JP5887840B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that can be mounted on a vehicle such as an automobile.

  2. Description of the Related Art Conventionally, an image processing apparatus that is mounted on a vehicle such as an automobile and detects raindrops and monitors the periphery of the automobile is known.

  For example, a lens that can take a first focal length for short distance for imaging raindrops attached to the vehicle and a second focal length for long distance for imaging the periphery of the vehicle is provided. , A camera unit capable of switching the focal length of the lens between a first focal length and a second focal length, and a raindrop for detecting the presence or absence of raindrops based on an image captured by the camera unit at the first focal length An image processing apparatus or the like that includes a detection unit and a monitoring unit that monitors the periphery of the vehicle based on an image captured by the camera unit at a second focal length (see, for example, Patent Document 1).

  However, the image processing apparatus described in Patent Document 1 uses a lens having a focal point for both near and near, focuses on raindrops when detecting raindrops adhering to the windshield surface, and detects vehicle surrounding information when detecting vehicle periphery information. Focus on the surroundings. For this reason, when raindrops are detected by focusing on the windshield surface, the background contrast is reflected in the raindrops, and the position of the bright spot changes according to the position of the light source, so that the raindrop region can be extracted. Have difficulty.

  That is, in the image processing apparatus described in Patent Document 1, it is difficult to capture raindrops attached to the windshield surface and vehicle periphery information far from the position of the windshield under appropriate conditions. .

  The present invention has been made in view of the above points, and is capable of capturing an image of an object attached to a transparent member such as a windshield and information far from the position of the transparent member under suitable conditions. It is an object to provide a processing apparatus.

The image processing apparatus includes a light source that irradiates irradiation light from one surface side of the transparent member toward the transparent member, an imaging lens in which a focal point is set farther from the position of the transparent member, and a subsequent stage of the imaging lens. And an image sensor composed of a plurality of two-dimensionally arranged pixels that receive light transmitted through the optical filter, and disposed on the same side as the light source with respect to the transparent member And the optical filter is formed on a substrate that transmits incident light incident through the imaging lens and a part of an effective imaging region on the substrate, and the light source among the incident light A spectral filter layer that selectively transmits light in a first wavelength band including the oscillation wavelength range of the optical filter, wherein the spectral filter layer is formed in a checkered pattern on the substrate, and the spectral filter layer includes: Formation An image of a deposit adhering to the other surface of the transparent member is captured with a first exposure amount by a pixel corresponding to the region, and the position of the transparent member is determined by a pixel in the region where the spectral filter layer is not formed. It is a requirement to capture a farther image with the second exposure amount.

  According to the disclosed technology, it is possible to provide an image processing apparatus capable of capturing an attached matter attached to a transparent member such as a windshield and information distant from the position of the transparent member under appropriate conditions.

1 is a schematic diagram illustrating a schematic configuration of an image processing apparatus 10 according to a first embodiment. It is the schematic diagram which expands and illustrates a part of FIG. It is a figure which illustrates the picked-up image data for raindrop detection in case the imaging lens 41 is focused on the raindrop. It is a figure which illustrates the picked-up image data for raindrop detection in case the imaging lens 41 is in focus at infinity. It is the figure which looked at the optical filter 42 from the sensor board | substrate 44 side. It is the figure which saw through the surface facing the optical filter 42 of the image sensor 43 from the sensor board | substrate 44 side. It is a figure which illustrates the correspondence of the positional relationship of the optical filter and image sensor 43 which concern on 1st Embodiment. It is sectional drawing which follows the AA line of FIG.5 and FIG.6. It is a figure which illustrates the polarizer formed with the wire grid structure. FIG. 6 is a diagram (part 1) illustrating transmittance characteristics of a spectral filter layer 49a; FIG. 6 is a diagram (part 1) illustrating transmittance characteristics of a spectral filter layer 49b; FIG. 10 is a second diagram illustrating the transmittance characteristics of the spectral filter layer 49a. FIG. 6B is a diagram illustrating the transmittance characteristic of the spectral filter layer 49b (part 2); It is a figure which shows the example of a captured image typically. It is a figure which illustrates the light ray in connection with raindrop detection and vehicle detection. It is a figure which illustrates a captured image. It is the figure which looked at 42 A of optical filters which concern on the modification 1 of 1st Embodiment from the sensor board | substrate 44 side. It is a figure which illustrates the correspondence of the positional relationship of 42 A of optical filters and image sensor 43 which concern on the modification 1 of 1st Embodiment. It is sectional drawing which follows the BB line of FIG. It is sectional drawing which follows the CC line of FIG. It is the figure which looked at the optical filter 42B which concerns on the modification 2 of 1st Embodiment from the sensor board | substrate 44 side. It is a figure which illustrates the correspondence of the positional relationship of the optical filter 42B and the image sensor 43 which concern on the modification 2 of 1st Embodiment. It is sectional drawing which follows the DD line | wire of FIG. It is sectional drawing which follows the EE line | wire of FIG. It is the figure which looked at 42 C of optical filters which concern on the modification 3 of 1st Embodiment from the sensor board | substrate 43 side. It is FIG. (The 1) which illustrates an opening restriction part. It is FIG. (The 2) which illustrates an opening restriction part. It is FIG. (The 3) which illustrates an opening restriction | limiting part. It is FIG. (The 4) which illustrates an opening restriction part. It is FIG. (5) which illustrates an opening restriction part. It is FIG. (6) which illustrates an opening restriction part. It is a schematic diagram which illustrates schematic structure of the vehicle equipment control system provided with the image processing apparatus which concerns on this Embodiment.

  Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In the drawings, the same components are denoted by the same reference numerals, and redundant description may be omitted.

  In the following embodiments, an example is shown in which an image processing apparatus is mounted on a vehicle such as an automobile and images of vehicle peripheral information such as raindrops attached to a windshield of the vehicle and headlights of other vehicles. .

[First Embodiment]
<Schematic configuration of image processing apparatus>
FIG. 1 is a schematic view illustrating a schematic configuration of an image processing apparatus 10 according to the first embodiment. FIG. 2 is a schematic view illustrating a part of FIG. 1 in an enlarged manner. Referring to FIGS. 1 and 2, the image processing apparatus 10 includes an imaging unit 20 and an image analysis unit 50. Reference numeral 101 denotes a windshield of the vehicle, and 102 denotes raindrops attached to the outer wall surface of the windshield 101. The windshield 101 is a typical example of the transparent member according to the present invention. Moreover, the raindrop 102 is a typical example of the deposit | attachment which concerns on this invention.

  The imaging unit 20 is installed, for example, in the vicinity of the vehicle windshield 101 (for example, in the vicinity of a room mirror (not shown)). For example, the imaging unit 20 may be installed in the vicinity of the windshield 101 in a state in which the imaging unit 20 is accommodated in the cover 103 having at least the windshield 101 side.

  Alternatively, the cover 103 having an opening on the windshield 101 side may be brought into close contact with the windshield 101 so that the imaging unit 20 is covered with the windshield 101 and the cover 103. In this case, even if the inner wall surface of the windshield 101 is clouded, it is possible to prevent the windshield 101 in the portion covered with the cover 103 from being clouded. As a result, the image analysis unit 50 can be prevented from being erroneously analyzed due to fogging of the windshield 101, and various control operations based on the analysis result of the image analysis unit 50 can be appropriately performed.

  However, for example, when the imaging unit 20 detects cloudiness of the windshield 101 and controls the air conditioning equipment of the host vehicle, the portion of the windshield 101 facing the imaging unit 20 is the same as the other parts. An air flow path may be formed in the cover 103 so that

  The imaging unit 20 includes a light source 30 and an imaging device 40. The light source 30 is a light source provided to detect an adhering matter adhering to the windshield 101, and has a function of irradiating irradiation light from one surface side (inner wall surface side) of the windshield 101 toward the windshield 101. Have The light source 30 is disposed at a position at which at least a deposit detection image area (raindrop detection image area) for detecting a deposit attached to the windshield 101 can be irradiated.

  For example, if the image area for adhering matter detection is the lower end side of the windshield 101, the light source 30 is disposed at a position where at least the lower end side of the windshield 101 can be irradiated. In addition, if the attached object detection image area is the upper end side and the lower end side of the windshield 101, the light source 30 is disposed at a position where at least the upper end side and the lower end side of the windshield 101 can be irradiated. Further, when the image area for detection of attached matter is the entire surface of the windshield 101, the light source 30 is disposed at a position where at least the entire surface of the windshield 101 can be irradiated. Note that a plurality of light sources 30 may be arranged in order to reliably irradiate the attached object detection image region.

  As the light source 30, for example, a light emitting diode (LED), a semiconductor laser (LD), or the like can be used. Further, as the oscillation wavelength of the light source 30, for example, visible light, infrared light, or the like can be used.

  However, it is necessary to avoid dazzling the driver or pedestrian of the oncoming vehicle with the light of the light source 30. For this reason, as the oscillation wavelength of the light source 30, a wavelength that is longer than the visible light and reaches the light receiving sensitivity of the image sensor 43 described later (for example, a wavelength in the infrared light region of 800 nm to 1000 nm) is selected. It is preferable. Hereinafter, a case where the light source 30 emits light having a wavelength in the infrared light region will be described as an example.

  Note that the irradiation light of the light source 30 is preferably substantially parallel light. By arranging a collimating lens or the like immediately after the light source 30, substantially parallel light can be generated. Thus, the light source 30 includes components such as a lens capable of adjusting the form of the irradiation light.

  The imaging device 40 is disposed on the same side as the light source 30 with respect to the windshield 101, and has a function of imaging reflected light of light emitted from the light source 30 to the raindrop 102 and incident light from outside the vehicle. In the present embodiment, the imaging device 40 is arranged so that the optical axis of the imaging lens 41 faces the horizontal direction (X direction), but the present invention is not limited to this. The light source 30 is disposed at a position where the reflected light of the light emitted from the light source 30 to the raindrop 102 enters the imaging device 40.

  The imaging device 40 includes an imaging lens 41, an optical filter 42, an image sensor 43, a sensor substrate 44, and a signal processing unit 45. The imaging lens 41 is composed of, for example, a plurality of lenses, and the focal point is set farther than the position of the windshield 101. The focal position of the imaging lens 41 can be set, for example, between infinity or infinity and the windshield 101.

  The optical filter 42 is disposed downstream of the imaging lens 41 and has a function of limiting the wavelength band of light incident on the image sensor 43. The image sensor 43 is composed of a plurality of two-dimensionally arranged pixels that receive light transmitted through the optical filter 42, and has a function of photoelectrically converting incident light for each pixel. The image sensor 43 is mounted on the sensor substrate 44. In addition, although each pixel of the image sensor 43 is drawn in a simplified manner in FIG. 6 and the like which will be described later, the image sensor 43 is actually composed of about several hundreds of thousands of pixels arranged two-dimensionally.

  As the image sensor 43, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like can be used. For example, a microlens or the like corresponding to each pixel may be provided on the incident side of the image sensor 43 to increase the light collection efficiency to the image sensor 43.

  The signal processing unit 45 generates captured image data obtained by converting the analog electrical signal (the amount of light incident on each pixel of the image sensor 43) that is photoelectrically converted by the image sensor 43 and output from the sensor substrate 44 into a digital electrical signal. Has a function to output. The signal processing unit 45 is electrically connected to the image analysis unit 50.

  The light applied to the raindrops 102 attached to the outer wall surface of the windshield 101 from the light source 30 is reflected at the interface between the raindrops 102 and the air, and enters the image sensor 43 via the imaging lens 41 and the optical filter 42. The light incident on the image sensor 43 is converted by the image sensor 43 into an electrical signal corresponding to the light intensity.

  When an electric signal (analog signal) is input from the image sensor 43 via the sensor substrate 44, the signal processing unit 45 determines the brightness of each pixel on the image sensor 43 as captured image data from the input electric signal. A digital signal (captured image data) indicating (luminance) is output to the subsequent image analysis unit 50 together with the horizontal / vertical synchronization signal of the image.

  The image analysis unit 50 has a function of controlling the imaging device 40 and a function of analyzing captured image data transmitted from the imaging device 40. Specifically, the image analysis unit 50 calculates an optimal exposure amount for each imaging region of the image sensor 43 from the captured image data transmitted from the imaging device 40, and optimal exposure for each imaging region of the image sensor 43. It has a function to set the amount (adjust to the optimum exposure time).

  Further, the image analysis unit 50 detects raindrops 102 and foreign matter adhering to the windshield 101 from the captured image data transmitted from the imaging device 40, or white lines on the road surface (in the imaging area) ( Or a detection object such as a lane marking).

  Further, the image analysis unit 50 has a function of calculating the position, direction, distance, and the like of another vehicle existing ahead of the host vehicle from the captured image data transmitted from the imaging device 40. Further, the image analysis unit 50 detects the preceding vehicle traveling in the same traveling direction as the own vehicle by identifying the tail lamp of the other vehicle from the captured image data transmitted from the imaging device 40, or the headlamp of the other vehicle. By identifying the oncoming vehicle that travels in the opposite direction to the host vehicle.

<Detailed description of image processing apparatus>
Here, the image processing apparatus 10 will be described in more detail. First, the detection of a deposit that is one of the functions of the image analysis unit 50 will be described in detail by taking a case where the deposit is a raindrop 102 as an example.

[Focus of imaging lens 41]
When raindrops 102 are not attached to the outer wall surface of the windshield 101, the light emitted from the light source 30 is reflected at the interface between the outer wall surface of the windshield 101 and the outside air, and the reflected light enters the imaging device 40. . On the other hand, as shown in FIG. 1, when raindrops 102 are attached to the outer wall surface of the windshield 101, the refractive index difference between the outer wall surface of the windshield 101 and the raindrop 102 is the same as that of the outer wall surface of the windshield 101. It becomes smaller than the refractive index difference from the outside air.

  Therefore, the light emitted from the light source 30 passes through the interface between the outer wall surface of the windshield 101 and the raindrop 102 and enters the raindrop 102. Then, the light incident on the raindrop 102 is reflected at the interface between the raindrop 102 and the air, and the reflected light enters the imaging device 40. Due to such a difference depending on the presence or absence of the raindrop 102, the image analysis unit 50 can detect the presence or absence of the raindrop 102 attached to the windshield 101 from the captured image data transmitted from the imaging device 40.

  In the present embodiment, the focal position of the imaging lens 41 is set between infinity or infinity and the windshield 101. Thereby, the image analysis unit 50 not only detects the raindrops 102 attached to the outer wall surface of the windshield 101 but also detects the preceding vehicle and the oncoming vehicle and the white line. Appropriate information can be acquired from the captured image data.

  For example, when detecting the raindrop 102 attached to the outer wall surface of the windshield 101, the shape of the image of the raindrop 102 on the captured image data is often circular. Therefore, the image analysis unit 50 determines whether the candidate image of the raindrop 102 on the captured image data is a circular shape, thereby identifying that the candidate image of the raindrop 102 is an image of the raindrop 102. I do.

  When such shape recognition processing is performed, as described above, the distance between the infinity or infinity and the windshield 101 is larger than the focus of the imaging lens 41 on the raindrops 102 attached to the outer wall surface of the windshield 101. If the image is in focus, the shape recognition rate (circular shape) of the raindrop 102 becomes higher and the detection performance of the raindrop 102 is improved.

  FIG. 3 is a diagram illustrating captured image data for raindrop detection when the imaging lens 41 is in focus on the raindrop 102. FIG. 4 is a diagram exemplifying captured image data for raindrop detection when the imaging lens 41 is focused at infinity. 3 and 4 are examples in the case where infrared light is applied to the raindrops 102 from the light source 30. FIG.

  Referring to FIG. 3, when the imaging lens 41 is focused on the raindrop 102 attached to the outer wall surface of the windshield 101, the background image 105 reflected on the raindrop 102 is captured. Such a background image 105 causes erroneous detection of the raindrop 102. Further, only the portion 106 of the raindrop 102 may have a high brightness such as a bow shape. In that case, the shape of the portion 106 with increased luminance changes depending on the direction of sunlight, the position of the streetlight, and the like. In order to cope with such a shape of the image of the raindrop 102 that changes variously by the shape recognition processing, the processing load is large, and the recognition accuracy is lowered.

  On the other hand, referring to FIG. 4, when the imaging lens 41 is focused at infinity, some blurring occurs. For this reason, the reflection of the background image 105 is not reflected in the captured image data, and erroneous detection of the raindrop 102 is reduced. In addition, the occurrence of some defocusing reduces the degree to which the shape of the raindrop image changes depending on the direction of sunlight, the position of the streetlight, etc., and the shape of the image of the raindrop 102 is always substantially circular. Thereby, the load of the shape recognition process of the raindrop 102 can be reduced, and the recognition accuracy of the raindrop 102 can be improved.

  However, when the imaging lens 41 is focused at infinity, the image sensor 43 may have about one pixel that receives the light of the tail lamp when identifying the tail lamp of a preceding vehicle traveling far away. is there. In this case, there is a possibility that the tail lamp cannot be recognized and the preceding vehicle cannot be detected.

  In order to avoid such a problem, it is preferable to focus the imaging lens 41 closer to infinity than infinity. As a result, the tail lamp of the preceding vehicle traveling far is out of focus, so the number of pixels that receive light from the tail lamp can be increased. As a result, since the tail lamp recognition accuracy is improved, the detection accuracy of the preceding vehicle can be improved.

[Configuration of Optical Filter 42]
FIG. 5 is a diagram of the optical filter 42 viewed from the sensor substrate 44 side. FIG. 6 is a perspective view of the surface of the image sensor 43 that faces the optical filter 42 from the sensor substrate 44 side. FIG. 7 is a diagram illustrating the correspondence of the positional relationship between the optical filter 42 and the image sensor 43 according to the first embodiment. FIG. 8 is a cross-sectional view taken along line AA in FIGS. 5 and 6.

  5 to 7, for convenience, the region corresponding to the spectral filter layer 49 b and the spectral filter layer 49 b on the image sensor 43 is drawn with a satin pattern. In the optical filter 42 shown in FIG. 7, a broken line drawn on the outer peripheral portion of the spectral filter layer 49b indicates an effective imaging region.

  5 to 8, in the optical filter 42, the filter substrate 46 is a transparent substrate that transmits incident light incident on the optical filter 42 via the imaging lens 41. A spectral filter layer 49a is formed in the entire effective image area (area corresponding to all pixels constituting the image sensor 43) on the surface of the filter substrate 46 on the imaging lens 41 side. The spectral filter layer 49a is a typical example of the second spectral filter layer according to the present invention.

  A polarizing filter layer 47 is formed on the surface of the filter substrate 46 on the image sensor 43 side. Although the polarizing filter layer 47 may not be formed, unnecessary light can be reduced by forming the polarizing filter layer 47, so that there is an advantage that the detection accuracy of raindrop detection and vehicle detection can be improved.

  A filler 48 is further formed so as to cover the polarizing filter layer 47. A spectral filter layer 49b is laminated on the upper end side and the lower end side of the effective image area on the surface of the filler 48 on the image sensor 43 side. The spectral filter layer 49b is a typical example of the spectral filter layer according to the present invention.

Of the light incident on the optical filter 42, the spectral filter layer 49a, the light transmitted through the polarizing filter layer 47, and the spectral filter layer 49b is incident on the pixel _43a 1 _{~43b 8} and _43g 1 _{~43h 8} image sensor 43 . Of the light incident on the optical filter 42, the light transmitted through the spectral filter layer 49 a and the polarizing filter layer 47 (light transmitted through the region where the spectral filter layer 49 b is not formed) is the pixel 43 c of the image sensor 43. incident on the _{1 ~43f 8.}

  As a material of the filter substrate 46, a transparent material that can transmit light in a use band (in the present embodiment, visible light region and infrared region), for example, glass, sapphire, crystal, or the like can be used. In the present embodiment, it is preferable to use glass, particularly quartz glass (refractive index: 1.46) and Tempax glass (refractive index: 1.51) which are inexpensive and durable.

  The polarizing filter layer 47 is formed so as to transmit only the P-polarized component of the light incident on the optical filter 42. The polarizing filter layer 47 has a polarizer formed in a wire grid structure as shown in FIG. 9, and the surface on the spectral filter layer 49b side is an uneven surface. The wire grid structure is a structure in which metal wires (conductor wires) made of metal such as aluminum and extending in a specific direction are arranged at a specific pitch. In FIG. 9, light with a polarization direction in the groove direction is blocked when incident, and light with a polarization direction perpendicular to the groove is transmitted.

  By setting the wire pitch of the wire grid structure to a sufficiently small pitch (for example, ½ or less) compared to the wavelength band of incident light (for example, the wavelength of visible light from 400 nm to 800 nm), the following effects are produced. That is, almost all the light of the electric field vector component that oscillates in parallel to the longitudinal direction of the metal wire is reflected and almost all the light of the electric field vector component that oscillates in the direction orthogonal to the longitudinal direction of the metal wire is transmitted. It can be used as a polarizer that produces a single polarized light.

  In general, a polarizer having a wire grid structure has an extinction ratio that increases as the cross-sectional area of the metal wire increases. Further, the transmittance of a metal wire having a predetermined width or more with respect to the period width decreases. Moreover, when the cross-sectional shape orthogonal to the longitudinal direction of the metal wire is a taper shape, the wavelength dispersion of transmittance and polarization degree is small in a wide band, and high extinction ratio characteristics are exhibited.

  Forming the polarizing filter layer 47 with a wire grid structure has the following effects. That is, the wire grid structure can be formed using a widely known semiconductor manufacturing process. Specifically, after depositing an aluminum thin film on the filter substrate 46, patterning is performed, and the sub-wavelength uneven structure of the wire grid may be formed by a technique such as metal etching. By such a manufacturing process, it is possible to adjust the longitudinal direction of the metal wire, that is, the polarization direction (polarization axis) corresponding to the imaging pixel size (several μm level) of the image sensor 43.

  Further, since the wire grid structure is made of a metal material such as aluminum, there is an advantage that it is excellent in heat resistance and can be suitably used even in a high-temperature environment such as a vehicle interior that is likely to become high temperature.

  When the spectral filter layer 49b is formed directly on the concave and convex surface of the polarizing filter layer 47, the spectral filter layer 49b is formed along the concave and convex surface of the polarizing filter layer 47, resulting in uneven thickness of the spectral filter layer 49b and the original spectral layer. Performance may not be obtained. Therefore, the upper surface side of the polarizing filter layer 47 in the stacking direction (spectral filter layer 49 b side) is filled with the filler 48 and planarized, and then the spectral filter layer 49 b is formed on the filler 48.

  The filler 48 is filled in the recesses between the metal wires of the polarizing filter layer 47. As the filler 48, an inorganic material having a refractive index lower than or equal to that of the filter substrate 46 can be suitably used. In addition, the filler 48 in this Embodiment is formed so that the upper direction of the lamination direction of the metal wire part of the polarizing filter layer 47 may also be covered.

  As the material of the filler 48, it is necessary to use a material that can flatten the uneven surface of the polarizing filter layer 47 and does not interfere with the function of the polarizing filter layer 47. Therefore, it is preferable to use a material that does not have a polarizing function. . Further, as the material of the filler 48, it is preferable to use a low refractive index material whose refractive index is as close as possible to the refractive index of air (refractive index = 1).

As a specific material of the filler 48, for example, a porous ceramic material formed by dispersing fine pores in ceramics is preferable. More specifically, porous silica (SiO ₂ ), porous magnesium fluoride (MgF), porous alumina (Al ₂ O ₃ ), and the like can be given.

  The degree of these low refractive indexes is determined by the number and size of pores in the ceramic (porosity). When the main component of the filter substrate 46 is made of quartz or glass, porous silica (n = 1.2-1.26) can be preferably used.

The method for forming the filler 48 is not limited to this, but, for example, an SOG (Spin On Glass) method can be suitably used. Specifically, a solvent in which silanol (Si (OH) ₄ ) is dissolved in alcohol is spin-coated on the polarizing filter layer 47 formed on the filter substrate 46, and then the solvent component is volatilized by heat treatment, thereby silanol. It can be formed by subjecting itself to a dehydration polymerization reaction.

  The polarizing filter layer 47 has a sub-wavelength sized wire grid structure, has low mechanical strength, and a metal wire is damaged by a slight external force. Since it is desired that the optical filter 42 of the present embodiment is disposed in close contact with the image sensor 43, there is a possibility that the optical filter 42 and the image sensor 43 come into contact with each other in the manufacturing stage.

  In the present embodiment, since the upper surface of the polarizing filter layer 47 in the stacking direction (the surface on the image sensor 43 side) is covered with the filler 48, the situation in which the wire grid structure is damaged when contacting the image sensor 43 is suppressed. it can. Further, by filling the filler 48 into the recesses between the metal wires in the wire grid structure of the polarizing filter layer 47 as in the present embodiment, it is possible to prevent foreign matter from entering the recesses.

  Note that the spectral filter layer 49b laminated on the filler 48 may be protected by a protective layer such as the filler 48. However, in the present embodiment, the spectral filter layer 49b is similar to the filler 48. No protective layer is provided. According to the experiments by the inventors, even when the spectral filter layer 49b contacts the image sensor 43, damage that affects the captured image did not occur. Is omitted.

  In addition, the height of the metal wire (convex portion) of the polarizing filter layer 47 is as low as half or less of the wavelength used, while the spectral filter layer 49b is as high as about several times the wavelength used. As the thickness of the filler 48 increases, it becomes more difficult to ensure the flatness of the upper surface, which affects the characteristics of the optical filter 42, so that there is a limit to increasing the thickness of the filler 48. Therefore, in the present embodiment, the spectral filter layer 49b is not covered with a filler.

  Here, the transmittance characteristics of the spectral filter layers 49a and 49b will be described. FIG. 10 is a diagram illustrating the transmittance characteristics of the spectral filter layer 49a. FIG. 11 is a diagram illustrating the transmittance characteristics of the spectral filter layer 49b.

  Referring to FIG. 10, the spectral filter layer 49 a transmits incident light in a so-called visible light region with a wavelength range of 400 nm to 670 nm and incident light in a so-called infrared region with a wavelength range of 940 to 970 nm, and a wavelength range of 670 nm to 940 nm. It has a transmittance characteristic that cuts incident light. The transmittance in the wavelength range of 400 nm to 670 nm and the wavelength range of 940 to 970 nm is preferably 30% or more, and more preferably 90% or more. The transmittance in the wavelength range of 670 nm to 940 nm is preferably 20% or less, and more preferably 5% or less.

  Incident light in the visible light region is used to detect vehicle periphery information, and incident light in the infrared light region is used to detect raindrop information. The reason why incident light in the wavelength range of 670 nm to 940 nm is not transmitted is that if this wavelength range is taken in, the obtained image data will be entirely red, and it will be difficult to extract the red part of the tail lamp. Because there are cases.

  Therefore, if a filter having a characteristic that cuts most of the wavelength range (670 nm to 940 nm) of the infrared light region as shown in FIG. 10 is formed, disturbance light can be removed, so that, for example, detection accuracy of the tail lamp is improved. be able to.

  From the viewpoint of removing disturbance light, it is desirable that the wavelength range 940 to 970 nm of the light source 30 is not transmitted. However, in the wavelength range of 940 to 970 nm, the amount of received light is sufficiently small because the band is narrower than the visible light band, and the influence on the detection of the tail lamp can be ignored.

  The wavelength range of 940 to 970 nm is a typical example of the first wavelength range according to the present invention. The wavelength range of 400 nm to 670 nm is a representative example of the second wavelength range according to the present invention.

  Referring to FIG. 11, the spectral filter layer 49b transmits the incident light in the so-called infrared light region in the wavelength range of 940 to 970 nm and cuts the incident light in the other wavelength ranges (up to 940 nm and 970 nm). Has characteristics. The transmittance in the wavelength range of 940 to 970 nm is preferably 30% or more, and more preferably 90% or more. The transmittance in other wavelength ranges (˜940 nm and 970 nm˜) is preferably 20% or less, and more preferably 5% or less.

  From the transmittance characteristics of FIGS. 10 and 11, in the region where only the spectral filter layer 49 a is formed (for example, the region corresponding to the central portion 1/2 of the image sensor 43), the wavelength depends on the characteristics of the spectral filter layer 49 a. Incident light in the so-called visible light region in the range of 400 nm to 670 nm and incident light in the so-called infrared region in the wavelength range of 940 to 970 nm are transmitted.

  Further, in a region where both the spectral filter layers 49a and 49b are formed (for example, a region corresponding to the upper and lower quarters of the image sensor 43), due to the characteristics of the combination of the spectral filter layer 49a and the spectral filter layer 49b, Only incident light in the wavelength range of 940 to 970 nm is transmitted.

  In order to increase the detection sensitivity in the infrared light region, it is desirable that the center value of the wavelength range of 940 to 970 nm and the oscillation wavelength of the light source 30 are substantially equal. In other words, it is desirable that the spectral filter layer 49b has a characteristic of selectively transmitting light in a wavelength band including the oscillation wavelength range of the light source 30 among incident light. By making the center value in the wavelength range of 940 to 970 nm substantially the same as the oscillation wavelength of the light source 30, the reflected light of the infrared wavelength light emitted from the light source 30 can be easily distinguished from a large amount of disturbance light.

  For example, when the spectral filter layers 49a and 49b are not present, when the reflected light of the infrared wavelength light emitted from the light source 30 reflected by the windshield 101 is captured by the imaging device 40, the image sensor 43 of the imaging device 40 In addition to the reflected light of the infrared wavelength light emitted from the light source 30, a large amount of disturbance light such as sunlight is also received.

  Therefore, in order to distinguish the infrared wavelength light from the light source 30 from such a large amount of disturbance light, it is necessary to make the light emission amount of the light source 30 sufficiently larger than the disturbance light. It is often difficult to use an amount of light source 30.

  Therefore, spectral filter layers 49 a and 49 b having transmittance characteristics shown in FIGS. 10 and 11 are provided on the optical path incident on the imaging device 40. Accordingly, since the light other than the reflected light of the infrared wavelength light emitted from the light source 30 is significantly cut, the reflected light of the infrared light emitted from the light source 30 can be received by the image sensor 43. At this time, if the central value of the wavelength range of 940 to 970 nm and the oscillation wavelength of the light source 30 are made substantially equal, the amount of reflected infrared light emitted from the light source 30 increases, so that the light source 30 emits light. The reflected infrared light can be accurately received by the image sensor 43.

  A region where only the spectral filter layer 49a is formed (for example, a region corresponding to the central portion 1/2 of the image sensor 43) is a vehicle detection used for detecting headlamps of oncoming vehicles, taillights of preceding vehicles, white lines, and the like. It becomes an image area. An area where both the spectral filter layers 49a and 49b are formed (for example, an area corresponding to the upper and lower quarters of the image sensor 43) is a raindrop detection image area (attachment detection image used for detecting raindrops). Area).

  However, instead of the transmittance characteristics shown in FIGS. 10 and 11 (characteristics including a band-pass filter in which the transmittance peak substantially matches the oscillation wavelength of the light source 30), the transmittance characteristics shown in FIGS. May be. That is, as shown in FIGS. 12 and 13, the light may be cut at a wavelength shorter than the oscillation wavelength of the light source 30. 10 and 13 may be used in combination, or FIG. 11 and FIG. 12 may be used in combination.

  In the present embodiment, the polarization filter 47 is used to transmit only the P-polarized component of the incident light to the optical filter 42 and cut the S-polarized component. Thus, for example, when detecting the presence or absence of raindrops, the S-polarized light component is removed, so S-polarized light such as light reflected on the road surface or light from the dashboard of the vehicle interior (reflection light), etc. Disturbances caused by unwanted light with strong components can be suppressed. Thereby, the identification rate of raindrops can be improved.

  Also, for example, when identifying white lines, headlamps of oncoming vehicles, tail lamps of preceding vehicles, etc., the S-polarized component is removed, so the headlamps and street lights reflected on the road surface and the dashboard inside the vehicle's interior It is possible to suppress a disturbance factor caused by unnecessary light having a strong S-polarized component such as light from the light (reflection light). Thereby, the identification rate of the white line, the headlamp of the oncoming vehicle, and the tail lamp of the preceding vehicle can be improved.

  In particular, it is generally known that the reflected light from the water surface covering the road surface has a lot of S-polarized light components in rainy roads. Therefore, by using the image from which the S-polarized light component has been removed for identifying the white line, it is possible to appropriately identify the white line below the water surface in the rainy road and improve the recognition rate.

  An area corresponding to the center half of the image sensor 43 is a vehicle detection image area, and an area corresponding to the upper and lower quarters of the image sensor 43 is a raindrop detection image area (attachment detection image area). This is an example, and the present invention is not limited to this.

  Next, the structure and manufacturing method of the spectral filter layer 49b will be described. The spectral filter layer 49b can be produced by a multilayer film structure in which a thin film having a high refractive index and a thin film having a low refractive index are alternately stacked. According to such a multilayer film structure, the degree of freedom in setting the spectral transmittance is high by using light interference, and by stacking thin films in multiple layers, for a specific wavelength (for example, a wavelength band other than infrared light). It is also possible to achieve reflectivity close to 100%.

  In the present embodiment, since the use wavelength range of the captured image data is approximately from the visible light wavelength band to the infrared light wavelength band, the image sensor 43 having sensitivity in the use wavelength range is selected. Since the spectral filter layer 49b only needs to transmit a part of infrared light, the transmission wavelength range of the multilayer film portion of the spectral filter layer 49b is set to, for example, 900 nm or more, and the other wavelength bands are reflected. (See FIG. 13) may be formed.

Such a cut filter is obtained by forming a multilayer film having a configuration of “substrate / (0.125L 0.25H 0.125L) p / medium A” in order from the lower side in the stacking direction of the optical filter 42. It is done. The “substrate” here means the filler 48 described above. In addition, “0.125L” is a film thickness marking method of a low refractive index material (for example, SiO ₂ ) in which nd / λ is 1 L. Therefore, the film of “0.125L” has an optical path of 1/8 wavelength. It means that the film is of a low refractive index material having such a long film thickness. “N” is a refractive index, “d” is a thickness, and “λ” is a cutoff wavelength.

Similarly, “0.25H” is obtained by setting nd / λ to 1H in the film thickness marking method of a high refractive index material (for example, TiO ₂ ). Therefore, the film of “0.25H” has a quarter wavelength. It means a film of a high refractive index material having a film thickness that becomes an optical path length. “P” indicates the number of times the combination of films shown in parentheses is repeated (laminated), and the more “p”, the more the influence of ripples and the like can be suppressed. The medium A is intended for air or a resin or adhesive for tight bonding with the image sensor 43.

The spectral filter layer 49b may be a band-pass filter (see FIG. 11) having a transmission wavelength range of 940 to 970 nm as a multilayer film structure that transmits only the infrared light wavelength band. With such a bandpass filter, it is possible to distinguish between an infrared region and a red region on the longer wavelength side than infrared light. Such a band-pass filter is, for example, “substrate / (0.125L 0.5M 0.125L) p (0.125L 0.5H 0.125L) q (0.125L 0.5M 0.125L) r / It can be obtained by producing a multilayer film having a configuration such as “medium A”. If titanium dioxide (TiO ₂ ) is used as the high refractive index material and silicon dioxide (SiO ₂ ) is used as the low refractive index material, a highly weather-resistant spectral filter layer 49b can be realized.

  An example of a method for producing the spectral filter layer 49 b will be described. First, the multilayer film described above is formed on the filler 48 formed on the filter substrate 46 and the polarizing filter layer 47. As a method for forming such a multilayer film, a known method such as vapor deposition may be used. Subsequently, the multilayer film is removed from a portion corresponding to a non-spectral region (for example, a region corresponding to the central portion 1/2 of the image sensor 43).

  As this removal method, a general lift-off processing method may be used. In the lift-off processing method, a pattern opposite to the target pattern is formed in advance on a layer of the filler 48 with a metal, a photoresist, or the like, a multilayer film is formed thereon, and then a non-spectral region. The multi-layer film corresponding to the above is removed together with the metal and photoresist.

  In this embodiment, since a multilayer film structure is adopted as the spectral filter layer 49b, there is an advantage that the degree of freedom in setting the spectral characteristics is high. In general, a color filter used in a color sensor or the like is formed of a resist agent, but it is difficult to control spectral characteristics with such a resist agent as compared with a multilayer film structure. In the present embodiment, since the multilayer filter structure is employed as the spectral filter layer 49b, the wavelength of the light source 30 and the wavelength band of the raindrop detection area can be substantially matched.

  Although the above description has been made centering on the spectral filter layer 49b, the spectral filter layer 49a can also be realized by a multilayer structure similar to the spectral filter layer 49b.

  Here, the reason why the raindrop detection image areas are provided above and below will be described. FIG. 14 is a diagram illustrating a captured image.

  The headlamp (not shown) of the oncoming vehicle, the tail lamp of the preceding vehicle 115, and the image of the white line 116 often exist mainly in the center of the captured image. Further, it is normal that an image of the road surface 117 immediately in front of the host vehicle exists at the lower part of the captured image and an image of the sky 118 exists at the upper part of the captured image. That is, information necessary for identifying the headlamp (not shown) of the oncoming vehicle, the tail lamp of the preceding vehicle 115, and the white line 116 is concentrated in the central portion of the captured image, and in the identification, information on the upper portion of the captured image and the lower portion of the captured image. Is not very important.

  Therefore, when both the detection of the oncoming vehicle, the preceding vehicle 115 or the white line 116 and the detection of the raindrop 102 are performed simultaneously from the single captured image data, as shown in FIG. It is preferable to use the image area 111 for the raindrop and the raindrop detection image areas 112 and 113 for the upper part of the captured image and the lower part of the captured image, respectively.

  Therefore, a spectral filter layer 49a as shown in FIG. 8 is provided corresponding to the vehicle detection image region 111 of FIG. 14, and the spectral as shown in FIG. 8 is provided corresponding to the raindrop detection image regions 112 and 113 of FIG. Filter layers 49a and 49b are provided.

  Note that when the imaging direction of the imaging device 40 is tilted downward, the hood of the host vehicle (the portion indicated by the broken line in the raindrop detection image area 113 in FIG. 14) may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the own vehicle, tail lamps of the preceding vehicle, etc. become disturbance light, which is included in the captured image data.

  Even in such a case, in the present embodiment, since the spectral filter layers 49a and 49b are provided at a position corresponding to the lower part of the captured image, disturbance light such as sunlight reflected by the bonnet and the tail lamp of the preceding vehicle 115 is removed. Is done. Therefore, the detection accuracy of the raindrop 102 can be improved.

  When the preceding vehicle 115 is detected, the preceding vehicle 115 is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light than the headlamp of the oncoming vehicle, and street lights, etc. There are also many disturbance lights. Therefore, it is difficult to detect the tail lamp with high accuracy only from mere luminance data.

  Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. However, since each pixel constituting the image sensor 43 of the present embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 43 receives light including the infrared wavelength band. The captured image is entirely reddish. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp.

  In the present embodiment, as described above, since the spectral filter layer 49a has a transmittance characteristic that cuts incident light in the wavelength range of 670 nm to 940 nm, the infrared wavelength band from the captured image data portion used for identification of the tail lamp is large. The portion is excluded, and the tail lamp identification accuracy can be improved.

  In FIG. 8, a gap is provided between the optical filter 42 and the image sensor 43, but it is preferable to have a configuration in which the optical filter 42 is in close contact with the image sensor 43. This is because the boundary between the regions of the polarizing filter layer 47 and the spectral filter layer 49b of the optical filter 42 and the boundary between the pixels on the image sensor 43 can be easily matched. That is, by tightly joining the optical filter 42 and the image sensor 43, the boundary between the raindrop detection area and the area for detecting the vehicle periphery information is clarified, and the accuracy of determining the presence or absence of raindrops can be increased.

  When the optical filter 42 is brought into close contact with the image sensor 43, for example, the optical filter 42 and the image sensor 43 may be bonded with a UV adhesive, or effective imaging is performed with a spacer supported outside the effective imaging area used for imaging. The four-sided region outside the region may be UV bonded or thermocompression bonded.

  It is possible to suppress warping of the optical filter 42 by forming the spectral filter layers 49a and 49b on both sides of the filter substrate 46. That is, if a multilayer film (spectral filter layer) is formed only on one side of the filter substrate 46, stress is applied and warpage occurs. However, when multilayer films (spectral filter layers) are formed on both sides of the filter substrate 46 as shown in FIG. 8, warping can be suppressed because the effect of stress is offset.

  Further, when the spectral filter layer 49b is provided only on one of the upper and lower sides, it is difficult to bond the optical filter 42 and the image sensor 43 in parallel, and it is inclined with respect to the Z axis. If the adhesive is tilted, the optical path length is changed between the upper part of the captured image and the lower part of the captured image, which causes deterioration of recognition accuracy such as reading error of the white line coordinates when detecting vehicle peripheral information, for example, white line coordinates. As shown in FIG. 8, such a problem can be solved by providing the spectral filter layers 49b above and below.

  However, even when the spectral filter layers 49a and 49b are formed on one side of the filter substrate 46, or when the spectral filter layer 49b is provided on only one of the upper and lower sides, a certain effect can be obtained.

  In the present embodiment, due to the characteristics of the imaging lens 41, the sight in the imaging area and the image on the image sensor 43 are reversed.

  Here, raindrop detection and vehicle detection will be described in more detail. FIG. 15 is a diagram illustrating light rays related to raindrop detection and vehicle detection.

  The incident angle of the light source 30 on the windshield 101 is set so that reflected light at any position on the boundary surface between the raindrop 102 and air is imaged by the imaging device 40. The layout in which the reflected light from the boundary surface between the raindrop 102 and the air is the strongest is such that the light source 30 is substantially opposite to the optical axis of the imaging lens 41 with respect to the normal of the windshield 101 intersecting with the optical axis of the imaging lens 41. And the light source 30 is installed so as to have substantially the same optical axis as the optical axis of the imaging lens 41.

  Further, the light reflected from the boundary surface between the raindrop 102 and the air is the weakest when the normal line of the windshield 101 intersecting with the optical axis of the imaging lens 41 and the optical axis of the light source 30 substantially coincide. . Here, as an example, it is assumed that the light source 30 is arranged such that the regular reflection light on the outer wall surface of the windshield 101 substantially coincides with the optical axis of the imaging lens 41.

  The light ray A is a light ray that is irradiated from the light source 30 and passes through the windshield 101. When the raindrops 102 are not attached to the outer wall surface of the windshield 101, the light emitted from the light source 30 toward the windshield 101 passes through the windshield 101 like the light ray A and is directly outside the vehicle. Leaks out. Therefore, it is preferable to select a light source having a wavelength / light quantity in the eye-safe band in consideration of the fact that the light hits human eyes. Further, as shown in FIG. 15, it is more preferable that the light emitted from the light source 30 toward the windshield 101 is directed upward in the vertical direction because the possibility of the light hitting human eyes is reduced.

  The light beam B is a light beam that is incident on the imaging device 40 after the light irradiated from the light source 30 is regularly reflected by the inner wall surface of the windshield 101. Part of the light traveling from the light source 30 toward the windshield 101 is regularly reflected by the inner wall surface of the windshield 101. It is known that the polarization component of the regular reflection light (ray B) is generally dominated by the S polarization component that oscillates in a direction orthogonal to the incident surface (a direction perpendicular to the paper surface of FIG. 15). It has been.

  The specularly reflected light (light ray B) emitted from the light source 30 and specularly reflected by the inner wall surface of the windshield 101 does not vary depending on the presence or absence of the raindrops 102 adhering to the outer wall surface of the windshield 101. Besides, it becomes disturbance light that lowers the detection accuracy of raindrop detection. In the present embodiment, since the light beam B (S-polarized component) is cut by the polarizing filter layer 47, it is possible to suppress the raindrop detection accuracy from being lowered by the light beam B.

  A light ray C is a light ray that is irradiated from the light source 30 through the inner wall surface of the windshield 101 and is then reflected by the interface between the raindrops 102 and the air adhering to the outer wall surface of the windshield 101 and incident on the imaging device 40. It is. A part of the light traveling from the light source 30 toward the windshield 101 is transmitted through the inner wall surface of the windshield 101, but the transmitted light is more in the P-polarized component than in the S-polarized component.

  When the raindrops 102 are attached to the outer wall surface of the windshield 101, the light transmitted through the inner wall surface of the windshield 101 does not leak outside like the light ray A, and the interface between the raindrops 102 and air. , And is transmitted through the windshield 101 again toward the imaging device 40 and enters the imaging device 40.

  At this time, since the spectral filter layer 49a is configured to transmit the oscillation wavelength (infrared light) of the light source 30, the light ray C passes through the spectral filter layer 49a. Moreover, since the polarizing filter layer 47 is formed with the longitudinal direction of the metal wire having a wire grid structure so as to transmit the P-polarized component, the light ray C mainly composed of the P-polarized component also transmits the polarizing filter layer 47. . Moreover, since the spectral filter layer 49b is configured to transmit the oscillation wavelength (infrared light) of the light source 30, the light ray C also transmits through the spectral filter layer 49b. Therefore, the light ray C reaches the raindrop detection image area of the image sensor 43, and the raindrop 102 can be detected based on the amount of received light.

  The light ray D is a light ray that passes through the windshield 101 from the outside of the windshield 101 and enters the raindrop detection image region of the image sensor 43 of the imaging device 40. Although the light ray D can be disturbance light at the time of raindrop detection, in the present embodiment, most of the infrared light is cut off by the spectral filter layers 49 a and 49 b, and most of the light ray D is from the image sensor 43. It does not reach the raindrop detection image area. Therefore, it is possible to suppress the raindrop detection accuracy from being lowered by the light ray D.

  The light beam E is a light beam that passes through the windshield 101 from the outside of the windshield 101 and enters the vehicle detection image area of the image sensor 43 of the imaging device 40. The P-polarized component of the light beam E is transmitted through the spectral filter layers 49a and 49b so that only part of visible light and infrared light are transmitted through the spectral filter layers 49a and 49b. Is reached and imaged.

  This captured image is used for detecting a head lamp of an oncoming vehicle, a tail lamp of a preceding vehicle, and a white line. Since the S-polarized light component of the light beam E is removed by the polarizing filter layer 47, light such as headlamps and street lights reflected on the road surface, light from a dashboard or the like in the interior of the vehicle (reflection light), etc. In addition, it is possible to suppress disturbance factors caused by unnecessary light having a strong S-polarized component.

Next, exposure amount adjustment when a captured image is acquired by the image processing apparatus 10 will be described. As an example, as surrounded by a one-dot chain line in FIG. 6, one unit of captured image data by a total of four areas (two pixels 43 a ₁ , 43 a ₂ , 43 b ₁ , 43 b ₂ ) adjacent to each other in two vertical and two horizontal directions. Consider the case where Light transmitted through the spectral filter layer 49a, the polarizing filter layer 47, and the spectral filter layer 49b is incident on the pixels 43a ₁ , 43a ₂ , 43b ₁ , and 43b ₂ . Accordingly, the pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b ₂ receive the P-polarized component of the infrared light in the incident light on the optical filter 42, and generate an image using the P-polarized component of the infrared light. For example, the presence or absence of raindrops can be detected using the image of the P-polarized component of the infrared light thus obtained.

  The light source 30 may emit light continuously (sometimes referred to as CW light emission), or may emit pulse light at a specific timing. If the timing of light emission and the timing of imaging are synchronized, the influence of disturbance light can be reduced. In addition, when a plurality of light sources are installed, the plurality of light sources may emit light simultaneously or sequentially. When light is emitted sequentially, the influence of disturbance light can be reduced by synchronizing the light emission timing with the imaging timing.

Next, one unit of picked-up image data is configured by an area composed of the pixels 43c ₁ , 43c ₂ , 43d ₁ , and 43d ₂ . Light that has passed through the spectral filter layer 49a and the polarizing filter layer 47 is incident on the pixels 43c ₁ , 43c ₂ , 43d ₁ , and 43d ₂ . Accordingly, the pixels 43c ₁ , 43c ₂ , 43d ₁ , and 43d ₂ receive mainly the P-polarized light component of the incident light on the optical filter 42, and generate an image mainly using the P-polarized light component of the visible light. The For example, vehicle periphery information can be detected using the image of the P-polarized component of visible light obtained in this manner.

  When imaging the vehicle detection image area, the light source 30 may be turned off, or the light source 30 may be constantly turned on.

Next, one unit of picked-up image data is constituted by an area composed of the pixels 43g ₁ , 43g ₂ , 43h ₁ , 43h ₂ . In the region composed of the pixels 43g ₁ , 43g ₂ , 43h ₁ , 43h ₂ , an image based on the P-polarized component of infrared light is generated as in the region composed of the pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b _2. , Can detect the presence or absence of raindrops. In addition, the light source 30 is made to emit light similarly to the case of the pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b ₂ .

  Similarly, an image is sequentially generated with a total of four areas of two adjacent vertical and horizontal two as one unit. As a result, an image of the entire image sensor 43 is generated. However, it is not always necessary to alternately generate images of the raindrop detection image region and the vehicle detection image region, and images of the same region may be generated successively.

  Although an example in which four pixels are one unit is shown here, the present invention is not limited to this. For example, when imaging the image area for raindrop detection arranged above and below the image sensor 43, the pixel area corresponding to the spot size of the reflected light from the light source 30 on the image sensor 43 can be set as one unit, When the image area for vehicle detection arranged at the center of the image sensor 43 is imaged, one pixel can be made one unit when a monochrome sensor is used as the image sensor.

  By the way, the raindrop detection image region (attachment detection image region) in which the spectral filter layer 49a, the polarizing filter layer 47, and the spectral filter layer 49b are formed, and the spectral filter layer 49a and the polarizing filter layer 47 are formed. The amount of acquired light is greatly different from that of the vehicle detection image area in which the filter layer 49b is not formed. This is because only the reflected light of the infrared light emitted from the light source 30 is mainly incident on the raindrop detection image area, but the visible light is mainly incident on the vehicle detection image area.

  Therefore, it is preferable to change the exposure amount when imaging the raindrop detection image region and when capturing the vehicle detection image region. Thereby, an image can be picked up with an optimum exposure amount for each of the raindrop detection image region and the vehicle detection image region. For example, when capturing a distant image, automatic exposure adjustment is performed based on the image information of the vehicle detection image area while detecting a portion of the vehicle detection image area, and when detecting a raindrop image, raindrop detection is performed. The automatic exposure adjustment may be performed based on the image information of the raindrop detection image area while detecting the image area portion.

  In order to change the exposure amount, for example, the exposure time may be changed when the raindrop detection image area is imaged and when the vehicle detection image area is imaged. For example, the image analysis unit 50 can change the exposure time by controlling the time during which the image sensor 43 converts incident light into an electrical signal. Further, as in Modification 3 of the first embodiment to be described later, the exposure amount is changed by providing an aperture limiting portion corresponding to each pixel of the image sensor 43 in an area where the spectral filter layer 49b is not formed. You can also.

  The vehicle detection image region has a large change in the amount of light around it. Since the illuminance around the vehicle imaged by the imaging device varies from several tens of thousands of lux in the daytime to 1 lux or less at nighttime, it is necessary to adjust the exposure time according to the imaging scene. Control may be performed. In the present embodiment, since the subject is in the vicinity of the road surface, it is desirable to perform exposure control based on the image information of the road surface area.

  On the other hand, the raindrop detection image region is set so as to capture only the reflected light from the attached material with respect to the irradiation light from the light source 30, and therefore the change in the amount of light due to the surrounding environment is small. Therefore, the raindrop detection image region can be imaged with a fixed exposure time (imaged with a fixed exposure amount).

  FIG. 16 is a diagram illustrating a captured image. 16A is an example of a captured image when raindrops are attached to the outer wall surface of the windshield 101, and FIG. 16B is an image when raindrops are not attached to the outer wall surface of the windshield 101. FIG. It is an example of an image. In the present embodiment, raindrop detection image areas are provided above and below the captured image, but FIG. 16 is an image excluding the raindrop detection image area at the top of the image.

  In the captured images of FIG. 16A and FIG. 16B, the lower part of the captured image indicated as “detected area” in the figure is the raindrop detection image region 113, and the rest is the vehicle detection image region 111. When raindrops are attached to the image area 113 for raindrop detection, light from the light source 30 is projected as shown in FIG. 16A, and when no raindrops are attached, it is shown in FIG. 16B. Thus, the light from the light source 30 is not projected.

  The display “Rain detected” indicates a state in which raindrops are recognized, and the display “Rain not detected” indicates a state in which raindrops are not recognized. Such a raindrop image recognition process in the raindrop detection image region 113 can be easily performed by adjusting the threshold value of the amount of reflected light from the attached matter with respect to the irradiation light from the light source 30. Note that the threshold value does not have to be a fixed value, and may be changed as appropriate according to changes in the situation around the host vehicle on which the imaging device 40 is mounted. For example, the threshold value may be changed by calculating an optimum value based on the exposure adjustment information of the vehicle detection image region 111 or the like.

[Modification 1 of the first embodiment]
The first modification of the first embodiment shows an example in which the region dividing method of the spectral filter layer on the image sensor 43 side is changed. In the first modification of the first embodiment, the description of the same components as those of the already described embodiment is omitted.

  FIG. 17 is a diagram of an optical filter 42A according to Modification 1 of the first embodiment viewed from the sensor substrate 44 side. FIG. 18 is a diagram illustrating the correspondence of the positional relationship between the optical filter 42 </ b> A and the image sensor 43 according to the first modification of the first embodiment. 19 is a cross-sectional view taken along line BB in FIG. 20 is a cross-sectional view taken along the line CC of FIG.

  A view of the surface of the image sensor 43 facing the optical filter 42A seen through from the sensor substrate 44 side is the same as FIG. In FIG. 17 and FIG. 18, for the sake of convenience, regions corresponding to the spectral filter layer 49c and the spectral filter layer 49c on the image sensor 43 are drawn in a satin pattern. In the optical filter 42A shown in FIG. 18, the broken line drawn on the outer peripheral portion of the spectral filter layer 49c indicates the effective imaging area.

  Referring to FIGS. 17 to 20, in the optical filter 42 </ b> A according to the first modification of the first embodiment, the spectral filter layer 49 c is formed in a checkered pattern corresponding to each pixel of the image sensor 43. This is different from the first embodiment.

For example, as surrounded by a one-dot chain line in FIG. 6, one unit of captured image data can be constituted by a total of four pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b ₂ adjacent to each other in two vertical and two horizontal directions. In one unit surrounded by an alternate long and short dash line, the light source 30 emits light, and the light emission timing and the imaging timing are synchronized, so that the P of infrared light out of the incident light to the optical filter 42A at the pixels 43a ₂ and 43b ₁ The polarization component is received, and an image by the P polarization component of infrared light is obtained.

Further, at the timing of turning off the light source 30, P-polarized light component of the visible light is received mainly of the incident light to the optical filter 42A in the pixel 43a ₁ and 43 b _2, mainly image by P-polarized light component of the visible light can get. Furthermore, an image corresponding to all the pixels of the image sensor 43 can be obtained by sequentially shifting one unit surrounded by a one-dot chain line and repeating the same operation.

  Using the image of the P-polarized component of the infrared light and the image of the P-polarized component of the visible light obtained in this way, as described above, the deposits such as raindrops attached to the windshield of the vehicle and other vehicles The vehicle periphery information such as the headlight can be imaged.

  In these captured image data, the number of image pixels is smaller than the number of captured pixels (although information on adjacent pixel areas is insufficient), a generally known image is obtained when obtaining a higher resolution image. Interpolation techniques may be used. For example, in order to obtain an image of a P-polarized component of infrared light with a higher resolution, the following may be performed.

That is, it uses the information of the P-polarized light component of the infrared light for the pixels 43a ₂ and 43 b _1, for the pixel 43 b _2, for example, the pixel _43a 2 surrounding the _periphery, 43 b 1, 43 b _3, and the average of 43c ₂ The value may be used as information on the P-polarized component of infrared light. The same applies when trying to obtain an image of the P-polarized component of visible light mainly at a higher resolution.

[Modification 2 of the first embodiment]
The second modification of the first embodiment shows another example in which the region dividing method of the spectral filter layer on the image sensor 43 side is changed. In the second modification of the first embodiment, the description of the same components as those of the already described embodiment is omitted.

  FIG. 21 is a diagram of an optical filter 42B according to Modification 2 of the first embodiment viewed from the sensor substrate 44 side. FIG. 22 is a diagram illustrating the correspondence of the positional relationship between the optical filter 42 </ b> B and the image sensor 43 according to the second modification of the first embodiment. 23 is a cross-sectional view taken along the line DD of FIG. 24 is a cross-sectional view taken along the line EE of FIG.

  A view of the surface of the image sensor 43 facing the optical filter 42 seen from the sensor substrate 44 side is the same as FIG. 21 and 22, for convenience, a region corresponding to the spectral filter layer 49d and the spectral filter layer 49d on the image sensor 43 is drawn with a satin pattern. Further, in the optical filter 42B shown in FIG. 22, a broken line drawn on the outer peripheral portion of the spectral filter layer 49d indicates an effective imaging area.

  Referring to FIGS. 21 to 24, in the optical filter 42 </ b> B according to the second modification of the first embodiment, the spectral filter layer 49 d is formed in a stripe pattern corresponding to each pixel of the image sensor 43. This is different from the first embodiment.

For example, as surrounded by a one-dot chain line in FIG. 6, one unit of captured image data can be constituted by a total of four pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b ₂ adjacent to each other in two vertical and two horizontal directions. In one unit enclosed by a one-dot chain line, the light source 30 to emit light, by synchronizing the timing of the timing and the imaging of the emission, the infrared light of the incident light to the optical filter 42B in the pixel 43a ₁ and 43 b ₁ P The polarization component is received, and an image by the P polarization component of infrared light is obtained.

Further, at the timing when the light source 30 is turned off, the pixels 43a ₂ and 43b ₂ receive mainly the P-polarized component of the visible light in the incident light to the optical filter 42B, and an image mainly composed of the P-polarized component of the visible light. can get. Furthermore, an image corresponding to all the pixels of the image sensor 43 can be obtained by sequentially shifting one unit surrounded by a one-dot chain line and repeating the same operation.

  Using the image of the P-polarized component of the infrared light and the image of the P-polarized component of the visible light obtained in this way, as described above, the deposits such as raindrops attached to the windshield of the vehicle and other vehicles The vehicle periphery information such as the headlight can be imaged. Note that an image interpolation technique may be used as in the first modification of the first embodiment.

  Effects unique to the second modification of the first embodiment are as follows. That is, in the second modification of the first embodiment, the spectral filter layer 49d is formed in a stripe pattern, so that the image sensor 43 of the image sensor 43 is compared with the checkered pattern in the first modification of the first embodiment. The positional deviation accuracy between each pixel and the spectral filter layer 49d forming portion of the optical filter 42B can be relaxed.

  That is, in the case of the checkered pattern as in the first modification of the first embodiment, in order to match each pixel of the image sensor 43 with the spectral filter layer 49c forming portion of the optical filter 42A, It is necessary to adjust the position in each Z direction. On the other hand, in the stripe pattern as in Modification 2 of the first embodiment, in order to match each pixel of the image sensor 43 with the spectral filter layer 49d forming portion of the optical filter 42B, Y Only the direction needs to be adjusted. Thereby, in the process of bonding the optical filter 42B and the image sensor 43, the assembly time can be shortened and the assembling apparatus can be simplified.

  The stripe direction (Z direction in FIG. 21) of the stripe pattern is desirably set in accordance with the irradiation direction of the light source 30 for irradiating the raindrops 102. More specifically, the stripe direction (Z direction in FIG. 21) of the stripe pattern is relative to the surface formed by the optical axis of the irradiation light toward the windshield 101 of the light source 30 and the optical axis of the imaging lens 41. It is desirable to be parallel.

  This is because when the raindrops 102 attached to the windshield 101 are imaged, an image in which the vertical direction (Z direction) is compressed is obtained. That is, as shown in FIG. 21, the vertical direction (Z direction) is compressed by having the stripe direction in the vertical direction (Z direction), thereby increasing the resolution for extracting infrared light information in the vertical direction (Z direction). Raindrops can be detected accurately from images.

[Modification 3 of the first embodiment]
In the third modification of the first embodiment, an example in which an opening limiting portion is provided in a region where the spectral filter layer is not formed is shown. Note that in the third modification of the first embodiment, the description of the same components as those of the already described embodiment will be omitted.

  FIG. 25 is a diagram of an optical filter 42C according to Modification 3 of the first embodiment viewed from the sensor substrate 43 side. Since the positional relationship and cross-sectional view of the optical filter 42C and the image sensor 43 are the same as those of the first modification of the first embodiment, the illustration is omitted. In FIG. 25, for the sake of convenience, the spectral filter layer 49b is drawn in a satin pattern, and the aperture limiting portion 49y is drawn in gray.

  Referring to FIG. 25, the optical filter 42C according to the third modification of the first embodiment includes an opening 49x corresponding to each pixel of the image sensor 43 and an opening restriction in an area where the spectral filter layer 49b is not formed. The point which provided the part 49y differs from the modification 1 of 1st Embodiment.

For example, as surrounded by a one-dot chain line in FIG. 6, one unit of captured image data can be constituted by a total of four pixels 43a ₁ , 43a ₂ , 43b ₁ , 43b ₂ adjacent to each other in two vertical and two horizontal directions. In one unit surrounded by an alternate long and short dash line, the light source 30 emits light, and the light emission timing and the imaging timing are synchronized, so that the infrared light P of the incident light to the optical filter 42C at the pixels 43a ₂ and 43b ₁ The polarization component is received, and an image by the P polarization component of infrared light is obtained.

Further, at the timing when the light source 30 was turned off, P-polarized component of the openings limited primarily visible light of the incident light to the optical filter 42C in the pixel 43a ₁ and 43 b ₂ is received, mainly visible which is open restricted An image based on the P-polarized light component is obtained. However, since the pixels 43a ₁ and 43b ₂ mainly receive the P-polarized component of the visible light whose aperture is limited, the image mainly using the P-polarized component of the visible light is modified example 1 of the first embodiment. Is generated with a smaller received light amount.

  Furthermore, an image corresponding to all the pixels of the image sensor 43 can be obtained by sequentially shifting one unit surrounded by a one-dot chain line and repeating the same operation.

  Using the image of the P-polarized component of the infrared light obtained in this way and the image of the P-polarized component of the visible light whose aperture is mainly limited, as described above, it is possible to attach raindrops attached to the windshield of the vehicle. Vehicle peripheral information such as kimono and headlights of other vehicles can be imaged. Note that an image interpolation technique may be used as in the first modification of the first embodiment.

  FIG. 26 shows an example of the opening restriction unit. In FIG. 26, the opening 49x has a circular wire grid structure, and an opening restricting portion 49y made of a solid aluminum film is formed around the opening 49x. Thereby, the amount of received light in the region where the spectral filter layer 49b is not formed can be limited.

  According to the configuration shown in FIG. 26, since incident light is shielded by the opening restricting portion 49y made of an aluminum solid film, the spectral filter layer 49b is formed by the width (opening ratio) of the opening 49x made of a wire grid structure. It is possible to limit the amount of received light that passes through an unfinished area.

  The shape of the opening 49x having a wire grid structure is not limited to the circular shape shown in FIG. 26, and may be, for example, a substantially square shape as shown in FIG. In the case of a shape having corners as shown in FIG. 27, it is preferable to give R to the corners because the shape dimensions can be easily obtained by etching or the like.

  The opening 49x and the opening restricting portion 49y can be produced simultaneously with the polarizing filter layer 47, for example. Specifically, after the aluminum film is uniformly formed on the filter substrate 46, the aluminum film in the region for forming the spectral filter layer 49b and the region for forming the opening 49x is removed by etching or the like, and the aluminum film is removed. What is necessary is just to form a wire grid structure in a part.

  As described above, when an aluminum light-shielding region is provided in the periphery of the wire grid structure to limit the opening, the opening is limited by processing so as to leave the aluminum film in the periphery when forming the wire grid structure. Can do. Since this wire grid structure becomes the opening 49x and also the polarizing filter layer 47, the manufacturing process can be simplified as compared with the case where the opening limiting process is performed separately from the polarizing filter layer 47.

  Note that an opening limiting portion 49y as shown in FIG. 28 may be provided in a layer different from the polarizing filter layer 47. In this case, a wire grid structure is not formed inside the opening 49x.

  Further, the opening limiting portion 49y is not limited to the reflection film such as the aluminum film as described above, and may be formed of a film that absorbs light, for example. For example, as shown in FIG. 29, the opening limiting portion 49y may be formed of a solid black resist film. Also in this case, the opening 49x is not limited to the circular shape shown in FIG. 29, and may be, for example, a substantially square shape as shown in FIG.

  Further, a plurality of openings 49x may be provided for each region corresponding to each pixel of the image sensor 43 in the region where the spectral filter layer 49b is not formed. A plurality of aperture restriction portions 49y may be provided for each region corresponding to each pixel of the image sensor 43 in a region where the spectral filter layer 49b is not formed. In addition, the opening restriction portion 49y is not necessarily provided around the opening portion 49x, and the opening restriction portion 49y may be provided in the opening portion 49x. For example, as shown in FIG. 31, the opening restricting portions 49y made of a solid aluminum film may be discretely arranged in the openings 49x made of a wire grid structure.

  Effects unique to the third modification of the first embodiment are as follows. That is, for example, if the aperture limiting portion 49y is produced so that the amount of light received by the P-polarized component of visible light is substantially equal to the amount of light received by the P-polarized component of infrared light, the exposure control is performed once without performing complicated exposure control. In this exposure, it is possible to obtain infrared light that detects the presence or absence of raindrops and visible light that detects vehicle periphery information.

<Application example>
Here, as an application example, an example in which the in-vehicle device control system is configured using the image processing apparatus according to the present embodiment will be described.

  FIG. 32 is a schematic view illustrating a schematic configuration of an in-vehicle device control system including the image processing apparatus according to this embodiment. Referring to FIG. 32, an in-vehicle device control system 300 includes an image processing apparatus 10 according to the present embodiment, a headlamp control unit 310, a wiper control unit 320, a vehicle travel control unit 330, a headlamp 350, And a wiper 360. In FIG. 32, reference numeral 400 denotes a host vehicle such as an automobile.

  The in-vehicle device control system 300 uses the captured image data in the front area (imaging area) in the traveling direction of the host vehicle captured by the imaging device 40 of the image processing apparatus 10 mounted on the host vehicle 400 to control the light distribution of the headlamps. In addition, it has a function of controlling driving of the wiper and controlling other in-vehicle devices.

  The captured image data captured by the imaging device 40 is input to the image analysis unit 50. As described above, the image analysis unit 50 analyzes the captured image data transmitted from the imaging device 40 to calculate, for example, the position, direction, distance, and the like of another vehicle existing in front of the host vehicle 400.

  Further, as described above, the image analysis unit 50 detects the preceding vehicle traveling in the same traveling direction as the own vehicle 400 by identifying the tail lamp of the other vehicle, or identifies the head lamp of the other vehicle. An oncoming vehicle that travels in the opposite direction to the vehicle 400 is detected. In addition, as described above, the image analysis unit 50 detects raindrops 102 and foreign matter adhering to the windshield 101, or a detection target such as a white line (partition line) on the road surface existing in the imaging region. To detect things.

  The analysis result of the image analysis unit 50 is sent to the headlamp control unit 310. For example, the headlamp control unit 310 generates a control signal for controlling the headlamp 350 that is an in-vehicle device of the host vehicle 400 from the distance data calculated by the image analysis unit 50.

  Specifically, for example, while avoiding the strong light of the headlamp of the own vehicle 400 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. In order to realize the driver's visibility, the switching of the high beam and the low beam of the headlamp 350 is controlled, or partial light shielding control of the headlamp 350 is performed.

  The analysis result of the image analysis unit 50 is also sent to the wiper control unit 320. The wiper control unit 320 controls the wiper 360 to remove the raindrops 102 and foreign matters attached to the windshield 101 of the host vehicle 400. The wiper control unit 320 receives a foreign object detection result detected by the image analysis unit 50 and generates a control signal for controlling the wiper 360. When the control signal generated by the wiper control unit 320 is sent to the wiper 360, the wiper 360 is operated to ensure the visibility of the driver of the host vehicle 400.

  The analysis result of the image analysis unit 50 is also sent to the vehicle travel control unit 330. The vehicle travel control unit 330, for example, to the driver of the host vehicle 400 when the host vehicle 400 is out of the lane area defined by the white line based on the white line detection result detected by the image analysis unit 50. Announce a warning. In addition, the vehicle travel control unit 330 performs travel support control such as controlling a handle and a brake of the host vehicle 400, for example.

  As described above, the in-vehicle device control system 300 can be configured using the image processing apparatus 10 according to the present embodiment. However, the image processing apparatus 10 according to the present embodiment is not limited to the in-vehicle device control system, and can be applied to, for example, other systems equipped with an object detection apparatus that performs object detection based on a captured image.

  The preferred embodiment and its modification have been described in detail above, but the present invention is not limited to the above-described embodiment and its modification, and the above-described implementation is performed without departing from the scope described in the claims. Various modifications and substitutions can be added to the embodiment and its modifications.

  For example, in the above-described embodiment and its modifications, the description has been made on the assumption that the image sensor 43 is a monochrome image pickup device. However, the image sensor 43 may be a color image sensor. When the image sensor 43 is a color image sensor, the light transmission characteristics of the polarizing filter layer 47 and the spectral filter layers 49a and 49b are adjusted according to the characteristics of the color filter attached to each pixel of the color image sensor. do it.

  In the above-described embodiment and the modification thereof, the spectral filter layer 49 a is formed on the surface of the filter substrate 46 on the imaging lens 41 side, and the spectral filter layer 49 b is formed on the surface of the filler 48 on the image sensor 43 side. showed that. However, the spectral filter layer 49 b may be formed on the surface of the filter substrate 46 on the imaging lens 41 side, and the spectral filter layer 49 a may be formed on the surface of the filler 48 on the image sensor 43 side.

  However, in this case, the same effect as the former is obtained, but the boundary between the spectral filter layer 49a and the spectral filter layer 49b is more blurred than the former, and the boundary between the raindrop detection image area and the vehicle detection image area is unclear. There is a risk of becoming. Therefore, the former structure is preferable in order to make the boundary between the raindrop detection image area and the vehicle detection image area clearer.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 20 Imaging unit 30 Light source 40 Imaging apparatus 41 Imaging lens 42, 42A, 42B, 42C Optical filter 43 Image sensor 44 Sensor board 45 Signal processing part 46 Filter board 47 Polarization filter layer 48 Filler 49a, 49b, 49c, 49d Spectral filter layer 49x Opening 49y Opening restriction 50 Image analysis unit 101 Windshield 102 Raindrop 103 Cover 105 Background image 106 Part of raindrop 111 Vehicle detection image area 112, 113 Raindrop detection image area 115 Predecessor vehicle 116 White line 117 Road surface 118 Sky 300 In-vehicle device control system 310 Head lamp control unit 320 Wiper control unit 330 Vehicle travel control unit 350 Head lamp 360 Wiper 400 Own vehicle

JP 2005-225250 A

Claims (6)

A light source that emits irradiation light toward the transparent member from one surface side of the transparent member;
Consists of an imaging lens whose focal point is set farther from the position of the transparent member, an optical filter arranged at a subsequent stage of the imaging lens, and a plurality of pixels arranged two-dimensionally to receive light transmitted through the optical filter An image sensor provided on the same side as the light source with respect to the transparent member,
The optical filter includes a substrate that transmits incident light incident through the imaging lens;
A spectral filter layer that is formed in a part of an effective imaging region on the substrate and selectively transmits light in a first wavelength band including an oscillation wavelength range of the light source among the incident light,
The spectral filter layer is formed in a checkered pattern on the substrate,
The pixel of the area | region in which the image of the deposit | attachment adhering to the other surface of the said transparent member was imaged with the 1st exposure amount by the pixel corresponding to the area | region in which the said spectral filter layer was formed, and the said spectral filter layer was not formed An image processing apparatus that captures an image far from the position of the transparent member with a second exposure amount. A light source that emits irradiation light toward the transparent member from one surface side of the transparent member;
Consists of an imaging lens whose focal point is set farther from the position of the transparent member, an optical filter arranged at a subsequent stage of the imaging lens, and a plurality of pixels arranged two-dimensionally to receive light transmitted through the optical filter An image sensor provided on the same side as the light source with respect to the transparent member,
The optical filter includes a substrate that transmits incident light incident through the imaging lens;
A spectral filter layer that is formed in a part of an effective imaging region on the substrate and selectively transmits light in a first wavelength band including an oscillation wavelength range of the light source among the incident light,
The spectral filter layer is formed in a striped pattern on the substrate,
The pixel of the area | region in which the image of the deposit | attachment adhering to the other surface of the said transparent member was imaged with the 1st exposure amount by the pixel corresponding to the area | region in which the said spectral filter layer was formed, and the said spectral filter layer was not formed An image processing apparatus that captures an image far from the position of the transparent member with a second exposure amount. The image processing apparatus according to claim 2 , wherein a stripe direction of the stripe pattern is parallel to a surface formed by an optical axis of the irradiation light of the light source and an optical axis of the imaging lens. A light source that emits irradiation light toward the transparent member from one surface side of the transparent member;
Consists of an imaging lens whose focal point is set farther from the position of the transparent member, an optical filter arranged at a subsequent stage of the imaging lens, and a plurality of pixels arranged two-dimensionally to receive light transmitted through the optical filter An image sensor provided on the same side as the light source with respect to the transparent member,
The optical filter includes a substrate that transmits incident light incident through the imaging lens;
A spectral filter layer that is formed in a part of an effective imaging region on the substrate and selectively transmits light in a first wavelength band including an oscillation wavelength range of the light source among the incident light,
In the region where the spectral filter layer on the substrate is not formed, an opening limiting portion for limiting the amount of transmitted light is formed,
The pixel of the area | region in which the image of the deposit | attachment adhering to the other surface of the said transparent member was imaged with the 1st exposure amount by the pixel corresponding to the area | region in which the said spectral filter layer was formed, and the said spectral filter layer was not formed An image processing apparatus that captures an image far from the position of the transparent member with a second exposure amount. A light source that emits irradiation light toward the transparent member from one surface side of the transparent member;
Consists of an imaging lens whose focal point is set farther from the position of the transparent member, an optical filter arranged at a subsequent stage of the imaging lens, and a plurality of pixels arranged two-dimensionally to receive light transmitted through the optical filter An image sensor provided on the same side as the light source with respect to the transparent member,
The optical filter includes a substrate that transmits incident light incident through the imaging lens;
A spectral filter layer that is formed in a part of an effective imaging region on the substrate and selectively transmits light in a first wavelength band including an oscillation wavelength range of the light source among the incident light,
The spectral filter layer is formed in a part of the effective imaging region of the image sensor side surface of the substrate,
A second spectral filter layer is formed on the entire effective imaging area of the surface of the substrate on the imaging lens side,
In the second spectral filter layer, the light in the first wavelength band of the incident light and the first wavelength band are discontinuous and the second wavelength filter layer has a second wavelength shorter than the first wavelength band. And light in the wavelength band of
The pixel of the area | region in which the image of the deposit | attachment adhering to the other surface of the said transparent member was imaged with the 1st exposure amount by the pixel corresponding to the area | region in which the said spectral filter layer was formed, and the said spectral filter layer was not formed An image processing apparatus that captures an image far from the position of the transparent member with a second exposure amount. The first exposure amount is fixed,
The second exposure, the image processing apparatus of any one of claims 1 to 5 is adjusted based on the image information of the area where the spectral filter layer is not formed.