JP6701542B2 - Detection device, mobile device control system, and detection program - Google Patents

Description

  The present invention relates to a detection device, a mobile device control system, and a detection program.

  Conventionally, a detection device that detects a detection target in an illumination area based on the amount of light received by the imaging means, using captured image data obtained by imaging the illumination area illuminated by the light source light from the light source device with the imaging means. Are known.

  For example, in Patent Document 1, reflected light from a windshield of an automobile illuminated by light source light from a light source device is received by an image sensor (imaging unit) to be imaged, and raindrops adhering to the windshield are detected. A raindrop detection device for detecting is disclosed. In this Patent Document 1, the light source light is totally reflected by the windshield to be incident on the image pickup means in the portion where the adhered matter is not attached, and the light source light is transmitted through the windshield in the portion where the adhered matter is attached. An example of a raindrop detection device that does not enter the imaging means is disclosed. In this example, the amount of light received by the image pickup means is reduced due to the deposition of raindrops on the windshield, and the brightness (pixel value) of the captured image is reduced. Therefore, by detecting the decrease in the brightness of the captured image on the windshield. Raindrops are detected.

  The raindrop detection device disclosed in Patent Document 1 suppresses the influence of ambient light other than the light source light that deteriorates raindrop detection accuracy, and therefore, a lighting image captured when the light source is on (during the illumination period) and a light source off. Raindrops are detected based on the brightness difference value between the image when the light is off and the image taken during (non-illumination period). Since the off-time image shows only ambient light, the brightness difference value obtained by subtracting the off-time image from the on-light image is the brightness value excluding the ambient light component from the on-light image, that is, the received light amount of the light source light. Become. Patent Document 1 describes that by performing raindrop detection based on such a brightness difference value, raindrop detection processing that suppresses the influence of ambient light can be performed.

  However, in a detection device that detects a detection target such as a raindrop based on the amount of light received by the image pickup unit, the factor that reduces the detection accuracy is not only the influence of ambient light other than the light source light. For example, when a light source such as an LED whose light emission amount has a large temperature dependency is used as the light source of the light source device, the light amount of the light source light received by the imaging means changes according to the temperature change. Therefore, the amount of light received by the imaging unit changes regardless of the presence or absence of the detection target, which may cause erroneous detection of the detection target. Further, for example, even when the optical characteristics of the optical member existing on the optical path of the light source light from the light source of the light source device to the image pickup unit change due to temperature change or deterioration over time, the light source light received by the image pickup unit. Since the amount of light changes, there is a possibility that the detection target may be erroneously detected.

In order to solve the above-mentioned problems, the present invention provides an illumination area illuminated by light source light from a light source device at the time of lighting and extinguishing obtained by imaging by an imaging means while repeatedly turning on and off the light source light. In a detection device equipped with detection processing executing means for executing detection processing for detecting a detection target in the illumination area based on the amount of light received by the imaging means using captured image data, the illumination area has a plate-shaped transmission. A region on a member, wherein the detection target is an adhered substance that adheres to one surface of the transmissive member, and the light source device and the imaging means are the one surface with the transmissive member interposed therebetween. It is arranged on the opposite surface side, and the frequency of the fluctuation component of the light receiving amount of the image pickup unit is based on the plurality of imaged image data taken by the image pickup unit during the lighting of the light source light within a predetermined period. Has a low-frequency fluctuation component detecting means for detecting a low-frequency fluctuation component that is a fluctuation component of a predetermined frequency or less, and the detection processing executing means uses the low-frequency fluctuation component detected by the low-frequency fluctuation component detecting means. And a detection device that executes the detection process.
However, the predetermined frequency is at least a fluctuation in which the amount of light received by the image pickup device fluctuates due to a change over time in the optical characteristics of an optical member existing on the optical path of the light source light from the light source of the light source device to the light receiving element of the image pickup device. It is equal to or higher than the frequency of the component, or equal to or higher than the frequency of the fluctuation component in which the amount of light received by the imaging unit fluctuates due to the change in the amount of light emission due to the temperature change of the light source of the light source device.

  According to the present invention, there is an excellent effect that it is possible to suppress erroneous detection of a detection target due to a temperature change of the light source, an optical member, or the like or deterioration with time.

It is a schematic diagram which shows the schematic structure of the vehicle-mounted apparatus control system in embodiment. It is explanatory drawing which shows schematic structure of the imaging part and image analysis unit in the same vehicle equipment control system. It is an explanatory view for explaining an optical system of the imaging unit. It is a perspective view which showed the schematic structure of the same imaging unit typically. (A) is a side view showing an image pickup unit in a state in which the windshield is attached to a vehicle having an inclination angle of 22° with respect to a horizontal plane. FIG. 7B is an explanatory diagram showing an optical system of the image pickup unit in the case where raindrops are not attached in the state of FIG. FIG. 7C is an explanatory diagram showing an optical system of the image pickup unit when raindrops are attached in the state of FIG. It is a perspective view which shows the reflection deflection prism of the imaging unit. It is a graph which shows the filter characteristic of the cut filter which can be applied to the picked-up image data for raindrop detection. It is a graph which shows the filter characteristic of the bandpass filter applicable to the picked-up image data for raindrop detection. It is a front view of a pre-stage filter provided in the optical filter of the imaging unit. It is explanatory drawing which shows the image example of the captured image data of the said imaging part. FIG. 3 is a schematic enlarged view of the optical filter and the image sensor of the imaging unit when viewed from a direction orthogonal to the light transmission direction. It is explanatory drawing which shows the area|region division pattern of the polarization filter layer and spectral filter layer of the same optical filter. It is sectional drawing which shows the layer structure of the optical filter in embodiment typically. It is explanatory drawing which shows the content of the information (information of each image pick-up pixel) corresponding to the light-reception amount which permeate|transmits the vehicle detection filter part of the optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the vehicle detection filter part of the optical filter and the image sensor which were cut|disconnected by the code|symbol AA shown in FIG. (B) is sectional drawing which represented typically the vehicle detection filter part of the optical filter and the image sensor which were cut|disconnected by the code|symbol BB shown in FIG. It is explanatory drawing which shows the content of the information (information of each image pick-up pixel) corresponding to the light reception amount which permeate|transmits the raindrop detection filter part of the optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the raindrop detection filter part of the optical filter and image sensor cut|disconnected by the code|symbol AA shown in FIG. FIG. 17B is a cross-sectional view schematically showing the raindrop detection filter portion of the optical filter and the image sensor, which are cut along the line BB shown in FIG. 16. It is a flow chart which shows a flow of vehicle detection processing in an embodiment. It is an explanatory view of raindrop detection processing in an embodiment. It is explanatory drawing which shows the relationship between the imaging frame and raindrop detection in embodiment. It is explanatory drawing which shows the example in which an exposure period is changed by automatic exposure control within the light irradiation period from a light source part. It is explanatory drawing which shows the example in which an exposure period is changed by automatic exposure control outside the light irradiation period from a light source part. It is a block diagram showing a flow of raindrop detection processing in an embodiment. It is a flow chart which shows a flow of raindrop detection processing in an embodiment. It is explanatory drawing about the various light rays relevant to raindrop detection in a modification.

Hereinafter, one embodiment in which the detection device according to the present invention is applied to an adhering matter detection device used in an in-vehicle device control system as a mobile device control system for controlling a target device mounted on a vehicle such as an automobile that is a mobile object The form will be described.
In addition, the detection device according to the present invention can be widely used in a system that executes various processes and controls by using a detection result of a detection target, and other mobile objects such as vehicles other than automobiles, ships, and aircraft. The present invention can also be used for a control system for various types of mobile device-mounted equipment mounted on a mobile phone, or a control system for a processing device such as FA (factory automation) that is not a device mounted on a mobile body.

FIG. 1 is a schematic diagram showing a schematic configuration of an in-vehicle device control system according to this embodiment.
The in-vehicle device control system uses the imaged image data obtained by capturing an image of the surroundings of the vehicle (particularly, the front area in the traveling direction) as an imaging area by the imaging unit mounted on the vehicle 100 such as an automobile, and distributes the light of the headlamp. The control, the drive control of the wiper, and the control of other in-vehicle devices are performed.

  The imaging unit provided in the in-vehicle device control system according to the present embodiment is provided in the imaging unit 101 and images a front area in the traveling direction of the traveling vehicle 100 as an imaging area. It is installed near the rearview mirror on the windshield 105. The captured image data captured by the image capturing unit of the image capturing unit 101 is input to the image analysis unit 102. The image analysis unit 102 includes a CPU, a RAM, and the like, analyzes captured image data transmitted from the image capturing unit, and uses the captured image data to determine the position (direction and distance) of another vehicle existing in front of the host vehicle 100. The calculation is performed, the attached matter (detection target) such as raindrops or foreign matter attached to the windshield 105 is detected, and the white line (compartment line) on the road surface existing in the imaging area is detected. When detecting another vehicle, a tail lamp of the other vehicle is identified to detect a preceding vehicle traveling in the same traveling direction as the own vehicle 100, and a head lamp of the other vehicle is identified to travel in the opposite direction of the own vehicle 100. The oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. The headlamp control unit 103 generates a control signal for controlling the headlamp 104, which is an in-vehicle device of the own vehicle 100, from the position data of the other vehicle calculated by the image analysis unit 102, for example. Specifically, for example, the driver of another vehicle is prevented from being dazzled by avoiding the strong light of the headlamp of the vehicle 100 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle. The headlamp 104 is controlled to switch between a high beam and a low beam so that the driver's visual field can be secured, or the headlamp 104 is partially shielded from light.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls the wiper 107 as a removing unit to remove the attached matter (detection target) such as raindrops and foreign matter attached to the windshield 105 of the vehicle 100. The wiper control unit 106 receives the adhering matter detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated so as to secure the visibility of the driver of the own vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle running control unit 108 warns the driver of the own vehicle 100 when the own vehicle 100 is out of the lane area divided by the white line. It gives notification and performs driving support control such as controlling the steering wheel and brake of the host vehicle. In addition, the vehicle traveling control unit 108 notifies the driver of the own vehicle 100 of a warning when the distance to the preceding vehicle is close based on the position data of the other vehicle detected by the image analysis unit 102, It also performs driving support control such as controlling the steering wheel and brakes of the vehicle.

FIG. 2 is an explanatory diagram showing a schematic configuration of the image pickup unit 200 provided in the image pickup unit 101 and the image analysis unit 102 constituting the adhering matter detection device.
The image pickup unit 200 mainly includes an image pickup lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 as an image pickup unit in which light receiving elements are two-dimensionally arranged, and an analog electrical output from the sensor substrate 207. The signal processing unit 208 includes a signal (amount of light received by each light receiving element on the image sensor 206) converted into a digital electric signal to generate and output captured image data. Further, in the present embodiment, the light source unit 202 as a light source device is mounted on the sensor substrate 207. The light source unit 202 is disposed on the inner wall surface (one surface) side of the windshield 105, and an adhered matter (hereinafter, the adhered matter is mainly a raindrop) attached to the outer wall surface (the other surface) is taken as an example. Is described below).

  In the present embodiment, the image pickup unit 101 is arranged so that the optical axis of the image pickup lens 204 coincides with the horizontal direction, but the present invention is not limited to this, and the horizontal direction (X direction in FIG. 2) is used as a reference. It may be an example of directing in a specific direction. As the image pickup lens 204, for example, a lens which is composed of a plurality of lenses and whose focus is set farther than the position of the windshield 105 is used. The focal position of the imaging lens 204 can be set to, for example, infinity or between infinity and the windshield 105.

  The image sensor 206 is composed of a plurality of two-dimensionally arranged light receiving elements that receive light that has passed through the cover glass that protects the sensor surface, and has a function of photoelectrically converting incident light for each light receiving element (imaging pixel). Although each pixel of the image sensor 206 is illustrated in a simplified manner in the drawings and the like described later, the image sensor 206 is actually composed of several hundreds of thousands of pixels arranged two-dimensionally. Examples of the image sensor 206 include a CCD (Charge Coupled Device) that simultaneously exposes all imaging pixels (global shutter) to read out signals of each imaging pixel, or a signal of each imaging pixel exposed by line exposure (rolling shutter). It is an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) and the like, and a photodiode can be used for its light receiving element.

  The signal processing unit 208 photoelectrically converts by the image sensor 206 and converts the analog electric signal output from the sensor substrate 207 (the amount of light incident on each light receiving element of the image sensor 206) into a digital electric signal to generate captured image data. It has the function of outputting. The signal processing unit 208 is electrically connected to the image analysis unit 102. When an electric signal (analog signal) is input from the image sensor 206 via the sensor substrate 207, the signal processing unit 208 uses the input electric signal to determine the brightness (luminance) of each imaging pixel on the image sensor 206. To generate a digital signal (captured image data). Then, the captured image data is output to the image analysis unit 102 in the subsequent stage together with the horizontal/vertical synchronization signals of the image.

  The image analysis unit 102 also has a function of controlling the image pickup operation of the image pickup unit 101 and a function of analyzing picked-up image data transmitted from the image pickup unit 101. The image analysis unit 102 uses the exposure amount control unit 102C as an exposure amount control unit to capture an image of the image sensor 206 (an object such as another vehicle existing in the front region of the own vehicle) from the captured image data transmitted from the image capturing unit 101. ), and has a function of setting the optimum exposure amount (exposure time in the present embodiment) for each imaging target of the image sensor 206. Further, the image analysis unit 102 has a function of controlling the light emission timing of the light source unit 202 and the like by the light source control unit 102B serving as a light source control unit while interlocking with the acquisition of the image signal from the signal processing unit 208. Further, the image analysis unit 102 has a function of detecting information regarding a road surface state, a road sign, and the like from the captured image data transmitted from the image capturing unit 101. Further, the image analysis unit 102 has a function of calculating the position, direction, distance, etc. of another vehicle existing in front of the own vehicle from the captured image data transmitted from the image pickup unit 101.

  Further, the image analysis unit 102 has a function of detecting the state of the windshield 105 (adhesion of raindrops, freezing, clouding, etc.) by the detection processing unit 102A. When detecting the attachment of raindrops, the detection processing unit 102A uses the captured image data transmitted from the imaging unit 101 and calculates the amount of raindrops on the windshield 105 according to the brightness value (pixel value) of the captured image data. To do. At this time, in the present embodiment, as will be described later in detail, based on a plurality of captured image data sent from the image capturing unit 101 within a predetermined period, the brightness value fluctuation component of the captured image data is equal to or lower than a predetermined frequency. The low-frequency fluctuation component detecting unit 102D is provided as a low-frequency fluctuation component detecting means for detecting the low-frequency fluctuation component. In this specification, the luminance value variation component is a luminance value time variation component, and the low frequency variation component is a variation component having a long time period.

FIG. 3 is an explanatory diagram for explaining an optical system of the image pickup unit 101 in this embodiment.
The light source unit 202 of the present embodiment emits light source light (illumination light) for detecting adhered substances (raindrops, freezing, cloudiness, etc.) attached to the windshield 105. The light source unit 202 of the present embodiment includes a plurality of LEDs as the light source, and is configured to emit the light source light emitted from each LED to the outside through an optical system such as a collimator lens. By providing a plurality of light sources in this way, the detection area for detecting the adhered matter on the windshield 105 is expanded and the accuracy of detecting the adhered matter on the windshield 105 is increased as compared with the case where the number of the light source is one. improves.

  In the present embodiment, since the LEDs are mounted on the same sensor substrate 207 as the image sensor 206, the number of substrates can be reduced and the cost can be reduced as compared with the case where these are mounted on different substrates. The LEDs are arranged in one row or a plurality of rows along the Y direction in FIG. 3, so that the windshield displayed below the image area in which the image of the vehicle front area is displayed, as described later. It is possible to make uniform the illumination for capturing an image. In the present embodiment, the light source unit 202 is mounted on the same sensor substrate 207 as the image sensor 206, but it may be mounted on a different substrate from the image sensor 206.

  The light source unit 202 is arranged on the sensor substrate 207 so that the optical axis direction of the light emitted by the light source unit 202 and the optical axis direction of the imaging lens 204 have a predetermined angle in advance. Further, the light source unit 202 is arranged on the windshield 105 illuminated by the light emitted from the light source unit 202 such that the illumination range is within the view angle range (viewing angle range) of the imaging lens 204. ing. As a light emission wavelength of the light source unit 202, it is preferable to avoid visible light so as not to dazzle a driver or a pedestrian of an oncoming vehicle. For example, the wavelength is longer than visible light and the light receiving sensitivity of the image sensor 206 extends. A range (for example, an infrared light wavelength range of about 800 to 1000 nm) is used. The drive control of the light emission timing of the light source unit 202 and the like is performed by the light source control unit 102B of the image analysis unit 102 in conjunction with the acquisition of the image signal from the signal processing unit 208.

  As shown in FIG. 3, the image pickup unit 101 of the present embodiment is provided with a reflection deflection prism 230 having a reflection surface 231 that reflects the light from the light source unit 202 and guides it to the windshield 105. The reflection-deflecting prism 230 is arranged such that one surface thereof is in close contact with the inner wall surface of the windshield 105 in order to appropriately guide the light from the light source unit 202 into the windshield 105. Specifically, even if the incident angle of the light from the light source unit 202 that enters the reflection deflection prism 230 changes within a predetermined range, the light is emitted from the light source unit 202 and regularly reflected by the reflection surface 231 of the reflection deflection prism 230. In the light, the specular reflection light specularly reflected at the non-adhered portion on the outer wall surface of the windshield 105 where the raindrop (detection target) is not attached is received by the image sensor 206, The reflection deflection prism 230 is attached to the inner wall surface of the windshield 105.

  When the reflection-deflecting prism 230 is attached to the inner wall surface of the windshield 105, it is preferable to interpose a filler such as a gel made of a light-transmitting material or a sealing material between them to enhance the adhesion. Thereby, it is possible to prevent air layers, bubbles, and the like from intervening between the reflection-deflecting prism 230 and the windshield 105, so that fogging is unlikely to occur between them. Further, the refractive index of the filling material is preferably an intermediate refractive index between the reflective deflection prism 230 and the windshield 105. This makes it possible to reduce Fresnel reflection loss between the filler and the reflection deflection prism 230 and between the filler and the windshield 105. The Fresnel reflection referred to here is reflection that occurs between materials having different refractive indexes.

  As shown in FIG. 3, the reflection-deflecting prism 230 specularly reflects the incident light from the light source unit 202 once by the reflecting surface 231, and folds it back toward the inner wall surface of the windshield 105. The folded light is configured to have an incident angle θ (θ≧about 42°) with respect to the outer wall surface of the windshield 105. This appropriate incident angle θ is a critical angle that causes total internal reflection on the outer wall surface of the windshield 105 due to the difference in refractive index between the air and the outer wall surface of the windshield 105. Therefore, in the present embodiment, when there is no attached matter such as raindrops on the outer wall surface of the windshield 105, the light reflected by the reflection surface of the reflection deflection prism 230 passes through the outer wall surface of the windshield 105. All are reflected without doing.

  On the other hand, when an adhered matter such as a raindrop (refractive index 1.38) different from air (refractive index 1) adheres to the outer wall surface of the windshield 105, this condition of total reflection is broken, and at the place where the raindrop adheres. The light passes through the outer wall surface of the windshield 105. Therefore, regarding the non-adhered portion where the raindrop does not adhere to the outer wall surface of the windshield 105, the reflected light is received by the image sensor 206 and becomes a high-luminance image portion, while the attached portion where the raindrop adheres The amount of the reflected light is reduced, and the amount of light received by the image sensor 206 is reduced, resulting in a low-luminance image portion. Therefore, on the captured image, the contrast between the raindrop attached portion and the non-raindrop attached portion can be obtained.

FIG. 4 is a perspective view schematically showing the schematic configuration of the image pickup unit 101 of the present embodiment.
The image pickup unit 101 of the present embodiment has the reflection deflection prism 230 fixedly supported therein, and the first module 101A as the first support member fixedly arranged on the inner wall surface of the windshield 105, the image sensor 206, and the LED 211 are mounted. The sensor substrate 207, the light guide 215, and the second module 101B as a second support member that fixedly supports the imaging lens 204 are configured.

  These modules 101A and 101B are rotatably connected by a rotary connection mechanism 240. The rotary coupling mechanism 240 has a rotary shaft 241 extending in a direction (front-back direction in FIG. 3) orthogonal to both the tilt direction and the vertical direction of the windshield 105. The one module 101A and the second module 101B can be relatively rotated. The above-described rotatable structure is aimed at the imaging direction of the imaging unit 200 of the second module 101B even when the first module 101A is fixedly arranged with respect to the windshield 105 having different inclination angles. This is so that it can be directed in a specific direction (horizontal direction in this embodiment).

  FIG. 5A is a side view showing the image pickup unit 101 in a state in which the windshield 105 is attached to a vehicle having an inclination angle θg of 22° with respect to the horizontal plane. FIG. 5B is an explanatory diagram showing the optical system of the image pickup unit 101 when raindrops are not attached in the state of FIG. FIG. 5C is an explanatory diagram showing the optical system of the image pickup unit 101 when raindrops are attached in the state of FIG.

  The light L1 from the light source unit 202 is specularly reflected by the reflection surface 231 of the reflection deflection prism 230, and the reflected light L2 is transmitted through the inner wall surface of the windshield 105. When raindrops are not attached to the outer wall surface of the windshield 105, the reflected light L2 is totally reflected on the outer wall surface, and the reflected light L3 is transmitted through the inner wall surface of the windshield 105 and is emitted toward the imaging lens 204. Goes on. On the other hand, when the raindrop 203 is attached to the outer wall surface of the windshield 105, the reflected light L2 specularly reflected by the reflection surface 231 of the reflection deflection prism 230 passes through the outer wall surface. In such a system, when the angle θg of the windshield 105 changes, the second module 101B is fixed to the inner wall surface of the windshield 105 while maintaining the posture of the second module 101B (the imaging direction is fixed in the horizontal direction). The attitude of the module 101B changes, and the reflection-deflecting prism 230 rotates together with the windshield 105 around the Y axis in the figure.

  Here, the positional relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 in the present embodiment is such that the entire outer wall surface of the windshield 105 is always within the rotation adjustment range of the rotary coupling mechanism 240. The reflected light L3 is received by the light receiving area of the image sensor 206 for detecting the change in the windshield state (hereinafter, referred to as “adhesion detecting light receiving area”). Therefore, even if the angle θg of the windshield 105 changes, the total reflected light L3 on the outer wall surface of the windshield 105 is received by the light receiving area for detecting the adhering matter of the image sensor 206, and proper raindrop detection can be realized.

  In particular, the positional relationship in the present embodiment is such that the principle of the corner cube is substantially established within the rotation adjustment range of the rotary coupling mechanism 240. Therefore, even if the angle θg of the windshield 105 changes, the angle θ formed by the optical axis direction of the totally reflected light L3 on the outer wall surface of the windshield 105 and the horizontal plane is substantially constant. Therefore, it is possible to suppress the fluctuation of the portion of the outer wall surface of the windshield 105 through which the optical axis of the totally reflected light L3 passes in the light receiving region for detecting adhered matter of the image sensor 206, and to realize more appropriate raindrop detection. ..

  If the arrangement relationship between the reflection surface 231 of the reflection-deflecting prism 230 and the outer wall surface of the windshield 105 is perpendicular to each other, the corner cube principle holds true, but substantially the corner is within the rotation adjustment range of the rotation coupling mechanism 240. As long as the cube principle is established, the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not limited to the mutually perpendicular relationship. Even if the reflective surface 231 of the reflective deflection prism 230 and the outer wall surface of the windshield 105 are not in a mutually perpendicular relationship, the angles of the other surfaces (the incident surface and the exit surface) of the reflective deflection prism 230 are adjusted. As a result, even if the inclination angle θg of the windshield 105 changes, the angle θ of the optical axis of the totally reflected light L3 traveling toward the imaging lens can be kept substantially constant.

FIG. 6 is a perspective view showing the reflective deflection prism 230 of this embodiment.
The reflection-deflecting prism 230 has an incident surface 233 on which light from the light source unit 202 is incident, a reflecting surface 231 that reflects the light L1 incident from the incident surface 233, and a contact surface 232 that is in close contact with the inner wall surface of the windshield 105. And an emission surface 234 that emits the reflected light L3 reflected by the outer wall surface of the windshield 105 toward the imaging unit 200. In the present embodiment, the incident surface 233 and the emitting surface 234 are configured to be parallel to each other, but they may be non-parallel surfaces.

  The material of the reflection-deflecting prism 230 may be any material that transmits at least the light from the light source unit 202, and glass, plastic, or the like can be used. Since the light from the light source unit 202 of the present embodiment is infrared light, the reflective deflection prism 230 may be made of a black material that absorbs visible light. By using a material that absorbs visible light, it is possible to prevent light other than light (infrared light) from the LED (visible light from the outside of the vehicle) from entering the reflective deflection prism 230.

  Further, the reflection-deflecting prism 230 is formed such that the total reflection condition for totally reflecting the light from the light source unit 202 on the reflection surface 231 is satisfied within the rotation adjustment range of the rotation coupling mechanism 240. Further, when it is difficult to satisfy the condition of total reflection by the reflection surface 231 within the rotation adjustment range of the rotation coupling mechanism 240, a film such as aluminum is vapor-deposited on the reflection surface 231 of the reflection deflection prism 230. A reflective mirror may be formed.

  Further, in the present embodiment, the reflecting surface 231 is a flat surface, but the reflecting surface may be a concave surface. By using such a concave reflecting surface, it is possible to collimate the diffused light flux incident on the reflecting surface. This makes it possible to suppress a decrease in illuminance on the windshield 105.

  Here, when the infrared ray light from the light source unit 202 reflected by the outer wall surface of the windshield 105 is imaged by the image pickup unit 200, the image sensor 206 of the image pickup unit 200 causes the infrared wavelength light from the light source unit 202 as well as the infrared wavelength light from the light source unit 202. A large amount of ambient light including infrared wavelength light such as sunlight is also received. Therefore, in order to distinguish the infrared wavelength light from the light source unit 202 from such a large amount of ambient light, it is necessary to make the light emission amount of the light source unit 202 sufficiently larger than the ambient light. In many cases, it is difficult to use the light source unit 202 that emits a large amount of light.

  Therefore, in the present embodiment, for example, a cut filter that cuts light having a wavelength shorter than the emission wavelength of the light source unit 202 as shown in FIG. 7 or a peak transmittance as shown in FIG. The image sensor 206 is configured to receive the light from the light source unit 202 through a bandpass filter having a light emission wavelength that is substantially the same as that of the light source unit 202. Accordingly, light other than the emission wavelength of the light source unit 202 can be removed and received, so that the amount of light received by the image sensor 206 from the light source unit 202 becomes relatively large with respect to ambient light. As a result, it is possible to distinguish the light from the light source unit 202 from the ambient light even if the light source unit 202 emits a large amount of light.

  However, in the present embodiment, not only the raindrops 203 on the windshield 105 are detected from the captured image data, but also preceding vehicles and oncoming vehicles and white lines are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source unit 202 is removed from the entire captured image, the image sensor 206 outputs light in the wavelength band necessary for detecting a preceding vehicle or an oncoming vehicle or detecting a white line. The light cannot be received, which hinders their detection. Therefore, in the present embodiment, the image area of the captured image data is a raindrop detection image area for detecting raindrops 203 on the windshield 105, and a vehicle for detecting a preceding vehicle or an oncoming vehicle or a white line. A filter for removing the wavelength band other than the infrared wavelength light emitted from the light source unit 202 is arranged in the optical filter 205, which is divided into the detection image area and only the portion corresponding to the raindrop detection image area.

FIG. 9 is a front view of the pre-stage filter 210 provided in the optical filter 205.
FIG. 10 is an explanatory diagram illustrating an image example of captured image data.
The optical filter 205 of the present embodiment has a structure in which a pre-stage filter 210 and a post-stage filter 220 are superposed in the light transmission direction. As shown in FIG. 9, the pre-stage filter 210 includes an infrared light cut filter area 211 and a raindrop detection image area 214, which are arranged at a position corresponding to the upper 2/3 of the captured image which is the vehicle detection image area 213. The area is divided into an infrared light transmitting filter area 212 arranged at a position corresponding to a lower one-third of a captured image. In the infrared light transmission filter area 212, the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. 8 is used.

  Images of headlamps of oncoming vehicles, taillights of preceding vehicles, and white lines are often located mainly from the center of the captured image to the upper part, and the image of the nearest road surface in front of the vehicle is usually present at the lower part of the captured image. Is. Therefore, the information necessary for identifying the headlight of the oncoming vehicle, the taillight of the preceding vehicle, and the white line is concentrated in the upper portion of the captured image, and the information in the lower portion of the captured image is not so important in the identification. Therefore, when the detection of the oncoming vehicle or the preceding vehicle or the white line and the detection of the raindrop are simultaneously performed from the single captured image data, as shown in FIG. 10, the lower portion of the captured image is the raindrop detection image area 214. It is preferable that the upper part of the remaining captured image is a vehicle detection image area 213, and the pre-stage filter 210 is divided into areas correspondingly.

  In the present embodiment, the raindrop detection image area 214 is provided below the vehicle detection image area 213 in the captured image, but the raindrop detection image area 214 is provided above the vehicle detection image area 213. Alternatively, the raindrop detection image area 214 may be provided both above and below the vehicle detection image area 213.

  When the imaging direction of the imaging unit 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, the sunlight reflected on the hood of the host vehicle or the tail lamp of the preceding vehicle becomes ambient light, and this is included in the captured image data, which causes misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in the present embodiment, since the cut filter shown in FIG. 7 and the bandpass filter shown in FIG. 8 are arranged in the portion corresponding to the lower part of the captured image, the sunlight reflected by the bonnet and the preceding Ambient light such as vehicle tail lights is eliminated. Therefore, the accuracy of identifying the headlight of the oncoming vehicle, the taillight of the preceding vehicle, and the white line is improved.

  In the present embodiment, due to the characteristics of the imaging lens 204, the top and bottom of the scene in the imaging area and the image on the image sensor 206 are upside down. Therefore, when the lower portion of the captured image is used as the raindrop detection image area 214, the upper portion of the pre-stage filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. ..

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image, but the tail lamp has a smaller amount of light than the head lamp of the oncoming vehicle, and the disturbance of street lights and the like. Since there is a lot of light, it is difficult to detect the tail lamp with high precision only from the brightness data. Therefore, it is necessary to use the spectral information to identify the tail lamp and identify the tail lamp based on the amount of received red light. Therefore, in the present embodiment, as will be described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the color of the tail lamp) that matches the color of the tail lamp is arranged in the post-stage filter 220 of the optical filter 205, and the red filter is provided. The amount of light received can be detected.

  However, each of the light receiving elements constituting the image sensor 206 of the present embodiment has sensitivity to light in the infrared wavelength band, and therefore, when the light including the infrared wavelength band is received by the image sensor 206, it can be obtained. The captured image becomes reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in this embodiment, in the pre-stage filter 210 of the optical filter 205, the portion corresponding to the vehicle detection image area 213 is the infrared light cut filter area 211. As a result, the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, so that the tail lamp identifying accuracy is improved.

FIG. 11 is a schematic enlarged view when the optical filter 205 and the image sensor 206 are viewed from a direction orthogonal to the light transmission direction.
The image sensor 206 is an image sensor using a CCD, a CMOS or the like as described above, and the photodiode 206A is used for its light receiving element. The photodiodes 206A are arranged in a two-dimensional array for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiodes 206A. The image sensor 206 is bonded to a PWB (printed wiring board) by a method such as wire bonding to form a sensor substrate 207.

  An optical filter 205 is arranged close to the surface of the image sensor 206 on the microlens 206B side. As shown in FIG. 11, the post-stage filter 220 of the optical filter 205 has a laminated structure in which a polarization filter layer 222 and a spectral filter layer 223 are sequentially formed on a transparent filter substrate 221. The polarization filter layer 222 and the spectral filter layer 223 are both divided into regions so as to correspond to one photodiode 206A on the image sensor 206.

  There may be a configuration in which there is a gap between the optical filter 205 and the image sensor 206, but a configuration in which the optical filter 205 is in close contact with the image sensor 206 causes the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205 to be close to each other. It becomes easy to match the boundary of each area with the boundary between the photodiodes 206A on the image sensor 206. The optical filter 205 and the image sensor 206 may be bonded by, for example, a UV adhesive, or the four side regions outside the effective pixel are UV bonded or thermocompression bonded while being supported by a spacer outside the effective pixel range used for imaging. Good.

FIG. 12 is an explanatory diagram showing a region division pattern of the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205 according to this embodiment.
The polarization filter layer 222 and the spectral filter layer 223 are such that two types of regions, a first region and a second region, are arranged corresponding to one photodiode 206A on the image sensor 206. As a result, the amount of light received by each photodiode 206A on the image sensor 206 is expressed as polarization information, spectral information, or the like, depending on the type of region of the polarization filter layer 222 and the spectral filter layer 223 through which the received light is transmitted. Can be obtained.

  In the present embodiment, the image sensor 206 will be described on the premise of a monochrome image pickup device, but the image sensor 206 may be configured by a color image pickup device. When the color image pickup device is used, the light transmission characteristics of the respective regions of the polarization filter layer 222 and the spectral filter layer 223 may be adjusted according to the characteristics of the color filter attached to each image pickup pixel of the color image pickup device.

Here, an example of the optical filter 205 in the present embodiment will be described.
FIG. 13 is a cross-sectional view schematically showing the layer structure of the optical filter 205 in this embodiment.
The post-stage filter 220 of the optical filter 205 in the present embodiment includes a vehicle detection filter section 220A corresponding to the vehicle detection image area 213 and a raindrop detection filter section 220B corresponding to the raindrop detection image area 214, and its layers. The composition is different. Specifically, the vehicle detection filter unit 220A includes the spectral filter layer 223, whereas the raindrop detection filter unit 220B does not include the spectral filter layer 223. Further, the vehicle detection filter unit 220A and the raindrop detection filter unit 220B have different configurations of the polarization filter layers 222 and 225.

FIG. 14 shows the content of information (information of each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 after passing through the vehicle detection filter unit 220A of the optical filter 205 according to this embodiment. FIG.
FIG. 15A is a cross-sectional view schematically showing the vehicle detection filter portion 220A of the optical filter 205 and the image sensor 206 taken along the line AA shown in FIG.
FIG. 15B is a cross-sectional view schematically showing the vehicle detection filter portion 220A of the optical filter 205 and the image sensor 206 taken along the line BB shown in FIG.

  As shown in FIGS. 15A and 15B, the vehicle detection filter portion 220A of the optical filter 205 according to the present embodiment has a polarization filter layer 222 formed on a transparent filter substrate 221, and then formed on the polarization filter layer 222. The spectral filter layer 223 is formed to have a laminated structure. The polarization filter layer 222 has a wire grid structure, and the upper surface (lower side surface in FIG. 15) in the stacking direction becomes an uneven surface. If the spectral filter layer 223 is formed on such an uneven surface as it is, the spectral filter layer 223 is formed along the uneven surface, and the spectral thickness of the spectral filter layer 223 becomes uneven, so that the original spectral performance is obtained. May not be. Therefore, in the optical filter 205 of the present embodiment, the upper surface side of the polarization filter layer 222 in the stacking direction is filled with a filler to be planarized, and then the spectral filter layer 223 is formed thereon.

  As the filling material, any material that does not interfere with the function of the polarization filter layer 222 whose uneven surface is flattened by the filling material may be used. Therefore, in the present embodiment, a material having no polarization function is used. Further, as the flattening treatment with the filler, for example, a method of applying the filler by a spin-on-glass method can be preferably adopted, but the flattening treatment is not limited thereto.

In the present embodiment, the first region of the polarization filter layer 222 is a vertical polarization region that selects and transmits only the vertical polarization component that oscillates in parallel to the vertical column of the image pickup pixels of the image sensor 206. The second region of the layer 222 is a horizontal polarization region that selects and transmits only the horizontal polarization component that oscillates in parallel to the row (horizontal direction) of the image pickup pixels of the image sensor 206.
The first region of the spectral filter layer 223 is a red spectral region in which only light in the red wavelength band (specific wavelength band) included in the usable wavelength band that can be transmitted through the polarization filter layer 222 is selected and transmitted. The second region of the filter layer 223 is a non-spectral region that transmits light without performing wavelength selection. Then, in the present embodiment, as indicated by the dashed line in FIG. 14, the captured image data is composed of a total of four adjacent two vertical and horizontal two imaging pixels (four imaging pixels a, b, e, and f). 1 image pixel of

In the imaging pixel a shown in FIG. 14, light transmitted through the vertical polarization region (first region) of the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel a receives the light P/R in the red wavelength band (indicated by R in FIG. 14) of the vertical polarization component (indicated by P in FIG. 14).
In the imaging pixel b shown in FIG. 14, light transmitted through the vertical polarization region (first region) of the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. .. Therefore, the image pickup pixel b receives the non-spectral light P/C (indicated by reference numeral C in FIG. 14) in the vertically polarized light component P.
In the imaging pixel e shown in FIG. 14, light transmitted through the horizontal polarization region (second region) of the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. .. Therefore, the image pickup pixel e receives the light S/C of the non-spectral light C in the horizontal polarization component (indicated by reference numeral S in FIG. 14).
In the imaging pixel f shown in FIG. 14, light transmitted through the vertical polarization region (first region) of the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the image pickup pixel f receives the light P/R in the red wavelength band R in the vertical polarization component P, similarly to the image pickup pixel a.

  With the above configuration, according to the present embodiment, one image pixel of the vertically polarized component image of red light is obtained from the output signals of the image pickup pixel a and the image pickup pixel f, and the non-spectral vertical image is obtained from the output signal of the image pickup pixel b. One image pixel for the polarization component image is obtained, and one image pixel for the non-spectral horizontal polarization component image is obtained from the output signal of the imaging pixel e. Therefore, according to this embodiment, three types of captured image data, that is, the vertically polarized component image of red light, the vertically polarized component image of non-spectral light, and the horizontally polarized component image of non-spectral light can be obtained by one image capturing operation. ..

Although the number of image pixels of these captured image data is smaller than the number of captured pixels, a generally known image interpolation technique may be used to obtain an image with higher resolution. For example, in the case of obtaining a vertically polarized component image of red light having a higher resolution, for the image pixels corresponding to the imaging pixel a and the imaging pixel f, the vertically polarized light of the red light received by these imaging pixels a and f. The information of the component P is used as it is, and for the image pixel corresponding to the image pickup pixel b, for example, the average value of the image pickup pixels a, c, f, and j surrounding the image pixel b is used as the information of the vertical polarization component of the red light of the image pixel. To use as.
Further, when trying to obtain a non-spectral horizontal polarization component image having a higher resolution, the information of the non-spectral horizontal polarization component S received by the imaging pixel e is used as it is for the image pixel corresponding to the imaging pixel e. For the image pixels corresponding to the image pickup pixels a, b, and f, the average value of the image pickup pixel e or the image pickup pixel g that receives the non-spectral horizontal polarization component around the image pixels is used, or the same as the image pickup pixel e. You may use the value.

  The vertically polarized component image of red light thus obtained can be used, for example, for identifying a tail lamp. In the vertically polarized component image of red light, the horizontal polarized component S is cut off, so that the red light reflected on the road surface and the red light (reflected light) from the dashboard in the room of the vehicle 100 are horizontal. It is possible to obtain a red image in which the disturbance factor due to the red light having a strong polarization component S is suppressed. Therefore, by using the vertically polarized component image of red light for identifying the tail lamp, the recognition rate of the tail lamp is improved.

  Further, the non-spectral vertically polarized component image can be used, for example, for identifying a white line or a headlamp of an oncoming vehicle. In the non-spectral horizontal polarization component image, the horizontal polarization component S is cut off, so white light from a headlamp, streetlight, etc. reflected on the road surface or white light (reflective light) from a dashboard or the like inside the vehicle 100 is reflected. It is possible to obtain a non-spectral image in which the disturbance factor due to the white light having a strong horizontal polarization component S is suppressed as in (1). Therefore, by using the non-spectral vertical polarization component image for identifying the white line and the headlamp of the oncoming vehicle, the recognition rate is improved. Particularly in rainy roads, it is generally known that the light reflected from the water surface covering the road surface has a large amount of horizontal polarization component S. Therefore, by using the non-spectral vertical polarization component image for identifying the white line, the white line under the water surface in the rain road can be appropriately identified, and the recognition rate is improved.

  Further, if a comparison image in which the index value obtained by comparing each pixel value between the non-spectral vertical polarization component image and the non-spectral horizontal polarization component image is used as the pixel value is used, as described later, It is possible to identify an object, a dry and wet condition of a road surface, a three-dimensional object in an imaging region, and a white line on a rainy road with high accuracy. As the comparison image used here, for example, a difference image in which the difference value of the pixel values between the non-spectral vertical polarization component image and the non-spectral horizontal polarization component image is the pixel value, and the pixel value between these images A ratio image with a pixel value as a ratio, or a difference polarization degree image with a pixel value that is the ratio of the difference value of the pixel values between these images to the sum of the pixel values between these images is used. can do.

FIG. 16 shows the content of information (information of each image pickup pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 after passing through the raindrop detection filter unit 220B of the optical filter 205 in this embodiment. FIG.
17A is a cross-sectional view schematically showing the raindrop detection filter portion 220B of the optical filter 205 and the image sensor 206, which are cut along the line AA shown in FIG.
17B is a cross-sectional view schematically showing the raindrop detection filter portion 220B of the optical filter 205 and the image sensor 206, which are cut along the line BB shown in FIG.

  As shown in FIGS. 17A and 17B, the raindrop detection filter portion 220B of the optical filter 205 of the present embodiment has a wire grid structure on the filter substrate 221 common to the vehicle detection filter portion 220A. The polarization filter layer 225 is formed. The polarization filter layer 225 is flattened with the polarization filter layer 222 of the vehicle detection filter section 220A by filling the upper surface side in the stacking direction with a filler. However, the raindrop detection filter unit 220B is different from the vehicle detection filter unit 220A in that the spectral filter layer 223 is not laminated.

  In the present embodiment, the landscape on the indoor side of the vehicle 100 may be reflected on the inner wall surface of the windshield 105. This reflection is due to the light regularly reflected by the inner wall surface of the windshield 105. Since this reflection is specularly reflected light, it is disturbance light having a relatively high light intensity. Therefore, when this glare is reflected in the raindrop detection image area 214 together with the raindrop, the raindrop detection accuracy is reduced. Further, when the specularly reflected light, which is the light from the light source unit 202 specularly reflected on the inner wall surface of the windshield 105, is also projected on the raindrop detection image area 214 together with the raindrops, it becomes disturbance light and the raindrop detection accuracy is reduced. Let

  Since the ambient light that reduces the detection accuracy of such raindrops is the regular reflection light that is regularly reflected by the inner wall surface of the windshield 105, most of its polarization components are polarization components whose polarization direction is perpendicular to the light source incident surface. That is, the horizontal polarization component S that oscillates in parallel to the row (horizontal direction) of the image pickup pixels of the image sensor 206. Therefore, the polarization filter layer 225 in the raindrop detection filter portion 220B of the optical filter 205 of the present embodiment has a virtual surface (including the optical axis of the light from the light source portion 202 toward the windshield 105 and the optical axis of the imaging lens 204 ( The transmission axis is set so that only the polarization component whose polarization direction is parallel to the light source incident surface), that is, the vertical polarization component P that oscillates parallel to the vertical column (vertical direction) of the image sensor 206 is transmitted. ing.

  As a result, the light transmitted through the polarization filter layer 225 of the raindrop detection filter unit 220B becomes only the vertical polarization component P, and the light is reflected on the inner wall surface of the windshield 105 or is regularly reflected by the inner wall surface of the windshield 105. It is possible to cut the horizontal polarization component S that occupies most of ambient light such as specularly reflected light from 202. As a result, the raindrop detection image area 214 becomes a vertically polarized image due to the vertical polarization component P that is less affected by ambient light, and the raindrop detection accuracy based on the imaged image data of the raindrop detection image area 214 is improved.

  In the present embodiment, the infrared light cut filter area 211 and the infrared light transmission filter area 212 that form the pre-stage filter 210 are formed by multilayer films having different layer configurations. As a method of manufacturing such a pre-stage filter 210, the infrared light cut filter region 211 is hidden by a mask while the infrared light transmission filter region 212 is formed by vacuum evaporation or the like, and then the infrared light is removed. There is a method of forming a film on the portion of the infrared light cut filter region 211 by vacuum evaporation while hiding the portion of the transmission filter region 212 with a mask.

  Further, in the present embodiment, both the polarization filter layer 222 of the vehicle detection filter unit 220A and the polarization filter layer 225 of the raindrop detection filter unit 220B have a wire grid structure that is divided into two-dimensional regions. However, in the former polarization filter layer 222, two types of areas (vertical polarization area and horizontal polarization area) whose transmission axes are orthogonal to each other are divided into image-capturing pixel units, and the latter polarization filter layer 225 is defined by vertical polarization. One type of area having a transmission axis that allows only the component P to be transmitted is divided into image pickup pixel units. When the polarization filter layers 222 and 225 having such different configurations are formed on the same filter substrate 221, for example, adjustment in the groove direction of a template (corresponding to a mold) for patterning a metal wire having a wire grid structure. Thus, the adjustment of the metal wire in each region in the longitudinal direction is easy.

In the present embodiment, the infrared light cut filter area 211 may not be provided in the optical filter 205, and for example, the infrared light cut filter area 211 may be provided in the imaging lens 204. In this case, the optical filter 205 can be easily manufactured.
Further, instead of the infrared light cut filter area 211, a spectral filter layer that transmits only the vertically polarized component P may be formed in the raindrop detection filter portion 220B of the post-stage filter 220. In this case, it is not necessary to form the infrared light cut filter area 211 in the pre-stage filter 210.
Further, the polarization filter layer does not necessarily have to be provided.
Further, in the optical filter 205 of the present embodiment, the post-stage filter 220 having the polarization filter layer 222 and the spectral filter layer 223 divided into regions as shown in FIG. 14 is provided closer to the image sensor 206 side than the pre-stage filter 210. However, the pre-stage filter 210 may be provided closer to the image sensor 206 side than the post-stage filter 220.

[Headlamp light distribution control]
The light distribution control of the headlamp in this embodiment will be described below.
In the light distribution control of the headlamp in the present embodiment, the image data captured by the image capturing unit 200 is analyzed to identify the tail lamp and the head lamp of the vehicle, the leading vehicle is detected from the identified tail lamp, and the identified head is detected. The oncoming vehicle is detected from the lamp. Then, the driver of the own vehicle 100 is prevented from being dazzled by avoiding the strong light of the headlamp of the own vehicle 100 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle, and the visibility of the driver of the own vehicle 100 is secured. In order to realize the above, the switching between the high beam and the low beam of the headlamp 104 is controlled, and the partial light shielding control of the headlamp 104 is performed.

  In the headlamp light distribution control of the present embodiment, of the information that can be acquired from the image pickup unit 101, the intensity of light (brightness information) emitted from each point (light source) in the image pickup area, the headlamp, and the like. Distance (distance information) between a light source such as a tail lamp (other vehicle) and the vehicle, spectral information obtained by comparing the red component and white component (non-spectral) of the light emitted from each light source, horizontal polarization component of the white component And the vertical polarization component by comparison between the vertical polarization component, the white polarization component with the horizontal polarization component cut, and the red polarization component information with the horizontal polarization component cut.

  Explaining the brightness information, when the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are imaged by the image capturing unit 200, the identification target object on the captured image data is detected. The headlamp of one oncoming vehicle, which is one, is projected brightest, and the taillamp of the preceding vehicle, which is one of the identification objects, is projected darker than that. In addition, when the reflector is reflected in the imaged image data, the reflector is not a light source that emits light by itself, but is reflected brightly by reflecting the headlamps of the own vehicle, so it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlights of the oncoming vehicle, the taillights of the preceding vehicle, and the reflector is observed to be darker on the image sensor 206 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, it is possible to perform primary identification of two types of identification objects (a head lamp and a tail lamp) and a reflector.

  In addition, distance information will be explained. Since most headlamps and tail lamps are composed of a pair of left and right pair lamps, the characteristics of this configuration are used to measure the distance between the headlamp or tail lamp (that is, another vehicle) and the host vehicle. It is possible to ask. The paired left and right lamps are projected close to each other in the same height direction position on the captured image data captured by the image capturing unit 200, and the lamp image areas projecting the lamps have substantially the same size, and The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, the lamp image areas that satisfy the conditions can be identified as paired lamps. Note that at a long distance, the left and right lamps that make up the pair of lamps cannot be recognized separately and are recognized as a single lamp.

  When the pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp or the tail lamp, which is the pair lamp configuration. That is, the distance between the left and right headlamps and the distance between the left and right taillamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the image pickup lens 204 in the image pickup unit 200 is known, the distance w1 between the two lamp image areas corresponding to the left and right lamps on the image sensor 206 of the image pickup unit 200 is calculated from the image pickup image data. As a result, the distance x between the light source of the headlamp or tail lamp, which is the paired lamp configuration, and the host vehicle can be obtained by a simple proportional calculation (x=f×w0/w1). If the distance x calculated in this way is within the proper range, the two lamp image areas used for the calculation can be identified as the headlamp and the tail lamp of another vehicle. Therefore, by using this distance information, the accuracy of distinguishing the head lamp and the tail lamp, which are the objects to be distinguished, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, the imaging pixel a on the image sensor 206 that receives the red light (vertical polarization component) P/R from the imaged image data captured by the imaging unit 200. By extracting only the pixel data corresponding to c, f, h, etc., it is possible to generate a red image in which only the red component within the imaging region is projected. Therefore, when there is an image area having a brightness equal to or higher than the predetermined brightness in the red image, the image area can be identified as a tail lamp image area showing the tail lamp.

  In addition, only the pixel data corresponding to the imaged pixels b and d on the image sensor 206 that receives the vertical polarization component P/C of white light (non-spectral light) is extracted from the imaged image data taken by the image pickup unit 200. Thus, it is possible to generate a monochrome luminance image (vertical polarization component) in the imaging area. Therefore, it is also possible to calculate the luminance ratio (red luminance ratio) between the image area on the red image and the image area on the monochrome luminance image corresponding to this image area. By using this red luminance ratio, it is possible to grasp the relative ratio of the red component included in the light from the object (light source body) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the tail lamp identification accuracy is improved by using this red luminance ratio.

  Further, the polarization information will be described. In the present embodiment, as described above, the imaging on the image sensor 206 that receives the vertical polarization component P/C of the white light (non-spectral light) from the imaged image data captured by the imaging unit 200. Pixel data corresponding to the pixels b and d, and pixel data corresponding to the imaging pixels e and g on the image sensor 206 that receives the horizontal polarization component S/C of white light (non-spectral light) are extracted, It is possible to obtain a comparison image in which pixel values (luminance) between these image data are compared for each image pixel. Specifically, for example, a difference image in which the difference value (SP) between the vertical polarization component P of white light (non-spectral) and the horizontal polarization component S of white light (non-spectral) is a pixel value is compared image. Can be obtained as According to such a comparative image, the image area of the direct light (the headlamp image area) that is directly incident on the imaging unit 200 from the headlamp and the incident light on the imaging unit 200 after being reflected from the water surface of the rain road from the headlamp. The contrast of the indirect light with the image area can be increased, and the accuracy of headlamp identification is improved.

  In particular, as a comparative image, a ratio image in which the ratio (S/P) of the vertical polarization component P of white light (non-spectral) and the horizontal polarization component S of white light (non-spectral) is a pixel value, and the differential polarization degree A differential polarization degree image having a pixel value of ((S−P)/(S+P)) can be preferably used. Generally, it is known that the light reflected by a horizontal mirror surface such as a water surface always has a strong horizontal polarization component. In particular, the ratio (S/P) of the horizontal polarization component S and the vertical polarization component P and the differential polarization degree are known. It is known that when ((S−P)/(S+P)) is taken, the ratio and the degree of differential polarization are maximum at a specific angle (Brewster angle). In a rainy road, water is spread on the asphalt surface, which is a scattering surface, and becomes a state close to a mirror surface. Therefore, the headlamp reflected light from the road surface is stronger in the horizontal polarization component S. Therefore, the image area of the headlamp reflected light from the road surface has a large pixel value (luminance) in the ratio image and the differential polarization degree image. On the other hand, since the direct light from the headlamp is basically non-polarized, its pixel value (luminance) is small in the ratio image and the differential polarization degree image. Due to this difference, it is possible to properly exclude the headlight reflected light from the rain road surface having the same light quantity as the direct light from the headlamp, and to distinguish the direct light from the headlamp by distinguishing it from such headlamp reflected light. be able to.

Next, a flow of detection processing of the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 18 is a flowchart showing the flow of vehicle detection processing in this embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the image capturing unit 200 to extract an image region that is considered to be an identification target. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of identification objects the type of light source body displayed in the image area is.

  First, in step S1, the image data in front of the vehicle captured by the image sensor 206 of the image capturing unit 200 is loaded into the memory. As described above, this image data includes a signal indicating the brightness at each image pickup pixel of the image sensor 206. Next, in step S2, information regarding the behavior of the host vehicle is fetched from the vehicle behavior sensor.

  In step S3, a high-luminance image region (high-luminance image region) that is considered to be an identification target (a tail lamp of a preceding vehicle and a bed lamp of an oncoming vehicle) is extracted from the image data captured in the memory. This high-brightness image area is a bright area having a brightness higher than a predetermined threshold brightness in image data, and a plurality of bright areas often exist, but all of them are extracted. Therefore, at this stage, the image area showing the reflected light from the rain road surface is also extracted as the high-luminance image area.

  In the high-luminance image region extraction processing, first, in step S31, the binarization processing is performed by comparing the luminance value of each image pickup pixel on the image sensor 206 with a predetermined threshold luminance. Specifically, a binary image is created by allocating “1” to pixels having a brightness equal to or higher than a predetermined threshold brightness and “0” to pixels not. Next, in step S32, if the pixels assigned with "1" are close to each other in this binarized image, a labeling process for recognizing them as one high-luminance image region is performed. As a result, a set of adjacent pixels having high brightness values is extracted as one high brightness image area.

  In step S4 executed after the above-described high-brightness image area extraction processing, the distance between the object and the vehicle in the image pickup area corresponding to each extracted high-brightness image area is calculated. In this distance calculation process, the pair of lamps on the left and right of the vehicle are used to detect the distance, and when the distance becomes long, the left and right lamps forming the pair lamp cannot be distinguished and recognized. A single lamp distance calculation process when the paired lamp is recognized as a single lamp is executed.

  First, for pair lamp distance calculation processing, in step S41, pair lamp creation processing, which is processing for creating lamp pairs, is performed. The pair of left and right lamps, which are paired, are close to each other in the image data picked up by the image pickup unit 200 and have substantially the same height, and the areas of the high-brightness image areas are almost the same, and the shape of the high-brightness image area is the same. The condition of being the same is satisfied. Therefore, the high-luminance image areas satisfying such a condition are paired with each other. High intensity image areas that cannot be paired are considered as single lamps. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation processing in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the image pickup unit 200 is known, the actual distance x to the pair of lamps can be calculated by a simple proportional calculation (x= f·w0/w1). A dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio of the red image of the vertical polarization component P and the white image of the vertical polarization component P (red luminance ratio) is used as the spectral information, and from this spectral information, two high-luminance image regions that are pair lamps are obtained. A lamp type identification process is performed to identify whether the light is from the headlamp or the tail lamp. In this lamp type identification process, first, in step S51, pixel data corresponding to the imaging pixels a and f on the image sensor 206 and pixels corresponding to the imaging pixel b on the image sensor 206 in the high-luminance image area that is a pair lamp. A red-ratio image with a pixel value corresponding to the ratio with the data is created. Then, in step S52, the pixel value of the red-ratio image is compared with a predetermined threshold value, and a high-luminance image area equal to or higher than the predetermined threshold value is a tail lamp image area due to light from the tail lamp, and is less than the predetermined threshold value. For a certain high-luminance image area, lamp type processing is performed that is a headlamp image area by light from the headlamp.

  Subsequently, in step S6, the differential polarization degree ((S−P)/(S+P)) is used as the polarization information for each of the image areas identified as the tail lamp image area and the head lamp image area, and the image is output from the tail lamp or the head lamp. Is performed to determine whether the direct light is reflected light or the reflected light received by being reflected by a mirror surface portion such as a rain road surface. In the reflection identification processing, first, in step S61, the differential polarization degree ((SP)/(S+P)) is calculated for the tail lamp image area, and a differential polarization degree image having the differential polarization degree as a pixel value is created. Similarly, the degree of polarization difference ((S−P)/(S+P)) is calculated for the headlamp image area, and a difference polarization degree image having the pixel value as the difference polarization degree is created. Then, in step S62, the pixel value of each differential polarization degree image is compared with a predetermined threshold value, and the tail lamp image area and the head lamp image area that are equal to or greater than the predetermined threshold value are determined to be due to reflected light, It is determined that these image areas do not show the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle and are excluded. The tail lamp image area and the head lamp image area remaining after performing this exclusion process are identified as showing the tail lamp of the preceding vehicle or showing the head lamp of the oncoming vehicle.

  Note that a rain sensor or the like may be mounted on the vehicle, and the above-described reflection identification processing S6 may be executed only when the rain sensor confirms that it is raining. Further, the above-described reflection identifying process S6 may be executed only when the driver (driver) is operating the wiper. In short, the above-described reflection identifying process S6 may be executed only in rainy weather when reflection from a rain road surface is expected.

  In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection processing described above are used for the light distribution control of the headlamp, which is an in-vehicle device of the own vehicle. Specifically, when the tail lamp is detected by the vehicle detection process and the vehicle approaches the rearview mirror of the preceding vehicle within the distance range where the headlamp illumination light of the own vehicle is incident, the preceding vehicle is illuminated by the headlamp illumination of the own vehicle. Control is performed such that a part of the headlamp of the own vehicle is blocked so that the light does not hit, or the light irradiation direction of the headlamp of the own vehicle is shifted in the vertical direction or the horizontal direction. Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range where the headlamp illumination light of the own vehicle hits, the oncoming vehicle is illuminated by the headlamp illumination light of the own vehicle. In order to prevent the vehicle from hitting, control is performed such that a part of the headlamp of the own vehicle is shielded from the light or the light irradiation direction of the headlamp of the own vehicle is shifted in the vertical direction or the horizontal direction.

[White line detection processing]
The white line detection processing in this embodiment will be described below.
In the present embodiment, a process of detecting a white line (compartment line) as an identification target is performed in order to prevent the own vehicle from deviating from the drivable area. The white line here includes all white lines such as a solid line, a broken line, a dotted line, and a double line that divide the road. It should be noted that it is possible to detect a division line of a color other than white such as a yellow line in the same manner.

  In the white line detection processing in the present embodiment, of the information that can be acquired from the imaging unit 101, the polarization information of the vertical polarization component P of the white component (non-spectral) is used. The vertically polarized component of the white component may include the vertically polarized component of cyan light. It is generally known that white lines and asphalt surfaces have flat spectral luminance characteristics in the visible light region. On the other hand, since cyan light includes a wide band in the visible light region, it is suitable for imaging asphalt and white lines. Therefore, by using the optical filter 205 in the configuration example 2 and including the vertically polarized component of cyan light in the vertically polarized component of the white component, the number of image pickup pixels to be used is increased, resulting in an increase in resolution and a distant white line. Can also be detected.

  In the white line detection processing of the present embodiment, many roads have white lines formed on the road surface of a color close to black, and in the image of the vertical polarization component P of the white component (non-spectral), the brightness of the white line portion is on the road surface. Larger than other parts. Therefore, the white line can be detected by determining the part of the road surface where the brightness is equal to or higher than the predetermined value as the white line. In particular, in the present embodiment, the image of the vertical polarization component P of the white component (non-spectral light) used has the horizontal polarization component S cut off, and therefore, an image in which reflected light from a rain road is suppressed is acquired. Is possible. Therefore, it is possible to perform white line detection without erroneously recognizing ambient light such as reflected light from a headlamp from a rain road at nighttime as a white line.

  In addition, in the white line detection processing according to the present embodiment, among the information that can be acquired from the imaging unit 101, polarization information obtained by comparing the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P, for example, white The degree of difference polarization ((S−P)/(S+P)) between the component (non-spectral) horizontal polarization component S and the vertical polarization component P may be used. Since the diffuse reflection component is usually dominant in the reflected light from the white line, the vertical polarization component P and the horizontal polarization component S of the reflected light are almost the same, and the differential polarization degree shows a value close to zero. On the other hand, the asphalt surface portion in which the white line is not formed exhibits the characteristic that the scattered reflection component is dominant in the dry state, and the differential polarization degree thereof shows a positive value. Further, in the asphalt surface portion where the white line is not formed, the specular reflection component becomes dominant in the wet state, and the differential polarization degree thereof shows a larger value. Therefore, it is possible to determine a portion where the obtained polarization difference value of the road surface portion is smaller than the predetermined threshold as a white line.

[Raindrop detection processing on the windshield]
Hereinafter, the raindrop detection process in this embodiment will be described.
In the present embodiment, for the purpose of controlling the drive of the wiper 107 and controlling the discharge of the washer liquid, a process of detecting raindrops as adhering substances is performed. In addition, here, the case where the attached matter on the windshield is a raindrop will be described as an example, but the same applies to the attached matter such as bird droppings and splashes on the road surface bounced from an adjacent vehicle. is there.

  In the raindrop detection process according to the present embodiment, the polarization filter layer 225 in the infrared light transmission filter region 212 of the pre-stage filter 210 and the polarization filter layer 225 of the raindrop detection filter unit 220B of the post-stage filter 220 is included in the information that can be acquired from the imaging unit 101. The polarization information of the vertical polarization component P of the raindrop detection image area 214 that receives the transmitted light is used. Therefore, it is necessary that the light to be incident on the windshield 105 from the light source unit 202 contains a large amount of the vertically polarized component P. For that purpose, for example, when a light emitting diode (LED) is used as a light source of the light source unit 202, it is preferable to dispose a polarizer that transmits only the vertically polarized component P between the light source unit 202 and the windshield 105. When a semiconductor laser (LD) is used as the light source of the light source unit 202, the LD can emit only the light of the specific polarization component, so that the light of only the vertical polarization component P is incident on the windshield 105. The axes of may be aligned.

  In the present embodiment, as described above, the illumination light (infrared light) emitted from the light source unit 202 and incident on the inner wall surface of the windshield 105 from the reflection deflection prism 230 has raindrops attached on the outer wall surface of the windshield 105. At the non-adhered portion, which is not attached, specular reflection occurs on the outer wall surface of the windshield 105. The specularly reflected light is received by the image sensor 206 and displayed on the raindrop detection image area 214. On the other hand, at a place where raindrops are attached to the outer wall surface of the windshield 105, the illumination light is transmitted through the outer wall surface of the windshield 105, and the transmitted light is not received by the image sensor 206. Therefore, in the raindrop detection image area 214 of the captured image data, the non-adhered portion where the raindrop is not attached to the outer wall surface of the windshield 105 is a high brightness image portion (high pixel value), while the raindrop is attached. The attached portion is a low-brightness image portion (low pixel value). From such a difference, it is possible to grasp not only the presence or absence of raindrops but also the amount of raindrops.

FIG. 19 is an explanatory diagram of the raindrop detection process in this embodiment.
In the raindrop detection process of the present embodiment, the information of the raindrop detection image area 214 of the captured image data acquired from the image pickup unit 101 is used, and when it is determined that the raindrop amount exceeds a certain amount, the wiper 107 is driven. Specifically, the raindrop detection image area 214 is divided into eight sections (x=1 to 8) in the lateral direction of the image as shown in FIG. 19, and the total luminance value y(x,t) in each raindrop detection section x is divided. The raindrop fluctuation component r(x, t) based on Note that “t” is the execution timing of the raindrop detection process. Then, when the raindrop variation component r(x, t) of any of the raindrop detection categories x falls below a predetermined threshold value rth, it is determined that the raindrop amount exceeds a certain amount, and the wiper 107 is driven. On the contrary, when the raindrop variation component r(x, t) of any of the raindrop detection sections x becomes equal to or larger than a predetermined threshold value rth, the drive of the wiper 107 is stopped. The conditions for starting and stopping the driving of the wiper 107 are not limited to this, and can be set as appropriate. For example, the threshold does not have to be a fixed value, and may be appropriately changed according to a change in the surroundings of the vehicle on which the image capturing unit 200 is mounted. The threshold values for the start condition and the stop condition may be the same value or different values.

FIG. 20 is an explanatory diagram showing the relationship between the imaging frame and raindrop detection in this embodiment.
Conventionally, an imaging frame (sensing frame) for detecting a white line (compartment line) on the road surface or an identification object such as another vehicle existing in the imaging area, and an imaging frame for detecting raindrops (raindrop detection) Frame) was a different imaging frame. In this case, since there is a time lag between the sensing frames before and after the raindrop detection frame, the identification target object is detected between the time when the previous sensing frame is imaged and the time when the later sensing frame is imaged. There is a time during which an image for identification cannot be obtained, which causes a problem such as deterioration in recognition accuracy of the identification target.

  On the other hand, in the present embodiment, as shown in FIG. 20, an image having an image (vehicle detection image area) for detecting the identification target and a raindrop detection image area for detecting the raindrop 203. Can be obtained in a single imaging frame frame2,4,6,8,10,12. In this case, since the dedicated image pickup frame for detecting raindrops (raindrop detection frame) is not required, the above-mentioned problems do not occur.

  Here, in the present embodiment, in order to obtain a suitable image area for vehicle detection (for example, an image area with high contrast), automatic exposure control (AEC) that changes the exposure amount according to the situation (brightness, etc.) of the imaging area. ) Is being implemented. Specifically, the exposure amount control unit 102C of the image analysis unit 102 adjusts the exposure time of the next imaging frame according to the brightness value of the central portion of the captured image (pixels in the vehicle detection image area 213) in the previous imaging frame. Automatic exposure control (AEC) for changing (exposure amount) is performed.

  When such automatic exposure control is carried out, even if the raindrops adhered on the windshield are exactly the same, the brightness and contrast of the raindrop detection image area may be changed by changing the exposure time. For example, as shown in FIG. 21, when the exposure period is changed by the automatic exposure control within the light irradiation period from the light source unit 202, the light receiving time of the light source light by the image sensor 206 (on the windshield is changed depending on the length of the exposure period). The time to receive the reflected light of the light source light from the non-raindrop adhering part of) changes. In this case, even if the amount of raindrops adhering to the windshield is the same, the light receiving amount of the light source light received by the image sensor 206 changes. Therefore, as a result of obtaining the raindrop detection image area 214 having different brightness, contrast, and the like, the raindrop amount cannot be properly detected, and a problem such as a malfunction of the wiper 107 may occur.

  Therefore, in the present embodiment, as shown in FIG. 22, the exposure period is set to be changed by the automatic exposure control outside the light irradiation period from the light source unit 202. In this case, since the light receiving time of the light source light by the image sensor 206 is constant during the light irradiation period even if the exposure period is changed, it is possible to appropriately detect the amount of raindrops and suppress problems such as malfunction of the wiper 107. it can.

Next, the raindrop amount calculation processing for calculating the raindrop fluctuation component r(x, t) from the total luminance value y(x, t) in each raindrop detection section x used in the raindrop detection processing will be described.
In the present embodiment, as described above, the amount of light received by the image sensor 206 from the light source unit 202 due to the deposition of raindrops on the windshield is reduced, and the brightness value of the raindrop detection image area is reduced. Use to detect raindrops on the windshield. That is, the total luminance value y(x, t) in each raindrop detection section x is the total luminance value y(x, t) and the reference luminance, when there is no fluctuation component other than the fluctuation component due to raindrop attachment. The difference from the value y(x,0) can be used as the raindrop variation component r(x,t), and the raindrop amount can be grasped from the raindrop variation component r(x,t).

  The reference brightness value y(x, 0) is a total brightness value acquired for each raindrop detection category x under a predetermined temperature environment under the condition that raindrops are not attached, the condition that ambient light does not exist. .. In the present embodiment, the light intensity of the light source light emitted from the light source unit 202 varies due to individual differences of the light sources, so that the reference luminance value y(x, 0) is obtained by actually analyzing the reference luminance value y(x, 0) in advance. It is preferably stored in the storage means of the unit 102.

  When there is a variation component in addition to the raindrop variation component in the total luminance value y(x, t) in each raindrop detection category x, the total luminance value y(x, t) and the reference luminance value y(x, If the difference from 0) is used as the raindrop fluctuation component r(x, t) as it is, the raindrop may be erroneously detected, and the wiper 107 may malfunction.

  Examples of the fluctuation component that causes erroneous detection of raindrops include a fluctuation component d(x,t) due to ambient light other than the light source light that is incident on the sensor portion of the image sensor 206 corresponding to the raindrop detection image area.

  In addition, the optical characteristics of optical members such as lenses, mirrors, and prisms, and the temperature fluctuation component that changes the luminance value of the image area for raindrop detection due to fluctuations in the light emission amount of the light source and the like, and the optical characteristics of these optical members. A variation component that deteriorates over time that varies the brightness value of the raindrop detection image region by varying the light emission amount of the light source or the like due to deterioration over time may also be a variation component that causes erroneous detection of raindrops. In particular, in the present embodiment, since the LED whose light emission amount has a high temperature dependency is used as the light source, the light emission amount of the light source may fluctuate enough to cause an erroneous detection of raindrops due to a temperature change. Further, in the present embodiment, the optical characteristics of the optical member such as the collimator lens provided in the light source unit 202 may change due to the temperature change so as to cause erroneous detection of raindrops. Therefore, in the present embodiment, as a fluctuation component that causes erroneous detection of raindrops, a temperature fluctuation component e(x, t) of the light source light caused by a fluctuation in the light amount of the light source light emitted from the light source unit 202 due to a temperature change. ) Is included.

  Since the temperature fluctuation component e(x,t) of the light source light also affects the raindrop fluctuation component, the raindrop fluctuation component when the temperature fluctuation component e(x,t) of the light source light is included. Is represented by r(x,t)×e(x,t).

  As described above, in the present embodiment, the fluctuation component included in the total luminance value y(x,t) in each raindrop detection section x used in the raindrop detection process is the raindrop fluctuation component r(x,t)×e(x, t), the ambient light fluctuation component d(x, t), and the temperature fluctuation component e(x, t) of the light source light.

  Here, the disturbance light fluctuation component d(x, t) can be obtained as follows, for example. That is, in the present embodiment, as shown in FIG. 20, the light source unit 202 is alternately turned on and off for imaging in each imaging frame. Specifically, in odd-numbered image pickup frames frame 1, 3, 5, 7, 9, and 11, image pickup image data (image data when light is turned off) imaged in a state where the light source unit 202 is turned off is imaged, and even-numbered image pickup frames are taken. In frames 2, 4, 6, 8, 10, and 12, the imaged image data (image data at the time of lighting) imaged while the light source unit 202 is turned on is imaged.

  At this time, the off-time image data of the raindrop detection image area obtained by the image frames frame 1, 3, 5, 7, 9, and 11 when the light is off is the disturbance light d(x, t) other than the light source light from the light source unit 202. ) Is the image data captured. Therefore, in the present embodiment, it is included in the total luminance value y(x, t) in each raindrop detection section x of the raindrop detection image area obtained from the imaged frame frames 2, 4, 6, 8, 10, and 12 at the time of lighting. As the disturbance light fluctuation component d(x, t), the sum of brightness values d(x, t) in each raindrop detection section x in the immediately preceding unlit imaging frames frame 1, 3, 5, 7, 9, and 11, respectively, is displayed. Will be used.

  On the other hand, regarding the temperature variation component e(x, t) of the light source light, for example, data indicating the relationship between the light amount of the light source light emitted from the light source unit 202 and the temperature is acquired in advance by an experiment or the like, and the light source unit It is possible to grasp the temperature fluctuation component e(x, t) of the light source light emitted from the light source unit 202 from the temperature detection result detected by providing the temperature sensor 202 and the data. However, in this method, it is necessary to previously obtain highly reliable data regarding the relationship between the light amount of the light source light emitted from the light source unit 202 and the temperature by experiments or the like, but such high reliability is required. It is difficult to obtain some data. Further, since it is necessary to provide a temperature sensor, space saving and cost increase are brought about.

  Further, a light amount sensor for detecting the light amount of the light source light emitted from the light source unit 202 is provided, and the light source is adjusted so that the light amount of the light source light emitted from the light source unit 202 becomes constant based on the detection result of the light amount sensor. By performing light source light control in which the drive current supplied is feedback-controlled, the light source emitted from the light source unit 202 even if the optical characteristics of the optical members in the light source unit 202 change or the light emission amount of the light source changes due to temperature change. The amount of light is constant. In this case, it is not necessary to consider the temperature fluctuation component e(x, t) of the light source light. However, such light source light control leads to space saving and cost increase because it is necessary to provide a light amount sensor, and a light source or control system that enables highly accurate control of the light amount of the light source light is provided. It becomes necessary and cost inevitably increases.

Therefore, in the present embodiment, the temperature fluctuation component e(x, t) of the light source light is grasped as follows without performing such light source light control.
The change in the optical characteristics of the optical member and the change in the light emission amount of the light source due to the temperature change that causes the temperature fluctuation component e(x, t) of the light source light cause the change in the raindrop amount that causes the raindrop fluctuation component r(x, t), It changes slowly compared to the change in the amount of ambient light that causes the ambient light fluctuation component d(x, t). That is, the temperature fluctuation component e(x,t) is a low frequency fluctuation component that changes at a lower frequency than the raindrop fluctuation component r(x,t) and the ambient light fluctuation component d(x,t). Therefore, in the present embodiment, a low-frequency fluctuation component of a predetermined frequency or less is extracted from the total luminance value y(x, t) obtained from the imaged frame frames 2, 4, 6, 8, 10, 12 at the time of lighting, Is the temperature fluctuation component e(x, t).

  The low-frequency fluctuation component extracted in this way is not only the fluctuation component caused by the change in the optical characteristics of the optical member due to the temperature change or the change in the light emission amount of the light source, but also in each raindrop detection category x at a low frequency. An unknown fluctuation component (such as a fluctuation component due to deterioration over time of an optical member or a light source) that causes fluctuations in the total luminance value y(x, t) can also be included. Therefore, according to the present embodiment, it is possible to suppress erroneous detection of the amount of raindrops due to such an unknown fluctuation component.

FIG. 23 is a block diagram showing the flow of raindrop detection processing in this embodiment.
FIG. 24 is a flowchart showing the flow of raindrop detection processing in this embodiment.
The light source control unit 102B of the image analysis unit 102 controls the light emission timing of the light source unit 202 while interlocking with the acquisition of the image signal from the signal processing unit 208. First, the light-off imaging frame frame1 and the light-on imaging frame frame2 are set. Is imaged (S11). In the present embodiment, since the drive current value supplied to the light source of the light source unit 202 is constant, when the optical characteristic change of the optical member in the light source unit 202 or the light emission amount of the light source changes due to the temperature change, the light source unit changes. The light amount of the light source light emitted from 202 fluctuates, and a temperature fluctuation component e(x,t) occurs in the total luminance value y(x,t) obtained from the imaged frame frame2 at the time of lighting.

  Next, the image analysis unit 102 calculates the sum d(x,t) of the brightness values in each raindrop detection section x of the raindrop detection image area in the imaging frame frame1 when the light is off (S12). Then, it is determined whether the calculated total brightness value d(x, t) exceeds a predetermined disturbance determination threshold value (S13). It should be noted that here, whether or not strong ambient light has entered is determined using the sum of pixel values, but the information is not limited to this as long as it is information that can be determined whether or not strong ambient light is incoming.

  At this time, when it exceeds the disturbance determination threshold value (Yes in S13), it is determined that the appropriate raindrop detection process cannot be performed, the raindrop detection process is terminated, and the process proceeds to the next raindrop detection process. On the other hand, when the disturbance determination threshold is not exceeded (No in S13), the image analysis unit 102 determines the total luminance value y(x in each raindrop detection section x of the raindrop detection image area in the imaged frame frame2 when the light is turned on. , T) is calculated (S14). Then, in the low frequency fluctuation component detection unit 102D, a low frequency fluctuation component is extracted from the calculated total luminance value y(x, t) in each raindrop detection section x, and this is used as a temperature fluctuation component e(x, t). It is detected (S15).

  Specifically, for each raindrop detection section x in the raindrop detection image area, the total luminance value y(x, t), y( of the raindrop detection section x calculated for the imaging frame imaged within the most recent predetermined period. low-frequency filter processing using a low-pass filter LPF that extracts low-frequency components below a predetermined frequency is performed on a data group yn consisting of x, t-1), y(x, t-2),.... Then, the temperature fluctuation component e(x, t) which is a low frequency fluctuation component is extracted. In this low frequency filter processing, the temperature fluctuation component e(x, t-1) calculated in the previous low frequency filter processing is held in the recording means without holding the data group yn in the storage means, and this temperature The current temperature fluctuation component e(x, t) can be extracted using the fluctuation component e(x, t−1) and the current total brightness value y(x, t).

  However, the temperature fluctuation component e(x, t) thus extracted is the initial temperature at the initial brightness value y(x, 1) which is the total brightness value of the first imaged frame imaged in the predetermined period. It is a relative temperature fluctuation component with respect to the fluctuation component e(x, 1). Therefore, unless the initial temperature fluctuation component e(x, 1) is grasped and the relative temperature fluctuation component e(x, t) extracted this time is converted into the absolute temperature fluctuation component at this time, The correction for excluding the temperature fluctuation component included in the total luminance value y(x, t) cannot be performed.

Therefore, the relative temperature fluctuation component e(x, t) of each raindrop detection section x extracted as described above is an absolute temperature fluctuation component e′ using the initial temperature fluctuation component e(x, 1). It is converted into (x, t), and this is calculated as a correction value y _trend for performing correction for excluding the temperature fluctuation component included in the total luminance value y(x, t) of each raindrop detection section x (S16). .. The initial time point in this embodiment is, for example, the initial brightness value y(x, which is the first total brightness value after the image analysis unit 102 is started and no raindrop is attached and the ambient light fluctuation component d is equal to or less than the threshold value. , 1) is acquired. It should be noted that this threshold value is set to a value lower than the above-described disturbance determination threshold value, and is set so that it can be handled as including no disturbance light fluctuation component d.

  If the raindrops have already adhered when the image analysis unit 102 is started, the initial luminance value y(x, 1) cannot be acquired as it is, so the timing after driving the wiper 107 to remove the raindrops. The initial luminance value y(x, 1) is acquired with.

  Here, after starting the image analysis unit 102, the first initial luminance value y(x, 1) in which raindrops are not attached and the ambient light fluctuation component d is less than or equal to the threshold value is normally held in the storage means. The reference brightness value y(x, 0) does not match. The difference between the initial luminance value y(x,1) and the reference luminance value y(x,0) is that the initial luminance value y(x,1) does not include the raindrop fluctuation component r(x,1), and Since it does not include the ambient light fluctuation component d(x,1), it can be considered to correspond to the temperature fluctuation component e(x,1) at the initial point. Therefore, the initial temperature fluctuation component e(x, 1) can be specified from the difference between the initial brightness value y(x, 1) and the reference brightness value y(x, 0).

By thus obtaining the initial temperature variation component e(x, 1), the relative temperature variation component e(x, t) extracted in the subsequent raindrop detection process and the initial temperature variation component e(x x, 1) is the absolute temperature fluctuation component e′(x, t)=y _trend in the total luminance value y(x, t) in the subsequent raindrop detection process and the initial luminance value y(x, t). It corresponds to the ratio with 1). Therefore, the absolute temperature variation component e′(x,t)=y _trend at the time of each raindrop detection process is equal to the initial temperature variation component e(x,t) extracted in the raindrop detection process. It can be calculated from the brightness value y(x, 1) and the initial temperature fluctuation component e(x, 1).

As described above, the correction value for excluding the variation components other than the raindrop variation component r(x,t) is obtained. That is, the correction value d(x, t) due to the ambient light fluctuation component was obtained in the processing step S12, and the correction value y _trend according to the temperature fluctuation component was obtained in the processing step S16. Therefore, in the image processing unit 102A, in the detection processing unit 102A, the total brightness value y(x, t) in each raindrop detection section x of the raindrop detection image area in the imaged frame frame2 at the time of lighting calculated in step S14 is calculated. , A correction for excluding the ambient light fluctuation component and the temperature fluctuation component is performed based on these correction values d(x, t) and y _trend , and the raindrop amount calculation for calculating the raindrop fluctuation component r(x, t) is performed (S17). ). The raindrop amount data obtained by the raindrop amount calculation is output from the image analysis unit 102 to the wiper control unit 106 (S18). The wiper control unit 106, for example, determines that the calculation result of the amount of raindrops is under a predetermined condition (for example, the condition that the count value of the amount of raindrops is 10 or more for each of the 20 time difference images that are continuously created). When the above condition is satisfied, drive control of the wiper 107 and discharge control of the washer liquid are performed.

  Regarding the data group yn that is a sample for extracting the temperature fluctuation component e(x,t), the initial luminance value y is set so that the temperature fluctuation component e(x,t) can be extracted even with a relatively small number of data. Similar to (x, 1), it is possible to use only the total luminance value of the raindrop detection section x calculated for an imaging frame that satisfies the condition that no raindrop is attached and the ambient light fluctuation component d is equal to or less than the threshold value. At this time, in a situation where it is raining, the data group yn cannot be stored as it is, but in synchronization with the operation of the wiper 107, the total luminance value y(x , T), it is possible to accumulate the data group yn even when it is raining. In addition, if a sufficient number of data is obtained as the data group yn, it is not necessary to limit to such conditions.

  Further, in the raindrop detection process of the present embodiment, the correction for excluding the disturbance light component d(x, t) is also performed, but the correction for excluding the disturbance light component d(x, t) is not necessarily performed. No need. In particular, in the present embodiment, the disturbance light component d is sufficiently suppressed in the raindrop detection image area by the infrared light transmission filter area 212 of the optical filter 205, the polarization filter layer 225, and the like. The correction for excluding (x,t) does not have to be performed.

[Modification]
Next, a modified example of the present embodiment will be described.
In the embodiment described above, the reflection deflection prism 230 is used, and the illumination light is transmitted through the outer wall surface of the windshield at the location where the raindrop is attached, and the illumination light is reflected at the outer wall surface of the windshield at the location where the raindrop is not attached to the image sensor 206. It was configured to receive light. On the other hand, in this modification, the illumination light is transmitted through the windshield outer wall surface at the raindrop non-adhered portion, and the illumination light transmitted through the windshield outer wall surface is reflected by the raindrop at the raindrop adhered portion and reflected by the image sensor 206. It is configured to receive light. Specifically, the configuration is as shown in FIG.

  Note that in the example of FIG. 25, the illumination light emitted from the light source unit 202 is directly incident on the inner wall surface of the windshield 105, but it may be configured via an optical path changing member such as a mirror. In this case, the light source unit 202 as the light source device can be mounted on the sensor substrate 207, as in the above-described embodiment.

  Light ray A in FIG. 25 is a light ray emitted from the light source unit 202 and passing through the windshield 105. When the raindrops 203 do not adhere to the outer wall surface of the windshield 105, the light emitted from the light source unit 202 toward the windshield 105 passes through the windshield 105 like the light ray A and directly passes through the windshield 105. It leaks to the outside.

  A light beam B in FIG. 25 is a light beam that is emitted from the light source unit 202 and specularly reflected by the inner wall surface of the windshield 105 and is incident on the imaging unit 200. A part of the light traveling from the light source unit 202 to the windshield 105 is specularly reflected by the inner wall surface of the windshield 105. The polarization component of this specularly reflected light (light ray B) is generally dominated by the S polarization component (horizontal polarization component S) that oscillates in a direction orthogonal to the plane of incidence (direction perpendicular to the paper surface of FIG. 25). Is known to be The specularly reflected light (light ray B) emitted from the light source unit 202 and specularly reflected on the inner wall surface of the windshield 105 does not change depending on the presence or absence of the raindrops 203 adhering to the outer wall surface of the windshield 105. In addition, it becomes ambient light that reduces the detection accuracy of raindrop detection. In the present modification, the light ray B (horizontal polarization component S) is cut by the polarization filter layer 225 of the raindrop detection filter unit 220B, so that it is possible to suppress the raindrop detection accuracy from being lowered by the light ray B.

  A light ray C in FIG. 25 is a light ray in which light emitted from the light source unit 202 passes through the inner wall surface of the windshield 105, is then reflected by raindrops adhering to the outer wall surface of the windshield 105, and is incident on the imaging unit 200. is there. Although a part of the light traveling from the light source unit 202 to the windshield 105 is transmitted through the inner wall surface of the windshield 105, the transmitted light is more in the vertical polarization component P than in the horizontal polarization component S. When the raindrop 203 is attached to the outer wall surface of the windshield 105, the light transmitted through the inner wall surface of the windshield 105 does not leak to the outside like the light ray A but is multiply reflected inside the raindrop. Then, the light passes through the windshield 105 again toward the imaging unit 200 side and enters the imaging unit 200. At this time, since the infrared light transmitting filter region 212 of the pre-stage filter 210 in the optical filter 205 of the image capturing unit 200 is configured to transmit the emission wavelength (infrared light) of the light source unit 202, the light ray C is red. The light passes through the outside light transmission filter region 212. Further, since the polarization filter layer 225 of the raindrop detection filter portion 220B of the succeeding post-stage filter 220 is formed in the longitudinal direction of the metal wire of the wire grid structure so as to transmit the vertically polarized component P, the light ray C reflects the polarization filter. Layer 225 is also transparent. Therefore, the light ray C reaches the image sensor 206, and raindrops are detected by the amount of received light.

  A light ray D in FIG. 25 is a light ray that has passed through the windshield 105 from the outside of the windshield 105 and is incident on the raindrop detection filter portion 220B of the imaging unit 200. This light ray D can also be disturbance light at the time of raindrop detection, but in this modification, most of the light ray D is cut by the infrared light transmitting filter area 212 of the pre-stage filter 210 in the optical filter 205. Therefore, it is possible to suppress the raindrop detection accuracy from being lowered by the light ray D.

  The light ray E in FIG. 25 is a light ray that has passed through the windshield 105 from the outside of the windshield 105 and is incident on the vehicle detection filter portion 220A of the imaging unit 200. The infrared ray band of the light ray E is cut by the infrared ray cut filter area 211 of the pre-stage filter 210 in the optical filter 205, and only light in the visible area is imaged. This picked-up image is used to detect headlamps of oncoming vehicles, tail lamps of preceding vehicles, white lines, and the like.

What has been described above is an example, and the following unique effects can be obtained.
(Aspect A)
Using imaged image data obtained by imaging an illumination area illuminated by light source light from a light source device such as the light source unit 202 with an image sensor 206 or the like, and within the illumination area based on the amount of light received by the imager. In the detection device such as the image analysis unit 102 including the detection processing execution unit such as the detection processing unit 102A that executes the detection processing to detect the detection target such as the raindrop, a plurality of images captured within the predetermined period by the imaging unit. A low-frequency fluctuation component detection unit such as a low-frequency fluctuation component detection unit 102D that detects a low-frequency fluctuation component of a predetermined frequency or less among the fluctuation components of the amount of light received by the imaging unit, The detection processing execution means executes the detection processing by using the low frequency fluctuation component detected by the low frequency fluctuation component detection means.
Of the change in the amount of received light of the light source light (time-varying component) by the image pickup unit used for the detection processing of the detection target, the light source light by the image pickup unit caused by temperature change or deterioration over time of the light source, optical member, etc. The change in the amount of received light changes at a lower frequency (longer time period) than the change in the amount of received light caused by the presence or absence of the detection target and the change in the amount of received light caused by ambient light. Therefore, in the present aspect, the low frequency fluctuation component in the received light amount of the light source light by the image pickup unit detected by the low frequency fluctuation component detection unit is caused by the temperature change of the light source, the optical member, or the like and deterioration over time. It can be obtained as a change in the amount of received light of the light source generated by the image pickup unit. In this aspect, since the detection process is executed using the low-frequency fluctuation component detected in this manner, the change in the amount of light received by the imaging unit (fluctuation component) causes a change in temperature of the light source, the optical member, or the like and deterioration over time. It is possible to reduce the influence of the low frequency fluctuation component caused by the above. Therefore, according to this aspect, it is possible to suppress the erroneous detection of the detection target due to the temperature change of the light source, the optical member, or the like, or the deterioration over time.

(Aspect B)
In the aspect A, the detection processing execution means corrects the light reception amount of the image pickup means used in the detection processing based on the low frequency fluctuation component detected by the low frequency fluctuation component detection means, and obtains the corrected light reception amount. It is characterized in that the detection target is detected based on the above.
According to this, it is possible to reduce the influence of the low-frequency fluctuation component caused by the temperature change of the light source, the optical member, or the like, or the deterioration over time, from the change (variation component) in the amount of light received by the imaging unit.

(Aspect C)
In the aspect A or B, the low-frequency fluctuation component detection unit is a low-frequency component equal to or lower than the predetermined frequency from a temporal fluctuation component of a data group yn of each received light amount of the imaging unit obtained from each of the plurality of captured image data. Is performed to detect the low-frequency fluctuation component.
According to this, it is possible to easily detect a change in the amount of light received by the image pickup unit that occurs due to a change in temperature of the light source, the optical member, or the like, or deterioration over time.

(Aspect D)
In any one of the aspects A to C, the predetermined frequency is a temporal change (temperature change) of an optical characteristic of an optical member existing on an optical path of a light source light from a light source of the light source device to a light receiving element of the imaging unit. Or a deterioration with time), the light receiving amount of the image pickup unit fluctuates or exceeds the frequency of a fluctuation component.
According to this, it is possible to detect a change in the amount of light received by the image pickup unit that occurs due to a change in temperature of the optical member or deterioration over time.

(Aspect E)
In any one of the aspects A to D, the predetermined frequency is equal to or higher than a frequency of a fluctuation component in which the amount of light received by the imaging unit fluctuates due to a change in the amount of light emission due to a temperature change of a light source of the light source device. To do.
According to this, it is possible to detect a change in the amount of light received by the image pickup unit that occurs due to a change in the temperature of the light source.

(Aspect F)
In any one of the above aspects A to E, the plurality of captured image data used by the low frequency variation component detection unit to detect the low frequency variation component does not have a detection target object in the illumination region. The feature is that it is sometimes captured.
According to this, even if the number of pieces of captured image data used for the low-frequency fluctuation component detecting means to detect the low-frequency fluctuation component is small, the low-frequency fluctuation component can be detected.

(Aspect G)
In the aspect F, the low-frequency fluctuation component detection means includes a wiper 107 or the like that removes the detection target object in the illumination area as the plurality of captured image data when the detection object exists in the illumination area. It is characterized in that picked-up image data obtained by picking up an image of the illumination area after the detection target is removed by operating the removing means is used.
According to this, it is possible to obtain the captured image data when the detection target does not exist even under the situation where the detection target adheres to the illumination area.

(Aspect H)
In any one of the aspects A to G, the detection processing detects the amount of the detection target object in the illumination area based on the amount of light received by the imaging unit.
When detecting the amount of the detection target object in the illumination area based on the amount of light received by the imaging unit, higher accuracy is required for the error in the amount of light received by the imaging unit than when detecting only the presence or absence of the detection target. Therefore, a change in the amount of light received by the image pickup unit caused by a change in temperature of the light source, the optical member, or the like or deterioration over time easily causes erroneous detection. According to this aspect, it is possible to suppress the erroneous detection of the amount of the detection target due to the temperature change of the light source, the optical member, or the like, or the deterioration over time.

(Aspect I)
Using imaged image data obtained by imaging an illumination area illuminated by light source light from a light source device such as the light source unit 202 with an image sensor 206 or the like, and within the illumination area based on the amount of light received by the imager. A detection device such as an image analysis unit 102 for detecting a detection target such as a raindrop, and a predetermined device such as a wiper 107 mounted on a moving body such as the own vehicle 100 based on a detection result of the detection device. In a mobile device control system such as an in-vehicle device control system including a mobile device control unit such as a wiper control unit 106, the detection device according to any one of the modes A to F is used as the detection device. Is characterized by.
According to this, it is possible to suppress the erroneous detection of the detection target due to the temperature change of the light source or the optical member or the deterioration over time, so that it is possible to highly accurately control the mobile device-equipped device based on the detection result. Becomes

(Aspect J)
Using imaged image data obtained by imaging an illumination area illuminated by light source light from a light source device such as the light source unit 202 with an image sensor 206 or the like, and within the illumination area based on the amount of light received by the imager. Based on a plurality of picked-up image data picked up by the image pick-up means within a predetermined period, a pick-up means for executing a detection process for detecting a detection object such as raindrop A computer of a detection device such as an image analysis unit 102 provided with low frequency fluctuation component detection means such as a low frequency fluctuation component detection unit 102D that detects a low frequency fluctuation component of a predetermined frequency or less among the fluctuation components of the received light amount of A detection program for functioning as the detection processing execution means and the low frequency fluctuation component detection means, wherein the detection processing execution means uses the low frequency fluctuation component detected by the low frequency fluctuation component detection means, It is characterized in that the detection process is executed.
According to this, it is possible to suppress the erroneous detection of the detection target due to the temperature change of the light source, the optical member, and the like, or the deterioration over time.
It should be noted that this program can be distributed or obtained while being recorded in a recording medium such as a CD-ROM. Also, by distributing this program and distributing or receiving the signal transmitted by a predetermined transmission device via a transmission medium such as a public telephone line, a leased line, or another communication network, distribution or acquisition Is possible. At the time of this distribution, at least a part of the computer program may be transmitted in the transmission medium. That is, it is not necessary that all the data making up the computer program exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave containing the computer program. Further, the transmission method of transmitting a computer program from a predetermined transmission device includes a case of continuously transmitting the data making up the program and a case of transmitting the data intermittently.

100 Own vehicle 101 Imaging unit 102 Image analysis unit 102A Detection processing unit 102B Light source control unit 102C Exposure amount control unit 102D Low frequency fluctuation component detection unit 105 Windshield 106 Wiper control unit 107 Wiper 200 Imaging unit 202 Light source unit 203 Raindrop 204 Imaging lens 205 Optical filter 206 Image sensor 207 Sensor substrate 208 Signal processing unit 230 Reflective deflection prism

JP, 2014-32174, A

Claims (8)

Using the imaged image data at the time of lighting and at the time of light-off obtained by imaging the illumination area illuminated by the light source light from the light source device with the light source light being repeatedly turned on and off, the received light amount of the image pickup means is used. In a detection device provided with a detection process executing means for executing a detection process of detecting a detection target in the illumination area based on:
The illumination area is an area on a plate-shaped transparent member, the detection target is an adherent that adheres to one surface of the transparent member, and the light source device and the image capturing unit include the transparent member. It is arranged on the side of the surface opposite to the one surface sandwiched between,
Based on the plurality of imaged image data captured by the image capturing unit during the lighting of the light source light within a predetermined period, a low frequency component of the variable component of the light receiving amount of the image capturing unit, which has a frequency equal to or lower than a predetermined frequency. Having a low-frequency fluctuation component detection means for detecting fluctuation components,
The detection device, wherein the detection processing execution means executes the detection processing by using the low frequency fluctuation component detected by the low frequency fluctuation component detection means.
However, the predetermined frequency is a variation component in which the amount of light received by the image pickup device fluctuates due to the temporal change of the optical characteristics of the optical member existing on the optical path of the light source light from the light source of the light source device to the light receiving element of the image pickup device. It is equal to or higher than the frequency, or equal to or higher than the frequency of the fluctuation component in which the amount of light received by the imaging unit fluctuates due to the change in the amount of light emission due to the temperature change of the light source of the light source device. The detection device according to claim 1,
The detection processing execution means corrects the light reception amount of the image pickup means used in the detection processing based on the low frequency fluctuation component detected by the low frequency fluctuation component detection means, and the detection target object based on the corrected light reception amount. A detection device characterized by detecting. The detection device according to claim 1 or 2,
The low-frequency fluctuation component detection means executes low-frequency filter processing for extracting a frequency component below the predetermined frequency from a temporal fluctuation component of a data group of each received light amount of the imaging means obtained from each of the plurality of captured image data. And detecting the low frequency fluctuation component. In the detection apparatus according to any one of claims 1乃Itaru 3,
The plurality of picked-up image data used by the low-frequency fluctuation component detection unit to detect the low-frequency fluctuation component is picked up when a detection target object does not exist in the illumination area. Detection device. The detection device according to claim 4 ,
When the detection target object exists in the illumination region, the low-frequency fluctuation component detection unit operates the removal unit that removes the detection target object in the illumination region as the plurality of captured image data, thereby performing the detection. A detection device characterized by using picked-up image data obtained by picking up an image of an illumination area after removing an object by the image pickup means. In the detection apparatus according to any one of claims 1乃Itaru 5,
The said detection process detects the quantity of the detection target object in the said illumination area based on the light reception amount of the said imaging means, The detection apparatus characterized by the above-mentioned. A detection device for detecting an object to be detected in the illumination area based on the amount of light received by the imaging means, using imaged image data obtained by imaging the illumination area illuminated by the light source light from the light source device. ,
Based on the detection result of the detection device, in a mobile device control system including a mobile device control means for controlling a predetermined device mounted on a mobile,
Examples detecting device, the mobile device control system, which comprises using a detection device according to any one of claims 1乃optimum 6. Using the imaged image data at the time of lighting and at the time of light-off obtained by imaging the illumination area illuminated by the light source light from the light source device with the light source light being repeatedly turned on and off, the received light amount of the image pickup means is used. Based on a plurality of the imaged image data imaged by the imager when the light source light is turned on within a predetermined period by the imager to detect a target object in the illumination area. A low-frequency fluctuation component detecting means for detecting a low-frequency fluctuation component whose frequency is a fluctuation component whose frequency is equal to or lower than a predetermined frequency among the fluctuation components of the amount of received light of the image pickup means ,
The illumination area is an area on a plate-shaped transparent member, the detection target is an adherent that adheres to one surface of the transparent member, and the light source device and the image capturing unit include the transparent member. sandwiched therebetween and the one surface of the computer detection device is arranged on the side opposite to a detection program for functioning as the detecting process execution means and the low frequency fluctuation component detecting means,
A detection program, wherein the detection processing execution means executes the detection processing using the low frequency fluctuation component detected by the low frequency fluctuation component detection means.
However, the predetermined frequency is a variation component in which the amount of light received by the image pickup device fluctuates due to the temporal change of the optical characteristics of the optical member existing on the optical path of the light source light from the light source of the light source device to the light receiving element of the image pickup device. It is equal to or higher than the frequency, or equal to or higher than the frequency of the fluctuation component in which the amount of light received by the imaging unit fluctuates due to the change in the amount of light emission due to the temperature change of the light source of the light source device.

Images