JP2016200528A - Object detection device, moving body-mounted equipment control system, and object detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a rolling shutter system image sensor which suppresses trouble such as erroneous detection of a detection object when changing exposure time, while achieving power saving of a light source device.SOLUTION: There is provided an object detection device 102 which images an illumination area illuminated by light source light from a light source device 202 using at least a part of sensor parts of a rolling shutter system image sensor 206 and detects a detection object within the illumination area on the basis of receiving light quantity of the light source light received by a part of the sensor parts. The object detection device includes: light source control means 102B for periodically turning on and off the light source light emitted from the light source device during an entire exposure period of the at least a part of the sensor parts; and exposure time control means 102C for changing the exposure time according to prescribed exposure conditions. The light source control means changes an on/off period of the light source light in accordance with the exposure time changed by the exposure time control means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object detection device, a mobile device control system, and an object detection program.

  Conventionally, an imaging area image for sensing for recognizing an object existing in an imaging area in front of the host vehicle and an object (detection target) such as raindrops adhering to the windshield of the host vehicle are detected. There is known an imaging apparatus that captures an adhered image with a single image sensor.

  For example, in Patent Document 1, reflected light of light source light emitted toward a windshield from a light source device is received by a part of the image sensor, and light from an imaging region in front of the host vehicle is received by another image sensor. A raindrop detection device including an imaging device that receives light at a portion is disclosed. This raindrop detection device employs a rolling shutter type image sensor, and uses the image data of the deposit image area captured by the part of the image sensor to determine the amount of light source light received by the part. Based on this, it detects raindrops on the windshield.

  However, in the raindrop detection apparatus disclosed in Patent Document 1, the exposure period of the first light receiving element line corresponding to the deposit image region starts until the exposure period of the last light receiving element line ends ( During the entire exposure period), the light source device continues to emit light from the light source device. In this case, the light irradiation time for which the light source device emits the light source light per imaging frame is long, which is not preferable from the viewpoint of power saving of the light source device.

  Therefore, the present inventor is developing an object detection device that periodically irradiates light source light from a light source device in a light irradiation period shorter than the exposure period of each light receiving element line. Specifically, for example, 100 light receiving element lines corresponding to the adhered image area are divided into four light receiving element lines, and one light irradiation period is assigned to each of the light receiving element lines. The light source light is periodically irradiated from the light source device so that the light irradiation periods overlap at the time when the exposure periods overlap between the element lines. In this case, the light irradiation time for which the light source device irradiates the light source light per imaging frame can be shortened, and power saving of the light source device can be achieved.

  However, if automatic exposure control is performed to change the exposure time when imaging the deposit image area, for example, when the exposure time becomes longer, the exposure period of some light receiving element lines belonging to one line segment is different from the other. A situation may occur where the light irradiation period corresponding to the line segment overlaps. In this case, the light amount of the light source light during each exposure period is not uniform between the light receiving element lines, causing problems such as erroneous detection of attached matter. Similarly, for example, when the exposure time is shortened, for some light receiving element lines belonging to one line section, the time during which the exposure period overlaps with the light irradiation period as compared to other light receiving element lines belonging to the line section. Shortening can happen. In this case as well, the light amount of the light source light during each exposure period is not uniform between the light receiving element lines, causing problems such as erroneous detection of adhered matter.

  In order to solve the above-described problems, the present invention captures an illumination area illuminated by light source light from a light source device with at least a part of a sensor in a rolling shutter type image sensor in which light receiving elements are two-dimensionally arranged. In the object detection apparatus for detecting the detection target in the illumination area based on the received light amount of the light source light received by the at least a part of the sensor using the captured image data obtained thereby. Light source control means for periodically turning on and off the light source light emitted from the light source device during the total exposure period in the sensor portion of the part, and exposure time control means for changing the exposure time according to a predetermined exposure condition, The light source control means changes the on / off cycle of the light source light according to the exposure time changed by the exposure time control means. That.

  Advantageous Effects of Invention According to the present invention, in a rolling shutter type image sensor, it is possible to suppress problems such as erroneous detection of an object to be detected when the exposure time changes by the exposure time control means while saving power in the light source device. The effect is played.

It is a mimetic diagram showing a schematic structure of an in-vehicle device control system in an embodiment. It is a schematic diagram which shows schematic structure of the deposit | attachment detection apparatus provided with the imaging unit in the same vehicle equipment control system. It is explanatory drawing for demonstrating the optical system of the imaging unit. It is the perspective view which showed typically schematic structure of the imaging unit. (A) is a side view which shows the imaging unit of the state attached to the vehicle whose inclination angle of a windshield with respect to a horizontal surface is 22 degrees. (B) is explanatory drawing which shows the optical system of the imaging unit in case the raindrop has not adhered in the state of the figure (a). (C) is explanatory drawing which shows the optical system of an imaging unit in case the raindrop has adhered in the state of the figure (a). It is a perspective view which shows the reflective deflection | deviation prism of the imaging unit. It is a graph which shows the filter characteristic of the cut filter applicable to the picked-up image data for raindrop detection. It is a graph which shows the filter characteristic of the band pass filter applicable to the picked-up image data for raindrop detection. It is a front view of the front | former stage filter provided in the optical filter of the imaging part. It is explanatory drawing which shows the image example of the captured image data of the imaging part. It is a model enlarged view when the optical filter and image sensor of the imaging part are seen from a direction orthogonal to the light transmission direction. It is explanatory drawing which shows the area | region division pattern of the polarizing filter layer and spectral filter layer of the same optical filter. It is sectional drawing which shows typically the layer structure of the optical filter in embodiment. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the filter part for vehicle detection of the same optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the filter part for vehicle detection of the optical filter and image sensor which were cut | disconnected by code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the vehicle detection filter part and image sensor of the optical filter which were cut | disconnected by the code | symbol BB shown in FIG. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the filter part for raindrop detection of the optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the filter part for raindrop detection and the image sensor of the optical filter cut | disconnected by code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the filter part for raindrop detection of the optical filter cut | disconnected by the code | symbol BB shown in FIG. 16, and an image sensor. It is a flowchart which shows the flow of the vehicle detection process in embodiment. It is explanatory drawing of the raindrop detection process in embodiment. It is explanatory drawing which shows the relationship between the conventional imaging frame and raindrop detection. It is explanatory drawing which shows the relationship between the imaging frame and raindrop detection in embodiment. It is explanatory drawing which shows an example of the relationship between the exposure period of each light receiving element line set by automatic exposure control, and the lighting period of a light source part in one image (imaging frame). In the example of FIG. 22, it is explanatory drawing which shows an example of the relationship between the exposure period of each light receiving element line and the lighting period of a light source part when exposure time becomes long, fixing the lighting period of a light source part. It is a flowchart which shows the flow of the lighting cycle control in embodiment. It is explanatory drawing which shows the other example of the relationship between the exposure period of each light receiving element line set by automatic exposure control, and the lighting period of a light source part in one image (imaging frame). It is explanatory drawing which shows the further another example of the relationship between the exposure period of each light receiving element line set by automatic exposure control, and the lighting period of a light source part in one image (imaging frame). It is explanatory drawing about the various light rays in connection with the raindrop detection in a modification.

Hereinafter, an embodiment in which an object detection device according to the present invention is used in an in-vehicle device control system as a mobile device control system that controls a target device mounted on a vehicle such as an automobile that is a mobile body will be described.
The object detection apparatus according to the present invention is used in a control system for various mobile body-mounted devices mounted on other mobile bodies such as vehicles other than automobiles, ships, and airplanes, and is mounted on mobile bodies. It can also be used in factory automation (FA) that is not a device.

FIG. 1 is a schematic diagram illustrating a schematic configuration of an in-vehicle device control system according to the present embodiment.
This in-vehicle device control system uses a picked-up image data obtained by picking up an area around the own vehicle (especially a forward area in the traveling direction) by an image pickup unit mounted on the own vehicle 100 such as an automobile, and distributes the headlamp light. Control, wiper drive control, and other in-vehicle devices are controlled.

  The imaging unit provided in the in-vehicle device control system of the present embodiment is provided in the imaging unit 101, and images an area in the traveling direction of the traveling host vehicle 100 as an imaging region. It is installed near the room mirror of the windshield 105. The captured image data captured by the imaging unit of the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 includes a CPU, a RAM, and the like, analyzes captured image data transmitted from the imaging unit, and determines the position (direction and distance) of another vehicle existing ahead of the host vehicle 100 in the captured image data. Calculation, detection of an adhering substance such as raindrops or foreign matters adhering to the windshield 105, or detection of an object to be detected such as a white line (partition line) on the road surface existing in the imaging region. In the detection of other vehicles, a preceding vehicle traveling in the same traveling direction as the own vehicle 100 is detected by identifying the tail lamp of the other vehicle, and traveling in the opposite direction to the own vehicle 100 by identifying the headlamp of the other vehicle. An oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. For example, the headlamp control unit 103 generates a control signal for controlling the headlamp 104 that is an in-vehicle device of the host vehicle 100 from the position data of the other vehicle calculated by the image analysis unit 102. Specifically, for example, while avoiding that the strong light of the headlamp of the own vehicle 100 is incident on the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. The switching of the high beam and the low beam of the headlamp 104 is controlled, and partial shading control of the headlamp 104 is performed so that the driver's visibility can be secured.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls the wiper 107 to remove deposits such as raindrops and foreign matters attached to the windshield 105 of the host vehicle 100. The wiper control unit 106 receives the attached matter detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated to ensure the visibility of the driver of the host vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle travel control unit 108 warns the driver of the host vehicle 100 when the host vehicle 100 is out of the lane area defined by the white line. Notification is performed, and driving support control such as controlling the steering wheel and brake of the host vehicle is performed. In addition, the vehicle travel control unit 108 notifies the driver of the host vehicle 100 of a warning when the distance from the preceding vehicle is close based on the position data of the other vehicle detected by the image analysis unit 102, Carry out driving support control such as controlling the steering wheel and brakes of the vehicle.

FIG. 2 is an explanatory diagram illustrating a schematic configuration of the attached matter detection apparatus including the imaging unit 200 provided in the imaging unit 101.
The imaging unit 200 mainly includes an imaging lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 in which light receiving elements are two-dimensionally arranged, and an analog electrical signal (image sensor) output from the sensor substrate 207. A signal processing unit 208 that generates and outputs captured image data obtained by converting a received light amount received by each light receiving element on the digital image 206 into a digital electric signal, and an internal memory 209. In the present embodiment, the light source unit 202 as a light source device is mounted on the sensor substrate 207. The light source unit 202 is disposed on the inner wall surface (one surface) side of the windshield 105, and an example is a case where the attached matter (hereinafter, the detection target is mainly raindrops) attached to the outer wall surface (the other surface). This is for detecting.

  In the present embodiment, the imaging unit 101 is arranged so that the optical axis of the imaging lens 204 coincides with the horizontal direction. However, the present invention is not limited to this, and the horizontal direction (X direction in FIG. 2) is used as a reference. It may be an example directed to a specific direction. For example, the imaging lens 204 includes a plurality of lenses and has a focal point set farther than the position of the windshield 105. The focal position of the imaging lens 204 can be set, for example, between infinity or infinity and the windshield 105.

  The image sensor 206 includes a plurality of two-dimensionally arranged light receiving elements that receive light transmitted through a cover glass that protects the sensor surface, and has a function of photoelectrically converting incident light for each light receiving element (imaging pixel). In the drawings and the like to be described later, each pixel of the image sensor 206 is illustrated in a simplified manner. However, the image sensor 206 is actually composed of about several hundred thousand pixels arranged two-dimensionally. The image sensor 206 according to the present embodiment is a CMOS (line shutter method) that captures images by a line exposure method (rolling shutter method) that sequentially exposes each light receiving element line constituting the image sensor and sequentially reads light reception data of each imaging pixel of each light receiving element line. Complementary Metal Oxide Semiconductor).

  Here, when using an image sensor of the simultaneous exposure method (global shutter method), the exposure period for all the pixels of the image sensor is the same, so the exposure period of the raindrop detection image region 214 described later and the vehicle detection image The exposure period of the region 213 matches. Therefore, the light source light emitted from the light source unit 202 during the exposure period of the raindrop detection image area 214 is also emitted during the exposure period of the vehicle detection image area 213, and the light source light is used as the vehicle detection image. There is a possibility that the accuracy of recognizing an object such as another vehicle based on the vehicle detection image region 213 becomes disturbance light in the region 213. On the other hand, when a line exposure type (rolling shutter type) image sensor is used, light reception is started according to the set exposure time with reference to the timing of the line readout signal for each light receiving element line of the image sensor. The time (the start time of the exposure period) is determined. Therefore, since the exposure periods of the respective light receiving element lines are shifted from each other, it is possible to irradiate the light source light from the light source unit 202 during the exposure period of the raindrop detection image area 214 so as to be out of the exposure period of the vehicle detection image area 213. It becomes possible. Therefore, disturbance light in the vehicle detection image area 213 due to the light source light can be reduced, and a reduction in recognition accuracy of an object such as another vehicle can be suppressed.

  The signal processing unit 208 generates captured image data obtained by converting an analog electrical signal (the amount of light incident on each light receiving element of the image sensor 206) that is photoelectrically converted by the image sensor 206 and output from the sensor substrate 207 into a digital electrical signal. Output function. The signal processing unit 208 is electrically connected to the image analysis unit 102. When an electric signal (analog signal) is input from the image sensor 206 via the sensor substrate 207, the signal processing unit 208 determines the brightness (luminance) of each imaging pixel on the image sensor 206 from the input electric signal. A digital signal (captured image data) is generated. Then, the captured image data is output to the subsequent image analysis unit 102 together with the horizontal / vertical synchronization signal of the image.

  Further, the image analysis unit 102 has a function of controlling the imaging operation of the imaging unit 101 and a function of analyzing captured image data transmitted from the imaging unit 101. The image analysis unit 102 detects an object to be imaged by the image sensor 206 (an object such as another vehicle existing in the front area of the host vehicle) from the captured image data transmitted from the imaging unit 101 by the exposure time control unit 102C as an exposure time control unit. ) To calculate the optimum exposure time for each image sensor 206 and set an optimum exposure time for each imaging target of the image sensor 206. Further, the image analysis unit 102 has a function of adjusting the on / off cycle of the light source light emitted from the light source unit 202 by the light source control unit 102B as a light source control unit in conjunction with the exposure time adjustment.

  In addition, the image analysis unit 102 has a function of detecting information related to a road surface condition, a road sign, and the like from captured image data transmitted from the imaging unit 101. Further, the image analysis unit 102 has a function of detecting the state of the windshield 105 (attachment of raindrops, freezing, cloudiness, etc.) by the detection processing unit 102A. Further, the image analysis unit 102 has a function of calculating the position, direction, distance, and the like of another vehicle existing ahead of the host vehicle from the captured image data transmitted from the imaging unit 101.

FIG. 3 is an explanatory diagram for explaining an optical system of the imaging unit 101 in the present embodiment.
The light source unit 202 of the present embodiment irradiates light source light (illumination light) for detecting a deposit (a raindrop, freezing, cloudiness, etc.) that is a detection countermeasure on the windshield 105. The light source unit 202 of the present embodiment is configured to include a plurality of LEDs as the light source. By providing a plurality of light sources in this manner, the detection area for detecting the deposit on the windshield 105 is broadened compared to the case where there is a single light source, and the detection accuracy of the deposit on the windshield 105 is improved. improves.

  In this embodiment, since the LEDs are mounted on the same sensor substrate 207 as the image sensor 206, the number of substrates can be reduced as compared with the case where these are mounted on separate substrates, and low cost can be realized. Further, the LED is arranged in one or more rows along the Y direction in FIG. 3 so that, as will be described later, the windshield is projected below the image area where the image of the vehicle front area is projected. It is possible to make the illumination for capturing an image uniform. In the present embodiment, the light source unit 202 is mounted on the same sensor substrate 207 as the image sensor 206, but may be mounted on a different substrate from the image sensor 206.

  The light source unit 202 is disposed on the sensor substrate 207 so that the optical axis direction of the light emitted from the light source unit 202 and the optical axis direction of the imaging lens 204 have a predetermined angle in advance. Further, the light source unit 202 is arranged on the windshield 105 illuminated by the light emitted from the light source unit 202 so that the illumination range is within the field angle range (viewing angle range) of the imaging lens 204. ing. As the emission wavelength of the light source unit 202, it is preferable to avoid visible light so as not to dazzle the driver or pedestrian of the oncoming vehicle. For example, the wavelength is longer than the visible light, and the light receiving sensitivity of the image sensor 206 extends. A range (for example, an infrared light wavelength range of about 800 to 1000 nm) is used.

  As shown in FIG. 3, the imaging unit 101 of the present embodiment is provided with a reflection deflection prism 230 as an optical member that includes a reflection surface 231 that reflects light from the light source unit 202 and guides it to the windshield 105. Yes. The reflection deflection prism 230 is disposed so that one surface thereof is in close contact with the inner wall surface of the windshield 105 in order to appropriately guide the light from the light source unit 202 into the windshield 105. Specifically, even if the incident angle of light from the light source unit 202 incident on the reflection deflection prism 230 changes within a predetermined range, the light is irradiated from the light source unit 202 and regularly reflected by the reflection surface 231 of the reflection deflection prism 230. Of the light, regular reflection light regularly reflected at a non-attached portion where no raindrop (detection target) on the outer wall surface of the windshield 105 is attached is received by the image sensor 206 so as to maintain the relationship. A reflection deflection prism 230 is attached to the inner wall surface of the windshield 105.

  When the reflection deflecting prism 230 is attached to the inner wall surface of the windshield 105, it is preferable to improve the adhesion by interposing a filler such as a gel made of a translucent material or a sealing material therebetween. Thereby, it is possible to prevent an air layer, bubbles, or the like from intervening between the reflection deflection prism 230 and the windshield 105, and to prevent fogging between them. The refractive index of the filler is preferably an intermediate refractive index between the reflective deflection prism 230 and the windshield 105. This is because the Fresnel reflection loss between the filler and the reflection deflecting prism 230 and between the filler and the windshield 105 can be reduced. Fresnel reflection here refers to reflection that occurs between materials having different refractive indexes.

  As shown in FIG. 3, the reflection deflection prism 230 specularly reflects incident light from the light source unit 202 once on the reflection surface 231, and folds it back toward the inner wall surface of the windshield 105. The folded light is configured to have an incident angle θ (θ ≧ about 42 °) with respect to the outer wall surface of the windshield 105. The appropriate incident angle θ is a critical angle that causes total reflection on the outer wall surface inside the windshield 105 due to a difference in refractive index between air and the outer wall surface of the windshield 105. Therefore, in the present embodiment, when no deposits such as raindrops adhere to the outer wall surface of the windshield 105, the light reflected by the reflection surface of the reflection deflection prism 230 is transmitted through the outer wall surface of the windshield 105. Without being reflected all.

  On the other hand, if deposits such as raindrops (refractive index 1.38) different from air (refractive index 1) adhere on the outer wall surface of the windshield 105, this total reflection condition breaks down, and in places where raindrops adhere Light passes through the outer wall surface of the windshield 105. Therefore, for the non-attached portion where no raindrop is attached to the outer wall surface of the windshield 105, the reflected light is received by the image sensor 206 to become a high-luminance image portion, while the attached portion where the raindrop is attached. Since the amount of reflected light is reduced and the amount of light received by the image sensor 206 is reduced, an image portion with low luminance is obtained. Therefore, on the captured image, the contrast between the raindrop-attached portion and the non-raindrop-attached portion can be obtained.

FIG. 4 is a perspective view schematically showing a schematic configuration of the imaging unit 101 of the present embodiment.
The imaging unit 101 of the present embodiment is mounted with a first module 101A as a first support member that fixedly supports the reflection deflection prism 230 and is fixedly disposed on the inner wall surface of the windshield 105, an image sensor 206, and an LED 211. The sensor board 207, the light guide 215, and the second module 101B as a second support member that fixes and supports the imaging lens 204 are configured.

  These modules 101 </ b> A and 101 </ b> B are rotatably connected by a rotation connecting mechanism 240. The rotation coupling mechanism 240 has a rotation shaft 241 extending in a direction orthogonal to both the inclination direction and the vertical direction of the windshield 105 (the front-rear direction in FIG. 3). The one module 101A and the second module 101B can be rotated relative to each other. Such a rotatable configuration is aimed at the imaging direction of the imaging unit 200 of the second module 101B even when the first module 101A is fixedly arranged with respect to the windshield 105 having different inclination angles. This is because it can be directed in a specific direction (horizontal direction in the present embodiment).

  FIG. 5A is a side view showing the imaging unit 101 in a state of being attached to a vehicle in which the windshield 105 has an inclination angle θg of 22 ° with respect to a horizontal plane. FIG. 5B is an explanatory diagram showing the optical system of the imaging unit 101 when no raindrops are attached in the state of FIG. FIG. 5C is an explanatory diagram showing the optical system of the imaging unit 101 when raindrops are attached in the state of FIG.

  The light L1 from the light source unit 202 is specularly reflected by the reflection surface 231 of the reflection deflection prism 230, and the reflected light L2 is transmitted through the inner wall surface of the windshield 105. When raindrops are not attached to the outer wall surface of the windshield 105, the reflected light L2 is totally reflected on the outer wall surface, and this reflected light L3 is transmitted through the inner wall surface of the windshield 105 toward the imaging lens 204. Goes on. On the other hand, when raindrops 203 are attached to the outer wall surface of the windshield 105, the reflected light L2 specularly reflected by the reflecting surface 231 of the reflecting deflection prism 230 is transmitted through the outer wall surface. In such a system, when the angle θg of the windshield 105 changes, the second module 101B is fixed to the inner wall surface of the windshield 105 while maintaining the attitude of the second module 101B (while the imaging direction is fixed in the horizontal direction). The posture of the module 101B is changed, and the reflection deflection prism 230 is rotated around the Y axis in the drawing together with the windshield 105.

  Here, the arrangement relationship between the reflecting surface 231 of the reflecting deflection prism 230 and the outer wall surface of the windshield 105 in the present embodiment is always the entire relationship on the outer wall surface of the windshield 105 within the rotation adjustment range of the rotary coupling mechanism 240. The reflected light L3 is received by the light receiving area of the image sensor 206 for detecting the windshield state change (hereinafter referred to as “attached substance detecting light receiving area”). Therefore, even if the angle θg of the windshield 105 changes, the total reflected light L3 on the outer wall surface of the windshield 105 is received by the adhering matter detection light receiving area of the image sensor 206, and appropriate raindrop detection can be realized.

  In particular, the arrangement relationship in the present embodiment is such that the corner cube principle is substantially established within the rotation adjustment range of the rotary coupling mechanism 240. Therefore, even if the angle θg of the windshield 105 changes, the angle θ formed between the optical axis direction of the total reflected light L3 on the outer wall surface of the windshield 105 and the horizontal plane is substantially constant. Therefore, it is possible to suppress the fluctuation of the portion through which the optical axis of the total reflected light L3 passes on the outer wall surface of the windshield 105 in the attached light detection region of the image sensor 206, thereby realizing more appropriate raindrop detection. .

  The corner cube principle is established if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is perpendicular to each other. If the cube principle is established, the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not limited to a perpendicular relationship. Even if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not perpendicular to each other, the angle of the other surface (incident surface or emission surface) of the reflection deflection prism 230 is adjusted. Thus, even if the inclination angle θg of the windshield 105 changes, the angle θ of the optical axis of the total reflected light L3 toward the imaging lens can be kept substantially constant.

FIG. 6 is a perspective view showing the reflective deflection prism 230 of the present embodiment.
The reflection deflection prism 230 includes an incident surface 233 on which light from the light source unit 202 is incident, a reflecting surface 231 that reflects the light L1 incident from the incident surface 233, and an adhesion surface 232 that is in close contact with the inner wall surface of the windshield 105. And an emission surface 234 that emits the reflected light L3 reflected by the outer wall surface of the windshield 105 toward the imaging unit 200. In the present embodiment, the incident surface 233 and the exit surface 234 are configured to be parallel to each other, but they may be non-parallel surfaces.

  The material of the reflection deflection prism 230 may be any material that transmits at least light from the light source unit 202, and glass, plastic, or the like can be used. Since the light from the light source unit 202 of the present embodiment is infrared light, a black material that absorbs visible light may be used as the material of the reflective deflection prism 230. By using a material that absorbs visible light, light (such as visible light from outside the vehicle) other than the light from the LED (infrared light) can be prevented from entering the reflective deflection prism 230.

  Further, the reflection deflection prism 230 is formed so that the total reflection condition for totally reflecting the light from the light source unit 202 by the reflection surface 231 is satisfied within the rotation adjustment range of the rotary coupling mechanism 240. In addition, when it is difficult to satisfy the condition of total reflection on the reflection surface 231 within the rotation adjustment range of the rotation coupling mechanism 240, a film such as aluminum is vapor-deposited on the reflection surface 231 of the reflection deflection prism 230. A reflection mirror may be formed.

  In the present embodiment, the reflecting surface 231 is a flat surface, but the reflecting surface may be concave. By using such a concave reflection surface 225, the diffused light beam incident on the reflection surface 225 can be collimated. This makes it possible to suppress a decrease in illuminance on the windshield 105.

  Here, when imaging the infrared wavelength light from the light source unit 202 reflected by the outer wall surface of the windshield 105 with the imaging unit 200, the image sensor 206 of the imaging unit 200 includes the infrared wavelength light from the light source unit 202. A large amount of disturbance light including infrared wavelength light such as sunlight is also received. Therefore, in order to distinguish the infrared wavelength light from the light source unit 202 from such a large amount of disturbance light, the light emission amount of the light source unit 202 needs to be sufficiently larger than the disturbance light. It is often difficult to use the light source unit 202 with a large light emission amount.

  Therefore, in the present embodiment, for example, a cut filter that cuts light having a wavelength shorter than the light emission wavelength of the light source unit 202 as shown in FIG. 7, or a transmittance peak as shown in FIG. The image sensor 206 is configured to receive light from the light source unit 202 through a bandpass filter that substantially matches the emission wavelength of the light source unit 202. Accordingly, light other than the light emission wavelength of the light source unit 202 can be removed and received, so that the amount of light from the light source unit 202 received by the image sensor 206 is relatively large with respect to disturbance light. As a result, light from the light source unit 202 can be distinguished from disturbance light even if the light source unit 202 does not have a large light emission amount.

  However, in the present embodiment, not only the raindrop 203 on the windshield 105 is detected from the captured image data, but also the preceding vehicle and the oncoming vehicle and the white line are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source unit 202 is removed from the entire captured image, the image sensor 206 transmits light in the wavelength band necessary for detection of the preceding vehicle and the oncoming vehicle and detection of the white line. The light cannot be received, and this hinders detection. Therefore, in the present embodiment, the image area of the captured image data, the raindrop detection image area for detecting the raindrop 203 on the windshield 105, the vehicle for detecting the preceding vehicle and the oncoming vehicle and the white line are detected. A filter that removes a wavelength band other than the infrared wavelength light irradiated by the light source unit 202 only on a portion corresponding to the raindrop detection image region is arranged in the optical filter 205.

FIG. 9 is a front view of the front-stage filter 210 provided in the optical filter 205.
FIG. 10 is an explanatory diagram illustrating an example of captured image data.
The optical filter 205 of the present embodiment has a structure in which a front-stage filter 210 and a rear-stage filter 220 are overlapped in the light transmission direction. As shown in FIG. 9, the front-stage filter 210 includes an infrared light cut filter region 211 and a raindrop detection image region 214 that are arranged at locations corresponding to the upper two-third captured image that is the vehicle detection image region 213. The region is divided into an infrared light transmission filter region 212 arranged at a position corresponding to a lower one third of a captured image. For the infrared light transmission filter region 212, the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. 8 is used.

  The headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line image are often present mainly from the center to the upper part of the captured image, and there is usually an image of the nearest road surface in front of the host vehicle at the lower part of the captured image. It is. Therefore, the information necessary for identifying the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line is concentrated on the upper portion of the captured image, and the information on the lower portion of the captured image is not so important in the identification. Therefore, when both oncoming vehicle and preceding vehicle or white line detection and raindrop detection are performed simultaneously from a single captured image data, the lower portion of the captured image is displayed in the raindrop detection image area 214 as shown in FIG. It is preferable that the upper portion of the remaining captured image is the vehicle detection image region 213 and the pre-filter 210 is divided into regions corresponding to this.

  In this embodiment, the raindrop detection image area 214 is provided below the vehicle detection image area 213 in the captured image. However, the raindrop detection image area 214 is provided above the vehicle detection image area 213. Alternatively, the raindrop detection image area 214 may be provided both above and below the vehicle detection image area 213.

  When the imaging direction of the imaging unit 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the host vehicle or the tail lamp of the preceding vehicle becomes disturbance light, which is included in the captured image data, which may cause misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in the present embodiment, the cut filter shown in FIG. 7 and the bandpass filter shown in FIG. Disturbance light such as the tail lamp of the vehicle is removed. Therefore, the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line identification accuracy are improved.

  In this embodiment, due to the characteristics of the imaging lens 204, the scene in the imaging area and the image on the image sensor 206 are upside down. Therefore, when the lower portion of the captured image is used as the raindrop detection image region 214, the upper portion of the upstream filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. .

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light compared to the headlamp of the oncoming vehicle, and disturbance such as street lights. Since there is a lot of light, it is difficult to detect the tail lamp with high accuracy only from mere luminance data. Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. Therefore, in this embodiment, as will be described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the tail lamp color) that matches the color of the tail lamp is disposed in the subsequent filter 220 of the optical filter 205, and the red color. The amount of light received can be detected.

  However, since each light receiving element constituting the image sensor 206 of the present embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 206 receives light including the infrared wavelength band. The captured image becomes reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in the present embodiment, the portion corresponding to the vehicle detection image region 213 in the upstream filter 210 of the optical filter 205 is the infrared light cut filter region 211. Thereby, since the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, the identification accuracy of the tail lamp is improved.

FIG. 11 is a schematic enlarged view when the optical filter 205 and the image sensor 206 are viewed from a direction orthogonal to the light transmission direction.
As described above, the image sensor 206 is an image sensor using a CCD, a CMOS, or the like, and a photodiode 206A is used as a light receiving element thereof. The photodiodes 206A are two-dimensionally arranged for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiode 206A. The image sensor 206 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 207.

  An optical filter 205 is disposed close to the surface of the image sensor 206 on the micro lens 206B side. As shown in FIG. 11, the post filter 220 of the optical filter 205 has a laminated structure in which a polarizing filter layer 222 and a spectral filter layer 223 are sequentially formed on a transparent filter substrate 221. Each of the polarizing filter layer 222 and the spectral filter layer 223 is divided into regions corresponding to one photodiode 206A on the image sensor 206.

  A configuration may be adopted in which there is a gap between the optical filter 205 and the image sensor 206, but the configuration in which the optical filter 205 is in close contact with the image sensor 206 is different from the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205. It becomes easy to make the boundary of each area | region and the boundary between photodiode 206A on the image sensor 206 correspond. The optical filter 205 and the image sensor 206 may be bonded with, for example, a UV adhesive, or UV bonding or thermocompression bonding is performed on the four side areas outside the effective pixel while being supported by the spacer outside the effective pixel range used for imaging. Also good.

FIG. 12 is an explanatory diagram showing a region division pattern of the polarizing filter layer 222 and the spectral filter layer 223 of the optical filter 205 according to the present embodiment.
In the polarizing filter layer 222 and the spectral filter layer 223, two types of regions, a first region and a second region, are arranged corresponding to one photodiode 206A on the image sensor 206, respectively. As a result, the amount of light received by each photodiode 206A on the image sensor 206 is determined as polarization information, spectral information, or the like depending on the types of regions of the polarizing filter layer 222 and the spectral filter layer 223 through which the received light is transmitted. Can be acquired.

  In the present embodiment, the image sensor 206 is described on the assumption of an image sensor for monochrome images, but the image sensor 206 may be configured with a color image sensor. In the case of the color image sensor, the light transmission characteristics of the polarizing filter layer 222 and the spectral filter layer 223 may be adjusted according to the characteristics of the color filter attached to each image pickup pixel of the color image sensor.

Here, an example of the optical filter 205 in the present embodiment will be described.
FIG. 13 is a cross-sectional view schematically showing the layer configuration of the optical filter 205 in the present embodiment.
The post filter 220 of the optical filter 205 in the present embodiment includes a vehicle detection filter unit 220A corresponding to the vehicle detection image region 213 and a raindrop detection filter unit 220B corresponding to the raindrop detection image region 214, and the layers thereof. The configuration is different. Specifically, the vehicle detection filter unit 220 </ b> A includes the spectral filter layer 223, while the raindrop detection filter unit 220 </ b> B does not include the spectral filter layer 223. Further, the configuration of the polarizing filter layers 222 and 225 is different between the vehicle detection filter unit 220A and the raindrop detection filter unit 220B.

FIG. 14 shows the contents of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the vehicle detection filter 220A of the optical filter 205 in this embodiment. It is explanatory drawing which shows.
FIG. 15A is a cross-sectional view schematically showing the vehicle detection filter unit 220A and the image sensor 206 of the optical filter 205 cut along the line AA shown in FIG.
FIG. 15B is a cross-sectional view schematically showing the vehicle detection filter unit 220A and the image sensor 206 of the optical filter 205 cut along the line BB shown in FIG.

  As shown in FIGS. 15A and 15B, the vehicle detection filter unit 220 </ b> A of the optical filter 205 in the present embodiment is formed on a polarizing filter layer 222 on a transparent filter substrate 221, and then on it. The spectral filter layer 223 is formed to have a laminated structure. The polarizing filter layer 222 has a wire grid structure, and the upper surface in the stacking direction (the lower side surface in FIG. 15) is an uneven surface. If an attempt is made to form the spectral filter layer 223 directly on such an uneven surface, the spectral filter layer 223 is formed along the uneven surface, resulting in uneven thickness of the spectral filter layer 223 and obtaining the original spectral performance. There may not be. Therefore, in the optical filter 205 of the present embodiment, the upper surface side in the stacking direction of the polarizing filter layer 222 is filled with a filler and planarized, and then the spectral filter layer 223 is formed thereon.

  As the filler, any material that does not hinder the function of the polarizing filter layer 222 whose uneven surface is flattened by the filler may be used. In this embodiment, a material having no polarization function is used. In addition, as the planarization treatment with the filler, for example, a method of applying the filler by a spin-on-glass method can be suitably employed, but the present invention is not limited to this.

In the present embodiment, the first region of the polarization filter layer 222 is a vertical polarization region that selectively transmits only the vertical polarization component that vibrates in parallel with the column (vertical direction) of the imaging pixels of the image sensor 206, and the polarization filter The second region of the layer 222 is a horizontal polarization region that selects and transmits only the horizontal polarization component that vibrates parallel to the row (horizontal direction) of the imaging pixels of the image sensor 206.
The first region of the spectral filter layer 223 is a red spectral region that selectively transmits only light in the red wavelength band (specific wavelength band) included in the usable wavelength band that can be transmitted through the polarizing filter layer 222. The second region of the filter layer 223 is a non-spectral region that transmits light without performing wavelength selection. In the present embodiment, as indicated by a one-dot chain line in FIG. 14, imaged image data is obtained by a total of four imaging pixels (4 imaging pixels of reference symbols a, b, e, and f) adjacent to each other in two vertical and two horizontal directions. 1 image pixel is formed.

In the imaging pixel a shown in FIG. 14, light that has passed through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel a receives light P / R in the red wavelength band (indicated by symbol R in FIG. 14) of the vertically polarized component (indicated by symbol P in FIG. 14).
In addition, in the imaging pixel b illustrated in FIG. 14, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel b receives light P / C of non-spectral (indicated by symbol C in FIG. 14) in the vertical polarization component P.
In addition, in the imaging pixel e illustrated in FIG. 14, light that has passed through the horizontal polarization region (second region) of the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel e receives the non-spectral C light S / C in the horizontal polarization component (indicated by symbol S in FIG. 14).
In the imaging pixel f shown in FIG. 14, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel f receives the light P / R in the red wavelength band R in the vertical polarization component P, similarly to the imaging pixel a.

  With the above configuration, according to the present embodiment, one image pixel for the vertical polarization component image of red light is obtained from the output signals of the imaging pixel a and the imaging pixel f, and the non-spectral vertical is obtained from the output signal of the imaging pixel b. One image pixel for the polarization component image is obtained, and one image pixel for the non-spectral horizontal polarization component image is obtained from the output signal of the imaging pixel e. Therefore, according to the present embodiment, three types of captured image data, that is, a red light vertical polarization component image, a non-spectral vertical polarization component image, and a non-spectral horizontal polarization component image, can be obtained by a single imaging operation. .

In these captured image data, the number of image pixels is smaller than the number of captured pixels, but a generally known image interpolation technique may be used to obtain a higher resolution image. For example, when trying to obtain a vertical polarization component image of red light having a higher resolution, the vertical polarization of the red light received by the imaging pixels a and f for the imaging pixel a and the imaging pixel f. For the image pixel corresponding to the imaging pixel b using the information of the component P as it is, for example, the average value of the imaging pixels a, c, f, and j surrounding the periphery is used as information on the vertical polarization component of the red light of the image pixel. Use as
Further, when obtaining a non-spectral horizontal polarization component image having a higher resolution, the information of the non-spectral horizontal polarization component S received by the imaging pixel e is used as it is for the image pixel corresponding to the imaging pixel e. For the image pixels corresponding to the imaging pixels a, b, and f, an average value of the imaging pixel e and the imaging pixel g that receives the non-spectral horizontal polarized light component around it is used, or the same as the imaging pixel e. Or a value may be used.

  The vertically polarized component image of red light obtained in this way can be used, for example, for identifying a tail lamp. Since the vertical polarization component image of the red light has the horizontal polarization component S cut off, the red light is horizontal, such as red light reflected on the road surface or red light (reflection light) from the dashboard of the vehicle 100 or the like. It is possible to obtain a red image in which a disturbance factor due to red light having a strong polarization component S is suppressed. Therefore, the recognition rate of the tail lamp is improved by using the vertical polarization component image of the red light for the identification of the tail lamp.

  The non-spectral vertical polarization component image can be used, for example, for identifying a white line or a headlamp of an oncoming vehicle. In the non-spectral horizontal polarization component image, since the horizontal polarization component S is cut off, white light such as a headlamp or a streetlight reflected on the road surface, white light from a dashboard in the interior of the vehicle 100, etc. (reflection light) A non-spectral image in which disturbance factors due to white light with a strong horizontal polarization component S are suppressed can be obtained. Therefore, the recognition rate is improved by using the non-spectral vertically polarized component image for identifying the white line and the headlamp of the oncoming vehicle. In particular, it is generally known that the reflected light from the water surface covering the road surface has a large amount of horizontal polarization component S in a rainy road. Therefore, by using the non-spectral vertical polarization component image for white line identification, it becomes possible to appropriately identify the white line under the water surface in the rainy road, and the recognition rate is improved.

  In addition, if a comparison image using pixel values as index values obtained by comparing pixel values between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image is used, as described later, the metal in the imaging region It is possible to accurately identify an object, a wet and dry state of a road surface, a three-dimensional object in an imaging region, and a white line on a rainy road. As a comparison image used here, for example, a difference image in which a pixel value is a difference value between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image, or a pixel value between these images. Use a ratio image with the ratio as the pixel value, or a differential polarization degree image with the ratio of the difference value of the pixel values between these images to the sum of the pixel values between these images (difference polarization degree) as the pixel value can do.

FIG. 16 shows the contents of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the raindrop detection filter unit 220B of the optical filter 205 in the present embodiment. It is explanatory drawing which shows.
FIG. 17A is a cross-sectional view schematically showing the raindrop detection filter unit 220B and the image sensor 206 of the optical filter 205 cut along the line AA shown in FIG.
FIG. 17B is a cross-sectional view schematically showing the raindrop detection filter unit 220B and the image sensor 206 of the optical filter 205 cut along the line BB shown in FIG.

  As shown in FIGS. 17A and 17B, the raindrop detection filter portion 220B of the optical filter 205 in this embodiment has a wire grid structure on the filter substrate 221 common to the vehicle detection filter portion 220A. A polarizing filter layer 225 is formed. Along with the polarizing filter layer 222 of the vehicle detection filter unit 220A, the polarizing filter layer 225 is flattened by filling the upper surface side in the stacking direction with a filler. However, unlike the vehicle detection filter unit 220A, the raindrop detection filter unit 220B has no spectral filter layer 223 laminated thereon.

  In the present embodiment, the indoor landscape of the host vehicle 100 may be reflected on the inner wall surface of the windshield 105. This reflection is due to the light regularly reflected by the inner wall surface of the windshield 105. Since this reflection is specular reflection light, it is disturbance light having a relatively high light intensity. Therefore, if this reflection is projected on the raindrop detection image area 214 together with the raindrop, the raindrop detection accuracy is lowered. In addition, specularly reflected light, which is specularly reflected by the inner wall surface of the windshield 105, is also reflected in the raindrop detection image area 214 together with raindrops, resulting in disturbance light that reduces raindrop detection accuracy. Let

  The disturbance light that reduces the detection accuracy of raindrops is specularly reflected light that is specularly reflected by the inner wall surface of the windshield 105, so that most of the polarization component is a polarization component whose polarization direction is perpendicular to the light source incident surface. That is, it is the horizontal polarization component S that vibrates in parallel with the row (horizontal direction) of the imaging pixels of the image sensor 206. Therefore, the polarizing filter layer 225 in the raindrop detection filter unit 220B of the optical filter 205 of the present embodiment has a virtual surface (including the optical axis of the light from the light source unit 202 toward the windshield 105 and the optical axis of the imaging lens 204). The transmission axis is set so as to transmit only the polarization component whose polarization direction is parallel to the light source incident surface), that is, the vertical polarization component P that vibrates parallel to the column (vertical direction) of the imaging pixels of the image sensor 206. ing.

  As a result, the light transmitted through the polarizing filter layer 225 of the raindrop detection filter unit 220B is only the vertical polarization component P, and the light source unit is reflected on the inner wall surface of the windshield 105 or regularly reflected by the inner wall surface of the windshield 105. The horizontally polarized component S occupying most of disturbance light such as specularly reflected light from 202 can be cut. As a result, the raindrop detection image area 214 becomes a vertically polarized image based on the vertical polarization component P, which is less affected by disturbance light, and the raindrop detection accuracy based on the captured image data of the raindrop detection image area 214 is improved.

  In the present embodiment, the infrared light cut filter region 211 and the infrared light transmission filter region 212 constituting the pre-stage filter 210 are each formed of multilayer films having different layer configurations. As a manufacturing method of such a pre-stage filter 210, after the infrared light transmission filter region 212 is formed by vacuum deposition or the like while concealing the infrared light cut filter region 211 with a mask, this time, infrared light is filtered. There is a method in which the part of the infrared light cut filter region 211 is formed by vacuum deposition or the like while the part of the transmission filter region 212 is hidden with a mask.

  In the present embodiment, both the polarizing filter layer 222 of the vehicle detection filter unit 220A and the polarizing filter layer 225 of the raindrop detection filter unit 220B have a wire grid structure in which regions are divided in a two-dimensional direction. However, the former polarizing filter layer 222 is obtained by dividing two types of regions (vertical polarizing region and horizontal polarizing region) whose transmission axes are orthogonal to each other, and the latter polarizing filter layer 225 is vertically polarized. One type of region having a transmission axis that transmits only the component P is divided into regions for each imaging pixel. When the polarizing filter layers 222 and 225 having such different configurations are formed on the same filter substrate 221, for example, adjustment of the groove direction of a template (corresponding to a mold) for patterning a metal wire having a wire grid structure Therefore, adjustment of the longitudinal direction of the metal wire in each region is easy.

In the present embodiment, the infrared light cut filter region 211 may not be provided in the optical filter 205, and the infrared light cut filter region 211 may be provided in the imaging lens 204, for example. In this case, the optical filter 205 can be easily manufactured.
Further, instead of the infrared light cut filter region 211, a spectral filter layer that transmits only the vertical polarization component P may be formed in the raindrop detection filter unit 220B of the post-stage filter 220. In this case, it is not necessary to form the infrared light cut filter region 211 in the upstream filter 210.
Further, the polarizing filter layer is not necessarily provided.
Further, in the optical filter 205 of the present embodiment, a rear-stage filter 220 having a polarizing filter layer 222 and a spectral filter layer 223 divided into regions as shown in FIG. 14 is provided on the image sensor 206 side with respect to the front-stage filter 210. However, the front-stage filter 210 may be provided closer to the image sensor 206 than the rear-stage filter 220.

[Light distribution control of headlamps]
Hereinafter, the light distribution control of the headlamp in the present embodiment will be described.
In the light distribution control of the headlamp in the present embodiment, the captured image data captured by the imaging unit 200 is analyzed to identify the tail lamp and head lamp of the vehicle, the preceding vehicle is detected from the identified tail lamp, and the identified head The oncoming vehicle is detected from the ramp. Then, the driver of the host vehicle 100 can be prevented from being dazzled by avoiding the strong light from the headlamp of the host vehicle 100 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle, and the driver's view of the host vehicle 100 is secured. In order to realize the above, the switching of the high beam and the low beam of the headlamp 104 is controlled, or partial light shielding control of the headlamp 104 is performed.

  In the headlamp light distribution control of the present embodiment, among the information that can be acquired from the imaging unit 101, the intensity (brightness information) of light emitted from each point (light source body) in the imaging region, the headlamp, The distance (distance information) between the light source body (other vehicle) such as the tail lamp and the own vehicle, the spectral information by comparing the red component and the white component (non-spectral) of the light emitted from each light source body, the horizontal polarization component of the white component Information of a white component with a horizontal polarization component cut off, and vertical polarization component information of a red component with a horizontal polarization component cut off are used.

  The brightness information will be described. When the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are imaged by the imaging unit 200, the detected object is detected on the captured image data. One headlamp of the oncoming vehicle is projected brightest, and the tail lamp of the preceding vehicle, which is one of the detection objects, is projected darker than that. In addition, when the reflector is displayed in the captured image data, the reflector is not a light source that emits light by itself, but merely a bright image that is reflected by reflecting the headlamp of the host vehicle, so that it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the reflector is gradually observed on the image sensor 206 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, primary identification of the two types of detection objects (head lamp and tail lamp) and the reflector is possible.

  Further, the distance information will be described. Most headlamps and tail lamps have a pair of left and right pair lamps. Therefore, the distance between the headlamps and tail lamps (that is, other vehicles) and the host vehicle is utilized using the characteristics of this configuration. Can be obtained. The pair of left and right lamps that are paired are displayed close to each other at the same height position on the captured image data captured by the image capturing unit 200, and the widths of the lamp image areas that project the lamps are substantially the same. The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, lamp image areas satisfying the conditions can be identified as pair lamps. At a long distance, the left and right lamps constituting the pair lamp cannot be distinguished and recognized as a single lamp.

  When a pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp and tail lamp which are the pair lamp structure. That is, the distance between the left and right headlamps and the distance between the left and right tail lamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the imaging lens 204 in the imaging unit 200 is known, the distance w1 between the two lamp image areas respectively corresponding to the left and right lamps on the image sensor 206 of the imaging unit 200 is calculated from the captured image data. Accordingly, the distance x between the light source of the headlamp or taillamp which is the pair lamp configuration and the host vehicle can be obtained by simple proportional calculation (x = f × w0 / w1). If the distance x calculated in this way is within an appropriate range, the two lamp image areas used for the calculation can be identified as headlamps and taillamps of other vehicles. Therefore, by using this distance information, the identification accuracy of the head lamp and tail lamp, which are detection objects, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, the imaging pixels a on the image sensor 206 that receive red light (vertical polarization component) P / R from the captured image data captured by the imaging unit 200. By extracting only pixel data corresponding to c, f, h, etc., it is possible to generate a red image in which only the red component in the imaging region is projected. Therefore, when there is an image area having a luminance equal to or higher than a predetermined luminance in the red image, the image area can be identified as a tail lamp image area in which a tail lamp is projected.

  In addition, only pixel data corresponding to the imaging pixels b and d on the image sensor 206 that receives the vertical polarization component P / C of white light (non-spectral) is extracted from the captured image data captured by the imaging unit 200. Thus, a monochrome luminance image (vertical polarization component) within the imaging region can be generated. Therefore, the luminance ratio (red luminance ratio) between the image area on the red image and the image area on the monochrome luminance image corresponding to the image area can be calculated. By using this red luminance ratio, it is possible to grasp the ratio of the relative red component contained in the light from the object (light source body) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the identification accuracy of the tail lamp is improved by using this red luminance ratio.

  Further, the polarization information will be described. In the present embodiment, as described above, imaging on the image sensor 206 that receives the white light (non-spectral) vertical polarization component P / C from the captured image data captured by the imaging unit 200. Extracting pixel data corresponding to the pixels b, d, and the like, and pixel data corresponding to the imaging pixels e, g, etc. on the image sensor 206 that receives the horizontal polarization component S / C of white light (non-spectral), For each image pixel, a comparison image in which pixel values (luminance) between these image data are compared can be obtained. Specifically, for example, a difference image having a pixel value as a difference value (SP) between a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) is compared with a comparison image. Can be obtained as According to such a comparative image, an image area (headlamp image area) of direct light directly incident on the imaging unit 200 from the headlamp and an incident on the imaging unit 200 after being reflected from the water surface of the rain road from the headlamp. The contrast with the image area of indirect light can be increased, and the identification accuracy of the headlamp is improved.

  In particular, as a comparative image, a ratio image with a ratio (S / P) of a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) as a pixel value, or a differential polarization degree A differential polarization degree image having ((S−P) / (S + P)) as a pixel value can be suitably used. In general, it is known that the light reflected by a horizontal mirror such as a water surface always has a high horizontal polarization component. In particular, the ratio (S / P) between the horizontal polarization component S and the vertical polarization component P and the differential polarization degree. When ((S−P) / (S + P)) is taken, it is known that the ratio and the differential polarization degree become maximum at a specific angle (Brewster angle). In rainy roads, water is applied to the asphalt surface, which is a scattering surface, in a state close to a mirror surface, so that the headlamp reflected light from the road surface is stronger in the horizontal polarization component S. Therefore, the image area of the headlamp reflected light from the road surface has a large pixel value (luminance) in the ratio image and the differential polarization degree image. On the other hand, since the direct light from the headlamp is basically non-polarized light, the pixel value (luminance) is small in the ratio image and the differential polarization degree image. Due to this difference, it is possible to appropriately exclude the headlamp reflected light from the rain road surface having the same amount of light as the direct light from the headlamp, and distinguish the direct light from the headlamp from such headlamp reflected light. be able to.

Next, a flow of detection processing for the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 18 is a flowchart showing a flow of vehicle detection processing in the present embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the imaging unit 200, and an image region that is considered to be a detection target is extracted. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of detection objects is the type of the light source displayed in the image area.

  First, in step S1, image data in front of the host vehicle captured by the image sensor 206 of the imaging unit 200 is taken into a memory. As described above, this image data includes a signal indicating the luminance in each imaging pixel of the image sensor 206. Next, in step S2, information related to the behavior of the host vehicle is taken from the vehicle behavior sensor.

  In step S3, an image area with high brightness (high brightness image area) that is considered to be a detection target (tail lamp of the preceding vehicle and bed lamp of the oncoming vehicle) is extracted from the image data captured in the memory. This high luminance image region is a bright region having a luminance higher than a predetermined threshold luminance in the image data, and there are many cases where a plurality of high luminance image regions are extracted, but all of them are extracted. Therefore, at this stage, an image area that reflects reflected light from the rainy road surface is also extracted as a high-luminance image area.

  In the high luminance image region extraction processing, first, in step S31, binarization processing is performed by comparing the luminance value of each imaging pixel on the image sensor 206 with a predetermined threshold luminance. Specifically, a binary image is created by assigning “1” to pixels having a luminance equal to or higher than a predetermined threshold luminance and assigning “0” to other pixels. Next, in step S32, when pixels assigned with “1” are close to each other in the binarized image, a labeling process for recognizing them as one high luminance image region is performed. Thereby, a set of a plurality of adjacent pixels having a high luminance value is extracted as one high luminance image region.

  In step S4, which is executed after the above-described high-intensity image region extraction process, the distance between the object in the imaging region corresponding to each extracted high-intensity image region and the host vehicle is calculated. In this distance calculation process, the pair lamp distance calculation process that detects the distance by using the pair of left and right lamps of the vehicle and the left and right lamps constituting the pair lamp cannot be distinguished and recognized at a long distance. A single lamp distance calculation process is executed when the pair lamp is recognized as a single lamp.

  First, for a pair lamp distance calculation process, in step S41, a pair lamp creation process, which is a process of creating a lamp pair, is performed. The pair of left and right lamps that are paired are in the position where they are close to each other and have substantially the same height in the image data captured by the imaging unit 200, and the area of the high-luminance image region is substantially the same, and the shape of the high-luminance image region is the same. Satisfy the condition of being the same. Therefore, the high-intensity image areas that satisfy such conditions are used as a pair lamp. High brightness image areas that cannot be paired are considered a single lamp. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation process in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f in the imaging unit 200 is known, the actual distance x to the pair lamps is calculated by a simple proportional calculation (x = x = x) by calculating the left / right lamp distance w1 on the image sensor 206 of the imaging unit 200. f · w0 / w1). It should be noted that a dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio (red luminance ratio) between the red image of the vertical polarization component P and the white image of the vertical polarization component P is used as spectral information. From this spectral information, two high-luminance image regions that are paired lamps are A lamp type identification process is performed for identifying whether the light is from the head lamp or from the tail lamp. In this lamp type identification process, first, in step S51, pixel data corresponding to the imaging pixels a and f on the image sensor 206 and pixels corresponding to the imaging pixel b on the image sensor 206 for the high-intensity image area that is a pair of lamps. A red ratio image having a pixel value as a ratio to the data is created. In step S52, the pixel value of the red ratio image is compared with a predetermined threshold, and a high-luminance image region that is equal to or higher than the predetermined threshold is determined to be a tail lamp image region by light from the tail lamp. For a certain high-intensity image area, a lamp type process is performed to determine that the area is a headlamp image area by light from the headlamp.

  Subsequently, in step S6, for each image area identified as the tail lamp image area and the head lamp image area, the difference polarization degree ((S−P) / (S + P)) is used as the polarization information from the tail lamp or the head lamp. A reflection identification process is performed for identifying whether the light is direct light or reflected light reflected by a mirror surface such as a rainy road surface. In this reflection identification process, first, in step S61, a differential polarization degree ((S−P) / (S + P)) is calculated for the tail lamp image region, and a differential polarization degree image with the differential polarization degree as a pixel value is created. Similarly, a difference polarization degree ((S−P) / (S + P)) is calculated for the headlamp image region, and a difference polarization degree image is created using the difference polarization degree as a pixel value. In step S62, the pixel value of each differential polarization degree image is compared with a predetermined threshold value, and it is determined that the tail lamp image area and the head lamp image area that are equal to or larger than the predetermined threshold value are caused by reflected light. Those image areas are excluded because they do not reflect the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle. The tail lamp image area and the head lamp image area remaining after performing this exclusion process are identified as those that reflect the tail lamp of the preceding vehicle or those that reflect the head lamp of the oncoming vehicle.

  Note that the reflection identification process S6 described above may be executed only when a rain sensor or the like is mounted on the vehicle and it is confirmed that the rain sensor is raining. Further, the above-described reflection identification process S6 may be executed only when the driver (driver) is operating the wiper. In short, the above-described reflection identification process S6 may be executed only in rainy weather when reflection from the rainy road surface is assumed.

  In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection process as described above are used for light distribution control of a headlamp that is an in-vehicle device of the host vehicle. Specifically, when the tail lamp is detected by the vehicle detection processing and approaches the distance range in which the headlamp illumination light of the host vehicle is incident on the rearview mirror of the preceding vehicle, the headlamp illumination of the host vehicle is approached to the preceding vehicle. Control is performed to block a part of the headlamp of the host vehicle or to shift the light irradiation direction of the headlamp of the host vehicle in the vertical direction or the horizontal direction so as not to be exposed to light. Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range where the headlamp illumination light of the own vehicle hits, the headlamp illumination light of the own vehicle is emitted to the oncoming vehicle. In order to avoid hitting, a part of the headlamp of the host vehicle is shielded from light, or the light irradiation direction of the headlamp of the host vehicle is shifted in the vertical direction or the horizontal direction.

[White line detection processing]
Hereinafter, the white line detection process in this embodiment will be described.
In the present embodiment, processing for detecting a white line (partition line) as a detection target is performed for the purpose of preventing the vehicle from deviating from the travelable area. Here, the white line includes all white lines that demarcate the road, such as a solid line, a broken line, a dotted line, and a double line. In addition, it can detect similarly about the division line of colors other than white, such as a yellow line.

  In the white line detection processing in the present embodiment, polarization information of the vertical polarization component P of the white component (non-spectral) among the information that can be acquired from the imaging unit 101 is used. In addition, the vertical polarization component of cyan light may be included in the vertical polarization component of white component. In general, it is known that white lines and asphalt surfaces have flat spectral luminance characteristics in the visible light region. On the other hand, since cyan light includes a wide band in the visible light region, it is suitable for imaging asphalt and white lines. Therefore, by using the optical filter 205 in the configuration example 2 and including the vertical polarization component of cyan light in the vertical polarization component of the white component, the number of imaging pixels to be used increases, resulting in an increase in resolution and a distant white line. Can also be detected.

  In the white line detection processing of this embodiment, in many roads, white lines are formed on the road surface of a color close to black, and the brightness of the white line portion in the image of the white component (non-spectral) vertical polarization component P is on the road surface. Big enough than other parts. Therefore, it is possible to detect a white line by determining a portion of the road surface portion having a luminance equal to or higher than a predetermined value as a white line. In particular, in the present embodiment, the image of the vertical polarization component P of the white component (non-spectral) to be used is obtained by suppressing the reflected light from the rainy road because the horizontal polarization component S is cut. Is possible. Therefore, it is possible to detect a white line without erroneously recognizing disturbance light such as reflected light from a headlamp from a rainy road at night as a white line.

  Also, in the white line detection processing in the present embodiment, out of the information that can be acquired from the imaging unit 101, polarization information obtained by comparing the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P, for example, white The differential polarization degree ((S−P) / (S + P)) of the component (non-spectral) horizontal polarization component S and vertical polarization component P may be used. Since the reflected light from the white line is usually dominated by the diffuse reflection component, the vertical polarization component P and the horizontal polarization component S of the reflected light are substantially equal, and the degree of differential polarization shows a value close to zero. On the other hand, the asphalt surface portion on which no white line is formed shows a characteristic in which the scattered reflection component is dominant in the dry state, and the differential polarization degree shows a positive value. Further, when the asphalt surface portion where the white line is not formed is in a wet state, the specular reflection component is dominant, and the differential polarization degree shows a larger value. Therefore, a portion where the polarization difference value of the obtained road surface portion is smaller than the predetermined threshold value can be determined as a white line.

[Raindrop detection processing on the windshield]
Hereinafter, the raindrop detection process in the present embodiment will be described.
In the present embodiment, for the purpose of performing drive control of the wiper 107 and discharge control of the washer liquid, processing for detecting raindrops as deposits is performed. In addition, here, the case where the deposit adhered on the windshield is a raindrop will be described as an example. However, the same applies to deposits such as bird droppings and splashes on the road surface from the adjacent vehicle. is there.

  In the raindrop detection process in the present embodiment, among the information that can be acquired from the imaging unit 101, the infrared light transmission filter region 212 of the front-stage filter 210 and the polarization filter layer 225 in the raindrop detection filter unit 220B of the rear-stage filter 220 are used. The polarization information of the vertical polarization component P of the raindrop detection image area 214 that receives the transmitted light is used. Therefore, it is necessary that the light incident on the windshield 105 from the light source unit 202 includes a large amount of the vertically polarized component P. For this purpose, for example, when a light emitting diode (LED) is used as the light source of the light source unit 202, a polarizer that transmits only the vertical polarization component P is preferably disposed between the light source unit 202 and the windshield 105. Further, when a semiconductor laser (LD) is used as the light source of the light source unit 202, the LD can emit only light of a specific polarization component, so that the light of only the vertical polarization component P is incident on the windshield 105. The axes may be aligned.

  In the present embodiment, as described above, the illumination light (infrared light) irradiated from the light source unit 202 and incident on the inner wall surface of the windshield 105 from the reflection deflection prism 230 has raindrops attached to the outer wall surface of the windshield 105. In the non-attached part which is not carried out, regular reflection is carried out by the outer wall surface of the windshield 105. FIG. The specularly reflected light is received by the image sensor 206 and displayed on the raindrop detection image area 214. On the other hand, at a place where raindrops are attached on the outer wall surface of the windshield 105, the illumination light is transmitted through the outer wall surface of the windshield 105, and the transmitted light is not received by the image sensor 206. Therefore, in the raindrop detection image area 214 of the captured image data, the non-attached portion where the raindrop does not adhere to the outer wall surface of the windshield 105 becomes a high brightness image portion (high pixel value), while the raindrop adheres. The attached part becomes a low-brightness image part (low pixel value). From these differences, it is possible to grasp not only the presence of raindrops but also the amount of raindrops.

FIG. 19 is an explanatory diagram of raindrop detection processing in the present embodiment.
In the raindrop detection process of the present embodiment, the wiper 107 is driven when it is determined that the raindrop amount exceeds a certain amount using information on the raindrop detection image area 214 of the captured image data acquired from the imaging unit 101. Specifically, the raindrop detection image area 214 is divided into eight sections in the horizontal direction of the image as shown in FIG. 19, for example, and total pixel values IL1 to IL8 in each section are calculated. Then, if the total pixel values IL1 to IL8 in any of the sections fall below a predetermined threshold value ILth, it is determined that the raindrop amount exceeds a certain amount, and the wiper 107 is driven. On the contrary, when the total pixel values IL1 to IL8 in any of the sections become equal to or greater than a predetermined threshold value ILth, the driving of the wiper 107 is stopped. The conditions for starting and stopping the driving of the wiper 107 are not limited to this, and can be set as appropriate. For example, the threshold value does not have to be a fixed value, and may be changed as appropriate according to a change in the situation around the host vehicle on which the imaging unit 200 is mounted. Moreover, the threshold value of the start condition and the stop condition may be the same value or different values.

FIG. 20 is an explanatory diagram showing a relationship between a conventional imaging frame and raindrop detection.
FIG. 21 is an explanatory diagram showing the relationship between the imaging frame and raindrop detection in the present embodiment.
Conventionally, as shown in FIG. 20, imaging frames (sensing frames) frames 1 to 4, 6 to 6 for detecting a detection target such as a white line (partition line) on the road surface and other vehicles existing in the imaging region. 9, 11 to 14 and the imaging frames (raindrop detection frames) frames 5 and 10 for detecting raindrops are different imaging frames. In this case, since there is a time gap between the sensing frames before and after the raindrop detection frame, the object to be detected is placed between the imaging time of the previous sensing frame and the imaging time of the subsequent sensing frame. There is a time during which an image for detection cannot be acquired, causing problems such as a reduction in recognition accuracy of a detection target.

  On the other hand, in this embodiment, as shown in FIG. 21, an image having an image (vehicle detection image region) for detecting the detection target and a raindrop detection image region for detecting the raindrop 203. Can be obtained with a single imaging frame frame2,4,6,8,10,12. In this case, a dedicated imaging frame (raindrop detection frame) for detecting raindrops is not necessary, and the above-described problems do not occur.

  Here, in this embodiment, in order to obtain a suitable vehicle detection image area (for example, an image area with high contrast), automatic exposure control (AEC) that changes the exposure time according to the situation (brightness, etc.) of the imaging area. ). Specifically, automatic exposure control is performed to change the exposure time of the next imaging frame in accordance with the luminance value of the central portion of the captured image (pixels in the vehicle detection image area 213) in the previous imaging frame.

  However, when such automatic exposure control is carried out, even if the raindrop adhesion situation on the windshield is exactly the same, the brightness and contrast of the raindrop detection image area may change due to the change in the exposure time. There is. In this case, in the raindrop detection process of this embodiment, the amount of raindrops detected from the information in the raindrop detection image area 214 cannot be properly determined, and problems such as a malfunction of the wiper 107 may occur.

FIG. 22 is an explanatory diagram showing an example of the relationship between the exposure period of each light receiving element line set by automatic exposure control and the lighting period of the light source unit 202 in one image (imaging frame).
In the present embodiment, the light source control unit 102B of the image analysis unit 102 performs the light source unit 202 during the total exposure period in the sensor portion of the image sensor 206 corresponding to the raindrop detection image region 214 (for 100 lines from the top of the sensor). The light source light emitted from is periodically turned on and off.

  In the exposure time set by the automatic exposure control as in the example shown in FIG. 22, the maximum number of light receiving element lines with overlapping exposure periods is four. In the present embodiment, the sensor portion of the image sensor 206 corresponding to the raindrop detection image region 214 corresponds to 100 lines (line numbers 1 to 100) from the top of the sensor. Therefore, as shown in FIG. 22, the 100 lines are divided into four lines, and the lighting cycle of the light source unit 202 is set so that one lighting period is assigned to each section. During the exposure period, the light source light having a light quantity corresponding to one lighting period is received. That is, the amount of light source light irradiated during each exposure period is the same between the light receiving element lines. As a result, there is no inconvenience such as erroneous detection of the amount of raindrops due to non-uniformity of the amount of light source light irradiated during the exposure period between the light receiving element lines.

  As described above, the light source light is periodically turned on and off during the total exposure period in the sensor portion of the image sensor 206 corresponding to the raindrop detection image region 214, so that the light source light is always turned on during the total exposure period. Compared to the case, power saving of the light source unit 202 can be achieved.

  In the example shown in FIG. 22, the exposure time is set such that the exposure period of the other light receiving element lines (light receiving element lines after four lines) is just started at the timing when the exposure period of one light receiving element line ends. Has been. Therefore, except for the first and last three lines of the light receiving element line group corresponding to the raindrop detection image area 214, the number of light receiving element lines having overlapping exposure periods is always four. However, the exposure time may be set such that the timing at which the exposure period of one light receiving element line ends and the timing at which the exposure period of another light receiving element line starts are different. In this case, in the light receiving element lines excluding the first and last three lines of the light receiving element line group corresponding to the raindrop detection image region 214, the exposure overlap period in which the exposure periods for five lines overlap and the lines for four lines. There is an exposure overlap period in which the exposure periods overlap.

  Even in such a case, if the length of the exposure overlap period in which the exposure periods for five lines overlap is longer than one lighting period (light source light irradiation period) by the light source unit 202, the example of FIG. Similarly to the above, a lighting cycle is set such that one lighting period is assigned to each section divided into the maximum number (5 lines) of light receiving element lines whose exposure periods overlap each other, and the lighting period is within the exposure overlapping period. Should be included. However, when the length of the exposure overlap period in which the exposure periods for five lines overlap is less than one lighting period (light source light irradiation period) by the light source unit 202, the exposure periods overlap each other. Even if the maximum number of light receiving element lines is 5, it is divided into four lines, and a lighting cycle is set such that one lighting period is assigned to each section.

  Here, when the lighting cycle set in this way is fixed, if the exposure time becomes longer by automatic exposure control, for example, the amount of light source light irradiated during the exposure period becomes non-uniform between the light receiving element lines. . For example, in the example shown in FIG. 23, the exposure time of the light receiving element line of line number 5 belonging to the division number 24 is assigned to the division number 24 as a result of the longer exposure time than the example shown in FIG. It overlaps not only the lighting period that is present, but also the lighting period that is assigned to the division number 25. Therefore, the light amount of the light source light irradiated during the exposure period of the other light receiving element lines (line numbers 6 to 8) belonging to the division number 24 corresponds to one lighting period, whereas the light receiving element line of the line number 5 The amount of light source light irradiated during the exposure period corresponds to two lighting periods. As described above, when the exposure time becomes longer due to the automatic exposure control, the light amount of the light source light irradiated during the exposure period becomes non-uniform between the light receiving element lines with the lighting cycle of the example shown in FIG. May cause problems such as false detection.

  Therefore, in the present embodiment, the light source control unit 102B performs lighting cycle control for changing the lighting cycle of the light source unit 202 according to the exposure time changed by the automatic exposure control, and even if the exposure time is changed, The amount of light of the light source irradiated during the exposure period is maintained at the same amount between the light receiving element lines. Specifically, the lighting cycle (on / off cycle) of the light source light is changed so that one lighting period is assigned for each maximum number of light receiving element lines whose exposure periods overlap each other in a time longer than one lighting period. The lighting cycle control is performed as described above.

FIG. 24 is a flowchart showing a flow of lighting cycle control in the present embodiment.
In the image analysis unit 102, when the light source control unit 102B obtains the exposure time of the next frame determined by the automatic exposure control by the exposure time control unit 102C (S11), a time longer than one lighting period is obtained from this exposure time. In step S12, the maximum number of light receiving element lines whose exposure periods overlap each other (maximum number of overlapping lines) is calculated. For example, in the example shown in FIG. 22, the maximum number of overlapping lines is calculated as 4 lines, and in the example shown in FIG. 23, the maximum number of overlapping lines is calculated as 4 lines.

  Then, the light source control unit 102B divides the light receiving line group for 100 lines corresponding to the raindrop detection image region 214 for each maximum overlapping line number calculated in the processing step S12, and one lighting period is provided for each division. The lighting cycle of the light source unit 202 is set so as to be assigned (S13). At this time, it is preferable to set the phase of the lighting cycle (lighting timing of the light source unit 202) so that the number of lighting periods is minimized. In this embodiment, lighting is performed so that the number of lighting periods is minimized. The phase of the cycle is determined (S14).

  Specifically, either the first light receiving element line (line number 1) or the last light receiving element line (line number 100) in the light receiving element line group corresponding to the raindrop detection image region 214 is used as a reference. Then, the light receiving element lines whose exposure periods overlap in a time equal to or longer than one lighting period with respect to the reference light receiving element line are classified into the same section, and the lighting period is set when the exposure periods overlap between the light receiving element lines in the section. Is determined so that the period of the lighting period (lighting timing) for the category is included. Assuming that the lighting period determined in this way is the phase reference of the lighting cycle, when the light source unit 202 is on / off controlled at the lighting cycle set in the processing step S13, the total light reception corresponding to the raindrop detection image region 214 is performed. The number of lighting periods required to allocate to the element line is minimized.

  When the exposure time is set as in the example shown in FIG. 25, the maximum number of overlapping lines is 6, and when the number of light receiving element lines corresponding to the raindrop detection image area 214 (100 lines) is divided, The number of light receiving element lines (line numbers 1 to 4) to which only the section (section number 17) belongs is four lines. Even in such a case, if the lighting cycle and phase are determined as described above, the four light receiving element lines (line numbers 1 to 4) belonging to the last section (section number 17) are also irradiated during each exposure period. The light amount of the light source light is equivalent to one lighting period as in the other sections, and the light amount of the light source light irradiated during each exposure period in all the light receiving element lines corresponding to the raindrop detection image region 214. Can be the same amount.

  In particular, in the present embodiment, when determining the phase of the lighting cycle (S14), the last light receiving element line (line number 100) in the light receiving element line group corresponding to the raindrop detection image region 214 is used as a reference. That is, in the present embodiment, a light receiving element line adjacent to the light receiving element line group (line numbers 101 to 101) corresponding to the vehicle detection image area 213 among the light receiving element line groups corresponding to the raindrop detection image area 214 is used as a reference. To do.

  If the last light receiving element line (line number 100) is used as a reference, the number of light receiving element lines (100 lines) corresponding to the raindrop detection image area 214 is the maximum number of overlapping lines as in the example shown in FIG. When it is not possible to just categorize with (6 lines), it corresponds to the vehicle detection image area 213 in addition to the four light receiving element lines (line numbers 97 to 100) corresponding to the raindrop detection image area 214 in the last division. Two light receiving element lines (line numbers 101 to 102) are included. In this case, the lighting period assigned to the last section overlaps the exposure periods of the two light receiving element lines (line numbers 101 to 102) corresponding to the vehicle detection image region 213. Therefore, since the light source light can be disturbance light of the two light receiving element lines corresponding to the vehicle detection image region 213, there is a possibility of causing misidentification due to the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the light source light of the white line. .

  On the other hand, when the last light receiving element line (line number 100) adjacent to the light receiving element line group (line number 101 to) corresponding to the vehicle detection image region 213 is used as a reference, as in the present embodiment, the last The lighting period set so as to overlap with the exposure period of the light receiving element line does not overlap with the exposure period of the light receiving element line group (line numbers 101 to 101) corresponding to the vehicle detection image region 213. As a result, naturally, the lighting period of the light source unit 202 does not overlap with the exposure period of the light receiving element line group corresponding to the vehicle detection image region 213. Therefore, since the light source light is not irradiated during each exposure period of the light receiving element line group corresponding to the vehicle detection image region 213, the light source light does not become disturbance light of the vehicle detection image region 213. Misidentification due to the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the light source light of the white line can be suppressed.

  When the lighting cycle and phase of the light source unit 202 are determined as described above, the light source control unit 102B periodically turns on and off the light source light at the determined lighting cycle and phase in synchronization with the imaging operation of the next imaging frame. Thus, drive control of the light source unit 202 is performed (S15). Thereby, even if the exposure time is changed by the automatic exposure control, the light amount of the light source light irradiated during each exposure period of the light receiving element line group corresponding to the raindrop detection image area 214 is the same amount between the light receiving element lines. It is maintained, and malfunctions such as erroneous detection of raindrop amount can be suppressed.

  Even if the exposure time is changed by the automatic exposure control, the amount of light source light irradiated during each exposure period of the light receiving element line group corresponding to the raindrop detection image area 214 is maintained at the same amount between the light receiving element lines. If so, the lighting cycle and phase may be set by another method. For example, in this embodiment, the lighting cycle and phase of each light receiving element line corresponding to the raindrop detection image region 214 are set so that one lighting period overlaps each exposure period. As shown, the lighting cycle and phase may be set in the light receiving element line corresponding to the raindrop detection image region 214 so that two lighting periods overlap during the exposure period. That is, in the example shown in FIG. 26, among the light receiving element lines corresponding to the raindrop detection image region 214, the light receiving element lines (line numbers 1, 3 to 5, 7 to 9) in which one lighting period overlaps the exposure period. 11-13) and light receiving element lines (line numbers 2, 6, 10, etc.) in which the two lighting periods overlap in the exposure period are mixed, but the ratio of the lighting period in each exposure period is Since they are the same, the total amount of light emitted from the light source unit 202 during each exposure period is the same.

  In this embodiment, the intensity of the light source light emitted from the light source unit 202 is constant. However, the power of the light source such as each LED of the light source unit 202 is controlled according to a predetermined condition, and the intensity of the light source light is appropriately set. You may perform control to change.

  Further, in the present embodiment, an image including a raindrop detection image area that displays an attachment such as raindrops attached to the windshield 105 and a vehicle detection image area that displays an imaging area in front of the host vehicle is captured in one imaging frame. However, the same applies to the case where an image of the raindrop detection image area is captured using a dedicated imaging frame.

[Modification]
Next, a modification of this embodiment will be described.
In the above-described embodiment, the reflection deflecting prism 230 is used, and the illumination light is transmitted through the outer surface of the windshield at the location where raindrops are attached, and the illumination light is reflected from the outer surface of the windshield at the location where no raindrops are attached. It was the structure which receives light. On the other hand, in this modified example, the illumination light is transmitted through the outer surface of the windshield at the location where the raindrops are not attached, and the illumination light transmitted through the outer surface of the windshield is reflected at the locations where the raindrops are attached. It is the structure which receives light. Specifically, the configuration is as shown in FIG.

  In the example of FIG. 27, the illumination light emitted from the light source unit 202 is directly incident on the inner wall surface of the windshield 105, but may be configured via an optical path changing member such as a mirror. In this case, similarly to the above-described embodiment, the light source unit 202 as the light source device can be mounted on the sensor substrate 207.

  A light ray A in FIG. 27 is a light ray emitted from the light source unit 202 and passing through the windshield 105. When the raindrop 203 does not adhere to the outer wall surface of the windshield 105, the light emitted from the light source unit 202 toward the windshield 105 passes through the windshield 105 like the light ray A and remains on the vehicle 100 as it is. Leak outside.

  A light beam B in FIG. 27 is a light beam that is emitted from the light source unit 202 and is regularly reflected by the inner wall surface of the windshield 105 and is incident on the imaging unit 200. Part of the light traveling from the light source unit 202 toward the windshield 105 is regularly reflected by the inner wall surface of the windshield 105. The polarization component of the specularly reflected light (ray B) is generally dominated by the S polarization component (horizontal polarization component S) that vibrates in a direction perpendicular to the incident plane (direction perpendicular to the paper surface of FIG. 27). Is known to be. The specularly reflected light (light ray B) irradiated from the light source unit 202 and specularly reflected by the inner wall surface of the windshield 105 does not vary depending on the presence or absence of the raindrop 203 adhering to the outer wall surface of the windshield 105. In addition, it becomes disturbance light that lowers the detection accuracy of raindrop detection. In this modification, since the light beam B (horizontal polarization component S) is cut by the polarizing filter layer 225 of the raindrop detection filter unit 220B, it is possible to suppress the raindrop detection accuracy from being lowered by the light beam B.

  A light ray C in FIG. 27 is a light ray that is emitted from the light source unit 202 through the inner wall surface of the windshield 105 and then reflected by raindrops attached to the outer wall surface of the windshield 105 and incident on the imaging unit 200. is there. A part of the light traveling from the light source unit 202 toward the windshield 105 passes through the inner wall surface of the windshield 105, but the transmitted light is more in the vertical polarization component P than in the horizontal polarization component S. When raindrops 203 are attached on the outer wall surface of the windshield 105, the light transmitted through the inner wall surface of the windshield 105 does not leak out like the light ray A, but is reflected multiple times inside the raindrop. Then, the light passes through the windshield 105 again toward the imaging unit 200 and enters the imaging unit 200. At this time, since the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205 of the imaging unit 200 is configured to transmit the emission wavelength (infrared light) of the light source unit 202, the light ray C is red. It passes through the outside light transmission filter region 212. Further, since the polarizing filter layer 225 of the raindrop detection filter unit 220B of the subsequent post-stage filter 220 is formed with the longitudinal direction of the metal wire having a wire grid structure so as to transmit the vertically polarized component P, the light ray C is a polarizing filter. Layer 225 is also transparent. Therefore, the light ray C reaches the image sensor 206, and raindrops are detected based on the amount of light received.

  A light ray D in FIG. 27 is a light ray that passes through the windshield 105 from the outside of the windshield 105 and enters the raindrop detection filter unit 220B of the imaging unit 200. Although this ray D can also be disturbance light at the time of raindrop detection, in this modification, most of the ray D is cut by the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205. Therefore, it is possible to suppress the raindrop detection accuracy from being lowered by the light ray D.

  A light ray E in FIG. 27 is a light ray that passes through the windshield 105 from the outside of the windshield 105 and enters the filter 220A for vehicle detection of the imaging unit 200. The light beam E is cut in the infrared light band by the infrared light cut filter region 211 of the pre-stage filter 210 in the optical filter 205, and only the light in the visible region is imaged. This captured image is used for detection of a head lamp of an oncoming vehicle, a tail lamp of a preceding vehicle, and a white line.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
The illumination area illuminated by the light source light from the light source device such as the light source unit 202 corresponds to at least a part of the rolling shutter type image sensor 206 in which the light receiving elements are two-dimensionally arranged (corresponding to the raindrop detection image area 214). The sensor part) detects the detection object such as raindrops in the illumination area based on the amount of received light source light received by at least a part of the sensor part. A light source control unit such as a light source control unit 102B for periodically turning on and off the light source light emitted from the light source device during a total exposure period in the at least some sensor portions. Exposure time control means such as an exposure time control unit 102C for changing the exposure time according to predetermined exposure conditions, and the light Control means, in accordance with the exposure time varies with the exposure time control means, and changes the on-off cycle of the source light.
In this aspect, in the image sensor of the rolling shutter system, the light source light emitted from the light source device is periodically turned on and off during the total exposure period in the sensor portion that images the illumination area of the light source light. Compared with the case of continuously irradiating light from the light source, it is possible to save power in the light source device.
Further, in this aspect, even if the exposure time is changed by the exposure time control means, the on / off cycle of the light source light is changed accordingly, so that the exposure time between the light receiving element lines in the sensor portion is changed. It becomes possible to make light quantity of light source light the same quantity. Therefore, it is possible to suppress problems such as erroneous detection of the detection target due to the non-uniform light amount of the light source light during each exposure period between the light receiving element lines in the sensor portion.

(Aspect B)
In the aspect A, the light source control means has the same amount of light source light emitted by the light source device during the exposure period of each light receiving element line between the light receiving element lines corresponding to the at least some sensor portions. Thus, the on / off cycle of the light source light is changed.
According to this, even if the exposure time is changed by the exposure time control means, the light amount of the light source light during each exposure period between the light receiving element lines in the sensor portion does not become uneven. Therefore, it is possible to suppress problems such as erroneous detection of the detection target due to the non-uniformity.

(Aspect C)
In the aspect A or B, the light source control unit irradiates the light source device during the exposure period of each light receiving element line corresponding to the at least some sensor portions before and after the change of the exposure time by the exposure time control unit. The on / off cycle of the light source light is changed so that the light amount of the light source light does not change.
According to this, even if the exposure time is changed by the exposure time control means, the light amount of the light source light irradiated during the exposure period of each light receiving element line in the sensor portion does not change and is constant. Therefore, when detecting a detection target object using the time change of the received light amount of the light source light received by the sensor part, it is possible to suppress problems such as erroneous detection of the detection target object.

(Aspect D)
In any one of the aspects A to C, the light source control unit is configured such that the light source control unit has a maximum number of light receiving element lines whose exposure periods overlap each other in a time longer than a single light source light irradiation period in which the light source device emits light source light. The on / off cycle of the light source light is changed so that one light source light irradiation period is assigned every time.
According to this, it is possible to minimize the number of light source light irradiation periods included in the total exposure period of the sensor portion, and power saving of the light source device can be achieved.

(Aspect E)
In any one of the aspects A to D, the image sensor is configured to display an illumination area on a light transmissive member such as the windshield 105 illuminated by the light source light from the light source device as a part of the image sensor. Raindrops adhering to the light-transmitting member by receiving light from a predetermined imaging region and receiving light from a predetermined imaging region by another sensor portion of the image sensor (sensor portion corresponding to the vehicle detection image region 213) Imaging an image including a deposit image area such as a raindrop detection image area 214 that projects an adhesion object such as a raindrop detection image area and an imaging area image area such as a vehicle detection image area 213 that projects the imaging area. Features.
When an imaging frame for obtaining an attachment image is inserted between imaging frames for obtaining an imaging area image, the time during which the imaging area image cannot be acquired becomes longer due to the insertion, and processing that uses temporal changes in the imaging area image ( There is a risk that the accuracy of recognition processing of an object existing in the imaging area will be reduced. According to this aspect, it is possible to obtain an image including the attachment image region and the imaging region image region with a single imaging frame, so that it is not necessary to insert an imaging frame for obtaining the attachment image. . Therefore, it is possible to suppress the time during which the imaging region image cannot be acquired is shortened and the accuracy of the processing using the time change of the imaging region image is reduced.

(Aspect F)
In the aspect E, the light source control means does not overlap with an exposure period of the light receiving element line in the other sensor portion in an exposure period of the light receiving element line of the partial sensor portion adjacent to the other sensor portion. The phase of the on / off cycle of the light source light is set so that a light source light irradiation period in which the light source device emits light source light overlaps an exposure period portion.
According to this, the light source light irradiation period (lighting period) set to overlap the exposure period in the light receiving element line (line number 100) of the part of the sensor part adjacent to the other sensor part is the other The exposure period of the light receiving element lines (line numbers 101 to 101) of the sensor portion of the sensor portion does not overlap. In this case, since any light source light irradiation period in the total exposure period of the part of the sensor portions does not overlap with the exposure periods of the other sensor parts, the light source light is not disturbed by the disturbance light in the imaging region image region. Can be suppressed.

(Aspect G)
The illumination area illuminated by the light source light from the light source device such as the light source unit 202 corresponds to at least a part of the rolling shutter type image sensor 206 in which the light receiving elements are two-dimensionally arranged (corresponding to the raindrop detection image area 214). The sensor part) detects the detection object such as raindrops in the illumination area based on the amount of received light source light received by at least a part of the sensor part. Vehicle-mounted device comprising: an object detection device that performs the control, and a mobile device control unit such as a wiper control unit 106 that controls a predetermined device such as the wiper 107 mounted on the mobile body based on a detection result of the object detection device In a mobile device control system such as a control system, the object detection device according to any one of the aspects A to F is used as the object detection apparatus. Characterized by using.
According to this, in the rolling shutter type image sensor, it is possible to suppress the malfunction such as the erroneous detection of the detection target when the exposure time is changed by the exposure time control means while reducing the power consumption of the light source device. Device control based on the detection result of the object can be performed with high accuracy.

(Aspect H)
An illumination region illuminated by light source light from a light source device such as the light source unit 202 is imaged by at least a part of a sensor part of a rolling shutter type image sensor 206 in which light receiving elements are two-dimensionally arranged, and imaging obtained thereby A computer of an object detection device such as an image analysis unit 102 that detects a detection target such as raindrops in the illumination area based on the amount of received light source light received by the at least some sensor portions using image data. A program for detecting an object for functioning, and a light source control unit such as a light source control unit 102B for periodically turning on and off the light source light emitted from the light source device during a total exposure period in the at least some sensor portions; And as an exposure time control means such as an exposure time control unit 102C for changing the exposure time according to predetermined exposure conditions, It is intended to function serial computer, the light source control means in accordance with the exposure time varies with the exposure time control means, and changes the on-off cycle of the source light.
In this aspect, in the image sensor of the rolling shutter system, the light source light emitted from the light source device is periodically turned on and off during the total exposure period in the sensor portion that images the illumination area of the light source light. Compared with the case of continuously irradiating light from the light source, it is possible to save power in the light source device.
Further, in this aspect, even if the exposure time is changed by the exposure time control means, the on / off cycle of the light source light is changed accordingly, so that the exposure time between the light receiving element lines in the sensor portion is changed. It becomes possible to make light quantity of light source light the same quantity. Therefore, it is possible to suppress problems such as erroneous detection of the detection target due to the non-uniform light amount of the light source light during each exposure period between the light receiving element lines in the sensor portion.
This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 102A Detection processing part 102B Light source control part 102C Exposure time control part 105 Windshield 106 Wiper control unit 107 Wiper 200 Imaging part 202 Light source part 203 Raindrop 205 Optical filter 206 Image sensor 208 Signal processing part 213 Image area 214 for vehicle detection Image area 230 for raindrop detection Reflection deflection prism

JP 2013-113781 A

Claims (8)

The illumination area illuminated by the light source light from the light source device is imaged by at least a part of the sensor in the image sensor of the rolling shutter system in which the light receiving elements are two-dimensionally arranged, and using the captured image data obtained thereby, In an object detection device for detecting a detection target in the illumination area based on a received light amount of light source light received by the at least some sensor parts,
Light source control means for periodically turning on and off the light source light emitted from the light source device during a total exposure period in the at least some sensor parts;
Exposure time control means for changing the exposure time according to predetermined exposure conditions,
The light source control means changes an on / off cycle of the light source light in accordance with an exposure time changed by the exposure time control means. The object detection apparatus according to claim 1,
The light source control means is configured so that the light amount of the light source light emitted by the light source device during the exposure period of each light receiving element line is the same between the light receiving element lines corresponding to the at least some sensor portions. An object detection apparatus characterized by changing an on / off cycle of light. In the object detection device according to claim 1 or 2,
The light source control means changes the light amount of the light source light emitted by the light source device during the exposure period of each light receiving element line corresponding to the at least some sensor portions before and after the change of the exposure time by the exposure time control means. So as not to change the on / off cycle of the light source light. The object detection device according to any one of claims 1 to 3,
The light source control means assigns one light source light irradiation period for each maximum number of light receiving element lines whose exposure periods overlap each other in a time longer than one light source light irradiation period in which the light source device emits light source light. As described above, the on-off cycle of the light source light is changed. In the object detection device according to any one of claims 1 to 4,
The image sensor receives an illumination area on a light transmissive member illuminated by light source light from the light source device by a part of the sensor of the image sensor, and receives light from a predetermined imaging area. By picking up light from another sensor part, an image including a deposit image area that projects the deposit adhered to the light transmissive member as the detection target and an imaging area image area that projects the imaging area is captured. An object detection device characterized by. The object detection apparatus according to claim 5,
The light source control means is arranged in an exposure period portion that does not overlap with an exposure period of a light receiving element line in the other sensor portion of an exposure period in a light receiving element line of the partial sensor portion adjacent to the other sensor portion. An object detection apparatus, wherein a phase of an on / off cycle of the light source light is set so that a light source light irradiation period in which the light source apparatus irradiates the light source light overlaps. The illumination area illuminated by the light source light from the light source device is imaged by at least a part of the sensor in the image sensor of the rolling shutter system in which the light receiving elements are two-dimensionally arranged, and using the captured image data obtained thereby, An object detection device for detecting a detection target in the illumination area based on a received light amount of light source light received by the at least some sensor parts;
In a mobile device control system comprising mobile device control means for controlling a predetermined device mounted on the mobile based on the detection result of the object detection device,
A mobile device control system using the object detection device according to any one of claims 1 to 6 as the object detection device. The illumination area illuminated by the light source light from the light source device is imaged by at least a part of the sensor in the image sensor of the rolling shutter system in which the light receiving elements are two-dimensionally arranged, and using the captured image data obtained thereby, An object detection program for causing a computer of an object detection device to detect a detection target in the illumination area based on a received light amount of light source light received by at least a part of the sensor part,
As light source control means for periodically turning on and off the light source light emitted from the light source device during the total exposure period in the at least some sensor parts, and as exposure time control means for changing the exposure time according to predetermined exposure conditions, Functioning the computer,
The light source control means changes the on / off cycle of the light source light according to the exposure time changed by the exposure time control means.

Images