JP2016146583A - Imaging device, mobile apparatus control system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain proper images for both imaging area image area and attachment image area, when capturing an image including the attachment image area and imaging area image area.SOLUTION: An imaging device for capturing an image including an attachment image area reflecting an attachment adhering to a light-permeable member and a capturing region image area reflecting the capturing region, by receiving the light source light irradiated from a light source device 202 toward the light-permeable member by means of a part of an image sensor 206, and receiving the light from a predetermined capturing region by means of other part of the image sensor, includes light source control means 102B executing light source light control for changing the amount of the light source light, received by a part of the image sensor during the exposure period with an exposure amount after change, according to the exposure amount changed by exposure amount control means 102C.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus, a mobile device control system, and a program.

  Conventionally, an imaging area image for sensing for recognizing an object existing in an imaging area in front of the host vehicle and an deposit image for detecting deposits such as raindrops attached to the windshield of the host vehicle An imaging apparatus that captures an image with a single image sensor is known.

  For example, Patent Literature 1 discloses an imaging frame (acquisition frame) for acquiring the image for detecting the attached matter every time an imaging frame (sensing frame) for acquiring the sensing image is captured 30 times continuously. There is disclosed an imaging apparatus that repeats the imaging of the adhering matter detection frame) once. In this imaging device, when imaging an adhering matter detection frame, the light source light is emitted from the light source device toward the windshield, and the reflected light is received so that the adhering portion and the adhering matter are An adhering matter detection frame having a contrast with a non-adhering portion is obtained. This imaging device captures an image of a sensing frame while automatically adjusting the exposure amount by automatic exposure control according to the brightness of the imaging region. On the other hand, the adhering matter detection frame is imaged with a fixed exposure amount because the light amount of the light source emitted from the light source device is constant.

  However, in an imaging apparatus that captures an attachment detection frame between sensing frames that are continuously imaged, time is left between sensing frames before and after the attachment detection frame. For this reason, there is a time during which no sensing image can be acquired between the time point when the previous sensing frame is captured and the time point when the subsequent sensing frame is captured, causing problems such as a reduction in object recognition accuracy.

  On the other hand, as a method for obtaining a sensing image and a deposit image with a single image sensor, the reflected light of the light source emitted from the light source device toward the windshield is received by a part of the image sensor, A method is conceivable in which light from the imaging area in front of the vehicle is received by another part of the image sensor. According to this method, an image area for adhering matter detection (adhering matter image area) that shows an adhering matter adhering to the windshield and an image area for sensing that shows an imaging area in front of the host vehicle in a single imaging frame. An image including an imaging region (image region) can be obtained. In this case, since the imaging frame (adherent detection frame) for obtaining the adhering image is unnecessary, the above-described problems do not occur.

  In order to obtain an image area for sensing suitable for sensing (for example, an image area with high contrast), it is desirable to perform automatic exposure control in which the exposure amount is changed according to the brightness of the imaging area. However, if the exposure amount is changed, the received light amount of the light source received by the image sensor changes, so that the brightness, contrast, etc. Therefore, a deposit image having a different value may be obtained, and a defect such as erroneous detection of the deposit may occur.

  In order to solve the above-described problems, the present invention receives light source light emitted from a light source device toward a light transmissive member by a part of an image sensor and transmits light from a predetermined imaging region to the other of the image sensor. In an imaging apparatus that captures an image including an adhered image area that reflects an adhered object that adheres to the light transmissive member and an imaging area image area that reflects the imaging area by receiving light at the portion in accordance with predetermined exposure conditions Exposure amount control means for changing the exposure amount, and the amount of light source light received by the part of the image sensor during the exposure period with the changed exposure amount according to the exposure amount changed by the exposure amount control means And light source light control means for executing light source light control for changing the light source.

  According to the present invention, even when an image including an attachment image region and an imaging region image region is captured, an appropriate imaging region image region is obtained by changing the exposure amount according to the state of the imaging region, and the exposure amount An excellent effect is obtained that it is possible to obtain an appropriate deposit image region with little change in luminance, contrast, and the like due to the change.

It is a mimetic diagram showing a schematic structure of an in-vehicle device control system in an embodiment. It is a schematic diagram which shows schematic structure of the deposit | attachment detection apparatus provided with the imaging unit in the same vehicle equipment control system. It is explanatory drawing for demonstrating the optical system of the imaging unit. It is the perspective view which showed typically schematic structure of the imaging unit. (A) is a side view which shows the imaging unit of the state attached to the vehicle whose inclination angle of a windshield with respect to a horizontal surface is 22 degrees. (B) is explanatory drawing which shows the optical system of the imaging unit in case the raindrop has not adhered in the state of the figure (a). (C) is explanatory drawing which shows the optical system of an imaging unit in case the raindrop has adhered in the state of the figure (a). It is a perspective view which shows the reflective deflection | deviation prism of the imaging unit. It is a graph which shows the filter characteristic of the cut filter applicable to the picked-up image data for raindrop detection. It is a graph which shows the filter characteristic of the band pass filter applicable to the picked-up image data for raindrop detection. It is a front view of the front | former stage filter provided in the optical filter of the imaging part. It is explanatory drawing which shows the image example of the captured image data of the imaging part. It is a model enlarged view when the optical filter and image sensor of the imaging part are seen from a direction orthogonal to the light transmission direction. It is explanatory drawing which shows the area | region division pattern of the polarizing filter layer and spectral filter layer of the same optical filter. It is sectional drawing which shows typically the layer structure of the optical filter in embodiment. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the filter part for vehicle detection of the same optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the filter part for vehicle detection of the optical filter and image sensor which were cut | disconnected by code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the vehicle detection filter part and image sensor of the optical filter which were cut | disconnected by the code | symbol BB shown in FIG. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the filter part for raindrop detection of the optical filter, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the filter part for raindrop detection and the image sensor of the optical filter cut | disconnected by code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the filter part for raindrop detection of the optical filter cut | disconnected by the code | symbol BB shown in FIG. 16, and an image sensor. It is a flowchart which shows the flow of the vehicle detection process in embodiment. It is explanatory drawing of the raindrop detection process in embodiment. It is explanatory drawing which shows the relationship between the conventional imaging frame and raindrop detection. It is explanatory drawing which shows the relationship between the imaging frame and raindrop detection in embodiment. It is explanatory drawing which shows the example by which an exposure period is changed by automatic exposure control within the light irradiation period from a light source part. It is explanatory drawing which shows the example by which an exposure period is changed by automatic exposure control outside the light irradiation period from a light source part. It is explanatory drawing which shows the relationship between the exposure time and light source power in embodiment. It is explanatory drawing about the various light rays in connection with the raindrop detection in a modification.

  Hereinafter, an embodiment in which an imaging device according to the present invention is used in an in-vehicle device control system as a mobile device control system that controls a target device mounted on a vehicle such as an automobile that is a mobile body will be described.

FIG. 1 is a schematic diagram illustrating a schematic configuration of an in-vehicle device control system according to the present embodiment.
This in-vehicle device control system uses a picked-up image data obtained by picking up an area around the own vehicle (especially a forward area in the traveling direction) by an image pickup unit mounted on the own vehicle 100 such as an automobile, and distributes the headlamp light. Control, wiper drive control, and other in-vehicle devices are controlled.

  The imaging unit provided in the in-vehicle device control system of the present embodiment is provided in the imaging unit 101, and images an area in the traveling direction of the traveling host vehicle 100 as an imaging region. It is installed near the room mirror of the windshield 105. The captured image data captured by the imaging unit of the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 includes a CPU, a RAM, and the like as processing execution means, analyzes captured image data transmitted from the imaging unit, and detects the position of another vehicle (in front of the host vehicle 100) in the captured image data ( (Direction and distance), detection of raindrops and foreign matter adhering to the windshield 105, detection of a detection target such as a white line (partition line) on the road surface existing in the imaging region, To do. In the detection of other vehicles, a preceding vehicle traveling in the same traveling direction as the own vehicle 100 is detected by identifying the tail lamp of the other vehicle, and traveling in the opposite direction to the own vehicle 100 by identifying the headlamp of the other vehicle. An oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. For example, the headlamp control unit 103 generates a control signal for controlling the headlamp 104 that is an in-vehicle device of the host vehicle 100 from the position data of the other vehicle calculated by the image analysis unit 102. Specifically, for example, while avoiding that the strong light of the headlamp of the own vehicle 100 is incident on the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. The switching of the high beam and the low beam of the headlamp 104 is controlled, and partial shading control of the headlamp 104 is performed so that the driver's visibility can be secured.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls the wiper 107 to remove deposits such as raindrops and foreign matters attached to the windshield 105 of the host vehicle 100. The wiper control unit 106 receives the attached matter detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated to ensure the visibility of the driver of the host vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle travel control unit 108 warns the driver of the host vehicle 100 when the host vehicle 100 is out of the lane area defined by the white line. Notification is performed, and driving support control such as controlling the steering wheel and brake of the host vehicle is performed. In addition, the vehicle travel control unit 108 notifies the driver of the host vehicle 100 of a warning when the distance from the preceding vehicle is close based on the position data of the other vehicle detected by the image analysis unit 102, Carry out driving support control such as controlling the steering wheel and brakes of the vehicle.

FIG. 2 is an explanatory diagram illustrating a schematic configuration of the attached matter detection apparatus including the imaging unit 200 provided in the imaging unit 101.
The imaging unit 200 mainly includes an imaging lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 in which light receiving elements are two-dimensionally arranged, and an analog electrical signal (image sensor) output from the sensor substrate 207. A signal processing unit 208 that generates and outputs captured image data obtained by converting a received light amount received by each light receiving element on the digital image 206 into a digital electric signal, and an internal memory 209. In the present embodiment, the light source unit 202 as a light source device is mounted on the sensor substrate 207. The light source unit 202 is disposed on the inner wall surface (one surface) side of the windshield 105, and an example is a case where the attached matter (hereinafter, the detection target is mainly raindrops) attached to the outer wall surface (the other surface). This is for detecting.

  In the present embodiment, the imaging unit 101 is arranged so that the optical axis of the imaging lens 204 coincides with the horizontal direction. However, the present invention is not limited to this, and the horizontal direction (X direction in FIG. 2) is used as a reference. It may be an example directed to a specific direction. For example, the imaging lens 204 includes a plurality of lenses and has a focal point set farther than the position of the windshield 105. The focal position of the imaging lens 204 can be set, for example, between infinity or infinity and the windshield 105.

  The image sensor 206 includes a plurality of two-dimensionally arranged light receiving elements that receive light transmitted through a cover glass that protects the sensor surface, and has a function of photoelectrically converting incident light for each light receiving element (imaging pixel). In the drawings and the like to be described later, each pixel of the image sensor 206 is illustrated in a simplified manner. However, the image sensor 206 is actually composed of about several hundred thousand pixels arranged two-dimensionally. As the image sensor 206, for example, a CCD (Charge Coupled Device) that reads all the imaging pixels simultaneously by exposing all the imaging pixels simultaneously (global shutter), or a signal of each imaging pixel exposed by line exposure (rolling shutter). Is an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) or the like, and a photodiode can be used as the light receiving element thereof.

  The signal processing unit 208 generates captured image data obtained by converting an analog electrical signal (the amount of light incident on each light receiving element of the image sensor 206) that is photoelectrically converted by the image sensor 206 and output from the sensor substrate 207 into a digital electrical signal. Output function. The signal processing unit 208 is electrically connected to the image analysis unit 102. When an electric signal (analog signal) is input from the image sensor 206 via the sensor substrate 207, the signal processing unit 208 determines the brightness (luminance) of each imaging pixel on the image sensor 206 from the input electric signal. A digital signal (captured image data) is generated. Then, the captured image data is output to the subsequent image analysis unit 102 together with the horizontal / vertical synchronization signal of the image.

  The image sensor 206 of the present embodiment includes a plurality of optical blacks (light receiving elements configured so that no light is incident) for performing dark correction (black level control: BLC) outside the effective imaging pixel region. Yes. The optical black output signal is sent to the signal processing unit 208 and used to measure the amount of dark current noise. Specifically, the signal processing unit 208 functions as a zero reference setting unit. For example, a signal level obtained by adding a predetermined margin to the average level of the output signal from the optical black is used as a zero reference (pixel value zero) of the captured image data. Set to. Thereby, among the analog electric signals output from the sensor substrate 207, those below the signal level corresponding to the zero reference are set to zero values.

  Further, an analog electric signal exceeding a signal level corresponding to the zero reference is converted into a digital electric signal having a digital value obtained by dividing the signal level of the zero reference and the maximum signal level by a predetermined number of gradations. Accordingly, it is possible to avoid a situation in which a light receiving element that does not receive light at all is erroneously detected as receiving light due to a dark current flowing through the light receiving element of the image sensor 206.

  Further, the image analysis unit 102 has a function of controlling the imaging operation of the imaging unit 101 and a function of analyzing captured image data transmitted from the imaging unit 101. The image analysis unit 102 detects an object to be imaged by the image sensor 206 (an object such as another vehicle existing in the front area of the host vehicle) from the captured image data transmitted from the imaging unit 101 by the exposure amount control unit 102C as an exposure amount control unit. ) And an optimum exposure amount (exposure time in this embodiment) for each imaging target of the image sensor 206 is set. Further, the image analysis unit 102 has a function of adjusting the amount of light emitted from the light source unit 202 by the light amount adjusting unit 102B as the light source light control unit in conjunction with the exposure amount adjustment. In addition, the image analysis unit 102 has a function of detecting information related to a road surface condition, a road sign, and the like from captured image data transmitted from the imaging unit 101. Further, the image analysis unit 102 has a function of detecting the state of the windshield 105 (attachment of raindrops, freezing, cloudiness, etc.) by the detection processing unit 102A. Further, the image analysis unit 102 has a function of calculating the position, direction, distance, and the like of another vehicle existing ahead of the host vehicle from the captured image data transmitted from the imaging unit 101.

FIG. 3 is an explanatory diagram for explaining an optical system of the imaging unit 101 in the present embodiment.
The light source unit 202 of the present embodiment irradiates light source light (illumination light) for detecting an adhering matter (raindrop, freezing, cloudiness, etc.) attached on the windshield 105. The light source unit 202 of the present embodiment is configured to include a plurality of LEDs as the light source. By providing a plurality of light sources in this manner, the detection area for detecting the deposit on the windshield 105 is broadened compared to the case where there is a single light source, and the detection accuracy of the deposit on the windshield 105 is improved. improves.

  In this embodiment, since the LEDs are mounted on the same sensor substrate 207 as the image sensor 206, the number of substrates can be reduced as compared with the case where these are mounted on separate substrates, and low cost can be realized. Further, the LED is arranged in one or more rows along the Y direction in FIG. 3 so that, as will be described later, the windshield is projected below the image area where the image of the vehicle front area is projected. It is possible to make the illumination for capturing an image uniform. In the present embodiment, the light source unit 202 is mounted on the same sensor substrate 207 as the image sensor 206, but may be mounted on a different substrate from the image sensor 206.

  The light source unit 202 is disposed on the sensor substrate 207 so that the optical axis direction of the light emitted from the light source unit 202 and the optical axis direction of the imaging lens 204 have a predetermined angle in advance. Further, the light source unit 202 is arranged on the windshield 105 illuminated by the light emitted from the light source unit 202 so that the illumination range is within the field angle range (viewing angle range) of the imaging lens 204. ing. As the emission wavelength of the light source unit 202, it is preferable to avoid visible light so as not to dazzle the driver or pedestrian of the oncoming vehicle. For example, the wavelength is longer than the visible light, and the light receiving sensitivity of the image sensor 206 extends. A range (for example, an infrared light wavelength range of about 800 to 1000 nm) is used. Drive control such as the light emission timing of the light source unit 202 is performed through the image analysis unit 102 in conjunction with the acquisition of the image signal from the signal processing unit 208.

  As shown in FIG. 3, the imaging unit 101 of the present embodiment is provided with a reflection deflection prism 230 as an optical member that includes a reflection surface 231 that reflects light from the light source unit 202 and guides it to the windshield 105. Yes. The reflection deflection prism 230 is disposed so that one surface thereof is in close contact with the inner wall surface of the windshield 105 in order to appropriately guide the light from the light source unit 202 into the windshield 105. Specifically, even if the incident angle of light from the light source unit 202 incident on the reflection deflection prism 230 changes within a predetermined range, the light is irradiated from the light source unit 202 and regularly reflected by the reflection surface 231 of the reflection deflection prism 230. Of the light, regular reflection light regularly reflected at a non-attached portion where no raindrop (detection target) on the outer wall surface of the windshield 105 is attached is received by the image sensor 206 so as to maintain the relationship. A reflection deflection prism 230 is attached to the inner wall surface of the windshield 105.

  When the reflection deflecting prism 230 is attached to the inner wall surface of the windshield 105, it is preferable to improve the adhesion by interposing a filler such as a gel made of a translucent material or a sealing material therebetween. Thereby, it is possible to prevent an air layer, bubbles, or the like from intervening between the reflection deflection prism 230 and the windshield 105, and to prevent fogging between them. The refractive index of the filler is preferably an intermediate refractive index between the reflective deflection prism 230 and the windshield 105. This is because the Fresnel reflection loss between the filler and the reflection deflecting prism 230 and between the filler and the windshield 105 can be reduced. Fresnel reflection here refers to reflection that occurs between materials having different refractive indexes.

  As shown in FIG. 3, the reflection deflection prism 230 specularly reflects incident light from the light source unit 202 once on the reflection surface 231, and folds it back toward the inner wall surface of the windshield 105. The folded light is configured to have an incident angle θ (θ ≧ about 42 °) with respect to the outer wall surface of the windshield 105. The appropriate incident angle θ is a critical angle that causes total reflection on the outer wall surface inside the windshield 105 due to a difference in refractive index between air and the outer wall surface of the windshield 105. Therefore, in the present embodiment, when no deposits such as raindrops adhere to the outer wall surface of the windshield 105, the light reflected by the reflection surface of the reflection deflection prism 230 is transmitted through the outer wall surface of the windshield 105. Without being reflected all.

  On the other hand, if deposits such as raindrops (refractive index 1.38) different from air (refractive index 1) adhere on the outer wall surface of the windshield 105, this total reflection condition breaks down, and in places where raindrops adhere Light passes through the outer wall surface of the windshield 105. Therefore, for the non-attached portion where no raindrop is attached to the outer wall surface of the windshield 105, the reflected light is received by the image sensor 206 to become a high-luminance image portion, while the attached portion where the raindrop is attached. Since the amount of reflected light is reduced and the amount of light received by the image sensor 206 is reduced, an image portion with low luminance is obtained. Therefore, on the captured image, the contrast between the raindrop-attached portion and the non-raindrop-attached portion can be obtained.

FIG. 4 is a perspective view schematically showing a schematic configuration of the imaging unit 101 of the present embodiment.
The imaging unit 101 of the present embodiment is mounted with a first module 101A as a first support member that fixedly supports the reflection deflection prism 230 and is fixedly disposed on the inner wall surface of the windshield 105, an image sensor 206, and an LED 211. The sensor board 207, the light guide 215, and the second module 101B as a second support member that fixes and supports the imaging lens 204 are configured.

  These modules 101 </ b> A and 101 </ b> B are rotatably connected by a rotation connecting mechanism 240. The rotation coupling mechanism 240 has a rotation shaft 241 extending in a direction orthogonal to both the inclination direction and the vertical direction of the windshield 105 (the front-rear direction in FIG. 3). The one module 101A and the second module 101B can be rotated relative to each other. Such a rotatable configuration is aimed at the imaging direction of the imaging unit 200 of the second module 101B even when the first module 101A is fixedly arranged with respect to the windshield 105 having different inclination angles. This is because it can be directed in a specific direction (horizontal direction in the present embodiment).

  FIG. 5A is a side view showing the imaging unit 101 in a state of being attached to a vehicle in which the windshield 105 has an inclination angle θg of 22 ° with respect to a horizontal plane. FIG. 5B is an explanatory diagram showing the optical system of the imaging unit 101 when no raindrops are attached in the state of FIG. FIG. 5C is an explanatory diagram showing the optical system of the imaging unit 101 when raindrops are attached in the state of FIG.

  The light L1 from the light source unit 202 is specularly reflected by the reflection surface 231 of the reflection deflection prism 230, and the reflected light L2 is transmitted through the inner wall surface of the windshield 105. When raindrops are not attached to the outer wall surface of the windshield 105, the reflected light L2 is totally reflected on the outer wall surface, and this reflected light L3 is transmitted through the inner wall surface of the windshield 105 toward the imaging lens 204. Goes on. On the other hand, when raindrops 203 are attached to the outer wall surface of the windshield 105, the reflected light L2 specularly reflected by the reflecting surface 231 of the reflecting deflection prism 230 is transmitted through the outer wall surface. In such a system, when the angle θg of the windshield 105 changes, the second module 101B is fixed to the inner wall surface of the windshield 105 while maintaining the attitude of the second module 101B (while the imaging direction is fixed in the horizontal direction). The posture of the module 101B is changed, and the reflection deflection prism 230 is rotated around the Y axis in the drawing together with the windshield 105.

  Here, the arrangement relationship between the reflecting surface 231 of the reflecting deflection prism 230 and the outer wall surface of the windshield 105 in the present embodiment is always the entire relationship on the outer wall surface of the windshield 105 within the rotation adjustment range of the rotary coupling mechanism 240. The reflected light L3 is received by the light receiving area of the image sensor 206 for detecting the windshield state change (hereinafter referred to as “attached substance detecting light receiving area”). Therefore, even if the angle θg of the windshield 105 changes, the total reflected light L3 on the outer wall surface of the windshield 105 is received by the adhering matter detection light receiving area of the image sensor 206, and appropriate raindrop detection can be realized.

  In particular, the arrangement relationship in the present embodiment is such that the corner cube principle is substantially established within the rotation adjustment range of the rotary coupling mechanism 240. Therefore, even if the angle θg of the windshield 105 changes, the angle θ formed between the optical axis direction of the total reflected light L3 on the outer wall surface of the windshield 105 and the horizontal plane is substantially constant. Therefore, it is possible to suppress the fluctuation of the portion through which the optical axis of the total reflected light L3 passes on the outer wall surface of the windshield 105 in the attached light detection region of the image sensor 206, thereby realizing more appropriate raindrop detection. .

  The corner cube principle is established if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is perpendicular to each other. If the cube principle is established, the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not limited to a perpendicular relationship. Even if the arrangement relationship between the reflection surface 231 of the reflection deflection prism 230 and the outer wall surface of the windshield 105 is not perpendicular to each other, the angle of the other surface (incident surface or emission surface) of the reflection deflection prism 230 is adjusted. Thus, even if the inclination angle θg of the windshield 105 changes, the angle θ of the optical axis of the total reflected light L3 toward the imaging lens can be kept substantially constant.

FIG. 6 is a perspective view showing the reflective deflection prism 230 of the present embodiment.
The reflection deflection prism 230 includes an incident surface 233 on which light from the light source unit 202 is incident, a reflecting surface 231 that reflects the light L1 incident from the incident surface 233, and an adhesion surface 232 that is in close contact with the inner wall surface of the windshield 105. And an emission surface 234 that emits the reflected light L3 reflected by the outer wall surface of the windshield 105 toward the imaging unit 200. In the present embodiment, the incident surface 233 and the exit surface 234 are configured to be parallel to each other, but they may be non-parallel surfaces.

  The material of the reflection deflection prism 230 may be any material that transmits at least light from the light source unit 202, and glass, plastic, or the like can be used. Since the light from the light source unit 202 of the present embodiment is infrared light, a black material that absorbs visible light may be used as the material of the reflective deflection prism 230. By using a material that absorbs visible light, light (such as visible light from outside the vehicle) other than the light from the LED (infrared light) can be prevented from entering the reflective deflection prism 230.

  Further, the reflection deflection prism 230 is formed so that the total reflection condition for totally reflecting the light from the light source unit 202 by the reflection surface 231 is satisfied within the rotation adjustment range of the rotary coupling mechanism 240. In addition, when it is difficult to satisfy the condition of total reflection on the reflection surface 231 within the rotation adjustment range of the rotation coupling mechanism 240, a film such as aluminum is vapor-deposited on the reflection surface 231 of the reflection deflection prism 230. A reflection mirror may be formed.

  In the present embodiment, the reflecting surface 231 is a flat surface, but the reflecting surface may be concave. By using such a concave reflection surface 225, the diffused light beam incident on the reflection surface 225 can be collimated. This makes it possible to suppress a decrease in illuminance on the windshield 105.

  Here, when imaging the infrared wavelength light from the light source unit 202 reflected by the outer wall surface of the windshield 105 with the imaging unit 200, the image sensor 206 of the imaging unit 200 includes the infrared wavelength light from the light source unit 202. A large amount of disturbance light including infrared wavelength light such as sunlight is also received. Therefore, in order to distinguish the infrared wavelength light from the light source unit 202 from such a large amount of disturbance light, the light emission amount of the light source unit 202 needs to be sufficiently larger than the disturbance light. It is often difficult to use the light source unit 202 with a large light emission amount.

  Therefore, in the present embodiment, for example, a cut filter that cuts light having a wavelength shorter than the light emission wavelength of the light source unit 202 as shown in FIG. 7, or a transmittance peak as shown in FIG. The image sensor 206 is configured to receive light from the light source unit 202 through a bandpass filter that substantially matches the emission wavelength of the light source unit 202. Accordingly, light other than the light emission wavelength of the light source unit 202 can be removed and received, so that the amount of light from the light source unit 202 received by the image sensor 206 is relatively large with respect to disturbance light. As a result, light from the light source unit 202 can be distinguished from disturbance light even if the light source unit 202 does not have a large light emission amount.

  However, in the present embodiment, not only the raindrop 203 on the windshield 105 is detected from the captured image data, but also the preceding vehicle and the oncoming vehicle and the white line are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source unit 202 is removed from the entire captured image, the image sensor 206 transmits light in the wavelength band necessary for detection of the preceding vehicle and the oncoming vehicle and detection of the white line. The light cannot be received, and this hinders detection. Therefore, in the present embodiment, the image area of the captured image data, the raindrop detection image area for detecting the raindrop 203 on the windshield 105, the vehicle for detecting the preceding vehicle and the oncoming vehicle and the white line are detected. A filter that removes a wavelength band other than the infrared wavelength light irradiated by the light source unit 202 only on a portion corresponding to the raindrop detection image region is arranged in the optical filter 205.

FIG. 9 is a front view of the front-stage filter 210 provided in the optical filter 205.
FIG. 10 is an explanatory diagram illustrating an example of captured image data.
The optical filter 205 of the present embodiment has a structure in which a front-stage filter 210 and a rear-stage filter 220 are overlapped in the light transmission direction. As shown in FIG. 9, the front-stage filter 210 includes an infrared light cut filter region 211 and a raindrop detection image region 214 that are arranged at locations corresponding to the upper two-third captured image that is the vehicle detection image region 213. The region is divided into an infrared light transmission filter region 212 arranged at a position corresponding to a lower one third of a captured image. For the infrared light transmission filter region 212, the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. 8 is used.

  The headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line image are often present mainly from the center to the upper part of the captured image, and there is usually an image of the nearest road surface in front of the host vehicle at the lower part of the captured image. It is. Therefore, the information necessary for identifying the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line is concentrated on the upper portion of the captured image, and the information on the lower portion of the captured image is not so important in the identification. Therefore, when both oncoming vehicle and preceding vehicle or white line detection and raindrop detection are performed simultaneously from a single captured image data, the lower portion of the captured image is displayed in the raindrop detection image area 214 as shown in FIG. It is preferable that the upper portion of the remaining captured image is the vehicle detection image region 213 and the pre-filter 210 is divided into regions corresponding to this.

  In this embodiment, the raindrop detection image area 214 is provided below the vehicle detection image area 213 in the captured image. However, the raindrop detection image area 214 is provided above the vehicle detection image area 213. Alternatively, the raindrop detection image area 214 may be provided both above and below the vehicle detection image area 213.

  When the imaging direction of the imaging unit 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the host vehicle or the tail lamp of the preceding vehicle becomes disturbance light, which is included in the captured image data, which may cause misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in the present embodiment, the cut filter shown in FIG. 7 and the bandpass filter shown in FIG. Disturbance light such as the tail lamp of the vehicle is removed. Therefore, the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line identification accuracy are improved.

  In this embodiment, due to the characteristics of the imaging lens 204, the scene in the imaging area and the image on the image sensor 206 are upside down. Therefore, when the lower portion of the captured image is used as the raindrop detection image region 214, the upper portion of the upstream filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 7 or the bandpass filter shown in FIG. .

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light compared to the headlamp of the oncoming vehicle, and disturbance such as street lights. Since there is a lot of light, it is difficult to detect the tail lamp with high accuracy only from mere luminance data. Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. Therefore, in this embodiment, as will be described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the tail lamp color) that matches the color of the tail lamp is disposed in the subsequent filter 220 of the optical filter 205, and the red color. The amount of light received can be detected.

  However, since each light receiving element constituting the image sensor 206 of the present embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 206 receives light including the infrared wavelength band. The captured image becomes reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in the present embodiment, the portion corresponding to the vehicle detection image region 213 in the upstream filter 210 of the optical filter 205 is the infrared light cut filter region 211. Thereby, since the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, the identification accuracy of the tail lamp is improved.

FIG. 11 is a schematic enlarged view when the optical filter 205 and the image sensor 206 are viewed from a direction orthogonal to the light transmission direction.
As described above, the image sensor 206 is an image sensor using a CCD, a CMOS, or the like, and a photodiode 206A is used as a light receiving element thereof. The photodiodes 206A are two-dimensionally arranged for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiode 206A. The image sensor 206 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 207.

  An optical filter 205 is disposed close to the surface of the image sensor 206 on the micro lens 206B side. As shown in FIG. 11, the post filter 220 of the optical filter 205 has a laminated structure in which a polarizing filter layer 222 and a spectral filter layer 223 are sequentially formed on a transparent filter substrate 221. Each of the polarizing filter layer 222 and the spectral filter layer 223 is divided into regions corresponding to one photodiode 206A on the image sensor 206.

  A configuration may be adopted in which there is a gap between the optical filter 205 and the image sensor 206, but the configuration in which the optical filter 205 is in close contact with the image sensor 206 is different from the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205. It becomes easy to make the boundary of each area | region and the boundary between photodiode 206A on the image sensor 206 correspond. The optical filter 205 and the image sensor 206 may be bonded with, for example, a UV adhesive, or UV bonding or thermocompression bonding is performed on the four side areas outside the effective pixel while being supported by the spacer outside the effective pixel range used for imaging. Also good.

FIG. 12 is an explanatory diagram showing a region division pattern of the polarizing filter layer 222 and the spectral filter layer 223 of the optical filter 205 according to the present embodiment.
In the polarizing filter layer 222 and the spectral filter layer 223, two types of regions, a first region and a second region, are arranged corresponding to one photodiode 206A on the image sensor 206, respectively. As a result, the amount of light received by each photodiode 206A on the image sensor 206 is determined as polarization information, spectral information, or the like depending on the types of regions of the polarizing filter layer 222 and the spectral filter layer 223 through which the received light is transmitted. Can be acquired.

  In the present embodiment, the image sensor 206 is described on the assumption of an image sensor for monochrome images, but the image sensor 206 may be configured with a color image sensor. In the case of the color image sensor, the light transmission characteristics of the polarizing filter layer 222 and the spectral filter layer 223 may be adjusted according to the characteristics of the color filter attached to each image pickup pixel of the color image sensor.

Here, an example of the optical filter 205 in the present embodiment will be described.
FIG. 13 is a cross-sectional view schematically showing the layer configuration of the optical filter 205 in the present embodiment.
The post filter 220 of the optical filter 205 in the present embodiment includes a vehicle detection filter unit 220A corresponding to the vehicle detection image region 213 and a raindrop detection filter unit 220B corresponding to the raindrop detection image region 214, and the layers thereof. The configuration is different. Specifically, the vehicle detection filter unit 220 </ b> A includes the spectral filter layer 223, while the raindrop detection filter unit 220 </ b> B does not include the spectral filter layer 223. Further, the configuration of the polarizing filter layers 222 and 225 is different between the vehicle detection filter unit 220A and the raindrop detection filter unit 220B.

FIG. 14 shows the contents of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the vehicle detection filter 220A of the optical filter 205 in this embodiment. It is explanatory drawing which shows.
FIG. 15A is a cross-sectional view schematically showing the vehicle detection filter unit 220A and the image sensor 206 of the optical filter 205 cut along the line AA shown in FIG.
FIG. 15B is a cross-sectional view schematically showing the vehicle detection filter unit 220A and the image sensor 206 of the optical filter 205 cut along the line BB shown in FIG.

  As shown in FIGS. 15A and 15B, the vehicle detection filter unit 220 </ b> A of the optical filter 205 in the present embodiment is formed on a polarizing filter layer 222 on a transparent filter substrate 221, and then on it. The spectral filter layer 223 is formed to have a laminated structure. The polarizing filter layer 222 has a wire grid structure, and the upper surface in the stacking direction (the lower side surface in FIG. 15) is an uneven surface. If an attempt is made to form the spectral filter layer 223 directly on such an uneven surface, the spectral filter layer 223 is formed along the uneven surface, resulting in uneven thickness of the spectral filter layer 223 and obtaining the original spectral performance. There may not be. Therefore, in the optical filter 205 of the present embodiment, the upper surface side in the stacking direction of the polarizing filter layer 222 is filled with a filler and planarized, and then the spectral filter layer 223 is formed thereon.

  As the filler, any material that does not hinder the function of the polarizing filter layer 222 whose uneven surface is flattened by the filler may be used. In this embodiment, a material having no polarization function is used. In addition, as the planarization treatment with the filler, for example, a method of applying the filler by a spin-on-glass method can be suitably employed, but the present invention is not limited to this.

In the present embodiment, the first region of the polarization filter layer 222 is a vertical polarization region that selectively transmits only the vertical polarization component that vibrates in parallel with the column (vertical direction) of the imaging pixels of the image sensor 206, and the polarization filter The second region of the layer 222 is a horizontal polarization region that selects and transmits only the horizontal polarization component that vibrates parallel to the row (horizontal direction) of the imaging pixels of the image sensor 206.
The first region of the spectral filter layer 223 is a red spectral region that selectively transmits only light in the red wavelength band (specific wavelength band) included in the usable wavelength band that can be transmitted through the polarizing filter layer 222. The second region of the filter layer 223 is a non-spectral region that transmits light without performing wavelength selection. In the present embodiment, as indicated by a one-dot chain line in FIG. 14, imaged image data is obtained by a total of four imaging pixels (4 imaging pixels of reference symbols a, b, e, and f) adjacent to each other in two vertical and two horizontal directions. 1 image pixel is formed.

In the imaging pixel a shown in FIG. 14, light that has passed through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel a receives light P / R in the red wavelength band (indicated by symbol R in FIG. 14) of the vertically polarized component (indicated by symbol P in FIG. 14).
In addition, in the imaging pixel b illustrated in FIG. 14, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel b receives light P / C of non-spectral (indicated by symbol C in FIG. 14) in the vertical polarization component P.
In addition, in the imaging pixel e illustrated in FIG. 14, light that has passed through the horizontal polarization region (second region) of the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel e receives the non-spectral C light S / C in the horizontal polarization component (indicated by symbol S in FIG. 14).
In the imaging pixel f shown in FIG. 14, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel f receives the light P / R in the red wavelength band R in the vertical polarization component P, similarly to the imaging pixel a.

  With the above configuration, according to the present embodiment, one image pixel for the vertical polarization component image of red light is obtained from the output signals of the imaging pixel a and the imaging pixel f, and the non-spectral vertical is obtained from the output signal of the imaging pixel b. One image pixel for the polarization component image is obtained, and one image pixel for the non-spectral horizontal polarization component image is obtained from the output signal of the imaging pixel e. Therefore, according to the present embodiment, three types of captured image data, that is, a red light vertical polarization component image, a non-spectral vertical polarization component image, and a non-spectral horizontal polarization component image, can be obtained by a single imaging operation. .

In these captured image data, the number of image pixels is smaller than the number of captured pixels, but a generally known image interpolation technique may be used to obtain a higher resolution image. For example, when trying to obtain a vertical polarization component image of red light having a higher resolution, the vertical polarization of the red light received by the imaging pixels a and f for the imaging pixel a and the imaging pixel f. For the image pixel corresponding to the imaging pixel b using the information of the component P as it is, for example, the average value of the imaging pixels a, c, f, and j surrounding the periphery is used as information on the vertical polarization component of the red light of the image pixel. Use as
Further, when obtaining a non-spectral horizontal polarization component image having a higher resolution, the information of the non-spectral horizontal polarization component S received by the imaging pixel e is used as it is for the image pixel corresponding to the imaging pixel e. For the image pixels corresponding to the imaging pixels a, b, and f, an average value of the imaging pixel e and the imaging pixel g that receives the non-spectral horizontal polarized light component around it is used, or the same as the imaging pixel e. Or a value may be used.

  The vertically polarized component image of red light obtained in this way can be used, for example, for identifying a tail lamp. Since the vertical polarization component image of the red light has the horizontal polarization component S cut off, the red light is horizontal, such as red light reflected on the road surface or red light (reflection light) from the dashboard of the vehicle 100 or the like. It is possible to obtain a red image in which a disturbance factor due to red light having a strong polarization component S is suppressed. Therefore, the recognition rate of the tail lamp is improved by using the vertical polarization component image of the red light for the identification of the tail lamp.

  The non-spectral vertical polarization component image can be used, for example, for identifying a white line or a headlamp of an oncoming vehicle. In the non-spectral horizontal polarization component image, since the horizontal polarization component S is cut off, white light such as a headlamp or a streetlight reflected on the road surface, white light from a dashboard in the interior of the vehicle 100, etc. (reflection light) A non-spectral image in which disturbance factors due to white light with a strong horizontal polarization component S are suppressed can be obtained. Therefore, the recognition rate is improved by using the non-spectral vertically polarized component image for identifying the white line and the headlamp of the oncoming vehicle. In particular, it is generally known that the reflected light from the water surface covering the road surface has a large amount of horizontal polarization component S in a rainy road. Therefore, by using the non-spectral vertical polarization component image for white line identification, it becomes possible to appropriately identify the white line under the water surface in the rainy road, and the recognition rate is improved.

  In addition, if a comparison image using pixel values as index values obtained by comparing pixel values between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image is used, as described later, the metal in the imaging region It is possible to accurately identify an object, a wet and dry state of a road surface, a three-dimensional object in an imaging region, and a white line on a rainy road. As a comparison image used here, for example, a difference image in which a pixel value is a difference value between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image, or a pixel value between these images. Use a ratio image with the ratio as the pixel value, or a differential polarization degree image with the ratio of the difference value of the pixel values between these images to the sum of the pixel values between these images (difference polarization degree) as the pixel value can do.

FIG. 16 shows the contents of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the raindrop detection filter unit 220B of the optical filter 205 in the present embodiment. It is explanatory drawing which shows.
FIG. 17A is a cross-sectional view schematically showing the raindrop detection filter unit 220B and the image sensor 206 of the optical filter 205 cut along the line AA shown in FIG.
FIG. 17B is a cross-sectional view schematically showing the raindrop detection filter unit 220B and the image sensor 206 of the optical filter 205 cut along the line BB shown in FIG.

  As shown in FIGS. 17A and 17B, the raindrop detection filter portion 220B of the optical filter 205 in this embodiment has a wire grid structure on the filter substrate 221 common to the vehicle detection filter portion 220A. A polarizing filter layer 225 is formed. Along with the polarizing filter layer 222 of the vehicle detection filter unit 220A, the polarizing filter layer 225 is flattened by filling the upper surface side in the stacking direction with a filler. However, unlike the vehicle detection filter unit 220A, the raindrop detection filter unit 220B has no spectral filter layer 223 laminated thereon.

  In the present embodiment, the indoor landscape of the host vehicle 100 may be reflected on the inner wall surface of the windshield 105. This reflection is due to the light regularly reflected by the inner wall surface of the windshield 105. Since this reflection is specular reflection light, it is disturbance light having a relatively high light intensity. Therefore, if this reflection is projected on the raindrop detection image area 214 together with the raindrop, the raindrop detection accuracy is lowered. In addition, specularly reflected light, which is specularly reflected by the inner wall surface of the windshield 105, is also reflected in the raindrop detection image area 214 together with raindrops, resulting in disturbance light that reduces raindrop detection accuracy. Let

  The disturbance light that reduces the detection accuracy of raindrops is specularly reflected light that is specularly reflected by the inner wall surface of the windshield 105, so that most of the polarization component is a polarization component whose polarization direction is perpendicular to the light source incident surface. That is, it is the horizontal polarization component S that vibrates in parallel with the row (horizontal direction) of the imaging pixels of the image sensor 206. Therefore, the polarizing filter layer 225 in the raindrop detection filter unit 220B of the optical filter 205 of the present embodiment has a virtual surface (including the optical axis of the light from the light source unit 202 toward the windshield 105 and the optical axis of the imaging lens 204). The transmission axis is set so as to transmit only the polarization component whose polarization direction is parallel to the light source incident surface), that is, the vertical polarization component P that vibrates parallel to the column (vertical direction) of the imaging pixels of the image sensor 206. ing.

  As a result, the light transmitted through the polarizing filter layer 225 of the raindrop detection filter unit 220B is only the vertical polarization component P, and the light source unit is reflected on the inner wall surface of the windshield 105 or regularly reflected by the inner wall surface of the windshield 105. The horizontally polarized component S occupying most of disturbance light such as specularly reflected light from 202 can be cut. As a result, the raindrop detection image area 214 becomes a vertically polarized image based on the vertical polarization component P, which is less affected by disturbance light, and the raindrop detection accuracy based on the captured image data of the raindrop detection image area 214 is improved.

  In the present embodiment, the infrared light cut filter region 211 and the infrared light transmission filter region 212 constituting the pre-stage filter 210 are each formed of multilayer films having different layer configurations. As a manufacturing method of such a pre-stage filter 210, after the infrared light transmission filter region 212 is formed by vacuum deposition or the like while concealing the infrared light cut filter region 211 with a mask, this time, infrared light is filtered. There is a method in which the part of the infrared light cut filter region 211 is formed by vacuum deposition or the like while the part of the transmission filter region 212 is hidden with a mask.

  In the present embodiment, both the polarizing filter layer 222 of the vehicle detection filter unit 220A and the polarizing filter layer 225 of the raindrop detection filter unit 220B have a wire grid structure in which regions are divided in a two-dimensional direction. However, the former polarizing filter layer 222 is obtained by dividing two types of regions (vertical polarizing region and horizontal polarizing region) whose transmission axes are orthogonal to each other, and the latter polarizing filter layer 225 is vertically polarized. One type of region having a transmission axis that transmits only the component P is divided into regions for each imaging pixel. When the polarizing filter layers 222 and 225 having such different configurations are formed on the same filter substrate 221, for example, adjustment of the groove direction of a template (corresponding to a mold) for patterning a metal wire having a wire grid structure Therefore, adjustment of the longitudinal direction of the metal wire in each region is easy.

In the present embodiment, the infrared light cut filter region 211 may not be provided in the optical filter 205, and the infrared light cut filter region 211 may be provided in the imaging lens 204, for example. In this case, the optical filter 205 can be easily manufactured.
Further, instead of the infrared light cut filter region 211, a spectral filter layer that transmits only the vertical polarization component P may be formed in the raindrop detection filter unit 220B of the post-stage filter 220. In this case, it is not necessary to form the infrared light cut filter region 211 in the upstream filter 210.
Further, the polarizing filter layer is not necessarily provided.
Further, in the optical filter 205 of the present embodiment, a rear-stage filter 220 having a polarizing filter layer 222 and a spectral filter layer 223 divided into regions as shown in FIG. 14 is provided on the image sensor 206 side with respect to the front-stage filter 210. However, the front-stage filter 210 may be provided closer to the image sensor 206 than the rear-stage filter 220.

[Light distribution control of headlamps]
Hereinafter, the light distribution control of the headlamp in the present embodiment will be described.
In the light distribution control of the headlamp in the present embodiment, the captured image data captured by the imaging unit 200 is analyzed to identify the tail lamp and head lamp of the vehicle, the preceding vehicle is detected from the identified tail lamp, and the identified head The oncoming vehicle is detected from the ramp. Then, the driver of the host vehicle 100 can be prevented from being dazzled by avoiding the strong light from the headlamp of the host vehicle 100 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle, and the driver's view of the host vehicle 100 is secured. In order to realize the above, the switching of the high beam and the low beam of the headlamp 104 is controlled, or partial light shielding control of the headlamp 104 is performed.

  In the headlamp light distribution control of the present embodiment, among the information that can be acquired from the imaging unit 101, the intensity (brightness information) of light emitted from each point (light source body) in the imaging region, the headlamp, The distance (distance information) between the light source body (other vehicle) such as the tail lamp and the own vehicle, the spectral information by comparing the red component and the white component (non-spectral) of the light emitted from each light source body, the horizontal polarization component of the white component Information of a white component with a horizontal polarization component cut off, and vertical polarization component information of a red component with a horizontal polarization component cut off are used.

  The brightness information will be described. When the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are imaged by the imaging unit 200, the detected object is detected on the captured image data. One headlamp of the oncoming vehicle is projected brightest, and the tail lamp of the preceding vehicle, which is one of the detection objects, is projected darker than that. In addition, when the reflector is displayed in the captured image data, the reflector is not a light source that emits light by itself, but merely a bright image that is reflected by reflecting the headlamp of the host vehicle, so that it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the reflector is gradually observed on the image sensor 206 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, primary identification of the two types of detection objects (head lamp and tail lamp) and the reflector is possible.

  Further, the distance information will be described. Most headlamps and tail lamps have a pair of left and right pair lamps. Therefore, the distance between the headlamps and tail lamps (that is, other vehicles) and the host vehicle is utilized using the characteristics of this configuration. Can be obtained. The pair of left and right lamps that are paired are displayed close to each other at the same height position on the captured image data captured by the image capturing unit 200, and the widths of the lamp image areas that project the lamps are substantially the same. The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, lamp image areas satisfying the conditions can be identified as pair lamps. At a long distance, the left and right lamps constituting the pair lamp cannot be distinguished and recognized as a single lamp.

  When a pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp and tail lamp which are the pair lamp structure. That is, the distance between the left and right headlamps and the distance between the left and right tail lamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the imaging lens 204 in the imaging unit 200 is known, the distance w1 between the two lamp image areas respectively corresponding to the left and right lamps on the image sensor 206 of the imaging unit 200 is calculated from the captured image data. Accordingly, the distance x between the light source of the headlamp or taillamp which is the pair lamp configuration and the host vehicle can be obtained by simple proportional calculation (x = f × w0 / w1). If the distance x calculated in this way is within an appropriate range, the two lamp image areas used for the calculation can be identified as headlamps and taillamps of other vehicles. Therefore, by using this distance information, the identification accuracy of the head lamp and tail lamp, which are detection objects, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, the imaging pixels a on the image sensor 206 that receive red light (vertical polarization component) P / R from the captured image data captured by the imaging unit 200. By extracting only pixel data corresponding to c, f, h, etc., it is possible to generate a red image in which only the red component in the imaging region is projected. Therefore, when there is an image area having a luminance equal to or higher than a predetermined luminance in the red image, the image area can be identified as a tail lamp image area in which a tail lamp is projected.

  In addition, only pixel data corresponding to the imaging pixels b and d on the image sensor 206 that receives the vertical polarization component P / C of white light (non-spectral) is extracted from the captured image data captured by the imaging unit 200. Thus, a monochrome luminance image (vertical polarization component) within the imaging region can be generated. Therefore, the luminance ratio (red luminance ratio) between the image area on the red image and the image area on the monochrome luminance image corresponding to the image area can be calculated. By using this red luminance ratio, it is possible to grasp the ratio of the relative red component contained in the light from the object (light source body) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the identification accuracy of the tail lamp is improved by using this red luminance ratio.

  Further, the polarization information will be described. In the present embodiment, as described above, imaging on the image sensor 206 that receives the white light (non-spectral) vertical polarization component P / C from the captured image data captured by the imaging unit 200. Extracting pixel data corresponding to the pixels b, d, and the like, and pixel data corresponding to the imaging pixels e, g, etc. on the image sensor 206 that receives the horizontal polarization component S / C of white light (non-spectral), For each image pixel, a comparison image in which pixel values (luminance) between these image data are compared can be obtained. Specifically, for example, a difference image having a pixel value as a difference value (SP) between a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) is compared with a comparison image. Can be obtained as According to such a comparative image, an image area (headlamp image area) of direct light directly incident on the imaging unit 200 from the headlamp and an incident on the imaging unit 200 after being reflected from the water surface of the rain road from the headlamp. The contrast with the image area of indirect light can be increased, and the identification accuracy of the headlamp is improved.

  In particular, as a comparative image, a ratio image with a ratio (S / P) of a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) as a pixel value, or a differential polarization degree A differential polarization degree image having ((S−P) / (S + P)) as a pixel value can be suitably used. In general, it is known that the light reflected by a horizontal mirror such as a water surface always has a high horizontal polarization component. In particular, the ratio (S / P) between the horizontal polarization component S and the vertical polarization component P and the differential polarization degree. When ((S−P) / (S + P)) is taken, it is known that the ratio and the differential polarization degree become maximum at a specific angle (Brewster angle). In rainy roads, water is applied to the asphalt surface, which is a scattering surface, in a state close to a mirror surface, so that the headlamp reflected light from the road surface is stronger in the horizontal polarization component S. Therefore, the image area of the headlamp reflected light from the road surface has a large pixel value (luminance) in the ratio image and the differential polarization degree image. On the other hand, since the direct light from the headlamp is basically non-polarized light, the pixel value (luminance) is small in the ratio image and the differential polarization degree image. Due to this difference, it is possible to appropriately exclude the headlamp reflected light from the rain road surface having the same amount of light as the direct light from the headlamp, and distinguish the direct light from the headlamp from such headlamp reflected light. be able to.

Next, a flow of detection processing for the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 18 is a flowchart showing a flow of vehicle detection processing in the present embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the imaging unit 200, and an image region that is considered to be a detection target is extracted. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of detection objects is the type of the light source displayed in the image area.

  First, in step S1, image data in front of the host vehicle captured by the image sensor 206 of the imaging unit 200 is taken into a memory. As described above, this image data includes a signal indicating the luminance in each imaging pixel of the image sensor 206. Next, in step S2, information related to the behavior of the host vehicle is taken from the vehicle behavior sensor.

  In step S3, an image area with high brightness (high brightness image area) that is considered to be a detection target (tail lamp of the preceding vehicle and bed lamp of the oncoming vehicle) is extracted from the image data captured in the memory. This high luminance image region is a bright region having a luminance higher than a predetermined threshold luminance in the image data, and there are many cases where a plurality of high luminance image regions are extracted, but all of them are extracted. Therefore, at this stage, an image area that reflects reflected light from the rainy road surface is also extracted as a high-luminance image area.

  In the high luminance image region extraction processing, first, in step S31, binarization processing is performed by comparing the luminance value of each imaging pixel on the image sensor 206 with a predetermined threshold luminance. Specifically, a binary image is created by assigning “1” to pixels having a luminance equal to or higher than a predetermined threshold luminance and assigning “0” to other pixels. Next, in step S32, when pixels assigned with “1” are close to each other in the binarized image, a labeling process for recognizing them as one high luminance image region is performed. Thereby, a set of a plurality of adjacent pixels having a high luminance value is extracted as one high luminance image region.

  In step S4, which is executed after the above-described high-intensity image region extraction process, the distance between the object in the imaging region corresponding to each extracted high-intensity image region and the host vehicle is calculated. In this distance calculation process, the pair lamp distance calculation process that detects the distance by using the pair of left and right lamps of the vehicle and the left and right lamps constituting the pair lamp cannot be distinguished and recognized at a long distance. A single lamp distance calculation process is executed when the pair lamp is recognized as a single lamp.

  First, for a pair lamp distance calculation process, in step S41, a pair lamp creation process, which is a process of creating a lamp pair, is performed. The pair of left and right lamps that are paired are in the position where they are close to each other and have substantially the same height in the image data captured by the imaging unit 200, and the area of the high-luminance image region is substantially the same, and the shape of the high-luminance image region is the same. Satisfy the condition of being the same. Therefore, the high-intensity image areas that satisfy such conditions are used as a pair lamp. High brightness image areas that cannot be paired are considered a single lamp. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation process in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f in the imaging unit 200 is known, the actual distance x to the pair lamps is calculated by a simple proportional calculation (x = x = x) by calculating the left / right lamp distance w1 on the image sensor 206 of the imaging unit 200. f · w0 / w1). It should be noted that a dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio (red luminance ratio) between the red image of the vertical polarization component P and the white image of the vertical polarization component P is used as spectral information. From this spectral information, two high-luminance image regions that are paired lamps are A lamp type identification process is performed for identifying whether the light is from the head lamp or from the tail lamp. In this lamp type identification process, first, in step S51, pixel data corresponding to the imaging pixels a and f on the image sensor 206 and pixels corresponding to the imaging pixel b on the image sensor 206 for the high-intensity image area that is a pair of lamps. A red ratio image having a pixel value as a ratio to the data is created. In step S52, the pixel value of the red ratio image is compared with a predetermined threshold, and a high-luminance image region that is equal to or higher than the predetermined threshold is determined to be a tail lamp image region by light from the tail lamp. For a certain high-intensity image area, a lamp type process is performed to determine that the area is a headlamp image area by light from the headlamp.

  Subsequently, in step S6, for each image area identified as the tail lamp image area and the head lamp image area, the difference polarization degree ((S−P) / (S + P)) is used as the polarization information from the tail lamp or the head lamp. A reflection identification process is performed for identifying whether the light is direct light or reflected light reflected by a mirror surface such as a rainy road surface. In this reflection identification process, first, in step S61, a differential polarization degree ((S−P) / (S + P)) is calculated for the tail lamp image region, and a differential polarization degree image with the differential polarization degree as a pixel value is created. Similarly, a difference polarization degree ((S−P) / (S + P)) is calculated for the headlamp image region, and a difference polarization degree image is created using the difference polarization degree as a pixel value. In step S62, the pixel value of each differential polarization degree image is compared with a predetermined threshold value, and it is determined that the tail lamp image area and the head lamp image area that are equal to or larger than the predetermined threshold value are caused by reflected light. Those image areas are excluded because they do not reflect the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle. The tail lamp image area and the head lamp image area remaining after performing this exclusion process are identified as those that reflect the tail lamp of the preceding vehicle or those that reflect the head lamp of the oncoming vehicle.

  Note that the reflection identification process S6 described above may be executed only when a rain sensor or the like is mounted on the vehicle and it is confirmed that the rain sensor is raining. Further, the above-described reflection identification process S6 may be executed only when the driver (driver) is operating the wiper. In short, the above-described reflection identification process S6 may be executed only in rainy weather when reflection from the rainy road surface is assumed.

  In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection process as described above are used for light distribution control of a headlamp that is an in-vehicle device of the host vehicle. Specifically, when the tail lamp is detected by the vehicle detection processing and approaches the distance range in which the headlamp illumination light of the host vehicle is incident on the rearview mirror of the preceding vehicle, the headlamp illumination of the host vehicle is approached to the preceding vehicle. Control is performed to block a part of the headlamp of the host vehicle or to shift the light irradiation direction of the headlamp of the host vehicle in the vertical direction or the horizontal direction so as not to be exposed to light. Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range where the headlamp illumination light of the own vehicle hits, the headlamp illumination light of the own vehicle is emitted to the oncoming vehicle. In order to avoid hitting, a part of the headlamp of the host vehicle is shielded from light, or the light irradiation direction of the headlamp of the host vehicle is shifted in the vertical direction or the horizontal direction.

[White line detection processing]
Hereinafter, the white line detection process in this embodiment will be described.
In the present embodiment, processing for detecting a white line (partition line) as a detection target is performed for the purpose of preventing the vehicle from deviating from the travelable area. Here, the white line includes all white lines that demarcate the road, such as a solid line, a broken line, a dotted line, and a double line. In addition, it can detect similarly about the division line of colors other than white, such as a yellow line.

  In the white line detection processing in the present embodiment, polarization information of the vertical polarization component P of the white component (non-spectral) among the information that can be acquired from the imaging unit 101 is used. In addition, the vertical polarization component of cyan light may be included in the vertical polarization component of white component. In general, it is known that white lines and asphalt surfaces have flat spectral luminance characteristics in the visible light region. On the other hand, since cyan light includes a wide band in the visible light region, it is suitable for imaging asphalt and white lines. Therefore, by using the optical filter 205 in the configuration example 2 and including the vertical polarization component of cyan light in the vertical polarization component of the white component, the number of imaging pixels to be used increases, resulting in an increase in resolution and a distant white line. Can also be detected.

  In the white line detection processing of this embodiment, in many roads, white lines are formed on the road surface of a color close to black, and the brightness of the white line portion in the image of the white component (non-spectral) vertical polarization component P is on the road surface. Big enough than other parts. Therefore, it is possible to detect a white line by determining a portion of the road surface portion having a luminance equal to or higher than a predetermined value as a white line. In particular, in the present embodiment, the image of the vertical polarization component P of the white component (non-spectral) to be used is obtained by suppressing the reflected light from the rainy road because the horizontal polarization component S is cut. Is possible. Therefore, it is possible to detect a white line without erroneously recognizing disturbance light such as reflected light from a headlamp from a rainy road at night as a white line.

  Also, in the white line detection processing in the present embodiment, out of the information that can be acquired from the imaging unit 101, polarization information obtained by comparing the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P, for example, white The differential polarization degree ((S−P) / (S + P)) of the component (non-spectral) horizontal polarization component S and vertical polarization component P may be used. Since the reflected light from the white line is usually dominated by the diffuse reflection component, the vertical polarization component P and the horizontal polarization component S of the reflected light are substantially equal, and the degree of differential polarization shows a value close to zero. On the other hand, the asphalt surface portion on which no white line is formed shows a characteristic in which the scattered reflection component is dominant in the dry state, and the differential polarization degree shows a positive value. Further, when the asphalt surface portion where the white line is not formed is in a wet state, the specular reflection component is dominant, and the differential polarization degree shows a larger value. Therefore, a portion where the polarization difference value of the obtained road surface portion is smaller than the predetermined threshold value can be determined as a white line.

[Raindrop detection processing on the windshield]
Hereinafter, the raindrop detection process in the present embodiment will be described.
In the present embodiment, for the purpose of performing drive control of the wiper 107 and discharge control of the washer liquid, processing for detecting raindrops as deposits is performed. In addition, here, the case where the deposit adhered on the windshield is a raindrop will be described as an example. However, the same applies to deposits such as bird droppings and splashes on the road surface from the adjacent vehicle. is there.

  In the raindrop detection process in the present embodiment, among the information that can be acquired from the imaging unit 101, the infrared light transmission filter region 212 of the front-stage filter 210 and the polarization filter layer 225 in the raindrop detection filter unit 220B of the rear-stage filter 220 are used. The polarization information of the vertical polarization component P of the raindrop detection image area 214 that receives the transmitted light is used. Therefore, it is necessary that the light incident on the windshield 105 from the light source unit 202 includes a large amount of the vertically polarized component P. For this purpose, for example, when a light emitting diode (LED) is used as the light source of the light source unit 202, a polarizer that transmits only the vertical polarization component P is preferably disposed between the light source unit 202 and the windshield 105. Further, when a semiconductor laser (LD) is used as the light source of the light source unit 202, the LD can emit only light of a specific polarization component, so that the light of only the vertical polarization component P is incident on the windshield 105. The axes may be aligned.

  In the present embodiment, as described above, the illumination light (infrared light) irradiated from the light source unit 202 and incident on the inner wall surface of the windshield 105 from the reflection deflection prism 230 has raindrops attached to the outer wall surface of the windshield 105. In the non-attached part which is not carried out, regular reflection is carried out by the outer wall surface of the windshield 105. FIG. The specularly reflected light is received by the image sensor 206 and displayed on the raindrop detection image area 214. On the other hand, at a place where raindrops are attached on the outer wall surface of the windshield 105, the illumination light is transmitted through the outer wall surface of the windshield 105, and the transmitted light is not received by the image sensor 206. Therefore, in the raindrop detection image area 214 of the captured image data, the non-attached portion where the raindrop does not adhere to the outer wall surface of the windshield 105 becomes a high brightness image portion (high pixel value), while the raindrop adheres. The attached part becomes a low-brightness image part (low pixel value). From these differences, it is possible to grasp not only the presence of raindrops but also the amount of raindrops.

FIG. 19 is an explanatory diagram of raindrop detection processing in the present embodiment.
In the raindrop detection process of the present embodiment, the wiper 107 is driven when it is determined that the raindrop amount exceeds a certain amount using information on the raindrop detection image area 214 of the captured image data acquired from the imaging unit 101. Specifically, the raindrop detection image area 214 is divided into eight sections in the horizontal direction of the image as shown in FIG. 19, for example, and total pixel values IL1 to IL8 in each section are calculated. Then, if the total pixel values IL1 to IL8 in any of the sections fall below a predetermined threshold value ILth, it is determined that the raindrop amount exceeds a certain amount, and the wiper 107 is driven. On the contrary, when the total pixel values IL1 to IL8 in any of the sections become equal to or greater than a predetermined threshold value ILth, the driving of the wiper 107 is stopped. The conditions for starting and stopping the driving of the wiper 107 are not limited to this, and can be set as appropriate. For example, the threshold value does not have to be a fixed value, and may be changed as appropriate according to a change in the situation around the host vehicle on which the imaging unit 200 is mounted. Moreover, the threshold value of the start condition and the stop condition may be the same value or different values.

FIG. 20 is an explanatory diagram showing a relationship between a conventional imaging frame and raindrop detection.
FIG. 21 is an explanatory diagram showing the relationship between the imaging frame and raindrop detection in the present embodiment.
Conventionally, as shown in FIG. 20, imaging frames (sensing frames) frames 1 to 4, 6 to 6 for detecting a detection target such as a white line (partition line) on the road surface and other vehicles existing in the imaging region. 9, 11 to 14 and the imaging frames (raindrop detection frames) frames 5 and 10 for detecting raindrops are different imaging frames. In this case, since there is a time gap between the sensing frames before and after the raindrop detection frame, the object to be detected is placed between the imaging time of the previous sensing frame and the imaging time of the subsequent sensing frame. There is a time during which an image for detection cannot be acquired, causing problems such as a reduction in recognition accuracy of a detection target.

  On the other hand, in this embodiment, as shown in FIG. 21, an image having an image (vehicle detection image region) for detecting the detection target and a raindrop detection image region for detecting the raindrop 203. Can be obtained with a single imaging frame frame2,4,6,8,10,12. In this case, a dedicated imaging frame (raindrop detection frame) for detecting raindrops is not necessary, and the above-described problems do not occur.

  Here, in this embodiment, in order to obtain a suitable vehicle detection image area (for example, an image area with high contrast), automatic exposure control (AEC) that changes the exposure amount according to the situation (brightness, etc.) of the imaging area. ). Specifically, automatic exposure control (AEC) for changing the exposure time (exposure amount) of the next imaging frame in accordance with the luminance value of the center of the captured image (pixels in the vehicle detection image area 213) in the previous imaging frame. )I do.

  However, when such automatic exposure control is performed, even if the raindrop adhesion state on the windshield is exactly the same, the brightness and contrast of the image area for raindrop detection change due to the change in the exposure time. Therefore, in the raindrop detection process of the present embodiment, the amount of raindrops detected from the information in the raindrop detection image area 214 cannot be properly determined, and problems such as a malfunction of the wiper 107 may occur.

  For example, as shown in FIG. 22, when the exposure period is changed by the automatic exposure control within the light irradiation period from the light source unit 202, the light source light receiving time (on the windshield) by the image sensor 206 depends on the length of the exposure period. The time during which the reflected light of the light source from the non-raindrop-attached portion is received) changes. In this case, even if the amount of raindrops adhering to the windshield is the same, the amount of received light of the light source received by the image sensor 206 is changed, and as a result, a raindrop detection image region 214 having different brightness and contrast is obtained. The amount cannot be detected properly.

  On the other hand, as shown in FIG. 23, when the exposure period is changed by automatic exposure control outside the light irradiation period from the light source unit 202, the light source light is received by the image sensor 206 even if the exposure period is changed. The time is constant in the light irradiation period. However, even in this case, even when the exposure time of the light source light by the image sensor 206 is constant when the exposure period is changed due to signal processing in the image sensor 206 (for example, the dark correction described above), The analog electrical signal output from the sensor substrate 207 of the image sensor 206 may change. Therefore, in this case as well, even if the amount of raindrops adhering to the windshield is the same, as a result of obtaining the raindrop detection image region 214 having different brightness, contrast, etc., it becomes impossible to appropriately detect the amount of raindrops.

  Therefore, in the present embodiment, the light source light received by the portion corresponding to the raindrop detection image area of the image sensor 206 during the exposure period with the changed exposure time in accordance with the exposure time changed by the automatic exposure control. The light source light control for changing the amount of light is executed. Specifically, the light amount adjustment unit 102B of the image analysis unit 102 functions as a light source light control unit. The light amount adjustment unit 102B acquires the exposure time calculated by the exposure amount control unit 102C of the image analysis unit 102, and in accordance with the exposure time, the intensity of the light source unit 202 that is turned on during the imaging frame for which the exposure time is set. PW control is performed to change. In this PW control, for example, the power of a light source such as each LED provided in the light source unit 202 is changed.

  In the present embodiment, PW control for changing the intensity of the light source unit 202 is adopted as light source light control for changing the amount of light source light received at a portion corresponding to the raindrop detection image area of the image sensor 206. However, other controls may be used. For example, the control may be to change the number of lighting of a plurality of LEDs provided in the light source unit 202. Further, for example, PWM control for controlling the irradiation time ratio of the light source light irradiated by the light source unit 202 during the exposure period may be used. Further, instead of controlling the light source unit 202, the light transmission amount of the light transmitting member that transmits the light source light emitted from the light source unit 202 is controlled, and the portion corresponding to the raindrop detection image region of the image sensor 206 is controlled. You may change the light quantity of the light source light received.

FIG. 24 is an explanatory diagram showing the relationship between the exposure time and the light source power in this embodiment.
In the present embodiment, the light amount adjustment unit 102B reduces the intensity (light source power) of the light source unit 202 if the exposure time is increased by the exposure amount control unit 102C, and the light source unit 202 if the exposure time is reduced by the exposure amount control unit 102C. Increase the strength. At this time, it is preferable to control the intensity of the light source unit 202 so that the amount of the light source light emitted from the light source unit 202 is maintained within a predetermined range during the exposure period. Specifically, for example, the intensity (light source power) of the light source unit 202 is controlled so that the time integration value of the light source light emitted from the light source unit 202 during the exposure period becomes a target value. According to this, the change in the brightness and contrast of the raindrop detection image region due to the change in the exposure time can be suppressed within a predetermined range, and the raindrop amount can be appropriately detected.

  The above PW control is not executed when the exposure period is changed by the automatic exposure control outside the light irradiation period from the light source unit 202 as shown in FIG. 23, and as shown in FIG. It may be executed only when the exposure period is changed by automatic exposure control within the light irradiation period from. As shown in FIG. 23, when the exposure period is changed by the automatic exposure control outside the light irradiation period from the light source unit 202, as described above, the light source light is received by the image sensor 206 even if the exposure period is changed. This is because since the time is constant, changes in the brightness and contrast of the raindrop detection image region due to the change in the exposure time may be sufficiently small.

  In addition, the image at the time of illumination when the light source unit 202 is illuminated and the image at the time of extinction when the light source unit 202 is extinguished are captured, and the above-described raindrop detection processing is performed from the difference image (comparison image). Also good. Of the disturbance light, sunlight or the like does not change greatly even after a certain amount of time has passed, but the headlight of an oncoming vehicle while the host vehicle 100 is traveling may change greatly after a short time. In this case, if the time interval between the two frames for obtaining the difference image is wide, the magnitude of the disturbance light changes between them, and the disturbance light cannot be canceled well when the difference image is generated. There is sex. In order to prevent this, it is preferable that the two frames for obtaining the difference image are continuous frames. In addition, if the exposure time changes during imaging of two frames for detecting raindrops, the amount of disturbance light received in each frame changes, and there is a possibility that disturbance light cannot be canceled appropriately by the difference image. is there. Therefore, for the two frames for detecting raindrops, for example, exposure control is preferably performed so that the exposure time is the same.

[Modification]
Next, a modification of this embodiment will be described.
In the above-described embodiment, the reflection deflecting prism 230 is used, and the illumination light is transmitted through the outer surface of the windshield at the location where raindrops are attached, and the illumination light is reflected from the outer surface of the windshield at the location where no raindrops are attached. It was the structure which receives light. On the other hand, in this modified example, the illumination light is transmitted through the outer surface of the windshield at the location where the raindrops are not attached, and the illumination light transmitted through the outer surface of the windshield is reflected at the locations where the raindrops are attached. It is the structure which receives light. Specifically, the configuration is as shown in FIG.

  In the example of FIG. 25, the illumination light emitted from the light source unit 202 is directly incident on the inner wall surface of the windshield 105, but may be configured via an optical path changing member such as a mirror. In this case, similarly to the above-described embodiment, the light source unit 202 as the light source device can be mounted on the sensor substrate 207.

  A light ray A in FIG. 25 is a light ray emitted from the light source unit 202 and passing through the windshield 105. When the raindrop 203 does not adhere to the outer wall surface of the windshield 105, the light emitted from the light source unit 202 toward the windshield 105 passes through the windshield 105 like the light ray A and remains on the vehicle 100 as it is. Leak outside.

  A light beam B in FIG. 25 is a light beam that is emitted from the light source unit 202 and is regularly reflected by the inner wall surface of the windshield 105 and is incident on the imaging unit 200. Part of the light traveling from the light source unit 202 toward the windshield 105 is regularly reflected by the inner wall surface of the windshield 105. In general, the polarization component of the regular reflection light (ray B) is dominated by the S polarization component (horizontal polarization component S) that vibrates in a direction perpendicular to the incident plane (direction perpendicular to the paper surface of FIG. 25). Is known to be. The specularly reflected light (light ray B) irradiated from the light source unit 202 and specularly reflected by the inner wall surface of the windshield 105 does not vary depending on the presence or absence of the raindrop 203 adhering to the outer wall surface of the windshield 105. In addition, it becomes disturbance light that lowers the detection accuracy of raindrop detection. In this modification, since the light beam B (horizontal polarization component S) is cut by the polarizing filter layer 225 of the raindrop detection filter unit 220B, it is possible to suppress the raindrop detection accuracy from being lowered by the light beam B.

  A light ray C in FIG. 25 is a light ray that is emitted from the light source unit 202 through the inner wall surface of the windshield 105, then reflected by raindrops attached to the outer wall surface of the windshield 105, and incident on the imaging unit 200. is there. A part of the light traveling from the light source unit 202 toward the windshield 105 passes through the inner wall surface of the windshield 105, but the transmitted light is more in the vertical polarization component P than in the horizontal polarization component S. When raindrops 203 are attached on the outer wall surface of the windshield 105, the light transmitted through the inner wall surface of the windshield 105 does not leak out like the light ray A, but is reflected multiple times inside the raindrop. Then, the light passes through the windshield 105 again toward the imaging unit 200 and enters the imaging unit 200. At this time, since the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205 of the imaging unit 200 is configured to transmit the emission wavelength (infrared light) of the light source unit 202, the light ray C is red. It passes through the outside light transmission filter region 212. Further, since the polarizing filter layer 225 of the raindrop detection filter unit 220B of the subsequent post-stage filter 220 is formed with the longitudinal direction of the metal wire having a wire grid structure so as to transmit the vertically polarized component P, the light ray C is a polarizing filter. Layer 225 is also transparent. Therefore, the light ray C reaches the image sensor 206, and raindrops are detected based on the amount of light received.

  A light beam D in FIG. 25 is a light beam that passes through the windshield 105 from the outside of the windshield 105 and enters the raindrop detection filter unit 220B of the imaging unit 200. Although this ray D can also be disturbance light at the time of raindrop detection, in this modification, most of the ray D is cut by the infrared light transmission filter region 212 of the front-stage filter 210 in the optical filter 205. Therefore, it is possible to suppress the raindrop detection accuracy from being lowered by the light ray D.

  A light ray E in FIG. 25 is a light ray that passes through the windshield 105 from the outside of the windshield 105 and enters the filter 220A for vehicle detection of the imaging unit 200. The light beam E is cut in the infrared light band by the infrared light cut filter region 211 of the pre-stage filter 210 in the optical filter 205, and only the light in the visible region is imaged. This captured image is used for detection of a head lamp of an oncoming vehicle, a tail lamp of a preceding vehicle, and a white line.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
A part of the image sensor 206 receives light source light emitted from a light source device such as the light source unit 202 toward a light transmissive member such as the windshield 105, and receives light from a predetermined imaging region in another part of the image sensor 206. Imaging of a deposit image area 214 such as a raindrop detection image area 214 and the like, and a vehicle detection image area 213 and the like that project the imaging area, by reflecting light received at the portion to reflect the deposit such as the raindrop 203 attached to the light transmissive member. Exposure amount control means such as an exposure amount control unit 102C that changes an exposure amount such as an exposure time in accordance with a predetermined exposure condition in an imaging apparatus including an imaging unit 101 and an image analysis unit 102 that capture an image including a region image region And receiving light at the part of the image sensor during the exposure period with the changed exposure amount according to the exposure amount changed by the exposure amount control means. And having a source light control means such as a light amount adjusting unit 102B to perform a source light control for changing the light quantity of the light source light.
According to this, when the exposure amount is changed by the exposure amount control unit, the light amount of the light source light received by the image sensor portion corresponding to the deposit image region can be changed by the light source light control unit. When the light amount of the light source light received by the image sensor portion corresponding to the deposit image area is changed, the brightness, contrast, etc. of the deposit image area can be changed. Therefore, by changing the light amount of the light source light by the light source light control means according to the exposure amount changed by the exposure amount control means, the brightness, contrast, etc. of the deposit image area resulting from the change of the exposure amount Change can be suppressed. Therefore, even when an image including an attachment image area and an imaging area image area is imaged, an appropriate imaging area image area is obtained by changing the exposure amount according to the state of the imaging area, and the exposure amount changes. An appropriate deposit image area in which changes in brightness, contrast, and the like are suppressed can be obtained.

(Aspect B)
In the aspect A, when the exposure amount is increased by the exposure amount control unit, the light source light control unit decreases the light amount of the light source light received by the part of the image sensor, and the exposure amount control unit When the exposure amount decreases, the light source light control is executed so that the amount of the light source light received by the part of the image sensor increases.
According to this, it is possible to easily obtain a deposit image area in which changes in brightness, contrast, and the like due to exposure amount changes are suppressed.

(Aspect C)
In the aspect A or B, the light source light control unit performs the light source light control by controlling the light source device.
According to this, the light amount of the light source light received by the image sensor portion corresponding to the deposit image area so that the brightness, contrast, etc. of the deposit image area can be suppressed when the exposure amount is changed. It is possible to easily realize light source light control for changing the angle.

(Aspect D)
In the aspect C, the light source light control unit performs the light source light control so that the light amount of the light source light irradiated by the light source device is maintained within a predetermined range during the exposure period. .
According to this, it is possible to obtain an appropriate deposit image area in which changes in luminance, contrast, and the like due to exposure amount changes are suppressed within a certain range.

(Aspect E)
In the aspect C or D, the light source light control unit performs the light source light control by controlling the intensity of the light source light emitted by the light source device during the exposure period.
According to this, as in the PW control described above, it is possible to easily change the amount of light source light received by the image sensor portion corresponding to the deposit image area during the exposure period.

(Aspect F)
In any one of the aspects C to E, the light source light control unit performs the light source light control by controlling an irradiation time ratio of the light source light irradiated by the light source device during the exposure period. It is characterized by.
According to this, as in the PWM control described above, it is possible to easily change the light amount of the light source light received by the image sensor portion corresponding to the deposit image area during the exposure period.

(Aspect G)
In any one of the aspects A to F, the exposure amount control means changes an exposure time of the image sensor as the exposure amount, and the light source light control means is exposed by the exposure amount control means. The light source light control is executed when the period is shorter than the light source light irradiation period, but the light source light control is not executed when the exposure amount control means causes the exposure period to be longer than the light source light irradiation period. It is characterized by that.
According to this, when the exposure period becomes longer than the irradiation period of the light source light by the exposure amount control means (as shown in FIG. 23), as described above, even if the exposure period is changed, the image sensor Since the light receiving time of the light source light is constant, the change in the brightness and contrast of the raindrop detection image region due to the change in the exposure time may be sufficiently small. In such a case, the execution of the unnecessary light source light control can be suppressed by not performing the light source light control.

(Aspect H)
Mobile device control means such as a wiper control unit 106 for controlling predetermined devices such as a wiper mounted on a mobile body such as the host vehicle 100 based on an attachment image area of an image captured by the imaging device. In a mobile device control system such as an in-vehicle device control system, the imaging device according to any one of the modes A to G is used as the imaging device.
According to this, even when an image including an attachment image region and an imaging region image region is captured, the exposure amount is changed according to the state of the imaging region to obtain an appropriate imaging region image region, and the exposure amount change Therefore, it is possible to obtain an appropriate deposit image area in which changes in brightness, contrast, and the like due to the image are suppressed, and thus it is possible to perform device control based on the deposit image area with high accuracy.

(Aspect I)
By receiving light source light emitted from the light source device toward the light transmissive member by a part of the image sensor and receiving light from a predetermined imaging region by another part of the image sensor, the light transmissive member A program for causing a computer of an image pickup apparatus that captures an image including an attached image area that displays an attached object image area and an imaging area image area that displays the imaging area, the exposure amount to be determined according to a predetermined exposure condition The amount of light source light received by the part of the image sensor is changed during the exposure period with the changed exposure amount according to the exposure amount control unit to be changed and the exposure amount changed by the exposure amount control unit. The computer is caused to function as light source light control means for executing light source light control.
According to this, even when an image including an attachment image region and an imaging region image region is captured, the exposure amount is changed according to the state of the imaging region to obtain an appropriate imaging region image region, and the exposure amount change Thus, it is possible to obtain an appropriate deposit image area in which changes in brightness, contrast, and the like due to the above are suppressed.
This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 102A Detection processing part 102B Light quantity adjustment part 102C Exposure amount control part 103 Headlamp control unit 104 Headlamp 105 Windshield 106 Wiper control unit 107 Wiper 108 Vehicle travel control unit 200 Imaging part 202 Light source Unit 203 raindrop 204 imaging lens 205 optical filter 206 image sensor 207 sensor substrate 208 signal processing unit 213 vehicle detection image region 214 raindrop detection image region 230 reflection deflecting prism

JP 2010-14494 A

Claims (9)

By receiving light source light emitted from the light source device toward the light transmissive member by a part of the image sensor and receiving light from a predetermined imaging region by another part of the image sensor, the light transmissive member In an imaging apparatus that captures an image including an attached object image area that displays an attached object that is attached to the imaging area and an imaging area image area that displays the imaging area,
Exposure amount control means for changing the exposure amount according to predetermined exposure conditions;
Light source light for executing light source light control for changing the light amount of the light source light received by the part of the image sensor during the exposure period with the changed exposure amount according to the exposure amount changed by the exposure amount control means And an imaging device. The imaging device according to claim 1,
When the exposure amount is increased by the exposure amount control unit, the light source light control unit decreases the light amount of the light source light received by the part of the image sensor, and the exposure amount control unit decreases the exposure amount. In this case, the light source light control is executed so that the amount of the light source light received by the part of the image sensor is increased. The imaging device according to claim 1 or 2,
The light source light control means executes the light source light control by controlling the light source device. The imaging device according to claim 3.
The imaging apparatus according to claim 1, wherein the light source light control unit performs the light source light control so that a light amount of the light source light irradiated by the light source device is maintained within a predetermined range during the exposure period. In the imaging device according to claim 3 or 4,
The imaging apparatus according to claim 1, wherein the light source light control means executes the light source light control by controlling an intensity of the light source light emitted by the light source device during the exposure period. The imaging apparatus according to any one of claims 3 to 5,
The imaging apparatus according to claim 1, wherein the light source light control means executes the light source light control by controlling an irradiation time ratio of the light source light emitted by the light source device during the exposure period. The imaging apparatus according to any one of claims 1 to 6,
The exposure amount control means changes the exposure time of the image sensor as the exposure amount,
The light source light control unit executes the light source light control when the exposure amount control unit makes the exposure period shorter than the light source light irradiation period, but the exposure amount control unit performs the light source light irradiation. The imaging apparatus is characterized in that the light source light control is not executed when the period is exceeded. In the mobile device control system including mobile device control means for controlling a predetermined device mounted on the mobile based on the attachment image area of the image captured by the imaging device,
A mobile device control system using the imaging device according to any one of claims 1 to 7 as the imaging device. By receiving light source light emitted from the light source device toward the light transmissive member by a part of the image sensor and receiving light from a predetermined imaging region by another part of the image sensor, the light transmissive member A program for causing a computer of an imaging apparatus to capture an image including an adhesion image area that displays an adhesion object that is attached to the imaging area and an imaging area image area that reflects the imaging area;
Exposure amount control means for changing the exposure amount in accordance with predetermined exposure conditions; and
Light source light for executing light source light control for changing the light amount of the light source light received by the part of the image sensor during the exposure period with the changed exposure amount according to the exposure amount changed by the exposure amount control means A program for causing the computer to function as control means.

Images