JP5936098B2 - Imaging apparatus, object detection apparatus including the same, and optical filter - Google Patents

Description

  In the present invention, light from an object existing in an imaging region is received by an image sensor configured by a pixel array in which light receiving elements are two-dimensionally arranged through an optical filter, thereby imaging the imaging region. The present invention relates to an imaging device, an object detection device including the same, an optical filter, and a method for manufacturing the same.

  Patent Document 1 discloses a system that can capture captured images with different exposure times with the front of the host vehicle as an imaging region using one camera mounted on the host vehicle. In this system, first, a high-luminance detection image (low transmittance image) with a short exposure time is captured, and then a low-luminance detection image (high transmittance image) with a long exposure time is captured. Then, an image analysis process for identifying the headlamp of the oncoming vehicle based on the high-intensity detection image captured first is performed, and an image analysis process for identifying the tail lamp of the preceding vehicle based on the low-intensity detection image captured later. Do.

  The tail lamps and headlamps of the same vehicle are usually one set of two that are separated by a certain distance in the lateral direction. Therefore, when detecting a preceding vehicle or an oncoming vehicle, when the image portions of two tail lamps or two head lamps in the captured image are separated by a certain distance in the lateral direction, the set of tail lamps is used. Alternatively, a set of headlamps is recognized as that of the preceding vehicle or the oncoming vehicle, and the preceding vehicle or the oncoming vehicle is detected.

  The light from the headlamp is larger than the light from the taillamp. In particular, the difference in the amount of light between the headlamp of the oncoming vehicle close to the host vehicle and the tail lamp of the preceding vehicle far from the host vehicle becomes very large. For this reason, if the exposure time of the camera is set so that even a far tail lamp can be properly identified, the amount of light received by the headlamp by the image sensor of the camera is saturated and a blooming phenomenon occurs. As a result, particularly for headlamps in the vicinity of the host vehicle, the image area of the headlamp expands to the periphery, and two headlamp image areas that should be recognized apart from each other are recognized as an integral image area. The detection accuracy of the oncoming vehicle is reduced. Conversely, if the camera exposure time is set so that even nearby headlamps can be properly identified, the amount of light received by the taillamp will be insufficient, the taillamp image area will not be properly recognized, and the detection accuracy of the preceding vehicle will be reduced. To do.

  According to the system described in Patent Document 1, a headlamp with a large amount of light is identified using a high-intensity detection image with a short exposure time, and a taillight with a small amount of light is identified with a low-luminance detection image with a long exposure time. Use to identify. Therefore, it is possible to appropriately identify both the tail lamp and the headlight, and it is possible to realize high detection accuracy for both the preceding vehicle and the oncoming vehicle.

  However, the system described in Patent Document 1 requires two imaging operations with different exposure times in order to properly identify both the headlamp and the taillamp. As in the same system, when a preceding vehicle or an oncoming vehicle that can move in the imaging region at a relatively high speed is set as a detection target, it is desired to detect these detection targets continuously at as short a time interval as possible. . However, in the system described in Patent Document 1, a low-intensity detection image for detecting a preceding vehicle and a high-intensity detection image for detecting an oncoming vehicle are sequentially captured by one camera. The time interval when each image is continuously acquired requires twice as long as the time interval (frame rate) at which continuous imaging can be performed with the camera to be used. Therefore, the system described in Patent Document 1 has a problem that it is impossible to continuously detect both the preceding vehicle and the oncoming vehicle at a time interval as short as the time interval at which continuous imaging can be performed by the camera used. It was.

  This problem is not limited to the case in which both the preceding vehicle and the oncoming vehicle are detected continuously by identifying the headlamp and taillight from the captured image, and there are a plurality of light source bodies with significantly different amounts of light in the same imaging area. In a system where it is required to appropriately recognize these light source bodies from the captured image, there is a problem as well.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image pickup that can continuously detect a plurality of light source bodies having greatly different light amounts at shorter time intervals. The present invention provides an apparatus, an object detection apparatus including the apparatus, an optical filter, and a manufacturing method thereof.

In order to achieve the above object, according to the present invention, light from an object existing in an imaging region is received by an image sensor including a pixel array in which light receiving elements are two-dimensionally arranged through an optical filter. Thus, in the image pickup apparatus for picking up an image in the image pickup region, the optical filter has a plurality of types of light transmission regions having different transmittances when uniform light is incident on one light receiving element or two on the image sensor. The unit area composed of the light receiving elements described above has a transmittance adjustment layer alternately arranged in the two-dimensional direction of the pixel array, and the transmittance adjustment layer selects only a polarization component in a specific direction. A first region composed of a light transmission part having a metal wire grid structure to be transmitted and a metal light-shielding part disposed around the light transmission part, and transmits light without selecting a polarization component Light transmission Or the second composed of a shielding portion of a metal disposed around the light transmitting portion and a light transmitting portion having a wire grid structure of metal to be transmitted by selecting only the polarization component in a direction different from the said specific direction region and is formed by a polarizing filter layer divided into areas above Kitan position region, the light transmission regions of low the transmittance is relatively kind, the transmittance is relatively more high types of light transmissive region Also, the area of the light transmission part is small .

  In the present invention, on the image sensor, a light receiving element that receives light transmitted through the light transmitting region with low transmittance in the transmittance adjusting layer, and light that has passed through the light transmitting region with high transmittance in the transmittance adjusting layer is received. The light receiving elements to be alternately positioned in the two-dimensional direction of the image array. As a result, a low transmittance image with a relatively small amount of received light can be obtained from the former light receiving element group, and a high transmittance image with a relatively large amount of received light can be obtained from the latter light receiving element group. it can. A light source body with a relatively large amount of light in the imaging region is detected using a low transmittance image, and a light source body with a relatively small amount of light in the imaging region is detected using a high transmittance image. Even if there are a plurality of light source bodies having different light amounts, these light source bodies can be appropriately detected. In the present invention, both a low transmittance image and a high transmittance image can be obtained by a single imaging operation. Therefore, the time interval at which these images can be continuously imaged can be shortened as compared with the conventional imaging device in which the low-transmittance image and the high-transmittance image are obtained by dividing them into two imaging operations.

  According to the present invention, since a light source body with a relatively small amount of light and a light source body with a relatively large amount of light can be appropriately identified from image data obtained by one imaging, a plurality of light sources having greatly different light amounts An excellent effect is obtained that the body can be continuously detected at shorter time intervals.

It is a mimetic diagram showing a schematic structure of an in-vehicle device control system in an embodiment. It is a schematic diagram which shows schematic structure of the imaging unit in the same vehicle equipment control system. It is explanatory drawing which shows schematic structure of the imaging device provided in the imaging unit. It is explanatory drawing which shows the infrared light image data which is the picked-up image data for raindrop detection, when the raindrop on the outer wall surface of the front glass of the own vehicle is focused. It is explanatory drawing which shows the infrared light image data which is the picked-up image data for raindrop detection in the case where the focus is infinite. It is a graph which shows the filter characteristic of the cut filter applicable to the picked-up image data for raindrop detection. It is a graph which shows the filter characteristic of the band pass filter applicable to the picked-up image data for raindrop detection. It is a front view of the front | former stage filter provided in the optical filter of the imaging device. It is explanatory drawing which shows the example of an image of the captured image data of the imaging device. 2 is an explanatory diagram illustrating details of the imaging apparatus. FIG. It is a model enlarged view when the optical filter and image sensor of the imaging device are seen from the direction orthogonal to the light transmission direction. It is explanatory drawing which shows the area | region division pattern of the polarizing filter layer and spectral filter layer of the same optical filter. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the optical filter in the structural example 1, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol BB shown in FIG. It is explanatory drawing which shows an example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the other example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the further another example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the further another example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the further another example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the further another example of the structure which restrict | limits the light reception amount of the light which permeate | transmits the non-spectral area | region of the spectral filter layer of the same optical filter. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the optical filter in the structural example 2, and is received by each photodiode on an image sensor. (A) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol BB shown in FIG. It is explanatory drawing which shows the content of the information (information of each imaging pixel) corresponding to the light reception amount which permeate | transmits the optical filter in the structural example 3, and is received with each photodiode on an image sensor. (A) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol AA shown in FIG. (B) is sectional drawing which represented typically the optical filter and image sensor which were cut | disconnected by the code | symbol BB shown in FIG. It is explanatory drawing which shows the longitudinal direction of the metal wire of the wire grid structure in the polarizing filter layer of the optical filter in the structural example 4. It is an enlarged view of the wire grid structure which comprises a polarizing filter layer. It is a graph which shows the filter characteristic of the cut filter applicable to a spectral filter layer. It is a graph which shows the filter characteristic of the band pass filter applicable to a spectral filter layer. It is a histogram when the direct light from a headlamp and the reflected light (reflected light) are imaged on the rainy road surface of a headlamp on the rainy day, and the difference polarization degree is calculated using the same imaging device. It is a schematic diagram which shows an example at the time of the own vehicle driving | running | working on a rainy road surface, and when the situation where both a preceding vehicle and an oncoming vehicle exist in substantially the same distance ahead of the advancing direction is imaged with the imaging device. It is a flowchart which shows the flow of the vehicle detection process in embodiment. (A) is explanatory drawing for demonstrating the change of reflected light when a road surface is a wet state. (B) is explanatory drawing for demonstrating the change of reflected light when a road surface is a dry state. It is a graph which shows the incident angle dependence of the horizontal polarization component Is and the vertical polarization component Ip of the reflected light with respect to the incident light of the light intensity I. It is explanatory drawing which shows the example of a setting of the process line with respect to the captured image at the time of detecting a road surface metal object. (A) is an example of an image showing a monochrome luminance image (non-spectral / non-polarized light) obtained by imaging an imaging region including a road surface metal object. (B) is an image example showing a non-spectral differential polarization degree image obtained by imaging the same imaging region. It is explanatory drawing which shows the example of a setting of the process line with respect to the captured image at the time of detecting a solid object. (A) is an example of an image showing a monochrome luminance image (non-spectral / non-polarized light) obtained by imaging an imaging region including a three-dimensional object. (B) is an image example showing a non-spectral differential polarization degree image obtained by imaging the same imaging region. It is explanatory drawing which shows the outline | summary of the experiment which changes the light source position with respect to the test object, and image | photographs the image of the horizontal polarization component S and the image of the vertical polarization component P with the camera fixedly arranged in a laboratory. Experiments in the laboratory where an asphalt surface and a painted surface were used as test objects, the position of the light source was changed, and an image of the horizontal polarization component S and the vertical polarization component P was captured using a fixed camera. It is a graph which shows a result. (A) is an image example showing a monochrome luminance image (non-spectral / non-polarized light) obtained by imaging an imaging region including a road edge. (B) is an image example showing a non-spectral differential polarization degree image obtained by imaging the same imaging region. Experiments in an experiment where an asphalt surface and a concrete surface were used as test objects, the position of the light source was changed, and an image of a horizontal polarization component S and an image of a vertical polarization component P were taken with a fixed camera. It is a graph which shows a result. (A) is an image example showing a monochrome luminance image (non-spectral / non-polarized light) obtained by imaging an imaging region including a white line in rainy weather and a non-spectral differential polarization degree image. (B) is an image example which shows the non-spectral difference polarization degree image which imaged the same imaging area at the time of rainy weather. It is explanatory drawing which shows the polarization state of the reflected light in a Brewster angle. (A) is the graph which showed the ratio of the light reception amount of the imaging device with respect to the emitted light amount of the light source in an imaging unit for every polarization | polarized-light component when the raindrop has not adhered to the outer wall surface of a windshield. (B) is the graph which showed the ratio of the light reception amount of the imaging device with respect to the light emission amount of the light source for each polarization component when raindrops are attached to the outer wall surface of the windshield. (A) is a graph which shows a differential polarization degree when the raindrop does not adhere to the outer wall surface of a windshield. (B) is a graph which shows a differential polarization degree when the raindrop has adhered to the outer wall surface of a windshield. It is an image example which shows the differential polarization degree image of the windshield to which the raindrop has adhered when arrange | positioning so that an incident angle may be near 50 degree | times.

Hereinafter, an embodiment in which an imaging device according to the present invention is used in an in-vehicle device control system will be described.
The imaging device according to the present invention is not limited to the in-vehicle device control system, and can be applied to other systems equipped with an object detection device that detects an object based on a captured image, for example.

FIG. 1 is a schematic diagram illustrating a schematic configuration of an in-vehicle device control system according to the present embodiment.
The in-vehicle device control system uses the captured image data of the front area (imaging area) in the traveling direction of the host vehicle captured by the imaging device mounted on the host vehicle 100 such as an automobile, and controls the light distribution of the headlamp and the wiper. It controls drive control and other in-vehicle devices.

  The imaging device provided in the in-vehicle device control system of the present embodiment is provided in the imaging unit 101, and images the traveling region forward area of the traveling vehicle 100 as an imaging region. The windshield 105 is installed in the vicinity of a room mirror (not shown). The captured image data captured by the imaging device of the imaging unit 101 is input to the image analysis unit 102. The image analysis unit 102 analyzes the captured image data transmitted from the imaging device, calculates the position, direction, and distance of another vehicle existing ahead of the host vehicle 100 in the captured image data, or attaches to the windshield 105. For example, a detection object such as a white line (partition line) on the road surface existing in the imaging region is detected. In the detection of other vehicles, a preceding vehicle traveling in the same traveling direction as the own vehicle 100 is detected by identifying the tail lamp of the other vehicle, and traveling in the opposite direction to the own vehicle 100 by identifying the headlamp of the other vehicle. An oncoming vehicle is detected.

  The calculation result of the image analysis unit 102 is sent to the headlamp control unit 103. For example, the headlamp control unit 103 generates a control signal for controlling the headlamp 104 that is an in-vehicle device of the host vehicle 100 from the distance data calculated by the image analysis unit 102. Specifically, for example, while avoiding that the strong light of the headlamp of the own vehicle 100 is incident on the driver of the preceding vehicle or the oncoming vehicle, the driver of the other vehicle is prevented from being dazzled. The switching of the high beam and the low beam of the headlamp 104 is controlled, and partial shading control of the headlamp 104 is performed so that the driver's visibility can be secured.

  The calculation result of the image analysis unit 102 is also sent to the wiper control unit 106. The wiper control unit 106 controls the wiper 107 to remove deposits such as raindrops and foreign matters attached to the windshield 105 of the host vehicle 100. The wiper control unit 106 receives a foreign object detection result detected by the image analysis unit 102 and generates a control signal for controlling the wiper 107. When the control signal generated by the wiper control unit 106 is sent to the wiper 107, the wiper 107 is operated to ensure the visibility of the driver of the host vehicle 100.

  The calculation result of the image analysis unit 102 is also sent to the vehicle travel control unit 108. Based on the white line detection result detected by the image analysis unit 102, the vehicle travel control unit 108 warns the driver of the host vehicle 100 when the host vehicle 100 is out of the lane area defined by the white line. Notification is performed, and driving support control such as controlling the steering wheel and brake of the host vehicle is performed.

FIG. 2 is a schematic diagram illustrating a schematic configuration of the imaging unit 101.
FIG. 3 is an explanatory diagram illustrating a schematic configuration of the imaging device 200 provided in the imaging unit 101.
The imaging unit 101 includes an imaging device 200, a light source 202, and an imaging case 201 that accommodates these. The imaging unit 101 is installed on the inner wall surface side of the windshield 105 of the host vehicle 100. As shown in FIG. 3, the imaging device 200 includes an imaging lens 204, an optical filter 205, and an image sensor 206. The light source 202 emits light toward the windshield 105 and is arranged so that the reflected light enters the imaging device 200 when the light is reflected by the outer wall surface of the windshield 105.

  In the present embodiment, the light source 202 is for detecting an adhering matter adhering to the outer wall surface of the windshield 105 (hereinafter, the case where the adhering matter is a raindrop will be described as an example). When raindrops 203 are not attached to the outer wall surface of the windshield 105, the light emitted from the light source 202 is reflected at the interface between the outer wall surface of the windshield 105 and the outside air, and the reflected light enters the imaging device 200. . On the other hand, as shown in FIG. 2, when raindrops 203 are attached to the outer wall surface of the windshield 105, the refractive index difference between the outer wall surface of the windshield 105 and the raindrop 203 is the same as that of the outer wall surface of the windshield 105. It becomes smaller than the refractive index difference from the outside air. For this reason, the light emitted from the light source 202 passes through the interface and does not enter the imaging device 200. Due to this difference, the raindrop 203 adhering to the windshield 105 is detected from the captured image data of the imaging apparatus 200.

  In the present embodiment, the imaging unit 101 covers the imaging device 200 and the light source 202 with the imaging glass 201 together with the windshield 105 as shown in FIG. By covering with the imaging case 201 in this way, it is possible to suppress a situation where the windshield 105 covered with the imaging unit 101 is fogged even in a situation where the inner wall surface of the windshield 105 is clouded. Therefore, a situation in which the image analysis unit 102 misanalyzes due to fogging of the windshield can be suppressed, and various control operations based on the analysis result of the image analysis unit 102 can be appropriately performed.

  However, when the fogging of the windshield 105 is detected from the captured image data of the imaging device 200 and, for example, the air conditioning equipment of the host vehicle 100 is controlled, the portion of the windshield 105 facing the imaging device 200 is different from the other portions. A passage through which air flows may be formed in a part of the imaging case 201 so as to achieve the same situation.

  Here, in this embodiment, the focal position of the imaging lens 204 is set between infinity or infinity and the windshield 105. Thereby, not only when detecting the raindrop 203 attached on the windshield 105, but also when detecting the preceding vehicle and the oncoming vehicle and detecting the white line, appropriate information is obtained from the captured image data of the imaging device 200. Can be acquired.

  For example, when the raindrop 203 attached on the windshield 105 is detected, the shape of the raindrop image on the captured image data is often circular, so whether the raindrop candidate image on the captured image data is circular. A shape recognition process is performed to determine whether the raindrop candidate image is a raindrop image. When performing such shape recognition processing, the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105, as described above, between infinity or infinity and the windshield 105. The in-focus state is slightly out of focus, the raindrop shape recognition rate (circular shape) is high, and the raindrop detection performance is high.

FIG. 4 is an explanatory diagram showing infrared light image data that is captured image data for raindrop detection when the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105.
FIG. 5 is an explanatory diagram showing infrared light image data, which is captured image data for raindrop detection, when the focus is set to infinity.
When the imaging lens 204 is focused on the raindrop 203 on the outer wall surface of the windshield 105, the background image 203a reflected in the raindrop is captured as shown in FIG. Such a background image 203 a causes erroneous detection of the raindrop 203. Further, as shown in FIG. 4, only a portion 203b of the raindrop may have a brightness that increases in a bow shape or the like, and the shape of the large brightness portion, that is, the shape of the raindrop image changes depending on the direction of sunlight, the position of the streetlight, etc. . In order to cope with the shape of such variously changing raindrop images by the shape recognition processing, the processing load is large and the recognition accuracy is lowered.

  On the other hand, when the image is focused at infinity, some blurring occurs as shown in FIG. For this reason, the reflection of the background image 203a is not reflected in the captured image data, and erroneous detection of the raindrop 203 is reduced. In addition, the occurrence of slight blurring reduces the degree to which the shape of the raindrop image changes depending on the direction of sunlight, the position of the streetlight, and the like. The shape of the raindrop image is always substantially circular. Therefore, the load of the shape recognition process of the raindrop 203 is small, and the recognition accuracy is high.

  However, when focusing at infinity, when identifying the tail lamp of a preceding vehicle traveling far, there may be about one light receiving element that receives the light of the tail lamp on the image sensor 206. In this case, as will be described in detail later, the light from the tail lamp may not be received by the red light receiving element that receives the tail lamp color (red). In this case, the tail lamp cannot be recognized and the preceding vehicle cannot be detected. In order to avoid such a problem, it is preferable that the imaging lens 204 is focused before infinity. As a result, the tail lamp of the preceding vehicle traveling far is out of focus, so that the number of light receiving elements that receive the light of the tail lamp can be increased, the recognition accuracy of the tail lamp is increased, and the detection accuracy of the preceding vehicle is improved.

  For the light source 202 of the imaging unit 101, a light emitting diode (LED), a semiconductor laser (LD), or the like can be used. Further, for example, visible light or infrared light can be used as the emission wavelength of the light source 202. However, in order to avoid dazzling the driver or pedestrian of the oncoming vehicle with the light of the light source 202, the wavelength is longer than the visible light and the light receiving sensitivity of the image sensor 206 reaches, for example, 800 nm or more. It is preferable to select a wavelength in the infrared light region of 1000 nm or less. The light source 202 of the present embodiment emits light having a wavelength in the infrared light region.

  Here, when infrared imaging light from the light source 202 reflected by the windshield 105 is imaged by the imaging device 200, the image sensor 206 of the imaging device 200 uses, for example, sunlight as well as infrared wavelength light from the light source 202. A large amount of disturbance light including the infrared wavelength light is also received. Therefore, in order to distinguish the infrared wavelength light from the light source 202 from such a large amount of disturbance light, it is necessary to make the light emission amount of the light source 202 sufficiently larger than the disturbance light. It is often difficult to use an amount of light source 202.

  Therefore, in the present embodiment, for example, a cut filter that cuts light having a wavelength shorter than the emission wavelength of the light source 202 as shown in FIG. 6, or a peak of transmittance as shown in FIG. The light from the light source 202 is received by the image sensor 206 through a bandpass filter that substantially matches the emission wavelength of 202. As a result, light other than the emission wavelength of the light source 202 can be removed and received, so that the amount of light from the light source 202 received by the image sensor 206 is relatively large with respect to disturbance light. As a result, even if the light source 202 does not have a large light emission amount, the light from the light source 202 can be distinguished from disturbance.

  However, in the present embodiment, not only the raindrop 203 on the windshield 105 is detected from the captured image data, but also the preceding vehicle and the oncoming vehicle and the white line are detected. Therefore, if the wavelength band other than the infrared wavelength light emitted by the light source 202 is removed from the entire captured image, the image sensor 206 receives light in the wavelength band necessary for detection of the preceding vehicle and oncoming vehicle and detection of the white line. This cannot be done and hinders these detections. Therefore, in the present embodiment, the image area of the captured image data, the raindrop detection image area for detecting the raindrop 203 on the windshield 105, the vehicle for detecting the preceding vehicle and the oncoming vehicle and the white line are detected. A filter that removes a wavelength band other than the infrared wavelength light emitted from the light source 202 only in a portion corresponding to the raindrop detection image region is arranged in the optical filter 205. The filter is divided into detection image regions.

FIG. 8 is a front view of the front-stage filter 210 provided in the optical filter 205.
FIG. 9 is an explanatory diagram illustrating an example of captured image data.
As shown in FIG. 3, the optical filter 205 of the present embodiment has a structure in which a front-stage filter 210 and a rear-stage filter 220 are overlapped in the light transmission direction. As shown in FIG. 8, the front-stage filter 210 includes an infrared light cut filter area 211 and a raindrop detection image area 214 that are arranged at locations corresponding to the upper two-third captured image, which is the vehicle detection image area 213. The region is divided into an infrared light transmission filter region 212 arranged at a position corresponding to a lower one third of a captured image. For the infrared light transmission filter region 212, the cut filter shown in FIG. 6 or the band-pass filter shown in FIG. 7 is used.

  In many cases, the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line image are mainly present in the upper part of the captured image, and the image of the nearest road surface in front of the host vehicle is usually present in the lower part of the captured image. Therefore, the information necessary for identifying the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line is concentrated on the upper portion of the captured image, and the information on the lower portion of the captured image is not so important in the identification. Therefore, when both oncoming vehicle, preceding vehicle, or white line detection and raindrop detection are performed simultaneously from a single captured image data, as shown in FIG. It is preferable that the upper portion of the remaining captured image is the vehicle detection image region 213 and the pre-filter 210 is divided into regions corresponding to this.

  When the imaging direction of the imaging apparatus 200 is tilted downward, the hood of the host vehicle may enter the lower part of the imaging area. In this case, sunlight reflected by the bonnet of the host vehicle or the tail lamp of the preceding vehicle becomes disturbance light, which is included in the captured image data, which may cause misidentification of the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line. Become. Even in such a case, in this embodiment, the cut filter shown in FIG. 6 and the bandpass filter shown in FIG. Disturbance light such as the tail lamp of the vehicle is removed. Therefore, the head lamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the white line identification accuracy are improved.

  In this embodiment, due to the characteristics of the imaging lens 204, the scene in the imaging area and the image on the image sensor 206 are upside down. Therefore, when the lower portion of the captured image is used as the raindrop detection image region 214, the upper portion of the upstream filter 210 of the optical filter 205 is configured by the cut filter shown in FIG. 6 or the bandpass filter shown in FIG. .

  Here, when the preceding vehicle is detected, the preceding vehicle is detected by identifying the tail lamp on the captured image. However, the tail lamp has a smaller amount of light compared to the headlamp of the oncoming vehicle, and disturbance such as street lights. Since there is a lot of light, it is difficult to detect the tail lamp with high accuracy only from mere luminance data. Therefore, it is necessary to identify the tail lamp based on the amount of received red light using spectral information for identifying the tail lamp. Therefore, in this embodiment, as will be described later, a red filter or a cyan filter (a filter that transmits only the wavelength band of the tail lamp color) that matches the color of the tail lamp is disposed in the subsequent filter 220 of the optical filter 205, and the red color. The amount of light received can be detected.

  However, since each light receiving element constituting the image sensor 206 of the present embodiment has sensitivity to light in the infrared wavelength band, it is obtained when the image sensor 206 receives light including the infrared wavelength band. The captured image becomes reddish as a whole. As a result, it may be difficult to identify the red image portion corresponding to the tail lamp. Therefore, in the present embodiment, the portion corresponding to the vehicle detection image region 213 in the upstream filter 210 of the optical filter 205 is the infrared light cut filter region 211. Thereby, since the infrared wavelength band is excluded from the captured image data portion used for identifying the tail lamp, the identification accuracy of the tail lamp is improved.

FIG. 10 is an explanatory diagram showing details of the imaging apparatus 200 in the present embodiment.
The imaging apparatus 200 mainly includes an imaging lens 204, an optical filter 205, a sensor substrate 207 including an image sensor 206 having a two-dimensionally arranged pixel array, and an analog electrical signal ( The signal processing unit 208 generates and outputs captured image data obtained by converting the received light amount received by each light receiving element on the image sensor 206 into a digital electric signal. Light from the imaging region including the subject (detection target) passes through the imaging lens 204, passes through the optical filter 205, and is converted into an electrical signal corresponding to the light intensity by the image sensor 206. In the signal processing unit 208, when an electrical signal (analog signal) output from the image sensor 206 is input, the brightness (luminance) of each pixel on the image sensor 206 is shown as captured image data from the electrical signal. The digital signal is output to the subsequent unit together with the horizontal / vertical synchronizing signal of the image.

FIG. 11 is a schematic enlarged view when the optical filter 205 and the image sensor 206 are viewed from a direction orthogonal to the light transmission direction.
The image sensor 206 is an image sensor using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and a photodiode 206A is used as a light receiving element thereof. The photodiodes 206A are two-dimensionally arranged for each pixel, and a microlens 206B is provided on the incident side of each photodiode 206A in order to increase the light collection efficiency of the photodiode 206A. The image sensor 206 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 207.

  An optical filter 205 is disposed close to the surface of the image sensor 206 on the micro lens 206B side. As shown in FIG. 11, the post filter 220 of the optical filter 205 has a laminated structure in which a polarizing filter layer 222 and a spectral filter layer 223 are sequentially formed on a transparent filter substrate 221. Each of the polarizing filter layer 222 and the spectral filter layer 223 is divided into regions corresponding to one photodiode 206A on the image sensor 206.

  A configuration may be adopted in which there is a gap between the optical filter 205 and the image sensor 206, but the configuration in which the optical filter 205 is in close contact with the image sensor 206 is different from the polarization filter layer 222 and the spectral filter layer 223 of the optical filter 205. It becomes easy to make the boundary of each area | region and the boundary between photodiode 206A on the image sensor 206 correspond. The optical filter 205 and the image sensor 206 may be bonded with, for example, a UV adhesive, or UV bonding or thermocompression bonding is performed on the four side areas outside the effective pixel while being supported by the spacer outside the effective pixel range used for imaging. Also good.

FIG. 12 is an explanatory diagram showing a region division pattern of the polarizing filter layer 222 and the spectral filter layer 223 of the optical filter 205 according to the present embodiment.
In the polarizing filter layer 222 and the spectral filter layer 223, two types of regions, a first region and a second region, are arranged corresponding to one photodiode 206A on the image sensor 206, respectively. As a result, the amount of light received by each photodiode 206A on the image sensor 206 is determined as polarization information, spectral information, or the like depending on the types of regions of the polarizing filter layer 222 and the spectral filter layer 223 through which the received light is transmitted. Can be acquired.

  In the present embodiment, the image sensor 206 is described on the assumption of an image sensor for monochrome images, but the image sensor 206 may be configured with a color image sensor. In the case of the color image sensor, the light transmission characteristics of the polarizing filter layer 222 and the spectral filter layer 223 may be adjusted according to the characteristics of the color filter attached to each image pickup pixel of the color image sensor.

[Configuration example 1 of optical filter]
Here, a configuration example of the optical filter 205 in the present embodiment (hereinafter, this configuration example will be referred to as “configuration example 1”) will be described. In the following description of the optical filter 205, the pre-filter 210 of the optical filter 205 is omitted, and only the post-filter 220 is described.
FIG. 13 is an explanatory diagram illustrating the content of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the optical filter 205 in the first configuration example. .
FIG. 14A is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line AA shown in FIG.
FIG. 14B is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line BB shown in FIG.

  As shown in FIGS. 14A and 14B, in the optical filter 205 of Configuration Example 1, after forming the polarizing filter layer 222 on the transparent filter substrate 221, the spectral filter layer 223 is formed thereon. Thus, a laminated structure is obtained. The polarizing filter layer 222 has a wire grid structure, and the upper surface in the stacking direction (the lower side surface in FIG. 14) is an uneven surface. If an attempt is made to form the spectral filter layer 223 directly on such an uneven surface, the spectral filter layer 223 is formed along the uneven surface, resulting in uneven thickness of the spectral filter layer 223 and obtaining the original spectral performance. There may not be. Therefore, in the optical filter 205 of the present embodiment, the upper surface side in the stacking direction of the polarizing filter layer 222 is filled with a filler and planarized, and then the spectral filter layer 223 is formed thereon.

  As the filler, any material that does not hinder the function of the polarizing filter layer 222 whose uneven surface is flattened by the filler may be used. In this embodiment, a material having no polarization function is used. In addition, as the planarization treatment with the filler, for example, a method of applying the filler by a spin-on-glass method can be suitably employed, but the present invention is not limited to this.

In the first configuration example, the first area of the polarization filter layer 222 is a vertical polarization area that selects and transmits only a vertical polarization component that vibrates parallel to the column (vertical direction) of the imaging pixels of the image sensor 206. The second region of the filter layer 222 is a horizontal polarization region that selects and transmits only the horizontal polarization component that vibrates in parallel with the row (horizontal direction) of the imaging pixels of the image sensor 206.
The first region of the spectral filter layer 223 is a red spectral region that selectively transmits only light in the red wavelength band (specific wavelength band) included in the usable wavelength band that can be transmitted through the polarizing filter layer 222. The second region of the filter layer 223 is a non-spectral region that transmits light without performing wavelength selection. And in this structural example 1, as enclosed with the dashed-dotted line in FIG. 13, it is a picked-up image by a total of four image pick-up pixels (4 image pick-up pixels of code | symbol a, b, e, and f) which adjoin 2 lengthwise and 2 width. One image pixel of data is constructed.

In the imaging pixel a shown in FIG. 13, light that has passed through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel a receives light P / R in the red wavelength band (indicated by symbol R in FIG. 13) of the vertically polarized component (indicated by symbol P in FIG. 13).
In addition, in the imaging pixel b illustrated in FIG. 13, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel b receives light P / C of non-spectral light (indicated by symbol C in FIG. 13) in the vertical polarization component P.
In addition, in the imaging pixel e illustrated in FIG. 13, light transmitted through the horizontal polarization region (second region) in the polarization filter layer 222 of the optical filter 205 and the non-spectral region (second region) of the spectral filter layer 223 is received. . Therefore, the imaging pixel e receives the non-spectral C light S / C in the horizontal polarization component (indicated by symbol S in FIG. 13).
In the imaging pixel f illustrated in FIG. 13, light transmitted through the vertical polarization region (first region) in the polarization filter layer 222 of the optical filter 205 and the red spectral region (first region) of the spectral filter layer 223 is received. Therefore, the imaging pixel f receives the light P / R in the red wavelength band R in the vertical polarization component P, similarly to the imaging pixel a.

  With the above configuration, according to the present configuration example 1, one image pixel for the vertical polarization component image of red light is obtained from the output signals of the imaging pixel a and the imaging pixel f, and non-spectral is obtained from the output signal of the imaging pixel b. One image pixel for the vertical polarization component image is obtained, and one image pixel for the non-spectral horizontal polarization component image is obtained from the output signal of the imaging pixel e. Therefore, according to the present configuration example 1, three types of captured image data, that is, a vertical polarization component image of red light, a non-spectral vertical polarization component image, and a non-spectral horizontal polarization component image can be obtained by a single imaging operation. it can.

In these captured image data, the number of image pixels is smaller than the number of captured pixels, but a generally known image interpolation technique may be used to obtain a higher resolution image. For example, when trying to obtain a vertical polarization component image of red light having a higher resolution, the vertical polarization of the red light received by the imaging pixels a and f for the imaging pixel a and the imaging pixel f. For the image pixel corresponding to the imaging pixel b using the information of the component P as it is, for example, the average value of the imaging pixels a, c, f, and j surrounding the periphery is used as information on the vertical polarization component of the red light of the image pixel. Use as
Further, when obtaining a non-spectral horizontal polarization component image having a higher resolution, the information of the non-spectral horizontal polarization component S received by the imaging pixel e is used as it is for the image pixel corresponding to the imaging pixel e. For the image pixels corresponding to the imaging pixels a, b, and f, an average value of the imaging pixel e and the imaging pixel g that receives the non-spectral horizontal polarized light component around it is used, or the same as the imaging pixel e. Or a value may be used.

  The vertically polarized component image of red light obtained in this way can be used, for example, for identifying a tail lamp. Since the vertical polarization component image of the red light has the horizontal polarization component S cut off, the red light is horizontal, such as red light reflected on the road surface or red light (reflection light) from the dashboard of the vehicle 100 or the like. It is possible to obtain a red image in which a disturbance factor due to red light having a strong polarization component S is suppressed. Therefore, the recognition rate of the tail lamp is improved by using the vertical polarization component image of the red light for the identification of the tail lamp.

  The non-spectral vertical polarization component image can be used, for example, for identifying a white line or a headlamp of an oncoming vehicle. In the non-spectral horizontal polarization component image, since the horizontal polarization component S is cut off, white light such as a headlamp or a streetlight reflected on the road surface, white light from a dashboard in the interior of the vehicle 100, etc. (reflection light) A non-spectral image in which disturbance factors due to white light with a strong horizontal polarization component S are suppressed can be obtained. Therefore, the recognition rate is improved by using the non-spectral vertically polarized component image for identifying the white line and the headlamp of the oncoming vehicle. In particular, it is generally known that the reflected light from the water surface covering the road surface has a large amount of horizontal polarization component S in a rainy road. Therefore, by using the non-spectral vertical polarization component image for white line identification, it becomes possible to appropriately identify the white line under the water surface in the rainy road, and the recognition rate is improved.

  In addition, if a comparison image using pixel values as index values obtained by comparing pixel values between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image is used, as described later, the metal in the imaging region It is possible to accurately identify an object, a wet and dry state of a road surface, a three-dimensional object in an imaging region, and a white line on a rainy road. As a comparison image used here, for example, a difference image in which a pixel value is a difference value between a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image, or a pixel value between these images. Use a ratio image with the ratio as the pixel value, or a differential polarization degree image with the ratio of the difference value of the pixel values between these images to the sum of the pixel values between these images (difference polarization degree) as the pixel value can do.

  In the optical filter 205 of the present embodiment, the rear-stage filter 220 having the polarizing filter layer 222 and the spectral filter layer 223 divided as shown in FIG. 13 is divided into upper and lower parts as shown in FIG. Although it is provided on the image sensor 206 side with respect to the front-stage filter 210, the front-stage filter 210 may be provided on the image sensor 206 side with respect to the rear-stage filter 220. Further, as shown in FIG. 13, the region-divided polarizing filter layer 222 and spectral filter layer 223 are not necessarily required for raindrop detection. Therefore, these polarizing filter layer 222 and spectral filter layer 223 are used for raindrop detection. You may comprise so that it may not provide in the location corresponding to the image area | region 214, ie, the location facing the infrared light transmission filter area | region 212 of the front | former stage filter 210. FIG.

  In the first configuration example, an aperture limiting unit for limiting the amount of received light is provided corresponding to the non-spectral region of the spectral filter layer 223. That is, the non-spectral region of the spectral filter layer 223 is configured to have a lower transmittance than the red spectral region. Thereby, in this configuration example 1, the non-spectral vertical polarization component image and the non-spectral horizontal polarization component image obtained through the non-spectral region of the spectral filter layer 223 are transmitted through the red spectral region of the spectral filter layer 223. Thus, it is generated with a smaller amount of received light than the vertically polarized component image of red light obtained in this way.

  As a configuration for limiting the amount of light received through the non-spectral region of the spectral filter layer 223 (that is, a configuration for reducing the transmittance), as shown in FIG. 15, corresponding to the non-spectral region of the spectral filter layer 223. A configuration in which a circular wire grid structure is formed in the central portion of the imaging pixel of the polarizing filter layer 222 as a transmittance adjusting layer, and the peripheral portion thereof is a solid aluminum film. According to this configuration, since the solid aluminum film is shielded from light, the amount of light transmitted through the non-spectral region of the spectral filter layer 223 is limited by the width (aperture ratio) of the region forming the wire grid structure. Can do. In addition, the area | region shape which forms a wire grid structure is not restricted to the circular shape shown in FIG. 15, For example, a substantially square shape as shown in FIG. 16 may be sufficient. As shown in FIG. 16, in the case of a shape having a corner, it is easier to obtain a shape dimension by etching or the like if the corner has R.

  For example, the polarizing filter layer 222 having a wire grid structure is generally manufactured by forming an aluminum film uniformly on the filter substrate 221 and then partially removing the aluminum film by etching or the like to obtain a wire grid structure. is there. In the case where an aluminum light-shielding region is provided in the periphery of the wire grid structure to limit the opening as in the configuration example 1, by processing so as to leave the aluminum film in the periphery when the wire grid structure is formed. The opening can be limited. Therefore, the manufacturing process can be simplified as compared with the case where the aperture limiting process is performed separately from the polarizing filter layer 222.

Of course, an aperture limiting layer (transmittance adjusting layer) as shown in FIG. 17 may be provided separately from the polarizing filter layer 222. In this case, a wire grid structure is not formed in the center of the imaging pixel of the aperture limiting layer, and the aperture is an aperture that transmits light as it is.
Further, the light-shielding region for limiting the opening is not limited to the above-described reflective film (light reflecting material) such as an aluminum film, and may be formed of, for example, a film that absorbs light (light absorbing material). . For example, as shown in FIG. 18, a light shielding region may be formed by a solid film of black resist. Also in this case, the opening is not limited to the circular shape shown in FIG. 18 and may be, for example, a substantially square shape as shown in FIG. As shown in FIG. 19, in the case of a shape having a corner, it is easier to obtain a shape dimension by etching or the like if the corner has R.
Further, the number of openings that transmit light is not necessarily one for one imaging pixel, and a plurality of openings or wire grid structure regions may be formed for one imaging pixel. Further, the number of light shielding regions is not necessarily one for one imaging pixel, and a plurality of light shielding portions may be formed for one imaging pixel. In particular, the light shielding region does not need to be provided in the peripheral portion of the imaging pixel. For example, as shown in FIG. 20, a solid aluminum film may be discretely arranged in a wire grid structure.

  In the first configuration example, a vertical polarization component image of red light obtained through a high-transmittance red spectral region that is not aperture-limited, and a non-spectral region that is aperture-limited and has low transmittance. Three types of captured image data can be obtained: a non-spectral vertical polarization component image and a non-spectral horizontal polarization component image. In the first configuration example, the preceding vehicle is detected from the result of identifying the tail lamp using the vertical polarization component image of red light, and the headlamp is mounted using the non-spectral vertical polarization component image and the non-spectral horizontal polarization component image. An oncoming vehicle is detected from the identified result. Tail lamps and head lamps are usually a set of two that are separated by a certain distance in the horizontal direction. Therefore, when detecting the preceding vehicle or the oncoming vehicle, when the image parts of the two tail lamps or the two head lamps in the captured image are separated by a certain distance, the one set of tail lamps or one set is used. Are recognized as those of the preceding vehicle or the oncoming vehicle. At this time, since the headlamp is a light source that has a larger amount of light than the tail lamp, if the light sensitivity is set so that the tail lamp can receive light appropriately, the received light quantity of the head lamp is saturated and recognized as one head lamp. The image area to be enlarged is enlarged, and two headlamp image areas that are originally recognized apart from each other become an integrated image area, and the image area of the headlamp cannot be properly recognized, and the recognition rate of the oncoming vehicle is increased. descend. On the other hand, if the light receiving sensitivity is set so that the light from the headlamp can be received properly, the amount of light received by the tail lamp, which is a light source with a small amount of light, is insufficient, and the tail lamp image area cannot be recognized properly. Recognition rate decreases.

  According to the first configuration example, the non-spectral vertical polarization component image and the non-spectral horizontal polarization component image used for identification of the headlamps are those in which the received light amount is limited by the aperture limitation described above. Therefore, even if the light reception sensitivity is set according to the tail lamp identified using the vertical polarization component image of red light whose light reception amount is not limited, the situation where the light reception amount of the headlamp is saturated is suppressed, and each head The image area of the lamp can be individually identified, and the decrease in the recognition rate of the oncoming vehicle is suppressed.

  In addition, for example, by identifying the headlamp and the tail lamp from the picked-up images picked up separately by switching the light receiving sensitivity, the head lamp and the tail lamp can be identified at the same time. However, in this case, a control mechanism such as switching the light receiving sensitivity is required, and the frame rate of the captured image data drops to half. On the other hand, according to the present configuration example 1, it is possible to achieve both headlamp and taillamp identification without such problems.

[Configuration example 2 of optical filter]
Next, another configuration example of the optical filter 205 in the present embodiment (hereinafter referred to as “configuration example 2”) will be described.
FIG. 21 is an explanatory diagram illustrating the contents of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the optical filter 205 in the second configuration example. .
FIG. 22A is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line AA shown in FIG.
FIG. 22B is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line BB shown in FIG.

  In the above configuration example 1, the first region of the spectral filter layer 223 is a red spectral region that selectively transmits only light in the red wavelength band, but in this configuration example 2, it can transmit through the polarizing filter layer 222. This is a cyan spectral region in which only light in the cyan wavelength band (indicated by reference numeral Cy in FIG. 21) included in the used wavelength band is selected and transmitted. Other configurations are the same as those in the first configuration example.

  According to the second configuration example, one image pixel for the vertical polarization component image of cyan light is obtained from the output signals of the imaging pixel a and the imaging pixel f, and the non-spectral vertical polarization component image is obtained from the output signal of the imaging pixel b. One image pixel is obtained, and one image pixel for the non-spectral horizontal polarization component image is obtained from the output signal of the imaging pixel e. Therefore, according to the second configuration example, it is possible to obtain three types of captured image data of a cyan light vertical polarization component image, a non-spectral vertical polarization component image, and a non-spectral horizontal polarization component image by a single imaging operation. it can.

In the second configuration example, the three types of captured image data obtained in this way improve the recognition rate of various identification objects (tail lamp, headlamp, white line, etc.) as in the first configuration example.
Furthermore, according to the second configuration example, it is possible to use a comparative image between a cyan light vertical polarization component image and a non-spectral vertical polarization component image. By using such a comparison image, the tail lamp has high accuracy. Identification becomes possible. That is, the amount of light received from the tail lamp is small in the imaging pixels that have passed through the cyan spectral region, and the light reception amount is large in the imaging pixels that have passed through the non-spectral region. Therefore, in order to reflect this difference, if the comparison image of the vertical polarization component image of cyan light and the non-spectral vertical polarization component image is generated, the contrast between the tail lamp and the surrounding background portion can be increased. Recognition rate is improved.

  Further, in the present configuration example 2, instead of the red spectral region using the red filter of the above configuration example 1, a cyan spectral region using a cyan filter that transmits only cyan light is used. Compared with the case of 1, the discriminating ability between the tail lamp when the preceding vehicle is close to the host vehicle and the head lamp of the oncoming vehicle is higher. When the red spectral region is used as in the configuration example 1, the tail light of the preceding vehicle close to the host vehicle may become saturated as the amount of light received through the red spectral region becomes larger as the light sensitivity is lost. Therefore, there is a possibility that the recognition rate of the tail lamp of the preceding vehicle close to the host vehicle is lowered. On the other hand, when the cyan spectral region is used as in Configuration Example 2, the amount of light received through the cyan spectral region is not saturated with respect to the tail lamp of the preceding vehicle close to the own vehicle, and the preceding vehicle close to the own vehicle is not saturated. It can suppress that the recognition rate of a tail lamp falls.

[Configuration Example 3 of Optical Filter]
Next, still another configuration example of the optical filter 205 in the present embodiment (hereinafter referred to as “configuration example 3”) will be described.
FIG. 23 is an explanatory diagram illustrating the content of information (information about each imaging pixel) corresponding to the amount of light received by each photodiode 206A on the image sensor 206 through the optical filter 205 in the third configuration example. .
FIG. 24A is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line AA shown in FIG.
FIG. 24B is a cross-sectional view schematically showing the optical filter 205 and the image sensor 206 cut along the line BB shown in FIG.

In the configuration example 1, the spectral filter layer 223 is provided in the optical filter 205, but in the configuration example 3, the spectral filter layer 223 is not provided in the optical filter 205.
Further, in the above configuration example 1, the polarization filter layer 222 of the optical filter 205 corresponds to three imaging pixels among the one image pixel configured by 2 × 2 imaging pixels, and the remaining one imaging The area was divided so that the horizontal polarization area corresponds to the pixel. On the other hand, in the configuration example 3, the vertical polarization region and the horizontal polarization region are alternately arranged in the imaging pixel unit in the horizontal direction and the vertical direction. Therefore, two image pickup pixels of one image pixel constituted by 2 × 2 image pickup pixels correspond to the vertical polarization region, and the remaining two image pickup pixels correspond to the horizontal polarization region.

  And in this structural example 3, as shown in FIG. 23, the opening restriction | limiting part for restrict | limiting a light reception amount is provided in one of the two vertical polarization areas corresponding to 1 image pixel, and 1 image pixel Is provided with an aperture limiting portion for limiting the amount of received light in one of the two horizontal polarization regions corresponding to. Therefore, according to the third configuration example, a non-spectral vertical polarization component image generated with a small amount of received light based on the output signal of the imaging pixel a is obtained, and with a small amount of received light based on the output signal of the imaging pixel b. A generated non-spectral horizontal polarization component image is obtained, and a non-spectral vertical polarization component image generated with a large amount of received light is obtained based on the output signal of the imaging pixel e. Based on the output signal of the imaging pixel f Thus, a non-spectral horizontal polarization component image generated with a large amount of received light can be obtained. That is, two non-spectral vertical polarization component images having different light reception amounts (exposure amounts) and two non-spectral horizontal polarization component images having different light reception amounts (exposure amounts) can be obtained by one imaging operation. .

  In the third configuration example, since the spectral filter layer 223 is not provided in the optical filter 205, spectral information cannot be obtained. However, as described above, two non-spectral vertically polarized light beams having different received light amounts (exposure amounts) are used. Four types of images are obtained: a component image and two non-spectral horizontally polarized component images having different received light amounts (exposure amounts). Even in this case, it is possible to detect the preceding vehicle and the oncoming vehicle by identifying the tail lamp and the head lamp from the difference in luminance.

  Further, according to the configuration example 3, a differential polarization degree image with a small amount of received light (exposure amount) can be obtained from a non-spectral vertical polarization component image and a horizontal polarization component image with a small amount of received light (exposure amount). In addition, a differential polarization degree image with a large amount of received light (exposure amount) can be obtained from a non-spectral vertical polarization component image and a horizontal polarization component image with a large amount of received light (exposure amount) by one photographing. Therefore, even when a plurality of types of light source bodies that can be identified using the differential polarization degree image exist in the imaging region and the light amounts of these light source bodies are greatly different, according to the present configuration example 3, By using two types of differential polarization degree images picked up with a light reception amount (exposure amount) suitable for the light source body, it is possible to appropriately identify these light source bodies continuously at shorter time intervals.

[Configuration example 4 of optical filter]
Next, still another configuration example of the optical filter 205 in the present embodiment (hereinafter referred to as “configuration example 4”) will be described.
As described above, the polarizing filter layer 222 provided in the post-stage filter 220 of the optical filter 205 selects only the vertical polarization region (first region) for transmitting only the vertical polarization component P and the horizontal polarization component S. And is divided into horizontal polarization areas (second areas) to be transmitted in units of imaging pixels. Accordingly, a vertical polarization component image in which the horizontal polarization component S is cut can be obtained based on the image data of the imaging pixel that receives light transmitted through the vertical polarization region. In addition, a horizontal polarization component image in which the vertical polarization component P is cut can be obtained from image data of an imaging pixel that receives light transmitted through the horizontal polarization region.

  If the surface of the windshield 105 is flat, reflection on the windshield 105 is performed by appropriately setting the polarization direction (transmission axis) of the vertical polarization region and the horizontal polarization region with respect to the surface of the windshield 105. It is possible to obtain a vertically polarized component image and a horizontally polarized component image that are appropriately cut. However, in general, the windshield 105 of an automobile is not only inclined downward toward the front to improve aerodynamic characteristics, but is also greatly curved backward from the center to both ends in the left-right direction. . Therefore, if the polarization direction (transmission axis) of the vertical polarization region or the horizontal polarization region in the polarization filter layer 222 of the optical filter 205 is uniform in any region, for example, in the central portion of the captured image, the image from the windshield 105 is reflected. Even if the image can be cut appropriately, the reflection from the windshield 105 may not be cut appropriately at the edge of the captured image.

FIG. 25 is an explanatory diagram illustrating the longitudinal direction of a metal wire having a wire grid structure in the polarizing filter layer 222 of the optical filter 205 according to the fourth configuration example.
The area division configuration of the polarizing filter layer 222 and the spectral filter layer 223 of the configuration example 4 is the same as that of the configuration example 1 described above. However, in the configuration example 4, the polarization direction of the vertical polarization region of the polarizing filter layer 222 ( The transmission axis is not uniform. Specifically, as shown in FIG. 25, the angle between the polarization direction (transmission axis) and the vertical direction in the vertical polarization region closer to the horizontal end of the polarizing filter layer 222 in accordance with the curvature of the windshield 105. The vertical polarization region of the polarizing filter layer 222 is formed so that the That is, the polarizing filter layer 222 of the configuration example 4 is configured such that the angle between the longitudinal direction of the metal wire of the wire grid structure and the horizontal direction becomes larger in the vertically polarized region closer to the horizontal end. In the present embodiment, since the vertical polarization region is formed with a wire grid structure, it is possible to form a large number of regions having different polarization directions in such a minute unit as an imaging pixel.

[Details of each part of the optical filter]
Next, details of each part of the post-stage filter 220 in the optical filter 205 will be described.
The filter substrate 221 is made of a transparent material, such as glass, sapphire, or quartz, that can transmit light in a use band (in this embodiment, visible light region and infrared region). In the present embodiment, glass, particularly inexpensive and durable quartz glass (refractive index 1.46) or Tempax glass (refractive index 1.51) can be suitably used.

  The polarizing filter layer 222 formed on the filter substrate 221 includes a polarizer formed with a wire grid structure as shown in FIG. The wire grid structure is a structure in which metal wires (conductor lines) made of metal such as aluminum and extending in a specific direction are arranged at a specific pitch. By setting the wire pitch of the wire grid structure to a sufficiently small pitch (for example, 1/2 or less) compared to the wavelength band of incident light, the electric field vector component light that vibrates in parallel to the longitudinal direction of the metal wire can be obtained. Since most of the electric field vector component that reflects and vibrates in the direction perpendicular to the longitudinal direction of the metal wire is transmitted, it can be used as a polarizer for producing a single polarized light.

  In general, a polarizer having a wire grid structure has an extinction ratio that increases as the cross-sectional area of the metal wire increases. Further, the transmittance of a metal wire having a predetermined width or more with respect to the period width decreases. Moreover, when the cross-sectional shape orthogonal to the longitudinal direction of the metal wire is a taper shape, the wavelength dispersion of transmittance and polarization degree is small in a wide band, and high extinction ratio characteristics are exhibited.

In the present embodiment, the polarizing filter layer 222 is formed with a wire grid structure, thereby providing the following effects.
The wire grid structure can be formed using a widely known semiconductor manufacturing process. Specifically, after depositing an aluminum thin film on the filter substrate 221, patterning is performed, and the sub-wavelength uneven structure of the wire grid may be formed by a technique such as metal etching. By such a manufacturing process, it is possible to adjust the longitudinal direction of the metal wire, that is, the polarization direction (polarization axis) corresponding to the imaging pixel size (several μm level) of the image sensor 206. Therefore, as in the present embodiment, it is possible to create the polarizing filter layer 222 in which the longitudinal direction of the metal wire, that is, the polarization direction (polarization axis) is different for each imaging pixel.
In addition, since the wire grid structure is made of a metal material such as aluminum, it has an advantage that it can be suitably used even in a high temperature environment such as a vehicle interior that has excellent heat resistance and is likely to become high temperature.

  The filler 224 used for planarizing the upper surface of the polarizing filter layer 222 in the stacking direction is filled in the recesses between the metal wires of the polarizing filter layer 222. As the filler 224, an inorganic material having a refractive index lower than or equal to that of the filter substrate 221 can be suitably used. In addition, the filler 224 in the present embodiment is formed so as to cover the upper surface in the stacking direction of the metal wire portion of the polarizing filter layer 222.

A specific material of the filler 224 is preferably a low refractive index material whose refractive index is as close as possible to the refractive index of air (refractive index = 1) so as not to deteriorate the polarization characteristics of the polarizing filter layer 222. . For example, a porous ceramic material formed by dispersing fine pores in ceramics is preferable. Specifically, porous silica (SiO ₂ ), porous magnesium fluoride (MgF), porous alumina (Al ₂ O) ₃ ). The degree of these low refractive indexes is determined by the number and size of pores in the ceramic (porosity). When the main component of the filter substrate 221 is made of silica crystal or glass, porous silica (n = 1.2-1.26) can be preferably used.

As a method for forming the filler 224, an SOG (Spin On Glass) method can be suitably used. Specifically, a solvent in which silanol (Si (OH) ₄ ) is dissolved in alcohol is spin-coated on the polarizing filter layer 222 formed on the filter substrate 221, and then the solvent component is volatilized by heat treatment. It is formed in such a way that it itself undergoes a dehydration polymerization reaction.

The polarizing filter layer 222 has a sub-wavelength sized wire grid structure, has low mechanical strength, and a metal wire is damaged by a slight external force. Since it is desired that the optical filter 205 of the present embodiment is disposed in close contact with the image sensor 206, there is a possibility that the optical filter 205 and the image sensor 206 come into contact with each other in the manufacturing stage. In the present embodiment, the upper surface in the stacking direction of the polarization 1 filter layer 222, that is, the surface on the image sensor 206 side is covered with the filler 224, so that the situation where the wire grid structure is damaged when contacting the image sensor 206 is suppressed. The
Further, by filling the filler 224 into the recesses between the metal wires in the wire grid structure of the polarizing filter layer 222 as in this embodiment, it is possible to prevent foreign matter from entering the recesses.

  In this embodiment, the spectral filter layer 223 laminated on the filler 224 is not provided with a protective layer like the filler 224. According to the experiments by the present inventors, even if the spectral filter layer 223 is in contact with the image sensor 206, damage that affects the captured image did not occur. The layer is omitted. Further, the height of the metal wire (convex portion) of the polarizing filter layer 222 is as low as half or less of the used wavelength, while the height of the filter layer portion (convex portion) forming the red spectral region or cyan spectral region of the spectral filter layer 223 is high. The height is about the same as the used wavelength to several times as high. As the thickness of the filler 224 increases, it becomes more difficult to ensure the flatness of the upper surface, which affects the characteristics of the optical filter 205, so that there is a limit to increasing the thickness of the filler 224. Therefore, in this embodiment, the spectral filter layer 223 is not covered with a filler.

  The filter layer portion that forms the red spectral region or the cyan spectral region in the spectral filter layer 223 of the present embodiment is made of a multilayer film structure in which a thin film having a high refractive index and a thin film having a low refractive index are alternately stacked. . According to such a multilayer film structure, the degree of freedom in setting the spectral transmittance is high by utilizing the interference of light, and by stacking thin films in multiple layers, it is 100 for a specific wavelength (for example, a wavelength band other than red). It is also possible to achieve reflectivity close to%. In the present embodiment, since the use wavelength range of the captured image data is a substantially visible light wavelength band (wavelength band of visible light and infrared light), the image sensor 206 having sensitivity in the use wavelength range is selected, A cut filter as shown in FIG. 27 may be formed in which the transmission wavelength range of the multilayer film portion is set to, for example, 600 nm or more and the other wavelength bands are reflected.

Such a cut filter can be obtained by fabricating a multilayer film having a configuration of “substrate / (0.125L0.25H0.125L) p / medium A” in order from the lower side of the optical filter 205 in the stacking direction. it can. The “substrate” here means the filler 224 described above. Further, “0.125L” is a film thickness marking method of a low refractive index material (for example, SiO ₂ ) in which nd / λ is 1 L. Therefore, the film of “0.125L” has an optical path of 1/8 wavelength. It means that the film is of a low refractive index material having such a long film thickness. “N” is a refractive index, “d” is a thickness, and “λ” is a cutoff wavelength. Similarly, “0.25H” is obtained by setting nd / λ to 1H in the film thickness marking method of a high refractive index material (for example, TiO ₂ ). Therefore, the film of “0.25H” has a quarter wavelength. It means a film of a high refractive index material having a film thickness that becomes an optical path length. “P” indicates the number of times the film combination shown in parentheses is repeated (laminated), and the more “p”, the more the influence of ripple and the like can be suppressed. The medium A is intended for air or a resin or adhesive for tight bonding with the image sensor 206. Also,

Further, the filter layer portion forming the red spectral region or the cyan spectral region in the spectral filter layer 223 is a bandpass filter having a filter characteristic as shown in FIG. 28 having a transmission wavelength range of 600 to 700 nm. May be. With such a bandpass filter, it is possible to distinguish the near-infrared region and red region on the longer wavelength side than red. Such a band pass filter is, for example, “substrate / (0.125L0.5M0.125L) p (0.125L0.5H0.125L) q (0.125L0.5M0.125L) r / medium A”. It can be obtained by producing a multilayer film having the structure. As described above, the spectral filter layer 223 having high weather resistance can be realized by using titanium dioxide (TiO ₂ ) as the high refractive index material and silicon dioxide (SiO ₂ ) as the low refractive index material.

  An example of a method for producing the spectral filter layer 223 of the present embodiment will be described. First, the above-described multilayer film is formed on the layer of the filler 224 formed on the filter substrate 221 and the polarizing filter layer 222. As a method for forming such a multilayer film, a well-known method such as vapor deposition may be used. Subsequently, the multilayer film is removed from the portion corresponding to the non-spectral region. As this removal method, a general lift-off processing method may be used. In the lift-off processing method, a pattern opposite to the target pattern is formed on a layer of the filler 224 in advance with a metal, a photoresist, or the like, a multilayer film is formed thereon, and then non-spectral The multilayer film corresponding to the region is removed together with the metal and photoresist.

  In the present embodiment, since the multilayer filter structure is employed as the spectral filter layer 223, there is an advantage that the degree of freedom in setting the spectral characteristics is high. In general, a color filter used in a color sensor or the like is formed of a resist agent, but it is difficult to control spectral characteristics with such a resist agent as compared with a multilayer film structure. In this embodiment, since the multilayer structure is adopted as the spectral filter layer 223, the spectral filter layer 223 optimized for the wavelength of the tail lamp can be formed.

[Light distribution control of headlamps]
Hereinafter, the light distribution control of the headlamp in the present embodiment will be described.
In the light distribution control of the headlamp in this embodiment, the captured image data captured by the imaging apparatus 200 is analyzed to identify the tail lamp and the head lamp of the vehicle, the preceding vehicle is detected from the identified tail lamp, and the identified head The oncoming vehicle is detected from the ramp. Then, the driver of the host vehicle 100 can be prevented from being dazzled by avoiding the strong light from the headlamp of the host vehicle 100 entering the eyes of the driver of the preceding vehicle or the oncoming vehicle, and the driver's view of the host vehicle 100 is secured. In order to realize the above, the switching of the high beam and the low beam of the headlamp 104 is controlled, or partial light shielding control of the headlamp 104 is performed.
In the following description, a case in which the configuration filter 1 described above is used as the post-filter 220 of the optical filter 205 will be described.

  In the headlamp light distribution control of the present embodiment, among the information that can be acquired from the imaging unit 101, the intensity (brightness information) of light emitted from each point (light source body) in the imaging region, the headlamp, The distance (distance information) between the light source body (other vehicle) such as the tail lamp and the own vehicle, the spectral information by comparing the red component and the white component (non-spectral) of the light emitted from each light source body, the horizontal polarization component of the white component Information of a white component with a horizontal polarization component cut off, and vertical polarization component information of a red component with a horizontal polarization component cut off are used.

  The brightness information will be described. When the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are imaged by the imaging device 200, the detected object is detected on the captured image data. One headlamp of the oncoming vehicle is projected brightest, and the tail lamp of the preceding vehicle, which is one of the detection objects, is projected darker than that. In addition, when the reflector is displayed in the captured image data, the reflector is not a light source that emits light by itself, but merely a bright image that is reflected by reflecting the headlamp of the host vehicle, so that it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the reflector is gradually observed on the image sensor 206 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, primary identification of the two types of detection objects (head lamp and tail lamp) and the reflector is possible.

  Further, the distance information will be described. Most headlamps and tail lamps have a pair of left and right pair lamps. Therefore, the distance between the headlamps and tail lamps (that is, other vehicles) and the host vehicle is utilized using the characteristics of this configuration. Can be obtained. The pair of left and right lamps that are paired are displayed close to each other at the same height position on the captured image data captured by the image capturing apparatus 200, and the widths of the lamp image areas that project the lamps are substantially the same. The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, lamp image areas satisfying the conditions can be identified as pair lamps. At a long distance, the left and right lamps constituting the pair lamp cannot be distinguished and recognized as a single lamp.

  When a pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp and tail lamp which are the pair lamp structure. That is, the distance between the left and right headlamps and the distance between the left and right tail lamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the imaging lens 204 in the imaging device 200 is known, the distance w1 between the two lamp image areas respectively corresponding to the left and right lamps on the image sensor 206 of the imaging device 200 is calculated from the captured image data. Accordingly, the distance x between the light source of the headlamp or taillamp which is the pair lamp configuration and the host vehicle can be obtained by simple proportional calculation (x = f × w0 / w1). If the distance x calculated in this way is within an appropriate range, the two lamp image areas used for the calculation can be identified as headlamps and taillamps of other vehicles. Therefore, by using this distance information, the identification accuracy of the head lamp and tail lamp, which are detection objects, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, the imaging pixels a on the image sensor 206 that receive red light (vertical polarization component) P / R from the captured image data captured by the imaging device 200. By extracting only pixel data corresponding to c, f, h, etc., it is possible to generate a red image in which only the red component in the imaging region is projected. Therefore, when there is an image area having a luminance equal to or higher than a predetermined luminance in the red image, the image area can be identified as a tail lamp image area in which a tail lamp is projected.

  Further, only pixel data corresponding to the imaging pixels b and d on the image sensor 206 that receives the vertical polarization component P / C of white light (non-spectral) is extracted from the captured image data captured by the imaging apparatus 200. Thus, a monochrome luminance image (vertical polarization component) within the imaging region can be generated. Therefore, the luminance ratio (red luminance ratio) between the image area on the red image and the image area on the monochrome luminance image corresponding to the image area can be calculated. By using this red luminance ratio, it is possible to grasp the ratio of the relative red component contained in the light from the object (light source body) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the identification accuracy of the tail lamp is improved by using this red luminance ratio.

  Further, the polarization information will be described. In the present embodiment, as described above, imaging on the image sensor 206 that receives the vertical polarization component P / C of white light (non-spectral) from the captured image data captured by the imaging apparatus 200. Extracting pixel data corresponding to the pixels b, d, and the like, and pixel data corresponding to the imaging pixels e, g, etc. on the image sensor 206 that receives the horizontal polarization component S / C of white light (non-spectral), For each image pixel, a comparison image in which pixel values (luminance) between these image data are compared can be obtained. Specifically, for example, a difference image having a pixel value as a difference value (SP) between a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) is compared with a comparison image. Can be obtained as According to such a comparative image, an image area (headlamp image area) of direct light directly incident on the imaging device 200 from the headlamp and an incident on the imaging device 200 after being reflected from the water surface of the rain road from the headlamp. The contrast with the image area of indirect light can be increased, and the identification accuracy of the headlamp is improved.

  In particular, as a comparative image, a ratio image with a ratio (S / P) of a vertical polarization component P of white light (non-spectral) and a horizontal polarization component S of white light (non-spectral) as a pixel value, or a differential polarization degree A differential polarization degree image having ((S−P) / (S + P)) as a pixel value can be suitably used. In general, it is known that the light reflected by a horizontal mirror such as a water surface always has a high horizontal polarization component. In particular, the ratio (S / P) between the horizontal polarization component S and the vertical polarization component P and the differential polarization degree. When ((S−P) / (S + P)) is taken, it is known that the ratio and the differential polarization degree become maximum at a specific angle (Brewster angle). In rainy roads, water is applied to the asphalt surface, which is a scattering surface, in a state close to a mirror surface, so that the headlamp reflected light from the road surface is stronger in the horizontal polarization component S. Therefore, the image area of the headlamp reflected light from the road surface has a large pixel value (luminance) in the ratio image and the differential polarization degree image. On the other hand, since the direct light from the headlamp is basically non-polarized light, the pixel value (luminance) is small in the ratio image and the differential polarization degree image. Due to this difference, it is possible to appropriately exclude the headlamp reflected light from the rain road surface having the same amount of light as the direct light from the headlamp, and distinguish the direct light from the headlamp from such headlamp reflected light. be able to.

  FIG. 29 shows an image of the direct light from the headlamp and the reflected light (reflected light) on the rainy road surface of the headlamp on the rainy day using the imaging device 200, and the respective differential polarization degrees ((SP)). This is a histogram when / (S + P)) is calculated. The vertical axis in FIG. 29 indicates the frequency, and is normalized to 1 here. The horizontal axis in FIG. 29 is the difference polarization degree ((SP) / (S + P)). As can be seen from FIG. 29, the distribution of the reflected light on the rainy road surface of the headlamp is shifted to the side where the horizontal polarization component S is relatively large (the right side in the figure) compared to the direct light of the headlamp. I understand.

FIG. 30 is a schematic diagram illustrating an example in which the imaging apparatus 200 captures an image of a situation in which both the preceding vehicle and the oncoming vehicle exist at substantially the same distance ahead in the traveling direction when the host vehicle is traveling on a rainy road surface. FIG.
In such a situation, the brightness information and distance information alone distinguish the taillight of the preceding vehicle, the taillight from the rainy road surface, the headlamp from the oncoming vehicle, and the headlight from the rainy road surface. It is difficult to detect.

  According to the present embodiment, even in such a situation, first, the distinction between the taillight reflected from the tail lamp and the rainy road surface of the preceding vehicle and the headlight from the oncoming vehicle and the headlamp from the rainy road surface. , And can be identified with high accuracy using the spectral information described above. Specifically, in the lamp image area narrowed down using the brightness information and the distance information, the image area in which the pixel value (brightness value) or the red luminance ratio of the red image described above exceeds a predetermined threshold is the tail lamp of the preceding vehicle. Alternatively, the tail lamp image area that reflects the taillight reflected light from the rainy road surface, and the image area that is equal to or less than the threshold is the headlamp image area that reflects the headlight of the oncoming vehicle or the headlamp from the rainy road surface. Identify it.

  Further, according to the present embodiment, the direct light and the reflected light from the tail lamp or the head lamp can be identified with high accuracy by using the polarization information described above for each lamp image area identified by the spectral information in this way. . Specifically, for example, regarding the tail lamp, the difference in frequency and intensity of the horizontal polarization component is used based on the pixel value (luminance value) of the red image of the horizontal polarization component S described above and the difference polarization degree thereof. Thus, the direct light from the tail lamp of the preceding vehicle and the reflected light of the tail lamp from the rain road surface are identified. In addition, for example, with respect to the headlamp, based on the pixel value (brightness value) of the white image of the horizontal polarization component described above, the difference polarization degree, and the like, the difference between the frequency and intensity of the horizontal polarization component is used. Discriminate between direct light from the headlamps of the vehicle and reflected light of the headlamps from the rain road surface.

Next, a flow of detection processing for the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 31 is a flowchart showing a flow of vehicle detection processing in the present embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the imaging device 200, and an image region that is considered to be a detection target is extracted. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of detection objects is the type of the light source displayed in the image area.

  First, in step S1, image data in front of the host vehicle imaged by the image sensor 206 of the imaging device 200 is taken into a memory. As described above, this image data includes a signal indicating the luminance in each imaging pixel of the image sensor 206. Next, in step S2, information related to the behavior of the host vehicle is fetched from a vehicle behavior sensor (not shown).

  In step S3, an image area with high brightness (high brightness image area) that is considered to be a detection target (tail lamp of the preceding vehicle and bed lamp of the oncoming vehicle) is extracted from the image data captured in the memory. This high luminance image region is a bright region having a luminance higher than a predetermined threshold luminance in the image data, and there are many cases where a plurality of high luminance image regions are extracted, but all of them are extracted. Therefore, at this stage, an image area that reflects reflected light from the rainy road surface is also extracted as a high-luminance image area.

  In the high luminance image region extraction processing, first, in step S31, binarization processing is performed by comparing the luminance value of each imaging pixel on the image sensor 206 with a predetermined threshold luminance. Specifically, a binary image is created by assigning “1” to pixels having a luminance equal to or higher than a predetermined threshold luminance and assigning “0” to other pixels. Next, in step S32, when pixels assigned with “1” are close to each other in the binarized image, a labeling process for recognizing them as one high luminance image region is performed. Thereby, a set of a plurality of adjacent pixels having a high luminance value is extracted as one high luminance image region.

  In step S4, which is executed after the above-described high-intensity image region extraction process, the distance between the object in the imaging region corresponding to each extracted high-intensity image region and the host vehicle is calculated. In this distance calculation process, the pair lamp distance calculation process that detects the distance by using the pair of left and right lamps of the vehicle and the left and right lamps constituting the pair lamp cannot be distinguished and recognized at a long distance. A single lamp distance calculation process is executed when the pair lamp is recognized as a single lamp.

  First, for a pair lamp distance calculation process, in step S41, a pair lamp creation process, which is a process of creating a lamp pair, is performed. The pair of left and right lamps that are paired are close to each other at substantially the same height in the image data captured by the imaging device 200, and the area of the high-luminance image region is substantially the same, and the shape of the high-luminance image region is the same. Satisfy the condition of being the same. Therefore, the high-intensity image areas that satisfy such conditions are used as a pair lamp. High brightness image areas that cannot be paired are considered a single lamp. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation process in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal distance f in the imaging apparatus 200 is known, the actual distance x to the pair lamps can be simply calculated by calculating the left / right lamp distance w1 on the image sensor 206 of the imaging apparatus 200 (x = f · w0 / w1). It should be noted that a dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio (red luminance ratio) between the red image of the vertical polarization component P and the white image of the vertical polarization component P is used as spectral information. From this spectral information, two high-luminance image regions that are paired lamps are A lamp type identification process is performed for identifying whether the light is from the head lamp or from the tail lamp. In this lamp type identification process, first, in step S51, pixel data corresponding to the imaging pixels a and f on the image sensor 206 and pixels corresponding to the imaging pixel b on the image sensor 206 for the high-intensity image area that is a pair of lamps. A red ratio image having a pixel value as a ratio to the data is created. In step S52, the pixel value of the red ratio image is compared with a predetermined threshold, and a high-luminance image region that is equal to or higher than the predetermined threshold is determined to be a tail lamp image region by light from the tail lamp. For a certain high-intensity image area, a lamp type process is performed to determine that the area is a headlamp image area by light from the headlamp.

  Subsequently, in step S6, for each image area identified as the tail lamp image area and the head lamp image area, the difference polarization degree ((S−P) / (S + P)) is used as the polarization information from the tail lamp or the head lamp. A reflection identification process is performed for identifying whether the light is direct light or reflected light reflected by a mirror surface such as a rainy road surface. In this reflection identification process, first, in step S61, a differential polarization degree ((S−P) / (S + P)) is calculated for the tail lamp image region, and a differential polarization degree image with the differential polarization degree as a pixel value is created. Similarly, a difference polarization degree ((S−P) / (S + P)) is calculated for the headlamp image region, and a difference polarization degree image is created using the difference polarization degree as a pixel value. In step S62, the pixel value of each differential polarization degree image is compared with a predetermined threshold value, and it is determined that the tail lamp image area and the head lamp image area that are equal to or larger than the predetermined threshold value are caused by reflected light. Those image areas are excluded because they do not reflect the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle. The tail lamp image area and the head lamp image area remaining after performing this exclusion process are identified as those that reflect the tail lamp of the preceding vehicle or those that reflect the head lamp of the oncoming vehicle.

  Note that the reflection identification process S6 described above may be executed only when a rain sensor or the like is mounted on the vehicle and it is confirmed that the rain sensor is raining. Further, the above-described reflection identification process S6 may be executed only when the driver (driver) is operating the wiper. In short, the above-described reflection identification process S6 may be executed only in rainy weather when reflection from the rainy road surface is assumed.

  In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection process as described above are used for light distribution control of a headlamp that is an in-vehicle device of the host vehicle. Specifically, when the tail lamp is detected by the vehicle detection processing and approaches the distance range in which the headlamp illumination light of the host vehicle is incident on the rearview mirror of the preceding vehicle, the headlamp illumination of the host vehicle is approached to the preceding vehicle. Control is performed to block a part of the headlamp of the host vehicle or to shift the light irradiation direction of the headlamp of the host vehicle in the vertical direction or the horizontal direction so as not to be exposed to light. Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range where the headlamp illumination light of the own vehicle hits, the headlamp illumination light of the own vehicle is emitted to the oncoming vehicle. In order to avoid hitting, a part of the headlamp of the host vehicle is shielded from light, or the light irradiation direction of the headlamp of the host vehicle is shifted in the vertical direction or the horizontal direction.

[Distinction processing of wet and dry road surface]
Hereinafter, the determination process of the wet and dry information in the present embodiment will be described.
In this embodiment, in order to determine whether the road surface is wet and the host vehicle is slippery, a determination process is performed to determine the wet and dry state of the road surface.
In the determination process of the road surface wet / dry state in the present embodiment, polarization information obtained by comparing the white component (non-spectral) horizontal polarization component and the vertical polarization component among the information that can be acquired from the imaging unit 101 is used.

FIGS. 32A and 32B are explanatory diagrams for explaining changes in reflected light when the road surface is in a wet state and in a dry state.
As shown in FIG. 32A, the wet road surface becomes close to a mirror surface when water accumulates on the uneven portions of the road surface. Therefore, the reflected light on the wet road surface exhibits the following polarization characteristics. That is, when the reflectances of the horizontal polarization component and the vertical polarization component of the reflected light are Rs and Rp, respectively, the horizontal polarization component Is and the vertical polarization component Ip of the reflected light with respect to the incident light having the light intensity I are expressed by the following formula (1). And (2), and the incident angle dependency is as shown in FIG.
Is = Rs × I (1)
Ip = Rp × I (2)

  As can be seen from FIG. 33, the reflectance Rs of the horizontal polarization component Is of the reflected light at the mirror surface becomes zero when the incident angle is equal to the Brewster angle (53.1 °), and the reflected light intensity of the horizontal polarization component Is is It becomes zero. Further, since the reflectance Rp of the vertical polarization component Ip of the reflected light at the mirror surface shows a characteristic that gradually increases as the incident angle increases, the reflected light intensity of the vertical polarization component Ip also increases gradually as the incident angle increases. On the other hand, as shown in FIG. 32B, the road surface in the dry state has a rough surface, so that irregular reflection is dominant, and the reflected light does not exhibit polarization characteristics, and the reflectance Rs, The difference in Rp becomes smaller.

It is possible to determine whether the road surface is wet or dry based on the difference in the polarization characteristics of the reflected light from the road surface. Specifically, in the present embodiment, the polarization ratio H shown in the following formula (3) is used to determine the wet and dry state of the road surface. The polarization ratio H is calculated, for example, for the image area that reflects the road surface by calculating the ratio (S / P) of the vertical polarization component P of white light (non-spectral) and the horizontal polarization component S of white light (non-spectral). It can be obtained from the average value. Since the polarization ratio H is a parameter that does not depend on the incident light intensity I as shown in the following formula (3), the wet / dry condition of the road surface can be determined stably without being affected by luminance fluctuations in the imaging region. be able to.
H = Is / Ip = Rs / Rp (3)

  When the polarization ratio H obtained in this way exceeds a predetermined threshold value, it is determined that the road surface is wet, and when it is equal to or less than the predetermined threshold value, it is determined that the road surface is dry. When the road surface is dry, the horizontal polarization component S and the vertical polarization component P are substantially equal, so the polarization ratio H is about 1. On the other hand, when the road surface is completely wet, the horizontal polarization component S takes a value much larger than the vertical polarization component P, so the polarization ratio H becomes a large value, and when the road surface is slightly wet, The ratio H is an intermediate value between them.

  In this embodiment, the determination result of the determination process of the wet / dry condition of the road surface as described above is used for driving support control such as warning to the driver of the host vehicle 100 and control of the steering wheel and brake of the host vehicle. Specifically, when it is determined that the road surface is wet, the determination result is sent to the vehicle travel control unit 108, and is used for, for example, control of the automatic brake system of the host vehicle 100, thereby causing a traffic accident. A reduction effect can be expected. Further, for example, information that warns that the road surface is slippery may be notified on the CRT screen of the car navigation system of the host vehicle to alert the driver.

[Detection of road surface metal objects]
Hereinafter, the detection process of the road surface metal object in this embodiment is demonstrated.
In the present embodiment, a process of detecting a road surface metal object as a detection target is performed for the purpose of preventing a side slip of the host vehicle 100 or suppressing erroneous recognition of a radar described later. The road surface metal object here is a metal object that exists on substantially the same plane as the road surface, and is, for example, a manhole cover on a general road or a metal joint on a highway. The manhole cover is a metal plate fitted into the opening of the manhole, and is generally made of strong and heavy cast iron.

  In the road surface metal object detection process according to the present embodiment, first, an image area that projects other than the road surface where the road surface metal object that is the detection target does not exist, specifically, an identification target area that excludes the upper area of the captured image is limited. . Although it is not always necessary to limit the identification target area, it is effective for shortening the processing time. Subsequently, a plurality of processing lines are set for the identification target area. As shown in FIG. 34, the processing line of the present embodiment is set for each pixel column arranged in one horizontal row in the identification target region. The direction of the processing line is not necessarily the horizontal direction, and may be the vertical direction or the oblique direction. In addition, the number of pixels in each processing line may be the same or different. In addition, the processing line is not necessarily set for all the pixels in the identification target area, and may be set for some appropriately selected pixels in the identification target area.

  In the detection process of the road surface metal object in the present embodiment, the polarization information obtained by comparing the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P among the information that can be acquired from the imaging unit 101 is used. Note that the vertical polarization component of red light may include the vertical polarization component of red light. In the present embodiment, as the polarization information, the differential polarization degree ((S−P) / (S + P)) between the white component (non-spectral) horizontal polarization component S and the vertical polarization component P is used. Specifically, from the image data of the white component (non-spectral) horizontal polarization component S and the white component (non-spectral) vertical polarization component P imaged by the imaging unit 101, the differential polarization degree ( A differential polarization degree image having a pixel value of (SP) / (S + P)) is generated. Then, a difference value between two adjacent pixel values (difference polarization degree) is calculated along the processing line described above, and the two adjacent pixels are calculated when the difference value is equal to or greater than a threshold value for the road surface metal object edge. Specify the interval as an edge. Then, the pixel value (difference of polarization) of the two pixels related to the specified edge is compared with a predetermined threshold value for specifying the road surface metal object. Extract as an edge.

  By performing such an edge extraction process for all the processing lines, an image area surrounded by an edge of the road surface metal object can be extracted as an image area candidate of the road surface metal object. After that, shape approximation recognition processing is performed on the image area candidate of the road surface metal object extracted in this way. Specifically, the shape of the extracted image area candidate of the road surface metal object is compared with the shape template of the road surface metal object stored in advance. When the shape of the image area candidate of the road surface metal object matches the shape template of the road surface metal object, the image area candidate is identified as the image area of the road surface metal object.

  In this shape approximation recognition process, an approximate curve is acquired by shape approximation recognition for the edges related to the image area candidate of the road surface metal object. As a method for recognizing the shape, a least square method, a Hough transform, a model equation, or the like is used. Note that when obtaining an approximate curve, it is desirable that an edge related to an image region candidate located in a lower portion of a highly reliable captured image is given a higher weight to a shape approximation vote value. In this way, even if there is an edge related to an image region candidate that is misrecognized in the upper part of the image with low reliability, the image region candidate that is normally recognized in the lower part of the image with high reliability If there is an edge related to, the road surface metal object can be appropriately identified.

Moreover, in order to improve the detection accuracy of road surface metal objects, the following processing may be added.
In the case of detecting a road surface metal object in real time, for an image region identified as a road surface metal object based on a differential polarization degree image obtained by continuously capturing images at predetermined time intervals by the imaging device 200, The processing result is stored in a predetermined memory. Using the processing result of the previous sheet or two or more sheets stored in this memory, the road surface metal object image area identified by this process is the road surface metal object even in the past processing result corresponding to the image area. If it is identified that the current processing result is high, the current processing result is determined to be highly reliable. And this reliability is utilized when identifying the image area | region of a road surface metal object in this process. The past processing result corresponding to the image region related to the current processing result is, for example, an image related to the corresponding past processing result from the position of the image region related to the current processing result and the traveling direction and speed of the own vehicle amount. The position of the area is searched and the corresponding past processing result is specified.

  In the above description, the case where the edge extraction process of the road surface metal object is performed along the processing line has been described. However, instead of the processing line, the processing may be performed in units of processing blocks (blocks each including two or more pixels in the vertical and horizontal directions). . In this case, for example, a plurality of processing blocks are set for the identification target region, and a standard deviation indicating a variation degree (scattering degree) of the pixel value (difference polarization degree) is calculated for each processing block. Can be determined that an edge is present in the processing block. Note that the processing block may be set in a rectangular area, or may be set in an area having another shape. The size of the processing block may be about 10 × 10 image pixels, for example. Each processing block may be the same size or a different size. Further, instead of the standard deviation, a statistic such as variance or average deviation may be used.

  Moreover, the threshold value used when detecting the road surface metal object may be switched according to a change in the environment. For example, it is possible to switch according to time zones such as daytime and nighttime, or according to weather such as rainy weather and fine weather. For such switching, information such as time information or a rain sensor or a sunshine sensor may be used.

Here, the reason why the road surface metal object can be distinguished from the road surface by using the differential polarization degree will be described.
When light is incident at an angle (incident angle) with respect to an interface between two materials having different refractive indexes, a polarization component parallel to the incident surface (vertical polarization component P in the present embodiment) and a perpendicular to the incident surface. The reflectance is different from that of the polarization component (horizontal polarization component S in this embodiment). Specifically, the reflectivity of the vertical polarization component P decreases to zero at a certain angle (Brewster angle) as the incident angle increases, and then increases. On the other hand, the reflectance of the horizontal polarization component S monotonously increases as the incident angle increases. As described above, since the reflection characteristics are different between the vertical polarization component P and the horizontal polarization component S, the degree of differential polarization ((SP) / (S + P)) also varies depending on the incident angle and the refractive index.

  In the present embodiment, the difference in polarization degree ((S−P) / (S + P)) varies depending on the difference in the material of the reflecting surface, that is, the difference in refractive index. Recognize with distinction. That is, the road surface is generally formed of asphalt, while the road surface metal object is formed of metal. If there is such a difference in material, since the refractive index is different, there is a difference in the differential polarization degree between the road surface and the road surface metal object. Due to this difference, as described above, the boundary (edge) between the road surface and the road surface metal object can be extracted, and the image area of the road surface metal object can be identified. Then, by the shape approximation recognition process, it is possible to specify the type of road surface metal object (whether it is a manhole cover or a metal connecting portion) using the shape template.

FIGS. 35A and 35B are image examples showing monochrome luminance images (non-spectral / non-polarized light) and non-spectral differential polarization degree images obtained by imaging the same imaging region.
Since the imaging region is dark, it can be seen that the contrast between the asphalt surface (road surface) and the manhole cover (road surface metal object) is small in the monochrome luminance image shown in FIG. In contrast, in the differential polarization degree image shown in FIG. 35B, the contrast between the asphalt surface (road surface) and the manhole cover (road surface metal object) is large. Therefore, even if it is difficult to identify the manhole cover in the monochrome luminance image, the manhole cover can be identified with high accuracy by using the differential polarization degree image.

  In the present embodiment, the result of the road surface metal object detection process as described above is used for driving support control such as warning of the driver of the host vehicle 100 and control of the steering wheel and brake of the host vehicle. Specifically, when it is determined that the road surface is a metal object, the determination result is sent to the vehicle travel control unit 108, and is used for controlling the automatic brake system of the host vehicle 100, for example, to reduce traffic accidents. We can expect effect. Further, for example, it can be used to alert the driver by notifying lane departure information on the CRT screen of the car navigation system of the host vehicle.

  Further, the result of the road surface metal object detection process can be used in a sensor fusion system that uses both the radar distance measurement result and the captured image of the imaging apparatus 200. If it demonstrates concretely, there exists a possibility that a road surface metal object may be misrecognized as collision avoidance objects, such as a preceding vehicle and a guardrail, by the measurement result of a radar. By correcting the measurement result of the radar based on the result of detecting the road surface metal object from the captured image of the imaging apparatus 200, it is possible to suppress erroneous recognition of the collision avoidance object in the radar. As a result, for example, it is possible to avoid a situation in which the road surface metal object is recognized as a collision avoidance object while the host vehicle is running, and the speed of the host vehicle is rapidly reduced due to the operation of the automatic brake system. it can.

  Further, the result of the road surface metal object detection process can be used as position information in car navigation, thereby improving the position specifying accuracy of the host vehicle. More specifically, the position information of manhole positions on the road surface is stored in a database. Then, from the detection result of the manhole cover, the distance and direction from the own vehicle to the manhole cover are specified, and the relative position information of the own vehicle relative to the manhole cover is generated, and the manhole cover is supported. Specify the manhole ID. Thereafter, the manhole position information corresponding to the specified manhole ID is read from the database, and the position of the own vehicle specified by the car navigation is corrected from the manhole position information and the relative position information of the own vehicle with respect to the manhole.

  When performing the road surface metal object detection process, the road surface metal object detection process may be performed on the differential polarization degree image from which the white line has been removed using the result of the white line recognition process described later. In this case, noise including a white line can be appropriately removed, and the road surface metal object identification accuracy can be improved.

[Three-dimensional object detection processing]
Hereinafter, the solid object detection process in the present embodiment will be described.
In the present embodiment, for the purpose of avoiding a collision with a three-dimensional object, a process of detecting a three-dimensional object as a detection target is performed. The three-dimensional objects mentioned here include other vehicles traveling on the road surface, guardrails, telephone poles, street lamps, signs, roadside steps such as roadside steps on the road surface, road surface Any three-dimensional object having an outer surface facing in a direction different from the traveling road surface, such as a person on an upper or shoulder road, an animal, or a bicycle, is included.

  In the three-dimensional object detection process in the present embodiment, polarization information obtained by comparing the white component (non-spectral) horizontal polarization component S and the vertical polarization component P among the information that can be acquired from the imaging unit 101 is used. Note that the vertical polarization component of red light may include the vertical polarization component of red light. In this embodiment, as this polarization information, as in the road surface metal object detection process, the differential polarization degree ((S−P) / (S + P) between the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P. ) Is used.

  First, from the image data of the white component (non-spectral) horizontal polarization component S and the image data of the white component (non-spectral) vertical polarization component P imaged by the imaging unit 101, the differential polarization degree ((S− A differential polarization degree image having P) / (S + P)) as a pixel value is generated. And a processing line is set similarly to the detection process of the said road surface metal object. However, since the three-dimensional object which is the detection target here can exist over the entire imaging region, the processing line is set for the entire captured image without limiting the identification target region as shown in FIG. In addition, about the setting method of a processing line (or processing block), it is the same as that of the detection process of the said road surface metal object.

  When the processing line is set in this way, a difference value between two adjacent pixel values (difference polarization degree) is calculated along the processing line described above, and the difference value is equal to or larger than a threshold for the three-dimensional object edge. In addition, the interval between the two adjacent pixels is specified as an edge. Then, the pixel value (difference polarization degree) of the two pixels related to the specified edge is compared with a predetermined threshold for specifying a three-dimensional object. Extract as there is.

  By performing such an edge extraction process for all the processing lines, an image region surrounded by the edges of the three-dimensional object can be extracted as an image region candidate for the three-dimensional object. Thereafter, a shape approximation recognition process is performed on the image region candidate of the three-dimensional object extracted in this way. Specifically, the shape of the extracted three-dimensional object image region candidate is compared with a shape object template stored in advance. When the shape of the candidate image area of the three-dimensional object matches the shape template of the three-dimensional object, the candidate image area is identified as the image area of the three-dimensional object. This shape approximation recognition process is the same as in the case of the road surface metal object detection process described above.

  Further, the threshold value used when detecting the three-dimensional object may be switched according to a change in environment. For example, it is possible to switch according to time zones such as daytime and nighttime, or according to weather such as rainy weather and fine weather. For such switching, information such as time information or a rain sensor or a sunshine sensor may be used.

Here, the reason why a three-dimensional object can be recognized by using the differential polarization degree will be described.
As already described, when light is incident at an angle (incident angle) with respect to the interface between two materials having different refractive indexes, the reflection characteristics are different between the vertical polarization component P and the horizontal polarization component S. The differential polarization degree ((SP) / (S + P)) varies depending on the incident angle and the refractive index. In the present embodiment, the difference in polarization degree ((S−P) / (S + P)) varies depending on the difference in the material of the reflecting surface, that is, the difference in refractive index. Can be distinguished and recognized. That is, while the road surface is generally formed of asphalt, solid objects such as other vehicles and guardrails existing in the imaging region are painted on the metal surface. If there is such a difference in material, since the refractive index is different, there is a difference in the degree of differential polarization between the road surface and the three-dimensional object. Due to this difference, as described above, it is possible to extract a boundary (edge) between a solid object such as another vehicle having a road surface and a painted surface or a guardrail, and it is possible to identify an image area of the solid object.

  In addition, the road surface is a substantially horizontal and flat surface, while a three-dimensional object such as another vehicle has a side surface facing a direction different from the road surface. Therefore, since the incident angle of the reflected light taken into the imaging device 200 is different between the road surface and the side surface of the three-dimensional object, the vertical polarization component P and the horizontal polarization component S of the reflected light are different between the road surface and the side surface of the three-dimensional object. coming out. In particular, when the side surface of the three-dimensional object is a surface substantially upright with respect to the road surface, the relative relationship between the vertical polarization component P and the horizontal polarization component S included in the reflected light from the side surface of the three-dimensional object is from the road surface. It approximates to the one in which the relative relationship between the vertical polarization component P and the horizontal polarization component S included in the reflected light is switched. In general, the relative relationship between the vertical polarization component P and the horizontal polarization component S included in the reflected light is such that the vertical polarization component P, which is a polarization component parallel to the incidence plane, is polarized perpendicular to the incidence plane. There is a relationship that the horizontal polarization component S which is a component is larger. Therefore, when the reflected light from the road surface or a plane parallel to the road surface is received by the imaging device 200, the horizontal polarization component S is stronger than the vertical polarization component P, and is from the side of the three-dimensional object substantially upright with respect to the road surface. When the reflected light is received by the imaging device 200, the vertical polarization component P is stronger than the horizontal polarization component S. By comparing the vertical polarization component P and the horizontal polarization component S in the reflected light received by the imaging device 200 due to the difference in polarization characteristics between the road surface and the three-dimensional object, if the horizontal polarization component S is strong, It can be understood that the reflected light is from a plane parallel to the road surface. If the vertical polarization component P is strong, it can be understood that the reflected light is from a plane perpendicular to the road surface. As a result, for example, by taking the difference value (or difference polarization degree) between the vertical polarization component P and the horizontal polarization component S included in the reflected light received by the imaging device 200, the difference value (or difference polarization degree) is obtained. Whether the object has a surface parallel to the road surface or an object having an outer surface facing in a direction different from the road surface, that is, a three-dimensional object, can be grasped.

  Due to the difference in material and incident angle as described above, it is possible to extract the boundary (edge) between the road surface and the three-dimensional object, and to identify the image area of the three-dimensional object. Then, by the shape approximation recognition process, it is possible to specify the type of three-dimensional object (whether it is an automobile or a guardrail, etc.) using the shape template.

FIGS. 37A and 37B are image examples showing monochrome luminance images (non-spectral / non-polarized light) and non-spectral differential polarization degree images obtained by imaging the same imaging region.
Since the imaging region is dark, it can be seen that in the monochrome luminance image shown in FIG. 37A, the contrast between the road surface and the preceding vehicle (three-dimensional object) is small. On the other hand, in the differential polarization degree image shown in FIG. 37 (b), the contrast between the road surface and the preceding vehicle (three-dimensional object) is large. Therefore, even if it is difficult to identify the preceding vehicle with the monochrome luminance image, the preceding vehicle can be identified with high accuracy by using the differential polarization degree image.

  Here, the difference in the degree of polarization due to the difference in the material described above is the difference in polarization reflection characteristics between the asphalt (road surface) that is the material constituting the road surface and the paint that constitutes the side surface of the automobile (three-dimensional object). By evaluating and analyzing, it was confirmed that the polarization reflection models were different, and as a result, it was confirmed that asphalt road surface and automobile could be distinguished. This will be specifically described below.

  Reflected light reflected from an object includes a specular reflection component that is a so-called “light”, a diffuse reflection component that is a matte reflection component that is a fine uneven structure on the surface of the object, and internal scattering that is scattered inside the object. Contains ingredients. The intensity of the reflected light is expressed as the sum of these three components. The specular reflection component can also be considered as a part of the diffuse reflection component. Although the diffuse reflection component and the internal scattering component are observed regardless of the direction of the light source that irradiates the object (that is, the dependency on the incident angle is low), the specular reflection component is the light receiving unit for the reflected light. On the other hand, it is a component having a strong incident angle dependency that is observed only when a light source is present in a substantially regular reflection direction. This is also true for polarization characteristics. As described above, the diffuse reflection component and the internal scattering component are observed regardless of the direction of the light source that irradiates the object, but their polarization characteristics are different from each other. Specifically, the diffuse reflection component can be assumed to divide the object surface into minute regions and satisfy the Fresnel reflection characteristics in each region. Therefore, when non-polarized light is incident, the horizontal polarization component S is There is a polarization characteristic that is larger than the vertical polarization component P. On the other hand, the internal scattering component is a component that is scattered inside the object, so when non-polarized light is incident, it is less affected by the polarization component of the light incident on the object, There is a polarization characteristic that the vertical polarization component P becomes stronger when it comes out.

  And like this embodiment, when photographing the front view from the host vehicle, the objects (asphalt, manhole cover, etc.) that can exist in the photographing region are mostly objects with irregularities on the surface. Therefore, it can be considered that the specular reflection component is small. Therefore, in the present embodiment, it can be considered that the diffused reflection component and the internal scattering component are dominant in the reflected light from the object existing in the imaging region of the imaging device 200. As a result, by comparing the strengths of the horizontal polarization component S and the vertical polarization component P in the reflected light, it can be understood that if the horizontal polarization component S is strong, the reflected light contains many diffuse reflection components, If the vertical polarization component P is strong, it can be understood that the reflected light contains many internal scattering components.

FIG. 38 is an explanatory diagram showing an outline of an experiment in which an image of a horizontal polarization component S and an image of a vertical polarization component P are captured by a fixedly arranged camera in a laboratory with a light source position changed. is there.
In this experiment, an image of the horizontal polarization component S and an image of the vertical polarization component P were photographed with a camera that was fixedly arranged by changing the light source position on the asphalt surface and the painted surface of steel coated with paint in the laboratory. The change in the degree of differential polarization was measured. FIG. 503 is an explanatory diagram of the evaluated optical system. A halogen lamp was used as the light source, and a vision camera was used as the camera. A polarizer was placed in front of the camera so that the direction of polarization could be selected.

FIG. 39 is a graph showing the results of this experiment.
In this graph, the horizontal axis represents the incident angle (light source position), and the vertical axis represents the differential polarization degree. The camera elevation angle is tilted 10 degrees from the horizontal. The differential polarization degree is calculated from luminance information at a substantially central portion of the captured image at each incident angle. However, since the differential polarization degree in this experiment is the ratio of the value obtained by subtracting the horizontal polarization component S from the vertical polarization component P to the total value of the vertical polarization component P and the horizontal polarization component S, the differential polarization degree in the present embodiment. The opposite is positive and negative. Therefore, the differential polarization degree in this experiment takes a positive value when the vertical polarization component P is stronger than the horizontal polarization component S, and when the horizontal polarization component S is stronger than the vertical polarization component P. It will take a negative value.

  As can be seen from the graph shown in FIG. 39, regarding the asphalt surface, the differential polarization degree takes a negative value over almost the entire incident angle. That is, the horizontal polarization component S is stronger than the vertical polarization component P. This is because the diffuse reflection component is dominant in the reflected light from the asphalt surface. On the other hand, with respect to the painted surface, when the incident angle exceeds about 60 °, the differential polarization degree takes a positive value. This is because the reflected light from the painted surface contains both internal scattering components and diffuse reflection components. Due to such differences, it is possible to distinguish and recognize the asphalt surface and the painted surface by the difference in the degree of differential polarization (difference in polarization characteristics).

  In the present embodiment, the result of the three-dimensional object detection process as described above is used for driving support control such as warning to the driver of the host vehicle 100 and control of the steering wheel and brake of the host vehicle. Specifically, when it is determined that the object is a three-dimensional object, the determination result is sent to the vehicle travel control unit 108, and is used for controlling the automatic brake system of the host vehicle 100, for example, to reduce the traffic accident. Etc. can be expected. Further, for example, it can be used to alert the driver by notifying lane departure information on the CRT screen of the car navigation system of the host vehicle.

[Road end detection processing]
Hereinafter, the road edge detection process in this embodiment will be described.
In this embodiment, the process which detects the road end as a detection target is performed for the purpose of preventing the own vehicle from deviating from the travelable region. The road edge here includes a step portion between the vehicle running surface and the pedestrian passage, a side groove portion, planting, a guardrail, a side wall made of concrete, and the like.

  In the road edge detection processing in the present embodiment, polarization information obtained by comparing the white component (non-spectral) horizontal polarization component S and the vertical polarization component P among the information that can be acquired from the imaging unit 101 is used. Note that the vertical polarization component of red light may include the vertical polarization component of red light. In the present embodiment, as the polarization information, the degree of differential polarization between the white component (non-spectral) horizontal polarization component S and the vertical polarization component P ((SP) / (S + P)) is used as the polarization information. Is used.

  First, from the image data of the white component (non-spectral) horizontal polarization component S and the image data of the white component (non-spectral) vertical polarization component P imaged by the imaging unit 101, the differential polarization degree ((S− A differential polarization degree image having P) / (S + P)) as a pixel value is generated. And a processing line is set like the detection process of the said solid object. Note that the processing line (or processing block) setting method is the same as the above three-dimensional object detection processing.

  When the processing line is set in this way, a difference value between two adjacent pixel values (difference polarization degree) is calculated along the processing line described above, and the difference value is equal to or greater than a threshold value for the road edge. In addition, the interval between the two adjacent pixels is specified as an edge. Then, the pixel value (difference polarization degree) of the two pixels related to the specified edge is compared with a predetermined threshold value for road edge identification. Extract as there is.

  By performing such edge extraction processing for all processing lines, an image region surrounded by road edge can be extracted as a road edge image region candidate. Thereafter, shape approximation recognition processing is performed on the image area candidates of the road edge extracted in this way. Specifically, the shape of the extracted road edge image region candidate is compared with a road edge shape template stored in advance. When the shape of the road edge image area candidate matches the road edge shape template, the image area candidate is identified as the road edge image area. This shape approximation recognition process is the same as the above-described solid object detection process.

  Moreover, the threshold value used when detecting the road edge may be switched according to a change in the environment. For example, it is possible to switch according to time zones such as daytime and nighttime, or according to weather such as rainy weather and fine weather. For such switching, information such as time information or a rain sensor or a sunshine sensor may be used.

  The reason why the road edge can be recognized by using the differential polarization degree is the same as in the case of the three-dimensional object. That is, since the difference polarization degree varies depending on the difference in material and incident angle, it becomes possible to extract the boundary (edge) between the road surface and the road end based on the difference polarization degree. Then, by the shape approximation recognition process, the type of road edge can be specified using the shape template.

FIGS. 40A and 40B are image examples showing a monochrome luminance image (non-spectral / non-polarized light) and non-spectral differential polarization degree image obtained by imaging the same imaging region.
This image example is taken when the host vehicle is traveling in a tunnel, and the inside of the imaging region is dark. Therefore, in the monochrome luminance image shown in FIG. 40A, it can be seen that the contrast between the road surface and the tunnel side wall (road end) is small. On the other hand, in the differential polarization degree image shown in FIG. 40B, the contrast between the road surface and the tunnel side wall (road end) is large. Therefore, even in a situation where it is difficult to identify the tunnel side wall in the monochrome luminance image, the tunnel side wall can be identified with high accuracy by using the differential polarization degree image.

  Here, the difference in polarization degree due to the difference in material described above is evaluated by analyzing the difference in polarization reflection characteristics between the asphalt (road surface) that is the material constituting the road surface and the concrete side wall (road end). It was confirmed that the polarized reflection models were different, and that it was possible to distinguish the asphalt road surface from the concrete side wall. This will be specifically described below.

FIG. 41 shows an experiment in which an asphalt surface and a concrete surface are used as test objects, the position of a light source is changed, and an image of a horizontal polarization component S and an image of a vertical polarization component P are captured by a fixedly arranged camera. It is a graph which shows the experimental result at the time.
In this experiment, the experimental apparatus shown in FIG. 38 is used, and the experimental conditions are the same as those in the experiment on the asphalt surface and the painted surface described above.

  As can be seen from the graph shown in FIG. 41, as described above, the asphalt surface has a negative degree of differential polarization over almost the entire incident angle, and the horizontal polarization component S is more than the vertical polarization component P. Is strong. On the other hand, the concrete surface shows a change similar to the painted surface, and it can be seen that the internal scattering component and the diffuse reflection component are mixed in the reflected light from the concrete surface. Due to such a difference, it is possible to distinguish and recognize the asphalt surface and the concrete surface by the difference in the degree of differential polarization (difference in polarization characteristics).

  In the present embodiment, the result of the road edge detection process as described above is used for driving support control such as warning to the driver of the host vehicle 100 and control of the steering wheel and brake of the host vehicle. Specifically, the road edge determination result is sent to the vehicle travel control unit 108 and, for example, it can be used for controlling the automatic brake system of the host vehicle 100, and the effect of reducing traffic accidents can be expected. Further, for example, it can be used to alert the driver by notifying lane departure information on the CRT screen of the car navigation system of the host vehicle.

[White line detection processing]
Hereinafter, the white line detection process in this embodiment will be described.
In the present embodiment, processing for detecting a white line (partition line) as a detection target is performed for the purpose of preventing the vehicle from deviating from the travelable area. Here, the white line includes all white lines that demarcate the road, such as a solid line, a broken line, a dotted line, and a double line. In addition, it can detect similarly about the division line of colors other than white, such as a yellow line.

  In the white line detection processing in the present embodiment, polarization information of the vertical polarization component P of the white component (non-spectral) among the information that can be acquired from the imaging unit 101 is used. In addition, the vertical polarization component of cyan light may be included in the vertical polarization component of white component. In general, it is known that white lines and asphalt surfaces have flat spectral luminance characteristics in the visible light region. On the other hand, since cyan light includes a wide band in the visible light region, it is suitable for imaging asphalt and white lines. Therefore, by using the optical filter 205 in the above configuration example 2 and including the vertical polarization component of cyan light in the vertical polarization component of the white component, the number of imaging pixels to be used increases, resulting in an increase in resolution and distant white lines. Can also be detected.

  In the white line detection processing of this embodiment, in many roads, white lines are formed on the road surface of a color close to black, and the brightness of the white line portion in the image of the white component (non-spectral) vertical polarization component P is on the road surface. Big enough than other parts. Therefore, it is possible to detect a white line by determining a portion of the road surface portion having a luminance equal to or higher than a predetermined value as a white line. In particular, in the present embodiment, the image of the vertical polarization component P of the white component (non-spectral) to be used is obtained by suppressing the reflected light from the rainy road because the horizontal polarization component S is cut. Is possible. Therefore, it is possible to detect a white line without erroneously recognizing disturbance light such as reflected light from a headlamp from a rainy road at night as a white line.

  Also, in the white line detection processing in the present embodiment, out of the information that can be acquired from the imaging unit 101, polarization information obtained by comparing the horizontal polarization component S of the white component (non-spectral) and the vertical polarization component P, for example, white The differential polarization degree ((S−P) / (S + P)) of the component (non-spectral) horizontal polarization component S and vertical polarization component P may be used. Since the reflected light from the white line is usually dominated by the diffuse reflection component, the vertical polarization component P and the horizontal polarization component S of the reflected light are substantially equal, and the degree of differential polarization shows a value close to zero. On the other hand, the asphalt surface portion where the white line is not formed shows a characteristic in which the scattered reflection component is dominant as shown in FIG. 39 and FIG. 41 in the dry state, and the differential polarization degree is a positive value (FIG. 39 and the experimental results shown in FIG. Further, when the asphalt surface portion where the white line is not formed is in a wet state, the specular reflection component is dominant, and the differential polarization degree shows a larger value. Therefore, a portion where the polarization difference value of the obtained road surface portion is smaller than the predetermined threshold value can be determined as a white line.

42A and 42B are image examples showing a monochrome luminance image (non-spectral / non-polarized light) and non-spectral differential polarization degree image obtained by capturing the same imaging region in rainy weather.
Since this image example is taken in rainy weather, the imaging area is relatively dark and the road surface is wet. Therefore, in the monochrome luminance image shown in FIG. 42A, the contrast between the white line and the road surface is small. On the other hand, in the differential polarization degree image shown in FIG. 42B, the contrast between the white line and the road surface is sufficiently large. Therefore, even in a situation where it is difficult to identify a white line in a monochrome luminance image, it is possible to identify the white line with high accuracy by using a differential polarization degree image.
In addition, since the white line in the right part of the image example is shaded, the contrast between the right white line and the road surface is particularly small in the monochrome luminance image shown in FIG. On the other hand, in the differential polarization degree image shown in FIG. 42B, the contrast between the white line on the right side and the road surface is sufficiently large. Therefore, white lines that are difficult to identify in a monochrome luminance image can be identified with high accuracy by using a differential polarization degree image.

[Raindrop detection processing on the windshield]
Hereinafter, the raindrop detection process in the present embodiment will be described.
In the present embodiment, for the purpose of performing the drive control of the wiper 107 and the discharge control of the washer liquid, a process of detecting raindrops as a detection target is performed. In addition, here, the case where the deposit adhered on the windshield is a raindrop will be described as an example. However, the same applies to deposits such as bird droppings and splashes on the road surface from the adjacent vehicle. is there.

  In the raindrop detection process in the present embodiment, the horizontal polarization component of the raindrop detection image area 214 that receives the light transmitted through the infrared light transmission filter area 212 of the pre-stage filter 210 among the information that can be acquired from the imaging unit 101. Polarization information obtained by comparing S and the vertical polarization component P is used. In the present embodiment, as the polarization information, the differential polarization degree ((S−P) / (S + P)) between the white component (non-spectral) horizontal polarization component S and the vertical polarization component P is used.

  As described above, in the imaging unit 101 of the present embodiment, the light source 202 is provided, and when the raindrop 203 is not attached to the outer wall surface of the windshield 105, the light emitted from the light source 202 is the windshield. The light is reflected at the interface between the outer wall surface 105 and the outside air 105, and the reflected light enters the imaging device 200. On the other hand, when raindrops 203 are attached to the outer wall surface of the windshield 105, the refractive index difference between the outer wall surface of the windshield 105 and the raindrop 203 is the refractive index between the outer wall surface of the windshield 105 and the outside air. Smaller than the difference. For this reason, the light emitted from the light source 202 passes through the interface and does not enter the imaging device 200.

FIG. 43 is an explanatory diagram showing the polarization state of the reflected light at the Brewster angle.
In general, when light is incident on a flat surface such as glass, the reflectance of the horizontal polarization component S monotonously increases with respect to the incident angle, whereas the reflectance of the vertical polarization component P is a specific angle (Brewster angle θB). As shown in FIG. 43, the vertical polarization component P is not reflected but only transmitted light. Therefore, the light source 202 is configured to irradiate only the light of the vertical polarization component P with an incident angle of Brewster angle θB from the vehicle interior side toward the windshield 105, whereby the inner wall surface (indoor side of the windshield 105) No reflected light is generated on the surface of the front glass 105, and the light of the vertically polarized light component P is irradiated on the outer wall surface (the vehicle outer surface) of the windshield 105. If there is reflected light on the inner wall surface of the windshield 105, the reflected light becomes disturbance light to the imaging device 200, which causes a reduction in the raindrop detection rate.

  In order to make the light incident on the windshield 105 from the light source 202 only the vertical polarization component P, for example, when a light emitting diode (LED) is used as the light source 202, the vertical polarization component is interposed between the light source 202 and the windshield 105. It is preferable to arrange a polarizer that transmits only P. Further, when a semiconductor laser (LD) is used as the light source 202, the LD can emit only light of a specific polarization component, so that the axis of the LD is adjusted so that only light of the vertical polarization component P is incident on the windshield 105. You may combine them.

FIG. 44A is a graph showing, for each polarization component, the ratio of the amount of light received by the imaging device 200 to the amount of light emitted from the light source 202 when no raindrops are attached to the outer wall surface of the windshield 105.
FIG. 44B is a graph showing, for each polarization component, the ratio of the amount of light received by the imaging device 200 to the amount of light emitted from the light source 202 when raindrops are attached to the outer wall surface of the windshield 105.
In these graphs, the horizontal axis represents the incident angle from the light source 202 to the windshield, and the vertical axis represents the ratio of the amount of light received by the imaging apparatus 200 to the amount of light emitted from the light source 202. The graph indicated by the symbol Is indicates a graph for the horizontal polarization component S, the graph indicated by the symbol Ip indicates a graph for the vertical polarization component P, and the graph indicated by the symbol I indicates an average of the horizontal polarization component S and the vertical polarization component P. A graph of the values is shown. These graphs are calculated by assuming that the refractive index of the windshield 105 is 1.5 and the refractive index of raindrops is 1.38.

  In the present embodiment, as described above, a polarizer is disposed between the light source 202 and the windshield 105 so that only the vertical polarization component P is incident on the windshield 105. Therefore, it is difficult to make adjustments such as adjusting the polarization axis of the polarizer so that only the light of the vertical polarization component P is completely incident on the windshield 105. Therefore, normally, the light of the horizontal polarization component S also enters the windshield 105. Therefore, normally, the light of the horizontal polarization component S is also received by the imaging device 200.

FIG. 45A is a graph showing the degree of differential polarization when raindrops are not attached to the outer wall surface of the windshield 105.
FIG. 45 (b) is a graph showing the degree of differential polarization when raindrops are attached to the outer wall surface of the windshield 105.
These graphs are also calculated assuming that the refractive index of the windshield 105 is 1.5 and the refractive index of raindrops is 1.38.

  Comparing the graphs shown in FIGS. 45 (a) and 45 (b), it can be seen that the incident angle characteristics of the differential polarization degree differ depending on the presence or absence of raindrops. According to these graphs, the difference in the degree of differential polarization between the two is most in the vicinity of 50 degrees where the incident angle is close to the Brewster angle. Therefore, by installing the light source 202 so as to have such an incident angle, it is possible to improve the accuracy of raindrop detection based on the differential polarization degree image.

FIG. 46 is an image example showing a differential polarization degree image when the incident angle is arranged to be around 50 degrees.
The image example in FIG. 46 is a differential polarization degree image when light is emitted from the light source 202 toward the windshield 105 from the vehicle interior in the dark, and the raindrops adhere to the image area of the windshield where the raindrops adhere. It can be seen that the contrast with the image area of the windshield that is not sufficiently high. Therefore, if the differential polarization degree image is used, it is possible to detect raindrops attached to the windshield 105 with high accuracy.

  In the raindrop detection process of this embodiment, first, the light source 202 is turned on, and the difference based on the horizontal polarization component S and the vertical polarization component P of the raindrop detection image region 214 corresponding to the light transmitted through the infrared light transmission filter region 212. A differential polarization degree image having the polarization degree ((S−P) / (S + P)) as a pixel value is generated. Then, edge detection processing is performed on the differential polarization degree image using a known Laplacian filter. By this edge detection process, it is possible to create an image in which the boundary between the raindrop image region candidate and the non-raindrop image region candidate is emphasized. Subsequently, a circular detection process is executed, and an image region detected as a circular is identified as an image region of raindrops. In this circular detection process, a known generalized Hough transform may be performed.

  Thereafter, in the present embodiment, the number of areas identified as raindrop image areas is measured, and the number is converted into rain to calculate rain. The wiper control unit 106 performs drive control of the wiper 107 and washer liquid discharge control based on the rainfall calculated in this way.

  Although the imaging apparatus 200 of the above embodiment has a so-called monocular camera configuration, the imaging apparatus of this modification is a stereo camera having a compound-eye camera configuration. As the configuration of the two camera units included in the stereo camera, the same configuration as that of the imaging device 200 in the above embodiment can be employed.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect A)
Imaging in which an image in the imaging area is picked up by receiving light from an object existing in the imaging area by the image sensor 206 configured by a pixel array in which the light receiving elements 206A are two-dimensionally arranged via the optical filter 205. In the apparatus 200, the optical filter 205 is a unit region in which a plurality of types of light transmission regions having different transmittances are configured by one light receiving element 206A or two or more light receiving elements 206A on the image sensor 206. It has a transmittance adjusting layer such as a polarizing filter layer 222 arranged alternately in the two-dimensional direction of the pixel array.
According to this, a light receiving element that receives light transmitted through a light-transmitting region having a low transmittance (the imaging pixels b and e in the configuration examples 1, 2, and 4 above. The imaging pixels a and b in the configuration example 3 above). Etc.) and a light-transmitting element that receives light transmitted through a light-transmitting region with high transmittance (images are taken in the configuration examples 1, 2, and 4). Pixels a, f, etc. In the above configuration example 3, the imaging pixels e, f, etc.) and a high transmittance image with a relatively large amount of received light can be obtained. As a result, a light source body such as a headlamp with a relatively large amount of light in the imaging area is detected using a low transmittance image, and a light source body such as a tail lamp or a white line with a relatively small amount of light in the imaging area is highly transmissive. By detecting using the rate image, even if there are a plurality of light source bodies with different amounts of light in the imaging region, these light source bodies can be appropriately detected. In addition, since both such a low transmittance image and a high transmittance image can be obtained by a single imaging operation, the low transmittance image and the high transmittance image were obtained by dividing the imaging operation into two times. Compared with the imaging apparatus, the time interval at which these images can be continuously captured can be shortened.

(Aspect B)
In the aspect A, the light transmission region of a type having a relatively low transmittance in the transmittance adjustment layer is such that a part of the light transmission region is a light absorbing material such as a metal reflecting film such as aluminum or a black resist film. It is characterized by being shielded by a material.
According to this, the transmittance can be adjusted without affecting the light transmission characteristics (polarization characteristics and spectral characteristics) of the optical filter.

(Aspect C)
In aspect B, the optical filter 205 transmits or transmits light without selecting a first region and a polarization component that select and transmit only a polarization component in a specific direction (vertical polarization component P). Includes a polarization filter layer 222 that is divided into the unit regions and a second region that selectively transmits only the polarization components in different directions (horizontal polarization component S), and the polarization filter layer 222 includes: The first region or the second region that transmits and transmits only the polarized component in a direction different from the specific direction is a polarizer region in which a part of a uniformly formed metal film is removed to form a wire grid structure. The polarizing filter layer 222 is configured as the transmittance adjusting layer by using the metal film as a light reflecting material configured to shield a part of a light transmitting region of a type having a relatively low transmittance. You .
According to this, it is not necessary to provide a transmittance adjusting layer separately from the polarizing filter layer 222 for the optical filter 205 having the polarizing filter layer 222. In addition, as described above, since the shielding region for limiting the transmittance (aperture ratio) can be formed at the stage of forming the wire grid structure of the polarizing filter layer 222, the manufacturing process can be simplified. .
In addition, the wire grid structure can be manufactured by a semiconductor process, and the polarization axis can be adjusted by changing the groove direction of the sub-wavelength uneven structure. Region) and a shielding region for limiting the transmittance (aperture ratio) can be formed. In addition, since the wire grid structure and the shielding region are made of metal, the optical filter 205 having high reliability in terms of heat resistance and light resistance can be realized. The light resistance here means resistance to deterioration of optical characteristics due to ultraviolet rays or the like. Thus, it is excellent also in heat resistance and light resistance, and can be suitably used for an in-vehicle imaging device.

(Aspect D)
In any one of the aspects A to C, the optical filter 205 selects the first region that transmits only light in a specific wavelength band (red wavelength band or cyan wavelength band) and transmits the light without performing wavelength selection. The spectral filter layer 223 in which the second region that transmits or selectively transmits light in a wavelength band other than the specific wavelength band is divided in the unit region in the light transmission direction together with the transmittance adjustment layer. It has a laminated structure.
According to this, spectral information can also be obtained from the captured image, and for example, it is possible to appropriately detect a plurality of light source bodies having different amounts of light for an arbitrary wavelength band.

(Aspect E)
In the aspect D, the second region of the spectral filter layer 223 transmits light without performing wavelength selection, and the spectral filter layer 223 uniformly forms a filter film serving as the first region. The filter film is formed by removing the filter film that is present at a location that will later become the second region.
According to this, it is possible to relatively easily realize the spectral filter layer 223 that is divided into small regions such as one pixel or several pixels of the light receiving element 206A.

(Aspect F)
In the aspect E, in the optical filter 205, the first region in the spectral filter layer 223 transmits light that passes through a light transmission region of a type having a high transmittance in the transmittance adjustment layer. The second region is configured to transmit light that passes through a light transmitting region of a type having a low transmittance in the transmittance adjusting layer.
According to this, a light source body (for example, a tail lamp) having a relatively small amount of light and a white light source body (for example, a headlamp) having a relatively large amount of light is appropriately selected from image data obtained by one imaging. Can be identified.

(Aspect G)
In the object detection apparatus having the object detection processing unit that performs the detection process of the detection target existing in the imaging region based on the captured image captured by the imaging unit, any one of modes A to F is used as the imaging unit. The imaging apparatus 200 according to the above is used.
According to this, since a low-transmittance image (image with a small amount of exposure) and a high-transmittance image (an image with a large amount of exposure) can be acquired by one imaging, the configuration is such that these images are alternately captured. In comparison, a low transmittance image and a high transmittance image can be obtained continuously at a high frame rate. Therefore, even if the detection target that can be detected from the low-transmittance image and the high-transmittance image moves at high speed in the imaging region, the movement can be appropriately grasped.

(Aspect H)
An oncoming vehicle that travels in the opposite direction of the host vehicle and a preceding vehicle that travels in the same traveling direction as the host vehicle based on the captured image captured by the imaging means that captures the front area in the traveling direction of the host vehicle as an imaging region. In the object detection apparatus having object detection processing means for performing detection processing as a detection target, the imaging device 200 of aspect F is used as the imaging means, and the first region of the spectral filter layer 223 transmits the light. The wavelength band is a wavelength band including the wavelength of the tail lamp color of the vehicle, and the object detection processing means outputs an output signal of the light receiving element 206A on the image sensor 206 that has received the transmitted light of the first region of the spectral filter layer 223. Based on the detection of the preceding vehicle and the output of the light receiving element 206A on the image sensor 206 that has received the transmitted light transmitted through the second region of the spectral filter layer 223. And detecting the oncoming vehicle based on the signal.
According to this, both tail lamps and head lamps having greatly different light amounts can be identified with high accuracy from image data obtained by one imaging.

(Aspect I)
In an optical filter arranged between an image sensor and an imaging region of an imaging device having an image sensor composed of a pixel array in which a light receiving element is two-dimensionally arranged, a plurality of types of light transmission regions having different transmittances are provided. In addition, the image sensor has a transmittance adjusting layer arranged alternately in a unit region constituted by one light receiving element or two or more light receiving elements on the image sensor.
According to this, since it becomes possible to appropriately discriminate a light source body having a relatively small amount of light and a light source body having a relatively large amount of light from image data obtained by a single imaging operation, It is possible to continuously detect the light source body at shorter time intervals.

(Aspect J)
In a manufacturing method of an optical filter arranged between an image sensor and an imaging region of an imaging device having an image sensor constituted by a pixel array in which a light receiving element is two-dimensionally arranged, only a polarization component in a specific direction is selected. The first region on the image sensor transmits the first region to be transmitted and the second region to transmit light without selecting the polarization component or to select and transmit only the polarization component in a direction different from the specific direction. When forming alternately arranged polarizing filter layers in a unit region composed of two light receiving elements or two or more light receiving elements, the first region is formed by removing a part after forming a metal film uniformly. Forming a wire grid structure and reducing the transmittance of at least one first region by leaving a metal film so that light is shielded by a part of the wire grid structure. And butterflies.
According to this, since it becomes possible to appropriately discriminate a light source body having a relatively small amount of light and a light source body having a relatively large amount of light from image data obtained by a single imaging operation, It is possible to continuously detect the light source body at shorter time intervals.

DESCRIPTION OF SYMBOLS 100 Own vehicle 101 Imaging unit 102 Image analysis unit 103 Headlamp control unit 104 Headlamp 105 Windshield 106 Wiper control unit 107 Wiper 108 Vehicle travel control unit 200 Imaging device 201 Imaging case 202 Light source 203 Raindrop 204 Imaging lens 205 Optical filter 206 Image Sensor 206A Photodiode (light receiving element)
210 Front filter 211 Infrared light cut filter region 212 Infrared light transmission filter region 213 Vehicle detection image region 214 Raindrop detection image region 220 Rear filter 221 Filter substrate 222 Polarizing filter layer 223 Spectral filter layer 224 Filler

JP 2005-92857 A

Claims (9)

In an imaging device that captures light in an imaging region by receiving light from an object existing in the imaging region through an optical filter by an image sensor configured by a pixel array in which a light receiving element is two-dimensionally arranged.
The optical filter is a unit region composed of one light receiving element or two or more light receiving elements on the image sensor, wherein a plurality of types of light transmitting regions having different transmittances when uniform light is incident thereon , Having transmittance adjustment layers alternately arranged in the two-dimensional direction of the pixel array;
The transmittance adjusting layer includes a light transmitting portion having a metal wire grid structure that selectively transmits only a polarized light component in a specific direction, and a metal light shielding portion disposed around the light transmitting portion . A light transmission part that transmits light without selecting a polarization component or a light transmission part having a metal wire grid structure that transmits only a polarization component in a direction different from the specific direction and the light; a second region composed of a shielding portion of a metal disposed around the transparent portion is formed by a polarizing filter layer divided into areas above Kitan displacement range,
The lower the type of the light transmission region transmittance is relatively imaging apparatus, wherein the area of the light-transmitting portion than the transmittance is relatively high types of light transmission region is small. The imaging device according to claim 1,
The light transmission region of a type having a relatively low transmittance is the first region or the second region that selectively transmits only the polarization component in a direction different from the specific direction. . The imaging device according to claim 1 or 2,
The optical filter is configured to transmit light without performing wavelength selection with the first region that selects and transmits only light in a specific wavelength band, or selects and transmits light in a wavelength band other than the specific wavelength band. An imaging apparatus comprising: a spectral filter layer in which two regions are divided by the unit region, and the transmittance adjusting layer stacked in the light transmission direction. The imaging device according to claim 3.
The second region of the spectral filter layer transmits light without performing wavelength selection,
The spectral filter layer is formed by removing the filter film that exists in the portion that becomes the second region after the filter film that becomes the first region is uniformly formed. apparatus. The imaging device according to claim 4.
The optical filter in the first region in the spectral filter layer is light transmitted through the light transmission region of the type described above having a high transmittance in the transmittance adjusting layer is transmitted, to the second region in the spectral filter layer the imaging apparatus characterized by light transmitted through the light transmission region type low the transmittance in the transmittance adjusting layer is to transmit configuration. In an object detection apparatus having object detection processing means for performing detection processing of a detection target existing in an imaging region based on a captured image captured by an imaging means,
An object detection apparatus using the imaging apparatus according to claim 1 as the imaging means. An oncoming vehicle that travels in the opposite direction of the host vehicle and a preceding vehicle that travels in the same traveling direction as the host vehicle based on the captured image captured by the imaging means that captures the front area in the traveling direction of the host vehicle as an imaging region. In an object detection apparatus having object detection processing means for performing detection processing as a detection target,
The imaging device of claim 5 is used as the imaging means,
The specific wavelength band through which the first region of the spectral filter layer transmits light is a wavelength band including a wavelength of a tail lamp color of a vehicle,
The object detection processing means detects a preceding vehicle based on the output signal of the light receiving element on the image sensor that has received the transmitted light of the first region of the spectral filter layer, and transmits the second region of the spectral filter layer. An on-coming vehicle is detected based on an output signal of a light receiving element on an image sensor that has received transmitted light.   Object detection processing means for performing detection processing of a detection target existing in the imaging area based on a captured image captured by an imaging means that captures the front area in the traveling direction of the host vehicle through the windshield of the host vehicle as an imaging area. In an object detection apparatus having
  As the imaging means, using the imaging device according to any one of claims 1 to 5,
  The first region of the polarizing filter layer is configured such that the angle between the longitudinal direction of the metal wires of the wire grid structure and the horizontal direction changes as it approaches the horizontal end in accordance with the curvature of the windshield. An object detection device characterized by being provided. In an optical filter arranged between the image sensor and an imaging region of an imaging device having an image sensor constituted by a pixel array in which light receiving elements are two-dimensionally arranged,
A plurality of types of light transmission regions having different transmittances when uniform light is incident are unit regions composed of one light receiving element or two or more light receiving elements on the image sensor. Having transmittance adjustment layers arranged alternately in the dimension direction;
The transmittance adjusting layer includes a light transmitting portion having a metal wire grid structure that selectively transmits only a polarized light component in a specific direction, and a metal light shielding portion disposed around the light transmitting portion . A light transmission part that transmits light without selecting a polarization component or a light transmission part having a metal wire grid structure that transmits only a polarization component in a direction different from the specific direction and the light; a second region composed of a shielding portion of a metal disposed around the transparent portion is formed by a polarizing filter layer divided into areas above Kitan displacement range,
The lower the type of the light transmission region transmittance is relatively optical filter, wherein the area of the light-transmitting portion than the transmittance is relatively high types of light transmission region is small.