JP5839253B2 - Object detection device and in-vehicle device control device including the same - Google Patents

Description

  The present invention relates to an object detection device that detects at least two types of detection objects that emit light in different wavelength bands based on captured images captured by an imaging unit, and an on-vehicle device control device including the same.

  Conventionally, at night, an in-vehicle camera (imaging device) captures a front area (imaging area) in the traveling direction of the host vehicle, and performs object detection processing based on the captured image to detect an object such as a preceding vehicle or an oncoming vehicle, A driver support system that reduces the driving load on the driver of the vehicle is known. Some driver support systems perform, for example, information provision control that notifies or warns the driver using detection results of preceding vehicles, oncoming vehicles, and the like. In addition, for example, by using the detection result of the preceding vehicle or the oncoming vehicle, control of switching the irradiation direction of the headlamp of the own vehicle (switching control between high beam and low beam, etc.), and control of ensuring visibility by the wiper of the own vehicle Some control the in-vehicle devices that perform the operation. Such a driver assistance system reduces the load on driving by the driver and contributes to traffic safety, and is a very useful system. However, the detection result of the preceding vehicle or the oncoming vehicle is used. In order to appropriately perform the various controls, it is important to improve the detection accuracy of the preceding vehicle, the oncoming vehicle, and the like.

  A conventional object detection device used in such a driver assistance system is a monochrome luminance camera (imaging device) that performs imaging based on brightness information (luminance information) of reflected light from an object existing in an imaging region. The one using is generally used. When imaging a forward area (imaging area) in the traveling direction of the vehicle in order to detect a preceding vehicle or an oncoming vehicle at night, the light from the imaging area is not limited to the tail lamp of the preceding vehicle or the headlamp of the oncoming vehicle. And various disturbance lights. For example, a light reflector such as a vending machine, a store or a light-emitting signboard around a road, or a light reflector that reflects light from a headlamp or other light source of the own vehicle or another vehicle (for example, installed on the road side) Light from a reflector or the like can become disturbance light. Therefore, the captured image captured by the monochrome luminance camera also includes other disturbance light other than the tail lamp of the preceding vehicle and the head lamp of the oncoming vehicle. Therefore, in order to detect the preceding vehicle and the oncoming vehicle with high accuracy at night, it is necessary to distinguish the taillight light of the preceding vehicle, the headlamp light of the oncoming vehicle, and other disturbance light with high accuracy. is there.

  Patent Document 1 discloses a system that detects a tail lamp of a preceding vehicle and a headlamp of an oncoming vehicle based on an image captured by an image sensor as an object detection device that can detect the preceding vehicle and the oncoming vehicle at night. . This system extracts strong white light from a headlamp and strong red light from a tail lamp, and detects an oncoming vehicle or a preceding vehicle from the extraction result (spectral information). Specifically, a lens formed with a red filter dye and a lens formed with a greenish blue filter dye are arranged in the front stage of the image sensor, and the input light from the imaging region is respectively received by these lenses. Focusing is performed at two locations on the image sensor. As a result, the red component of the input light is extracted from the luminance value of the light focused by the former lens, and the blue component of the input light is extracted from the luminance value of the light focused by the latter lens. When the red component of the input light is equal to or higher than the predetermined brightness (luminance), it is assumed that the tail lamp of the preceding vehicle exists in the imaging region, and the tail lamp of the preceding vehicle is detected, and the blue component of the input light has the predetermined brightness. When the brightness is equal to or greater than (luminance), the headlamp of the oncoming vehicle is detected in the imaging region, and the headlamp of the oncoming vehicle is detected.

  However, since the system disclosed in Patent Document 1 merely detects the luminance information of the red component and the luminance information of the blue component of the input light from the imaging region, a detection target (headlamp or There is a problem that a non-detection object having disturbance light of the same wavelength having the same brightness as that of the light of the tail lamp and the disturbance light is erroneously detected as light of the detection object. For example, when there is a specular reflector (such as a water pool on the road surface or a metal surface on or on the road surface) that specularly reflects light on the road surface or around the road, the headlamps and tail lamps outside the imaging area In some cases, light or other light strikes a specular reflector in the imaging region, and light having the same wavelength as the light of the headlamp or tail lamp is emitted from the specular reflector. In this case, when the detection is performed with the luminance and wavelength of the input light as in the system disclosed in Patent Document 1, the light of the head lamp or tail lamp, which is the detection target, and such disturbance light are distinguished and recognized. Can not.

  This problem is not limited to the object detection process of the preceding vehicle or the oncoming vehicle. Specifically, it is a case where at least two types of objects (detection target objects) that emit light in different wavelength bands are detected, and the same wavelength band as the light emitted from the objects (including reflected light; the same applies hereinafter). This is a problem that can occur in the same way when disturbing light that includes and exhibits the same level of brightness can exist in the imaging region.

The present invention has been made in consideration of the aforementioned problems, it is an object of non-detection object that emits ambient light that indicates the detection of contain and comparable to the same wavelength band as light emitted from the object luminance An object detection device capable of highly accurate detection even when an object is present and an in-vehicle device control device including the same are provided.

In order to achieve the above object, the invention of claim 1 includes a spectral member that transmits a spectral component having a predetermined wavelength band and a polarizing member that transmits a polarized component that vibrates in a predetermined direction, and exists in the imaging region. Imaging means for receiving light emitted from an object to be picked up by an image sensor and imaging the inside of the imaging area; object detection processing means for performing detection processing of a detection target including a vehicle based on a captured image captured by the imaging means; The spectral information acquisition unit acquires spectral information of the detection target from the captured image captured by the imaging unit, and the spectral information acquisition unit acquires from the captured image captured by the imaging unit. Polarization information acquisition means for acquiring polarization information corresponding to the spectral information to be obtained, and the object detection processing means includes a first detection process based on the spectral information acquired by the spectral information acquisition means, and the polarization information. The first polarization information about the polarization component that vibrates in the predetermined direction acquired by the information acquisition means and the second polarization information about the light that vibrates in the direction orthogonal to the predetermined direction calculated from the first polarization information. The detection process of the detection target is performed by performing the second detection process.
According to a second aspect of the present invention, in the object detection device according to the first aspect, the image sensor has a pixel array in which light receiving elements are two-dimensionally arranged, and the spectroscopic member receives light from the imaging region. The light is split into a plurality of different detection wavelength bands, and the light of each detection wavelength band is incident on different light receiving elements on the image sensor, and the polarizing member has each detection wavelength incident on the image sensor. The light having a polarization region that blocks only a specific polarization component of the band light and a transmission region that transmits the light without being polarized, and the spectral information acquisition unit receives each light of each detection wavelength band. An index value based on the amount of light received by the element is acquired as spectral information for each detection wavelength band, and the polarization information acquisition unit uses the index value based on the amount of light received by the light receiving element corresponding to the polarization region as first polarization information. An index value based on an amount obtained by subtracting the amount of light received by the light receiving element corresponding to the polarization region from the amount of light received by the light receiving element corresponding to the transmission region is obtained as second polarization information. .
Further, the invention of claim 3 is the object detection device of claim 1 or 2, wherein the imaging means images an area in front of the traveling direction of the host vehicle as an imaging area. A vehicle that travels in a direction approaching the vehicle and another vehicle that travels in a direction away from the host vehicle, and the spectral information acquisition unit includes spectral information about at least a wavelength band of a tail lamp color of the vehicle and a head of the vehicle. The spectral information about the wavelength band of the lamp color is acquired.
According to a fourth aspect of the present invention, in the object detection apparatus of the third aspect, in the second detection process performed by the object detection processing unit, the first polarization information and the second polarization information acquired by the polarization information acquisition unit. Is used to identify regular spectral information about light directly incident from a tail lamp of another vehicle and a headlamp of another vehicle, and non-regular spectral information about light from a road surface having the same wavelength band as the regular spectral information. The other spectroscopic information is used to detect another vehicle traveling in a direction approaching the own vehicle and another vehicle traveling in a direction away from the own vehicle.
Further, the invention according to claim 5 is based on an object detection means for picking up an image of the periphery of the vehicle as an image pickup area, detecting a detection target including the vehicle existing in the image pickup area, and a detection result of the object detection means. In the in-vehicle device control apparatus which has a control means which controls operation of the in-vehicle equipment carried in the above-mentioned vehicle, the object detection device given in any 1 paragraph of Claims 1 thru / or 4 was used as said object detection means. It is characterized by.
According to a sixth aspect of the present invention, in the in-vehicle device control apparatus according to the fifth aspect, the object detection device according to the third or fourth aspect is used as the object detection means, and the control means uses the detection result of the object detection means. Based on this, control is performed to change the irradiation direction of a headlamp which is an in-vehicle device mounted on the vehicle or to change the emission intensity of the headlamp.

In the present invention, (including as to reflect what the light itself emitted.) Contact Keru objects within the imaging region as a detection object, the spectral information spectral information acquisition means in an imaging image of the detection object obtained from, it detects the detection object in the imaging region by the object detection processing unit based on the spectral information. In the detection processing of the detection target by the object detection processing unit, the polarization information acquired from the captured image by the polarization information acquisition unit for the detection wavelength band corresponding to the spectral information acquired by the spectral information acquisition unit is also used. Therefore, even if there is a non-detection target in the imaging region that has the same wavelength as the light from the detection target and emits disturbance light having the same level of brightness, polarized light is detected between the detection target and the non-detection target. If there is a difference in components, these can be distinguished and recognized. For example, between the direct light from the object to be detected and the light from the object to be detected reflected by the specular reflector, the S-polarized component contained in the latter light is generally strong. Can do.

As described above, according to the present invention, even when the non-detection object that emits ambient light that indicates the luminance of contain and comparable to the same wavelength band as light emitted from the detection object is present, the detection object and the non-detection target If there is a difference in the polarization component between the object and the object, the detection object can be distinguished from the non-detection object, and an excellent effect can be obtained that the detection object can be detected with high accuracy.

It is a functional block diagram of the headlamp control system concerning an embodiment. It is explanatory drawing which shows schematic structure of the imaging device in the headlamp control system. It is an enlarged view which shows the optical filter and image sensor in the imaging device. It is explanatory drawing which shows the correspondence of the area | region division pattern of the same optical filter, and the pixel of the same image sensor. It is an enlarged photograph of the wire grid polarizer which can be used as a polarization region of the optical filter. It is an image which shows a measurement result when the light of an S polarization component is entered into the same optical filter. 5 is an explanatory diagram of an optical filter and an image sensor in Configuration Example 1. FIG. It is explanatory drawing of the optical filter and image sensor in the structural example 2. FIG. It is explanatory drawing of the optical filter and image sensor in the structural example 3. FIG. It is explanatory drawing which piled up and showed the polarization area of the optical filter in the structural example 4 on an image sensor. It is an image which shows a measurement result when the light of an S polarization component is entered into the same optical filter. It is a histogram which shows the result of having imaged the light (direct light) directly incident from a headlamp and the light (light reflected) which the light from the headlamp reflected on the rainy road surface using the imaging device on a rainy day . It is a schematic diagram which shows an example at the time of the own vehicle driving | running | working on a rainy road surface, when the situation where both a preceding vehicle and an oncoming vehicle exist in the substantially same distance ahead of the advancing direction is imaged with the imaging device. It is a flowchart which shows the flow of the vehicle detection process in embodiment.

Hereinafter, an embodiment in which an object detection device according to the present invention is used in a headlamp control system as an in-vehicle device control device will be described.
The object detection device according to the present invention is not limited to the headlamp control system, and for example, the object detection device detects a detection target object such as a preceding vehicle or an oncoming vehicle at night and displays or warns the driver. The same applies to the driver assistance system.

FIG. 1 is a functional block diagram of the headlamp control system according to the present embodiment.
This headlamp control system mainly includes an imaging device 11 as an imaging means, a vehicle behavior sensor 12, a vehicle detection control device 13 as an object detection device, and a headlamp control device 14 as a control means. ing.

  The imaging device 11 is mounted on the host vehicle so that the front area in the traveling direction of the host vehicle can be shot. Specifically, for example, it is attached to the vicinity of the windshield on the back side of the rearview mirror attached to the vehicle interior, built into the rearview mirror, or built into the headlamp. When the image pickup apparatus 11 is mounted, the image pickup direction of the image pickup apparatus 11 is adjusted and fixed so as to coincide with a predetermined reference direction (for example, the horizontal direction). The imaging device 11 is configured to be able to adjust the shutter speed, the frame rate, the gain of the digital signal output to the vehicle detection control device 13, and the like according to an instruction from a built-in control unit (not shown). Has been. And the imaging device 11 outputs the digital signal which shows the brightness (luminance) for every pixel of the image | photographed image as captured image data to the vehicle detection control apparatus 13 with the horizontal / vertical synchronizing signal of an image.

  The vehicle behavior sensor 12 is composed of, for example, a stroke sensor installed on a four-wheel suspension of the host vehicle, and detects behavior changes in the pitch direction and the roll direction of the host vehicle. When the host vehicle travels and acceleration occurs in the front, rear, left, and right directions, the body of the host vehicle swings in the pitch direction and the roll direction, and accordingly, the imaging direction of the imaging device 11 deviates from the reference direction. The vehicle behavior sensor 12 gives vehicle behavior information to the vehicle detection control device 13 for calculating how much the imaging direction of the imaging device 11 deviates from the reference direction.

  The vehicle detection control device 13 performs predetermined image processing using the captured image data input from the imaging device 11 and the above-described vehicle behavior information from the vehicle behavior sensor 12, so that the light source included in the image is detected. , Determine whether the tail lamp of the preceding vehicle (another vehicle traveling in a direction away from the host vehicle) or the headlamp of an oncoming vehicle (another vehicle traveling in a direction approaching the host vehicle) is applicable or not To do. When it is determined that the light source included in the image corresponds to the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle, detection information regarding the preceding vehicle or the oncoming vehicle is output to the headlamp control device 14.

  The headlamp control device 14 controls the direction of the headlamp (light irradiation direction) based on the detection information regarding the preceding vehicle and the oncoming vehicle input from the vehicle detection control device 13. For example, when the distance to the preceding vehicle or the oncoming vehicle specified based on the detection information is equal to or less than a predetermined distance, the headlamp direction is set to a low beam. This makes it difficult for the driver of the preceding vehicle or the oncoming vehicle to feel dazzled by the light of the headlamp of the host vehicle. On the other hand, when the distance to the preceding vehicle or the oncoming vehicle exceeds a predetermined distance, or when the preceding vehicle or the oncoming vehicle is not detected, the direction of the headlamp is set as a high beam. As a result, the field of view of the driver of the host vehicle can be secured even further at night. The headlamp control device 14 can switch the headlamp orientation OFF by the user, and can manually switch between the high beam and the low beam according to the user's recognition.

FIG. 2 is an explanatory diagram illustrating a schematic configuration of the imaging apparatus 11 according to the present embodiment.
The imaging device 11 mainly includes an imaging lens 1, an optical filter 2, a sensor substrate 3 including an image sensor 4 having a two-dimensionally arranged pixel array, and an analog electrical signal output from the sensor substrate 3 ( The signal processing unit 5 generates and outputs captured image data obtained by converting a received light amount received by each light receiving element on the image sensor 4 into a digital electric signal. Light from the imaging region including the subject (test object) passes through the imaging lens 1, passes through the optical filter 2, and is converted into an electrical signal corresponding to the light intensity by the image sensor 4. When the electrical signal (analog signal) output from the image sensor 4 is input to the signal processing unit 5, the brightness (luminance) of each pixel on the image sensor 4 is indicated as captured image data from the electrical signal. The digital signal is output to the subsequent device together with the horizontal / vertical synchronizing signal of the image.

FIG. 3 is an enlarged view showing the optical filter 2 and the image sensor 4.
The image sensor 4 is an image sensor using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and a photodiode 4a is used as a light receiving element thereof. The photodiodes 4a are two-dimensionally arranged for each pixel, and a microlens 4b is provided on the incident side of each photodiode 4a in order to increase the light collection efficiency of the photodiodes 4a. The image sensor 4 is bonded to a PWB (printed wiring board) by a technique such as wire bonding to form a sensor substrate 3. The optical filter 2 is disposed close to the surface of the image sensor 4 on the microlens 4b side. As shown in FIG. 4, the surface of the optical filter 2 on the image sensor 4 side is a transmission region 2a that transmits light, and polarized light that functions as a polarizer that transmits P-polarized components but shields S-polarized components. It has a lattice pattern (region division pattern) that is arranged in a lattice pattern so that the regions 2b are alternately adjacent to each other in the two-dimensional direction. As shown in FIG. 4, each transmission region 2 a and polarization region 2 b are arranged so as to correspond to two adjacent pixels (two photodiodes 4 a adjacent to each other) on the image sensor 4.

  In such an optical filter 2, as the polarization region 2b, for example, a polarizer made of an organic material using iodine or a dye used for a liquid crystal display or the like may be used, or a wire grid having excellent durability. A polarizer may be used. As shown in the photograph of the cross-sectional structure of the wire grid in FIG. 5, the wire grid polarizer is formed by arranging conductor wires made of metal such as aluminum in a lattice pattern at a specific pitch. If the pitch of the wire grid polarizer is sufficiently shorter than the incident light (for example, 400 to 800 nm which is the wavelength of visible light) (for example, ½ or less of the incident light), it is parallel to the conductor line. Since most of the oscillating electric field vector component light is reflected and almost all the electric field vector component light perpendicular to the conductor line can be transmitted, it can be used as a polarizer for producing a single polarized light.

  The wire grid polarizer can be manufactured by a well-known semiconductor process, that is, by patterning after depositing an aluminum thin film and forming the subwavelength uneven structure of the wire grid by a technique such as metal etching. In forming the lattice pattern as in the present embodiment, a method of removing the wire grid portion from the uniform wire grid layer according to the lattice pattern described above, and after uniformly depositing aluminum, It is conceivable that the concave formation of the wire grid is made at once. The latter requires fewer steps.

  In addition, when using a wire grid polarizer, it is mentioned that the extinction ratio increases as the cross-sectional area of the metal wire increases. Moreover, it is mentioned that the transmittance | permeability will reduce with the metal wire more than the predetermined width with respect to a period width. Moreover, when the cross-sectional shape orthogonal to the longitudinal direction of the metal wire is a tapered shape, the wavelength dispersion of the transmittance and the polarization degree is small in a wide band, and high extinction ratio characteristics are exhibited. It is preferable to seal with a resin or the like in order to improve the scratch resistance and antifouling property of the surface on which the metal wire is disposed.

  FIG. 6 is an image showing a measurement result when light of an S-polarized component is incident on the optical filter 2 according to the present embodiment that has been prototyped by the present inventors. The prototype optical filter 2 is formed by forming a region with a wire grid and a region without a wire grid on a glass plate so that the stripe width is about 6 μm. The former region is a polarizing region 2c, and the latter This area is a transmission area 2a. The measurement is the same as in the case of the above configuration example 1. As shown in FIG. 6, the black portion indicates that the light is shielded, and it is confirmed that the light of the S-polarized component is well cut in the polarization region 2c where the wire grid is formed. it can.

  In the present embodiment, the pixels (photodiode 4a) on the image sensor 4 form one group with four pixels, and each group has a wavelength band (red component) of the tail lamp color (red) of the vehicle as shown in FIG. R that receives light of non-polarized light, a pixel W that receives light of a vehicle headlamp color (white) wavelength band (white component = non-spectral) with non-polarized light, and P included in red component light The pixel RP which receives a polarization component and the pixel WP which receives a non-spectral P polarization component are included.

  In the present embodiment, in order to divide the pixels on the image sensor 4 so as to function as the four types of pixels R, W, RP, and WP, the image sensor 4 has only a red component. As shown in FIG. 4, spectral filters are used that are arranged in a checkered pattern so that the area of the color filter that passes through and the area without the color filter are alternately adjacent in the two-dimensional direction. Thereby, the pixels on the image sensor 4 are arranged in a checkered pattern so that the pixels R and RP that receive the red component and the pixels W and WP that receive the white component (non-spectral) are alternately adjacent in the two-dimensional direction. Is divided into regions. In addition, the color filter to be used can use what was formed by the semiconductor process which permeate | transmits only red light (for example, light with a wavelength of 600 nm or more). With such a filter, the filter function is less deteriorated when mounted in the imaging device 11 installed in a special environment such as a passenger compartment where the temperature changes drastically, as compared with a filter using a dye.

  As described above, the optical filter 2 in the present embodiment is arranged so that each transmission region 2a and polarization region 2b correspond to two adjacent pixels on the image sensor 4, respectively. Therefore, two adjacent pixels facing the polarization region 2b (a pixel that receives the red component and a pixel that receives the non-spectral light) receive only the P-polarized light component from which the S-polarized light component has been removed. Therefore, two adjacent pixels facing the polarization region 2b are a pixel RP that receives the P-polarized component included in the red component light, and a pixel WP that receives the P-polarized component included in the white component light (non-spectral). Function as. On the other hand, two adjacent pixels facing the transmission region 2a function as a pixel R that receives a non-polarized red component and a pixel W that receives non-polarized and non-spectral light. As a result, according to the present embodiment, as shown in FIG. 4, the pixels on the image sensor 4 are divided into regions so that the four types of pixels R, W, RP, and WP form a 2 × 2 pixel group. Pattern can be obtained.

  The pixels on the image sensor 4 are divided into four types of pixels R, W, RP, and WP in such a pattern, thereby obtaining various images with 2 × 2 pixels on the image sensor 4 as one pixel. be able to. For example, a red component P-polarized image corresponding to the pixel RP (P-polarized red image), a white component corresponding to the pixel WP, that is, a non-spectral P-polarized image (P-polarized monochrome luminance image), and a non-corresponding pixel R. A polarized red image and a non-polarized monochrome luminance image corresponding to the pixel W can be obtained. Further, the difference between the luminance value of the pixel R that receives the non-polarized red component and the luminance value of the pixel RP that receives the P-polarized component included in the red component light adjacent thereto is the light of the red component. Corresponds to the S-polarized light component contained in. Therefore, by using the difference between the luminance value of the pixel R and the luminance value of the pixel RP adjacent thereto, an S-polarized image of a red component using the difference value as the pixel value can be obtained. Similarly, by using the difference between the luminance value of the pixel W and the luminance value of the pixel WP adjacent thereto, a white component having the difference value as the pixel value, that is, a non-spectral S-polarized image can be obtained.

[Configuration example 1]
Here, a configuration example of the optical filter 2 and the image sensor 4 in the present embodiment (hereinafter referred to as “configuration example 1”) will be described.
FIG. 7 is an explanatory diagram of the optical filter 2 and the image sensor 4 in the first configuration example.
The optical filter 2 according to Configuration Example 1 includes a transmission region 2a that transmits non-polarized light, a polarization region 2b that transmits a P-polarized component but shields an S-polarized component, and a filter that transmits only a non-polarized red component. The region 2c, a filter that allows only the red component to pass through, and a filter polarization region 2d that combines a polarizer that transmits the P-polarized component but shields the S-polarized component, form a 2 × 2 pixel group, and each group is two-dimensional. It has area division patterns arranged in a grid pattern in the direction. If such an optical filter 2 is used, a pattern in which the pixels of the image sensor 4 are divided into regions as in the above-described embodiment can be obtained without providing a color filter in the image sensor 4. Therefore, it is possible to obtain various images similar to those in the above embodiment in which 2 × 2 pixels on the image sensor 4 are one pixel.

  In the first configuration example, as the image sensor 4, it is possible to use a monochrome-dedicated image sensor in which various types exist. Therefore, it is preferable in that a sensor satisfying a desired sensor specification can be easily obtained.

[Configuration example 2]
Further, another configuration example of the optical filter 2 and the image sensor 4 in the present embodiment (hereinafter referred to as “configuration example 2”) will be described.
FIG. 8 is an explanatory diagram of the optical filter 2 and the image sensor 4 in the second configuration example.
In the optical filter 2 according to Configuration Example 2, a transmissive region 2a that transmits non-polarized light and a polarizing region 2b that transmits a P-polarized component but shields an S-polarized component are in a specific direction (left-right direction in the figure). Have stripe-shaped area division patterns alternately arranged. The optical filter 2 having such a stripe pattern may be adjusted only in the specific direction when adjusting the position with the image sensor 4. In comparison, there is an advantage that the mounting accuracy is relaxed.

  On the other hand, in the image sensor 4 of Configuration Example 2, as in the above embodiment, the color filter region that transmits only the red component and the region without this color filter are alternately arranged in a two-dimensional direction as shown in FIG. The ones arranged in a checkered pattern so as to be adjacent to each other are used. As a result, by combining with the optical filter 2, it is possible to obtain a pattern in which the pixels on the image sensor 4 are divided into regions so that the four types of pixels R, W, RP, and WP form a 2 × 2 pixel group. it can. However, the arrangement of the types of pixels R, W, RP, and WP in each group in Configuration Example 2 is different from that in the above embodiment. However, it is possible to obtain various images similar to those in the above embodiment in which 2 × 2 pixels on the image sensor 4 are one pixel.

[Configuration example 3]
Further, still another configuration example of the optical filter 2 and the image sensor 4 in the present embodiment (hereinafter referred to as “configuration example 3”) will be described.
FIG. 9 is an explanatory diagram of the optical filter 2 and the image sensor 4 in the third configuration example.
In the image sensor 4 of Configuration Example 3, as in the above embodiment, the color filter region that transmits only the red component and the region without this color filter are alternately adjacent in the two-dimensional direction as shown in FIG. In this way, a checkered pattern is used. On the other hand, the optical filter 2 according to the configuration example 3 faces one of two pixels (two pixels that receive non-spectral light) that do not have a color filter on the image sensor 4 in the group of 2 × 2 pixels. As described above, a polarization region 2b that transmits the P-polarized component but shields the S-polarized component is disposed, and the rest has a region division pattern in which the transmissive region 2a that transmits non-polarized light is disposed.

  In the configuration example 3, the P-polarized component of the red component is not received by the image sensor 4. That is, as shown in FIG. 9, the pixels on the image sensor 4 are divided into regions so that three types of pixels (two pixels R and one pixel W and WP) form a 2 × 2 pixel group. Pattern can be obtained. Therefore, according to the configuration example 3, the white component corresponding to the pixel WP, that is, the non-spectral P-polarized image, the non-polarized red image corresponding to the pixel R, and the non-polarized monochrome luminance image corresponding to the pixel W A monochrome S-polarized image can be obtained. As will be described in detail later, the configuration example 3 can be employed when the tail lamp of the preceding vehicle does not distinguish (or does not need to distinguish) disturbance light such as reflected light from the road surface.

[Configuration Example 4]
Further, still another configuration example of the optical filter 2 and the image sensor 4 in the present embodiment (hereinafter referred to as “configuration example 4”) will be described.
FIG. 10 is an explanatory diagram showing the polarization region 2 c of the optical filter 2 in the configuration example 4 superimposed on the image sensor 4.
The optical filter 2 according to Configuration Example 4 has the lattice pattern in which the transmission regions 2a and the polarization regions 2c are alternately arranged in the one-dimensional direction and have a lattice pattern divided into stripes. This is the same as the configuration example 2. However, in the configuration example 2, the longitudinal direction of the polarization region 2c is parallel to the pixel row on the image sensor 4, whereas in the configuration example 4, the longitudinal direction of the polarization region 2c is on the image sensor 4. The pixel array is different in that it is slanted. In the optical filter 2 of Configuration Example 4, as shown in FIG. 10, the polarization region 2c has a width corresponding to one pixel of the image sensor 4 in the horizontal direction in the figure, and two pixels of the image sensor 4 in the vertical direction in the figure. It is arranged obliquely with respect to the pixel row on the image sensor 4 so that one horizontal pixel in the figure is displaced by two vertical pixels in the figure.

  On the other hand, as shown in FIG. 10, the image sensor 4 of the present configuration example 4 includes a color filter region that transmits only a red component and a region without this color filter. (Pixel rows extending in the direction) are used that are arranged in stripes. As a result, by combining with the optical filter 2, it is possible to obtain a pattern in which the pixels on the image sensor 4 are divided into regions so that the four types of pixels R, W, RP, and WP form a 2 × 2 pixel group. it can. However, the arrangement of the types of pixels R, W, RP, and WP in each group in the configuration example 4 is different from that in the above embodiment and the above configuration example 2. However, it is possible to obtain various images similar to those in the above embodiment in which 2 × 2 pixels on the image sensor 4 are one pixel.

  FIG. 11 is an image showing a measurement result when light of an S-polarized component is incident on the optical filter 2 in the present configuration example 4 made by the present inventors. The prototype optical filter 2 is formed by forming a region with a wire grid and a region without a wire grid on a glass plate so that the stripe width is about 6 μm. The former region is a polarizing region 2c, and the latter This area is a transmission area 2a. The measurement is the same as in the case of the above configuration example 2. As shown in FIG. 11, the black portion indicates that the light is shielded, and it is confirmed that the light of the S-polarized component is well cut in the polarization region 2c where the wire grid is formed. it can.

  The optical filter 2 of the present embodiment including these configuration examples 1 to 4 is installed in the front stage of the image sensor 4 of the present embodiment including these configuration examples 1 to 4. In the installation method, the concave-convex structure surface of the optical filter 2 is arranged in the proximity so as to face the light receiving surface of the image sensor 4 and then fixed. As a fixing method, a spacer is disposed around the light receiving surface of the image sensor 4 to hold the optical filter 2, or an adhesive is filled between the concavo-convex structure surface of the optical filter 2 and the image sensor 4 for bonding. The method etc. are mentioned. In the case of the latter method, since an adhesive is filled between the optical filter 2 and the image sensor 4, a high aspect ratio is obtained in order to obtain the same transmittance performance as when the space is an air layer. Therefore, it is difficult to manufacture the structure. In this respect, the former method is preferable.

  As a method of attaching the optical filter 2 to the image sensor 4, a method of fixing the optical filter 2 to the sensor substrate 3 installed on the PWB by a method such as wire bonding (ASSY) later, or an optical filter 2. And a method of mounting the image sensor 4 fixed in advance on the PWB. More desirably, it can be manufactured at a low cost by dicing a wafer obtained by bonding the wafer of the image sensor 4 and the wafer on which the optical filter 2 is formed.

  Next, various pieces of information are obtained from captured image data (a pixel data group indicating the amount of received light received by each light receiving element on the image sensor 4) captured by the imaging device 11 described above. Specifically, the intensity (brightness information) of light emitted from each point (light source) in the imaging region, the distance (distance information) between the light source (other vehicle) such as a headlamp and tail lamp and the own vehicle, Spectral information on red and white components (non-spectral) of light emitted from the light source, polarization information on P-polarized component (or S-polarized component) on the white component (non-spectral) of light emitted from each light source, from each light source Polarization information on the P-polarized component (or S-polarized component) for the red component of the emitted light is obtained.

  The brightness information will be described. When the preceding vehicle and the oncoming vehicle are present at the same distance from the own vehicle at night, when the preceding vehicle and the oncoming vehicle are photographed by the imaging device 11, the detected object is indicated on the photographed image data. One headlamp of the oncoming vehicle is projected brightest, and the tail lamp of the preceding vehicle, which is one of the detection objects, is projected darker than that. In addition, when the reflector is displayed in the captured image data, the reflector is not a light source that emits light by itself, but merely a bright image that is reflected by reflecting the headlamp of the host vehicle, so that it is darker than the tail lamp of the preceding vehicle. Become. On the other hand, the light from the headlamp of the oncoming vehicle, the tail lamp of the preceding vehicle, and the reflector is gradually darkened on the image sensor 4 that receives the light as the distance increases. Therefore, by using the brightness (luminance information) obtained from the captured image data, primary identification of the two types of detection objects (head lamp and tail lamp) and the reflector is possible.

  Further, the distance information will be described. Most headlamps and tail lamps have a pair of left and right pair lamps. Therefore, the distance between the headlamps and tail lamps (that is, other vehicles) and the host vehicle is utilized using the characteristics of this configuration. Can be obtained. The pair of left and right lamps that are paired are displayed close to each other on the same image in the height direction on the captured image data captured by the imaging device 11, and the widths of the lamp image areas that project the lamps are substantially the same. The shape of the lamp image area is almost the same. Therefore, if these characteristics are used as conditions, lamp image areas satisfying the conditions can be identified as pair lamps. At a long distance, the left and right lamps constituting the pair lamp cannot be distinguished and recognized as a single lamp.

  When a pair lamp can be identified by such a method, it is possible to calculate the distance to the light source of the head lamp and tail lamp which are the pair lamp structure. That is, the distance between the left and right headlamps and the distance between the left and right tail lamps of the vehicle can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f of the imaging lens 1 in the imaging device 11 is known, the distance w1 between the two lamp image areas respectively corresponding to the left and right lamps on the image sensor 4 of the imaging device 11 is calculated from the captured image data. Accordingly, the distance x between the light source of the headlamp or taillamp which is the pair lamp configuration and the host vehicle can be obtained by simple proportional calculation (x = f × w0 / w1). If the distance x calculated in this way is within an appropriate range, the two lamp image areas used for the calculation can be identified as headlamps and taillamps of other vehicles. Therefore, by using this distance information, the identification accuracy of the head lamp and tail lamp, which are detection objects, is improved.

  Further, the spectral information will be described. In the present embodiment, as described above, by extracting only pixel data corresponding to the pixel R on the image sensor 4 from the captured image data captured by the imaging device 11, It is possible to generate a non-polarized red image in which only the red component is projected. Similarly, by extracting only pixel data corresponding to the pixel W on the image sensor 4 from the captured image data captured by the imaging device 11, a non-polarized monochrome luminance image in the imaging region can be generated. Therefore, when there is an image region having a luminance equal to or higher than a predetermined luminance in the non-polarized red image, the image region can be identified as a tail lamp image region in which a tail lamp is projected. Furthermore, the luminance ratio (red luminance ratio) between the image area on the non-polarized red image and the image area on the non-polarized monochrome luminance image corresponding to the image area can be calculated. By using this red luminance ratio, it is possible to grasp the ratio of the relative red component contained in the light emitted from the object (light source) existing in the imaging region. Since the red luminance ratio of the tail lamp is sufficiently higher than that of the headlamp and most other light sources, the identification accuracy of the tail lamp is improved by using this red luminance ratio.

Further, the polarization information will be described. In the present embodiment, as described above, by extracting only the pixel data corresponding to the pixel RP on the image sensor 4 from the captured image data captured by the imaging device 11, It is possible to generate a P-polarized red image in which only the P-polarized component included in the red component is projected. Similarly, by extracting only the pixel data corresponding to the pixel WP on the image sensor 4 from the captured image data captured by the imaging device 11, only the non-spectral P-polarized component in the imaging region is projected. A luminance image can be generated.
The S-polarized component can be calculated by subtracting the luminance value of the P-polarized component from the non-polarized luminance value. Therefore, the pixel data corresponding to the pixel RP and the pixel data corresponding to the pixel R on the image sensor 4 are extracted from the captured image data captured by the imaging device 11, and the pixel data of the pixel RP is extracted from the pixel data of the pixel R. By calculating the subtracted pixel data, the pixel data of the S-polarized component included in the red component can be obtained, and an S-polarized red image in which only the S-polarized component included in the red component in the imaging region is displayed is generated. You can also. Similarly, pixel data corresponding to the pixel WP on the image sensor 4 and pixel data corresponding to the pixel W are extracted from the captured image data captured by the imaging device 11, and the pixel data of the pixel WP is extracted from the pixel data of the pixel W. By subtracting the pixel data, pixel data of the non-spectral S-polarized component can be obtained, and an S-polarized monochrome luminance image showing only the non-spectral S-polarized component in the imaging region can be generated. it can.

When such polarization information is used, it is difficult to identify with high accuracy whether or not it is a lamp image area in which a headlamp or a tail lamp is projected only with the brightness information, distance information, and spectral information described above. However, the identification accuracy can be improved.
For example, in general, it is known that light reflected by a mirror surface has an S-polarized component that is always strong, and in particular, the ratio S / P between the S-polarized component and the P-polarized component and the differential polarization degree (SP) / (S + P). It is known that the index value indicating the relative relationship between the S-polarized component and the P-polarized component such as) becomes maximum at a specific angle (Brewster angle). A road surface such as an asphalt surface that does not have water on the road surface on a sunny day is generally a scattering surface. However, when water is attached to the road surface due to rain or the like, the road surface (hereinafter referred to as “rain road surface”). ") Is close to a mirror surface. Therefore, the light reflected on the rainy road surface has a stronger S-polarized component than the light incident on the rainy road surface. Note that the S-polarized light component refers to a laterally polarized light component. The light that is directly incident on the imaging device 11 from the headlamp or tail lamp is usually in a non-polarized state. On the other hand, when light from the headlamp or tail lamp is reflected on the rainy road surface and enters the imaging device 11, the reflected light has a strong S-polarized component.

FIG. 12 shows light directly incident from a headlamp (direct light) and light reflected from a rainy road surface (reflected light) using the imaging device 11 of the present embodiment on a rainy day. It is a histogram which shows the result of having imaged.
In this histogram, the horizontal axis represents differential polarization degree (SP) / (S + P), and the vertical axis represents frequency (normalized to 1). As can be seen from this histogram, the distribution of reflected light is shifted in the direction in which the S-polarized light component is stronger than the direct light.

FIG. 13 is a schematic diagram illustrating an example in which the imaging device 11 captures an image of a situation in which both the preceding vehicle and the oncoming vehicle exist at approximately the same distance ahead in the traveling direction when the host vehicle is traveling on a rainy road surface. FIG.
In such a situation, only the brightness information and distance information distinguish the taillight of the preceding vehicle, the taillight reflected from the rainy road surface, the headlamp of the oncoming vehicle, and the headlamp reflected from the rainy road surface. It is difficult to detect.

  According to the present embodiment, even in such a situation, first, the distinction between the taillight reflected from the tail lamp and the rainy road surface of the preceding vehicle and the headlight from the oncoming vehicle and the headlamp from the rainy road surface. , And can be identified with high accuracy using the spectral information described above. Specifically, in the lamp image area narrowed down using the brightness information and the distance information, the image area in which the pixel value (brightness value) or the red luminance ratio of the red image described above exceeds a predetermined threshold is the tail lamp of the preceding vehicle. Alternatively, the tail lamp image area that reflects the taillight reflected light from the rainy road surface, and the image area that is equal to or less than the threshold is the headlamp image area that reflects the headlight of the oncoming vehicle or the headlamp from the rainy road surface. Identify it.

  Further, according to the present embodiment, the direct light and the reflected light from the tail lamp or the head lamp can be identified with high accuracy by using the polarization information described above for each lamp image area identified by the spectral information in this way. . Specifically, for example, with respect to the tail lamp, the preceding vehicle is utilized by using the difference in the frequency and intensity of the S-polarized component based on the pixel value (luminance value) of the S-polarized red image and the differential polarization degree. Discriminate between direct light from the taillight and taillight reflected from the rain surface. For example, regarding the headlamp, based on the pixel value (brightness value) of the above-described S-polarized monochrome luminance image and the differential polarization degree thereof, the difference in the frequency and intensity of the S-polarized component is used. The direct light from the headlamp and the reflected light of the headlamp from the rain road surface are distinguished.

Next, a flow of detection processing for the preceding vehicle and the oncoming vehicle in the present embodiment will be described.
FIG. 14 is a flowchart showing a flow of vehicle detection processing in the present embodiment.
In the vehicle detection process of the present embodiment, image processing is performed on the image data captured by the imaging device 11, and an image region that is considered to be a detection target is extracted. Then, the preceding vehicle and the oncoming vehicle are detected by identifying which of the two types of detection objects is the type of light source displayed in the image area.

  First, in step S1, image data in front of the host vehicle imaged by the image sensor 4 of the imaging device 11 is taken into a memory. As described above, this image data includes a signal indicating the luminance in each pixel of the image sensor 4. Next, in step S <b> 2, information regarding the behavior of the host vehicle is acquired from the vehicle behavior sensor 12.

  In step S3, an image area with high brightness (high brightness image area) that is considered to be a detection target (tail lamp of the preceding vehicle and bed lamp of the oncoming vehicle) is extracted from the image data captured in the memory. This high luminance image region is a bright region having a luminance higher than a predetermined threshold luminance in the image data, and there are many cases where a plurality of high luminance image regions are extracted, but all of them are extracted. Therefore, at this stage, an image area that reflects reflected light from the rainy road surface is also extracted as a high-luminance image area.

  In the high-luminance image region extraction processing, first, in step S31, binarization processing is performed by comparing the luminance value of each pixel on the image sensor 4 with a predetermined threshold luminance. Specifically, a binary image is created by assigning “1” to pixels having a luminance equal to or higher than a predetermined threshold luminance and assigning “0” to other pixels. Next, in step S32, when pixels assigned with “1” are close to each other in the binarized image, a labeling process for recognizing them as one high luminance image region is performed. Thereby, a set of a plurality of adjacent pixels having a high luminance value is extracted as one high luminance image region.

  In step S4, which is executed after the above-described high-intensity image region extraction process, the distance between the object in the imaging region corresponding to each extracted high-intensity image region and the host vehicle is calculated. In this distance calculation process, the pair lamp distance calculation process that detects the distance by using the pair of left and right lamps of the vehicle and the left and right lamps constituting the pair lamp cannot be distinguished and recognized at a long distance. A single lamp distance calculation process is executed when the pair lamp is recognized as a single lamp.

  First, for a pair lamp distance calculation process, in step S41, a pair lamp creation process, which is a process of creating a lamp pair, is performed. The pair of left and right lamps that are paired are in the position where they are close to each other and have substantially the same height in the image data captured by the imaging device 11, the area of the high-luminance image region is substantially the same, and the shape of the high-luminance image region is the same. Satisfy the condition of being the same. Therefore, the high-intensity image areas that satisfy such conditions are used as a pair lamp. High brightness image areas that cannot be paired are considered a single lamp. When a pair lamp is created, the distance to the pair lamp is calculated by the pair lamp distance calculation process in step S42. The distance between the left and right headlamps of the vehicle and the distance between the left and right taillamps can be approximated by a constant value w0 (for example, about 1.5 m). On the other hand, since the focal length f in the imaging device 11 is known, the actual distance x to the pair lamps is calculated by a simple proportional calculation (x = f · w0 / w1). It should be noted that a dedicated distance sensor such as a laser radar or a millimeter wave radar may be used to detect the distance to the preceding vehicle or the oncoming vehicle.

  In step S5, the ratio between the non-polarized red image and the non-polarized monochrome luminance image is used as spectral information. From this spectral information, whether the two high-intensity image regions that are paired lamps are due to light from the headlamps, A lamp type identification process for identifying whether the light is from the tail lamp is performed. In this lamp type identification process, first, in step S51, the ratio of the pixel data corresponding to the pixel R on the image sensor 4 to the pixel data corresponding to the pixel W on the image sensor 4 for the high-intensity image area that is a pair lamp. A red ratio image with the pixel value as is created. In step S52, the pixel value of the red ratio image is compared with a predetermined threshold, and a high-luminance image region that is equal to or higher than the predetermined threshold is determined to be a tail lamp image region by light from the tail lamp. For a certain high-intensity image area, a lamp type process is performed in which a headlamp image area is formed by light from the headlamp.

  Subsequently, in step S6, for each image region identified as the tail lamp image region and the head lamp image region, light is directly received from the tail lamp or the head lamp using the difference polarization degree (SP) / (S + P) as the polarization information. A reflection identification process is performed to identify whether the light is from the direct light or the light from the tail lamp or the head lamp reflected by a mirror surface such as a rainy road surface. In this reflection identification process, first, in step S61, with respect to the tail lamp image area, the differential polarization degree (S) is calculated using pixel data corresponding to the pixel RP on the image sensor 4 and pixel data corresponding to the pixel W on the image sensor 4. -P) / (S + P) is calculated, and a differential polarization degree image with the differential polarization degree (S-P) / (S + P) as a pixel value is created. Similarly, with respect to the headlamp image region, the degree of differential polarization (S−P) / () using pixel data corresponding to the pixel WP on the image sensor 4 and pixel data corresponding to the pixel W on the image sensor 4. (S + P) is calculated, and a differential polarization degree image with the differential polarization degree (S−P) / (S + P) as a pixel value is created. In step S62, the pixel value of each differential polarization degree image is compared with a predetermined threshold value, and it is determined that the tail lamp image area and the head lamp image area that are equal to or larger than the predetermined threshold value are caused by reflected light. Those image areas are excluded because they do not reflect the tail lamp of the preceding vehicle or the head lamp of the oncoming vehicle. The tail lamp image area and the head lamp image area remaining after performing this exclusion process are identified as those that reflect the tail lamp of the preceding vehicle or those that reflect the head lamp of the oncoming vehicle.

.
Note that the reflection identification process S6 described above may be executed only when a rain sensor or the like is mounted on the vehicle and it is confirmed that the rain sensor is raining. Further, the above-described reflection identification process S6 may be executed only when the driver (driver) is operating the wiper. In short, the above-described reflection identification process S6 may be executed only in rainy weather when reflection from the rainy road surface is assumed.

In the present embodiment, the detection results of the preceding vehicle and the oncoming vehicle detected by the vehicle detection process as described above are used for controlling the light irradiation direction of the headlamp that is an in-vehicle device of the host vehicle.
Specifically, when the tail lamp is detected by the vehicle detection process and approaches the distance range A where the headlight illumination light of the host vehicle is incident on the rearview mirror of the preceding vehicle, the headlight of the host vehicle is approached to the preceding vehicle. Control is performed to block a part of the headlight of the host vehicle or to shift the light irradiation direction of the headlight of the host vehicle in the vertical direction or the horizontal direction so that the illumination light does not strike.
Further, when the bed lamp is detected by the vehicle detection process and the driver of the oncoming vehicle approaches within the distance range B where the headlight illumination light of the own vehicle hits, the headlight illumination light of the own vehicle is approached to the oncoming vehicle. In order not to hit the vehicle, a part of the headlight of the host vehicle is blocked, or the light irradiation direction of the headlight of the host vehicle is shifted in the vertical direction or the horizontal direction.

In the above description, the case where the object detection device according to the present invention is used in a headlamp control system as an in-vehicle device control device has been described as an example. However, the present invention is not limited to this. For example, the object detection device according to the present invention controls an on-board device display monitor to warn of the approach of another vehicle, avoids the collision of the own vehicle with another vehicle, or reduces the impact at the time of the collision The present invention includes an example in which a brake that is an in-vehicle device is controlled in order to automatically adjust the speed of the host vehicle by controlling an accelerator adjusting device that is an in-vehicle device in order to maintain the inter-vehicle distance from the preceding vehicle. Can be applied to various in-vehicle device control devices.
The present invention is not limited to such an in-vehicle device control device, and can be applied to various object identification devices in which it is desired to distinguish between the direct light and the reflected light described above.

  In the above description, the example in which the polarization information is used to distinguish the reflected light (reflected light) from the mirror surface portion such as the rainy road surface from the direct light from the detection target is described. In some cases, this is effective for identification processing other than the distinction between direct light and reflected light. That is, according to the present invention, when there is a difference in polarization information between the direct light from the detection target and other disturbance light, these can be effectively identified.

As described above, the headlamp control system according to the present embodiment captures the surroundings of the host vehicle as an imaging region, and includes at least two types of detection objects that emit light in different wavelength bands among objects existing in the imaging region. Controls the operation of an object detection device that is an object detection means for detecting a headlamp of an oncoming vehicle and a tail lamp of a preceding vehicle, and a headlamp that is an in-vehicle device mounted on the host vehicle based on the detection result of the object detection device. It is an in-vehicle device control device having a headlamp control device 14 as control means. The object detection device used in this headlamp control system includes an imaging device 11 as an imaging unit that receives light emitted from an object existing in the imaging region by the image sensor 4 and images the imaging region, and the imaging device 11. Has a vehicle detection control device 13 as object detection processing means for performing detection processing of the headlamp of the oncoming vehicle and the tail lamp of the preceding vehicle based on the captured image captured by the vehicle. The vehicle detection control device 13 obtains spectral information about each detection wavelength band (red and white) that is a wavelength band of light emitted from the headlamp of the oncoming vehicle and the tail lamp of the preceding vehicle from the captured image captured by the imaging device 11. The differential polarization degree (S) which is polarization information about the detection wavelength band (red and white) corresponding to the spectral information acquired by the spectral information acquisition unit from the acquired spectral information acquisition unit and the captured image captured by the imaging device 11. It functions as polarization information acquisition means for acquiring (−P) / (S + P). And the vehicle detection control apparatus 13 performs the detection process of the headlamp of an oncoming vehicle, and the tail lamp of a preceding vehicle using the spectral information which the spectral information acquisition means acquired, and the polarization information which the polarization information acquisition means acquired. As a result, direct light directly incident from the headlamps of the oncoming vehicle and the tail lamp of the preceding vehicle and the reflected light (disturbance light) from the rainy road surface, which is difficult to identify using only brightness information, distance information, and spectral information Can be identified with high accuracy using polarization information. Therefore, the headlamp operation control by the headlamp control device 14 can be performed with high accuracy.
In particular, in the present embodiment, the image sensor 4 has a pixel array in which the light receiving elements 4a are two-dimensionally arranged, and the imaging device 11 uses light from the imaging region in each detection wavelength band (red and white). A spectral filter as a spectral member that splits light of each detection wavelength band into different light receiving elements on the image sensor 4 and a specific polarization component for the light of each detection wavelength band incident on the image sensor 4 It has an optical filter 2 as a polarizing member having a polarizing region 2b that shields only the S-polarized component and a transmission region 2a that transmits the S-polarized component. Then, an index value R / W based on the amount of light received by each of the light receiving elements R and W that has received light in each detection wavelength band (red and white) is acquired as spectral information for each detection wavelength band. Further, the degree of differential polarization (SP) / (S + P), which is an index value based on the amount of light received by the light receiving elements RP and WP corresponding to the polarization region 2b, is acquired as polarization information. With such a configuration, it is possible to acquire both spectral information and polarization information from one captured image data imaged by one imaging operation by the imaging device 11. Therefore, high-speed object identification processing can be realized.
Further, in the present embodiment, the imaging device 11 captures an image of a forward area in the traveling direction of the host vehicle as an imaging region, and the two types of detection objects are other vehicles that travel in a direction approaching the host vehicle. An oncoming vehicle and a preceding vehicle that is another vehicle traveling in a direction away from the host vehicle, and includes at least spectral information on the wavelength band of the tail lamp color (red wavelength band) of the preceding vehicle and the headlamp color of the oncoming vehicle. Spectral information about the wavelength band (white wavelength band) is acquired. Thereby, a preceding vehicle and an oncoming vehicle can be detected with high accuracy.
In particular, in the present embodiment, using polarization information, normal spectral information about direct light directly incident from a tail lamp of a preceding vehicle and a head lamp of an oncoming vehicle, and a road surface having the same wavelength band as the normal spectral information. The non-regular spectral information about the reflected light is identified, and the preceding vehicle and the oncoming vehicle are detected using the regular spectral information. In the captured image data, if the reflected light reflected from the mirror surface part such as the rain road surface is reflected from the tail lamp of the preceding vehicle or the headlamp of the oncoming vehicle, the reflected light is in the same wavelength band as the direct light. Since the luminance is comparable, it is difficult to distinguish between the regular spectral information about the direct light and the non-regular spectral information about the reflected light. According to this embodiment, since these can be identified with high accuracy, it is possible to suppress erroneous detection of reflected light as a preceding vehicle or an oncoming vehicle.

DESCRIPTION OF SYMBOLS 1 Imaging lens 2 Optical filter 2a Transmission area 2b Polarization area 2c Filter area 2d Filter polarization area 3 Sensor substrate 4 Image sensor 4a Photodiode (light receiving element)
4b Microlens 5 Signal processing unit 11 Imaging device 12 Vehicle behavior sensor 13 Vehicle detection control device 14 Headlamp control device

JP 2004-189229 A

Claims (6)

An image sensor comprising: a spectral member that transmits a spectral component having a predetermined wavelength band; and a polarizing member that transmits a polarized component that vibrates in a predetermined direction. The image sensor receives light emitted from an object existing in the imaging region. Imaging means for imaging the interior;
In an object detection apparatus having object detection processing means for performing detection processing of a detection target including a vehicle based on a captured image captured by the imaging means,
Spectral information acquisition means for acquiring spectral information of the detection object from a captured image captured by the imaging means;
Polarization information acquisition means for acquiring polarization information corresponding to spectral information acquired by the spectral information acquisition means from a captured image captured by the imaging means;
The object detection processing unit includes a first detection process based on the spectral information acquired by the spectral information acquisition unit, a first polarization information about the polarization component oscillating in the predetermined direction acquired by the polarization information acquisition unit, and the The detection process of the detection target is performed by performing a second detection process based on the second polarization information for light oscillating in a direction orthogonal to the predetermined direction calculated from the first polarization information. An object detection device. The object detection device according to claim 1.
The image sensor has a pixel array in which light receiving elements are two-dimensionally arranged,
The spectroscopic member separates light from the imaging region into a plurality of different detection wavelength bands and makes the light in each detection wavelength band enter different light receiving elements on the image sensor,
The polarizing member includes a polarization region that blocks only a specific polarization component of light in each detection wavelength band incident on the image sensor, and a transmission region that transmits unpolarized light,
The spectral information acquisition means acquires an index value based on the amount of light received by each light receiving element that receives light in each detection wavelength band as spectral information for each detection wavelength band,
The polarization information acquisition means acquires, as first polarization information, an index value based on the light reception amount of the light receiving element corresponding to the polarization region, and receives light corresponding to the polarization region from the light reception amount of the light reception element corresponding to the transmission region. An object detection apparatus characterized in that an index value based on an amount obtained by subtracting an amount of light received by an element is acquired as second polarization information. The object detection device according to claim 1 or 2,
The imaging means captures an area in the traveling direction of the host vehicle as an imaging area,
The detection objects include other vehicles that travel in a direction approaching the host vehicle and other vehicles that travel in a direction away from the host vehicle,
The spectral information acquisition means acquires spectral information about at least a wavelength band of a tail lamp color of a vehicle and spectral information about a wavelength band of a head lamp color of the vehicle. The object detection device according to claim 3.
In the second detection process performed by the object detection processing unit, the first polarization information and the second polarization information acquired by the polarization information acquisition unit are directly incident from the tail lamp of the other vehicle and the head lamp of the other vehicle. Identifying regular spectral information about light and non-regular spectral information about light from a road surface having the same wavelength band as the regular spectral information, and using the regular spectral information, proceeding in a direction approaching the host vehicle An object detection apparatus for detecting a vehicle and another vehicle traveling in a direction away from the host vehicle. An object detection means for imaging the periphery of the vehicle as an imaging region, and detecting a detection target including the vehicle existing in the imaging region;
In an in-vehicle device control device having a control means for controlling the operation of the in-vehicle device mounted on the vehicle based on the detection result of the object detection means,
An in-vehicle device control device using the object detection device according to any one of claims 1 to 4 as the object detection means. In the in-vehicle device control device according to claim 5,
Using the object detection device according to claim 3 or 4 as the object detection means,
The control means performs control to change the irradiation direction of a headlamp which is an in-vehicle device mounted on the vehicle or to change the emission intensity of the headlamp based on the detection result of the object detection means. In-vehicle device control device.