JP2010079706A - Object detector for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve accuracy of detecting an obstacle while simplifying a device configuration and reducing costs in an obstacle detection device 1 which is provided with a front monitor camera 3 for capturing a view ahead of a vehicle through a windshield glass from the inside of a car to detect an obstacle ahead of the vehicle on the basis of the image to be processed captured by the front monitor camera 3. <P>SOLUTION: The object detector for a vehicle is provided with: a predetermined condition image storage part 103 for storing a predetermined condition image in which surroundings are reflected in a windshield glass; a reflected image estimation part 115 for estimating a reflected image included in the image to be processed on the basis of the predetermined condition image stored in the predetermined condition image storage part 103; a differential image generation part 108 for generating a differential image between the image to be processed and the reflected image; and an obstacle detection part 109 for detecting an object on the basis of the generated differential image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention includes an imaging camera that is mounted on a vehicle and images the outside of the vehicle through a window glass from the interior of the vehicle, and an object that exists outside the vehicle based on a processing target image captured by the imaging camera. The present invention belongs to a technical field related to a vehicle object detection device to be detected.

  Conventionally, a vehicle object detection that detects white lines and obstacles on a traveling road by performing image processing on an image outside the vehicle imaged through a window glass by an in-vehicle camera provided in the vehicle interior of the vehicle. The device is known. In such a detection device, for example, when imaging the outside of the vehicle through the windshield with an in-vehicle camera, the surrounding dashboard and the mounted objects on the dashboard are reflected on the windshield by reflection of light, There is a problem that the reflected image is erroneously detected as a detection target during image processing, and various techniques have been proposed so far to prevent this erroneous detection.

For example, in Patent Document 1, an illumination device that alternately turns on and off in accordance with the imaging timing of the in-vehicle camera is provided near the dashboard, and the difference between the captured image during illumination and the captured image during non-illumination is provided. An image is created, and a reflected image is detected by extracting a pixel having a small temporal variation in the pixel value of each pixel of the created difference image.
JP 2002-230563 A

  However, in the vehicle object detection device shown in the above-mentioned Patent Document 1, since it is necessary to mount the illumination device in the vehicle interior, there is a problem that the number of parts increases and the cost of the device increases, In order to prevent a reduction in the detection accuracy of the reflected image due to variations in the illuminance of the light source, a dedicated feedback circuit for controlling the illuminance is required, which causes a problem that the apparatus is complicated and the cost is further increased.

  The present invention has been made in view of such a point, and an object of the present invention is to include an imaging camera that images the outside of the vehicle from the interior of the vehicle through the window glass, and the captured image ( For the vehicle object detection device that detects an object existing outside the vehicle based on the processing target image), the device configuration is simplified and the cost is reduced by devising the configuration. The object is to improve the detection accuracy of the object.

  In order to achieve the above object, in the present invention, a predetermined condition image storage unit that stores in advance a predetermined condition image in which the periphery is reflected on the window glass, and an image pickup camera based on the predetermined condition image. An image estimation unit that estimates a reflected image included in the processing target image, and a difference image generation unit that generates a difference image between the processing target image and the predetermined condition image, based on the generated difference image Object detection was performed.

  Specifically, the invention of claim 1 includes an imaging camera that is mounted on a vehicle and images the outside of the vehicle through a window glass from the interior of the vehicle, and the vehicle is based on a processing target image captured by the imaging camera. The object is a vehicle object detection device that detects an object existing outside.

  And the predetermined condition image storage part which memorize | stores the predetermined condition image imaged with the said imaging camera under the predetermined conditions which the surroundings reflect on the said window glass, and the predetermined condition memorize | stored in the said predetermined condition image storage part A reflected image estimation unit that estimates a reflected image corresponding to the reflected portion included in the processing target image captured by the imaging camera based on a condition image, and a processing target captured by the imaging camera A difference image generation unit that generates a difference image by performing a difference process between an image and a reflection image in the processing target image estimated by the reflection image estimation unit, and a difference generated by the difference image generation unit An object detection unit that detects the object based on the image is provided.

  According to this configuration, the predetermined condition image captured by the imaging camera under the predetermined condition in which the surrounding reflection of the windshield occurs is stored in the predetermined condition image storage unit, and the reflection image estimation unit performs the predetermined condition. A reflection image included in the processing target image is estimated based on the image, and a difference image estimation unit generates a difference image between the estimated reflection image and the processing target image, and the target detection unit Thus, the object detection is executed based on the difference image.

  As described above, the reflected image estimation unit estimates the reflected image in the processing target image based on the predetermined condition image stored in advance in the predetermined condition image storage unit. Without being provided, it is possible to reliably estimate the reflected image included in the processing target image with a simple configuration, and it is possible to reliably prevent erroneous detection in the target object detection unit.

  According to a second aspect of the present invention, in the first aspect of the invention, the predetermined condition is a condition in which a fixed vehicle around the window glass is reflected, and the reflected image estimation unit is the predetermined condition image storage unit. A fixed object image estimation unit that estimates a reflected image of the vehicle fixed object included in the processing target image based on a predetermined condition image stored in the processing target image, and a predetermined period before the imaging of the processing target image A mounted object image estimation unit that estimates a reflected image of the mounted object on the vehicle fixed object included in the processing target image based on a plurality of sample images captured by the imaging camera, and the fixed The reflected image of the vehicle fixed object estimated by the object image estimation unit and the reflected image of the placement object estimated by the above-described object image estimation unit are superimposed, and the superimposed image is used for the difference processing. And the above reflection image Assume that a that the image superimposing section.

  According to this configuration, the fixed object image estimation unit estimates the reflected image of the vehicle fixed object included in the processing target image based on the predetermined condition image stored in the predetermined condition image storage unit, and the mounted object In the image estimation unit, based on the sample image captured by the imaging camera within a predetermined period before imaging of the processing target image, the reflected image of the mounted object on the fixed vehicle included in the processing target image is displayed. The estimated reflected image of the vehicle fixed object and the reflected image of the placed object are superimposed in the image superimposing unit, and the reflected image for the difference processing (hereinafter referred to as the all reflected image) is obtained. Generated.

  Here, with respect to a vehicle fixed object such as a dashboard, since there is almost no secular change in position and shape, the reflected image is based on the predetermined condition image stored in advance in the predetermined condition image storage unit as described above. By estimating, the estimation can be performed easily and with high accuracy. In addition, since the position and shape of magazines and the like placed on a fixed vehicle are frequently changed, as described above, based on the sample images captured by the imaging camera within a predetermined period. By estimating the reflected image, the estimation can be performed with high accuracy. Therefore, it is possible to improve the estimation accuracy of the entire reflected image in which both the vehicle fixed object and the mounted object image are reflected, and further, it is possible to further prevent erroneous detection in the object detection unit. It becomes.

  In the invention of claim 3, in the invention of claim 2, the above-mentioned figurine image estimation unit compares a plurality of sample images taken by the imaging camera within the predetermined period, and a variance value of pixel values is predetermined. It is assumed that a pixel that is less than the value is calculated, and a reflected image of the above-described figurine included in the processing target image is estimated based on the calculated pixel.

  According to this configuration, the reflected image of the placement object can be reliably estimated with high accuracy in the placement object image estimation unit. That is, unlike the landscape image, the reflected image of the mounted object has a small temporal change in the pixel value of each pixel, so that the variance of the pixel value of each pixel between the sample images is lower than that of the landscape image. . Therefore, as in the present invention, the placement object image estimation unit calculates pixels whose variance value is less than the predetermined value, and estimates the reflected image of the placement object included in the processing target image based on the calculation result. By doing so, the estimation accuracy can be improved.

  According to a fourth aspect of the present invention, in the second aspect of the present invention, the imaging camera is a stereo camera including a pair of cameras that capture the outside of the vehicle, and the processing target image is one of the pair of cameras. And a parallax amount calculating unit that searches for a corresponding point between the images captured by the pair of cameras and calculates a parallax amount between the two images for each corresponding point, The above-described figurine image estimation unit is configured to estimate an image of a corresponding point area where the parallax amount calculated by the parallax amount calculation unit is larger than a predetermined amount as a reflected image of the above-described figurine And

  According to this configuration, the reflected image of the placement object can be reliably estimated with high accuracy in the placement object image estimation unit. That is, when the stereo camera is arranged on the upper side of the vehicle fixed object as in the present invention, the placed object is positioned closer to the stereo camera than the vehicle fixed object, and thus the image is captured by the stereo camera. The amount of parallax of the reflected image of the mounted object (the amount of parallax between the images captured by the pair of cameras) is larger than the amount of parallax (predetermined amount) of the reflected image of the fixed vehicle. Therefore, as in the present invention, the placement object image estimation unit extracts a corresponding point area in which the parallax amount calculated by the parallax amount calculation unit is larger than a predetermined amount from the processing target image, thereby The reflected image can be estimated with high accuracy.

In addition, in the invention according to claim 5, it is possible to reduce the processing load of the corresponding point search in the processing target image by setting an area where the reflected image of the vehicle fixed object and the mounted object is likely to be generated as the predetermined area. 5. The image sensor according to claim 1, further comprising an illuminance sensor that detects ambient illuminance of the vehicle fixed object, wherein the reflected image estimation unit estimates the reflected image. Comparing the detected illuminance of the illuminance sensor at the time of capturing the predetermined condition image with the processing target image and correcting the pixel value of each pixel of the predetermined condition image based on the comparison result It is assumed that

  According to this configuration, even if the ambient illuminance (detected illuminance detected by the illuminance sensor) of the vehicle stationary object at the time of capturing the processing target image is different from the illuminance at the time of capturing the predetermined condition image, the reflected image estimation unit In estimating the image, the detected illuminance of the illuminance sensor at the time of both imaging is compared, and the pixel value of the predetermined condition image is corrected based on the comparison result. For this reason, the reflected image in the processing target image can be estimated with high accuracy regardless of the ambient illuminance.

  According to a sixth aspect of the present invention, in the invention according to any one of the first to fifth aspects, when the reflected image estimation unit estimates the reflected image, the imaging camera captures the predetermined condition image, and Comparing at least one of the camera gain and exposure value of the imaging camera at the time of imaging the predetermined target image with the imaging camera, and correcting the pixel value of each pixel of the predetermined condition image based on the comparison result It shall be configured.

  According to this configuration, even if the setting value (at least one of the camera gain and the exposure value) of the imaging camera at the time of processing target image imaging is different from the setting value at the time of imaging the predetermined condition image, the reflected image estimation unit In estimating the reflected image, the set values at the time of both imaging are compared, and the pixel value of each pixel of the predetermined condition image is corrected based on the comparison result. For this reason, even if the setting value is changed after the predetermined condition image is captured by the imaging camera, the reflected image in the processing target image can be estimated with high accuracy.

  According to a seventh aspect of the present invention, in the first to sixth aspects of the invention, an image storage mode for storing the predetermined condition image in the predetermined condition image storage unit, and an object detection mode for performing the object detection, It is further assumed that mode switching means for switching and setting is provided.

  According to this configuration, the mode switching means can switch between the image storage mode for storing the predetermined condition image and the object detection mode for detecting the object. For this reason, even if the color of the vehicle fixed object changes over time, the reflected image in the reflected image estimation unit is set by setting the image storage mode with the mode switching means and recapturing the predetermined condition image with the imaging camera. It is possible to prevent a decrease in the estimation accuracy.

  According to an eighth aspect of the present invention, in the seventh aspect of the invention, the mode switching means includes an ignition key switch of the vehicle, and is configured to be able to switch between the two modes by a predetermined operation of the key switch. It shall be.

  As described above, the mode switching means is configured by using the existing ignition key switch installed in the vehicle, so that the vehicle occupant can easily perform the mode switching, and the overall cost of the apparatus is low. Can be achieved.

  According to a ninth aspect of the invention, in the invention according to any one of the first to eighth aspects, the windshield is a front windshield of a vehicle, and the vehicle fixed object is a vehicle mounted along a lower edge of the front windshield. It includes a dashboard provided on the indoor side.

  According to this configuration, the reflected image of the dashboard (vehicle fixed object) on the windshield and the reflected image of a magazine or the like (mounted object) placed on the dashboard by the occupant is displayed. Thus, the object detection unit can detect the object in front of the vehicle with high accuracy and reliability.

  As described above, according to the vehicle object detection device of the present invention equipped with an imaging camera that is mounted on a vehicle and images the outside of the vehicle from the interior of the vehicle through the windshield, the surroundings are reflected on the windshield. A predetermined condition image storage unit that stores a predetermined condition image in advance, an image estimation unit that estimates a reflected image included in the processing target image by the imaging camera based on the predetermined condition image, the processing target image and the predetermined condition A difference image generation unit that generates a difference image with the image, and by performing the object detection based on the generated difference image, the reflected image included in the processing target image is used as the object. It is possible to reliably prevent erroneous detection and improve object detection accuracy.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a vehicle A equipped with an obstacle detection device 1 as a vehicle object detection device according to an embodiment of the present invention. This vehicle A is equipped with a vehicle driving support system that detects an obstacle (target object) in front of the vehicle and notifies its presence to the occupant, and the obstacle detection device 1 is applied as a part of this system. ing.

  The obstacle detection device 1 includes a front monitoring camera 3 (for example, a CCD camera or a CMOS camera) as an imaging camera that is disposed in the front part of the passenger compartment and images the front of the vehicle through the windshield 2. An obstacle in front of the vehicle is detected by performing image processing on an image captured by the front monitoring camera 3 (hereinafter referred to as a processing target image). When the obstacle detection device 1 performs image processing of the processing target image, the vehicle stationary object 10 (in this embodiment, on the dashboard 6 and the dashboard 6 reflected in the windshield 2 (see FIG. 5)). The speaker 8 and the meter hood 9) and the placed object 7 (for example, a magazine) placed thereon are prevented from being erroneously determined as an obstacle.

  Here, in the present embodiment, the “vehicle fixed object” means that which is fixedly installed on the vehicle A from the time of factory shipment, and the “placed object” is placed on the vehicle fixed object 10 after factory shipment. It shall mean the one (including the one fixedly installed).

  As shown in FIG. 2, the obstacle detection device 1 includes a detection ECU 100 that processes the image data output from the front monitoring camera 3 to detect an obstacle, and the detection ECU 100 detects the obstacle. Sometimes, an operation signal is output to the alarm device 110 so as to issue an alarm to the occupant. As the alarm device 110, a speaker 8 that emits a warning sound upon receiving an operation signal, a display device that displays a warning on a display, or the like can be employed.

  In the present embodiment, only one front monitoring camera 3 is disposed at the center in the vehicle width direction of the front part of the passenger compartment (on the back side of the rearview mirror), and the captured image is converted into luminance information (luminance value (pixel value)). Output to the detection ECU 100 as an image made up of n (= i rows × j columns) pixels.

  The detection ECU 100 includes a well-known microcomputer including a CPU, a ROM, a RAM, and the like. The detection ECU 100 determines the illuminance around the ignition key switch 4 for starting the engine, which is also used for mode setting described later, and the dashboard 6. The illuminance sensor 5 to be detected and the forward monitoring camera 3 are connected so as to be able to send and receive signals. The mode setting unit 101, the illuminance storage unit 102, the predetermined condition image storage unit 103, and the captured image storage unit 104 , A fixed object image estimation unit 105, a placement object image estimation unit 106, an image superimposition unit 107, a difference image generation unit 108, and an obstacle detection unit 109.

  When the ignition key switch 4 is operated by an occupant, the mode setting unit 101 detects an object detection mode for detecting an object and a predetermined condition image storage mode for storing a predetermined condition image, which will be described later, based on the operation signal. Switch between and. Specifically, when the occupant performs a predetermined operation of the ignition key switch 4 (for example, an operation of executing switching of ACC on and off three times in succession), the mode setting unit 101 Assuming that there is a mode switching request by the passenger, the currently set mode is switched to a different mode.

  The illuminance storage unit 102 stores the detected illuminance of the illuminance sensor 5 in time series. Note that the illuminance detected by the illuminance sensor 5 at the time of capturing a predetermined condition image described later is also stored in the illuminance storage unit 102.

  The predetermined condition image storage unit 103 stores an image captured by the front monitoring camera 3 under a predetermined condition as a predetermined condition image when the predetermined condition image storage mode is set. Here, in the present embodiment, the predetermined condition is that the placement object 7 does not exist on the vehicle fixed object 10, and the ambient illuminance of the vehicle fixed object 10 is reflected on the front windshield 2. It is a condition that it is in the illuminance range in which the confusion occurs.

  The captured image storage unit 104 sequentially stores, as sample images, images captured by the front monitoring camera 3 before the processing target image is captured.

  The fixed object image estimation unit 105 displays the reflected image of the vehicle fixed object 10 included in the processing target image based on the predetermined condition image stored in the predetermined condition image storage unit 103 and the detected illuminance of the illuminance sensor 5. presume. Specifically, the fixed object image estimation unit 105 sets the pixel value of each pixel other than the region corresponding to the vehicle fixed object 10 in the predetermined condition image to 0, and sets the pixel value of each pixel to L (= processing). The reflected image of the vehicle fixed object 10 is estimated by performing pixel value correction that is multiplied by the detected illuminance of the illuminance sensor 5 at the time of capturing the target image / the detected illuminance of the illuminance sensor 5 at the time of capturing the predetermined condition image.

  The placement object image estimation unit 106 displays the reflected image of the placement object 7 included in the processing target image based on N (N = 5 in this embodiment) sample images stored in the captured image storage unit 104. presume. Specifically, a plurality of sample images are compared to determine a pixel number whose variance value is less than a predetermined value, that is, a pixel number whose pixel value change with time changes is small. Then, after extracting the pixel area corresponding to the pixel number from the processing target image, the pixel area corresponding to the vehicle fixed object 10 is erased (the pixel value is set to 0), so that the placement object 7 is reflected. Estimate the embedded image.

  The image superimposing unit 107 superimposes an image obtained by superimposing the reflected image of the vehicle fixed object 10 estimated by the fixed object image estimating unit 105 and the reflected image of the mounted object 7 estimated by the mounted object image estimating unit 106. Estimated as all reflected images in the processing target image. Thus, the fixed object image estimation unit 105, the placement object image estimation unit 106, and the image superimposing unit 107 constitute a reflected image estimation unit 115.

  As shown in FIG. 6, the difference image generation unit 108 performs a difference process on the processing target image and the entire reflected image estimated by the image superimposing unit 107 to generate a difference image.

  The obstacle detection unit 109 performs an obstacle detection process on the difference image generated by the difference image generation unit 108. Specifically, in this embodiment, pedestrians, vehicles, two-wheeled vehicles, and the like are detected as obstacles, and the obstacle detection unit 109 performs the above operation based on these template images stored in advance. By performing the pattern matching process on the difference image, it is detected whether there is an obstacle ahead of the vehicle. The obstacle detection unit 109 outputs an operation signal to the alarm device 110 when an obstacle is detected.

  The alarm device 110 receives an operation signal from the obstacle detection unit 109 and issues a warning sound to the occupant to notify that there is an obstacle ahead of the vehicle.

  Next, the obstacle detection process by the detection ECU 100 will be described with reference to the flowchart of FIG.

  In the first step S1, various information from the front monitoring camera 3, the ignition key switch 4, and the illuminance sensor 5 is read.

  In step S2, it is determined whether or not the object detection mode is set. When this determination is YES, the process proceeds to step S3, and when it is NO, the process proceeds to step S9.

  In step S3, it is determined whether or not the detected illuminance detected by the illuminance sensor 5 is within a predetermined illuminance range set in advance. When this determination is YES, the process proceeds to step S4, and when it is NO, the process returns to step S1. Here, the predetermined illuminance range is an illuminance range in which the surroundings (the vehicle fixed object 10 and the mounted object 7) are reflected on the windshield 2, and is obtained in advance through experiments or the like.

  In step S4, a predetermined period before the current captured image (processing target image) is captured (the time required to capture N images by the front monitoring camera 3 within the immediately preceding predetermined period). N images captured by the front monitoring camera 3 are read from the captured image storage unit 104 as sample images. Then, based on these sample images, a reflected image of the placement object 7 included in the current captured image (processing target image) is estimated. Details of the reflected image estimation processing of the mounted object 7 will be described later.

  In step S5, the predetermined condition image stored in the predetermined condition image storage unit 103 is read, and the above-described pixel value correction is performed on the predetermined condition image based on the detected illuminance of the illuminance sensor 5, thereby fixing the vehicle fixed object. Ten reflected images are estimated.

  In step S6, the reflected image of the placed object 7 and the reflected image of the vehicle fixed object 10 estimated in steps S4 and S5, respectively, are superimposed (added) to estimate the entire reflected image.

  In step S7, the forward monitoring camera 3 performs a difference process on the captured image (processing target image) captured this time and the entire reflected image estimated in step S6 to generate a difference image.

  In step S8, the obstacle detection process is executed on the difference image generated in step S7 (detecting whether there is an obstacle), and if an obstacle is detected, an alarm is issued. An activation signal is output to the device 110.

  In step S9 that proceeds when the determination in step S2 is NO (step S9 that proceeds when the image storage mode is set), the captured image of the front monitoring camera 3 is stored as a predetermined condition image in the predetermined condition image storage unit 103. Let

  Next, the reflected object image estimation process included in the obstacle detection process by the detection ECU 100 will be described with reference to the flowchart of FIG.

  First, in the first step SA41, the pixel number k of the read N sample images is designated. When the processing of this step SA41 is performed for the first time, the pixel number is set to 1, and thereafter, the designated number is incremented by one each time the processing of this step SA41 is executed. In addition, the relationship between the pixel located in x row xy column in each sample image and the pixel number k is expressed as k = (x-1) × j + y.

  In step SA42, for each of the N sample images, a pixel with the pixel number k designated in step SA41 is extracted, and an average value ave of these pixel values is calculated.

  In step SA43, the variance value V is calculated based on the following equation.

・・・・・（１）

(1)

  In step SA44, it is determined whether or not the variance value V is smaller than a predetermined value. When this determination is NO, the process proceeds to step SA46, and when it is YES, the process proceeds to step SA45.

  In step SA45, the current designated pixel number k is stored.

  In step SA46, it is determined whether or not the designated pixel number is equal to the total number of pixels n, that is, whether or not the calculation process of the variance value V for all the pixels of each sample image has been completed. While returning to step SA41, if YES, the process proceeds to step SA47.

  In step SA47, an area corresponding to the pixel number k stored in step SA45, that is, an area where the variance value V is less than a predetermined value is extracted from the processing target image.

  In step SA48, an image in which the pixel value of the region corresponding to the vehicle fixed object 10 in the image extracted in step SA47 is set to 0 is estimated as a reflected image of the mounted object 7, and then the process returns.

  As described above, in the first embodiment, the reflected image estimation unit 115 (image superimposing unit 107) estimates the entire reflected image in the processing target image, and the difference image generation unit 108 determines the reflected image and the processing target image. And the obstacle detection unit 109 executes obstacle detection processing on the generated difference image (see FIG. 6). Here, the reflected image estimation unit 115 reflects the reflected image of the vehicle fixed object 10 estimated by the fixed object image estimation unit 105 and the reflected image of the placement object 7 estimated by the placement object image estimation unit 106. Are superimposed to estimate the total reflected image. Accordingly, it is possible to reliably prevent erroneous detection of the vehicle fixed object 10 or the placement object 7 included in the processing target image as an obstacle.

  In the first embodiment, the fixed object image estimation unit 105 (the reflected image estimation unit 115) is based on the detected illuminance of the illuminance sensor 5 when the front monitoring camera 3 captures the predetermined condition image and when the processing target image is captured. Furthermore, the reflected image of the vehicle fixed object 10 included in the processing target image is estimated by correcting the luminance of each pixel of the predetermined condition image stored in advance in the predetermined condition image storage unit 103. Thus, the reflected image can be estimated easily and accurately without being affected by the ambient illuminance of the vehicle fixed object 10.

  In the first embodiment, the placement object image estimation unit 106 compares the n sample images to calculate a pixel number k whose variance value is less than a predetermined value (performs low-pass filter processing). The region image corresponding to the calculated pixel number k in the processing target image is estimated as the reflected image of the placement object 7. Here, in the first embodiment, the low-pass filter processing in the placed object image estimation unit 106 is performed only when the illuminance detected by the illuminance sensor 5 is within a predetermined illuminance range, that is, within the illuminance range in which reflection on the windshield 2 occurs. By doing so, it is possible to improve the processing accuracy of the low-pass filter processing, and as a result, the estimation accuracy of the reflected image of the mounted object 7 in the processing target image is made as much as possible. It becomes possible to improve.

  Further, in the first embodiment, an image storage mode for storing the predetermined condition image in the predetermined condition image storage unit 103 and an object detection mode for detecting an object are determined by the predetermined key of the ignition key switch 4. It can be easily switched by operation.

  By doing this, even after the vehicle A is shipped from the factory, by operating the ignition key switch 4 and setting the image storage mode, the front monitoring camera 3 can re-capture a predetermined condition image. Can do. Therefore, for example, even if the surface color of the dashboard 6 that is the vehicle fixed object 10 changes with time, the image of the reflected image in the fixed object image estimation unit 105 can be obtained by re-imaging the predetermined condition image as described above. A decrease in estimation accuracy can be reliably prevented.

(Embodiment 2)
7 and 8 show the second embodiment of the present invention, in which the configuration of the front monitoring camera 3 and the detection ECU 100 and the reflected image estimation processing of the mounted object 7 by the detection ECU 100 are different from those of the first embodiment. It is. In FIG. 7, the same components as those in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted as appropriate.

  In this embodiment, the front monitoring camera 3 is configured by a stereo camera 3 including a pair of left and right cameras 3L and 3R, and the left camera 3L and the right camera 3R are arranged so that their imaging areas overlap each other. ing. In the following description, an image captured by the left camera 3L is referred to as a left captured image GL, and an image captured by the right camera 3R is referred to as a right captured image GR.

  The detection ECU 100 includes a mode setting unit 101, an illuminance storage unit 102, a predetermined condition image storage unit 103, a captured image storage unit 104, a fixed object image estimation unit 105, a placement object image estimation unit 106, and an image superimposition unit. In addition to 107, the difference image generation unit 108, and the obstacle detection unit 109, it further includes a parallax amount calculation unit 111 and a parallax map storage unit 112.

  The parallax amount calculation unit 111 searches for a corresponding point with the right camera image GR in a predetermined region U (see FIG. 9) of the left camera image GL, and calculates a parallax amount between the two images GL and GR for each corresponding point. Based on the corresponding point information from the parallax amount calculation unit 111, the placement object image estimation unit 106 estimates the reflected image of the placement object 7 included in the processing target image. The corresponding points are points that are associated with each other between the both captured images GL and GR. For example, the corresponding point search can be performed based on an area-based matching method that is a known method.

  Further, the parallax map storage unit 112 stores a parallax map M used for image estimation in the mounted object image estimation unit 106. The parallax map M is obtained by imaging the virtual vehicle fixed object that the vehicle fixed object 10 reflected on the windshield 2 actually exists there with the two cameras 3L and 3R. The parallax amount Dd (hereinafter referred to as the theoretical parallax amount Dd) of each corresponding point is mapped. The parallax map M is a geometric relationship (positional relationship and shape) among the vehicle fixed object 10 (dashboard 6, speaker 8 and meter hood 9), the windshield 2, and the cameras 3L and 3R, which are known in advance. Is created based on the principle of triangulation and stored in advance in the parallax map storage unit 112 before shipment from the factory.

  In the present embodiment, the object detection process by the detection ECU 100 is executed exclusively on the left-side captured image GL. Specifically, what is obtained by changing the “processing target image” to the “left captured image” in the flowchart shown in FIG. 3 is the target object detection processing of the present embodiment. The placed object reflection image estimation process (the process of step S4) in the detection ECU 100, which is different from 1, will be described in detail with reference to FIG.

  First, in the first step SB41, a corresponding point with the right image GR is searched within a predetermined area U of the left captured image GL. Here, the predetermined region U is a region that is set in advance as a region where a reflected image of the vehicle fixed object 10 and the mounted object 7 is likely to be generated based on the positional relationship between the vehicle fixed object 10 and the windshield 2. It is.

  In step SB42, a parallax amount D between the images GL and GR (hereinafter referred to as measured parallax amount D) is calculated for each corresponding point searched in step SB41.

  In step SB43, the theoretical parallax amount Dd of each corresponding point is read from the parallax map M stored in the parallax map storage unit 112.

  In step SB44, the corresponding point area where the actually measured parallax amount D is larger than the theoretical parallax amount Dd is extracted from the left-side captured image GL and is estimated as a reflected image of the placement object 7, and then the present estimation process ends.

  As described above, in the above-described embodiment, the placement object image estimation unit 106 extracts from the processing target image the corresponding point region in which the actual parallax amount D calculated by the parallax amount calculation unit 111 is larger than the theoretical parallax amount Dd. Thus, it is estimated as a reflected image of the mounted object 7. The estimation principle is as follows. That is, when the placing object 7 is placed on the vehicle fixed object 10, the distance between the virtual placing object and the front monitoring camera 3 on the assumption that the placing object 7 reflected on the windshield 2 actually exists there. Is larger than the distance between the virtual vehicle fixed object and the front monitoring camera 3. Therefore, the amount of parallax between the images GL and GR when the virtual placement object is imaged with both cameras 3L and 3R is the same between the images GL and GR when the virtual vehicle fixed object is imaged with both cameras 3L and 3R. Is less than the theoretical parallax amount Dd. Therefore, as described above, the placement object image estimation unit 106 extracts the corresponding point region in which the measured parallax amount D is larger than the theoretical parallax amount Dd from the processing target image, thereby accurately displaying the reflected image of the placement object 7. It becomes possible to estimate well and reliably.

(Other embodiments)
The configuration of the present invention is not limited to the above embodiment, but includes various other configurations. That is, in each of the above embodiments, the fixed object image estimation unit 105 (the reflected image estimation unit 115) compares the detected illuminance of the illuminance sensor 5 when the front monitoring camera 3 captures the predetermined condition image and when the processing target image is captured. The luminance correction of the predetermined condition image is performed based on the above. For example, the luminance correction of the predetermined condition image is performed based on the comparison of the camera setting value (at least one of the camera gain and the exposure value) at the time of both imaging. You may make it perform. Specifically, for example, if the camera gain at the time of capturing the processing target image is I times that at the time of capturing the predetermined condition image, the fixed object image estimation unit 105 multiplies the pixel value of each pixel of the predetermined condition image by I. The image may be estimated as a reflected image of the mounted object. Similarly, if the exposure value at the time of processing target image capturing is J times that at the time of capturing the predetermined condition image, the fixed object image estimating unit 105 performs the predetermined condition image. An image obtained by multiplying the pixel value of each pixel by J may be estimated as a reflected image of the mounted object.

  In each of the above embodiments, the vehicle fixed object 10 is defined as being fixedly installed on the vehicle A from the time of factory shipment, and the mounted object 7 is defined as being placed on the vehicle fixed object 10 after factory shipment. However, the present invention is not limited to this. For example, a vehicle such as a radar detector on the dashboard 6 that is fixed on the vehicle fixed object 10 and hardly moves after the vehicle A is shipped from the factory. It may be included in the object 10.

  In each of the above embodiments, the obstacle detection in front of the vehicle is performed based on the difference image generated by the difference image generation unit 108. However, the present invention is not limited to this. For example, white line detection is performed. May be performed.

  In the above embodiments, the front of the vehicle is imaged through the front window glass 2 by the front monitoring camera 3. However, the present invention is not limited to this. For example, the rear of the vehicle is imaged through the rear window glass. May be.

  In the second embodiment, the parallax map M is created based on the principle of triangulation based on the positional relationship and shape between the vehicle fixed object 10 and the cameras 3L and 3R. For example, based on the captured images GL and GR actually captured by the cameras 3L and 3R of the vehicle stationary object 10 reflected on the windshield 2, the correspondence between the images GL and GR The parallax map M may be created by mapping the parallax amount of the points.

  In the above embodiment, the mode is switched by predetermined operation of the ignition key switch 4. However, the present invention is not limited to this. For example, a ground terminal provided in the engine room is grounded. Alternatively, the image storage mode may be set, and the object detection mode may be set by releasing the ground.

  The present invention includes an imaging camera that is mounted on a vehicle and images the outside of the vehicle through a windshield from a vehicle interior, and detects an object existing outside the vehicle based on a captured image captured by the imaging camera. The present invention is useful for a vehicle object detection device, and particularly useful for a vehicle object detection device that images the front of a vehicle through a windshield using an imaging camera.

It is the schematic diagram which looked at the vehicle carrying the obstacle detection apparatus 1 as a vehicle object detection apparatus which concerns on embodiment of this invention seen from the upper side. 3 is a block diagram illustrating a control configuration of the obstacle detection apparatus 1. FIG. It is a flowchart which shows the obstruction detection process in detection ECU100. It is a flowchart which shows the mounting object reflection image estimation process included in the obstruction detection process in detection ECU100. It is a schematic diagram which shows the reflection image to a windshield. It is a schematic diagram which shows the difference process of a process target image and a all reflected image. FIG. 3 is a view corresponding to FIG. FIG. 5 is a view corresponding to FIG. It is a schematic diagram which shows the picked-up image by a stereo camera. It is a schematic diagram which shows the parallax map used for discrimination | determination of the reflected image of the mounting object contained in a process target image.

Explanation of symbols

A Vehicle 1 Obstacle detection device (vehicle object detection device)
2 Front window glass (wind glass)
3 Front surveillance camera (imaging camera, stereo camera)
3L left camera (imaging camera)
3R Right camera (imaging camera)
4 Ignition key switch (mode switching means)
5 Illuminance sensor 6 Dashboard (vehicle stationary)
7 Placed object 10 Vehicle fixed object 101 Mode setting section (mode switching means)
103 Predetermined Condition Image Storage Unit 105 Fixed Object Image Estimation Unit 106 Placed Object Image Estimation Unit 107 Image Superimposition Unit
108 Difference image generation unit 109 Obstacle detection unit (object detection unit)
111 Parallax amount calculation unit 115 Reflected image estimation unit

Claims (9)

For a vehicle that includes an imaging camera that is mounted on a vehicle and images the outside of the vehicle through a windshield from a vehicle interior, and that detects an object existing outside the vehicle based on a processing target image captured by the imaging camera An object detection device comprising:
A predetermined condition image storage unit for storing a predetermined condition image captured by the imaging camera under a predetermined condition in which the surrounding reflection of the windshield occurs;
Based on the predetermined condition image stored in the predetermined condition image storage unit, a reflected image estimation unit that estimates a reflected image corresponding to the reflected portion included in the processing target image captured by the imaging camera. When,
A difference image generation unit that generates a difference image by performing a difference process between the processing target image captured by the imaging camera and the reflection image in the processing target image estimated by the reflection image estimation unit;
An object detection device for a vehicle, comprising: an object detection unit that detects the object based on the difference image generated by the difference image generation unit. The vehicle object detection device according to claim 1,
The predetermined condition is a condition in which a fixed vehicle around the window glass is reflected,
The reflected image estimation unit is
A fixed object image estimating unit that estimates a reflected image of the vehicle fixed object included in the processing target image based on the predetermined condition image stored in the predetermined condition image storage unit;
Based on a plurality of sample images captured by the imaging camera within a predetermined period prior to the time when the processing target image is captured, the placement object on the vehicle fixed object included in the processing target image is reflected. A placement object image estimation unit for estimating an image;
The reflected image of the vehicle fixed object estimated by the fixed object image estimating unit and the reflected image of the mounted object estimated by the above-described figurine image estimating unit are superimposed, and the superimposed image is converted into the difference. An object detection apparatus for a vehicle, comprising: an image superimposing unit that uses the reflected image for processing. The vehicle object detection device according to claim 2,
The above-mentioned figurine image estimation unit compares a plurality of sample images captured by the imaging camera within the predetermined period, calculates pixels whose variance value of pixel values is less than a predetermined value, and calculates the calculation result An object detection device for a vehicle characterized by being configured to estimate a reflected image of the above-mentioned figurine included in the processing object image. The vehicle object detection device according to claim 2,
The imaging camera is composed of a stereo camera composed of a pair of cameras that capture the outside of the vehicle, and is disposed on the upper side with respect to the vehicle fixed object,
The processing target image is an image captured by one of the pair of cameras,
In a predetermined area in the processing target image captured by the one camera, a corresponding point with the image captured by the other camera is searched, and a parallax amount between the two images is calculated for each corresponding point. A parallax amount calculating unit;
The figurine image estimation unit described above is configured to estimate an image of a corresponding point area where the parallax amount calculated by the parallax amount calculation unit is larger than a predetermined amount as a reflected image of the above figurine. An object detection device for vehicles. The vehicle object detection device according to any one of claims 1 to 4,
An illuminance sensor for detecting the illuminance around the vehicle fixed object is further provided,
The reflected image estimation unit compares the detected illuminance of the illuminance sensor between the time when the predetermined image is captured by the imaging camera and the time when the target image is captured when the reflected image is estimated. A vehicle object detection device configured to correct a pixel value of each pixel of the predetermined condition image based on a result. In the vehicle object detection device according to any one of claims 1 to 5,
When the reflected image estimation unit estimates the reflected image, at least one of the camera gain and the exposure value of the imaging camera is captured when the predetermined condition image is captured by the imaging camera and when the processing target image is captured. Comparing and correcting the pixel value of each pixel of the predetermined condition image based on the comparison result, the vehicle object detection device characterized by the above. The vehicle object detection device according to any one of claims 1 to 6,
The apparatus further comprises mode switching means for switching between an image storage mode for storing the predetermined condition image in the predetermined condition image storage unit and an object detection mode for performing the object detection. An object detection device for vehicles. The vehicle object detection device according to claim 7,
The vehicle object detection device, wherein the mode switching means includes an ignition key switch of the vehicle, and is configured to be able to switch between the two modes by a predetermined operation of the key switch. The vehicle object detection device according to any one of claims 1 to 8,
The windshield is the front windshield of the vehicle,
The vehicle object detection device, wherein the vehicle fixed object includes a dashboard provided on a vehicle interior side along a lower edge of the windshield.

Images