JP2007293672A - Photographing apparatus for vehicle and soiling detection method for photographing apparatus for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing apparatus for a vehicle detecting soiling of the photographing apparatus at high precision and a soiling detection method for the photographing apparatus for the vehicle. <P>SOLUTION: The photographing apparatus for the vehicle 1 equipped with a photographing means 13 for photographing the perimeter of one's own vehicle is provided with a soiling self diagnosis means 14d detecting soiling of the photographing means 13 and a soiling diagnosis permitting means 14e permitting detection of the soiling to the soiling self diagnosis means 14d when an object detecting means 2 detecting an object in the perimeter of the vehicle detects the object. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a vehicular imaging apparatus and a dirt detection method for a vehicular imaging apparatus, and more particularly to a vehicular imaging apparatus and a vehicular imaging apparatus that can perform a self-diagnosis of a dirt.

2. Description of the Related Art A vehicular imaging apparatus is known that assists a driver in driving a vehicle by capturing an image of the front or rear of a vehicle with a camera to detect an obstacle or a traveling lane. Since the performance of a vehicular photographing apparatus deteriorates when dirt is attached to a lens of a camera or the like, a technique for detecting dirt or a technique for executing predetermined fail-safe control when detected is proposed (for example, Patent Documents). 1). In Patent Document 1, when the image processing apparatus cannot detect a lane from an image in front of the vehicle, the number of times is counted, and when the number of times within a predetermined time exceeds a threshold value, it is determined that dirt is attached to the optical system. To do. Since the adhesion of dirt is detected based on the image thus taken, an additional device for detecting dirt is not necessary, and cost and installation space can be saved.
JP-A-8-202998

  However, the dirt detection method described in Patent Document 1 has a problem that, for example, dirt cannot be detected on a road without a white line, or it is erroneously determined that dirt is attached because a white line cannot be detected.

  Under certain conditions, if the contamination of the vehicle photographing device cannot be detected, the detection of the performance deterioration is delayed, and if it is erroneously determined, the driver may feel annoyed by sounding an alarm or wiping the lens. There is not preferable.

  In view of the above problems, an object of the present invention is to provide a vehicular shooting apparatus and a vehicular shooting apparatus dirt detection method capable of accurately detecting dirt on the shooting apparatus.

  In order to solve the above-described problems, the present invention provides a vehicular imaging apparatus including an imaging unit that captures an image of the surroundings of the host vehicle, a dirt self-diagnosis unit that detects contamination of the imaging unit, and an object detection that detects an object around the vehicle. When the means detects an object, it has dirt diagnosis permission means for permitting the dirt self-diagnosis means to detect dirt.

  According to the present invention, dirt self-diagnosis is permitted when an object is detected. Therefore, even when the amount of light is not sufficient, such as at night, the image of the detected object is accurately collected using the image of the detected object. Can be detected.

  It is possible to provide a vehicular imaging apparatus and a vehicular imaging apparatus dirt detection method capable of accurately detecting contamination of the imaging apparatus.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

  FIG. 1 shows an example of a schematic configuration diagram of an object detection device 2 connected to a vehicle photographing device 1 and an in-vehicle LAN 18 in the present embodiment. Since the vehicular photographing apparatus 1 permits a self-diagnosis of dirt, which was conventionally prohibited during night driving, when a preceding vehicle or the like is detected, dirt can be accurately detected even during night driving. . In the present embodiment, a three-dimensional object such as a preceding vehicle around the vehicle or a laying object such as a guard rail may be referred to as an object.

  The vehicle photographing device 1 includes cameras 13R and 13L (cameras 13R and 13L are simply referred to as a camera 13) configured by photographing devices 12R and 12L and lenses 11R and 11L, and constitutes a known stereo camera. The imaging devices 12R and 12L include, for example, CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device) imaging elements. Image data (hereinafter referred to as a frame) taken by each camera is output to a stereo ECU (Electronic Control Unit) 14. In addition to the cameras 13R and 13L, another camera may be provided.

  The stereo ECU 14 includes a CPU (Central Processing Unit) that executes a program, a ROM that stores the program, a RAM that temporarily stores data and calculation results, a non-volatile memory (NV-RAM) that stores data and parameters, and other ECUs An input / output interface for inputting / outputting data between them is configured as one unit connected by a bus, and is also called a microcomputer or a computer.

  The stereo ECU 14 includes a geometric conversion processing unit 14a, a LUT (Look Up Table) 14c, an image recognition processing unit 14b, a dirt self-diagnosis unit 14d, and a dirt diagnosis permission unit 14e, and the CPU executes a geometric conversion process by executing a program. The unit 14a, the image recognition processing unit 14b, the dirt self-diagnosis unit 14d, and the dirt diagnosis permission unit 14e are realized.

  Light incident on the cameras 13R and 13L from the front of the vehicle is photoelectrically converted by the image sensor, and the accumulated charge is read out as a voltage, amplified and A / D converted, and then a predetermined luminance gradation (for example, 256 gradations). Converted to a digital image. The cameras 13R and 13L shoot at a frame rate of about 30 to 60 times / second, for example, and sequentially output the captured frames to the stereo ECU 14 while holding them in a predetermined buffer or the like.

  As will be described later, since the stereo ECU 14 detects the distance to the object based on the parallax of the left and right cameras 13R and 13L, the left and right cameras 13R and 13L cause error factors other than parallax (lens distortion, optical axis deviation, focal length deviation). And image sensor distortion etc.). Since the error factor is stored in the LUT 14c as calibration data by calibration, the geometric conversion processing unit 14a removes the error factor by referring to the LUT and converting whatever camera 13R, 13L is connected. Output a frame.

  The image recognition processing unit 14b performs image recognition processing based on the geometrically converted frame from the geometric conversion processing unit 14a. The image recognition process is a process of detecting a three-dimensional object and a distance to the three-dimensional object for each frame imaged by, for example, a triangulation technique.

  In the image recognition processing, for example, the same point in the three-dimensional space is associated with the left and right frames taken almost simultaneously by the cameras 13R and 13L arranged on the left and right, and the three-dimensional position of the point is obtained in the manner of triangulation. .

  The image recognition processing unit 14b calculates the distance to the imaging object by obtaining the parallax at the same point in the frames shot by the cameras 13R and 13L. When the imaging target is ahead, the imaging target is shifted in the horizontal direction when the frame by the camera 13R and the frame by the camera 13L are overlapped. While shifting one frame pixel by pixel, for example, the luminance between the pixels is compared, and the shift amount having the highest correlation (for example, the city block distance) between the pixels is detected as parallax.

  Assuming that the number of pixels shifted to have the highest correlation is n, the focal length of the lens is f, the distance between the optical axes is m, and the pixel pitch is d, the distance L to the imaging object is L = (f The relational expression m) / (n · d) holds. This (n · d) is called parallax. Note that the height of the same point from the road surface in the three-dimensional space can be calculated from the height of the cameras 13R and 13L from the road surface.

  The dirt self-diagnosis unit 14d detects dirt on the left and right cameras 13R and 13L. When the cameras 13R and 13L are inside the vehicle, the dirt is a sticker attached to the front windshield or rear glass, adhering mud, accessories hung inside the vehicle, etc., and when the cameras 13R and 13L are outside the vehicle, the dirt Is mud or the like adhering to the lenses 11R and 11L. That is, the dirt is an object between the cameras 13R and 13L and an object in front of the vehicle that hinders normal photographing. Hereinafter, the camera 13 is simply referred to as dirt regardless of the form of dirt.

  The dirt self-diagnosis unit 14d compares the frames with the left and right cameras 13R and 13L to detect dirt on the other or both cameras. Further, the dirt diagnosis permission unit 14e permits the dirt self-diagnosis unit 14d to detect dirt based on whether or not the object detection device 2 has detected an object. Details of these will be described later.

  By mounting the vehicle photographing device 1 on the vehicle, if there is a preceding vehicle or a white line ahead of the vehicle, the distance is detected, and the image recognition processing result is required via the in-vehicle LAN 18 such as CAN (Controller Area Network). The result is transmitted to the other ECU. Other ECUs include, for example, an ECU that controls a collision avoidance / collision reduction system, an ECU that controls a lane keeping support system and a lane departure warning system, an inter-vehicle control ECU, a brake ECU, and the like.

  For example, in the lane keeping support system, an area having a luminance equal to or higher than a predetermined threshold is searched upward from the bottom of the frame based on, for example, the luminance of the captured frame. Since the white line has edges that are high-frequency components at both ends, if the frame in front of the vehicle is differentiated in the horizontal direction, peaks are obtained at both ends of the white line, and the peaks are within the white line, from the white line to the white line, and from the white line to the white line. Since it is obtained in the positive and negative directions, the white line portion can be estimated. By applying a white line enhancement filter that performs such processing to the frame, the white line portion can be emphasized. From the frame in which the white line is emphasized, there are characteristics such as white line characteristics, high brightness, and a linear shape. A white line can be detected in the region by a technique such as matching. The ECU of the lane keeping support system recognizes a white line up to, for example, about 100 m, determines the approximate center as a target travel line, and assists the steering torque so that the host vehicle does not deviate more than a predetermined distance from the target travel line.

  The object detection device 2 detects an object in front of or around the vehicle using a millimeter wave radar, a laser radar, or the like. The radar device 15 detects the distance to the obstacle and the relative speed based on the time from when the radar is transmitted until it is received, and sends it to the obstacle ECU 16.

  The radar device 15 includes an antenna unit, a high-frequency circuit unit, and a signal processing unit. The high-frequency circuit unit transmits a radio signal whose frequency linearly increases (or decreases) with time from the transmitting antenna, and the receiving antenna Receives the reflected wave returned from the object.

  Since the received wave travels a distance that is longer than the transmitted wave by the distance L to the object, it has a time delay with respect to the transmitted wave by 2 × L / speed of light. It will be low (high). For this reason, interference occurs between the transmission wave and the reception wave according to the distance to the object (a beat signal is output), and the signal processing unit detects the distance to the object based on the beat signal. Further, the relative speed can be obtained by analyzing the frequency of the beat signal.

  Since the speed of the host vehicle is acquired by the vehicle speed sensor in the obstacle ECU 16, the speed of the object can be detected. For example, when an abnormal approach to an object is detected, a signal is output to a pre-crash ECU, a brake ECU, or the like, and these ECUs perform vehicle control for object avoidance or support.

  By the way, the space A1 in which the object detection device 2 detects an object and the space A2 in which the vehicle imaging device 1 images and detects an object do not completely overlap in the three-dimensional space. FIG. 2 is a plan view showing a state where the host vehicle is traveling. A range A1 indicates a space in which an object can be detected by the radar device 15, a range A2 indicates a space in which the object can be detected by the vehicle imaging device 1, and a hatched portion indicates a space overlapping the spaces A1 and A2.

  The vehicle photographing apparatus 1 of the present embodiment utilizes the fact that an object is detected by both the vehicle photographing apparatus 1 and the object detection apparatus 2 in a space that overlaps the spaces A1 and A2.

  Next, a procedure for permitting the self-diagnosis of the cameras 13R and 13L by the dirt diagnosis permission unit 14e will be described with reference to the flowchart of FIG. FIG. 3A is a flowchart for determining whether or not to allow the dirt self-diagnosis unit 14d to perform the dirt self-diagnosis. The determination procedure of FIG. 3A is performed at predetermined time intervals while the vehicle is traveling at a predetermined speed or higher.

  The dirt diagnosis permission unit 14d first determines whether or not the current time is night (S1). Whether it is night or not means that the brightness required for photographing the front and surroundings of the vehicle by the vehicle photographing device 1 is not obtained from the sun, and is determined by, for example, the shutter time. Since the exposure of the frame changes according to the shutter time (shutter speed), the cameras 13R and 13L secure the light amount by extending the shutter time if the surroundings are dark. Therefore, when the shutter time is equal to or greater than the predetermined threshold value, it is understood that there is not enough brightness for photographing the front and surroundings of the vehicle, and it is nighttime. If it determines in this way, unlike the case where it determines by the vehicle-mounted clock, it can be determined that it is night at the time of driving | running | working a tunnel in the daytime. In addition, you may make an illuminance meter whether it is nighttime.

  When it is not nighttime (No in S1), that is, during the daytime, a self-diagnosis of dirt is permitted (S2). Diagnosis is permitted during the day because the surroundings are bright and a frame with sufficient information is captured in the camera, so that dirt can be detected accurately.

  In the nighttime (Yes in S1), the dirt diagnosis permission unit 14d determines whether or not the object detection device 2 detects a preceding vehicle ahead of the host vehicle (S3). When the object detection device 2 detects the preceding vehicle ahead of the host vehicle (Yes in S3), the dirt diagnosis permission unit 14d permits the dirt self-diagnosis (S2).

  When a preceding vehicle is detected ahead of the host vehicle by the radar device 15, the same preceding vehicle should have been photographed by the vehicle imaging device 1, and it has been found that there is a preceding vehicle even at night. If this is the case, it can be determined that self-diagnosis of the dirt can be performed by performing image processing on the frame in which the preceding vehicle is photographed.

  Therefore, the space in which the preceding vehicle is detected by the object detection device 2 is a space where at least the spaces A1 and A2 in FIG. 2 overlap, and more preferably within a predetermined distance (for example, within 20 m) where the preceding vehicle is photographed to a certain extent. ).

  When the object detection device 2 does not detect the preceding vehicle ahead of the host vehicle (No in S3), the dirt diagnosis permission unit 14d prohibits the self diagnosis of dirt (S4). That is, when the preceding vehicle is not detected at night, the dirt self-diagnosis is not performed.

  Next, the self-diagnosis of dirt will be described based on the flowchart of FIG. The dirt self-diagnosis in FIG. 3B is executed immediately after it is determined to be permitted by the determination procedure of whether to permit the self-diagnosis in FIG.

  When the object detection device 2 detects the preceding vehicle, the dirt self-diagnosis unit 14d preferably determines whether or not the frame captured by the cameras 13R and 13L includes edge information more than a predetermined value immediately after that (S10). ). The edge information is a change in the pixel value of adjacent pixels, but in the present embodiment, the change in the edge information is a predetermined change or more.

  Several methods have been proposed for detecting edge information. As a known method, for example, the pixel value of a frame is obtained by first-order differentiation or second-order differentiation in the horizontal direction and the vertical direction. Specifically, a 3 × 3 matrix is multiplied by an operator as a frame to obtain a total of nine calculation results. Since the result is an analog amount including a negative value, the absolute value thereof is taken, and compared with a certain threshold value, if it is larger than the threshold value, it is set as edge information.

  During the day, even when there is no preceding vehicle, edge information more than a predetermined value is detected in a frame obtained by photographing the front of the host vehicle by photographing a white line, a guardrail, a pedestrian, and the like. Further, when it is determined that a preceding vehicle is present at night, predetermined edge information such as a preceding vehicle and background, a tail lamp of the preceding vehicle, a license plate, a boundary between the body and the rear glass, and the like are detected.

  Further, when dirt is attached, the dirt adheres to unfocused positions such as the lenses 11R and 11L, the windshield 17, etc., so that the change in the pixel value of the adjacent pixels becomes gradual, and a predetermined number of edges or more. Information is not detected.

  In step S3 in FIG. 3 (a), the presence / absence of the preceding vehicle at night is determined in order to obtain edge information. Since edge information is detected when a typical laying object is also photographed, the presence or absence of the laying object as well as the preceding vehicle may be determined.

  Therefore, when the edge information of a predetermined number or more is not detected (No in S10), the dirt self-diagnosis unit 14d determines that dirt is attached to the camera 13R or the camera 13L (S20). The determination may be based on the ratio of the edge information to the frame area instead of the number of edge information. In addition, when edge information is not detected from both the camera 13R or the camera 13L, it can be determined that both are dirty.

  When a predetermined number or more of edge information is detected (Yes in S10), it is determined whether there is a difference between frames taken by the camera 13R and the camera 13L (S30). Since the vehicular imaging apparatus 1 is calibrated so that there is no difference other than parallax, it is determined that there is no difference between frames with respect to the difference in parallax. For example, the dirt self-diagnosis unit 14d shifts one frame by parallax (n · d) and compares the pixel values for each pixel. For example, if the total difference in pixel values is greater than or equal to a predetermined value, it is determined that there is a difference greater than or equal to the parallax between the left and right frames.

  When there is a difference between frames taken by the camera 13R and the camera 13L (No in S30), the dirt self-diagnosis unit 14d determines that dirt is attached to the camera 13R and the camera 13L (S20). For example, when dirt is attached to a part of the shooting range, edge information of a predetermined number or more may be detected. However, it is possible to detect a part of the shooting range by determining the frame difference. it can.

  If there is no difference between the frames photographed by the camera 13R and the camera 13L (Yes in S30), the dirt self-diagnosis unit 14d determines that dirt is not attached to the camera 13R and the camera 13L and ends the process (S40).

  Even when dirt is detected, it may be erroneously detected as dirt due to noise in the frame or dirt detection process, etc., because only one frame is detected for a predetermined time or a predetermined number of frames. It is preferable to determine that dirt has been detected.

  Further, when there is only one camera that captures an object, self-diagnosis of dirt can be performed with only one camera by determining the presence or absence of dirt by the presence or absence of edge information (S10).

  When the dirt is detected, the dirt self-diagnosis unit 14d sends a signal to the navigation ECU and the panel to notify the driver that the camera 13R or 13L is dirty. As a result, the driver can take appropriate measures such as removing dirt.

  As described above, according to the vehicular imaging apparatus 1 of the present embodiment, it is possible to detect camera dirt even under conditions such as nighttime where a sufficient amount of light cannot be obtained. Preferably, if a preceding vehicle or a frame with edge information is photographed, the self-diagnosis of the dirt can be executed, so that the dirt can be accurately detected without selecting conditions such as a road without a white line. Further, there is no need for an additional device for detecting dirt, and cost and installation space can be saved.

It is a schematic block diagram of the object detection apparatus connected with the imaging device for vehicles by in-vehicle LAN. It is a top view of a mode that the own vehicle is drive | working. It is a flowchart figure which shows the procedure of permission of self-diagnosis of the dirt of a camera, and a self-diagnosis.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle imaging device 2 Object detection apparatus 11R, 11L Lens 12R, 12L Image pick-up element 13R, 13L Camera 14 Stereo ECU
15 Radar equipment
16 Obstacle ECU
17 Windshield 18 Car LAN

Claims (3)

In the vehicular shooting device provided with shooting means for shooting the surroundings of the own vehicle,
A dirt self-diagnosis means for detecting dirt on the photographing means;
When the object detection means for detecting an object around the vehicle detects an object, a dirt diagnosis permission means for permitting the dirt self-diagnosis means to detect dirt;
An imaging device for a vehicle, comprising:   2. The vehicle photographing apparatus according to claim 1, wherein when the object is detected within a predetermined distance in front of the host vehicle, the dirt diagnosis permission means permits the detection of dirt. In the method for detecting dirt in a vehicle photographing device including photographing means for photographing the surroundings of the host vehicle,
When an object is detected around the vehicle by the object detection means for detecting an object around the vehicle, the stain diagnosis permission means permits the detection of the stain; and
If the detection of dirt is permitted, a step of detecting dirt on the photographing means by a dirt self-diagnostic means;
A dirt detection method for a vehicle photographing device.

Images