JP2007172504A - Adhering matter detection device and adhering matter detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect that adhering matter to one's own vehicle or a camera is present in a photographed image of the camera photographing a part in front of the own vehicle by an extremely little image processing burden. <P>SOLUTION: Each pixel point of an image characteristic amount is detected from image data of the photographed image of the monocular camera 2 by an image characteristic detection means, a detection result is held in a storage means, each the pixel point wherein the detection results of the image characteristic detection means accord continuously at least for a prescribed time is detected by an accordance point detection means, a basic retrieval area and a sub retrieval area shifted therefrom are set in the photographed image by a retrieval area setting means, and an area of the adhering matter having a counted value of a threshold value or above is detected by processing having the extremely little burden of a decision means counting the number of the detection accordance points of the accordance detection means about each area of both the retrieval areas to decide that there is the adhering matter. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a deposit detection apparatus and a deposit detection method for detecting a deposit on a host vehicle or a camera included in a captured image of a camera that captures the front of the host vehicle.

  Conventionally, in a vehicle equipped with a color or black and white camera such as a CCD monocular camera or a stereo camera for the purpose of target recognition in front of the host vehicle, the front of the host vehicle is continuously moved by the camera while the host vehicle is running. Take a picture, apply known image processing such as differential binarization to the captured image (luminance image) from moment to moment, and recognize the target and white line of the preceding vehicle etc. from each differential binary image obtained by the image processing Is going.

  By the way, the camera is attached to a rearview mirror mounting column or the like in the vehicle, and images the front (outside of the vehicle) of the vehicle through the imaging lens and the windshield of the vehicle.

  For this reason, for example, the vehicle or camera deposits such as sludge or raindrops attached to the windshield of the vehicle, dust attached to the windshield, or dust adhering to the shooting lens may appear in the captured image as noise. A recognition error or the like occurs due to the noise of the adhered matter.

Therefore, each pixel point of the contour (edge) is detected as a feature amount of the captured image of the camera and connected to form a line segment of the contour, and there is no time change in the shape and position of the contour of the line segment. From the above, it is proposed to detect the noise of the deposit by specifying the outline of the deposit (see, for example, Patent Document 1).
Japanese Patent No. 365541 (Claims, [0007], [0018]-[0023], FIG. 1, FIG. 3, etc.)

  As in the prior art, each pixel point of the contour (edge) of the photographed image of the camera is connected to form a contour line segment, and the contour of the deposit in the captured image is identified from the line segment, and there is an deposit Is detected, complicated image processing such as the above-mentioned connection and contour line differentiation is necessary, and there is a problem that the image processing load increases.

  It is an object of the present invention to detect the presence of an own vehicle or camera deposits in a photographed image of a camera that photographs the front of the own vehicle with a very small image processing load.

  In order to achieve the above-described object, an attached matter detection apparatus according to the present invention is an attached matter detection device that detects the presence of an attached matter in a photographed image preceding a host vehicle of a camera mounted on the own vehicle. Image feature detection means for detecting each pixel point of the image feature amount from the image data of the image, storage means for holding the detection result of the image feature detection means, and the image feature detection means for the latest image data of the captured image And the detection result of the image feature detection unit for the image data of the past photographed image held in the storage unit, and the detection result of the image feature detection unit coincides at least for a predetermined time. A matching point detecting means for detecting each pixel point, a basic search area composed of a plurality of areas obtained by dividing the photographed image in a matrix, and the photographed image in a matrix Search area setting means for setting a sub search area in which the basic search area is shifted to an arbitrary position in the vertical and horizontal directions so that each divided area straddles each area of the basic search area; And a determination unit that counts the number of coincidence points detected by the coincidence point detection unit for each area of each region, and detects an adhering area where the count value is equal to or greater than a threshold value, and determines that there is an adhering matter. It is characterized (claim 1).

  Further, the attached matter detection method of the present invention is an attached matter detection method for detecting an attached matter in a photographed image preceding a host vehicle of a camera mounted on the host vehicle, wherein an image feature amount is detected from image data of the photographed image. Each pixel point is detected and the detection result is retained, and the feature amount detection result for the latest image data of the captured image and the feature amount detection result for the retained image data of the past captured image are stored. Comparing and detecting each pixel point where the detection results coincide with each other for at least a predetermined time, and in the captured image, a basic search region including a plurality of areas obtained by dividing the captured image into a matrix, and the captured image A sub-search area in which the basic search area is shifted to an arbitrary position in the vertical and horizontal directions so that each area divided into a matrix crosses each area of the basic search area, and the both search areas Counting the detected matching score of each area in respectively, the count value is and determines that there deposits by detecting the area of deposits equal to or greater than the threshold value (claim 2).

  According to the first and second aspects of the present invention, each pixel point of the deposit from which the image feature amount is detected continuously for a predetermined period or longer of the captured image of the camera has each of the basic search areas obtained by dividing the captured image into a matrix. The area and the basic search area are divided and counted in units of each area of the sub search area shifted to an arbitrary position in the vertical and horizontal directions.

  At this time, even if each pixel point of the adhering substance is located at a position where the pixel points are distributed over a plurality of areas in the division of the basic search area, those pixel points are included in one area in the division of the sub search area. The count value is not small. On the other hand, even if each pixel point of the adhering substance is located at a position that is distributed across a plurality of areas in the division of the sub search area, those pixel points are included in one area in the division of the basic search area. The count value is not small. For this reason, with respect to the attached matter, the count value of the pixel point in each area at the position of either the basic or the sub-search area is equal to or greater than the threshold value, and it can be reliably detected that the attached matter is present.

  Then, as in the prior art, each pixel point of the contour (edge) of the photographed image of the camera is connected and line segmentation (grouping) is performed to form a line segment of the contour, and the adhered matter in the photographed image is formed from the line segment Instead of detecting the presence of an adhering substance by specifying the contour of the image, it is determined that there is an adhering substance from the count of pixel points for each area where the captured image is divided, so the process is simple and the image processing burden is light. It is possible to detect the presence of the subject vehicle or camera deposits in the photographed image of the camera that photographs the front of the subject vehicle with a very small image processing load.

  Next, in order to describe the present invention in more detail, an embodiment will be described in detail with reference to FIGS.

  FIG. 1 is a block configuration diagram of the own vehicle 1, FIG. 2 shows a search state of the monocular camera 2 mounted on the own vehicle 1, and FIGS. 3A and 3B are times tn and tn + 1 of the monocular camera 2. Photographed images Pn and Pn + 1 including the attached matter TG are shown.

  4 is an explanatory diagram of pixel points of a captured image of the control ECU, which will be described later, and FIGS. 5A and 5B are explanatory diagrams of edge pixel points and an enlarged view of a part thereof. (A), (b) are explanatory diagrams of the basic search area and the sub-search area, and FIG. 7 is a flowchart for explaining the processing of the control ECU.

  As shown in FIG. 1, the host vehicle 1 detects a target such as a preceding vehicle in front of the host vehicle or a white line on the road surface, determines the possibility of collision with the target, etc., and displays an alarm display, automatic brake control, steering In order to perform avoidance control and the like, a monocular camera 2 of a color or black and white CCD and a radar 3 such as a laser radar and a millimeter wave radar are mounted, and the monocular camera 2 and the radar 3 repeatedly search in front of the vehicle.

  Further, the traveling state of the host vehicle 1 is detected by various sensors such as a vehicle speed sensor (not shown), a rudder angle sensor 4 and a yaw rate sensor 5.

  Then, the control ECU 6 having a microcomputer configuration that controls the entire host vehicle 1 performs image processing on the captured image of the monocular camera 2 every moment to detect a target such as a preceding vehicle in front of the host vehicle, and the radar 3 sometimes Process every moment exploration results to detect similar targets, and recognize and collide with the above targets by well-known sensor fusion processing based on both detection results and detection results of each sensor 4, 5 etc. Judging sex etc. Then, based on this determination and the switching of the control switch 7, the so-called following travel (distance between vehicles) is normally controlled by the throttle opening control of the throttle control unit 8, the brake control of the brake control unit 9, and the AT control of the AT control unit 10. (Constant) travel control. Here, when the possibility of collision becomes high, the display warning unit 11 warns the driver, performs automatic braking control of the units 8 to 10, and further performs power steering collision avoidance control of the steering control unit 12 as necessary. I do.

  As shown in FIG. 2, the monocular camera 2 is attached, for example, to the support base of the rearview mirror 13 in the interior of the host vehicle 1 so as to face the front of the host vehicle, and has a photographing range facing diagonally below the solid lines a and b in the figure. Shoot.

  At this time, as shown in (a) and (b) of FIG. 3, dirt adhered to the windshield 14 of the vehicle 1, sludge and raindrops attached to the windshield 14, or dust attached to the photographing lens of the monocular camera 2. For example, in the captured images Pn and Pn + 1 at time tn and tn + 1, the deposit TG is present at the same position in the image.

  When the host vehicle speed becomes equal to or higher than the set value, the control ECU 6 executes the adhering matter detection program as one of the programs for image processing of the captured images of the monocular camera 2 every moment, and includes the following units. Note that when the vehicle speed is lower than the set value or when the vehicle stops, such as when the signal is stopped, the portions other than the deposit TG of the captured images Pn and Pn + 1 are the same, and the detection of the deposit TG is difficult. The attached matter detection program is not executed.

(A) Image feature detection means This means detects each pixel point of the image feature amount from the image data of the photographed image, and rewrites the detection result for the set time in a RAM or the like as a storage means of the control ECU 6. . Specifically, the image data of the captured image is differential binary image data (edge binary image data) of the brightness of the captured image, and the differential binary as an image feature amount of the differential binary image data has an edge. Each pixel point of the luminance edge that becomes “1” is detected.

  At this time, as shown in FIG. 4, each pixel point p of the luminance edge that becomes “1” for each pixel point p at the upper, lower, left and right matrix-like pixel points p in the approximately central region of interest ROI of the photographed image Pt. When the inspection is performed, the number of pixel points p to be processed is extremely large, and not only the processing load of the control ECU 6 is increased, but also the stationary deposit TG is slightly shifted for each captured image Pt depending on the traveling state. There is also.

  For this reason, as shown in FIGS. 5A and 5B, in this embodiment, for example, detection pixel points pi that are search points are set at intervals of 8 pixel points p in the vertical and horizontal directions, For each detection pixel point pi, the total number of 9 (3 × 3) pixel points pi, p of the eight pixel points p surrounding each detection pixel point pi and the pixel points pi is greater than the set number. Whether or not the detected pixel point pi has an edge of “1” is determined depending on whether or not. Only the detection point pixels pi determined in this way are searched for and detected whether or not there is an edge of “1”. Note that the dashed arrow in FIG. 5A indicates the scanning direction (detection direction).

(B) Matching point detection means This means includes the detection result of the image feature detection means for the latest (currently obtained) image data of the monocular camera 2 and the past ( The detection result of the image feature detection unit with respect to the image data of the captured image (up to a certain time before the present) is compared, and each detection pixel point pi where the detection result of the image feature detection unit matches at least for a predetermined time. Are detected as edge pixel points. The predetermined time and the predetermined time are, for example, time in seconds set by experiment or the like.

(C) Search area setting means This means is that a basic search area composed of a plurality of areas obtained by dividing the photographed image Pt in a matrix form and each area obtained by dividing the photographed image Pt in a matrix form is basically retrieved. A sub-search area in a state where the basic search area is shifted to an arbitrary position in the vertical and horizontal directions so as to straddle each area of the area is set.

  Specifically, as shown in FIG. 6A, a basic search region Km in which the portion of the region of interest ROI of the captured image Pt is divided into matrix-like areas α, and as shown in FIG. 6B. The region of interest ROI is subdivided into matrix-like areas β shifted from the basic inspection region Km by a length shorter than one area α in the vertical and horizontal directions, for example, a half of the area α. A search area Ks is set.

  The areas α and β have the same size including a plurality of the detection pixel points pi in the vertical and horizontal directions. Further, a plurality of sub-search areas Ks may be set by changing the amount of shift, but in this embodiment, only one is set in consideration of the processing load and the like.

(D) Determination means This means counts the number of coincidence points detected by the coincidence point detection means for each of the areas α and β in both search areas Km and Ks, and adds the count value to a set threshold value or more. The areas α and β of the kimono TG are detected and it is determined that there is a deposit.

  By providing the above means, the control ECU 6 repeatedly executes, for example, the detection processes in steps S1 to S3 in FIG.

  In the sticking point detection in step S1, each detection point pixel pi having an edge of “1” in the differential binary image is detected by the image feature detection unit, and continuously at least for a predetermined time by the coincidence point detection unit. Each detection pixel point pi that matches the detection result of the image feature detection means is detected as an edge pixel point.

  In the sticking area detection in step S2, the edge pixel points of the areas α and β set by the search area setting means are determined, and the areas α and β including the edge pixel points are detected.

  Further, in the fixed object detection in step S3, the determination unit detects the areas α and β of the deposit TG where the count value is equal to or larger than the set threshold value, and the areas α and β of the deposit TG are detected. If any one of them is present, it is determined that there is a deposit.

  By doing this, it is possible to detect the presence of adhering matter by counting edge pixel points that appear at the same position for a predetermined time or more in the units of areas α and β obtained by dividing the photographed image Pt. Compared to the case of performing complex image processing, the processing is simple, the image processing burden is light, and the captured image Pt of the monocular camera 2 that captures the front of the own vehicle with an extremely small image processing burden is included in the own vehicle 1 or monocular. It is possible to detect the presence of an attachment on the camera 2. Moreover, the position of the deposit TG on the image can be grasped from the areas α and β including the edge pixel points.

  Therefore, it is possible to prevent erroneous recognition of the target by removing the adhered substance TG from the captured image Pt.

  In addition, since the basic search area Km and the sub search area Ks are set and the edge image points are divided and counted in units of the respective areas α and β, and the adhering substance TG is detected, a plurality of basic search areas Ks are divided. Even if each edge pixel point of the deposit TG at a position where the count value of each area α decreases with a spread over the area α, the count value is included in one area β in the division of the sub search region Ks. Similarly, even in the division of the sub search area Ks, even if each edge pixel point of the deposit TG is located at a position where the count value of each area β is reduced by spreading over a plurality of areas β, the basic search area In the division of Km, the count value is not decreased by being included in one area α. Therefore, for the deposit TG, the count value of the edge pixel points of the areas α and β at the positions of the basic and sub-search areas Km and Ks must be equal to or greater than the threshold value. Can be detected, and reliability is significantly improved.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit thereof, for example, the basic search area Km, the sub-search, and the like. If the region Ks is set, the upper and lower, left and right peripheral portions do not reach the widths of the areas α and β, and are wasted. For example, as shown in FIG. The upper and lower, left and right outer peripheral search areas Ke are set as the sub search areas, and these outer peripheral areas Ke are divided into areas γ in the same manner as the basic search areas Km and the sub search areas Ks, and the adhered TGs are divided in units of areas. The presence or absence may be detected, and the detection accuracy of the deposit TG may be further improved.

  In the present invention, it is needless to say that the basic search area Km and the sub search areas Ks and Ke may have different sizes, shapes, and the like.

  Next, the size and number of areas α and β of the basic search region Km and the sub search region Ks, the continuous time for recognizing the deposit TG, and the like may be appropriately set based on experiments or the like. Of course.

  Further, the present invention can be similarly applied when a stereo camera or the like is mounted on the own vehicle 1 instead of the monocular camera 2.

It is a block diagram of one embodiment of the present invention. It is a perspective view of the example of mounting of the monocular camera of FIG. (A), (b) is explanatory drawing of the example of a sequential picked-up image of the monocular camera of FIG. It is explanatory drawing of the pixel point of the picked-up image of control ECU of FIG. (A), (b) is explanatory drawing of an edge pixel point, and the one part enlarged view. (A), (b) is explanatory drawing of a basic search area | region and a subsearch area | region. It is a flowchart for process description of control ECU of FIG. It is explanatory drawing of the other example of a setting of the search area | region of a picked-up image.

Explanation of symbols

1 Own vehicle 2 Monocular camera 6 Control ECU
TG deposit

Claims (2)

An attached matter detecting device for detecting the presence of attached matter in a captured image of a camera preceding a vehicle of an own vehicle, wherein the image feature detecting means detects each pixel point of an image feature amount from image data of the captured image. Storage means for holding the detection results of the image feature detection means, detection results of the image feature detection means for the latest image data of the captured images, and images of past captured images held in the storage means A matching point detection unit that compares the detection result of the image feature detection unit for data and detects each pixel point that the detection result of the image feature detection unit matches at least for a predetermined time;
The basic search area composed of a plurality of areas obtained by dividing the photographed image into a matrix, and the basic search area so that each area obtained by dividing the photographed image in a matrix straddles each area of the basic search area. A search area setting means for setting a sub search area in a state where the search area is shifted to an arbitrary position in the vertical and horizontal directions;
A determination unit that counts the number of coincidence points detected by the coincidence point detection unit for each area of the both search areas, and that determines an adhering matter area where the count value is equal to or greater than a threshold value. An adhering matter detection device characterized by that. An attached matter detection method for detecting an attached matter in a captured image of a preceding camera of a camera mounted on the own vehicle,
Detecting each pixel point of the image feature amount from the image data of the captured image and holding the detection result;
The feature value detection result for the latest image data of the captured image is compared with the feature value detection result for the stored image data of the past captured image, and the detection result is continuously at least for a predetermined time. Detect each matching pixel point,
The basic search area composed of a plurality of areas obtained by dividing the photographed image into a matrix, and the basic search area so that each area obtained by dividing the photographed image in a matrix straddles each area of the basic search area. Set the sub search area with the search area shifted to any position in the vertical and horizontal directions,
A deposit detection method, comprising: counting the number of coincident detection points in each of the two search areas, detecting an deposit area where the count value is equal to or greater than a threshold value, and determining the presence of deposits.

Images