JP2010273014A - Camera device for mobile object and method of detecting attachment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera device for a mobile object, the camera device highly accurately detecting a part where droplets or dirt or the like is attached and alarming a driver. <P>SOLUTION: The camera device for a mobile object includes: a camera, installed to the mobile object, for imaging the periphery of the mobile object successively at different timing; an image conversion processing part for shifting the first image of the first time between the first image and a second image which are photographed by the camera and are sequential in time corresponding to the moving amount of the mobile object, and converting it to the same phase as the second image; a difference processing part for extracting a difference between the second image and the converted image; a detection part for integrating a difference image extracted in the difference processing part and detecting the image of attachment attached to the lens of the camera; and an alarm part for reporting an alarm when the area of the image of the attachment becomes equal to or larger than a preset area. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a moving body camera device that captures an obstacle around a moving body with a camera installed on the moving body such as a vehicle and assists a driver, and a method for detecting an adhering matter attached to a camera lens.

  2. Description of the Related Art Conventionally, there is known a moving body camera device in which a camera is mounted on a vehicle or the like, and a surrounding obstacle such as a front or background of the vehicle is photographed and displayed on a display device. Since the camera for moving bodies captures surrounding images while the vehicle is traveling, water drops adhere to the camera lens when it rains. Also, dirt and insects may stick to the shooting. For this reason, when a lot of deposits such as water droplets and dirt adhere to the camera, the driver is encouraged to clean.

  In Patent Document 1, when a vehicle is moving, the difference between density values included in images taken at different timings is extracted, and the density values of the extracted images are integrated to remove dirt attached to the camera. An example of detection is disclosed. Further, Patent Document 2 calculates an optical flow of a predetermined pixel from a plurality of images taken at different timings, estimates an optical flow from vehicle speed information, and compares the measured optical flow with the estimated optical flow. Thus, an example in which dirt attached to the camera is detected is disclosed.

  However, in the example of Patent Document 1, when the road surface is black and has a uniform texture, it is difficult to distinguish the road surface from water droplets and dirt. In the example of Patent Document 2, road surface paint is used as an object for calculating an optical flow. However, there is a possibility that water droplets cannot be detected when there is no paint on the road surface. In addition, when there is a three-dimensional object (such as a guard rail column) near the paint, there is a possibility that the three-dimensional object is erroneously detected as dirt.

JP 2003-259358 A JP 2008-60874 A

  In the conventional mobile camera device, when the road surface is black and has a uniform texture, it is difficult to distinguish the road surface from water droplets and dirt. Further, when there is no paint on the road surface, water droplets cannot be detected, or when there is a three-dimensional object near the paint, there is a possibility that the three-dimensional object will be erroneously detected as dirt.

  In view of such circumstances, an object of the present invention is to provide a moving body camera device and a method for detecting an adhering substance that can detect an adhering portion such as water droplets and dirt with high accuracy and warn a driver. To do.

  According to a first aspect of the present invention, there is provided a camera device for a moving body that is installed on a moving body and sequentially captures the periphery of the moving body at different timings. An image conversion processing unit that shifts the first image at the previous time in accordance with the amount of movement of the moving body and converts the first image into the same phase image as the second image, the second image, and the image A difference processing unit that extracts a difference from the converted image, a detection unit that integrates the difference images extracted by the difference processing unit to detect an image of the attached matter attached to the lens of the camera, and And a warning unit that issues a warning when the area of the image exceeds a preset area.

  The present invention according to claim 5 is a method for detecting an adhering matter adhering to a lens of a camera, wherein the surroundings of the moving body are sequentially imaged at different timings by a camera installed on the moving body, and the images are captured by the camera Of the first and second images that move back and forth in time, the first image at the previous time is shifted according to the amount of movement of the moving body, converted into an image in phase with the second image, and the first image The difference between the image 2 and the converted image is extracted, the extracted difference images are integrated to generate a grayscale image, and the image obtained by binarizing the density of the grayscale image with a preset threshold value is used. It is detected as an image, and a warning is notified when the area of the image of the deposit is equal to or larger than a preset area.

  According to the moving body camera device of the present invention, even if the road surface has a uniform texture, candidates for water droplets and dirt can be extracted, and the candidate images such as water droplets and dirt can be integrated. It is possible to separate an image such as a three-dimensional object and detect a deposit such as a water droplet or dirt with high accuracy.

The block diagram which shows the camera apparatus for moving bodies which concerns on one Embodiment of this invention. Explanatory drawing which shows the extraction operation | movement of the difference image in the embodiment. Explanatory drawing which shows the extraction process of the difference image in the embodiment in time series. Explanatory drawing which shows the integration process of the difference image in the embodiment. Explanatory drawing which shows the detection processings, such as a water droplet and dirt in the same embodiment. The flowchart explaining the deposit | attachment detection method which concerns on one Embodiment of this invention. The subsequent flowchart which shows the deposit | attachment detection method in the embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing the overall configuration of a mobile camera apparatus according to an embodiment of the present invention.

  In FIG. 1, the moving body camera device 100 includes a camera 10, an image processing unit 20, a vehicle movement amount measurement unit 30, and a warning unit 40.

  The camera 10 is a photographing apparatus using a CCD, a CMOS sensor, or the like, and is attached to the front or rear of a moving body (for example, a vehicle), and images the periphery of the moving body sequentially at different timings. Below, the case where a mobile body is a vehicle is demonstrated. The image processing unit 20 includes an image input unit 21 that inputs an image captured by the camera 10, an image memory 22 that stores image data, and an image conversion processing unit 23 that converts the image stored in the image memory 22.

  In addition, a difference processing unit 24 that extracts a difference between the image output from the image input unit 21 and the image converted by the image conversion processing unit 23, and a candidate for an adhering substance such as a water droplet or a dirt is extracted from the extracted difference image. An attached matter candidate extraction unit 25 for detecting the presence of an attached matter such as water droplets or dirt on the camera 10.

  The vehicle movement amount measurement unit 30 includes a speedometer 31 and a gyro 32, and generates vehicle movement information and supplies it to the image conversion processing unit 23. The vehicle movement information includes information indicating the movement amount of the vehicle. That is, the movement amount indicated by the movement distance and angle of the vehicle is calculated by measuring the speed and rotation angle of the vehicle. The warning unit 40 includes a display device, a speaker, and the like. When a predetermined amount or more of water or dirt adheres to the lens of the camera 10, a warning is issued to the driver to encourage cleaning.

  Next, the operation of the moving body camera device 100 of the present invention will be described with reference to FIGS. FIG. 2 is an explanatory diagram showing operations of the image input unit 21, the image memory 22, the image conversion processing unit 23, and the difference processing unit 24.

  The camera 10 continuously captures the periphery of the vehicle at a predetermined time interval (for example, 30 images per second). The captured images are sequentially input to the image input unit 21. A1 in FIG. 2 indicates an image captured at time t1, and A2 indicates an image captured at time t2 that is slightly delayed. The images input to the image input unit 21 are sequentially stored in the image memory 22.

  In FIG. 2, the image A2 is shown as an image at the current time, and the image A1 is shown as an image at the previous time. The image output from the image input unit 21 is supplied to the difference processing unit 24, and the image read from the image memory 22 is supplied to the image conversion processing unit 23. When the image supplied from the image input unit 21 to the difference processing unit 24 is the image A2 at the current time, the image A1 at the previous time is read from the image memory 22 and input to the image conversion processing unit 23. The image conversion processing unit 23 processes the image A1 at the previous time, and obtains an image B1 converted into an image having the same phase as the image A2 at the current time.

  The conversion process in the image conversion processing unit 23 is performed based on the vehicle information from the vehicle movement amount measurement unit 30. That is, assuming that the distance when the vehicle moves between the previous time t1 and the current time t2 is L1, the previous time image A1 is shifted according to the moving distance L1 of the vehicle and is in phase with the current time image A2. Convert to an image. In addition, when the vehicle handle is turned between the previous time t1 and the current time t2, the image A1 at the previous time is shifted by incorporating information on the rotation angle.

  The image conversion processing unit 23 will be described in detail. In FIG. 2, it is assumed that the camera 10 is attached to the front portion of the vehicle and the front peripheral portion of the vehicle is sequentially photographed at a predetermined time interval. If the time difference between the previous time t1 and the current time t2 is obtained and the vehicle speed is multiplied by the time difference, the travel distance L1 from the previous time t1 to the current time t2 can be obtained.

  When the image conversion processing unit 23 regards all the pixels on the image A1 taken at the previous time t1 as points on the road surface, how the image A1 changes after traveling the distance L1. It is estimated and converted into an image having the same phase as the image A2 at the current time. For example, when the vehicle travels a distance L1 from the previous time, the previous time image A1 becomes an image shifted backward (downward) of the vehicle as indicated by B1 in FIG. In the image A1 at the previous time, a line 51 indicating a center line or roadside belt drawn on the road, a marking mark 52 (paint), and a three-dimensional object 53 (a guardrail post, etc.) near the road are forward (upper). In this example, water drops 61 and dirt 62 are attached to the surface of the camera 10.

  When the vehicle travels from the previous time t1 by the distance L1, the previous time image A1 becomes an image whose viewpoint is shifted backward (downward), and as shown by B1, the positions of the line 51, the marking mark 52, and the three-dimensional object 53 The positions of the water droplet 61 and the dirt 62 are also shifted downward. When the vehicle travels by the distance L1, the previous time image A1 is somewhat enlarged because the vehicle approaches the marking mark 52 or the three-dimensional object 53. In this way, the image A1 at the previous time is converted according to the vehicle information, and converted into the image B1 having the same phase as the image A2 at the current time.

  The difference processing unit 24 obtains the difference image C by taking the difference between the image A2 at the current time and the converted image B1. In the difference image C, the road and the line 51 and the marking mark 52 are canceled and deleted, and the image of the three-dimensional object 53, the water drop 61, and the dirt 62 remains as it is. If the road is curved, a part of the road, the line 51, and the marking mark 52 may remain without being canceled, but most of them are deleted.

  The adhering matter candidate extraction unit 25 binarizes the difference image C with a preset threshold value, and obtains an image of candidate regions for adhering matters such as water droplets and dirt. The adhering matter detection unit 26 integrates candidate area images such as water droplets and dirt in the time direction to generate a grayscale image. Further, the adhering matter detection unit 26 detects an area where the pixel value of the accumulated grayscale image is equal to or larger than a preset value as an adhering matter adhering to the lens. That is, an image obtained by binarizing the density of the grayscale image with a preset threshold value is detected as an image of the attached matter.

  And when the area | region of deposits, such as a water drop and dirt, exceeds an area more than fixed, it alert | reports by the warning part 40, and warns a driver | operator. This can prompt the driver to clean the lens surface. As a warning method to the driver, for example, a method of flashing and displaying the outer periphery of a water droplet or dirt area displayed on the camera image, a method of displaying a message prompting the lens to be washed with characters on the display, or a method of reporting from a speaker and so on.

  FIG. 3 is a diagram showing an image captured by the camera 10, an image subjected to image conversion processing, and a difference image in time series. The captured image changes sequentially as indicated by A1 to A4 according to the traveling of the vehicle. To do. The captured image A1 is a past image in time, and the captured image A4 is the latest image.

  Images obtained by converting the captured images A1, A2, and A3 by the image conversion processing unit 23 are B1, B2, and B3, respectively, and a difference image between the captured image A2 and the converted image B1 is C1. Similarly, the difference image between the photographed image A3 and the converted image B2 is C2, and the difference image between the photographed image A4 and the converted image B3 is C3. As can be seen from the difference images C1, C2, and C3, the image of the water droplet 61 and the dirt 62 remains, and the temporarily captured image such as the three-dimensional object 53 disappears on the way.

  The difference images C1, C2, C3... Are binarized by the adhering matter candidate extraction unit 25 and accumulated in the time axis direction by the adhering matter detection unit 26 as shown in FIG. When the difference images C1, C2, C3... Are integrated over time, the images of the water droplet 61 and the stain 62 become darker and the other images become lighter.

  In this way, as shown in FIG. 4B, an area where the pixel value of the grayscale image is equal to or larger than the preset value is detected as an image area of an adhering substance such as a water drop 61 and dirt 62. The driver can be informed if the warning unit 40 notifies when the area of the deposit area exceeds a predetermined area.

  Note that the grayscale image of the deposit detected by the deposit detection unit 26 is subjected to a labeling process as shown in FIG. The labeling process is a process in which the pixels 61 (62) of water droplets or dirt are adjacent to each other, and the warning unit 40 is displayed when the total pixel (area) of the labeled region 63 exceeds a predetermined value. Notify through.

  6A and 6B are flowcharts showing the operation of the deposit detection method of the present invention. In FIG. 6A, when the detection operation starts, an image taken by the camera 10 is input to the image input unit 21 in step S1. Further, it is sequentially stored in the image memory 22. In step S2, the image immediately before the current time image (previous time image) is read from the image memory 22.

  In step S3, vehicle information from the vehicle movement amount measurement unit 30 is obtained, and the movement amount of the vehicle is input to the image processing unit 20 (image conversion processing unit 23). In step S4, it is determined whether or not the vehicle has moved from the previous time. If it has moved, the process proceeds to step S5, and if not, the process returns to step S1.

  In step S5, the image at the previous time is shifted according to the amount of movement of the vehicle based on the amount of movement of the vehicle, and converted to an image in phase with the image at the current time. In step S6, the difference between the converted image and the current time image is taken to generate a difference image. In step S7, the difference candidate image is binarized by the adhering matter candidate extraction unit 25, and in step S8, it is determined whether or not an adhering matter candidate such as a water droplet or dirt has been detected. The operations in steps S1 to S8 are repeated to detect candidates such as water droplets or dirt adhering to the lens.

  After step S8, the process proceeds to step S9 in FIG. 6B. In step S9, the binarized image (image of the candidate area of the deposit) by the deposit detection unit 26 is integrated in the time direction to generate a grayscale image. In step S10, the grayscale image is binarized, and in step S11, the density of the grayscale image is determined. When an image having a predetermined density or more is detected as a region of adhered matter such as water droplets and dirt in step S11, a labeling process of the adhered region is performed in step S12.

  After labeling, in step S13, the area of the labeled region is calculated, and in step S14, it is determined whether the area is equal to or greater than a threshold value. When the area of a region such as water droplets or dirt is equal to or larger than the threshold value, in step S15, the warning unit 40 notifies the warning and warns the driver. When the area of the deposit region is not more than the threshold value, the process returns to step S13. Thus, the driver can be urged to clean the lens surface when a large amount of deposits adheres to the lens surface of the camera 10.

  As described above, according to the embodiment of the present invention, the image at the previous time is shifted according to the amount of movement of the vehicle, converted into an image having the same phase as the image at the current time, and converted into the image at the current time. By taking the difference between the images, even if the road surface is uniform, dirt, water droplets, and the like can be extracted as candidates for the deposit.

  Also, by binarizing the difference image and integrating it in the time axis direction, it is possible to detect water droplets, dirt, etc. separately from the three-dimensional object or moving object. Therefore, it is possible to detect the area of the deposit with high accuracy, and to issue a warning to the driver when there is a possibility that the obstacle detection system using the camera may not operate correctly.

  The embodiment of the present invention has been described by way of example, and is not limited to the configuration described above, and various modifications can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 100 ... Mobile body camera apparatus 10 ... Camera 21 ... Image input part 22 ... Image memory 23 ... Image conversion process part 24 ... Difference process part 25 ... Adhesion candidate extraction part 26 ... Adherence detection part 30 ... Vehicle movement amount measurement part 31 ... Speedometer 32 ... Gyro 40 ... Warning section 51 ... Line image,
52 ... Marking mark image 53 ... Solid object image 61 ... Water drop image 62 ... Dirt image

Claims (5)

A camera that is installed on a moving body and sequentially captures the periphery of the moving body at different timings;
Of the first and second images taken before and after the time taken by the camera, the first image at the previous time is shifted according to the amount of movement of the moving body, and the image has the same phase as the second image. An image conversion processing unit to convert;
A difference processing unit for extracting a difference between the second image and the converted image;
A detection unit that integrates the difference images extracted by the difference processing unit and detects an image of an adhering matter attached to the lens of the camera;
A moving body camera device, comprising: a warning unit that issues a warning notification when the area of the image of the adhering material exceeds a preset area. The image conversion processing unit acquires movement information of the moving body from a time when the first image is captured to a time when the second image is captured,
The mobile camera device according to claim 1, wherein the first image is shifted according to a moving distance and an angle of the moving body and converted into an image having the same phase as the second image.   The detection unit binarizes the difference image extracted by the difference processing unit and extracts a candidate image of a deposit, and generates a grayscale image by integrating the candidate images of the deposit, The moving body camera device according to claim 1, further comprising an attached matter detection unit that detects an image obtained by binarizing the density of the grayscale image with a preset threshold as an image of the attached matter.   The detection unit performs labeling processing on the image of the deposit from the deposit detection unit, and outputs a warning signal to the display unit when the area of the image subjected to the labeling processing exceeds a predetermined area. The moving body camera device according to claim 3. A method for detecting deposits attached to a lens of a camera,
The surroundings of the moving body are sequentially imaged at different timings with a camera installed on the moving body,
Of the first and second images taken before and after the time taken by the camera, the first image at the previous time is shifted according to the amount of movement of the moving body, and the image has the same phase as the second image. Converted,
Extracting a difference between the second image and the converted image;
The extracted difference images are integrated to generate a gray image,
An image obtained by binarizing the density of the grayscale image with a preset threshold value is detected as an image of the attached matter,
An adhering matter detection method, wherein a warning is notified when an area of the image of the adhering matter exceeds a preset area.