DE102014219424A1 - Camera lens dirt detection device - Google Patents

Abstract

Objective: To provide a camera lens smear detection device that can increase the accuracy of detecting dirt adhesion. Means for solving: A camera lens dirt detection means for performing dirt detection processing for detecting adhesion of dirt (B) to a lens (4) from a camera (1) mounted in a vehicle based on a photograph taken by the camera (1) Image (D) includes a control section (10) for calculating a rate of change (luminance gradient) (R) between a luminance of a predetermined area (P) and a luminance of an area of the predetermined area (P) in the image (D). The lens (4) is disposed at a position closer to the camera (1) than a range of depth of field (F) from the camera (1). The control section 10 sets an area in which predetermined areas (P) exist whose rates of change (R) are within set values (ST), successively as a dirt candidate area (E), and the control / Control section (10) performs the dirt detection processing to determine that the dirt candidate area (E) results from the dirt (B). The dirt detection processing is started while the power supply (6) is turned on by the control section (10).

Description

Technical area

The present invention relates to a camera lens dirt detection device, and more particularly to a camera lens dirt detection device which detects dirt adhering to a lens of a digital camera or the like.

Technical background

A method has been well known which detects dirt adhering to a surface of a lens of a digital camera having an image sensor comprising a plurality of light receiving elements or the like on the basis of photographed image data.

Japanese Patent Laid-Open Publication No. JP 2007-189369 (hereinafter referred to as Patent Document 1) discloses a camera lens duster detection device which identifies an area in which luminance between adjacent pixels changes upon processing of photographed image data and, in a case where a small luminance change occurs in the identified area, even if there is Sensitivity of the camera is increased, determines that the part is not an image that shows the outside, but results from dirt, which adheres to a lens and tells the lens dirt to a user.

Overview of the invention

Problem to be solved by the invention

However, the camera is focused on the outside, which is farther away from at least the position of the lens, and thus an output image is obtained in which the contours of dirt are blurred and gradually changed in luminance, as if gradation would be applied to the contour. Consequently, luminance differences on the periphery of the dirt do not occur significantly. It is therefore difficult to improve the accuracy of dirt detection with the technology described in Patent Document 1.

It is an object of the present invention to provide a camera lens load detection device which solves the problem of the above conventional technology and which can improve the accuracy of detection of dirt adhesion.

Means of solving the problem

In order to achieve the above object, according to a first feature of the present invention, there is provided a camera lens drop detecting means for performing a dirt detection processing to prevent adhesion of dirt (B) to a lens (FIG. 4 ) or a protective screen ( 4 ) from a camera installed in a vehicle ( 1 ) based on a through the camera ( 1 ) photographed image (D), wherein the camera lens adder detecting means comprises a control section (Fig. 10 ) to calculate a rate of change rate (R) between a luminance of a predetermined range (P) and a luminance of an environment of the predetermined range (P) in the image (D), wherein the control / Control section ( 10 ) an area in which predetermined areas (P) exist whose rates of change (R) are within set values (ST), successively set as a dirt candidate area (E), and the control section (FIG. 10 ) performs the dirt detection processing to determine that the soil candidate region (E) results from the soil (B).

In addition, according to a second feature of the present invention, the control section starts ( 10 ) lens lens detection processing by applying a power supply ( 6 ) from the control section ( 10 ) is turned on.

In addition, according to a third feature of the present invention, the lens is ( 4 ) or the protective screen ( 4 ) are arranged at a position facing the camera ( 1 ) is closer than a range of depth of focus (F) from the camera ( 1 ).

Furthermore, according to a fourth feature of the present invention, the control section ( 10 ) preferably a dirt candidate region having both a large vertical length and a large horizontal length as the soil candidate region (E) from the soil (B), and determines that the soil candidate Area from the dirt (B) results.

Effects of the invention

According to the first feature, the control section for calculating the rate of change between the luminance of the predetermined area and the luminance of the circumference of the predetermined area in the image is provided. The control section sets a region in which predetermined regions whose change rates are within the set values successively as a dirt candidate region, and the control section performs the dirt detection processing to Determine that the dirt-candidate area results from the dirt. Thus, a dirt-adhering state can be detected by determining whether the image of a dirt portion adhering to the lens or the like is an actual image or shows the dirt adhering to the lens or the like, using a characteristic of a Luminance gradually changing in the contour portion from the dirt portion. This eliminates a need for a complicated procedure such that a portion which does not change in luminance even if, for example, the luminance of an entire image is changed is determined to be a dirt, and a dirt detection can be made directly on the basis of Image data to be performed.

According to the second feature, when the power supply from the control section is turned on, the control section starts the lens dirty detection processing. Thus, an occupant may be instructed by the vehicle about dirt on the lens and be prompted to remove the debris before starting to drive. Therefore, the driving can be started in a state in which the dirt is removed, and thus functions are sufficiently exerted by the camera. In addition, the inconvenience of stopping the traveling vehicle and cleaning the lens is eliminated, so that comfort can be improved.

According to the third feature, the lens or the protective glass is disposed at a position closer to the camera than the range of depth of field from the camera. Thus, the camera is not focused on the dirt on the lens or the like. Therefore, soil determination can be effectively performed using rates of change in luminance.

According to the fourth feature, the control section preferably selects a dirt candidate region having both a large vertical length and a large horizontal length as the soil candidate region from the soil and determines that the soil Candidate area out of the dirt results. It is thus possible to ignore the effect of dust, scratches, and the like, which do not pose a problem when the image is inspected, and therefore to improve the convenience of the camera lens art detection device.

Brief description of the drawings

1 Fig. 10 is a block diagram showing a general construction of a camera lens load detection device according to an embodiment of the present invention.

2 Fig. 12 shows an example of image data photographed in a state in which dirt adheres to a lens.

3 Fig. 10 is a flowchart of a procedure for lens fouling detection processing.

4 Fig. 10 is a flowchart of a procedure for the operation of the camera and a control section.

5 Fig. 10 is a conceptual diagram showing an arrangement setting of a pixel P in a V-picture.

6 Figure 11 is a conceptual diagram showing a position setting for calculating a luminance gradient.

7 Fig. 10 is a flowchart of a procedure for a dirty candidate area extraction processing performed by a dirty candidate area extraction means.

8th is a V-picture after binarization processing.

9 FIG. 10 is a conceptual diagram of a method of large-dirt candidate area extraction processing. FIG.

10 FIG. 10 is a conceptual diagram of the method of large-dirt candidate area extraction processing. FIG.

Mode for carrying out the invention

A preferred embodiment of the present invention will be described below in detail with reference to the drawings. 1 Fig. 10 is a block diagram showing a general construction of a camera lens load detection device according to an embodiment of the present invention. A camera 1 is a digital camera or a digital video camera that has an image sensor 3 comprising a plurality of light receiving elements. The camera 1 has a lens 4 which is in contact with the outside, away from a main body portion 2 which comprises a plurality of inner lenses. For the rest, the lens 4 a disc for protecting the front of another lens or the like.

The camera 1 According to the present embodiment, a vehicle-mounted camera is assumed to be For example, for the detection of traffic signs in a drive recorder or a navigation device that records a driving video is used. However, regarding the application of a dirt detection technology, the camera may 1 a portable camera, which is worn by a user for photographing or the like.

Because it is difficult to use the lens 4 during the drive from the vehicle, it is intended that the camera lens drop detecting device according to the present invention notify a driver that dirt is on the lens to cause the driver to clean the lens before the vehicle starts to drive, in particular the lapse of a predetermined time since the power is turned on to the vehicle.

The camera lens load detecting device is in a control section 10 for processing one from the camera 1 provided photographed image. The control section 10 includes an RGB HSV conversion means 11 , a luminance extractant 12 , a luminance gradient computing means 13 , a Dirt Candidates Area Extractor 14 , a large-dirt candidate area extractant 15 and a timer 16 , The control section 10 is also associated with a resource 5 to operate the camera 1 , an energy supply 6 , which is formed by a vehicle battery, a cell or the like, and a warning means 7 to inform the driver about the lens dirt. Incidentally, each of the control section 10 , the resources 5 , the energy supply 6 and the warning means 7 separately from the camera 1 or integral with the camera 1 be educated. The resource 5 includes a recording start switch, a recording stop switch, and the like.

1 shows a condition of dirt B which is on the surface of the lens 4 adheres. It is intended that the camera 1 According to the present embodiment, it is attached to the vehicle and photographs an object concentrating on a road in front of the vehicle. Thus, the lens 4 normally located at a position closer to the camera than a range of depth of field F, in which area the camera 1 is focused.

2 Fig. 13 shows an example of image data D photographed in a state in which the dirt B on the lens 4 adheres. In the example of this figure, the dirt B appears on the lens 4 clings, essentially in the middle of a road sign O as an object.

As described above, the dirt B, which is on the lens 4 attaches, outside the range of the depth of field F from the camera 1 , Thus, the camera 1 not focused on the dirt B and the contour of the dirt B, which of the image sensor 3 is photographed is extremely blurred or blurry compared to other objects.

The camera lens occupancy detector according to the present invention discriminates the image of the dirt B from the image of the road sign O, with the luminance (value) of an image from the image data D as a reference.

3 Fig. 10 is a flowchart of a procedure for a camera body detection processing. In a step S1, image data D is put into the control section 10 read. In a step S2, the RGB-HSV conversion means is converted 11 (please refer 1 ) converts the read image data from RGB data of red, green and blue to HSV data of hue, saturation and value (luminance). This conversion from RGB to HSV is performed by a well-known image processing technology.

The image converted to HSV data can be decomposed into three elements of hue (H), saturation (S) and luminance (V). In the present embodiment, the luminance extractant extracts 12 only a V-picture representing the luminance among the three elements, by values 0 to 255 (256 steps from 0 from black to 255 from white), to use the V-image for the lens smear detection processing.

In a step S3, the luminance gradient calculating means calculates 13 a luminance gradient R in the extracted V-picture. The luminance gradient R represents a rate of change of luminance between adjacent pixels (which rate of change may be referred to as a rate of change R) and is between all pixels of the image sensor 3 calculated. In a step S4, the soil candidate extracting agent extracts 14 Dirt candidate areas based on the luminance gradients.

In a step S5, the large-dirt-candidate-area extracting agent extracts 15 only a large Dirt Candidates Area from the Dirt Candidate Areas Extracted. This serves the effect of a small dirt, a scratch on the lens 4 or the like, which need not be communicated to the occupant to eliminate.

Then, in a step S6, the temporal continuity of the large dirt candidate area is made using the timer 16 certainly. This increases the accuracy of one Dirt determination, since the dirt determination is performed only when a predetermined time has elapsed, in a state in which a large dirt candidate area is in a same position. A concrete method from the preceding steps S4 to S6 will be described later.

4 is a flowchart of a procedure for the operation of the camera 1 and the control section 10 , As described above, because the camera 1 According to the present embodiment, in a drive recorder or a navigation device for a vehicle, the dirt determination is started when power to the vehicle is turned on. The following description is of a river in a case where the camera 1 is used in a drive recorder according to the present embodiment.

In a step S10, after the power is turned on, the dirt determination for the camera becomes 1 performed as an introductory diagnosis. If it is determined in step S10 that there is dirt on the lens 4 gives the warning means 7 , which is provided on the vehicle, a message (warning) to the occupants. That is, the occupant can be notified and made to take the lens 4 to clean before the vehicle begins to drive. The warning device 7 may be formed, for example, by a warning light, which is provided on a meter device of the vehicle, a buzzer, a liquid crystal display or the like.

In step S11, an operation signal is read from a recording switch SW to give an instruction to record read image data in a memory. In a step S12, it is determined whether the recording switch SW is turned on or not. If an affirmative determination is made in step S12, the processing proceeds to step S13.

In step S13, it is determined whether recording is being performed or not. If an affirmative determination is made, the processing returns to step S11. If a negative determination is made, the processing proceeds to a step S14 to perform a recording start processing. If a negative determination is made in step S12, the processing proceeds to step S15 to determine whether the recording is being performed or not. If an affirmative determination is made, the processing proceeds to step S16 where recording stop processing is performed, and the series of control steps is ended. On the other hand, if a negative determination is made in step S15, the processing returns to step S11.

A method of calculating a luminance gradient by the luminance gradient calculating means 13 is described below with reference to the 5 and 6 described. 5 Fig. 10 is a conceptual diagram showing an arrangement adjustment of a pixel P in a V-picture (V). 6 Figure 11 is a conceptual diagram showing a position setting for calculating a luminance gradient. The pixel P as a smallest unit of luminance information has position information at a pixel position [i, j] on the V-picture (V).

Luminance information from 0 to 255 is set for each pixel position [i, j].

The luminance gradient indicates an amount of change in luminance in a certain range. The luminance gradient is calculated for each pixel position. In particular, when a quadrilateral having a pixel position [i, j] as a center and having a vertical and a horizontal length of 2s, in FIG 6 is set, a luminance gradient x in an x-direction from this range A [i, j] is given by a calculation equation x = | (v [i, j -s] -v [i, j + s]) / 25 | receive. Similarly, a luminance gradient y in a y-direction from the region A [i, j] is given by a calculation equation y = | (v [i-s, j] -v [i + s, j]) / 2s | receive. A luminance gradient xy (rate of change R) from the area A [i, j] is obtained as an average of the luminance gradient in the x-direction and the luminance gradient y in the y-direction, that is, by a calculation equation xy = (x + y) / 2. Incidentally, the calculation of the luminance gradient is not performed only for all the pixel positions, but is also modifiable, for example, whereby calculation accuracy is increased by making position adjustments so that pixels are superposed on each other or in which pixel positions for which the calculation is performed / will be reduced to reduce an operation time.

The calculated luminance gradient xy shows a rate of change in luminance between adjacent pixel positions. When the luminance gradient xy is zero, a difference in luminance between the adjacent pixel positions is zero. By increasing the luminance gradient xy, the luminance gradient xy shows a larger difference in luminance.

As described above, the image moves from the dirt B, which is on the lens 4 adheres, not in focus and light is deflected on the back surface side of the dirt B. Thus, the contour of the dirt B is blurred, and a gradation is formed in the contour portion, so that the luminance is gradually reduced toward the inside of the dirt B. In this gradation area or gradation area, small luminance gradients occur in succession. On the other hand, in image data in focus (for example, the road sign O in 2 ), the contour of the object or different colors are shown relatively clearly, so that a luminance gradation or luminance gradation is not to be expected.

7 FIG. 10 is a flowchart of a procedure for dirty candidate area extraction processing performed by the dirty candidate area extracting means. FIG 14 is carried out. In the present embodiment, after the luminance gradient xy is calculated at all of the pixel positions, the V-image is divided into two kinds of parts, ie, parts in which the luminance gradient xy is within set values ST and parts in which the luminance gradient xy is outside the set values ST. The set values ST are, for example, 0 to 10. When the luminance gradient xy is within the set values ST (for example, 7), a dirt determination value (Mask_dirt [i, j]) is set to 255. If the luminance gradient xy is outside the set value ST (for example, 12), the soil determination value is set to 0.

In a step S20, a pixel position at which a dirt determination value is to be set is set to [i = 0, j = 0]. In a step S21, it is determined whether or not the luminance gradient xy (Hsv_deri [i, j]) exceeds the upper limit value (UPPER) from the set values. If an affirmative determination is made in step S21, the processing proceeds to step S22 to set the soil determination value (Mask_dirt [i, j]) to 0.

On the other hand, if a negative determination is made in step S21, the processing proceeds to step S23 to determine whether the luminance gradient xy (Hsv_deri [i, j]) exceeds the lower limit value (LOWER) from the set values. If an affirmative determination is made in step S23, the processing proceeds to step S24 to set the soil determination value (Mask_dirt [i, j]) to 255. On the other hand, if a negative determination is made in step S23, the dirt determination value (Mask_dirt [i, j]) is set to 0.

Steps S25 to S29 are a procedure to set a dirt determination value for all of the pixel positions in turn. In a step S25, it is determined whether j = columns is reached or not, i. That is, whether the value of j has reached the last columns of the pixels or not. If an affirmative determination is made in step S25, the processing proceeds to step S26 to set j = 0, and advances to set dirt determination values in a next row. On the other hand, if a negative determination is made in step S25, the processing proceeds to step S27 to increment the value of j, and then returns to the determination in step S21.

Then, in a step S28, it is determined whether or not i = rows are reached, i. That is, whether the value of i has reached the last row of the pixels or not. If a negative determination is made in step S28, the processing proceeds to step S29 to increment the value of i and then returns to the determination in step S21.

The dirty candidate area extraction processing described above binarizes the dirt determination values at all of the pixel positions in the V-picture to either 0 or 255.

8th is a V-picture (V1) after the above binarization processing, which picture is painted in two different colors of white and black by setting the fouling determination value at 0 or 255 at each pixel position. According to the binarization processing, a region in which a gradation is seen in the RGB image is shown in white due to small luminance gradients xy (small changes in luminance), and a region with large luminance gradients xy is shown in black. That is, in the example of 8th For example, an area that appears white is recognized as a "dirt candidate area" which is most likely on the lens 4 adhering dirt shows.

However, if it is determined that all white areas show dirt, a warning is issued against the adhesion of even little dust or the like, which is not a problem when the occupant checks the image data, for example, so that the comfort is lowered. In addition, there may be a part with small luminance gradients xy, which do not result from dirt but result from the actual colors of the object, as in 8th shown. Thus, in the present invention, the large-dirt candidate area extractant extracts 15 as "dirt candidate areas" only parts which in 8th appear as large white areas, ie areas in which pixel locations whose soil determination value is set to 255 appear consecutively.

The 9 and 10 FIG. 10 are conceptual diagrams of a method of a large-dirt-candidate-area extraction processing. FIG. The diagrams show a state in which dirt candidate areas E, E1 and E2 are extracted by dirt candidate area extraction processing become. Of the three dirt candidate regions, the region E having dimensions of h in a vertical direction and W in a horizontal direction is the largest one. In the present embodiment, an area whose vertical and horizontal dimensions are both large is extracted. Thus, the smaller ones of the vertical and horizontal dimensions of each of the dirt candidate areas E, E1 and E2 are defined as a "dirt length". That is, the dirt lengths of the dirt candidate regions E, E1 and E2 are L, L1 and L2, respectively.

The dirt candidate areas E, E1 and E2 are classified according to the dirt lengths. In the example of the 9 and 10 the dirt lengths are such that L>L2> L1. Thus, the dirt candidate area E is ranked at the first rank, the dirt candidate area E2 is ranked at the second rank, and the dirt candidate area E1 is ranked at the third rank. Incidentally, only three dirt candidate areas are shown in the present embodiment. However, for example, in one case of actual data, the dirt lengths of 50 extracted dirt candidate areas are calculated and only the top 10 of the dirt candidate areas are extracted and stored as "large dirt candidate areas". In this case, dirt candidate areas in eleventh place and below are ignored as dirt candidate areas which pose no problem when the image is checked.

Finally, the control section determines 10 in that the stored large dirt-candidate area shows dirt on the lens 4 adheres when a predetermined time (for example, five seconds) has elapsed, with the stored large dirt candidate area remaining in a same position. This is to prevent an erroneous determination in the dirt detection even if a gradation is caused by, for example, the actual colors of the object, as described above. When the dirt detection processing is started by, for example, turning on an ignition switch of the vehicle, particularly in the case of a motorcycle being driven while sitting astride the vehicle body from the vehicle, the position is taken from the camera 1 often by handling the handlebar or the like even in a short time before the motorcycle starts to drive changed. If actually dirt on the lens 4 is, the position of the dirt candidate is not changed even if the camera 1 is moved. However, if a gradation is caused by the colors of the article or the like, the position of the dirt candidate is changed while the camera 1 is moved. Thus, the colors of the article or the like can be distinguished from the actual dirt.

It should be noted that the structure of the camera and the configuration of the control section, the shape and the number of pixels of the image sensor, the shape of the lens or the protective glass, the processing of image conversion of RGB data into HSV Data, the setting of the predetermined area P when the luminance gradient is calculated, the luminance gradient calculating area, the dirt candidate area setting method and the dirt length setting, the set values used for the binarization processing, the different colors which are used to paint the V-picture, the set time used for the dirt determination according to the lapse of time, and the like which are not limited to the foregoing embodiment but are variously changeable. The camera lens amount detecting device according to the present invention is not limited to cameras mounted on vehicles, but is applicable to work surveillance cameras fixed to machine tools, various types of portable type cameras, and the like.

Objective: To provide a camera lens smear detection device that can increase the accuracy of detecting dirt adhesion.

Means for Solving: A camera lens dirt detection device for performing a dirt detection processing to adhere dirt B to a lens 4 from a camera 1 to detect which is mounted in a vehicle based on a through the camera 1 photographed image D, comprises a control section 10 for calculating a rate of change (luminance gradient) R between a luminance of a predetermined area P and a luminance of an area of the predetermined area P in the image D. The lens 4 is located at a position facing the camera 1 closer than a range of depth of field F from the camera 1 , The control section 10 sets an area in which predetermined areas P exist whose rates of change R are within set values ST, successively as a dirt candidate area E, and the control section 10 performs the dirt detection processing to determine that the soil candidate region E results from the soil B. The dirt detection processing is started while the power supply 6 from the control section 10 is turned on.

Description of reference numerals

• 1 ... camera, 5 ... resources, 6 ... energy intake, 7 ... warning means, 10 ... control section, 11 ... RGB HSV conversion means, 12 ... luminance extractant, 13 ... luminance gradient calculation means, 14 ... Dirt Candidate Area Extractants, 15 ... large-dirt-candidate-area extractant, 16 ... Timer, B ... Dirt, D ... Image Data, E, E1, E2 ... Dirt Candidate Area, E ... Large Dirt Candidate Area, O ... Object, P. .. pixel (predetermined range), R ... rate of change in luminance (luminance gradient), V ... V image

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• JP 2007-189369 [0003]

Claims (4)

Camera lens dander detection device configured to perform dirt detection processing to prevent adhesion of dirt (B) to a lens (FIG. 4 ) or a protective screen ( 4 ) from a camera installed in a vehicle ( 1 ) based on a through the camera ( 1 ) photographed image (D), wherein the camera lens amount detecting means comprises a control section (Fig. 10 ) configured to calculate a rate of change (R) between a luminance of a predetermined range (P) and a luminance of an environment of the predetermined range (P) in the image (D), the control portion ( 10 ) an area in which predetermined areas (P) exist whose rates of change (R) are within set values (ST), successively set as a dirt candidate area (E), and the control section (FIG. 10 ) performs the dirt detection processing to determine that the soil candidate region (E) results from the soil (B). A camera lens load detecting device according to claim 1, wherein the control section ( 10 ) the lens smear detection processing starts by applying a power supply ( 6 ) from the control section ( 10 ) is turned on. Camera lens duster detection device according to claim 1 or 2, wherein the lens ( 4 ) or the protective screen ( 4 ) is disposed at a position facing the camera ( 1 ) is closer than a range of depth of focus (F) from the camera ( 1 ). A camera lens load detecting device according to any one of claims 1 to 3, wherein the control section ( 10 ) as the soil candidate area (E) of the soil (B) preferably selects a soil candidate area having both a large vertical length and a large horizontal length, and determines that the soil candidate area from the dirt (B) results.

Images