JP2015070566A - Device for detecting lens dirt of camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for detecting lens dirt of a camera, capable of enhancing detection accuracy for adhesion of dirt.SOLUTION: A device for detecting lens dirt of a camera that performs a dirt detection process of detecting that dirt B is adhered on a lens 4 of a camera 1 mounted on a vehicle on the basis of an image D captured by the camera 1, comprises a controller 10 calculating a change rate (a luminance gradient) R of a luminance in a predetermined region P of the image D and an ambient luminance. The lens 4 is placed at a position closer to the camera 1 than a range of a depth of field F of the camera 1. The controller 10 sets a region where a predetermined region P where the change rate R is within a setting value ST is continuous as a dirt candidate region E, and then, executes the dirt detection process of determining that the dirt candidate region E is caused by the dirt B. The dirt detection process is started with the turn-on of a power supply 6 of the controller 10.

Description

  The present invention relates to a lens dirt detection device for a camera, and more particularly to a lens dirt detection device for a camera that detects that dirt is attached to a lens of a digital camera or the like.

  2. Description of the Related Art Conventionally, in a digital camera or the like having an image sensor having a plurality of light receiving elements, there is known a method for detecting that dirt is attached to the surface of a camera lens based on photographed image data.

  In Patent Document 1, in the processing of captured image data, if an area where the luminance changes between adjacent pixels is specified, and the sensitivity change of the specified area is small even when the sensitivity of the camera is increased, It is determined that this portion is not an image showing the outside world but is caused by dirt adhering to the lens, and a lens dirt detection device for a camera is disclosed which notifies the user of lens dirt.

JP 2007-189369 A

  However, since the focal point (focus) of the camera is at least suitable for the outside world farther than the position of the lens, the outline portion of the dirt becomes an output image in which the luminance gradually changes as if the gradation is blurred. Therefore, a difference in brightness does not appear clearly around the dirt, and it has been difficult for the technique described in Patent Document 1 to improve the accuracy of dirt detection.

  An object of the present invention is to solve the above-mentioned problems of the prior art and provide a lens dirt detection device for a camera that can improve the accuracy of detecting the adhesion of dirt.

  In order to achieve the above object, the present invention provides a lens (4) or a protective screen (4) of the camera (1) based on an image (D) photographed by a camera (1) mounted on a vehicle. In a lens dirt detection device for a camera that performs dirt detection processing for detecting that dirt (B) is attached, a change rate (R) between the brightness of a predetermined area (P) of the image (D) and the surrounding brightness is calculated. A control unit (10) that performs a predetermined region (P) in which the change rate (R) is within a set value (ST) as a stain candidate region (E). In addition, the first feature is that a stain detection process is performed to determine that the stain candidate region (E) is caused by the stain (B).

  The control unit (10) has a second feature in that the lens contamination detection process is started when the power source (6) of the control unit (10) is turned on.

  A third feature is that the lens (4) or the protective screen (4) is disposed at a position closer to the camera (1) than the range of the depth of field (F) of the camera (1). There is.

  Further, the control unit (10) preferentially selects a region (E) of the dirt (B) having both large vertical and horizontal lengths, and determines that the region is caused by the dirt (B). There is a fourth feature.

  According to the first feature, the image processing apparatus includes a control unit that calculates a change ratio between the luminance of the predetermined area of the image and the surrounding luminance, and the control section includes an area in which the predetermined area in which the change ratio is within the set value is continuous. Since a stain detection process is performed to determine that the stain candidate area is caused by the stain after setting it as a stain candidate region, the brightness of the image of the stain portion attached to the lens or the like gradually changes in the contour portion. It is possible to determine whether the image is an actual image or dirt adhered to a lens or the like by using the characteristic of performing the detection, and detect the dirt adhesion state. Thereby, for example, a troublesome procedure for determining that a portion where the luminance does not change even when the luminance of the entire image is changed is determined to be dirty can be performed, and the contamination detection can be directly executed based on the image data.

  According to the second feature, the control unit starts the lens dirt detection process as the control unit is turned on, so that the vehicle occupant is informed of the lens before starting traveling. It is possible to notify the dirt and prompt the removal of the dirt. As a result, it is possible to start traveling in a state where the dirt is removed and the camera function is sufficiently exerted, and there is no need to stop the traveling vehicle and clean the lens, thereby improving convenience. be able to.

  According to the third feature, since the lens or the protective screen is disposed at a position closer to the camera than the range of the depth of field of the camera, the camera is not focused against dirt on the lens or the like. Therefore, it is possible to effectively perform the dirt determination using the luminance change rate.

  According to the fourth feature, the control unit preferentially selects a dirt area having both large vertical and horizontal lengths and determines that the area is caused by dirt. It is possible to ignore the influence of dust and scratches that have no problem and to improve the convenience of the camera lens dirt detection device.

1 is a block diagram illustrating an overall configuration of a lens dirt detection device for a camera according to an embodiment of the present invention. It is an example of the image data image | photographed in the state in which the dirt adhered to the lens. It is a flowchart which shows the procedure of a lens dirt detection process. It is a flowchart which shows the procedure of operation | movement of a camera and a control part. It is a conceptual diagram which shows the arrangement | positioning setting of the pixel P in V image. It is a conceptual diagram which shows the position setting for calculating a luminance gradient. It is a flowchart which shows the procedure of the dirt candidate area | region extraction process performed by the dirt candidate area | region extraction means. It is a V image after binarization processing. It is a conceptual diagram which shows the method of a big dirt candidate area | region extraction process. It is a conceptual diagram which shows the method of a big dirt candidate area | region extraction process.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a camera lens dirt detection device according to an embodiment of the present invention. The camera 1 is a digital camera or a digital video camera including an image sensor 3 having a plurality of light receiving elements, and includes a lens 4 that is in contact with the outside, outside the main body 2 including a plurality of internal lenses. The lens 4 may be a screen that protects the front of other lenses.

  The camera 1 according to the present embodiment is an in-vehicle camera that is assumed to be applied to a road recorder recognition or the like of a drive recorder or a navigation device that records a running video. It may be a portable camera for photographing.

  Since the lens dirt detection device for a camera according to the present invention is difficult to clean the lens 4 during traveling of the vehicle, before the vehicle starts traveling, specifically, a predetermined time after the vehicle is turned on. The purpose of this is to notify the driver that lens contamination has occurred before the time elapses and to encourage cleaning.

  The lens dirt detection device for a camera is included in the control unit 10 that processes an image captured by the camera 1. The control unit 10 includes an RGB-HSV conversion unit 11, a luminance extraction unit 12, a luminance gradient calculation unit 13, a stain candidate region extraction unit 14, a large stain candidate region extraction unit 15, and a timer 16. . The control unit 10 is connected to an operation unit 5 for operating the camera 1, a power source 6 including an in-vehicle battery or a battery, and a warning unit 7 for notifying the driver of the occurrence of lens contamination. The control unit 10, the operation unit 5, the power source 6, and the warning unit 7 can be configured separately or integrally with the camera 1, respectively. The operation means 5 includes a recording start switch, a stop switch, and the like.

  FIG. 1 shows a state in which dirt B adheres to the surface of the lens 4. Since the camera 1 according to the present embodiment is intended to shoot a subject in front of the vehicle centered on a road that is fixed to the vehicle, the lens 4 is usually an object field that is within the focus range of the camera 1. It is disposed at a position closer to the camera than the range of depth F.

  FIG. 2 is image data P showing an example of image data D photographed with dirt B attached to the lens 4. In the example of this figure, the dirt B adhering to the lens 4 is reflected in the approximate center of the road sign O as the subject.

  As described above, since the dirt B attached to the lens 4 is outside the range of the depth of field F of the camera 1, the focus (focus) of the camera 1 does not match the dirt B, and the image sensor 3 takes a picture. The dirt B to be made is that the outline thereof is greatly blurred compared to other subjects.

  In the camera lens dirt detection device according to the present invention, the imaging of the dirt B is distinguished from the imaging of the road sign O on the basis of the luminance (brightness) of the image of the image data P.

  FIG. 3 is a flowchart showing the procedure of the camera dirt detection process. In step S <b> 1, image data D is read into the control unit 10. In step S2, RGB-HSV conversion means 11 (see FIG. 1) converts the read image data from RGB data composed of red, green, and blue into HSV data composed of hue, saturation, brightness (luminance). . This RGB → HSV conversion is performed by a known image processing technique.

  An image converted into HSV data can be decomposed into three elements of hue (H), saturation (S), and luminance (V). In the present embodiment, among the three elements, only the V image whose luminance is represented by a value of 0 to 255 (256 levels from black zero to white 255) is extracted by the luminance extracting unit 12, and lens contamination detection processing is performed. Used for.

  In step S3, the brightness gradient calculating means 13 calculates the brightness gradient R of the extracted V image. The luminance gradient R indicates a luminance change rate (hereinafter, also referred to as a change rate R) between adjacent pixels (pixels), and is calculated between all the pixels of the image sensor 3. In the subsequent step S4, a candidate stain region is extracted by the candidate stain region extraction means 14 based on the luminance gradient.

  In step S5, the large dirt candidate area extracting unit 15 extracts only a large dirt candidate area from the extracted dirt candidate areas. This is to remove the influence of small dirt and scratches on the lens 4 that are not so large as to notify the passenger.

  In step S6, the timer 16 is used to determine the time continuity of a large dirt candidate region. As a result, the accuracy of the dirt determination can be improved by performing the dirt determination only when a predetermined time has passed in a state where the large dirt candidate region is at the same position. A specific method of the above steps S4 to S6 will be described later.

  FIG. 4 is a flowchart showing a procedure of operations of the camera 1 and the control unit 10. As described above, since the camera 1 according to the present embodiment is applied to a drive recorder or a navigation device of a vehicle, the dirt determination is started when the vehicle is turned on. Below, the flow when applied to a drive recorder will be described.

  In step S10, after the power is turned on, the contamination determination of the camera 1 is executed as an initial diagnosis. When the dirt determination is made in step S10, an alarm (warning) for the occupant is executed by the warning means 7 provided in the vehicle. That is, it is possible to notify the occupant before the vehicle starts running to prompt the lens 4 to be cleaned. The warning means 7 can be constituted by, for example, a warning light, a buzzer, a liquid crystal screen display or the like provided in a vehicle meter device.

  In step S11, an operation signal of the recording switch SW for instructing recording of the read image data in the memory is read. In step S12, it is determined whether or not the recording switch SW is turned on. If a positive determination is made in step S12, the process proceeds to step S13.

  In step S13, it is determined whether or not recording is in progress. If an affirmative determination is made, the process returns to step S11. If a negative determination is made, the process proceeds to step S14 to execute a recording start process. If a negative determination is made in step S12, the process proceeds to step S15 to determine whether or not recording is in progress. If an affirmative determination is made, the process proceeds to step S16 to execute a recording stop process and perform a series of controls. finish. On the other hand, if a negative determination is made in step S15, the process returns to step S11.

  Hereinafter, with reference to FIGS. 5 to 6, a luminance gradient calculation method by the luminance gradient calculation unit 13 will be described. FIG. 5 is a conceptual diagram showing the arrangement setting of the pixels (pixels) P in the V image (V). FIG. 6 is a conceptual diagram showing position setting for calculating the luminance gradient. The pixel P which is the minimum unit of luminance information has position information of the pixel position [i, j] on the V image (V). Luminance information of 0 to 255 is set for each pixel position [i, j].

  The luminance gradient indicates the amount of change in luminance in a certain area, and is calculated for each pixel position. That is, in FIG. 6, when a square having a vertical and horizontal length of 2 s centered on the pixel position [i, j] is set, the luminance gradient x in the x direction of this area A [i, j] is x = | (v [i, j−s] −v [i, j + s]) ÷ 2s | Similarly, the luminance gradient y in the y direction of the region A [i, j] is obtained by a calculation formula of y = | (v [i−s, j] −v [i + s, j]) ÷ 2s |. The luminance gradient xy (change rate R) of the region A [i, j] is an average value of the luminance gradient x in the x direction and the luminance gradient y in the y direction, that is, a calculation formula of xy = (x + y) / 2. Is required. In addition to calculating the brightness gradient at all pixel positions, it is possible to modify the position by overlapping the pixels to increase the calculation accuracy, or to reduce the pixel position to be calculated to shorten the calculation time. is there.

  The calculated luminance gradient xy indicates the rate of change in luminance between adjacent pixel positions. When the luminance gradient xy is zero, the difference in luminance between the adjacent pixel positions is zero, and the luminance gradient xy is large. It shows that the difference in luminance is large as

  As described above, the image of the stain B attached to the lens 4 is not focused on the stain B, and the light also circulates on the back side of the stain B, so that the contour is blurred. A gradation in which the luminance decreases stepwise toward the outside of B is formed. In this gradation region, a small luminance gradient appears continuously. On the other hand, in the focused image data (for example, the road sign O in FIG. 2), the contour of the subject and the coloring of the color are relatively clear, so that luminance gradation hardly occurs.

  FIG. 7 is a flowchart showing the procedure of the dirt candidate area extraction process executed by the dirt candidate area extracting unit. In the present embodiment, after calculating the luminance gradient xy at all pixel positions, the luminance gradient xy is divided into two parts, that is, a part within the set value ST and a part outside the set value ST. The set value ST is, for example, 0 to 10, and if the luminance gradient xy is within the set value ST (for example, 7), the stain determination value (Mask_dirt [i, j]) is set to 255, If it is outside the set value (for example, 12), the stain determination value is set to 0.

  In step S20, the pixel position for setting the stain determination value is set to [i = 0, j = 0]. In step S21, it is determined whether or not the luminance gradient xy (Hsv_deri [i, j]) exceeds the upper limit value (UPPER) of the set value. If an affirmative determination is made in step S21, the process proceeds to step S22, and the stain determination value (Mask_dirt [i, j]) is set to 0 (zero).

  On the other hand, if a negative determination is made in step S21, the process proceeds to step S23, in which it is determined whether or not the luminance gradient xy (Hsv_deri [i, j]) exceeds the lower limit value (LOWER) of the set value. If an affirmative determination is made in step S23, the process proceeds to step S24, where the stain determination value (Mask_dirt [i, j]) is set to 255. On the other hand, if a negative determination is made in step S23, the stain determination value (Mask_dirt [i, j]) is set to zero.

  Steps S <b> 25 to S <b> 29 are procedures for sequentially executing setting of the dirt determination value for all pixel positions. In step S25, it is determined whether J = cols has been reached, that is, whether the value of j has reached the last column of pixels. If an affirmative determination is made in step S25, the process proceeds to step S26, where J = 0 is set, and the process proceeds to the setting of the dirt determination value for the next line. On the other hand, if a negative determination is made in step S25, the process proceeds to step S27, the value of J is incremented, and the process returns to the determination in step S21.

  In step S28, it is determined whether i = rows has been reached, that is, whether the value of i has reached the last row of pixels. If a negative determination is made in step S28, the process proceeds to step S29, the value of i is incremented, and the process returns to the determination in step S21.

  By the above-described stain candidate area extraction process, the stain determination value is binarized to either zero or 255 at all pixel positions of the V image.

  FIG. 8 shows the V image (V1) after the binarization process, in which the stain determination value at each pixel position is set to zero or 255, and is divided into two colors of white and black. According to this binarization processing, since the luminance gradient xy is small (the change in luminance is small), a region that appears to be gradation in the RGB image is shown in white, and a region where the luminance gradient xy is large is shown in black. . That is, in the example of this figure, the white-colored area is recognized as a “dirt candidate area” that is highly likely to be dirt attached to the lens 4.

  However, if all the white areas are determined to be dirty, for example, a warning is issued even if there is a small dust that does not cause a problem for the occupant to check the image data, and convenience is reduced. Further, even if it is not caused by dirt, there may be a portion where the luminance gradient xy becomes small due to the actual color of the subject as shown in FIG. Therefore, in the present invention, only a portion where the white range in FIG. 8 appears to be large, that is, a region where pixel positions where the stain determination value is set to 255 continuously appear is displayed as a “dirt candidate region” by the large stain candidate region extraction unit 14. ".

  FIGS. 9 and 10 are conceptual diagrams showing a method of extracting a large dirt candidate region. In this figure, the dirt candidate areas E, E1, and E2 are extracted by the dirt candidate extraction process. Of these three, the region E having the dimensions in the vertical direction h and the horizontal direction W is the largest. In the present embodiment, in order to extract a region having both large vertical and horizontal dimensions, in each of the dirt candidate regions E, E1, and E2, the smaller of the vertical and horizontal dimensions is defined as “dirt length”. That is, the dirt lengths of E, E1, and E2 are L, L1, and L2, respectively.

  The dirt candidate areas E, E1, and E2 are ranked according to the dirt length. In the example of this figure, the stain length is L> L2> L1, and therefore the stain candidate regions are ranked first for E, second for E2, and third for E1. In the present embodiment, only three dirt candidate areas are shown. However, in actual data, for example, the dirt length of 50 extracted dirt candidate areas is calculated, and only the top ten of them are calculated. A process of extracting and storing as a “large stain candidate area” is executed. In this case, the 11th and lower dirt candidate regions are ignored as having no trouble in confirming the image.

  Finally, the control unit 10 determines that this is dirt attached to the lens 4 when a predetermined time (for example, 5 seconds) elapses while the stored large dirt candidate region exists at the same position. This is to prevent erroneous detection of contamination detection even when gradation is generated due to the actual color of the subject as described above. For example, even when the dirt detection process is started by turning on the ignition switch of the vehicle, in particular, in a motorcycle that rides across the vehicle body, the camera 1 can be operated by a steering wheel operation or the like even in a short time until the vehicle starts running. There are many people who change the position. If the lens 4 is actually dirty, the position of the stain candidate does not change even if the position of the camera 1 moves. However, if the lens 4 is caused by the color of the subject, the position of the stain candidate as the camera 1 moves. Can be distinguished from actual dirt.

  In addition, the structure of the camera, the configuration of the control unit, the form of the image sensor and the number of pixels, the form of the lens or the protective screen, the image conversion process from RGB data to HSV data, the setting of the predetermined area P when calculating the luminance gradient, The calculation range of the brightness gradient, the setting method of the dirt candidate region and the dirt length, the setting value used for the binarization process, the color for painting the V image, the setting time used for the dirt judgment over time, etc. are not limited to the above embodiment. However, various modifications are possible. The camera lens dirt detection device according to the present invention is not limited to a camera fixed to a vehicle, but can be applied to a work monitoring camera fixed to a machine tool, various portable cameras, and the like.

  DESCRIPTION OF SYMBOLS 1 ... Camera, 5 ... Operation means, 6 ... Power supply, 7 ... Warning means, 10 ... Control part, 11 ... RGB-HSV conversion means, 12 ... Luminance extraction means, 13 ... Luminance gradient calculation means 14 ... Dirt candidate area extracting means, 15 ... Large dirt candidate area extracting means, 16 ... Timer, B ... Dirt, D ... Image data, E, E1, E2 ... Dirt candidate area, E ... Large dirt candidate area, O ... Subject , P: pixel (predetermined area), R: luminance change rate (luminance gradient), S: subject, V: V image

Claims (4)

Based on the image (D) photographed by the camera (1) mounted on the vehicle, the stain for detecting that the stain (B) is attached to the lens (4) or the protective screen (4) of the camera (1). In the lens dirt detection device of the camera that performs the detection process,
A controller (10) for calculating a change rate (R) between the luminance of the predetermined region (P) of the image (D) and the surrounding luminance;
The control unit (10) sets a region where the predetermined region (P) where the change rate (R) is within a set value (ST) as a stain candidate region (E),
A lens dirt detection device for a camera, which performs a dirt detection process for determining that the dirt candidate region (E) is caused by the dirt (B).   2. The camera according to claim 1, wherein the control unit (10) starts the lens contamination detection process when the power source (6) of the control unit (10) is turned on. Lens dirt detector.   The lens (4) or the protective screen (4) is disposed at a position closer to the camera (1) than a range of the depth of field (F) of the camera (1). Or a lens dirt detection device for a camera according to 2;   The control unit (10) preferentially selects a region (E) of the dirt (B) having both large vertical and horizontal lengths, and determines that the region is caused by the dirt (B). The camera lens dirt detection device according to any one of claims 1 to 3.