JP2018197666A - Imaging apparatus, method for detecting dirt on lens, and program detecting dirt on lens - Google Patents

Abstract

To provide a technique for determining the extent of dirt on a camera lens with high accuracy.SOLUTION: In an imaging apparatus 100, a dirt-on-lens determination unit 50 extracts the feature amount indicating a luminance signal level per area of a reference pattern from a captured image obtained by imaging the reference pattern, which has a plurality of areas with a brightness difference in a one-dimensional or two-dimensional manner as part of a subject, determining dirt on a lens by comparing the extracted feature amount and a reference value. A cleaning instruction output unit 80 outputs, to a cleaning apparatus control unit 210, a signal to request cleaning of the lens on the basis of a determination result of the dirt on the lens from the dirt-on-lens determination unit 50. The cleaning apparatus control unit 210 causes a cleaning device 200 to clean a lens 10.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technique for detecting dirt on a lens of an imaging apparatus.

  In the case of an outdoor camera or an in-vehicle camera, the lens is easily soiled, so if a person looks at the image taken with the camera and suspects that the lens is dirty, the lens is manually cleaned. Since there are a wide variety of places where cameras can be mounted and usage patterns, it is necessary to automate lens cleaning.

  Patent Document 1 discloses an in-vehicle device that provides a light-shielding region in a photographing region of a camera, detects an adhering matter attached to a photographing lens, and removes the adhering matter based on an image obtained by photographing the light-shielding region. ing. Further, Patent Document 2 discloses an imaging apparatus that determines adhesion of water droplets on an imaging window in accordance with an evaluation value of an image signal captured through the imaging window and removes the adhered water droplets.

JP 2014-13449 A JP 2015-46851 A

  The in-vehicle device described in Patent Literature 1 detects an adhering substance based on an image obtained by photographing a light-shielding region, thereby preventing an image of the adhering substance from being mixed with another image and being erroneously detected. However, it is difficult to accurately determine the degree of contamination of the camera lens. For example, a state where foreign matter is attached can be detected relatively easily, but detection accuracy is insufficient for fine mist-like uniform dirt.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for accurately determining the degree of contamination of a camera lens.

  In order to solve the above-described problems, an imaging apparatus according to an aspect of the present invention is configured to capture the reference pattern from a captured image obtained by capturing a reference pattern having a plurality of areas having one-dimensional or two-dimensional brightness differences as part of a subject. A feature amount indicating a luminance signal level for each area is extracted, and a determination unit (50) that determines lens contamination by comparing the extracted feature amount with a reference value, and a determination result of lens contamination by the determination unit. A cleaning instruction output unit (80) for outputting a signal for instructing lens cleaning.

  Another aspect of the present invention is a lens dirt detection method. In this method, a feature amount indicating a luminance signal level for each area of the reference pattern is extracted from a captured image obtained by capturing a reference pattern having a plurality of areas having a one-dimensional or two-dimensional brightness difference as a part of a subject. And a determination step of determining lens contamination by comparing the extracted feature amount with a reference value.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to accurately determine the degree of contamination of a camera lens.

1 is an external view of an imaging apparatus according to an embodiment. FIG. 2A and FIG. 2B are diagrams for explaining an example of a reference pattern provided on the back side of the light shielding portion. FIG. 3A and FIG. 3B are diagrams illustrating a captured image by the imaging device. It is a functional lineblock diagram of an imaging device. It is a figure which shows the signal waveform obtained at the time of imaging of the reference pattern of the black-and-white checkered pattern of Fig.2 (a). FIG. 6A and FIG. 6B are flowcharts of lens dirt detection processing by the imaging apparatus.

  FIG. 1 is an external view of an imaging apparatus 100 according to the embodiment. The imaging apparatus 100 is installed on a ceiling or a wall as an example, and includes a light shielding unit 110 to shield light from an indoor lighting device from directly entering a lens of the imaging apparatus 100. Since the light shielding unit 110 is provided, the area indicated by reference numeral 140 is a light shielding area with respect to the shooting angle of view 130 of the imaging apparatus 100. In the light shielding region 140, shooting of the outside world is blocked, and the back side of the light shielding unit 110 (side facing the lens of the imaging device 100) is shot. A reference pattern 120 is provided on the back side of the light shielding unit 110.

  2A and 2B are diagrams illustrating an example of the reference pattern 120 provided on the back side of the light shielding unit 110. FIG. The reference pattern 120 is a pattern having a plurality of areas having a one-dimensional or two-dimensional light and dark difference. For example, a black and white checkered pattern as shown in FIG. 2A, a black and white pattern as shown in FIG. It is a white striped pattern. If the brightness difference is clear, it is not limited to black and white, and may be red and white, for example. The reference pattern 120 may be provided on the entire inner surface of the light shielding part 110 or may be provided on a part of the inner side of the light shielding part 110.

  FIG. 3A and FIG. 3B are diagrams for explaining a captured image 300 by the imaging apparatus 100. FIG. 3A shows a captured image 300 in the case where the reference pattern 120 is not provided in the light shielding unit 110 for reference. A region 310 in the upper part of the captured image 300 is an image obtained by photographing the light shielding region 140 in FIG. 1, and the outside is not photographed, and the inside of the light shielding unit 110 is reflected.

  FIG. 3B shows a captured image 300 when the light shielding unit 110 is provided with the reference pattern 120 shown in FIG. In the upper area 320 of the captured image 300, the reference pattern 120 provided inside the light shielding unit 110 is reflected.

  The imaging apparatus 100 analyzes an image of the reference pattern 120 that is reflected in a part of the captured image 300 and detects a decrease in MTF (Modulation Transfer Function) that occurs in the optical system of the imaging apparatus 100. Thus, the lens contamination of the imaging apparatus 100 is determined. MTF is an index for evaluating the imaging performance of a lens, and expresses the degree to which the contrast of a subject can be reproduced as a spatial frequency characteristic.

  Although not shown in FIG. 1, a cleaning device for cleaning the lens of the imaging device 100 is provided, and when lens contamination is detected, the cleaning device cleans the lens of the imaging device 100. The cleaning apparatus may further have a function of cleaning the reference pattern 120 provided in the light shielding unit 110. The cleaning device includes, for example, a wiper and a nozzle that injects cleaning liquid.

  FIG. 4 is a functional configuration diagram of the imaging apparatus 100. The lens 10 collects light from the subject and forms an image on the image sensor 20. The image sensor 20 is a photoelectric conversion sensor, and converts a received subject image into an electrical signal. The signal processing unit 30 arranges the sensor output from the image sensor 20 into a video signal. The video signal output unit 40 outputs the video signal supplied from the signal processing unit 30 to an external device.

  The control unit 60 controls the operation of each functional block of the imaging device 100. The control performed by the control unit 60 includes adjustment of shooting conditions such as the exposure time of the image sensor. The drive unit 70 drives the zoom, focus, and iris of the lens according to instructions from the control unit 60.

  The lens dirt determination unit 50 analyzes the image signal from the signal processing unit 30, extracts a feature amount indicating contrast from the image of the reference pattern 120 in the captured image, evaluates a decrease in MTF of the optical imaging system, and Judge dirt. When the lens contamination determination unit 50 determines that the lens contamination is caused by a decrease in MTF, the lens contamination determination unit 50 notifies the control unit 60 of the occurrence of lens contamination.

  When receiving the lens contamination occurrence notification from the lens contamination determination unit 50, the control unit 60 supplies a cleaning instruction to the cleaning instruction output unit 80. The cleaning instruction output unit 80 supplies a cleaning instruction signal to the cleaning device control unit 210.

  The cleaning device controller 210 controls the operation of the cleaning device 200 based on the cleaning instruction signal from the imaging device 100 and causes the cleaning device 200 to clean the lens 10 or the reference pattern 120.

  Hereinafter, the lens contamination detection method by the lens contamination determination unit 50 will be described in detail.

Lens contamination is caused by foreign matter adhering to the lens surface.
(A) When a foreign substance adheres to a limited area of a very small part of the lens;
(B) There are cases where deposits are finely and uniformly deposited in a mist.
In either case, it is necessary to detect and determine as lens contamination.

  The lens dirt determination unit 50 calculates a feature amount of a signal waveform obtained from an image obtained by capturing the reference pattern 120 in a reference state without lens dirt, and stores it in advance as a reference value. During actual lens dirt evaluation, the lens dirt determination unit 50 extracts a feature value of a signal waveform from an image obtained by photographing the reference pattern 120, and compares the extracted feature value with a reference value stored in advance. Evaluate the condition of lens dirt.

  As a feature amount of the signal waveform, a feature amount indicating a spatial frequency characteristic of the lens such as contrast (brightness / darkness ratio) or MTF is extracted from the image.

  The lens dirt determination unit 50 extracts the feature amount of the signal waveform from the image area of the reference pattern 120 in the captured image of the imaging device 100 while the imaging device 100 is in operation, and extracts the extracted feature amount and the reference value. If the difference exceeds a predetermined threshold, it is determined that the lens is dirty, and the cleaning instruction output unit 80 outputs a cleaning instruction signal.

  Since the reference pattern 120 is a pattern having a contrast difference such as a black and white checkered pattern, a feature amount indicating a contrast and a spatial frequency component amount can be calculated from a captured image of the reference pattern 120.

  A feature amount indicating a luminance signal level is calculated for each area having a difference in brightness of the reference pattern from an image of one frame in which the reference pattern 120 is captured. The feature value of the luminance signal level for each area is extracted from the maximum value / minimum value of the luminance signal level, the average luminance level, the histogram indicating the distribution of luminance values, the histogram of frequency components included in the luminance signal, the contrast value, and the like.

  In the case of the black and white checkered reference pattern 120 as shown in FIG. 2A, the maximum value of the luminance signal in the area of the reference pattern 120 is the white peak value, and the minimum value of the luminance signal is the black bottom value. The average luminance level is an average value of luminance values in the area. The maximum value and the minimum value of the luminance signal level can be acquired by the sample / hold function of the minimum value / maximum value normally provided in the imaging apparatus 100.

  A histogram obtained by tabulating each luminance value in the area of the reference pattern 120 and the number of pixels having the luminance value indicates a distribution of luminance values in the area. As the luminance signal level histogram, a histogram obtained by counting the number of pixels having a luminance value larger than a predetermined threshold or the number of pixels having a luminance value smaller than the predetermined threshold in the area may be used. With the histogram of the luminance signal level for each area of the reference pattern 120, the high-frequency component amount of the luminance signal can be evaluated in the original image area. On the other hand, the frequency component histogram of the luminance signal is obtained by performing spatial frequency conversion on the original image and evaluating the amount of the high frequency component in the frequency domain.

  The contrast value of the reference pattern 120 is obtained as the difference between the maximum value and the minimum value of the luminance signal level. The contrast value increases as the brightness difference in the area of the reference pattern 120 increases. When the lens surface is uniformly soiled, the contrast difference in the area of the reference pattern 120 is lowered and the contrast value is reduced.

  Here, the maximum value of the luminance signal level is the white peak value, but the luminance signal level varies depending on the environmental conditions during imaging. For the purpose of eliminating the influence due to the difference in environmental conditions, for example, an operation for scaling the measured value is performed or an electronic shutter is set so that the white peak value at the time of shooting becomes the same level as the reference value stored in advance. The imaging condition such as the exposure time of the image sensor 20 is adjusted. As a result, it is possible to compare the measured value of the feature amount with the reference value in a state where the white peak values are aligned, eliminating the difference in environmental conditions.

  When a foreign object adheres to the lens surface, the amount of incident light on the image sensor decreases due to the foreign object adhering to a specific area of the lens, and light shielding occurs in the specific area. The quantity deviates from the reference value. By detecting a change from the reference value, foreign matter adhesion can be determined.

  On the other hand, when mist-like water droplets uniformly adhere to the lens surface, the incident light on the lens is irregularly reflected by the water droplet group on the lens surface, and the white peak value of the image obtained through the image sensor is attenuated on the whole. The image looks whitish and has a haze, and in the black portion of the image, the black portion of the image appears gray and a “black float” occurs. The same is true when sand dust or the like uniformly adheres to the lens surface, but the white peak value is slightly attenuated due to the light shielding effect of sand.

  In this way, when fine particles such as mist and sand uniformly adhere to the lens surface as dirt, measuring the MTF of the lens optical system causes a change in the spatial frequency characteristics that particularly lowers the transmission characteristics in the high frequency band, As a result, a “round” occurs in the rectangular wave of the signal. In addition, the white level is lowered and the black floating phenomenon occurs due to light blocking and scattering. By detecting such a decrease in MTF, it is possible to determine uniform contamination on the lens surface.

  A method for detecting waveform rounding due to a decrease in MTF of the lens optical system will be described in detail.

  FIG. 5 is a diagram illustrating signal waveforms obtained when the black and white checkered reference pattern of FIG. 2A is captured. The checkerboard lattice size is N pixels × N pixels. In the figure, the luminance value of the pixel is graphed for the pixel column in the i-th row (i = 1,..., N), the horizontal axis is the pixel number of the pixel column, and the vertical axis is the luminance value. . Here, changes in luminance are shown in two areas having a difference in contrast between the white lattice and the black lattice. The white lattice area is a high luminance area, and the black lattice area is a low luminance area.

  The signal waveform obtained when the black and white checkered reference pattern is imaged is determined by the transfer characteristics of the optical imaging system. If the transfer characteristics are sufficiently good up to the high frequency band, the signal waveform is a rectangular wave. If the MTF in the high frequency band is reduced, the harmonic component of the rectangular wave is attenuated, and the signal waveform is a sine wave of the fundamental frequency. Close shape.

  Graph G1 is a signal waveform when there is no dirt on the lens surface, and is a substantially rectangular wave as a standard state. The luminance value has a peak value of 2.0 in the white lattice area, and the luminance value in the black lattice area. Takes a bottom value of zero.

  When viewed from the frequency plane, the rectangular wave has a structure in which the fundamental wave of the fundamental frequency and the respective harmonics are combined. However, when uniform dirt adheres to the lens optical system and the MTF is lowered, the rectangular wave has a frequency plane. The configuration seen in (1) changes and the high frequency component is attenuated, so that waveform rounding occurs.

  Graphs G <b> 2 to G <b> 7 show signal waveforms when uniform dirt attached to the lens optical system increases. When the high frequency component is attenuated from the rectangular wave in the graph G1, the signal is distorted at the boundary of the lattice, and the boundary of the area is not clear. Further, as the uniform stain increases, the white peak value becomes smaller than 2.0 while the black bottom value becomes larger than 0.

  A method for evaluating the frequency component of the rectangular wave in the original image will be described. To distinguish between a square wave and a sine wave, the number of effective pixels having a signal level greater than the high-level pixel counting threshold in the high-luminance area and an effective signal level having a signal level smaller than the low-level pixel counting threshold in the low-luminance area The number of pixels may be compared as a feature amount. The larger the number of effective pixels in the high luminance area and the number of effective pixels in the low luminance area, the greater the amount of high frequency components, and thus the signal can be detected as a signal close to a rectangular wave. In other words, the degree of attenuation of the high-frequency component of the image signal can be estimated by counting the number of pixels present in the high luminance / low luminance area where the contrast is large.

  Specifically, as shown in FIG. 5, a first threshold value a smaller than a white peak value 2.0 in a reference state without lens contamination is set, and a signal level larger than the first threshold value a is set for each white area. The number of pixels that are included is counted as the number of effective pixels, a second threshold value b that is larger than the black bottom value 0 in the reference state is set, and the number of pixels that have a signal level smaller than the second threshold value b for each black area Count. The number of effective pixels in the high brightness area and the number of effective pixels in the low brightness area are counted for each horizontal N pixel for the i-th row of pixel columns, and i = 1,... The number of effective pixels of each black lattice can be obtained.

  In the reference state where there is no lens contamination as in the graph G1, since it is a rectangular wave, the number of effective pixels for each area is N × N for both the white and black lattices of the reference pattern 120, and this is the reference value. As the uniform dirt on the lens surface increases as in the graphs G2 to G7, the waveform rounding gradually increases and the number of effective pixels for each area decreases. The degree of waveform rounding can be evaluated by comparing the number of effective pixels in the white area and the number of effective pixels in the black area with reference values.

  As another determination method, the difference between the maximum value and the minimum value of the luminance signal level may be evaluated as a contrast value, and the degree of waveform rounding may be determined based on the magnitude of the contrast value. As shown in FIG. 5, in the reference state without lens contamination, the maximum value of the luminance signal level is 2.0 of the white peak value and the minimum value is 0 of the black bottom value, so the contrast value is 2.0. . When lens contamination occurs, the white peak value decreases and the black bottom value increases. For example, when the white peak value decreases to 1.8 in the high luminance area and the black bottom value increases to 0.2 in the low luminance area, the contrast value becomes 1.6. The degree of lens contamination can be evaluated by the decrease in contrast value.

  The method for detecting the attenuation of high-frequency components in the original image region has been described. As another method, the frequency histogram is obtained by converting the image signal into the spatial frequency region by discrete cosine transform or the like, and the attenuation of the high-frequency components is directly performed. May be measured. Specifically, a black and white checkered reference pattern is imaged in a reference state free from lens contamination, spatial frequency conversion is performed on the image signal, and the spatial frequency component is stored as a reference value. When lens contamination occurs, the harmonic component of the image signal is attenuated, so that the lens contamination can be detected by comparing with the reference value. Alternatively, a change in the configuration of the spatial frequency component may be detected by filtering out one or more specific spatial frequency components and measuring the amount thereof.

  As preprocessing for lens contamination determination, the signal processing unit 30 performs automatic gain control, automatic white balance control, and the like so that a good imaging result can be obtained stably at all times. When determining lens contamination, it is possible to always make a correct determination regardless of the day and night weather by setting only the region of the reference pattern in which light is shielded as the region for automatic gain control.

  It is possible to detect a decrease in the MTF of the optical system due to lens contamination by evaluating the result of imaging a known reference pattern having a difference in brightness by the determination method as described above. By interpreting the decrease in MTF as lens contamination, it is possible to detect adhesion of uniform contamination such as fog and sand. The above-described determination method can detect not only the uniform dirt on the lens surface but also the case where a foreign substance adheres to a part of the lens.

  FIGS. 6A and 6B are flowcharts of lens dirt detection processing by the imaging apparatus 100. FIG. In the lens dirt detection process of FIG. 6A, when lens dirt is determined, the lens is cleaned immediately. In the lens dirt detection process of FIG. 6B, after the lens dirt is determined, the reference pattern is first cleaned, and when the lens dirt is still determined, the lens cleaning is performed.

  Reference is made to FIG. The image sensor 20 is driven, one frame is imaged, and a video signal is acquired (S10). The lens stain determination unit 50 calculates a feature amount indicating contrast from the video signal in the reference pattern area (S12).

  The lens contamination determination unit 50 determines the lens contamination state by comparing the feature amount with a reference value (S14). If it is determined that the lens is not dirty (NO in S14), the process returns to step S10, the next frame is imaged, and the feature amount is calculated from the video signal in the reference pattern area of the next frame.

  When it is determined that the lens is dirty (YES in S14), the cleaning instruction output unit 80 supplies a cleaning instruction signal to the cleaning device control unit 210, and the cleaning device control unit 210 drives the cleaning device 200, so that the lens 10 Is washed (S16). The cleaning device controller 210 can drive the cleaning device 200 to clean the reference pattern 120 as well.

  Reference is made to FIG. The image sensor 20 is driven, one frame is imaged, and a video signal is acquired (S20). The lens stain determination unit 50 calculates a feature amount indicating contrast from the video signal in the reference pattern area (S22).

  The lens contamination determination unit 50 determines the lens contamination state by comparing the feature amount with the reference value (S24). If it is determined that the lens is not dirty (NO in S24), the process returns to step S10, the next frame is imaged, and the feature amount is calculated from the video signal in the reference pattern area of the next frame.

  When it is determined that the lens is dirty (YES in S24), the cleaning instruction output unit 80 supplies a cleaning instruction signal for cleaning the reference pattern 120 to the cleaning device control unit 210, and the cleaning device control unit 210 causes the cleaning device 200 to operate. Driven to clean the reference pattern 120 (S26).

  The image sensor 20 is driven, one frame is imaged, and a video signal is acquired (S28). The lens stain determination unit 50 calculates a feature amount indicating contrast from the video signal in the reference pattern area (S30).

  The lens contamination determination unit 50 determines the lens contamination state by comparing the feature amount with the reference value (S32). If it is determined that the lens is not dirty (NO in S32), the process returns to step S10 and the subsequent steps are repeated.

  When it is determined that the lens is dirty (YES in S32), the cleaning instruction output unit 80 supplies a cleaning instruction signal for cleaning the lens 10 to the cleaning device control unit 210, and the cleaning device control unit 210 drives the cleaning device 200. Then, the lens 10 is washed (S34).

  In the case of FIG. 6B, even if it is determined that the lens is dirty from the feature amount of the image signal, the lens 10 is not immediately cleaned, and the reference pattern 120 is first cleaned. This assumes that the lens 10 is not dirty and the reference pattern 120 is dirty. While the cleaning device 200 cleans the lens 10, the subject cannot be photographed and the video is interrupted. By adding a step of cleaning the reference pattern 120 before cleaning the lens 10, it is possible to prevent the image from being interrupted by cleaning the lens 10 when the lens 10 is not dirty.

  Since the reference pattern 120 is provided on the back side of the light shielding unit 110, the shooting conditions such as the exposure time suitable for shooting the reference pattern 120 may be different from the shooting conditions for shooting an external subject. is there. Therefore, the shooting mode will be described separately for the following three cases.

(1) When performing lens contamination determination by simultaneously capturing both the subject area and the reference pattern area without performing imaging with varying exposure time Normal imaging is performed, and lens contamination determination is also performed in the process. The brightness of the entire subject is evaluated, and the exposure time of the image sensor 20 is controlled so that it can be maintained satisfactorily. Since there is no need to switch the shooting mode, it is the simplest. However, when the outside world is extremely bright and the back side of the light-shielding part 110 is dark, the image of the reference pattern 120 becomes dark, which affects the accuracy of stain detection. Sometimes.

(2) In the case of providing a lens dirt determination shooting mode separately from the normal imaging mode In addition to the “normal shooting mode”, the reference pattern area can be satisfactorily imaged and the exposure time suitable for determining lens dirt, etc. A “lens dirt determination shooting mode” in which shooting conditions are set is provided, the normal shooting mode is switched to the lens dirt determination shooting mode, the reference pattern area is shot, and lens dirt determination is performed. For example, a “lens dirt determination imaging mode” is set in which the exposure time and signal processing are adjusted so that the amplitude of the signal waveform obtained by imaging the reference pattern becomes a predetermined value.

  The rule for shifting from the normal shooting mode to the lens dirt determination shooting mode is set according to the environment in which the imaging apparatus 100 is used. For example, when used as a back camera of a car, the mode shifts to a dirt determination mode when the car gear is not in the back, and the normal shooting mode is always set while the car gear is in the back. To control.

(3) In the normal imaging mode, when changing the exposure time according to the dynamic range of the image and performing lens contamination determination Generally, when performing wide dynamic range imaging, the exposure time of the image sensor is changed for each frame. A wide dynamic range image is generated by combining the long exposure image and the short exposure image. If a wide dynamic range imaging method is used, two types of exposure images can be acquired by setting two exposure conditions independently. A short exposure image is picked up with a short exposure time suitable for the external subject and a long exposure image is picked up with a long exposure time suitable for the reference pattern according to the brightness difference between the external subject and the reference pattern. In this way, as in the above (2) “Lens Dirt Determination Shooting Mode”, the shooting conditions such as the exposure time are adjusted in accordance with the reference pattern provided on the back side of the light-shielding portion 110, and the lens dirt is set. The determination can be made satisfactorily.

  In the method (1) for simultaneously capturing both the subject area and the reference pattern area in the normal imaging mode, when there is a large difference between the brightness of the subject area and the brightness of the reference pattern area, the above (2) It is impossible to set the signal level suitable for the reference pattern as in the “lens dirt determination photographing mode”. Therefore, when evaluating lens contamination, it is necessary to perform control to appropriately change parameters such as a threshold according to the brightness of the subject. On the other hand, in the method (3) using the wide dynamic range imaging, the shooting conditions such as the exposure time are independently adjusted according to the brightness of the subject area and the brightness of the reference pattern area, and the subject area and the reference Each pattern area can be imaged satisfactorily. Further, the control for switching the photographing mode as in the above (2) is unnecessary, and the lens contamination can be evaluated in the reference pattern area at the same time as imaging the subject area.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above-described embodiment, the reference pattern 120 is provided on the back side of the light shielding unit 110 and the reference pattern 120 is photographed. However, when the imaging device 100 is not provided with the light shielding unit 110, the reference pattern 120 is provided on, for example, a wall or a floor. The lens dirt may be determined from the image of the reference pattern 120 that is installed and photographed as a part of the subject in the outside world. Alternatively, the orientation of the imaging apparatus 100 may be controlled so that the reference pattern 120 installed on a wall or the like can be photographed, the photograph is focused on the reference pattern 120, and lens contamination may be determined from the image of the reference pattern 120. .

  The functional configuration of each device described in the embodiment can be realized by hardware resources or software resources, or by cooperation of hardware resources and software resources. Processors, ROM, RAM, and other LSIs can be used as hardware resources. Programs such as operating systems and applications can be used as software resources.

  DESCRIPTION OF SYMBOLS 10 Lens, 20 Image sensor, 30 Signal processing part, 40 Video signal output part, 50 Lens dirt determination part, 60 Control part, 70 Drive part, 80 Cleaning instruction | indication output part, 100 Imaging device, 110 Light-shielding part, 120 Reference pattern, 200 Cleaning device, 210 Cleaning device controller.

Claims (7)

A feature amount indicating a luminance signal level for each area of the reference pattern is extracted from a captured image obtained by capturing a reference pattern having a plurality of areas having one-dimensional or two-dimensional brightness differences as a part of the subject, and the extracted features A determination unit that determines lens contamination by comparing the amount with a reference value;
An imaging apparatus comprising: a cleaning instruction output unit that outputs a signal for instructing lens cleaning based on a determination result of lens contamination by the determination unit.   The imaging apparatus according to claim 1, wherein the reference pattern is provided in a light shielding portion of a camera.   The imaging apparatus according to claim 1, wherein the reference pattern is a striped pattern or a checkered pattern.   The determination unit counts the number of effective pixels having a luminance signal level that is significant compared to a predetermined threshold value for each area of the reference pattern, and compares the effective pixel number for each area with a reference value as a feature amount. The imaging apparatus according to claim 1, wherein   5. The imaging apparatus according to claim 1, wherein the cleaning instruction output unit outputs a signal instructing cleaning of the reference pattern prior to lens cleaning. Extracting a feature amount indicating a luminance signal level for each area of the reference pattern from a captured image obtained by capturing a reference pattern having a plurality of areas having one-dimensional or two-dimensional brightness differences as part of a subject;
And a determination step for determining lens contamination by comparing the extracted feature amount with a reference value. Extracting a feature amount indicating a luminance signal level for each area of the reference pattern from a captured image obtained by capturing a reference pattern having a plurality of areas having one-dimensional or two-dimensional brightness differences as part of a subject;
A lens dirt detection program which causes a computer to execute a determination step of judging lens dirt by comparing an extracted feature amount with a reference value.

Images