JP5207057B2 - Dirt detection device and method for detecting contamination of an imaging device - Google Patents

Description

  The present invention relates to a detection apparatus and a detection method for detecting dirt on an imaging apparatus, particularly dirt on its optical system, using stereo images.

2. Description of the Related Art Conventionally, there has been known an image monitoring apparatus that installs a camera toward a monitoring target area and detects the appearance of an intruder based on an optical image acquired thereby.
There are various monitoring target areas where the cameras of the image monitoring apparatus are installed. For this reason, the lens surface of the camera and the surface of the monitoring window provided on the camera storage case may be soiled indoors, for example, with cigarette dust, etc., and outdoors, for example, dirt or mud due to wind and rain. Sometimes. Such dirt can blur or blur the image and make it unclear, which can hinder surveillance based on the image.

Therefore, it is conventionally known to detect such dirt based on an image acquired by a camera.
For example, when the image becomes unclear due to dirt, the sharpness (referred to as edge amount) of the change in pixel value at the portion (so-called edge) indicating the outer edge of the subject such as the boundary between subjects in the image is reduced. In addition, an image monitoring apparatus that detects dirt on a lens surface or the like based on the fact that the edge amount has become a predetermined value or less is known.
In this image monitoring device, when dirt is attached to the lens surface of the camera, the reduction in the area with a large edge amount in the image is relatively large, while the decrease in the area with a small edge amount is relatively small. On the other hand, when the illumination changes, the contamination of the lens surface is detected by utilizing the fact that the portion with the normal edge amount and the portion with the larger edge amount change to the same extent, and erroneous recognition is performed. (Refer to patent document 1).
However, since this image monitoring apparatus determines the sharpness of the edge image, it cannot be detected unless the contamination is severe to some extent. In addition, if an object with originally blurred brightness is captured, there is a problem that the lens is misidentified as dirty.

Separately from this, a lens cover-shaped light guide is provided in front of the image sensor and the lens to reflect the LED light to detect dirt and water droplets adhering to the lens cover surface. An imaging device is known (see Patent Document 2).
This in-vehicle photographing apparatus has a problem that it cannot be used in an existing system because it needs to have a unique mechanism called a lens cover-shaped light guide unit and a light emitting unit such as an LED. In addition, since a method of detecting dirt or the like from the difference between the images obtained when the light emitting unit is turned on and off, the difference between frames is used continuously every frame. There is also a problem that image recognition is greatly limited.
Japanese Patent No. 4,087,600 JP 2008-066430 A

  The present invention has been made in order to solve the problems of the prior art relating to the detection of dirt on the lens of a surveillance camera, and the object thereof is applicable to an existing surveillance system. It is to ensure that maintenance of the surveillance camera can always be performed accurately by detecting the dirt quickly and reliably without confusing foreign matters in front of the camera with dirt such as water droplets on the lens surface.

The present invention is a dirt detection device including a photographing device that captures a stereo image of a subject and an image processing device that processes the captured stereo image, and the image processing device includes means for obtaining parallax from the stereo image; Means for creating a distance image and an evaluation value image based on the obtained parallax; means for storing a distance image and an evaluation value image serving as a reference as a distance background image and an evaluation value background image; and the distance background image and the evaluation value by comparing the distance image and evaluation value image created from a stereo image captured by the background image and any time, characterized in that it comprises means, for detecting the contamination of the optical system of the imaging device.
Another aspect of the present invention includes the steps of obtaining a stain detection method for detecting contamination of the imaging apparatus based on the stereo image obtained by shooting an object, comprising the steps of shooting a stereo image of an object, the disparity from the stereo image, determined a step of creating a distance image and evaluation value image-out based on parallax, and a step of storing the reference become distance image and evaluation value image as respective distances background image and evaluation value background image, said distance background image and evaluation by comparing the distance image and evaluation value image created from a stereo image captured by the value background image and any time, characterized in that it and a step of detecting the contamination of the optical system of the imaging device.

  Since an existing camera can be used for early detection without confusing foreign matter in front of the camera and dirt on the lens surface, the camera can always be easily maintained in a normal state.

Hereinafter, the dirt detection apparatus and the detection method using the parallax in the stereo image of the present invention will be described. Before that, the parallax is detected using the stereo image (both left and right images) photographed using two cameras. A method of obtaining will be described with reference to the accompanying drawings.
FIG. 1 is a perspective view seen from an oblique upper side of the camera 20 showing a state in which the foreign object E placed in the room is photographed by the two cameras 20L and 20R, and FIG. 2 shows each camera 20 (L, R). The photographed image (stereo image) photographed in step 1 is shown.
In the shooting environment of the stereo camera shown in FIG. 1, when the same object (foreign matter) E is shot as a stereo image using the two cameras 20 (L, R), the lenses of the two cameras 20 (L, R) are used. Assuming that all of the distortion caused by this and the difference in internal parameters between the two cameras 20 have been corrected, the photograph (image) taken is the same image slightly shifted in position on the left and right as shown in FIG. become.
That is, due to the difference in the installation positions of the two cameras 20 (L, R), parallax due to the difference in the arrangement positions of the left and right lenses of the cameras 20 (L, R) occurs in the left and right images. In this case, since the cameras are installed side by side in the horizontal direction, parallax occurs in the horizontal direction.

FIG. 3 is a diagram showing the L image (the captured image of the camera 20L set on the left side) and the R image (the captured image of the camera 20R set on the right side) arranged vertically on the XY coordinates.
In the figure, the arrow indicates parallax, and the length indicates the magnitude of the parallax. As is clear from this figure, the parallax is larger for objects closer to the front as viewed from the camera and smaller for objects at the back. Therefore, an object that is sufficiently far away from the distance between the cameras 20 (L, R) is photographed. In this case, the parallax is substantially zero.

Next, with reference to FIG. 3, a stereo matching process for determining how much parallax an object in the captured left and right images will be described.
In parallel viewing, a point corresponding to a certain point in the L image must be on the left side in the horizontal direction in the R image. Therefore, when the image is placed on the XY coordinates, the Y coordinate is fixed. The position of the point is searched while shifting the coordinates on the X coordinate one by one, that is, while scanning.

  As shown in FIG. 3, when a specific area S is defined in the L image and the area S corresponding to the specific area S is searched for in the R image, first, the same X coordinate is examined, and then scanned to the left from there. To go. As a typical evaluation value used at this time, an absolute value of a luminance difference between pixels at positions corresponding to the L image and the R image is obtained for each pixel in the specific region S, and this is summed in the region S. Since there is the obtained sum of absolute differences (SAD), the SAD is obtained between the upper region and the lower region in the figure according to the X coordinate. By arranging the SADs in order with the X coordinate on the vertical axis and the horizontal axis, an evaluation value graph can be formed.

FIG. 4 shows a graph of evaluation values created in this way.
In the above graph, the evaluation value is the minimum at approximately 60 of the X coordinate, so that the target point in the L / R image corresponds to that portion, indicating that the parallax in this case is 60. Yes.

The method for obtaining the parallax using both images in the case where the L / R images are taken using the horizontally installed camera has been described above. Next, an optical system (such as a lens of the camera) using this method ( Here, a dirt detection method according to an embodiment of the present invention for distinguishing between dirt on a transparent lens cover and the like and an actual photographing object will be described.
The stain detection method according to the present embodiment uses image data (herein, referred to as evaluation value image data for the sake of convenience) created based on the obtained distance image and the evaluation value. Then, the minimum evaluation value for each pixel is obtained, and that value is stored in association with each pixel for each pixel constituting the image, that is, for each pixel.

FIG. 5 shows a past (background) image of one image (here, L image) and an image in a state where dirt is attached to the current L image photographing lens, respectively, an input image, a distance image, and an evaluation value. It is the figure divided and shown to the image.
Here, the distance image is an image in which the distance from the camera is obtained for each pixel of the stereo image and the distance data is provided (stored) corresponding to each pixel, that is, an image to which the distance data is added. In addition, the evaluation value image is an image obtained by adding the SAD minimum value obtained for each pixel in association with the above image to each pixel, that is, an image including the difference absolute value sum data.

Here, for example, the distance image is represented by dividing the image into 3 parts by dividing the distance image into 3 parts by dividing the image by 3 m or more into a large distance, 1.5 m to 3 m in the distance, and less than 1.5 m into the small distance. A drawn portion indicates a region belonging to a large distance, a portion drawn by a one-dot chain line indicates a region belonging to a medium distance, and a portion drawn by a broken line indicates a region belonging to a small distance.
The evaluation value image is obtained by comparing the evaluation value, that is, the minimum value of the sum of absolute differences with a predetermined threshold value, and by dividing the evaluation value smaller than the predetermined threshold value into three parts, that is, evaluation large, medium, and small, It is an image that is displayed in such a manner that the image is distinguished and can be identified.

FIG. 5A (1) is a background image taken by the left camera 20L, for example, and is a reference past image. On the other hand, FIG. 5A (2) is a current image (input image) photographed by the same camera in order to determine dirt.
As shown in the drawing, in the input image of FIG. 5A (2), dust (hatched circle) is shown in the center. When the adhering dust is fine dust, the field of view is not obstructed as shown in the figure, but it appears to be circularly blurred within a certain range.

When the parallax is obtained from the input image, a change occurs as compared with the parallax obtained from the past background image. For example, the parallax cannot be obtained in a part of the part (circle) to which dirt is attached. That is, since the dirt is dirt attached to the lens of the L camera, in FIG. 5A (2), the L image shows the dirt, and that part is blurred. On the other hand, there is no dirt at the same location of the R image. Even if the two images are compared, a portion where the parallax cannot be obtained occurs. Therefore, a change occurs in the distance image in the past background image and the input image.
Similarly, a change occurs in the evaluation value image. That is, the lower the evaluation value is, the higher the matching accuracy is. Therefore, the correspondence (matching accuracy) in the left and right images is lowered and the evaluation value is increased in a portion where dirt is attached. As a result, FIG. 5C (2) shows that the portion where the evaluation value was small becomes large, and in this case, it is barely within the threshold value (it is not displayed when the threshold value is exceeded).

  In the present invention, the distance image and the evaluation value image are created from both the left and right images by the two cameras arranged on the left and right as described above, and this is used as a past reference background image (image with no lens stain). By comparing the current image (input image) with the current image, it is possible to easily and quickly perform maintenance management of the surveillance camera by accurately determining dirt on the lens.

Next, a description will be given of a dirt detecting device for detecting lens dirt on a camera and an image processing apparatus of the same for implementing the present invention together with the camera 20 and the like.
FIG. 6 is a block diagram showing a dirt detection apparatus including the image processing apparatus.
The dirt detection device according to the present embodiment includes an image processing device 10, a camera 20, an input device 30, and a display unit 40 as illustrated.
The camera 20 is a camera provided with a photoelectric conversion element such as a CCD (Charge Coupled Device), for example, and inputs a photographed image to the image processing apparatus 10 as image data. The input device 30 is used for inputting necessary data such as operation and setting of the image processing apparatus. The display unit 40 includes display means such as a liquid crystal (LCD), and displays image data output from the image processing apparatus 10.
The image processing apparatus 10 includes an interface 12 for connection to an external device such as the camera 20, a CPU (Central Processing Unit) 14, and a RAM (Random Access Memory) 16. The CPU 14 further includes a stereo calculation unit. 141, the other calculation unit 142, the background update unit 143, and the stain detection comparison determination unit 144. The RAM 16 stores the acquired image data storage unit 161, the intermediate result data storage unit 162, and the background data storage. And a read only memory (ROM) (not shown) that stores a program for driving and controlling the image processing apparatus.

  Stereo input image data input as image data from a camera such as a CCD camera 20 is input to the image processing apparatus 10 via the interface 12. The input stereo image data is stored in the acquired image data storage unit 161 of the RAM 16. The CPU 141 reads stereo input image data from the acquired image data storage unit 161 and performs a calculation process in the stereo calculation unit 141. Here, in the pre-processing unit 141a, a parallax for each pixel of the left and right images is obtained, and calculation processing for obtaining the SAD is performed based on the parallax. Perform arithmetic processing.

  The calculation results in the stereo calculation unit 141 are stored in the distance image data storage unit 162a and the evaluation value image data storage unit 162b of the intermediate result data storage unit of the RAM 16, respectively. The CPU 14 reads out the distance image data and the evaluation value image data from the distance image data storage unit 162a and the evaluation value image data storage unit 162b, respectively. Based on these data in the other calculation unit 142, for example, a distance Evaluation value image data is read out and compared with a predetermined threshold value, and evaluation value image data smaller than that is further divided into large, medium, and small sizes, and an image is created by adding each to pixel data.

On the other hand, the background update unit 143 of the CPU 14 uses the read distance image data and evaluation value image data as background data, and uses these as the distance background image data storage unit 163a and the evaluation value background image data storage unit 163b of the background data storage unit 163, respectively. The process to memorize is performed.
The stain detection comparison / determination unit 144 of the CPU 14 reads the current distance image data and the evaluation value image data from the midway result data storage unit 162 of the RAM 16, and simultaneously reads the distance background image data serving as a reference read from the background data storage unit 163. The evaluation value is compared with the background image data to determine the presence or absence of dirt on the optical system such as a lens from the dirt on the image.
The distance image based on the distance image data and the evaluation value image data, the evaluation value image, and the determination result of the stain detection comparison determination unit 144 are displayed on the display screen of an external liquid crystal (LCD) display unit 40 via the interface. The

FIG. 7 is a flowchart illustrating a processing flow for detecting contamination of an optical system such as a lens using the image processing apparatus 10 described above.
First, the left and right images of the monitoring area are acquired by operating the two monitoring cameras (S101). The image processing apparatus 10 performs a series of processes for obtaining the parallax between the pixels of both images using the two acquired images (S102). Subsequently, SAD is obtained based on the parallax, an evaluation value image is obtained from the SAD (S103), and similarly, a distance image is obtained based on the parallax (S104). Either one may be acquired first). Here, a process for detecting contamination of the optical system of the monitoring camera is performed, or other processing, for example, a determination to perform foreign matter monitoring processing using the obtained stereo image is performed (S105). When performing the detection process (S105, YES), first, it is determined whether or not to update the background (S107).

  When the background is updated (S107, YES), the background update unit 143 receives the input R.D. The evaluation value background image and (S108) the distance background image are acquired from the L image (S109). When the background is not updated (S107, NO), the stain detection comparison / determination unit 144 continues to use the existing evaluation value background image. Are used to compare the evaluation value image acquired in step S103 with the evaluation value background image (S110). As a result, when the evaluation value of the pixel to be inspected of the current image is equal to or smaller than that of the same pixel of the evaluation value background image (S111, YES), both images match for the pixel. The pixel inspection is terminated.

  In step S111, when the evaluation value of the current pixel to be inspected is larger than that of the corresponding pixel of the evaluation background image (S111, NO), the CPU 14 determines whether a distance image is obtained (S112). If not found (S112, NO), a stain flag is set for the pixel as a specific designation by means for designating the stain of the pixel here, and if a distance image is found (S112, YES), The stain detection comparison / determination unit 144 compares the distance image created from the input image for the pixel with the distance background image (S113). If the distance between the current image and the background image is equal (S114, YES), the distance is determined. Although the evaluation value was higher than that of the background image even though the value did not change, it was determined that the optical system was contaminated and the pixel was stained. A flag. When the distance between the current image and the background image at the pixel is not equal (S114, NO), it is considered that the distance has changed due to the entry of an object that is not in the background. To do.

When the inspection for the pixel has been completed, it is determined whether or not the inspection has been completed for all of the pixels to be inspected (S117). If the inspection has not yet been completed, the process proceeds to step S111 for the next pixel. Return and repeat the above process.
When the detection for all the pixels to be detected is completed (S117, YES), next, a dirt flag image (when all the pixels have been inspected, all the pixels are flagged ( (ON) or not standing (OFF) data is given, so that this is regarded as a binary image (herein referred to as a stain flag image) (S118), and the final stain is based on the stain flag image. A determination is made (S119). The processing is terminated after this dirt determination.

FIG. 8 is a flowchart showing in detail step S119 in FIG.
In this embodiment, there are three methods 1 to 3 described below in the present embodiment for determining a stain in the stain flag image in step S119.
About the first method Specifically, the pixel with the dirt flag set is counted among the pixels of the entire image, and when the numerical value exceeds a predetermined threshold, it is determined that the camera lens is dirty.
FIG. 8A is a flowchart showing the first method.
The number of pixels for which the stain flag is set is counted by the processing shown in the flowchart of FIG. 7 (S119 (1)), and when the count number exceeds the threshold value (S119 (2) YES), it is determined that the pixel is dirty (S119). (3)). On the other hand, when the count number does not exceed the threshold value, it is determined that there is no dirt and the process is terminated.

About the second method In this method, the target image is divided into several blocks, and each block is determined by the same method as the first method.
According to this method, it is possible to know which area of the image is soiled, that is, which side of the lens is soiled.
In this case, for example, when a block exceeding a predetermined number in all blocks is dirty, it may be determined that the entire block is dirty.
FIG. 8B is a flow diagram of this second method.
In this method, first, the dirt flag image is divided into a plurality of blocks (S119 (4)), and then the number of pixels in which the dirt flag is set is counted among the pixels in each block (S119 (5)). When the number of pixels exceeds a predetermined threshold (S119 (6) YES), it is determined that the block is dirty (S119 (7)). Here, it is determined whether or not this stain determination processing has been performed for all blocks for each block (S119 (8)). If not (S119 (8) NO), the process returns to step S119 (5). When the above process is repeated and the determination is completed for all the blocks (S119 (8) YES), it is determined whether or not the entire stain determination is performed (S119 (9)), and when the determination is made (S119 (9)). YES), the blocks determined to be dirty blocks are counted (S119 (10)), and when the count exceeds a predetermined value (threshold value) (S119 (11) YES), the entire image is dirty. (S119 (12)) and the stain determination process is terminated.
In step S119 (9), when the overall stain determination is not performed (S119 (9), NO), the stain determination process is terminated.

About the third method In this method, the dirt flag image is regarded as a binary image by turning on and off the dirt flag, the dirt pixels are labeled, and the label size of the labeled pixels is a predetermined fixed size. If it exceeds, it is determined that there is dirt. Labeling is performed by a conventionally known method.
Since this method is based on labeled pixels, it can be determined which part of the image is dirty.
FIG. 8C is a flow diagram of this third method.
First, labeling is performed based on the dirt flag of the pixels constituting the dirt flag image (S119 (13)), and then the area of the pixels that are labeled and grouped for each label is calculated (S119 (14)). When the area exceeds a predetermined threshold value (S119 (15) YES), it is determined that the image is dirty (S119 (16)), and the process is terminated, and the threshold value is not exceeded in step S119 (15)). If so (S119 (15), NO), the process ends.
The above processing is performed on the left and right images, that is, the L image and the R image.

In the present embodiment, for a certain pixel, the current and past evaluation values are compared, and if they are equal or the current is lower, it is better to match. On the contrary, if the current one is higher, there is a possibility that it is dirty.
If there is a pixel whose distance cannot be obtained in the current image, there is a possibility of contamination, and a contamination flag is set for that pixel (initial value is contamination flag OFF).
When determining the evaluation value, a margin may be provided for the purpose of noise reduction or the like.

  The processing for detecting dirt on an optical system such as a lens based on a distance image based on parallax and an evaluation value image has been described above. For example, when dirt (for example, water droplets) attached to a lens is large, dirt is attached. The part suddenly loses its parallax (the evaluation value also changes), and the attached part changes to a completely different brightness compared to the past, so the change in brightness exceeds a predetermined threshold It is also possible to determine that the lens is dirty.

The above explanation has been given taking the case of processing using a stereo camera (parallel viewing) as an example, but any camera can be used as long as it is a multi-camera configured to obtain parallax. It is.

It is the perspective view seen from the camera upper side which shows the state which image | photographs the foreign material E set | placed indoors with two cameras. The left and right photographed images taken with the camera are shown. It is the figure which rearranged and showed L image and R image on the XY coordinate up and down. The graph of an evaluation value is shown. FIG. 6 is a diagram showing a past (background) image of one image and an image in a state where dirt is attached to the current imaging lens, which are divided into an input image, a distance image, and an evaluation value (SAD) image, respectively. is there. It is a block diagram which shows the stain | pollution | contamination detection apparatus containing an image processing apparatus. It is a flowchart explaining the processing flow for the stain | pollution | contamination detection of a lens using an image processing apparatus. It is the flowchart which showed step S119 in FIG. 7 in detail.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image processing apparatus, 12 ... Interface, 14 ... CPU, 141 ... Stereo calculating part, 142 ... Other calculating parts, 143 ... Background update part, 144 ... Dirt Comparison determination unit for detection, 16 ... RAM, 161 ... acquired image data storage unit, 162 ... intermediate result data storage unit, 163 ... background data storage unit, 164 ... various setting value storage units , 20... Camera, 30... Input device, 40.

Claims (11)

A dirt detection device comprising a photographing device for photographing a stereo image of an object and an image processing device for processing the photographed stereo image,
The image processing apparatus includes means for obtaining parallax from a stereo image, means for creating a distance image and an evaluation value image based on the obtained parallax, and a distance image and an evaluation value background as a reference distance image and evaluation value image, respectively. Means for storing as an image, and comparing the distance background image and the evaluation value background image with the distance image and the evaluation value image created from the stereo image photographed at an arbitrary time point, the contamination of the optical system of the photographing device A dirt detection device comprising a detecting means. In the dirt detector according to claim 1,
The means for creating the distance image and the evaluation value image adds the distance information obtained for each pixel of the acquired image to create the distance image, and adds the evaluation value information obtained for each pixel of the acquired image. And producing the evaluation value image. In the dirt detector according to claim 1 or 2,
The means for detecting contamination of the optical system of the photographing apparatus compares the evaluation value background image and the evaluation value image, and the evaluation value is higher than that of the evaluation value background image in the pixels of the evaluation value image. If the distance information about the pixel cannot be obtained on condition that there is a large pixel, the pixel is designated as a dirty pixel, and the optical system of the photographing apparatus is cleaned based on the dirty pixel. A dirt detecting device characterized by determining. In the dirt detector according to claim 1 or 2,
The means for detecting contamination of the optical system of the photographing device compares the evaluation value background image and the evaluation value image, and the evaluation value is higher than that of the evaluation value background image in the pixels of the evaluation value image. When the distance information of the pixel is equal to that of the distance background image on the condition that there is a large pixel, there is means for designating the pixel as a dirty pixel, and the contamination of the optical system of the photographing apparatus based on the dirty pixel A dirt detection apparatus characterized by determining whether In the dirt detection device according to claim 3 or 4,
The means for detecting contamination of the optical system of the photographing apparatus has means for counting the number of pixels designated as the dirty pixel, and the optical system of the photographing apparatus becomes dirty when the counted value exceeds a predetermined threshold value. It is determined that there is a dirt detection device. In the dirt detection device according to claim 3 or 4,
It means for detecting the contamination of the optical system of the imaging apparatus includes means for counting the number of pixels specified as the dirty picture element image in each image block is divided into a plurality of number of dirty pixels counted a predetermined threshold value And a means for determining whether or not the image block has exceeded , wherein when the counted number of dirty pixels exceeds a predetermined threshold, the image block is determined to be dirty. In the dirt detection device according to claim 3 or 4,
Means for detecting dirt in the optical system of the photographing apparatus includes means for labeling the dirty pixels designated as dirty pixels, means for calculating the area of the labeled pixel region, and the calculated labeling area and a predetermined threshold. And a means for comparing, and as a result of the comparison, when the labeling area exceeds the threshold, it is determined that the labeling region is contaminated. A dirt detection method for detecting dirt on a photographing device based on a stereo image taken of a subject,
A step of capturing a stereo image of an object, and obtaining a parallax stereo image, a step of creating a based-out distance image and evaluation value image to parallax calculated, a primary distance image and evaluation value image, respectively The step of storing as a distance background image and an evaluation value background image, and comparing the distance background image and the evaluation value background image with the distance image and the evaluation value image created from a stereo image photographed at an arbitrary time point, And a step of detecting dirt on the optical system of the apparatus. In the dirt detection method according to claim 8,
The step of creating the distance image and the evaluation value image includes adding the distance information obtained for each pixel of the acquired image to create the distance image, and adding the evaluation value information obtained for each pixel of the acquired image. Then, a stain detection method, which is a step of creating the evaluation value image. In the dirt detection method according to claim 8 or 9,
The step of detecting dirt on the optical system of the photographing apparatus compares the evaluation value background image with the evaluation value image, and the pixels of the evaluation value image have a larger evaluation value than that of the evaluation value background image. When the distance of the pixel cannot be obtained on the condition that the pixel exists, the method includes a step of designating the pixel as a dirty pixel and a step of determining the contamination of the optical system of the photographing apparatus based on the dirty pixel. A method for detecting dirt. In the dirt detection method according to claim 8 or 9,
The step of detecting dirt on the optical system of the photographing apparatus compares the evaluation value background image with an evaluation value image created at an arbitrary time, and the evaluation value is applied to the pixels of the evaluation value image. When the distance information of the pixel is equal to the distance information of the distance background image on the condition that there is a pixel larger than that of the pixel, the step of designating the pixel as a dirty pixel and the imaging device based on the designated pixel And a step of determining contamination of the optical system.

Images