KR100861514B1 - Photographing apparatus and method - Google Patents

Abstract

The imaging device according to the present invention includes a CCD device 110 for continuously photographing a plurality of images by changing the aperture 106 of the lens unit 102 with an aperture, and selecting one image from a plurality of images. A provisional particle candidate extraction unit 151 for determining the presence or absence of particles of the element, a provisional particle position extraction unit 152 for acquiring the position of the pixels of the imaging device that is believed to be attached to the dust, and one image of a plurality of images The dust recognition unit 154 and the dust recognition unit recognized by the dust recognition unit in which the dust is attached to the position of the pixel when the signal level is different from the predetermined level by comparing the signal level of the position of the pixel in another image different from There is provided an image pickup device including a particle display portion 156 for indicating an attachment position.

Description

Imaging Apparatus and Imaging Method {Photographing apparatus and method}

1 is an explanatory diagram for explaining an imaging device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram illustrating the configuration of the imaging device of FIG. 1. FIG.

3 is an explanatory diagram illustrating the configuration of the CPU of FIG. 1.

4 is a flowchart illustrating an image pickup method according to an embodiment of the present invention.

FIG. 5 is an explanatory diagram showing an example of a program diagram used to calculate the aperture value and the shutter speed when the aperture is opened in the imaging method of FIG. 4.

6 is an explanatory diagram showing an example of a program diagram used to calculate the aperture value and the shutter speed when the aperture is cut in the imaging method of FIG. 4.

FIG. 7A is an explanatory diagram showing an example of a picked-up image obtained by bouncing down an iris when dust is attached to an image pickup device according to an embodiment of the present invention. FIG.

7B is an explanatory diagram showing an example of a picked-up image in which an aperture is photographed when dust is attached to an image pickup device according to an embodiment of the present invention.

8 is an explanatory diagram illustrating an arrangement of color filters according to an embodiment of the present invention.

FIG. 9 is a flowchart for describing the provisional particle candidate extraction process of the imaging method shown in FIG. 4.

FIG. 10 is a flowchart for explaining a process of extracting provisional dust of the imaging method shown in FIG. 4.

FIG. 11 is a flowchart for explaining a particle recognition process of the imaging method shown in FIG. 4.

12 is a flowchart for explaining the flow of the dust cleaning mode in the imaging method shown in FIG. 4.

Fig. 13 is an explanatory diagram showing a display example of a picked-up image when the image is picked up with the dust attached according to the embodiment of the present invention.

14 is an explanatory diagram showing an example of an image in the case of displaying a particle position according to an embodiment of the present invention.

15 is an explanatory diagram illustrating an outline of another embodiment of the present invention.

FIG. 16 is an explanatory diagram showing an example of inspection results when dust is not attached to an image pickup device according to another embodiment of the present invention. FIG.

17 is an explanatory diagram showing an example of test results when dust is attached to an image pickup device according to another embodiment of the present invention.

* Explanation of symbols for main parts of the drawings

100: imaging device 132: RAM

102 and 202: lens unit 151: provisional dust candidate extracting unit

104: lens cap 152: provisional dust position extraction unit

106: aperture 154: dust recognition unit

110: CCD element 156: particle display unit

128: CPU 158: proper AE calculation unit

130: ROM 200: inspection device

The present invention relates to an imaging apparatus and an imaging method, and more particularly, to improve the detection accuracy of dust attached to an imaging element, and to specify the attachment position of the dust, an imaging apparatus and an imaging apparatus in which dust removal can be easily performed. It is about a method.

Background Art [0002] Imaging apparatuses for photographing still and moving images using solid-state imaging devices such as charge coupled devices (CCD) devices and complementary metal oxide semiconductor (CMOS) devices are widely used. In the imaging device using a CCD device or a CMOS device, a large number of imaging devices are available on the market in which a lens according to a situation can be used by attaching and detaching a lens like a conventional silver salt camera.

However, in the lens interchangeable imaging device which can attach and detach a lens in this way, there is a possibility that dust or dust (hereinafter, referred to collectively as "dust") adheres to the imaging element when the lens is replaced. When photographing in a state where dust is attached on the image pickup device, light is not incident on the pixel of the image pickup device on which the particle is attached, and thus a portion of the image is darkened, resulting in a problem in the image.

When shooting, it is difficult to recognize the presence of attached dust, and it is often realized for the first time when an image is displayed on a PC or the like after shooting or when printed with a printing device such as a printer. In addition, the dust adhered to the image pickup device is often of a small size which is invisible, and it is difficult to determine whether it is attached to a part of the image pickup device or the like. In addition, inadvertent work to remove the adhered dust may damage the low pass filter provided in front of the image pickup device.

Therefore, a technique is known in which a photographer can easily remove the dust by analyzing the captured image and detecting the presence or absence of the dust attached to the imaging device (see Patent Document 1 and Imaging Method 3). .

-Patent Document 1: Japanese Patent Publication No. 3461482

-Patent Document 2: Japanese Patent Laid-Open No. 2001-157087

-Patent Document 3: Japanese Patent Publication No. 2005-341381

However, in the method shown in Patent Document 1, the so-called pixel defects which lost not only the dust adhered to the imaging element but also the photoelectric conversion function are recognized as dust like dust adhered. Unlike the adhered dust, the image defect is meaningless even if it is removed, and there is a problem of damaging the imaging element (low pass filter) by performing the removal operation.

Moreover, with the method shown in patent document 2, since the position of the particle displayed on a picked-up image differs from the position of the particle actually attached on the imaging element surface, it is difficult to discriminate the location of the particle, and the particle on the imaging element is attached. There was a problem that the removal operation was performed at a position different from the existing position, thereby causing damage to the imaging element (low pass filter) even in this method.

Moreover, the method shown in patent document 3 judges that part as a sticking part of the dust when the same part of the image | photographed from the picked-up image is continuously lower than a predetermined signal level, and in this method, a continuous opportunity is rare. Since it is expected that there is a problem in the detection accuracy of dust, and also as patent document 1, there existed a problem that it was difficult to distinguish from a pixel defect.

SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus and an imaging method that can improve the detection accuracy of dust attached to an imaging element and simplify the removal work of the dust by specifying the attachment position of the dust.

An imaging device according to an aspect of the present invention sequentially includes a lens unit including an aperture mechanism for adjusting the amount of incident light from a subject, and an image of a subject imaged on an imaging surface with the lens unit interposed successively as image data. And a second image having the largest aperture value among the plurality of images so that imaging of a plurality of images is performed by changing an aperture value of the aperture mechanism part during continuous imaging by the imaging unit. And an aperture control unit for adjusting an amount of incident light from a subject so that the first image having an aperture smaller than the second image is included in the images, and an image of the second image among the image data sequentially output from the imaging unit. The imaging surface is compared by comparing the value of each pixel of the data with the representative value of the second image representing the image data of the second image. If the foreign matter attachment prediction unit extracts a pixel predicted to have a foreign matter attached to the tentative dust candidate, and the potential dust candidates extracted from the foreign matter attachment prediction unit are in successive positions, the tentative dust candidate is extracted as the tentative dust, and the same tentative dust is extracted. A pixel position detector that calculates the number of pixels included in a region considered to be a value; and a pixel value of the first image and the value of the pixel of the first image at the same position as the pixel regarded as the provisional dust in the second image by the pixel position detector Comparing the representative value of the first image representing the image data to calculate the number of pixels regarded as the tentative dust in the first image, compared with the number of pixels of the tentative dust of the second image, the difference in the number of pixels more than a predetermined value If present, the foreign matter determination unit for determining the provisional dust as dust having foreign matter attached thereto is included.

According to the above configuration, the lens unit receives the image light from the subject, and the imaging unit sequentially outputs the image of the subject formed on the image forming surface with the lens unit therebetween as image data continuously using the imaging element, The aperture control unit adjusts the light amount of incident light from the subject by changing the aperture value of the aperture mechanism unit included in the lens unit, and the foreign matter attaching prediction unit is adapted to at least two or more image data having different aperture values among the image data sequentially output from the imaging unit. On the basis of this, it is predicted whether foreign matter is attached on the imaging surface. As a result, it is possible to improve the detection accuracy of dust attached to the image pickup device by analyzing at least two or more image data having different aperture values continuously photographed.

delete

According to the above configuration, the pixel position detection unit detects the pixel position on the image plane where the adhesion of the foreign matter is predicted based on the prediction result of the foreign matter adhesion prediction unit, and the foreign matter determination unit compares the luminance level of each image data to a predetermined threshold value. When the above difference occurs, it is determined that the foreign matter is attached to the pixel position. As a result, it is possible to improve the detection accuracy of dust attached to the image pickup device by comparing the luminance levels of the respective image data.

The foreign matter adhesion prediction unit calculates, as a representative value of the second image, the average value of the luminance levels of the entire image data of the second image for each component color of the color filter included in the imaging unit, When the luminance level of the pixel is less than the predetermined value than the average value of the second image, the pixel may be extracted as a potential particle candidate.
In addition, the foreign matter determination unit calculates the average value of the luminance levels of the entire image data of the first image for each of the constituent colors of the color filter included in the imaging unit as the representative value of the first image, and the pixel position detection unit calculates the second image. If the luminance level of the pixel of the first image at the same position as the pixel regarded as the tentative dust is less than the predetermined value than the average value of the first image, the pixel may be extracted as the tentative dust of the first image.

According to the above configuration, the foreign matter determination unit determines that the foreign matter is attached to the pixel position when the luminance levels of the pixel positions on the image plane in each image data are all less than the average value. As a result, since successive pixels satisfying the predetermined condition are extracted as the pixels to which foreign matters are temporarily attached, it is possible to distinguish between pixels with foreign matters and simple pixel defects.

delete

According to the above configuration, the foreign matter adhesion prediction unit selects image data obtained from incident light when the aperture value of the aperture mechanism unit is the largest among the image data sequentially output from the imaging unit. As a result, the detection accuracy of the pixel to which the foreign matter adheres can be improved.

The imaging device may further include a display unit that displays a determination result by the foreign matter determination unit.

According to the said structure, a display part displays the determination result by a foreign material determination means. As a result, the removal of the dust can be simplified by specifying the result of the adhesion of the foreign matter.

The display unit may display the attachment position of the foreign material corresponding to the imaging area on the imaging surface.

According to the said structure, a display part displays the attachment position of a foreign material corresponding to the imaging area | region on the said imaging surface. As a result, since the displayed position and the attachment position of the foreign matter actually correspond, the removal work of the foreign matter can be simplified more.

The imaging method according to one aspect of the present invention continuously uses an lens unit including an aperture mechanism portion for adjusting the amount of incident light from an object, and continuously forms an image of a subject imaged on an imaging surface of an imaging portion with a lens unit therebetween. An imaging step of sequentially outputting as image data and a plurality of images in which the aperture value of the aperture mechanism unit is changed during continuous imaging in the imaging step so that imaging of a plurality of images is performed, An aperture control step of adjusting the amount of incident light from the subject so that the second image and the first image having a smaller aperture value than the second image are included in the images, and a second of image data sequentially output in the imaging step. Representative value of the second image representing the value of each pixel of the image data of the image representing the image data of the second image In comparison, the foreign matter adhesion prediction step of extracting the pixels predicted to have foreign matters attached to the image forming surface as potential dust candidates, and the potential dust candidates are extracted as provisional dusts when the potential dust candidates extracted in the foreign matter adhesion prediction step are in successive positions. And a pixel position detection step of calculating the number of pixels included in the site considered to be the same tentative dust, and a pixel of the first image at the same position as the pixel regarded as the tentative dust in the second image by the pixel position detection step. Compares the value of and the representative value of the first image representing the image data of the first image to calculate the number of pixels regarded as the provisional dust in the first image, and compares the number of pixels of the provisional dust of the second image with a predetermined value. If there is a difference in the number of pixels above the value, the foreign matter determination step of judging the provisional dust as dust having foreign matter attached thereto. It includes.

According to the method, the imaging step sequentially outputs the image of the subject imaged on the image forming surface successively as image data, with the lens unit incident on the image light from the subject interposed, and the aperture control step being included in the lens unit. The amount of incident light from the subject is adjusted by changing the aperture value of the aperture mechanism unit, and the foreign matter adhesion prediction step is performed based on at least two or more image data having different aperture values among the image data sequentially output from the imaging unit. Predict whether it is attached. As a result, it is possible to improve the detection accuracy of dust attached to the image pickup device by analyzing at least two or more image data having different aperture values continuously photographed.

delete

According to the method, the pixel position detection step detects the pixel position on the image plane where the adhesion of the foreign matter is predicted based on the prediction result in the foreign matter adhesion prediction step, and the foreign matter determination step compares the luminance level of each image data. In the case where a difference of more than a predetermined threshold occurs, it is determined that foreign matter is attached to the pixel position. As a result, it is possible to improve the detection accuracy of the dust attached to the image pickup device by comparing the luminance levels of the image data photographed continuously.

The foreign matter adhesion prediction step calculates the average value of the luminance levels of the entire image data of the second image for each component color of the color filter included in the image pickup unit as the representative value of the second image, When the luminance level of each pixel is less than a predetermined value than the average value of the second image, the pixel may be extracted as a potential particle candidate.
In the foreign material determination step, the average value of the luminance levels of the entire image data of the first image for each of the constituent colors of the color filter included in the image pickup unit is calculated as the representative value of the first image, and the second position is determined by the pixel position detection step. When the luminance level of the pixel of the first image at the same position as the pixel regarded as the provisional dust in the image is less than the predetermined value than the average value of the first image, the pixel may be extracted as the provisional dust of the first image.

According to the method, in the foreign matter determination step, it is determined that the foreign matter is attached to the pixel position when the luminance levels of the pixel positions on the image plane in each image data are all less than the average value. As a result, since successive pixels satisfying the predetermined condition are extracted as pixels to which foreign matter is temporarily attached, it is possible to distinguish between pixels with foreign matter attached and simple pixel defects.

delete

According to the above method, in the foreign matter adhesion prediction step, the image data obtained from the incident light when the aperture value of the aperture mechanism part is the largest among the image data sequentially output in the imaging step is selected. As a result, the detection accuracy of the pixel to which the foreign matter adheres can be improved.

The imaging method may further include a display step of displaying a determination result by the foreign matter determination step. According to the method, the display step displays the determination result by the foreign matter determination step. As a result, the removal work of the dust can be simplified by specifying the attachment position of the dust.

In the displaying step, the attachment position of the foreign matter may be displayed corresponding to the imaging area on the imaging surface.

According to the above method, in the display step, the attachment position of the foreign matter is displayed corresponding to the imaging area on the imaging surface. As a result, since the displayed position and the actual attachment position of the foreign matter correspond, the removal work of the foreign matter can be simplified more.

EMBODIMENT OF THE INVENTION Hereinafter, the structure and operation | movement of the imaging device and imaging method which concern on this invention are demonstrated in detail through embodiments of an accompanying drawing. In the specification and drawings, components having substantially the same functional configuration will be omitted by the same reference numerals.

(One embodiment)

First, an image capturing apparatus and an image capturing method according to an embodiment of the present invention will be described.

1 is an explanatory diagram for explaining an imaging device according to an embodiment of the present invention. Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to FIG. 1.

The imaging device 100 is an imaging device according to an embodiment of the present invention. The imaging device 100 has a structure in which the lens unit 102 can be mounted. Further, when inspecting dust using the imaging method according to the embodiment of the present invention, the imaging device 100 may attach the light diffusing lens cap 104 to the lens unit 102. By using the light diffusing lens cap 104, light incident from a subject can be diffused to detect dust more effectively.

In the above, the imaging device which concerns on one Example of this invention of FIG. 1 was demonstrated. Next, the structure of the imaging device which concerns on one Embodiment of this invention is demonstrated.

FIG. 2 is an explanatory diagram illustrating the configuration of the imaging device of FIG. 1. FIG. Hereinafter, the structure of the imaging device which concerns on an Example using FIG. 2 is demonstrated.

As shown in FIG. 2, the imaging device 100 according to the embodiment of the present invention includes a CCD element 110, an amplifier integrated CDS (correlated double sampling) circuit 112, and an A / D converter 114. And an image input controller 116, an image signal processor 118, a compression processor 120, a video encoder 122, an image display unit 124, a timing generator 126, and a CPU (central). processing unit (128), read only memory (ROM) 130, random access memory (RAM) 132, media controller 134, recording media 136, motor driver 138, , A shutter button 140, and an operation unit 142.

In addition, the lens unit 102 to which the imaging device 100 according to the embodiment of the present invention may be mounted includes an aperture 106 and a driving device 108.

The aperture 106 adjusts the amount of light incident on the CCD element 110 when the image is captured. The control of the diaphragm 106 is performed by the drive device 108. In the present embodiment, the range of the aperture can be adjusted between AV (aperture value) 3 to AV7 (F2.8 to F11). In the present invention, the range of the aperture is not limited thereto.

The CCD element 110 is an example of the imaging element of the present invention and is an element for converting light incident from the lens unit 102 into an electrical signal. In this embodiment, the time for extracting the electric signal by controlling the incident light is controlled by the electronic shutter. The time for extracting the electric signal may be adjusted by controlling the incident light by using the mechanical shutter. In one embodiment of the present invention, the lens unit 102 and the CCD element 110 are formed of an imaging unit.

In the present embodiment, the CCD device 110 is used as the imaging device, but the present invention is not limited to the above example, and instead of the CCD device 110, a CMOS (complementary metal oxide semiconductor) device may be used. You can also use other image sensors. Since the CMOS element can convert the video light of the subject into an electrical signal at a higher speed than the CCD element, it is possible to shorten the time from photographing the subject to displaying the image on the image display unit 124.

The CDS circuit 112 is a circuit in which a CDS circuit, which is a kind of sampling circuit, which removes noise of an electrical signal output from the CCD element 110, and an amplifier that amplifies the electrical signal after removing the noise, are integrated. In the present embodiment, the imaging device 100 is configured by using a circuit in which the CDS circuit and the amplifier are integrated, but the CDS circuit and the amplifier may be configured as circuits, respectively.

The A / D converter 114 converts the electric signal generated in the CCD element 110 into a digital signal to generate raw data (image data) of the image.

The image input controller 116 controls the input of the raw data (image data) of the image generated by the A / D converter 114 to the RAM 132.

The image signal processing unit 118 adjusts the gain correction and the white balance of the amount of light with respect to the raw data of the image generated by the A / D converter 114.

The compression processing unit 120 performs a compression process of compressing the image generated by the A / D converter 114 into image data of an appropriate format. The compressed format of the image may be a reversible format or an irreversible format. Examples of suitable formats may be converted to the JPEG (joint photographic experts group) format or the JPEG 2000 format.

The image display unit 124 performs live view display before photographing operation, display of photographed images, and the like. In addition to displaying an image, various setting screens of the imaging device 100 may be displayed. The display of the image data and various kinds of information of the imaging device 100 on the image display unit 124 is performed with the video encoder 122 interposed. An LCD (liquid crystal display) may be used as the image display unit 124.

The timing generator 126 inputs a timing signal to the CCD element 110. The shutter speed is determined by the timing signal from the timing generator 126. That is, the driving of the CCD element 110 is controlled by the timing signal from the timing generator 126, and the electric light that is the basis of the image data is generated by injecting the image light from the subject within the time when the CCD element 110 is driven. Is generated.

The CPU 128 executes a signal system command to the CCD element 110, the CDS circuit 112, or the like, or executes an operation system command for the operation of the operation unit 142. In this embodiment, only one CPU is included, but the instructions of the signal system and the instruction of the operation system may be executed by each other CPU.

The ROM 130 stores a program for controlling the imaging device 100.

The RAM 132 temporarily stores the captured image. Reading and writing of the image to the RAM 132 are controlled by the image input controller 116. As the RAM 132, a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a ferroelectric RAM (FeRAM), and a magnetoresistive RAM (MRAM) may be used.

The recording medium 136 records the photographed image. Input / output to the recording medium 136 is controlled by the media controller 134. As the recording medium 136, a memory card which is a card-type storage device for recording data in a flash memory can be used.

The motor driver 138 controls the drive device 108 for operating the aperture 106 included in the lens unit 102. The amount of light of the subject is adjusted by operating the aperture 106 with the motor driver 138 interposed therebetween.

The shutter button 140 is for photographing operation. The shutter button 140 combines the subject in a half-pressed state (hereinafter, referred to as a half-pressed state of the shutter button 140, also referred to as an "S1 state"), and a fully pressed state (hereinafter, The object in the fully depressed state of the shutter button 140 is also referred to as " S2 state "

In the operation unit 142, a member for performing an operation of the imaging apparatus 100 or performing various settings during shooting is disposed. On the member disposed on the operation unit 142, a power button, a cross key for selecting a shooting mode or a shooting drive mode, a selection button, and the like are disposed.

In the above, the structure of the imaging device which concerns on one Example of this invention was demonstrated using FIG. Next, the configuration of the CPU of the imaging device according to the embodiment of the present invention will be described.

3 is an explanatory diagram illustrating the configuration of the CPU of FIG. 1. Hereinafter, the structure of the CPU of the imaging device which concerns on one Embodiment of this invention using FIG. 3 is demonstrated.

As shown in FIG. 3, the CPU 128 according to an embodiment of the present invention includes a provisional particle candidate extraction unit 151, a provisional particle position extraction unit 152, a particle recognition unit 154, The particle | grain display part 156 and the appropriate AE calculation part 158 are comprised.

The temporary dust candidate extracting unit 151 is an example of the foreign matter adhesion predicting unit according to the present invention. The potential dust candidate extracting unit 151 analyzes the photographed image to determine whether dust is attached on the CCD element 110, and if the dust is attached, determines the detected portion. It is extracted as a potential dust candidate.

The provisional particle position extraction unit 152 is an example of the pixel position detection unit of the present invention, and acquires the pixel position on the CCD element 110 in which the particles are predicted to adhere to the captured image. A method of extracting a portion of the photographed image that is considered to be dust will be described later.

The dust recognition part 154 recognizes the site | part which a particle actually adheres to in the site | part predicted that the particle extracted by the provisional particle position extraction part 152 is affixed as an example of the foreign material determination part of this invention. The method of recognizing the site | part to which dirt adheres is mentioned later.

The particle | grain display part 156 makes the image display part 124 display the site | part by which the particle | grains recognized by the particle | grain recognition part 154 adhere. When the portion where the dust is attached is displayed on the image display portion 124, the CCD element 110 may be displayed at a position corresponding to the position where the dust is actually attached.

The appropriate AE calculation unit 158 performs automatic exposure to the imaging device 100 when photographing a subject to obtain an EV (exposure value) value. The appropriate exposure time and the shutter speed are determined based on the acquired EV value. As for the EV value, when the aperture value is F1 and the shutter speed is 1 second, the amount of light to obtain appropriate exposure is set to EV = 0, and the EV value is changed by changing the aperture value and the shutter speed. The EV value can be obtained by EV = log ₂ (F ² / T) with F as the aperture value and T as the shutter speed. Therefore, with the same aperture value, the higher the shutter speed, the higher the EV value. At the same shutter speed, the larger the aperture value, the higher the EV value.

In the above, the CPU which concerns on one Embodiment of this invention was demonstrated using FIG. Next, an imaging method according to an embodiment of the present invention will be described.

4 is a flowchart illustrating an image pickup method according to an embodiment of the present invention. Hereinafter, the imaging method which concerns on one Embodiment of this invention using FIG. 4 is demonstrated.

First, the operation unit 142 is operated to set the shooting mode to the mote shooting mode. The dust adhered to the CCD element 110 is detected by photographing by setting to the dust photographing mode. If the shooting mode is set to the mote shooting mode, the CPU 128 determines whether the shutter button 140 is pressed or not (step S110). If the shutter button 140 is not pressed, the process returns to step S110 to repeat the pressing determination process of the shutter button 140. On the other hand, if the shutter button 140 is pressed, the appropriate AE calculation unit 158 performs metering processing to determine the appropriate AE calculation unit 158 shutter speed and aperture to obtain an exposure value EV0 for proper exposure (step). S120). As a result of the photometric processing, it is determined whether or not the exposure value is within an appropriate exposure interlocking range in the particle photographing mode (step S130).

In the present embodiment, the range of the shutter speed for not affecting the detection of dust is predetermined in the range of TV5 to TV10 (1/32 second to 1/1024 second). In the present embodiment, the minimum aperture fluctuation range of the diaphragm 106 is AV2. Therefore, in the present embodiment, the exposure interlocking range in the mote shooting mode is 10 (AV3 + AV2 + TV5) to 16 (AV8-AV2 + TV10), and the range of the exposure value EV0 at which the appropriate exposure is obtained in the mote shooting mode. From the exposure interlocking range of 10≤EV0≤16. Therefore, in the case of EV0 < 10 or EV0 > 16, the image display unit 124 indicates that the exposure is out of range because the dust is not photographed (step S240), and the processing ends. The range of the shutter speed and the minimum fluctuation range of the aperture are not limited to the above examples.

As a result of performing the metering process, when the exposure value EV0 is in the range 10 ≦ EV0 ≦ 16, the aperture value and the shutter speed are obtained. The aperture value and the shutter speed are obtained in order to perform shooting when the aperture is opened for the first time (hereafter, shooting when the aperture is opened) is also referred to as "first shooting" (step S140). The aperture value and the shutter speed are calculated using the program diagram. 5 is an explanatory diagram showing an example of a program diagram used to calculate the aperture value and shutter speed when the aperture according to the present embodiment is opened. For example, when the exposure value EV0 is 10 as a result of the metering processing, the shutter speed TV is set to 7 (1/128 seconds) and the aperture value AV is set to 3 (F2.8). . As a result of the photometric processing, when the exposure value can take a plurality of shutter speeds and aperture values, the black circled portion of the program diagram shown in FIG. 5 is employed. For example, when the exposure value EV0 is 13, the shutter speed TV is set to 10 (1/1024 seconds) and the aperture value AV is set to 3 (F2.8).

After the calculation of the shutter speed and the aperture value is finished, first shooting is performed using the calculated shutter speed and the aperture value (step S150). An image photographed by the first photographing (hereinafter, also referred to as an "first image" is photographed by the first photographing) is temporarily stored in the RAM 132.

When the first photographing is completed, the aperture value and the shutter speed are obtained to perform photographing when the aperture is squeezed (hereinafter, referred to as "second photographing" when the aperture is squeezed) (step S160). The aperture value and the shutter speed are calculated using the program diagram as in the first photographing. FIG. 6 is an explanatory diagram showing an example of a program diagram used to calculate the aperture value and the shutter speed when the aperture is cut. For example, as a result of performing the metering process, when the exposure value EV0 is 10, the shutter speed TV is set to 5 (1/32 seconds) and the aperture value AV is set to 5 (F5.6). As in the case of the first photographing, as a result of performing the photometric processing, when the exposure value can take a plurality of shutter speeds and aperture values, the black circled portion of the program diagram shown in FIG. 6 is employed. For example, when the exposure value EV0 is 12, the shutter speed TV is set to 7 (1/128 seconds) and the aperture value AV to 5 (F5.6).

After calculating the shutter speed and the aperture value, the second photographing is performed using the calculated shutter speed and the aperture value (step S170). An image captured by the second photographing (hereinafter, also referred to as an "second image" is photographed by the second photographing) is also temporarily stored in the RAM 132 similarly to the first image.

The reason why the photographing is performed when the aperture is opened or when it is narrowed in this manner is to facilitate the determination of the dust attached to the image pickup device. 7A and 7B are explanatory diagrams showing an example of a picked-up image in the case of photographing when dust is attached to the image pickup device according to the embodiment of the present invention. FIG. 7A shows an example of the image when the aperture is taken by shooting, and FIG. 7B shows an example of the image when the aperture is taken by shooting.

Shooting with the aperture open will blur other than the focused subject, while shooting with the aperture compressed will surely illuminate the subject. Similarly, in the case where dust is attached to the image pickup device, if the image is captured while the aperture is compressed, the particle attached to the image pickup device is reliably reflected as shown in FIG. 7A, and when the image is captured while the aperture is opened, FIG. As shown in (b), dust adhered to the image pickup device is blurred. In the present embodiment, such two types of images are used to specify the place where the particles are attached.

Here, the first shooting and the second shooting are performed on the same object. The object to be photographed preferably has a bright color such as white or gray in order to improve the extraction accuracy of the dust.

When the second shooting is finished, in the second image picked up by the tentative dust position extraction unit 152, a tentative dust extraction process of acquiring the pixel position where dust is expected to be attached is performed (step S180). Hereinafter, the provisional dust extraction process according to an embodiment of the present invention will be described in detail.

In one embodiment of the present invention, the extraction of the temporary dust is performed by analyzing the second image photographed in step S170. A process of first analyzing the second image to extract the tentative dust candidate is performed, and then a process of extracting the tentative dust candidate at a continuous position as the tentative dust from the extraction result of the tentative dust candidate. In one embodiment of the present invention, the extraction of the provisional dust candidate is performed by the provisional dust candidate extraction unit 151 and the extraction of the provisional dust is performed by the provisional dust location extraction unit 152.

8 is an explanatory diagram for explaining the arrangement of the color filters of the CCD element 110 according to the embodiment of the present invention. As shown in Fig. 8, the color filter of the CCD element 110 according to the embodiment of the present invention has three primary colors of red (R), green (G), and blue (B), and the arrangement is Bayer. Bayer pattern.

Fig. 9 is a flowchart for explaining the provisional particle candidate extraction process according to the embodiment of the present invention, and the provisional particle candidate extraction process (step S175) in Fig. 4 will be described in detail. Hereinafter, the tentative dust candidate extraction process according to an embodiment of the present invention will be described with reference to FIG. 9.

First, by analyzing the second image taken by shooting the diaphragm 106, the presence or absence of dust and the position where the dust is considered to be attached are found. First, the average signal level of each color of the color filter of the CCD element 110 is calculated (step S210). The average value of each calculated RGB color is called Rav, Gav, and Bav, respectively.

When calculation of the average value of each color is completed, the signal level of each pixel and the intra-pixel average signal level of each color corresponding to each computed pixel are then compared in all the pixels on the CCD element 110. When the difference between the signal level and the average signal level of each pixel is equal to or greater than a predetermined value, it is determined that particles are attached to the pixel, and extracted as a potential particle candidate.

In one embodiment of the present invention, first, the temporary dust position extractor 152 performs initialization of the variable (step S220). In this embodiment, the provisional particle position extraction unit 152 initializes two types of variables, n and count. The variable n indicates the number of pixels, and the variable count indicates the number of pixels considered as potential dust candidates. The value of n ranges from 0 to (all pixels number-1). Next, it is determined whether or not the value of n is less than the number of all the pixels of the CCD element 110 (step S230). If the value of n is less than the number of all the pixels of the CCD element 110, it is judged whether or not the pixel is tentative dust (step S240). Specifically, for example, it is determined whether the signal level of the pixel corresponding to the value of n is less than a predetermined value than the average signal level in the pixels of each color corresponding to the pixel.

In one embodiment of the present invention, whether the signal level of each pixel is less than a predetermined value than the average signal level in the pixel corresponding to the color of the pixel is expressed by the following equations (1) to (3). To judge. When the color of the color filter is R, it is determined by Equation 1, when the color of the color filter is B, by Equation 2, and when the color of the color filter is G, by Equation 3.

Rav-Pn> Rst

Bav-Pn> Bst

Gav-Pn> Gst

In an embodiment of the present invention, when the signal level of the pixel is less than a predetermined value than the average signal level in the pixel corresponding to the color of the pixel, the pixel is extracted as a potential particle candidate. In this embodiment, the provisional particle position extracting unit 152 records in the array variable DD [count] in order to record the value of the variable n at that time (step S250). Then, the value of variable count is increased by one (step S260), and the value of variable n is increased by one (step S270). On the other hand, if the signal level of the pixel is not less than the predetermined value than the average signal level in the pixel corresponding to the color of the pixel, the process moves directly to step S270.

The provisional particle position extraction unit 152 performs the above series of processes on all pixels, and if the value of n is not less than the number of all pixels of the CCD element 110 in step S230, that is, the value of n If the number is the same as all the pixels, the process ends.

In the above, the extraction process of the tentative particle candidate which concerns on one Example of this invention was demonstrated. Next, the extraction process of the temporary dust concerning one Example of this invention is demonstrated.

FIG. 10 is a flowchart for explaining a process for extracting provisional dust according to an embodiment of the present invention, and the provisional dust extraction process (step S180) in FIG. 4 will be described in detail. Hereinafter, the extraction process of the temporary dust concerning one Example of this invention is demonstrated using FIG.

In the tentative dust extraction process according to the embodiment of the present invention, the tentative dust position extraction unit 152 initially initializes the variables k, cc, count, group, and cg (step S310). Subsequently, it is determined whether or not the value of the variable k is less than the number of acquired potential candidates (step S312). If the value of the variable k is less than the number of acquired temporary candidates, it is further determined whether the number of variables cc is less than the number of acquired potential candidates (step S314).

The tentative dust position extracting unit 152 determines whether the tentative dust candidate exists at a position adjacent to the pixel extracted as the tentative dust candidate in the process of extracting the tentative dust candidate when the number of variables cc is less than the number of the tentative dust candidates acquired. . In one embodiment of the present invention, the determination of whether there is a temporary dust candidate at an adjacent position is performed specifically by whether the array variables DD [k] and DD [cc] are adjacent (step S316). If the array variables DD [k] and DD [cc] are not adjacent to each other, the tentative dust position extraction unit 152 increases the value of the variable cc by one (step S318), and the tentative dust candidate for which the number of variables cc is once again acquired. After determining whether or not the number is less than (step S314), it is processed whether or not the array variables DD [k] and DD [cc] are adjacent to each other.

When there is a potential dust candidate at a position adjacent to the pixel extracted as the potential dust candidate, the potential dust position extracting unit 152 records the pixel as the temporary dust. In one embodiment of the present invention, specifically, when the array variables DD [k] and DD [cc] are adjacent to each other, the value of the variable k at that time is recorded as the array variable DUST [count] (step S320). .

The tentative dust position extraction unit 152 further records the recorded pixels in the unit of tentative dust when the pixels are recorded as the tentative dust. Specifically, in one embodiment of the present invention, the provisional particle position extraction unit 152 determines whether the value of the variable k is greater than the value of the variable cc (step S332). As a result of the determination, when the value of the variable k is larger than the value of the variable cc, the provisional particle position extraction unit 152 stores the value of the CD [cc] in the array variable CD [count] (step S324), If the value is equal to or less than the value of the variable cc, the value of the variable group is stored in the array variable CD [count], and the value of the variable group is increased by one (step S326). The provisional particle position extraction unit 152 increases the value of the variable count by one (step S328), and also initializes the value of the variable cc to 0 by increasing the value of the variable k by one (step S330). The provisional particle position extraction unit 152 increases the value of the variable k by one and initializes the value of the variable cc to 0, and returns to step S312 to determine whether or not the value of the variable k is smaller than the number of potential mote candidates.

In step S314, if the number of variable cc is equal to or greater than the acquired number of potential dust candidates, the process directly goes to step S330 to increase the value of variable k by one to initialize the value of variable cc to zero.

In this way, the provisional particle position extraction unit 152 repeats the process from step S312 to step S330 until the value of the variable k becomes equal to or more than the number of provisional dust candidates, thereby calculating the position and number of provisional particles. have.

When the tentative dust position extraction unit 152 detects the position and number of the tentative dust, the number of pixels included in the portion of the tentative dust position extraction unit 152 regarded as the same tentative dust is next calculated. In one embodiment of the present invention, the provisional dust position extraction unit 152 initially initializes the variables cc, cg, and k (step S332). When the variable is initialized, the provisional particle position extraction unit 152 determines whether or not the value of the variable cg is less than (group-1) (step S334). As a result of the determination by the provisional dust position extraction unit 152, if the value of the variable cg is less than (group-1), the value of the variable k in the provisional dust position extraction unit 152 is less than (count-1). It is determined whether or not it is (step S336). If the value of the variable k is less than (count-1), the provisional particle position extraction unit 152 determines whether or not the value of the array variable CD [k] matches cg (step S340). If the value of CD [k] coincides with cg, the provisional particle position extraction unit 152 increases the value of variable cc by one (step S342), and also increases the value of variable k by one (step S344). On the other hand, if the value of the array variable CD [k] does not match cg, the flow moves directly to step S344, and the provisional dust position extractor 152 increases the value of the variable k by one. After increasing the value of the variable k by one, the process returns to step S336 to determine whether the value of the variable k is less than (count-1) in the provisional particle position extraction unit 152.

If the value of the variable k is greater than or equal to (count-1) in step S336, the provisional particle position extraction unit 152 stores the value of the variable cc in the array variable ND0 [cg]. The value of the variable cc at this time is the number of pixels contained in one provisional particle. Therefore, the array variable ND0 [] stores the number of pixels in one tentative particle. The provisional particle position extraction unit 152 stores the value of the variable cc in the array variable ND0 [cg] and increases the value of the variable cg by one to initialize the value of the variable cc to 0 (step S338).

The provisional particle position extraction unit 152 repeats such a series of processes until the value of the variable cg is equal to or greater than (group-1), and in step S334 the value of the variable cg is greater than or equal to (group-1). If so, the process ends.

In the above, the extraction process of the tentative particle which concerns on one Example of this invention was demonstrated. The extraction process of the temporary dust in this invention is not limited to the said example.

When the tentative dust is extracted, an image different from the image used for extracting the tentative dust is analyzed using the extracted tentative dust information, and a dust recognition process is performed to recognize the extracted tentative dust as dust. In one embodiment of the present invention, the dust recognition unit 154 recognizes the dust by analyzing the first image photographed by the first photographing.

FIG. 11 is a flowchart illustrating the dust recognition process according to the embodiment of the present invention, and the dust recognition process (step S190) of FIG. 4 will be described in detail. Hereinafter, the dust recognition process which concerns on one Example of this invention is demonstrated using FIG.

First, the particle recognition unit 154 calculates an average signal level of each of the R, G, and B colors of the first image (step S410). The calculation method performed in the provisional particle candidate extraction process is used for calculation of an average signal level. When the average signal level of each of the R, G, and B colors is calculated, the particle recognition unit 154 initializes each variable used in the particle recognition process (step S412). In the particle recognition process according to the embodiment of the present invention, five types of variables k, c, g, n and count are prepared, initialized to 0 for variables k, c, and g, and variable count. Initializes the number of tentative dusts extracted by the tentative dust extraction process.

When the average signal level of each of the R, G, and B colors of the first image is calculated, the tentative recognition unit 154 tentatively grits the pixels of the first image at the same position as the pixels regarded as the tentative dust in the second image. On the basis of the above expressions (1) to (3) in the extraction process, a processing is performed to determine whether the signal level of the pixel is less than a predetermined value than the average signal level of each of the R, G, and B colors of the first image.

In one embodiment of the present invention, specifically, the dust recognition unit 154 first determines whether or not the value of the variable g is less than the value of (count-1) (step S414). If the value of the variable g is less than the value of (count-1), the particle recognition unit 154 checks whether the value of the variable g is equal to the value of the array variable CD [k] (step S416). . When the value of the variable g is less than the value of the array variable CD [k], the particle acknowledgment unit 154 stores the value of the array variable DUST [k] in the variable n (step S418). Then, it is checked whether or not the value of the variable k is less than (count-1) (step S420), and when the value of the variable k is less than (count-1), the dust recognition unit 154 further checks the array variable P2 [n]. It is determined whether 을 satisfies the condition of Equations 1 to 3 (step S422). Here, the array variable P2 [] indicates the signal level in each pixel of the first image captured by the first imaging. When the array variable P2 [n] satisfies Equations 1 to 3, the dust recognition unit 154 increases the value of the variable c by one (step S424), and increases the value of the variable k by one more. Subsequently (step S426), the process returns to the process of checking whether or not the value of the variable g of the step S416 is less than the value of the array variable CD [k].

In the above step S416, when the value of the variable g is different from the value of the array variable CD [k] and when the array variable P2 [n] does not satisfy the conditions shown in the equations (1) to (3) in the step S422, Moving to step S426, the dust recognition unit 154 increases the value of the variable k by one.

In addition, in step S420, when the value of the variable k is equal to or greater than (count-1), the value of the variable c is stored in the array variable nD1 [k] (step S428). Here, the array variable nD1 [] represents the number of pixels included in one provisional particle. Therefore, the array variable nD1 [] stores the number of pixels in one tentative particle. After increasing the value of the variable g by one (step S430), the process returns to step S414 to determine whether the value of the variable g is less than the value of (count-1).

The particle acknowledgment unit 154 repeats such a series of processes until the value of the variable g becomes equal to or greater than the value of (count-1), so that the number of pixels included in one provisional particle can be obtained. When the value of the variable g becomes equal to or greater than the value of (count-1) in step S414, the process of obtaining the number of pixels included in one provisional particle ends.

The number of pixels included in one temporary particle thus obtained is compared in the same provisional dust extraction portion between the first image and the second image. If there is a difference in the number of pixels of a predetermined value or more between the pixels included in the predetermined provisional dust of the second image and the pixels included in the provisional dust of the same portion of the first image, the particles adhere to the imaging device in that portion. The dust recognition unit 154 recognizes the provisional dust as the dust in consideration of the deterioration.

The dust recognition unit 154 stores the position of the dust on the recognized image when the temporary dust is recognized as dust. In one embodiment of the present invention, the X and Y coordinates of the pixel which is the center of the set of pixels recognized as one particle are obtained and stored as Xc and Yc, respectively.

In the above, the dust recognition process which concerns on one Example of this invention was demonstrated. The dust recognition process in this invention is not limited to the said example.

When it is admitted that dust adheres to the imaging element in the dust recognition process, the user of the imaging apparatus 100 is notified of the presence of the dust, and the dust cleaning mode is moved. In the dust cleaning mode for performing the dust removal operation, the dust may be forcibly moved after the dust recognition process is finished, or the user may select whether to move after the dust recognition process is completed. Hereinafter, the dust cleaning mode according to an embodiment of the present invention will be described.

12 is a flowchart for explaining the flow of the dust cleaning mode according to the embodiment of the present invention. Hereinafter, the flow of the dust cleaning mode according to the embodiment of the present invention will be described with reference to FIG. 12.

In the dust cleaning mode, the shutter in front of the CCD element 110 is completely opened for the first time to easily clean the dust (step S510). Subsequently, the position of the dust recognized by the dust recognition process is calculated (step S520). In the dust recognition process, since the position of the dust on the image was calculated, the dust was converted from the dust position on the image to the dust position on the imaging element surface, and the dust was present at a portion corresponding to the dust position on the converted imaging element surface. The attachment information of the contents is displayed (step S530). In one embodiment of the present invention, the particle display unit 156 performs the conversion to the particle position on the image pickup device surface and the adhesion information of the particle, and the adhesion information of the particle is displayed on the image display unit 124. The attachment information of the dust can be displayed, for example, by a symbol or by an icon or a character.

In one embodiment of the present invention, the conversion to the coordinates Xd and Yd of the image display unit 124 corresponding to the particle position on the image pickup element surface converted at the particle position on the image is performed by the following equation. Here, Hd is the number of display horizontal dots in the image display unit 124, Vd is the number of display vertical dots in the image display unit 124, H is the number of horizontal pixels in the image data, and V is the number of vertical pixels in the image data.

Xd = Xc × Hd / H

Yd = Vd-(Yc × Vd / V)

FIG. 13 shows an example when a photographed image is displayed on the image display unit 124 when the image is taken with the dust attached to the CCD element 110, which is an example of an image pickup device according to one embodiment of the present invention. 14 is an explanatory diagram showing an example of an image in a case where the particle position after position conversion according to an embodiment of the present invention is displayed on the image display unit 124. In an embodiment of the present invention, a case where dust is attached to three places of the CCD element 110 will be described.

As shown in FIG. 13, the particle | grains 160 are attached to three places in a picked-up image. On the imaging surface of the CCD element 110, the portion to which dust is actually attached becomes a position of vertical symmetry. Therefore, an icon or a character is repeatedly displayed as the attachment information of the particle on the image displayed on the image display unit 124 so as to be a mote position in the picked-up image and an up-down object.

In this way, the dust recognition unit 154 locates the dust position on the image pickup device surface, and the dust display unit 156 clearly indicates the position of the dust on the image display unit 124, whereby the user of the imaging apparatus 100 Blowers can be used to facilitate the removal of dust. The imaging apparatus 100 prompts the user to confirm whether or not the cleaning is completed (step S540), terminates the processing if the cleaning is completed, and continues to prompt the confirmation of whether or not the cleaning is completed. .

In the above, the flow of the dust cleaning mode which concerns on one Example of this invention was demonstrated using FIG.

The imaging method may be executed by a computer program. The CPU 128 can execute the imaging method by calling a computer program recorded in the ROM 130.

As described above, according to the imaging device and the imaging method according to the embodiment of the present invention, the detection accuracy of the dust adhered to the imaging element is improved by comparing the different images of the aperture and detecting the position of the dust. In addition, the detection accuracy of the dust is improved, thereby reducing the inadvertent removal work and the problem of damaging the optical low pass filter in front of the image pickup device. In addition, by displaying the attachment position of the dust, the user of the imaging device can easily perform the cleaning operation.

(Other embodiment)

In one embodiment of the present invention, an image pickup device and an image pickup method capable of facilitating a cleaning operation for a user of an image pickup device by detecting a location of a particle by comparing different images of an aperture and displaying an attachment location of a particle. Explained. In another embodiment of the present invention, an inspection apparatus and an imaging method for inspecting whether dust is attached to the imaging element by comparing different images of the aperture in the lens incorporating the imaging element will be described. .

15 is an explanatory diagram illustrating an outline of another embodiment of the present invention. Hereinafter, another Example of this invention is described using FIG.

As shown in FIG. 15, another embodiment of the present invention includes an imaging device 200 and a lens unit 202 incorporating an imaging device. The imaging device 200 is an example of the imaging device of the present invention. The imaging device may be a CCD device, a CMOS device, or other image sensor. Inside the lens unit 202, an aperture is built in as in the exemplary embodiment. The inspection apparatus 200 and the lens unit 202 are connected to the connection cable 204.

When inspecting dust using the imaging method according to another embodiment of the present invention, the light diffusing lens cap 104 may be attached to the lens unit 202 as in the embodiment. By using the light diffusing lens cap 104, since light incident from the subject is diffused, dust can be detected more effectively.

Since the structure of the test | inspection apparatus 200 which concerns on other Example of this invention has the same structure as the imaging device 100 which concerns on one Embodiment of this invention, detailed description is abbreviate | omitted. However, although the imaging device 100 according to the embodiment of the present invention has a configuration including the CCD device 110 which is an example of the imaging device, in another embodiment of the present invention, the imaging device is the lens unit 202. It is not necessary to include the imaging element in the inspection apparatus 200 because it is built in the.

In the above, the outline | summary of the other Example of this invention was demonstrated using FIG.

The imaging method according to another embodiment of the present invention recognizes the position of the dust by changing the aperture and taking another first image and the second image of the aperture, and analyzing the image as in the embodiment. And when it is recognized that dust adheres to the imaging element, the content that dust adheres to the display part 210 of the test | inspection apparatus 200 is displayed.

Here, the first image capture and the second image capture are performed on the same object as in the exemplary embodiment. In addition, the object to be photographed preferably has a bright color such as white or gray in order to improve the extraction accuracy of the dust.

16 and 17 are explanatory diagrams showing an example of inspection results by an inspection apparatus according to another embodiment of the present invention. FIG. 16 is an explanatory diagram showing an example of inspection results when dust is not attached to an imaging device provided in the lens unit 202 according to another embodiment of the present invention, and FIG. 17 is another embodiment of the present invention. It is explanatory drawing which shows an example of the test result in case particle | grains are adhered to the imaging element provided in the lens unit 202 which concerns on an Example.

As shown in FIG. 16, when dust is not attached to the image pickup device provided in the lens unit 202, the display unit 210 displays that the dust is not attached. As described above, when the dust is attached to the image pickup device provided in the lens unit 202, the inspection apparatus 200 displays the indication that the dust is attached to the display unit 210. When the dust is attached to the imaging device, the inspection apparatus 200 may display a place where the dust is attached as well as an indication that the dust is attached. By indicating the place where the dust is attached, it is possible to facilitate the removal work of the dust attached to the imaging element.

As described above, according to the inspection apparatus and the image pickup method according to another embodiment of the present invention, as in the embodiment, a lens having a built-in image pickup element captures another image of the aperture and compares the particles so that dust is attached to the image pickup element. You can check whether it is present.

Although the present invention has been described with reference to the above-described embodiments, these are merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention will be defined by the appended claims.

For example, in the above-described embodiments of the present invention and another embodiment, it was determined whether dust adhered to the image pickup device by capturing two images having different aperture values and then comparing the two images. It is not limited to the said example. For example, it is possible to determine whether dust adheres to the imaging device by photographing images having three or more different aperture values and comparing them.

The imaging device and imaging method of the present invention as described above can simplify the removal work of the dust by specifying the attachment position of the dust.

Claims (12)

A lens unit including an aperture mechanism unit for adjusting an amount of light incident from a subject, An imaging unit that sequentially outputs the image of the subject formed on the image forming surface with the lens unit therebetween as image data successively; During continuous imaging by the imaging unit, the aperture value of the aperture mechanism unit is changed to allow imaging of a plurality of images, and a second image having the largest aperture value among the plurality of images and an aperture smaller than the second image. An aperture control unit for adjusting an amount of light of incident light from the subject so that the first image of the value is included in the images; Foreign matter adheres to the image forming surface by comparing the value of each pixel of the image data of the second image among the image data sequentially output from the imaging unit with a representative value of the second image representing the image data of the second image. A foreign matter adhesion prediction unit that extracts a pixel predicted to be a potential dust candidate, A pixel position detection unit for extracting the tentative dust candidates as tentative dusts when the tentative dust candidates extracted by the foreign matter attachment predicting unit are in a continuous position, and calculating the number of pixels included in the site considered to be the same tentative dust; The pixel position detection unit compares a value of a pixel of the first image at the same position as a pixel regarded as a tentative dust in the second image with a representative value of a first image representing image data of the first image. If the number of pixels regarded as the provisional dust in the first image is calculated, and there is a difference in the number of pixels greater than or equal to a predetermined value compared to the number of the pixels in the provisional dust of the second image, the provisional dust is dust with foreign matter adhered thereto. An imaging device comprising a foreign matter determination unit that is determined as. The method of claim 1, The foreign matter adhesion prediction unit calculates an average value of luminance levels of the entire image data of the second image for each component color of the color filter included in the imaging unit as the representative value of the second image, and calculates the second image. And extracting the pixel as the tentative dust candidate when the luminance level of each pixel in the image data is less than a predetermined value than the average value of the second image. The method according to claim 1 or 2, The foreign matter determining unit calculates an average value of luminance levels of the entire image data of the first image for each component color of the color filter included in the imaging unit as the representative value of the first image, and calculates the pixel position detection unit. The pixel of the first image when the luminance level of the pixel of the first image at the same position as the pixel regarded as the tentative dust in the second image is less than a predetermined value than the average value of the first image. An imaging device which extracts by dust. delete The method of claim 1, And a display unit for displaying the determination result by the foreign matter determination unit. The method of claim 5, And the display unit displays the attachment position of the foreign matter corresponding to the imaging area on the imaging surface. An imaging step of sequentially outputting, as image data, an image of the subject imaged on an image forming section of an imaging unit with the lens unit interposed therebetween, using a lens unit including an aperture mechanism portion for adjusting the amount of incident light from the subject; Wow, During continuous imaging in the imaging step, the aperture value of the aperture mechanism unit is changed to allow imaging of a plurality of images, and a second image having the largest aperture value among the plurality of images and an aperture smaller than the second image. An aperture control step of adjusting an amount of light of incident light from the subject so that the first image of the value is included in the images; Foreign matter adheres to the image forming surface by comparing the value of each pixel of the image data of the second image among the image data sequentially output in the imaging step with a representative value of the second image representing the image data of the second image. A foreign matter adhesion prediction step of extracting a pixel predicted to be a potential dust candidate, A pixel position detection step of extracting the tentative dust candidates as tentative dusts if the tentative dust candidates extracted in the foreign matter adhesion prediction step are in a continuous position, and calculating the number of pixels included in the region regarded as the same tentative dust; By the pixel position detecting step, the value of the pixel of the first image at the same position as the pixel regarded as the tentative dust in the second image is compared with the representative value of the first image representing the image data of the first image. The number of pixels which are regarded as the provisional dust in the first image is calculated, and if there is a difference in the number of pixels more than a predetermined value compared to the number of the pixels in the provisional dust of the second image, foreign matter is attached to the provisional dust. An imaging method including a foreign matter determination step of determining as dust. The method of claim 7, wherein In the foreign matter adhesion prediction step, the average value of the luminance levels of the entire image data of the second image for each component color of the color filter included in the imaging unit is calculated as the representative value of the second image, and the second value is calculated. And extracting the pixel as the tentative dust candidate when the luminance level of each pixel in the image data of the image is less than a predetermined value than the average value of the second image. The method according to claim 7 or 8, In the foreign material determination step, the average value of the luminance levels of the entire image data of the first image for each of the constituent colors of the color filter included in the imaging unit is calculated as the representative value of the first image, and the pixel position detection step is performed. If the luminance level of the pixel of the first image at the same position as the pixel regarded as the tentative dust in the second image is less than a predetermined value than the average value of the first image, the pixel of the first image is The imaging method which extracts with a temporary particle. delete The method of claim 7, wherein And a display step of displaying the determination result by the foreign matter determination step. The method of claim 11, And the display step displays the attachment position of the foreign matter corresponding to the imaging area on the imaging surface.

Images