JP2009296362A - Imaging apparatus, control method thereof and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve quality of dust position information by: capturing the opportunity of detecting a dust position even while a user is not conscious of the detection; and acquiring the dust position information. <P>SOLUTION: An imaging apparatus includes: an imaging device for performing photoelectric conversion upon a subject image; an imaging section for producing an image signal from the imaging device; a foreign substance detection section for detecting, from the image signal captured by the imaging section, foreign substance information that is information about a position and a size of a foreign substance inside an imaging screen of the imaging section; and an auxiliary foreign substance information detection section which exposes the imaging device in an over-exposure state, causes the imaging section to produce a highlight-clipping image and detects auxiliary foreign substance information, which becomes a support of the foreign substance information, from the highlight-clipping image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for suppressing image quality deterioration due to foreign matter adhering to the surface of an optical low-pass filter or the like in an imaging apparatus using an imaging element such as a CCD or CMOS sensor.

  Conventionally, in a digital camera, when foreign objects such as dust adhere to the optical system (especially the surface of an optical element such as a low-pass filter disposed in front of the image sensor), the shadows of the foreign objects are shadowed on the captured image. There is a problem that the image quality is deteriorated.

  In order to solve this problem, a technique has been proposed in which a foreign substance adhering to an image sensor is removed by using a vibrating means (see Patent Document 1).

Further, the following method has been proposed. First, a uniform subject such as a white wall is photographed, and an image in which only foreign matter in the optical system is reflected is photographed. Then, dust correction data, which is information on the position and size of the foreign matter in the optical system, is created from the image, and a correction process is performed on the captured image based on the dust correction data to improve the image quality (see Patent Document 2). ).
JP 07-151946 A JP 2004-222231 A

  However, it is necessary for the user to consciously perform the information acquisition operation for the position information of dust. As an operation performed by the user, for example, a method of selecting a dust position detection mode from a menu screen and photographing a uniform white wall or the like can be considered. The dust position information is updated when such an operation is performed by the user. In other words, the dust position information is not updated unless the user consciously performs such an operation.

  On the other hand, since the position of dust changes with the passage of time, the frequency of dust position detection is better. Even when the user does not perform an operation for detecting the dust position, it is desirable to capture the dust position detection opportunity, acquire the dust position information, and maintain the freshness of the dust position information.

  Therefore, the present invention has been made in view of the above-described problems, and its purpose is to capture the opportunity of dust position detection even when the user is not conscious, acquire dust position information, and improve the quality of dust position information. It is to improve.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging device that photoelectrically converts a subject image, an imaging unit that generates an image signal from the imaging device, and an imaging unit. Foreign matter detection means for detecting foreign matter information, which is information on the position and size of the foreign matter within the imaging screen of the imaging means, from the obtained image signal, and exposing the imaging device to an overexposed state, and the imaging means And an auxiliary foreign matter information detecting means for generating auxiliary white matter information for assisting the foreign matter information from the whiteout image.

  An imaging apparatus control method according to the present invention is a method for controlling an imaging apparatus including an imaging element that performs photoelectric conversion of a subject image and an imaging unit that generates an image signal from the imaging element. A foreign matter detection step of detecting foreign matter information, which is information on the position and size of the foreign matter within the imaging screen of the imaging means, from the image signal obtained by the means, and exposing the imaging device to an overexposed state, And an auxiliary foreign matter information detecting step of causing the imaging means to generate a whiteout image and detecting auxiliary foreign matter information that assists the foreign matter information from the whiteout image.

  According to the present invention, even when the user does not perform an operation for dust position detection, an opportunity for dust position detection is captured, and the frequency of tracking the change in dust position that changes over time is increased. The freshness can be kept.

  Hereinafter, embodiments in which the present invention is applied to an interchangeable lens digital single-lens reflex camera will be described in detail with reference to the accompanying drawings.

(First embodiment)
In this embodiment, dust is detected by the camera body, dust correction data is attached to the image data, and image processing is performed from the image data by image processing using the dust correction data attached to the image data by an image processing apparatus outside the camera. A case where dust removal processing is performed will be described.

  FIG. 1 is a block diagram showing a circuit configuration of a lens-interchangeable single-lens reflex digital camera as an imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, a microcomputer 402 controls the operation of the entire camera, including processing of image data output from an image sensor (CCD in this embodiment) 418 that photoelectrically converts a subject image and display control of an LCD monitor device 417. To do.

  The switch (SW1) 405 is turned on when the release button 114 (see FIG. 2) is half-pressed. When the switch (SW1) 405 is turned on, the digital camera according to the present embodiment is in a shooting preparation state. The switch (SW2) 406 is turned on when the release button 114 is pressed to the end (fully pressed state), and when the switch (SW2) 406 is turned on, the digital camera of this embodiment starts a photographing operation.

  The lens control circuit 407 performs communication control with the photographic lens 200 (see FIG. 3) and drive control of the photographic lens 200 and drive control of the diaphragm blades during AF (autofocus).

  In FIG. 1, an external display control circuit 408 controls an external display device (OLC) 409 and a display device (not shown) in the viewfinder. The switch sense circuit 410 transmits signals of a large number of switches including the electronic dial 411 provided in the camera to the microcomputer 402.

  The strobe light emission dimming control circuit 412 is grounded via the X contact 412a and controls the external strobe. The distance measuring circuit 413 detects the defocus amount with respect to the subject for AF. The photometry circuit 414 measures the luminance of the subject.

  Note that the microcomputer 402 sets an optimal exposure condition and an overexposure condition based on the luminance measured by the photometry circuit 414.

  A shutter control circuit 415 controls the shutter, and performs appropriate exposure and exposure that causes whiteout on the image sensor.

  The LCD monitor device 417 and the backlight illumination device 416 constitute an image display device. The recording device 419 is, for example, a hard disk drive or a semiconductor memory card that can be attached to and detached from the camera body.

  Further, the microcomputer 402 includes an A / D converter 423, a buffer memory 424, an image processing circuit 425 including a DSP, and a pixel defect position memory 426 for storing that a predetermined pixel in the image sensor itself is defective. It is connected. Also connected is a dust position memory 427 that stores the pixel position in the image pickup device causing the image defect due to dust. Note that the pixel defect position memory 426 and the dust position memory 427 are preferably non-volatile memories. Further, the pixel defect position memory 426 and the dust position memory 427 may be stored using different addresses in the same memory space.

  Reference numeral 428 denotes a nonvolatile memory that stores programs executed by the microcomputer 402.

  FIG. 2 is a perspective view showing the external appearance of the digital camera according to the present embodiment, and FIG. 3 is a vertical sectional view of FIG.

  In FIG. 2, an eyepiece window 111 for finder observation, an AE (automatic exposure) lock button 112, an AF distance measuring point selection button 113, and a release button 114 for performing a photographing operation are provided on the upper part of the camera body 100. ing. An electronic dial 411, a shooting mode selection dial 117, and an external display device 409 are also provided. The electronic dial 411 is a multi-function signal input device that is used in combination with other operation buttons to input numerical values to the camera and to switch the shooting mode. The external display device 409 includes a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture, and shooting mode, and other information.

  Also, on the back of the camera main body 100, an LCD monitor device 417 for displaying captured images and various setting screens, a monitor switch 121 for turning on / off the LCD monitor device 417, a cross switch 116, and a menu button 124 is provided.

  The cross placement switch 116 has four buttons arranged vertically and horizontally and a SET button arranged in the center. The user instructs the camera to select and execute menu items displayed on the LCD monitor device 417. Used for.

  The menu button 124 is a button for causing the LCD monitor device 417 to display a menu screen for performing various camera settings. For example, when selecting and setting the shooting mode, after pressing the menu button 124, the user operates the up / down / left / right buttons of the cross switch 116 to select the desired mode, and the desired mode is selected. Setting is completed by pressing the SET button.

  The menu button 124 and the cross switch 116 are used for setting a normal dust acquisition mode, selecting whether to acquire dust information at the start of live view, or the like.

  Since the LCD monitor device 417 of the present embodiment is a transmissive type, an image cannot be visually recognized only by driving the LCD monitor device, and a backlight illumination device 416 is necessarily provided on the back surface thereof as shown in FIG. is there. As described above, the LCD monitor device 417 and the backlight illumination device 416 constitute an image display device.

  As shown in FIG. 3, the taking lens 200 that is an imaging optical system can be attached to and detached from the camera body 100 via a lens mount 202. In FIG. 3, 201 denotes a photographing optical axis, and 203 denotes a quick return mirror.

  The quick return mirror 203 is disposed in the photographing optical path, and a position for guiding the subject light from the photographing lens 200 to the finder optical system (referred to as a position shown in FIG. 3, an oblique position) and a position for retracting outside the photographing optical path (retraction position). Is called).

  In FIG. 3, the subject light guided from the quick return mirror 203 to the finder optical system is imaged on the focus plate 204. Reference numeral 205 denotes a condenser lens for improving the visibility of the finder, and 206 denotes a pentagonal roof prism, which guides the subject light that has passed through the focus plate 204 and the condenser lens 205 to an eyepiece lens 208 and a photometric sensor 207 for finder observation.

  Reference numerals 209 and 210 respectively denote a rear curtain and a front curtain that constitute a shutter, and an image sensor 418 that is a solid-state image sensor disposed behind the rear curtain 209 and the front curtain 210 is exposed for a necessary time. A captured image converted into an electrical signal for each pixel by the image sensor 418 is processed by the A / D converter 423, the image processing circuit 425, and the like, and is recorded in the recording device 419 as image data. An optical element 418a such as an optical low-pass filter is disposed in front of the image sensor 418, and shadows of foreign matters such as dust adhering to the surface of the optical element 418a appear in the photographed image.

  Under the control of the microcomputer 402, an electronic shutter is realized with the rear curtain 209 and the front curtain 210 open.

  The image sensor 418 is held on the printed circuit board 211. Behind this printed board 211, a display board 215, which is another printed board, is arranged. An LCD monitor device 417 and a backlight illumination device 416 are disposed on the opposite surface of the display substrate 215.

  Reference numeral 419 denotes a recording device for recording image data, and 217 denotes a battery (portable power supply). The recording device 419 and the battery 217 are detachable from the camera body.

(Normal garbage detection process)
FIG. 4 is a flowchart for explaining dust detection processing (detection processing of a pixel position where an image defect is caused by dust) in the digital camera according to the present embodiment. This processing is performed by the microcomputer 402 executing a dust detection processing program stored in the memory 428.

  The dust detection process is performed by capturing a dust detection image (foreign matter detection image). When performing dust detection processing, a camera is installed with the photographing optical axis 201 of the lens 200 facing a surface having a uniform color, such as an emission surface of a surface light source device or a white wall, and preparation for dust detection is performed.

  After the preparation is completed, for example, when the start of dust detection processing is instructed from the cross arrangement switch 116, the microcomputer 402 first sets the aperture. The image formation state of dust near the image sensor changes depending on the aperture value of the lens, and the position changes depending on the pupil position of the lens. Accordingly, the dust correction data (foreign matter information) needs to hold the aperture value and the pupil position of the lens in addition to the dust position and size.

  However, it is not always necessary to hold the aperture value in the dust correction data if it is determined in advance that the same aperture value is always used even when different lenses are used at the stage of creating dust correction data. Similarly, by using a light unit or permitting the use of only a specific lens for the pupil position, it is not always necessary to hold the pupil position in the dust correction data. In other words, at the stage of creating dust correction data, if you want to allow multiple lenses to use or change the aperture value to narrow down appropriately, it is necessary to keep the aperture value and lens pupil position at the time of detection in the dust correction data It can be said that there is. Here, the pupil position refers to the distance from the imaging plane (focal plane) of the exit pupil.

  In the normal dust detection process, the user shoots a uniform subject such as a white wall and designates F22 as a limit at which a stain on the wall does not appear (step S21).

  Next, the microcomputer 402 causes the lens control circuit 407 to perform aperture blade control of the photographing lens 200, and sets the aperture to the aperture value designated in step S21 (step S22). Further, the focus position is set to infinity (step S23).

  When the aperture value and focus position of the photographing lens are set, photographing in the dust detection mode is executed (step S24). Details of the imaging processing routine performed in step S24 will be described later with reference to FIG. The captured image data is stored in the buffer memory 424.

  When shooting is completed, the aperture value and lens pupil position at the time of shooting are acquired (step S25). Data corresponding to each pixel of the captured image stored in the buffer memory 424 is called to the image processing circuit 425 (step S26). The image processing circuit 425 performs the processing shown in FIG. 8, and acquires the position and size of the pixel where dust is present in the imaging screen (step S27). The position and size of the pixel where the dust exists in step S27 and the aperture value and lens pupil position information acquired in step S25 are registered in the dust position memory 427 (step S28).

  Here, in step S28, the position of the defective pixel (pixel defect) from the time of manufacture recorded in the pixel defect position memory 426 in advance is compared with the position of the read pixel data to check whether the pixel defect is present. . Only the area determined not to be due to the pixel defect may be registered in the dust position memory 427.

  An example of the data format of dust correction data stored in the dust position memory 427 is shown in FIG. As shown in FIG. 7, the dust correction data stores lens information and dust position and size information when the detection image is captured. This dust correction data is added to the image together with the shooting time information of the image data during normal shooting, and used in dust removal processing described later.

  Specifically, the actual aperture value (F value) at the time of capturing the detected image and the lens pupil position at that time are stored as lens information at the time of capturing the detected image. Subsequently, the number of detected dust areas (integer value) is stored in the storage area, and subsequently, individual specific dust area parameters are repeatedly stored by the number of dust areas. The parameter of the dust area is a set of three numerical values: a dust radius (for example, 2 bytes), a center x coordinate (for example, 2 bytes) in the effective image area, and a similar y coordinate (for example, 2 bytes).

  When the dust correction data size is limited due to the size of the dust position memory 427 or the like, the data is stored with priority from the head of the dust area obtained in step S27. This is because in the dust area acquisition routine in step S27, the dust areas are sorted in the order of noticeable dust, as will be described later.

(Trash area acquisition routine)
Next, details of the dust region acquisition routine performed in step S27 will be described with reference to FIGS.

  As shown in FIG. 9, the called image data is developed on a memory and processed in units of predetermined blocks. This is to cope with peripheral dimming caused by lens and sensor characteristics. Peripheral dimming is a phenomenon in which the luminance at the peripheral portion is lower than that at the central portion of the lens, and it is known that it is reduced to some extent by reducing the aperture of the lens. However, there is a problem that even if the aperture is reduced, if the dust position is determined based on a predetermined threshold value for the photographed image, dust in the peripheral portion cannot be detected accurately depending on the lens. Therefore, the image is divided into blocks to reduce the influence of peripheral dimming.

  When the blocks are simply divided, there is a problem that the detection result of the dust straddling the blocks is shifted when the threshold values are different between the blocks. Therefore, the blocks are overlapped, and pixels determined to be dust in any of the blocks constituting the overlap region are treated as dust regions.

  The dust region determination in the block is performed according to the processing flow shown in FIG. First, the maximum luminance Lmax and average luminance Lave in the block are calculated, and the threshold value T1 in the block is calculated using the following equation.

T1 = Lave × 0.6 + Lmax × 0.4
Next, pixels that do not exceed the threshold value are defined as dust pixels (step S61), and isolated regions constituted by dust pixels are each defined as one dust region di (i = 0, 1,..., N) (step S62). . As shown in FIG. 10, for each dust region, the maximum value Xmax and the minimum value Xmin of the horizontal coordinate of the pixels constituting the dust region, the maximum value Ymax and the minimum value Ymin of the vertical coordinate are obtained, and the dust region di A radius ri representing the size of is calculated by the following equation (step S63).

ri = √ [{(Xmax−Xmin) / 2} ² + {(Ymax−Ymin) / 2} ² ]
The relationship between Xmax, Xmin, Ymax, Ymin and ri is shown in FIG.

  Thereafter, in step S64, an average luminance value for each dust region is calculated.

  In some cases, the data size of the dust correction data is limited due to the limitation due to the size of the dust position memory 427. In order to cope with such a case, the dust position information is sorted according to the size and the average luminance value of the dust region (step S65). In this embodiment, sorting is performed in descending order of ri. When ri is equal, the sort is performed in ascending order of the average luminance value. In this way, it is possible to preferentially register noticeable dust in the dust correction data. The sorted dust area is Di, and the radius of the dust area Di is Ri.

  If there is a dust area larger than a predetermined size, it may be excluded from sorting and placed at the end of the sorted dust area list. This is because a large dust region may lower the image quality if interpolation processing is performed later, and is preferably treated as the lowest priority for correction.

(Auxiliary garbage data image generation routine)
Next, generation of a whiteout image for auxiliary dust data detection (auxiliary foreign matter information detection) will be described with reference to FIG.

  In this embodiment, in addition to the user intentionally performing dust detection on the camera, the auxiliary dust detection image is linked to the live view operation between this intentional dust detection and the next intentional dust detection. (Auxiliary dust data image) is acquired and dust detection processing is performed. This dust detection process is referred to as auxiliary dust data acquisition process. By performing auxiliary dust data acquisition processing, the frequency of dust detection processing is increased, and dust correction data can be updated more quickly.

  The auxiliary dust data image generation processing replaces the processing from step S21 to step S24 in FIG. In other words, the reason for starting the processing is that the user performs a photographing operation such as a white wall in the normal dust correction data image generation, whereas the live view starts in the generation of the auxiliary dust data image. Different.

  First, when the live view is started (step S71), the quick return mirror 203 is mirrored up (step S72). The rear curtain 209 and the front curtain 210 constituting the shutter are opened (step S73), and the image sensor 418 is activated (step S74). Here, it is checked whether dust detection is set. If it is set, the process proceeds to step S76, and it is determined whether or not whitening (shooting with overexposure) is possible. If the setting for proceeding to dust detection has not been made, the process proceeds to step S81 to set an appropriate exposure. The setting of dust detection can be set by the user operating the menu button 124 and the cross placement switch 116. The set information is recorded in a memory (not shown) in the camera.

  In step S76, it is determined whether or not whitening is possible. For the determination, the microcomputer 402 uses the luminance measured by the photometry circuit 414. The higher the brightness, the more the whiteout can be achieved with a shorter exposure time. On the other hand, if the brightness is too low, the exposure time must be long. Based on the exposure time, the microcomputer 402 determines whether it is in an appropriate state for acquiring a whiteout image. That is, it is determined whether or not whitening is possible.

  Note that, since the shadow of the foreign matter attached to the optical element 418a is captured even when overexposed, the whiteout image is acquired here, and an image in which only the foreign matter attached to the optical element 418a is captured by photographing the whiteout image. It is because it can obtain. Then, dust position data can be acquired from this whiteout image.

  If it is determined in step S76 that whitening is possible, the process proceeds to step S77, and the diaphragm blades are squeezed to capture dust. Here, it is assumed that F22 is set. If it is determined that whitening cannot be performed, the process proceeds to step S81, and the exposure value is set to an appropriate value.

  When the aperture is set to F22 in step S77, defocusing is performed (step S78), and exposure is performed so that whiteout occurs with the electronic shutter by the microcomputer 402 (step S79), and an image for detecting auxiliary dust data is obtained (step S80). ).

  When the auxiliary dust data detection image is obtained, the exposure value is reset to an appropriate value (step S81), and normal frame capturing is started (step S82). Finally, live view display is started on the LCD monitor device 417 (step S83).

  As described above, the auxiliary dust data image generation process is performed after the live view is started and before the live view image is displayed on the LCD monitor device 417. Indistinguishable from. That is, auxiliary garbage data can be generated without the user being aware of it.

  Further, it goes without saying that the user may be aware of the auxiliary dust data image generation processing. That is, if a message indicating that data for dust correction is being acquired is displayed on the LCD monitor device 417 immediately after the live view is started and the user consciously shoots a uniform subject, High quality auxiliary dust data images can be obtained.

  For the acquired auxiliary dust data image, the dust region acquisition routine shown in FIG. 8 is performed in the same manner as when generating normal dust correction data, and auxiliary dust data is generated and registered in the dust position memory 427. To do.

(Difference generation routine)
FIG. 6 is a flowchart showing a process for preventing correction data from overlapping between normal dust correction data and auxiliary dust data.

  First, one piece of dust information obtained in the auxiliary dust data acquisition step is selected (step S41). The normal dust correction data is searched for a match with the dust information selected in step S41 (step S42). If a match is found as a result of the search, the dust information of the matching auxiliary dust data is deleted from the auxiliary dust data candidates. That is, it is not stored as auxiliary dust data (step S44). The next dust information of the auxiliary dust data is selected (step S45), and the same processing is repeated for all dust information.

  Note that the dust information match determination may be performed by determining contact between surfaces derived from coordinates and radii, or by determining a threshold value for the distance between the coordinates. For example, if the coordinate distance threshold is 5 pixels or less, the dust information coordinates of the auxiliary dust data are (X, Y) = (100 pixels, 100 pixels), and the normal dust correction data is (X, Y) = If the information (100 pixels, 105 pixels) exists, the dust information of the auxiliary dust data is not stored as auxiliary dust data.

(Imaging processing routine)
Next, details of the imaging processing routine performed in step S24 of FIG. 4 will be described using the flowchart shown in FIG. The processing is performed by the microcomputer 402 executing an imaging processing program stored in the memory 428.

  When this imaging processing routine is executed, in step S201, the microcomputer 402 operates the quick return mirror 203 shown in FIG. 3, performs so-called mirror up, and retracts the quick return mirror 203 outside the imaging optical path.

  Next, charge accumulation in the image sensor is started in step S202, and in the next step S203, exposure is performed by running the shutter front curtain 210 and rear curtain 209 shown in FIG. In step S204, the charge accumulation of the image sensor is terminated, and in the next step S205, the image signal is read from the image sensor, and the image data processed by the A / D converter 423 and the image processing circuit 425 is temporarily stored in the buffer memory 424. .

  When reading of all image signals from the image sensor is completed in the next step S206, the quick return mirror 203 is mirrored down in step S207, the quick return mirror is returned to the oblique position, and a series of imaging operations is completed.

  In step S208, it is determined whether normal shooting or dust detection image shooting is performed. In normal shooting, the process proceeds to step S209, and the dust correction data shown in FIG. Record at 419.

  Specifically, for example, association can be realized by adding dust correction data to the Exif area, which is the header area of an image file in which camera setting values at the time of shooting are recorded. Alternatively, the dust correction data can be independently recorded as a file, and the association can be realized by recording only the link information to the dust correction data file in the image data. However, if the image file and the dust correction data file are recorded separately, the link relationship may be lost when the image file is moved. Therefore, it is desirable to hold the dust correction data integrally with the image data.

(Dust removal process)
Next, the flow of dust removal processing will be described. The dust removal process is performed not on the digital camera body but on a separately prepared image processing apparatus.

  FIG. 12 is a diagram showing an outline of the system configuration of the image processing apparatus.

  The CPU 1001 controls the operation of the entire system, and executes a program stored in the primary storage unit 1002. The primary storage unit 1002 is mainly a memory, and reads and stores programs stored in the secondary storage unit 1003. The secondary storage unit 1003 corresponds to, for example, a hard disk. Generally, the capacity of the primary storage unit is smaller than the capacity of the secondary storage unit, and programs and data that cannot be stored in the primary storage unit are stored in the secondary storage unit. Data that must be stored for a long time is also stored in the secondary storage unit. In the present embodiment, the program is stored in the secondary storage unit 1003, read into the primary storage unit 1002 when the program is executed, and the CPU 1001 performs an execution process.

  Examples of the input device 1004 include a mouse and keyboard used for system control, as well as a card reader, scanner, film scanner, and the like necessary for inputting image data. Examples of the output device 1005 include a monitor and a printer. Various other configurations of the apparatus configuration method are conceivable, but the description thereof is omitted because it is not the main point of the present invention.

  The image processing apparatus is equipped with an operating system capable of executing a plurality of programs in parallel, and an operator can operate a program operating on the apparatus using a GUI.

  FIG. 13 is a diagram showing a GUI (Graphical User Interface) of an image editing program in the image processing apparatus. The window includes a close button 1100 and a title bar 1101. When the close button 1100 is pressed, the program is terminated. When a correction target image is specified by dragging and dropping the file into the image display area 1102 and the correction target image is determined, the file name is displayed in the title bar 1101 and then the target image is displayed as Fit in the image display area 1102. To do. When the execute button 1103 is pressed, dust removal processing described later is executed, and the processed image is displayed in the image display area 1102. When a step execution button 1104 is pressed, a dust removal process, which will be described later, is executed, and the processed image is displayed in the image display area 1102 when processing is completed for all dust areas. When a save button 1105 is pressed, the processed image is saved.

  FIG. 14 shows the flow of dust removal processing in the image processing apparatus.

  First, normal photographed image data attached with dust correction data from a recording device 419 removed from the digital camera or from the digital camera is taken into the image processing device and stored in the primary storage unit 1002 or the secondary storage unit 1003 (step). S90).

  Next, dust correction data added to the photographed image in step S209 is extracted from the normally photographed image data (image to be subjected to dust removal processing) (step S91). Further, it is confirmed whether or not auxiliary dust data exists (step S92), and if it exists, it is read (step S93) and combined with normal dust correction data (step S94). For the combination, the coordinates, size, aperture value, and pupil position of normal dust correction data and auxiliary dust data may be held as a simple data string, or normal dust correction data and auxiliary dust data are held separately. May be.

  Next, from the dust correction data extracted in steps S91 to S94, the coordinate sequence Di (i = 1, 2,... N), the radius sequence Ri (i = 1, 2,..., N), the aperture value f1, and the lens pupil position. L1 is obtained (step S95). Here, Ri is the size of the dust at the coordinate Di calculated in step S65 of FIG. In step S96, the aperture value f2 and the lens pupil position L2 at the time of shooting of the normally shot image are acquired, and in step S97, Di is converted by the following equation. Here, d is a distance from the image center to the coordinate Di, and H is a distance between the surface of the image sensor 418 and dust. The coordinate Di ′ after conversion and the radius Ri ′ after conversion are defined by the following equation, for example.

Di ′ (x, y) = (L2 × (L1−H) × d / ((L2−H) × L1)) × Di (x, y)
Ri ′ = (Ri × f1 / f2 + 3) (1)
The unit here is a pixel, and “+3” for Ri ′ is a margin amount.

  In step S98, dust in the area indicated by the coordinates Di 'and the radius Ri' is detected, and interpolation processing is applied as necessary. Details of the interpolation processing will be described later. In step S99, it is determined whether or not dust removal processing has been applied to all coordinates. If processing has been completed for all coordinates, the processing is terminated; otherwise, the processing returns to step S98.

(Interpolation routine)
Next, details of dust region interpolation processing will be described. A flowchart showing the flow of the interpolation routine is shown in FIG. First, in step S1201, dust region determination is performed. The dust area is an area that satisfies all of the following conditions.
(1) Next, using the average luminance Yave and the maximum luminance Ymax of the pixels included in the center coordinate Di ′ and radius Ri ′ (Di ′, Ri ′ obtained in Expression (1)) calculated in step S97 of FIG. An area darker than the threshold value T2 obtained by the equation.

T2 = Yave × 0.6 + Ymax × 0.4
(2) A region not in contact with the circle having the center coordinate Di ′ and the radius Ri ′.
(3) A region in which the radius value calculated by the same method as in step S63 in FIG. 8 is not less than l1 pixels and less than l2 pixels with respect to the isolated region constituted by the low-luminance pixels selected in (1).
(4) A region including the center coordinates Di of the circle.

  In this embodiment, l1 is 3 pixels and l2 is 30 pixels. In this way, only an isolated small area can be handled as a dust area. If the lens pupil position cannot be obtained accurately, the condition (4) may have a width. For example, if the region of interest includes coordinates within a range of ± 3 pixels in the X direction and the Y direction from the coordinate Di, a condition such as determining as a dust region can be considered.

  If there is such an area in step S1202, the process advances to step S1203 to perform dust area interpolation. If there is no such area, the process ends. The dust region interpolation processing executed in step S1203 is performed by a known defect region interpolation method. Known defect area interpolation methods include, for example, pattern replacement disclosed in JP-A-2001-223894. In Japanese Patent Laid-Open No. 2001-223894, a defective region is specified using infrared light. In this embodiment, the dust region detected in step S1201 is treated as a defective region, and the dust region is treated as a normal pixel by pattern replacement. Interpolate with. For pixels that are not filled by pattern replacement, p normal pixels are selected in the order closest to the interpolation target pixel from the image data after pattern interpolation, and interpolation is performed using the average color.

  By attaching dust correction data to an image in this way, there is an advantage that it is not necessary to be aware of the correspondence between dust correction image data and captured image data. In addition, since the dust correction data is compact data composed of position, size, and conversion data (aperture value, distance information on the lens pupil position), the captured image data size does not become extremely large. Further, by performing interpolation processing only on the region including the pixel specified by the dust correction data, the probability of erroneous detection can be greatly reduced.

  Furthermore, in the present embodiment, auxiliary dust data is acquired between the time when dust correction data is acquired and the time when the next dust correction data is acquired, so the frequency of tracking the change in dust position that changes over time is set. And the freshness of the dust position information can be maintained.

(Second Embodiment)
In the first embodiment, when the auxiliary dust data is acquired, the difference from the dust correction data is obtained and saved and used later. However, the auxiliary dust data may be used as verification of the dust correction data. That is, the degree of difference between auxiliary dust data and dust correction data can be measured to know the freshness of dust correction data.

  FIG. 16 is a flowchart for explaining an example of a method for verifying dust correction data using auxiliary dust data.

  First, a counter that counts the difference between dust correction data and auxiliary dust data is reset to zero (step S101). Next, dust correction data is loaded (step S102), and one piece of dust coordinate information is extracted from the data (step S103). If the dust coordinates of the auxiliary dust data are not included in the distance within the dust radius from the extracted coordinates in step S104, the process proceeds to step S105, and the counter for counting the difference is counted up. If the dust coordinates of the auxiliary dust data are included, the process proceeds to step S106. In step S106, it is determined whether all the dust correction data has been investigated. If all the data has been investigated, the process proceeds to step S107, and the difference count is stored in the memory 428 or the like so that it can be referred to. . If the survey is not completed, the process returns to step S103, and one next dust correction data is extracted.

  The difference data generated in this way can be used to warn the user when the difference count exceeds a certain value, which contributes to improving the freshness of the dust correction data.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above. In the first and second embodiments, the garbage The dust position information is generated from the position information generation image and stored. Needless to say, the dust position information generation image can be stored and the dust position information can be extracted and used by a PC or the like. Yes.

  Furthermore, it goes without saying that the dust position information generation image may be stored as a moving image frame, as well as being stored as a still image.

1 is a block diagram showing a circuit configuration of a lens interchangeable digital single-lens reflex camera as an imaging apparatus according to a first embodiment of the present invention. 1 is an external perspective view of a digital camera according to a first embodiment. It is a vertical sectional view showing the internal structure of the digital camera according to the first embodiment. 5 is a flowchart for explaining dust detection processing in the digital camera according to the first embodiment. It is a flowchart explaining the overexposure image acquisition process at the time of the live view start in the digital camera which concerns on 1st Embodiment. 5 is a flowchart for explaining processing for removing data overlapping dust correction data from auxiliary dust data in the digital camera according to the first embodiment. It is a figure which shows the data format example of dust correction data. It is a figure explaining the detail of the dust area | region acquisition routine performed by step S27 of FIG. It is a figure which shows the processing unit of the dust area | region determination process performed by step S62 of FIG. It is a figure which shows the outline | summary of dust area size calculation performed by step S63 of FIG. FIG. 5 is a flowchart illustrating details of an imaging process routine performed in step S <b> 24 of FIG. 4. FIG. 1 is a diagram illustrating an outline of a system configuration of an image processing apparatus. It is a figure which shows the example of GUI in an image processing apparatus. It is a flowchart explaining the detail of a dust removal process. It is a flowchart explaining the detail of an interpolation routine. It is a flowchart explaining the example of the verification process of the dust correction data in the digital camera which concerns on 2nd Embodiment.

Claims (4)

An image sensor that photoelectrically converts a subject image;
Imaging means for generating an image signal from the imaging element;
Foreign matter detection means for detecting foreign matter information, which is information on the position and size of the foreign matter in the imaging screen of the imaging means, from an image signal obtained from the imaging means;
Auxiliary foreign matter information detection means for exposing the imaging device to an overexposed state, causing the imaging means to generate a whiteout image, and detecting auxiliary foreign matter information to assist the foreign matter information from the whiteout image;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the auxiliary foreign matter information is used for verification of foreign matter information detected by the foreign matter detection unit. A method of controlling an imaging device comprising: an imaging device that photoelectrically converts a subject image; and an imaging means that generates an image signal from the imaging device,
A foreign substance detection step of detecting foreign substance information, which is information on the position and size of the foreign substance in the imaging screen of the imaging means, from an image signal obtained from the imaging means;
An auxiliary foreign matter information detecting step of exposing the imaging device to an overexposed state, causing the imaging means to generate a whiteout image, and detecting auxiliary foreign matter information that assists the foreign matter information from the whiteout image;
An image pickup apparatus control method comprising:   A program for causing a computer to execute the control method according to claim 3.

Images