JP2008042348A - Image processor, image processing method, imaging apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent inappropriate foreign matter image correction processing from being applied to an image photographed by using a special lens. <P>SOLUTION: The image processor for correcting a photographed image, which is photographed by a lens interchangeable imaging apparatus and in which the shadow of a foreign matter present on the photographing optical path of the imaging apparatus is included, so as to reduce the influence of the shadow of the foreign matter comprises: a storage part for storing at least foreign matter information which is information relating to the position and size of the foreign matter present on the photographing optical path of the imaging apparatus; a correction part for correcting the photographed image so as to reduce the influence of the shadow of the foreign matter; a lens information acquisition part for acquiring the information of the lens mounted on the imaging apparatus when the photographed image is photographed; and a control part for inhibiting the correction of the photographed image by the correction part on the basis of the information of the lens acquired by the lens information acquisition part. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for correcting an image in which foreign matter such as dust attached to the surface of an optical member arranged in front of an image pickup device in an image pickup apparatus is reflected.

  In recent years, a large number of image pickup apparatuses that use an image pickup device such as a CCD, such as a digital camera or a digital video camera, and record an image as data have come to the market. In a digital camera, a photosensitive film that has been used as a conventional recording medium is no longer necessary, and instead of this, a data image is recorded on a data recording medium such as a semiconductor memory card or a hard disk device. Since these data recording media can be written and erased any number of times unlike films, the cost for consumables can be reduced and it is very convenient.

  Usually, a digital camera is equipped with an LCD monitor device capable of displaying captured images as needed and a detachable mass storage device.

  Using a digital camera equipped with these two devices not only eliminates the need for a film as a recording medium that has been used as a consumable item in the past, but also displays the captured image on an LCD monitor device and immediately on the spot. I can confirm. Therefore, image data that cannot be satisfied can be erased on the spot, or re-photographed as necessary, even in the field of photography. Compared with silver salt photography using film, the efficiency of photography is improved. It can be said that it has increased dramatically.

Due to such convenience and technological innovations such as increasing the number of pixels in the image sensor, the range of use of digital cameras is expanding, and in recent years, there are more digital cameras capable of exchanging lenses, such as single-lens reflex cameras.
JP 2004-317377 A

  However, in a digital camera, dust or dust (hereinafter collectively referred to as an image sensor, a surface of an image sensor protection glass fixed to the image sensor, an optical filter, or an optical system (hereinafter collectively referred to as an image sensor optical component). (Abbreviated as “foreign matter”) in some cases. When a foreign substance adheres to the image sensor optical component, there is a problem that the quality of the photographed image is deteriorated, for example, light is blocked by the foreign substance and the portion is not photographed.

  Not only digital cameras but also cameras using silver halide film have a problem of being captured if there is dust on the film, but in the case of film, the film moves from frame to frame, so it is the same for all frames It is very rare for trash to appear.

  However, since the image sensor of the digital camera does not move and images are taken with a common image sensor, once the dust adheres to the image sensor optical system parts, the same dust appears in many frames (photographed images). . Particularly in an interchangeable lens digital camera, there is a problem that dust easily enters the camera when the lens is replaced.

  Therefore, the photographer must always pay attention to the adhesion of the foreign matter to the optical element of the image sensor, and has spent a lot of labor on checking and cleaning the foreign matter. In particular, since the image sensor is disposed relatively deep inside the camera, it is not easy to clean or check for foreign matter.

  In order to solve such problems, an image pickup device jig for detecting the position of a foreign object existing on the imaging optical path, and an image correction based on the position of the foreign object detected using the image pickup device jig A foreign image correction method has been proposed. The imaging device jig can be attached to and detached from a lens attachment / detachment portion of the camera, and the camera photographs a shadow of a foreign object using an illumination light source provided in the jig. Then, the shadow of the foreign object is detected from the captured image by image processing, the position of the foreign object on the imaging optical path is calculated from the position of the shadow of the foreign object, and the image defect due to the foreign object in the captured image is corrected (Patent Document 1). reference).

  However, the above conventional image correction method has the following problems.

  As a photographic lens that can obtain a special effect, for example, a lens that can obtain a deep focus without tilting the photographic lens with respect to the image sensor surface (hereinafter, this movement is referred to as tilt) is known. Yes. Alternatively, there is known a lens having a tilt mechanism that performs perspective correction by moving the lens in parallel with the imaging element surface (hereinafter, this movement is referred to as shift).

  That is, the foreign object detection method and the correction method as in Patent Document 1 are based on the premise that the center of the lens is at the center of the optical axis, and thus correction of an image photographed with the lens having the tilt mechanism is not considered.

  FIG. 12A is a diagram illustrating a deviation between the position of a foreign object on an image sensor optical component and the position of a shadow of a foreign object projected on the image sensor when a normal photographing lens is used. FIG. 12B is a diagram illustrating a deviation between the position of the foreign matter on the image sensor optical component and the position of the shadow of the foreign matter projected on the image sensor when the lens having the tilt mechanism is shifted.

  In a normal photographic lens, as shown in FIG. 12A, there is a shift of x between the position of a foreign object on the image sensor optical component and the shadow of the foreign object projected on the image sensor depending on the angle of incident light from the lens. . In the description of Patent Document 1, the relationship expressed by the following equation is established between the distance r from the optical axis to the position where the foreign object is attached and the distance r ′ from the optical axis to the shadow of the foreign object projected onto the image sensor. .

r = r ′ · (L−d) / L (1)
However,
L: Distance from the image sensor surface to the lens center (pupil position)
d: Distance from the image sensor surface to a foreign object (distance from the image sensor surface to the optical component surface)
It is.

  That is, in the foreign object image correction process performed in Patent Document 1, the correction process is performed with the distance difference x between r and r 'expected. On the other hand, if the lens having the tilt mechanism is used for shifting and photographing, the optical axis of the lens and the optical axis of the image sensor are shifted. As shown in FIG. 12B, a deviation of the distance x ′ occurs. The deviation distance x ′ is larger than the deviation distance x in a normal lens. Further, since the lens optical axis and the image sensor optical axis are deviated, the above equation (1) does not hold.

  For this reason, if foreign object image correction processing is performed on the image captured using the shift function with the lens having the tilt mechanism described above by the method of Patent Document 1, an unnecessary portion is detected due to erroneous detection of the foreign object position. Image correction is performed to remove foreign matter, leading to image degradation.

  In addition, even with a photographic lens having an image blur correction function, when the image blur correction function is activated, the lens optical axis and the image sensor optical axis are shifted as in the case of the lens having the tilt mechanism described above. And the position of the shadow of the foreign object projected on the image sensor. For this reason, if the foreign object image correction process is performed by the method of Patent Document 1 described above on an image captured using an image blur correction function with a photographic lens having an image blur correction function, it is not possible due to erroneous detection of the foreign object position. Image correction is performed to remove foreign matters at necessary portions, leading to image degradation.

  In addition, when an image is taken using an extender lens that allows the focal length to be changed by being attached to the taking lens, the distance L in the above equation (1) cannot be read accurately. For this reason, when the foreign object image correction processing is performed on the image photographed using the extender lens by the method of Patent Document 1 described above, image correction for removing foreign objects in unnecessary portions due to erroneous detection of the foreign object position is performed. Done, leading to image degradation.

  Further, in the above equation (1), the correction is performed using the distance L (pupil position). Therefore, unless the distance L is accurately known, accurate foreign object image correction processing cannot be performed.

  Therefore, the present invention has been made in view of the above-described problems, and an object thereof is to prevent improper foreign object image correction processing from being performed on an image photographed using a special lens. .

In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention is photographed by an image-capturing apparatus capable of exchanging lenses, and a shadow of a foreign object existing on the photographing optical path of the image-capturing apparatus is reflected. An image processing apparatus that corrects a captured image so as to reduce the influence of the shadow of the foreign object, and stores foreign object information that is at least information regarding the position and size of the foreign object existing on the imaging optical path of the imaging device. A correction unit that corrects the captured image based on foreign matter information stored in the storage unit so as to reduce an influence of a shadow of the foreign matter, and a storage unit;
Based on the lens information acquired by the lens information acquisition means, the lens information acquisition means for acquiring information of the lens mounted on the imaging device when the captured image was taken, and the correction means by the correction means And a control means for prohibiting correction of the photographed image.

  An image pickup apparatus according to the present invention is an image pickup apparatus in which a lens can be exchanged, and includes an image pickup unit that photoelectrically converts a subject image, and the above-described image processing apparatus.

  In addition, the image processing method according to the present invention reduces the influence of the shadow of a foreign object on a photographed image that is captured by an imaging device with interchangeable lenses and includes a shadow of a foreign object existing on the imaging optical path of the imaging device. A storage step of storing foreign matter information that is at least information on the position and size of the foreign matter existing on the photographing optical path of the imaging device, and the foreign matter stored in the storage step Based on the information, a correction process for correcting the photographed image so as to reduce the influence of the shadow of the foreign object, and information on a lens attached to the imaging device when the photographed image is photographed are acquired. A lens information acquisition step, and a control step for prohibiting correction of the captured image in the correction step based on the lens information acquired in the lens information acquisition step. Characterized in that it.

  A program according to the present invention causes a computer to execute the above image processing method.

  According to the present invention, it is possible to prevent an inappropriate foreign matter image correction process from being performed on an image photographed using a special lens.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a circuit configuration example of a lens-interchangeable digital single-lens reflex camera 100 that is an image pickup apparatus incorporating the image processing apparatus according to the first embodiment of the present invention. In FIG. 1, a microcomputer 201 controls the operation of the entire camera, including processing of image data output from an image sensor 212 composed of a CCD, a CMOS element, etc., and display control of an LCD monitor device 217.

  The switch SW1 (202) is a switch that is turned on (ON) when the release button 304 (see FIG. 2) is half-pressed. When the switch SW1 (202) is turned on, the digital camera 100 is ready to shoot. The switch SW2 (203) is a switch that is turned on when the release button 304 is pressed to the end (fully pressed state). When the switch SW2 (203) is turned on, the digital camera 100 starts a photographing operation.

  The lens control circuit 204 communicates with the photographing lens 500 and transmits control signals for driving the focus lens and the diaphragm blades in the photographing lens 500 during AF (autofocus) to the lens-side microcomputer 223. The control signal is transmitted via the lens side contact 224a and the camera side contact 224b. The lens-side microcomputer 223 receives the control signal from the lens control circuit 204 and controls the driving of the focus lens in the lens 500 and the driving of the diaphragm blades. Lens information unique to the photographing lens 500 is stored in the lens-side microcomputer 223 and is transmitted to the microcomputer 201 via the lens control circuit 204.

  In FIG. 1, an external display control circuit 205 controls an external display device (OLC) 206 and a display device (not shown) in the viewfinder. The switch sense circuit 207 transmits signals of a number of switches including the electronic dial 208 provided in the camera to the microcomputer 201.

  The strobe light emission dimming control circuit 209 is grounded via the X contact 210 and controls the external strobe. The distance measuring circuit 213 detects a defocus amount with respect to a subject for AF (autofocus). The photometry circuit 214 measures the luminance of the subject.

  A shutter control circuit 215 controls the shutter and performs appropriate exposure on the image sensor 212. The LCD monitor device 217 and the backlight illumination device 216 constitute an image display device. In the image display device, a menu display of a foreign substance image correction mode, which will be described later, and a list display of captured images are performed. The external storage device 211 is, for example, a hard disk drive or a semiconductor memory card.

  In addition, the microcomputer 201 has an A / D converter 220, an image buffer memory 221, an image processing circuit 222 including a DSP, and a foreign object position storing a pixel position in the image sensor 212 causing an image defect due to the foreign object. A memory 218 is connected. Hereinafter, the information indicating the pixel position and the size in the image sensor 212 causing the image defect due to the foreign matter is also referred to as a foreign matter map. A lens data memory 219 storing a lens database necessary for determining the lens type is also connected to the microcomputer 201. The foreign object position memory 218 and the lens data memory 219 are preferably non-volatile memories.

  FIG. 2 is a perspective view showing the appearance of the digital camera of the present embodiment, and FIG. 3 is a vertical sectional view of the digital camera having the configuration shown in FIG.

  In FIG. 2, an eyepiece window 301 for finder observation, an AE (automatic exposure) lock button 302, an AF (autofocus) range-finding point selection button 303, and a release button for performing a photographing operation are shown at the top of the camera body 300. 304 is arranged. An electronic dial 208, a shooting mode selection dial 307, and an external display device 206 are also arranged. The electronic dial 208 is a multi-function signal input device that is used in combination with other operation buttons to input numerical values to the camera and switch the shooting mode. The external display device 206 includes a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture value, shooting mode, and other information.

  On the back of the camera body 300, an LCD (Liquid Crystal Display) monitor device 217 for displaying captured images and various setting screens, and a monitor switch 308 for turning on / off the LCD monitor device 217 are arranged. Yes. In addition, a cross arrangement switch 306 and a menu button 309 are also arranged.

  The cross placement switch 306 has four buttons arranged vertically and horizontally and a SET button arranged in the center. The user instructs the camera to select and execute a menu item displayed on the LCD monitor device 217. Used for.

  The menu button 309 is a button for causing the LCD monitor device 217 to display a menu screen for performing various camera settings. For example, when selecting and setting the shooting mode, after pressing the menu button 309, the desired mode is selected by operating the up / down / left / right buttons of the cross placement switch 306, and the desired mode is selected. Setting is completed by pressing the SET button. The menu button 309 and the cross arrangement switch 306 are also used for setting a foreign object image correction mode, which will be described later, and a selection operation in the foreign object image correction mode.

  Since the LCD monitor device 217 of the present embodiment is a transmissive type, the image cannot be visually recognized only by driving the LCD monitor device, and a backlight illumination device 216 is necessarily provided on the back side as shown in FIG. It is. As described above, the LCD monitor device 217 and the backlight illumination device 216 constitute an image display device.

  As shown in FIG. 3, the photographing lens 500 constituting the imaging optical system can be exchanged with respect to the camera body 300 via a body mount 400. A camera side contact 224b is provided in the main body mount 400 so as to be in contact with the lens side contact 224a. Reference numeral 401 denotes a photographing optical axis, and 402 denotes a quick return mirror.

  The quick return mirror 402 is disposed in the photographic optical path, and is a position for guiding the subject light from the photographic lens 500 to the finder optical system (the position shown in FIG. 3 is called an oblique position) and a position for retreating out of the photographic optical path ( (Referred to as a retreat position).

  In FIG. 3, the focus plate 403 forms an image of subject light guided from the quick return mirror 402 to the finder optical system. Reference numeral 404 denotes a condenser lens for improving the visibility of the finder, and reference numeral 405 denotes a pentagonal roof prism.

  Reference numerals 408 and 409 respectively denote a rear curtain and a front curtain that constitute a shutter, and by opening the rear curtain 408 and the front curtain 409, necessary exposure is given to the image sensor 212 arranged at the rear. A captured image converted into an electrical signal for each pixel by the image sensor 212 is processed by the A / D converter 220, the image processing circuit 222, and the like, and is recorded in the external storage device 211 as image data.

  The image sensor 212 is held on the printed circuit board 410. Behind this printed circuit board 410, a display board 411, which is another printed circuit board, is arranged. An LCD (Liquid Crystal Display) monitor device 217 and a backlight illumination device 216 are arranged on the opposite surface of the display substrate 411.

  211 is an external storage device for recording image data, and 412 is a battery (portable power supply). The external storage device 211 and the battery 412 are detachable.

  4A and 4B are diagrams illustrating a lens having a tilt mechanism as an example of an imaging lens. FIG. 4A is a diagram illustrating a lens in a state where the tilt mechanism function is not used, and FIG. 4B is a diagram illustrating a lens in a state where the tilt mechanism shift function is used.

  4A and 4B, the main lens barrel 501 includes a lens 501a constituting a photographing lens and a support member 501b thereof. An operation member 501c for shifting the main lens barrel 501 is provided on a side surface of the main lens barrel 501. The shift seat 502 supports the main lens barrel 501 and is configured to be slidable along a tangential plane with the main lens barrel 501. The lens side mount 503 fixedly supports the shift seat 502 and is detachably attached to the camera body 300.

  The lens-side microcomputer 223 is mounted on a substrate (not shown) and stores lens information such as a lens manufacturing number and an ID number. The connector 504 is a flexible connector that connects the lens-side microcomputer 223 and the lens-side contact 224a. The lens side contact 224a is connected to the camera side contact 224b to supply power and transmit signals.

  Next, the operation of the photographing lens under the above-described configuration will be described.

  The photographing lens 500 is shifted by rotating the operation member 501c provided on the side surface of the main lens barrel 501, and the shift amount of the photographing lens 500 is adjusted according to the rotation amount of the operation member 501c. When the operation member 501c is rotated, the main lens barrel 501 slides along a tangent plane with the shift seat 502. As the main lens barrel 501 slides, the center of the lens 501a and the center of the lens side mount 503 are shifted. When the photographing lens 500 is attached to the camera body 300 in this state, the center of the lens 501a and the center of the imaging element 212 of the camera are shifted. When the operation member 501c is operated and the shift function of the tilt mechanism is used at the time of photographing, the lens side microcomputer 223 recognizes this and stores that the shift function of the tilt mechanism is used.

  Next, a correction operation for removing a shadow of a foreign object from a captured image in the imaging apparatus configured as described above will be described. First, an outline of the operation will be described.

  First, in order to perform correction for removing the shadow of a foreign object from a captured image, it is necessary to create a foreign object map that is information indicating the position and size of the foreign object attached to the optical element of the image sensor for the digital camera. There is.

  This foreign matter map is created as follows.

  First, a uniform subject such as a white wall is photographed by a target digital camera. When such a uniform subject is photographed, an image in which only the shadow of the foreign matter attached to the image sensor optical component is reflected can be obtained from the image sensor 212. Then, the position and size of the foreign matter are detected by image processing in the image processing circuit 222 from the image in which only the shadow of the foreign matter is reflected. Then, a foreign matter map, which is information on the position and size of the foreign matter, is stored in the foreign matter position memory 218 described above.

  When shooting a uniform subject such as a white wall, do not use a special lens such as an ultra-wide-angle lens, a lens with the tilt mechanism described above, or a lens with an image blur correction function as a lens to be attached to a digital camera. To. As described above, when such a lens is used, the displacement between the position of the foreign matter adhering to the optical element of the image sensor and the position of the shadow of the foreign object formed on the surface of the image sensor increases. This is because it becomes difficult to detect accurately.

  When the foreign matter map as described above is prepared, the actual photographing is performed by the digital camera. Then, the position of the shadow of the foreign object reflected in the captured image is predicted from the information on the foreign object map, and the image processing circuit 222 corrects the image of the portion where the shadow of the foreign object is reflected. This correction can be performed by a method such as interpolating an image of a part in which a shadow of a foreign object is reflected with information of surrounding pixels.

  In the present embodiment, when the lens that was attached to the digital camera at the time of the actual shooting was a special lens such as a lens having a tilt mechanism or a lens having an image blur correction function, the actual shooting was performed. The correction for removing the shadow of the foreign matter from the image (foreign matter image correction processing) is not performed. As described above, this is because, in an image taken using a special lens such as a lens having a tilt mechanism or a lens having an image blur correction function, the position of the shadow of the foreign matter on the image sensor 212 is a foreign matter in the foreign matter map. This is because there is a possibility that the position is greatly deviated. In other words, if a correction that removes the shadows of foreign objects is performed on an image shot using such a lens, a portion of the image that does not need correction may be erroneously corrected, which deteriorates the image quality. It is because there is a possibility that it will end.

  The above is the outline of the operation for removing the shadow of the foreign matter from the captured image in the present embodiment.

  Next, the operation for removing the shadow of the foreign matter from the captured image will be described in detail. FIG. 5 is a flowchart showing an image processing operation in the present embodiment.

  As shown in FIG. 5, when the foreign object image correction mode is started, the photographing lens information when the microcomputer 201 performs the main photographing is read in step S101. Reading of photographing lens information is performed by reading lens information attached to photographed image data obtained by actual photographing by a method described later. Specifically, information indicating the type of the lens such as an ID number or a manufacturing number of the photographing lens 500, or information included in the specific lens such as the use state of the shift function of the tilt mechanism is read. This is performed for all the photographed image data stored in the external storage device 211.

  When the photographic lens information is read, the process proceeds to step S102 to determine the photographic lens. In step S102, the lens ID is read by the microcomputer 201 in step S101 and the information on the manufacturing number is collated with a lens information database, which will be described later, to specify the lens model. It is determined whether or not the specified lens model has a tilt mechanism.

  If it is determined in step S102 that the image data is taken with a photographing lens having a tilt mechanism, the process proceeds to step S103. In step S103, the microcomputer 201 determines whether or not the image data is imaged using the shift function of the tilt mechanism, based on the information indicating the use status of the shift mechanism of the tilt mechanism.

  If it is determined in step S103 that the image data was captured using the shift function of the tilt mechanism, the process proceeds to step S104. In step S <b> 104, the microcomputer 201 performs a foreign object image correction process prohibition setting for image data captured using the shift function of the tilt mechanism. The foreign object image correction processing prohibition setting is set to prevent the photographed image data from being subjected to the foreign object image correction process described above, and the setting information of the foreign object image correction processing prohibition setting is attached to the photographed image data.

  When the foreign image correction processing prohibition setting for the photographed image is performed in step S104, the process proceeds to step S105, and the microcomputer 201 performs the photographing image data correction selection disable setting. The image correction selection disabled setting in step S105 is a setting for preventing selection of images as foreign object image correction target images when thumbnails are displayed as a list in the foreign object image correction mode. The setting information of the disabled setting is attached to the captured image data.

  When the correction selection disable setting of the photographed image is performed in step S105, the microcomputer 201 performs setting so that the photographed image data is displayed in gray out in step S106. The gray-out display setting in step S106 is performed by a known technique, and when the captured images are displayed as a list of thumbnails in the foreign object image correction mode, image data for which foreign object image correction processing is prohibited is displayed in gray scale. It is. Accordingly, the user can easily distinguish between an image for which the foreign object image correction processing is prohibited and an image for which it is not.

  When the grayout display setting is performed in step S106, the process proceeds to a thumbnail list display screen for selecting an image to be subjected to foreign object image correction.

  If there is photographed image data in which reading of lens information is not normally performed in step S101, the process proceeds to step S111. In step S111, the LCD monitor 217 displays that the user has photographed image data for which photographing lens information has not been read correctly, and that the foreign object image correction processing may not be optimally performed due to this. Then, the process proceeds to step S112.

  In step S112, the LCD monitor 217 performs a display requesting the user to determine whether or not photographed image data for which lens information cannot be read can be selected as a foreign object image correction processing target.

  If it is determined in step S112 that the captured image data whose lens information cannot be read is a target for foreign object image correction processing, the captured image data whose lens information cannot be read is not grayed out in step S113. Display settings are made. In the normal display setting, captured images are displayed in color when displayed as a list of thumbnails.

  When the normal display setting is performed in step S113, the process proceeds to a thumbnail list display screen for selecting an image to be subjected to foreign object image correction.

  If it is determined in step S102 that the captured image data has been captured with a photographing lens that does not have a tilt mechanism, the process proceeds to step S113, where normal display is set, and then an image to be subjected to foreign object image correction is selected. Moves to the thumbnail list display screen for selection.

  If it is determined in step S103 that the captured image data is captured without using the shift function of the tilt mechanism, the process proceeds to step S113, where normal display is set, and then an image to be subjected to foreign object image correction is selected. To move to the thumbnail list display screen.

  If it is determined in step S112 that the captured image data whose lens information cannot be read is not subject to the foreign object image correction process, the captured image data whose lens information cannot be read is also detected in step S104. Correction processing prohibition is set. Thereafter, the processes in steps S104 to S106 are performed in the same manner as described above.

  Next, the main imaging process and lens information attachment timing will be described with reference to the flowchart shown in FIG.

  If it is detected in step S601 that the switch SW2 is turned on in the shooting mode, the microcomputer 201 activates the quick return mirror 402 shown in FIG. 3 in step S602. That is, so-called mirror-up is performed, and the quick return mirror 402 is retracted outside the photographing optical path.

  When the mirror is raised in step S602, charge accumulation in the image sensor 212 is started in step S603, and in the next step S604, the shutter front curtain 409 (see FIG. 3) and the rear curtain 408 (see FIG. 3) are displayed. Each travels and is exposed. In step S605, the charge accumulation of the image sensor 212 (see FIG. 1) is terminated. In the next step S606, the image signal is read from the image sensor 212, and the captured image data processed by the A / D converter 220 and the image processing circuit 222 is temporarily stored in the buffer memory 221.

  When reading of all the image signals from the image sensor 212 is completed in the next step S607, the quick return mirror 402 (see FIG. 3) is lowered in a so-called mirror-down manner in step S608. As a result, the quick return mirror 402 returns to a position for guiding subject light to the viewfinder optical system (an oblique position).

  In the next step S609, the captured image data temporarily stored in the buffer memory 221 is written in the external storage device 211. When the captured image data is written in the external storage device 211 in step S609, the microcomputer 201 performs the following processing. That is, information indicating the lens type such as the lens ID number and manufacturing number sent from the lens-side microcomputer 223 and information indicating the use state of the photographing lens, such as whether or not the shift function is used, are attached to the photographed image data. As described above, the data is written in the external storage device 211. As described above, a series of imaging processing ends.

  FIG. 7 is a diagram showing the configuration of the captured image data.

  The captured image data is written in the external storage device 211 in the order of shooting. The captured image data is roughly classified into header information 701 and image signal information 702. The image signal information 702 is information obtained by processing the signal accumulated in the image sensor 212 by the image processing circuit 222 and stored in the external storage device 211. The header information 701 includes main information 703, sub information 704, and manufacturer-specific information 705. The main information 703 includes information such as the manufacturer's name and model name of the camera used, and the sub information includes information related to shooting conditions such as exposure time, lens F value, and shooting date / time. The manufacturer-specific information is an information area that can be freely written for each manufacturer, and information that is not written in other information areas such as white balance and photometry mode is written therein. The lens specific information 706 is written in the manufacturer unique information. The lens unique information 706 includes information such as the ID number and manufacturing number of the photographing lens, the presence / absence of a special function, and the usage status of the special function.

  FIG. 8 is a flowchart of foreign object image correction processing in the present embodiment.

  When the foreign object image correction process is selected for a predetermined captured image in the foreign object image correction mode, the captured image data is read from the external storage device 211 in step S701. When the captured image data is read in step S701, the pixel address of the foreign object in the foreign object map stored in the foreign object position memory 218 is collated in step S702, and the pixel to be corrected is picked up from the selected captured image data.

  In the next step 703, the brightness of the pixel picked up in step S702 is compared with the brightness of the surrounding pixels, and it is determined whether or not there is a difference of a predetermined value or more. If it is determined that the brightness of the pixel picked up in step 703 is different from the brightness of the surrounding pixels by a predetermined value or more, the pixel is corrected in step S704. Here, various methods can be considered as a pixel correction method. In this embodiment, interpolation is performed using values of normal pixel data in the vicinity.

  Thereafter, in step S705, it is confirmed whether or not processing has been performed on all the pixel data picked up in step S702. In step S705, when it is confirmed that the processing has been performed on all the pixel data picked up, the image data corrected in step S706 is overwritten and saved on the image data before correction, and the foreign matter image correction processing is completed. To do.

  In step S703, if the difference in brightness between the picked-up pixel and its surrounding pixels is less than a predetermined value, the pixel is not corrected. Here, the predetermined value of the brightness difference may be set in advance, or may be set freely by the user.

  If there is unprocessed pixel data in step S705, the process returns to step S702 and the above-described processing is repeated.

  FIG. 9 is a diagram showing an example of a display screen on the LCD monitor 217 in the foreign object image correction mode of the camera body 300 of the present embodiment.

  Many conventional digital cameras can display a list of a plurality of photographed images as thumbnails as shown in FIG. 9, and the photographed images are also displayed in a list on the LCD monitor 217 in this embodiment. As described above, the list display of the photographed images is displayed after collating the lens manufacturing number or lens ID number attached to the photographed image information with the lens database. That is, it is determined whether or not the image is taken with a lens having a tilt mechanism, and an image photographed by using a shift function with a lens having a tilt mechanism is displayed in gray out (801, 802), and other images are normal. It is displayed in color display (803, 804).

  The operation for selecting a captured image is performed by operating the electronic dial 206. The frame of the selected image is highlighted as shown by 804 in FIG. Images are sequentially selected by the selection operation of the electronic dial 206. However, an image photographed using a shift function with a lens having a tilt mechanism is selected because the image selection is disabled as described above. The next captured image is selected without being skipped.

  FIG. 10 is a diagram showing an example of a display screen when photographed image data that cannot read lens information is detected when entering the foreign body image correction mode of the camera body 300.

  As described above, when photographed image data whose lens information cannot be read is detected, it is displayed that there is a possibility that the foreign matter image correction process may not be optimally performed due to this. In addition, as indicated by reference numeral 901 in the figure, a display for requesting the user to determine whether or not photographed image data for which lens information cannot be read can be selected as a target for foreign object image correction processing is performed. In accordance with the user's selection at this time, captured image data whose lens information cannot be read is set to gray-out display or normal display.

  In the above description, the lens having a specific function is a lens having a tilt mechanism. However, the lens is not necessarily limited to this, and the same applies to a lens having an image blur correction function or a master lens magnification function. You may make it perform the process of.

(Second Embodiment)
Next, a digital camera provided with an image processing apparatus according to the second embodiment of the present invention will be described. The basic configuration of the digital camera according to the present embodiment is the same as that of the digital camera according to the first embodiment shown in FIGS. The processing operation will be described.

  FIG. 11 is a flowchart showing an image processing operation according to the second embodiment.

  As shown in FIG. 11, when the foreign object image correction mode is started, the microcomputer 201 reads the photographic lens information in step S201. Reading of photographic lens information is to read lens information attached to photographic image data by the method described in the first embodiment. Specifically, information indicating the type of the lens such as an ID number or a manufacturing number of the photographic lens, and information indicating a use state of the photographic lens at the time of photographing such as a pupil position and a focal length are read. This is performed for all the photographed image data stored in the external storage device 211.

  When the photographing lens information is read, the process proceeds to step S202, and the lens information is determined. In step S202, it is determined whether or not the lens information read by the microcomputer 201 in step S101 includes pupil position information of the photographing lens at the time of photographing.

  If it is determined in step S202 that the image data does not have pupil position information, the process proceeds to step S203. In step S203, the microcomputer 201 performs a foreign object image correction process prohibition setting for image data that does not have pupil position information. The foreign object image correction processing prohibition setting is set to prevent the photographed image data from being subjected to the foreign object image correction process described in the first embodiment, and the setting information of the foreign object image correction processing prohibition setting is attached to the photographed image data. The

  When the foreign object image correction processing prohibition setting of the photographed image is performed in step S203, the process proceeds to step S204, and the microcomputer 201 performs the photographing image data correction selection disable setting. The image correction selection disabled setting in step S204 is a setting for preventing selection of images as foreign object image correction target images when thumbnails are displayed as a list in the foreign object image correction mode. The setting information of the disabled setting is attached to the captured image data.

  When the correction selection disabled setting of the photographed image is performed in step S204, the microcomputer 201 performs setting so that the photographed image data is displayed in gray out in step S205. The gray-out display setting in step S206 is performed by a known technique. When the photographed images are displayed as a list of thumbnails in the foreign object image correction mode, the image data for which the foreign object image correction process is prohibited is displayed in gray scale. Thereby, the user can easily distinguish an image for which the foreign object image correction processing is prohibited from an image for which the foreign object image correction processing is prohibited.

  When the gray-out display setting is performed in step S206, the process proceeds to a thumbnail list display screen for selecting an image to be subjected to foreign object image correction.

  If there is photographed image data in which reading of lens information has not been normally performed in step S201, the process proceeds to step S211. In step S211, the LCD monitor 217 displays on the LCD monitor 217 that the user has photographed image data for which the photographing lens information has not been read correctly, and that the foreign object image correction processing may not be optimally performed due to this. Then, the process proceeds to step S212.

  In step S212, the LCD monitor 217 performs a display requesting the user to determine whether or not photographed image data whose lens information cannot be read can be selected as a foreign object image correction processing target.

  If it is determined in step S212 that the captured image data whose lens information cannot be read is the target of the foreign object image correction processing, the normal display setting is also performed for the captured image data whose lens information cannot be read in step S213. Is made. In the normal display setting, captured images are displayed in color when displayed as a list of thumbnails.

  When the normal display setting is performed in step S213, the screen shifts to a thumbnail list display screen for selecting an image to be subjected to foreign object image correction.

  If it is determined in step S202 that the captured image data has pupil position information, the process proceeds to step S213, where normal display is set, and then a thumbnail list display for selecting an image to be subjected to foreign object image correction is displayed. Move to the screen.

  If it is determined in step S212 that the captured image data whose lens information cannot be read is not to be subjected to foreign object image correction processing, the process proceeds to step S203. In step S203, the foreign image correction processing prohibition setting is also made for the captured image data whose lens information cannot be read, and then the processing of steps S203 to S205 is performed in the same manner as described above.

  As described above, according to the above-described embodiment, when the lens used for shooting is a shooting lens having a specific function, the foreign object image correction processing is prohibited. Therefore, it is possible to prevent the correction of improper foreign matter caused by.

  Further, when the lens used for photographing is a photographing lens having a specific function, it is determined whether or not the specific function is used at the time of photographing. For this reason, even if an image is captured with a photographic lens having a specific function, foreign matter correction processing can be performed if the effect of the specific function is not reflected in the captured image.

  In addition, the foreign object image correction processing of images taken with a lens that has any of the tilt mechanism, image blur correction function, and master lens scaling function is prohibited. Inappropriate foreign object image correction processing can be prevented.

  Further, since the lens information is read by reading the lens ID number or the manufacturing number, it is possible to accurately determine the photographing lens.

  In addition, since the foreign object image correction process is prohibited for the captured image to which the pupil position information is not attached, an inappropriate foreign object image correction process due to the lack of the pupil position information can be prevented.

  In addition, in the foreign object image correction mode, an image for which the foreign object image correction process is prohibited cannot be selected, so that the user can be prevented from performing the foreign object image correction process by mistake.

  Further, in the foreign object image correction mode, the display of the captured image for which the foreign object image correction processing is prohibited is displayed differently from the other captured images, so that the user can visually recognize the image for which the foreign object image correction processing is prohibited. Can communicate.

  Further, when the lens information cannot be read, the user can select whether or not to perform the foreign object image correction process, so that the foreign object image correction process can be performed even if the lens information cannot be read. .

  In the above-described embodiment, the case where the foreign matter image correction process is performed by the image processing device 222 in the digital camera has been described. However, the digital camera may only take an image to generate image data, attach lens information to the image data, and store the image data in the external storage device 211. Then, the external storage device 211 may be attached to an image processing apparatus external to the digital camera to deliver image data, and the foreign image processing may be performed by the external image processing apparatus. In this case, the external image processing apparatus is configured similarly to the image processing apparatus 222 in the digital camera, and performs the operations shown in the flowcharts of FIGS. The external image processing apparatus may be configured as a dedicated image processing apparatus that performs foreign object image correction processing, or may be configured to perform image processing operations by operating application software on a personal computer.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, a CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is a block diagram illustrating a circuit configuration example of a lens-interchangeable digital single-lens reflex camera that is an imaging device including an image processing apparatus according to a first embodiment of the present invention. It is a perspective view which shows the external appearance of the digital camera of 1st Embodiment. FIG. 3 is a vertical sectional view of a digital camera having the configuration shown in FIG. 2. It is a block diagram of the imaging lens which has a specific function. It is a block diagram of the imaging lens which has a specific function. It is a flowchart which shows the flow of the image correction mode in 1st Embodiment. 3 is a flowchart illustrating a flow of a shooting operation of the imaging apparatus according to the first embodiment. It is a figure which shows the structure of picked-up image data. 6 is a flowchart illustrating a flow of image correction processing according to the first embodiment. It is a figure which shows the example of a display in image correction mode. It is a figure which shows the example of a warning display in image correction mode. It is a flowchart which shows the flow of the image correction mode in 2nd Embodiment. It is a figure which shows that the projection position of the foreign material on an optical component changes with the positions of an imaging lens. It is a figure which shows that the projection position of the foreign material on an optical component changes with the positions of an imaging lens.

Explanation of symbols

201 Microcomputer 212 Image Sensor 217 LCD Monitor 218 Foreign Object Position Memory 219 Lens Data Memory 223 Lens Side Microcomputer 300 Camera Body 306 Cross Position Switch 309 Menu Button 500 Shooting Lens 501 Main Lens Barrel 501a Lens 501c Operation Member 502 Shift Seat
706 Lens specific information

Claims (9)

An image processing apparatus that corrects a captured image captured by an image-capturing image-capturing device and including a shadow of a foreign object existing on the imaging optical path of the image-capturing device so as to reduce the influence of the shadow of the foreign object. ,
Storage means for storing foreign matter information that is at least information on the position and size of the foreign matter existing on the photographing optical path of the imaging device;
Correction means for correcting the captured image based on the foreign substance information stored in the storage means so as to reduce the influence of the shadow of the foreign substance,
Lens information acquisition means for acquiring information of a lens attached to the imaging device when the captured image is captured;
Control means for prohibiting correction of the captured image by the correction means based on the lens information acquired by the lens information acquisition means;
An image processing apparatus comprising:   2. The lens information includes information indicating whether or not a lens attached to the imaging apparatus when the captured image is captured is a lens having a specific function. An image processing apparatus according to 1.   The lens information means that a lens having a specific function that was attached to the imaging device when the captured image was captured was operating the specific function when the captured image was captured. The image processing apparatus according to claim 2, further comprising information indicating whether or not.   The image processing apparatus according to claim 2, wherein the specific function is any one of a tilt function, an image blur correction function, and a master lens scaling function.   The image processing apparatus according to claim 1, wherein the lens information includes an ID number or a manufacturing number of the lens.   The image processing apparatus according to claim 1, wherein the lens information includes pupil position information of the lens. An imaging device with interchangeable lenses,
An imaging means for photoelectrically converting a subject image;
An image processing apparatus according to claim 1;
An imaging apparatus comprising: An image processing method for correcting a photographed image captured by an imaging device capable of exchanging a lens and including a shadow of a foreign object existing on a photographing optical path of the imaging device so as to reduce the influence of the shadow of the foreign material. ,
A storage step of storing foreign matter information that is information on the position and size of the foreign matter existing at least on the imaging optical path of the imaging device;
A correction step for correcting the captured image based on the foreign matter information stored in the storage step so as to reduce the influence of the shadow of the foreign matter,
A lens information acquisition step of acquiring information of a lens attached to the imaging device when the captured image is captured;
A control step for prohibiting correction of the captured image in the correction step based on the lens information acquired in the lens information acquisition step;
An image processing method comprising:   A program causing a computer to execute the image processing method according to claim 8.

Images