JP2009017057A - Image processing unit, imaging apparatus, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress degradation in precision of correcting a defective pixel and a pixel with a foreign matter. <P>SOLUTION: An image processing device removes an influence of the shade of a foreign matter adhering to the surface of an optical element arranged in front of an image pickup device from an image captured by the image pickup device. The image processing device comprises: a defective pixel information storage section for storing information on the position of a defective pixel in the image pickup device; a foreign matter adhesion detection section for detecting information on the position of a foreign matter, based on an image signal outputted from the image pickup device; a foreign matter information storage section for storing information on a foreign matter detected by the foreign matter adhesion detection section; a foreign matter correction section for correcting an image signal from a foreign matter pixel that is a pixel of the shade of a foreign matter by an image signal from a pixel around the foreign matter pixel, based on information stored in the foreign matter information storage section; and a control section for storing information on the defective pixel in the foreign matter information storage section by regarding the defective pixel stored in the defective pixel information storage section as a foreign matter pixel. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for correcting an image in which foreign matter such as dust attached to the surface of an optical member arranged in front of an imaging element in an imaging apparatus such as a digital camera is reflected.

  2. Description of the Related Art An imaging device such as a digital camera that records and reproduces still images and moving images captured by a solid-state imaging device (hereinafter referred to as an imaging device) such as a CCD or CMOS sensor using a memory card having a solid-state memory element as a recording medium .

  In the imaging apparatus, external light incident through the optical lens is received by an imaging element formed by two-dimensionally arranging a large number of light receiving elements (hereinafter referred to as pixels) in a plane. Each pixel receives reflected light from the object to be imaged and outputs an analog electric signal having a magnitude corresponding to the amount of light received. The output of each pixel of the image sensor is scanned and sequentially converted into a digital signal by an A / D converter, and then supplied to a signal processing circuit. The signal processing circuit converts a digital signal into a display signal and supplies it to a display device or a recording medium. As a result, an image corresponding to the two-dimensionally arranged pixels is displayed on the screen of the display device.

  It is known that in some of the pixels arranged in two dimensions in the image pickup device, local sensitivity failure of the semiconductor may occur during or after the manufacturing process. When this phenomenon occurs, a charge output corresponding to the amount of incident light cannot be obtained, so that there is a problem that the acquired image appears as a so-called defective pixel in which white spots and black spots can be seen regardless of the subject.

  In order to correct the image quality degradation caused by such defective pixels by signal processing, the following methods have been proposed conventionally. First, when the image pickup device is shipped from the factory, the output during a predetermined accumulation time in a predetermined environment is evaluated. For a pixel determined to be defective, the defect type (black point, white point) and the pixel address (defect) Pixel position information (x, y)) and information on the degree of defect (level) are obtained. The obtained information is stored in a non-volatile memory provided in the imaging device, and when a captured image is acquired, defective pixels are corrected based on this information (Patent Document 1).

  In addition, since defective pixels tend to increase over time, there is also an imaging device configured to detect a newly generated defective pixel, additionally store information on the defective pixel, and correct the defective pixel (patent). Reference 2).

  For correcting defective pixels, a technique is known in which an average value or an interpolation value of pixels of the same color around the correction target pixel is replaced with an image of a pixel defect portion.

  An example of this technique will be described with reference to FIG. 5A. FIG. 5A is a diagram schematically showing a part of pixels of the image sensor, in which pixels of three colors R, G, and B are arranged in a regular combination. Now, assuming that G21 located in the center is a defective pixel, the average value of the image signals of G16, G17, G25, and G26 that are adjacent pixels of the same color is obtained, and the defective pixel is obtained by replacing the G21 image signal with the average value. Correct.

  In such a correction method, when a foreign substance such as dust adheres to the cover glass surface, which is the package light receiving surface of the image sensor, or the optical filter surface attached to the upper surface of the cover glass, as shown in FIG. Problems occur. That is, when the shadow of the attached foreign matter is applied to a pixel used for correction, the output signal of the pixel is lowered, and the defective pixel is corrected using the signal with the reduced output, so that the correction accuracy is lowered. Resulting in.

  In view of this, there has been proposed a camera equipped with a device that automatically removes foreign matter adhering to the cover glass of the image sensor package or the surface of the optical filter (Patent Document 3). This is also very effective for the problem that the correction accuracy is deteriorated due to the foreign matter when the defective pixel is corrected.

However, even if the foreign matter removing function is used, there are some foreign matters that cannot be removed. For this reason, the following method has also been proposed for correcting shadows of foreign matters that cannot be removed. First, a white wall or the like is imaged with the lens aperture closed, the position information of the foreign matter on the screen is calculated from the captured image, and the obtained foreign matter position information is made into a database (hereinafter referred to as a dust profile). . Then, using this dust profile, the signal of the pixel that has become the shadow of the foreign object is corrected by the same method as that for the defective pixel described above. Therefore, a high-quality image can be obtained by correcting the foreign matter adhered pixels after correcting the defective pixels.
JP 2000-23051 A JP 2005-123873 A JP 2003-018440 A

  However, if the correction of the defective pixel and the correction of the foreign matter adhesion pixel are performed separately, that is, if the defective pixel is corrected first and then the foreign matter adhesion pixel is corrected, the following problem occurs. In other words, if a foreign object adheres to the same color pixel around the correction target pixel used for correcting the defective pixel, as described above, the pixel whose signal is lowered due to the shadow of the foreign object is used for correcting the defective pixel. It will be. For this reason, the correction accuracy of defective pixels is lowered. Furthermore, the signal of the defective pixel corrected in a state where the correction accuracy is reduced in this way may be used next for correcting a nearby foreign object adhesion pixel, and the correction accuracy of the foreign object adhesion pixel also decreases. End up.

  In addition, when a foreign object is shadowed on the defective pixel, the defective pixel correction and the foreign object adhesion pixel correction may be double-corrected for the defective pixel, which also reduces the correction accuracy. To do.

  Therefore, the present invention has been made in view of the above-described problems, and an object of the present invention is to be able to suppress a decrease in correction accuracy of defective pixels and foreign matter-attached pixels.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention is disposed in front of an image pickup device from an image picked up by an image pickup device in which pixels are two-dimensionally arranged. An image processing apparatus for removing the influence of a shadow of a foreign substance attached to the surface of an optical element, the defective pixel information storing means for storing information on at least the position of the defective pixel in the image sensor, and output from the image sensor A foreign matter adhesion detecting means for detecting information on at least the position of the foreign matter, a foreign matter information storage means for storing information on the foreign matter detected by the foreign matter adhesion detecting means, and the foreign matter information storage Based on the information stored in the means, the image signal from the foreign pixel, which is a pixel that is a shadow of the foreign matter, is obtained from the surrounding pixels of the foreign pixel. A foreign matter correcting means for correcting using an image signal, a control means for regarding the defective pixel stored in the defective pixel information storage means as the foreign pixel, and storing information on the defective pixel in the foreign matter information storage means; It is characterized by providing.

  An image pickup apparatus according to the present invention includes an image pickup element in which pixels are two-dimensionally arranged, an optical element arranged in front of the image pickup element, and the image processing apparatus. .

  In addition, the image processing method according to the present invention is based on the influence of the shadow of the foreign matter attached to the surface of the optical element arranged in front of the image pickup element from the image picked up by the image pickup element in which the pixels are two-dimensionally arranged. And a defective pixel information storage step of storing in the defective pixel information storage means information regarding at least the position of the defective pixel in the image sensor, and an image signal output from the image sensor. A foreign matter adhesion detection step for detecting information on at least the position of the foreign matter, a foreign matter information storage step for storing information on the foreign matter detected in the foreign matter adhesion detection step in foreign matter information storage means, and the foreign matter information storage means. Based on the information stored in the image, the image signal from the foreign object pixel that is a pixel that is a shadow of the foreign object is converted to a pixel around the foreign object pixel. A foreign substance correcting step of correcting using the image signal, and a control step of regarding the defective pixel stored in the defective pixel information storage means as the foreign pixel and storing information on the defective pixel in the foreign information storage means And.

  According to the present invention, it is possible to suppress a decrease in the correction accuracy of defective pixels and foreign matter adhered pixels.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing a block configuration of a digital camera which is an embodiment of an imaging apparatus of the present invention.

  In FIG. 1, noise reduction and gain adjustment are performed by the CDS / AGC circuit 2 on the output of the imaging device 1 including a CCD, a CMOS sensor, etc. in which a plurality of photoelectric conversion units (pixels) are two-dimensionally arranged. The The signal output from the CDS / AGC circuit 2 is converted from an analog signal to a digital signal by the A / D converter 3 and input to the engine 4 which is a digital signal processing IC. The engine 4 functions as an image processing device.

  The engine 4 includes a foreign matter pixel influence determination unit 5, a defective pixel information storage unit 6, a foreign matter adhesion detection unit 7, a foreign matter information storage unit 8, a pixel correction unit 10, a development processing unit 11, and a switch 9. In the present embodiment, the pixel correction unit 10 actually functions as a foreign matter correction unit that corrects foreign matter pixels.

  The digital image signal input to the engine 4 is processed by the pixel correction unit 10 to correct defective pixels and pixels that are affected by foreign matter adhesion (referred to as foreign matter pixels or foreign matter attached pixels). The image signal subjected to these corrections is subjected to development processing such as color interpolation processing in the development processing section 11 and stored in the memory 20.

  In the foreign pixel effect determination unit 5 described above, the defective pixel unit is obtained from the position data of the defective pixel and the position data of the foreign pixel obtained from the information in the defective pixel information storage unit 6 and the dust profile in the foreign matter information storage unit 8. It is determined whether or not the image signal used for the correction is affected by foreign matter.

  In the present embodiment, the defective pixel portion is not corrected as a defective pixel portion, but is corrected as a foreign matter adhesion pixel portion. However, the image signal used for correcting the defective pixel portion is generally It means an image signal of a pixel used in a defective pixel correction method performed.

  Here, a correction method for a defective pixel portion that is generally performed will be described.

  For example, FIG. 2A is a diagram schematically showing a part of the pixels of the image sensor, in which pixels of three colors R, G, and B are arranged in a regular combination. If G21 in the center is a defective pixel, the average value of the image signals of G16, G17, G25, and G26 (pixels around the defective pixel) that are adjacent pixels of the same color is obtained, and the G21 image signal is the average value. The image signal from the defective pixel portion is corrected by replacing with. This is a general correction method for defective pixel portions. In the present embodiment, when it is assumed that the defect pixel influence determination unit 5 corrects the defective pixel portion by a general method in this way, pixels assumed to be used for correction are affected by the foreign substance. It is determined whether or not.

  The defective pixel information storage unit 6 stores information such as position data of defective pixels, a defect level, and a defect type acquired when the image pickup device 1 is shipped from the factory. In addition, information such as the position and size of foreign matter on the surface of the cover glass of the image sensor 1 or the surface of the optical filter 21 attached to the cover glass surface of the image sensor 1 is detected by the foreign matter adhesion detection unit 7. Information such as the position and size of the detected foreign matter is stored in the foreign matter information storage unit 8. Here, the surface of the cover glass of the image sensor 1 or the surface of the optical filter 21 attached to the cover glass surface of the image sensor 1 is described as “image sensor surface” for simplicity.

  An image developed by the development processing unit 11 is displayed on a display (LCD) 12 such as a TFT liquid crystal display, or is output from a video terminal (VIDEO) 13 or the like to an external monitor.

  The developed image can be stored in an external memory 14, for example, a CF card (flash memory card). The engine 4 is connected to an interface 15 such as a USB interface, which is a serial communication device, through which image data can be output to an external computer or the like. Further, the engine 4 drives a timing generator (TG / VDR) 16 to generate a clock signal and the like to be supplied to the image sensor 1.

  Further, a microprocessor (CPU) 19 is connected to the engine 4, and various settings are performed according to instructions from the CPU 19, and the operation is managed by the CPU 19. The memory 20 is used as a work area for storing a program to be provided to the CPU 19 or temporarily storing a photographed image. The lens 17 is an optical system for photographing, and has an actuator for adjusting its aperture and focus. The actuator is controlled in accordance with an instruction from the CPU 19. Further, the foreign matter removing unit 18 has an actuator (for example, a piezoelectric element that vibrates the optical filter 21) for removing foreign matter on the surface of the optical filter (optical element) 21 attached to the image pickup device 1, and for example, the CPU 19 at the time of power activation The drive is controlled by.

  The optical filter 21 is bonded or assembled in a highly sealed state in a state where foreign matters on the surface of the cover glass of the image sensor 1 and the surface of the optical filter 21 are sufficiently removed.

  Next, the operation of the foreign matter adhesion detection unit 7 provided in the engine 4 will be described with reference to the flowchart of FIG.

  In FIG. 3, when the main power supply (not shown) of the camera is activated, the CPU 19 controls the foreign matter removing unit 18 to remove foreign matter on the surface of the optical filter 21 attached to the cover glass of the image sensor 1 for a predetermined time. Automatically executed (step S201). In this embodiment, this operation is executed when the main power supply of the camera is started. However, the operation timing is not limited to this, and may be automatically executed periodically, or may be arbitrarily executed by the user. It doesn't matter. The details of the foreign matter removing operation may be those described in, for example, Japanese Patent Application Laid-Open No. 2003-018440, or any other configuration that can effectively remove foreign matter on the surface of the image sensor.

  The purpose of the foreign matter removing operation is to remove the foreign matter attached to the surface of the optical filter 21 attached to the cover glass of the image sensor 1. However, there is a limit to this removal capability, and most of the attached foreign matter can be reliably removed, but there are some very fine foreign matters of several pixels of the image sensor that cannot be removed. Therefore, it is detected how much foreign matter remains on the surface of the image sensor 1 after step S202.

  When the foreign matter removal operation is completed, the CPU 19 initializes the contents of the dust profile in the foreign matter information storage unit 8 in the engine 4 (step S202), and the switch 9 is turned on to detect the foreign matter adhesion from the image signal from the A / D converter 3. The state is input to the unit 7 (step S203). Then, a foreign object detection image is taken (step S204). Since a shadow is formed on the photographed image (foreign pixel) at a position where foreign matter has adhered to the surface of the image sensor, in step S204, a relatively bright uniform light that easily detects the foreign matter is photographed to generate a foreign matter detection image. . As a light incident method, a dedicated light source may be provided inside the camera as in JP-A-2002-300442, or light from the outside of the camera may be used.

  The output of the image sensor 1 by uniform light imaging is input to the engine 4 through the CDS / AGC circuit 2 and the A / D converter 3. The image signals input to the engine 4 are sequentially input to the foreign matter adhesion detection unit 7 for each image signal corresponding to one pixel (step S205). A defective pixel determination is performed on the image signal input to the foreign matter adhesion detection unit 7 based on information in the defective pixel information storage unit 6 (step S206). It does not carry out (step S206: Yes) and transfers to step S209. An image signal that is not a defective pixel (Step S206: No) is compared with a specified value (a specified luminance value of the image signal) (Step S207). Judge that foreign matter is attached. Then, information such as position data, size, and image signal level is written as a dust profile in the foreign matter information storage unit 8 (step S208). On the other hand, when the compared image signal is equal to or greater than the specified value, it can be determined that no foreign matter is attached, and therefore writing to the foreign matter information storage unit 8 is not performed. Every time the detection operation for one pixel is completed, it is confirmed whether the detection operation for all pixels is completed (step S209), and steps S205 to S209 are repeated until the detection operation is completed.

  Next, information on defective pixels recorded in the defective pixel information storage unit 6 is sequentially read out to the foreign object pixel influence determination unit 5 (step S210). Based on the read information of the defective pixel, the foreign object pixel influence determination unit 5 compares the pixel assumed to be used for correcting each defective pixel with the dust profile created in steps S201 to S209 described above. (Step S211). When the pixel assumed to be used for correcting the defective pixel matches the dust profile (step S211: Yes), an area including the defective pixel portion is added to the dust profile (step S212).

  For example, as shown in FIG. 2B, when the image signal of G17, which is one of the same color pixels adjacent to the defective pixel G21, is affected by the adhesion of foreign matter, an area including G21 is added to the dust profile. More specifically, a dust profile is created by enlarging the foreign matter adhesion region so as to include the defective pixel G21. In other words, the defective pixel G21 is treated as a foreign matter adhesion pixel, not a defective pixel. In addition to this, as shown in FIG. 2C, the foreign matter adhesion region may be further enlarged to create a dust profile with R8 and B11 as the foreign matter adhesion region.

  If the pixel assumed to be used for correcting the defective pixel does not match the dust profile (No at Step S211), the process proceeds to Step S213. In step S213, the target defective pixel is added to the dust profile as a single defective pixel. In other words, the target pixel is not treated as a defective pixel, but as a foreign object adhesion pixel.

  Then, it is confirmed whether the determination of all defective pixels is completed (step S214), and steps S210 to S214 are repeated until the determination is completed.

  The latest foreign substance adhesion pixel information (dust profile) for the target digital camera is created by the operation of the flowchart of FIG.

  Next, referring to the flowchart of FIG. 4 for the operation of correcting the photographed image by the pixel correction unit 10 provided in the engine 4 using the foreign substance adhesion pixel information (dust profile) created as described above. I will explain.

  In FIG. 4, when the user performs normal shooting that is not dust detection shooting, the output of the imaging device 1 that receives the reflected light from the shooting target is the CDS / AGC circuit 2 and the A / D converter 3. And input to the engine 4 (step S301).

  In step S302, it is determined whether or not correction of pixels affected by the adhesion of foreign matter is performed by the imaging device. When the correction is not performed by the imaging device (step S302: No), the correction processing is not performed and the development processing unit. The image signal is transmitted to 11, and the process ends. In addition, when correction of the pixel affected by the foreign matter adhesion is performed by the imaging apparatus (step S302: Yes), the process proceeds to step S303.

  The image signal input to the engine 4 is sequentially input to the pixel correction unit 10 for each image signal corresponding to one pixel (step S303).

  The image signal input to the pixel correction unit 10 is subjected to foreign object adhesion pixel determination based on a dust profile that is information in the foreign object information storage unit 8 (step S304). If the image signal is not for a foreign object adhesion pixel (step S304: No), the process proceeds to step S307 without correcting the image signal. If this image signal is for a foreign substance adhesion pixel (step S304: Yes), a correction value is obtained using an image signal from a predetermined pixel around the foreign substance adhesion pixel (step S305), and the obtained correction value is obtained. Is replaced with the image signal of the foreign substance adhesion pixel (step S306).

  In this method, for example, as shown in FIG. 2A, an average value of image signals of the same color pixels G16, G17, G25, and G26 adjacent to the defective pixel G21 is obtained, and the average value is replaced with the image signal of G21. This is the same as the method of performing correction.

  After that, every time the correction operation for one pixel is completed, it is confirmed whether the foreign matter adhesion pixel determination and the image signal correction operation for all pixels are completed (step S307), and steps S303 to S307 are repeated until this is completed. .

  As described above, in the present embodiment, a defective profile and a foreign object adhesion pixel are all handled as a foreign object adhesion pixel and a dust profile is created. Then, the correction of the defective pixels of the image sensor and the correction of the foreign matter adhesion pixels are not performed individually, but all are regarded as foreign matter adhesion pixels and are performed simultaneously. This makes it possible to provide an image with high correction accuracy without performing correction using an image signal with low accuracy or performing double correction on the same pixel.

  Further, in the present embodiment, the correction method for the foreign substance adhered pixels is correction using peripheral pixels of the same color, but the present invention is not limited to this correction method. Similarly, the position information acquisition method of the foreign matter adhesion part, the dust profile creation method, the addition method of the defective pixel to the dust profile, and the correction method are not limited to the present embodiment.

  Further, in the present embodiment, the correction of defective pixels and the correction of foreign matter adhesion portions are performed by the digital camera body, but these corrections may be performed by an external device such as a personal computer. In that case, the external device provides a configuration corresponding to the engine 4 shown in FIG.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above-mentioned storage medium, the storage medium stores program codes corresponding to the procedure described above.

It is a figure which shows the block configuration of the digital camera which is one Embodiment of the imaging device of this invention. It is the figure which extracted and extracted typically a part of pixel of the image sensor for demonstrating the operation | movement which correct | amends the image signal from a general defective pixel part. It is the figure which extracted and extracted typically a part of pixel of the image sensor for demonstrating the operation | movement which correct | amends the image signal from a defective pixel part. It is the figure which extracted and extracted typically a part of pixel of the image sensor for demonstrating the operation | movement which handles a defective pixel part as a foreign material adhesion pixel in one Embodiment of this invention. It is a flowchart explaining the preparation method of the information of a foreign material adhesion pixel. It is a flowchart explaining the correction | amendment operation | movement of a defective pixel and a foreign material adhesion pixel. It is the figure which extracted and extracted typically a part of pixel of the image pick-up element for demonstrating the operation | movement which correct | amends the image signal from the conventional defective pixel part. It is the figure which extracted and extracted typically a part of pixel of the image pick-up element for demonstrating the operation | movement which correct | amends the image signal from the conventional defective pixel part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up element 2 CDS / AGC circuit 3 A / D converter 4 Engine 5 Foreign object pixel influence determination part 6 Defective pixel information storage part 7 Foreign object adhesion detection part 8 Foreign object information storage part 9 Switch 10 Pixel correction part 11 Development processing part 12 Display 13 Video terminal
14 External memory 15 Interface 16 Timing generator 17 Lens 18 Foreign matter removing unit 19 Microprocessor 20 Memory 21 Optical filter

Claims (5)

An image processing apparatus for removing the influence of a shadow of a foreign substance attached to the surface of an optical element arranged in front of the imaging element from an image captured by an imaging element in which pixels are two-dimensionally arranged. ,
Defective pixel information storage means for storing information on at least the position of the defective pixel in the image sensor;
Foreign matter adhesion detection means for detecting information on at least the position of the foreign matter based on an image signal output from the image sensor;
Foreign matter information storage means for storing information on the foreign matter detected by the foreign matter adhesion detection means;
Based on the information stored in the foreign matter information storage means, the image signal from the foreign pixel which is a pixel shadowed by the foreign matter is corrected using the image signal from the pixels around the foreign matter pixel. Foreign matter correcting means;
Control means for regarding the defective pixel stored in the defective pixel information storage means as the foreign pixel, and storing information on the defective pixel in the foreign information storage means;
An image processing apparatus comprising:   The control means is affected by the shadow of the foreign object when a pixel around the defective pixel assumed to be used for correcting the defective pixel is included in an area affected by the shadow of the foreign object. The image processing apparatus according to claim 1, wherein a region including the defective pixel is expanded and stored in the foreign matter information storage unit. An image sensor in which pixels are two-dimensionally arranged;
An optical element disposed in front of the imaging element;
The image processing apparatus according to claim 1 or 2,
An imaging apparatus comprising: An image processing method for removing the influence of a shadow of a foreign substance attached to the surface of an optical element arranged in front of the image sensor from an image captured by an image sensor in which pixels are two-dimensionally arranged. ,
A defective pixel information storage step of storing information on at least the position of the defective pixel in the image sensor in the defective pixel information storage means;
A foreign matter adhesion detection step for detecting information on at least the position of the foreign matter based on an image signal output from the imaging element;
A foreign matter information storage step of storing information on the foreign matter detected in the foreign matter adhesion detection step in a foreign matter information storage means;
Based on the information stored in the foreign matter information storage means, the image signal from the foreign pixel which is a pixel shadowed by the foreign matter is corrected using the image signal from the pixels around the foreign matter pixel. Foreign matter correction process;
A control step in which the defective pixel stored in the defective pixel information storage means is regarded as the foreign pixel, and information regarding the defective pixel is stored in the foreign information storage means;
An image processing method comprising:   A program causing a computer to execute the image processing method according to claim 4.

Images