JP6218408B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  In particular, the present invention relates to an image processing apparatus, an image processing method, and a program suitable for use in correcting defective pixels.

  In recent years, solid-state imaging devices such as CCDs and CMOSs are widely used in video cameras and digital still cameras. These semiconductor devices are for generating image data by converting light incident on an image sensor into electrical signals. However, a defective pixel that cannot convert incident light into a normal electrical signal may occur due to a crystal defect or the like generated in the manufacturing process, and this defective pixel causes a decrease in yield. Yes. In order to solve such problems, it is common to record the position information of defective pixels detected in the manufacturing process in a memory, and correct normal data by referring to the position information of defective pixels at the time of shooting. Has been done.

  On the other hand, even if the pixel is determined to be normal when inspecting the manufacturing process, there are defective pixels that are generated due to factors such as cosmic rays and changes over time, and these defective pixels should be detected and corrected as appropriate. Is desired. In such a case, there are a plurality of defective pixel data such as position information of defective pixels detected in the manufacturing process and position information of detecting defective pixels that have occurred thereafter. A method of performing pixel correction is generally performed.

  As such a technique, for example, a method of updating stored data every time a defective pixel is detected, such as a method described in Patent Document 1, is known. The method described in Patent Document 1 enters a defective pixel detection mode when the power is turned on or off, and checks each time whether or not the position of the detected defective pixel exists in the registered data. As a result, if the pixel position does not exist in the registered defective pixel data, it is added as new data.

  In addition, as a simple method for determining whether or not a target pixel is a defective pixel in order to correct a defective pixel, for example, if it is a defective pixel as in the method described in Patent Document 2, it is replaced with a specific value. The method of doing is known. For example, if the pixel of interest is a defective pixel, the value of the pixel is replaced with a value of 0 or the like, and if the pixel of interest is not a defective pixel and the value of the pixel is 0, a value is used to distinguish it from the defective pixel. Replace with 1 etc. In this way, in the defective pixel correction process, it is possible to easily determine whether or not the pixel is a defective pixel by looking at the pixel value of the target pixel.

JP-A-8-18873 JP 2010-21858 A

  In the method of Patent Document 1 described above, for example, when two types of defective pixel data are combined into one data, as described above, the newly detected defective pixel data is obtained at an appropriate timing such as when the power is turned on or when the power is turned off. Usually, it is configured to be stored in the ROM. When defective pixel correction is performed, it is more convenient that the defective pixel data to be read are arranged in the pixel arrangement order. Actually, the defective pixel data is read from the memory and combined into one data in consideration of the data arrangement order. It will be necessary. For this reason, the method of collectively recording defective pixel data in a nonvolatile memory such as a FLASH ROM has a problem that the processing overhead increases as described above.

  Furthermore, when defective pixel data detected after shipment from the factory is stored as one data in the nonvolatile memory, the data may be destroyed if the camera battery is removed during the recording operation. At this time, in order not to lose the defective pixel data detected during the manufacturing process, the defective pixel data detected after shipment from the factory is recorded in a different area from the data detected at the time of shipment from the factory. It is common to collect defective pixel data. However, even in such a system, there is a problem that it takes a long time until the camera is ready for shooting after the camera is started, and the user of the camera waits until the start is completed.

  Moreover, the method of replacing the pixel value of the defective pixel described in Patent Document 2 with a specific value is not a technique based on the premise that defective pixel data is combined into one data. Therefore, it is not possible to handle the case where there is detected defective pixel data before factory shipment and defective pixel data detected after factory shipment. For example, considering that the processing described in Patent Document 2 is first performed on defective pixel data at the time of shipment from the factory, the pixel value of the defective pixel is replaced with 0, and in the case of a normal pixel having a pixel value of 0, The process of replacing with 1 is performed. Next, when the same processing is performed on the defective pixel detected after factory shipment for the image data subjected to this processing, the pixel value 0 replaced as a defective pixel in the first processing is replaced with 1. , It is regarded as not a defective pixel. In order to obtain a good correction result of defective pixels, in addition to defective pixel data at the time of shipment from the factory, it is usually performed to use a result of detecting a defective pixel generated thereafter. The described technique requires improvements to the problems described above.

  An object of the present invention is to make it possible to easily and accurately perform defective pixel correction using a plurality of defective pixel data that are detected at different times.

  The image processing apparatus according to the present invention includes a storage unit that stores first defective pixel data indicating position information of a defective pixel, and second defective pixel data detected at a time different from the first defective pixel data. First flag addition processing means for replacing a pixel flag corresponding to the first defective pixel data with a specific value for the input image data, and the flag is replaced by the first flag addition processing means. The flag is replaced by the second flag addition processing unit that replaces the flag of the pixel corresponding to the second defective pixel data with the specific value, and the second flag addition processing unit. The image data includes correction means for performing defective pixel correction on a pixel whose flag is the specific value.

  According to the present invention, even when there are a plurality of types of defective pixel data, it is not necessary to combine the data into one, and the defective pixels can be corrected easily and accurately.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment of the present invention. It is a flowchart which shows an example of the imaging | photography process sequence which concerns on the 1st Embodiment of this invention. It is a figure explaining operation | movement of the flag addition process which concerns on the 1st Embodiment of this invention. It is a flowchart which shows an example of the imaging | photography process sequence which concerns on the 3rd Embodiment of this invention. It is a figure explaining operation | movement of the 1st flag addition process which concerns on the 2nd Embodiment of this invention. It is a figure explaining operation | movement of the 2nd flag addition process which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the imaging device which concerns on the 3rd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 100 according to the present embodiment.
In FIG. 1, an imaging apparatus 100 includes a photographing lens 101, an imaging element 102, an A / D converter 103, a ROM 104, a RAM 105, a flag addition processing unit 106, a defective pixel correction unit 107, a DSP 108, a display 109, and a recording medium 110. doing.

  The taking lens 101 receives an image of a subject, and the image sensor 102 receives the image. As the image sensor 102, for example, a CMOS sensor or a CCD sensor is generally used. The A / D converter 103 converts the light received by the image sensor 102 into a digital signal. The ROM 104 stores defective pixel data indicating positional information of defective pixels detected after the manufacturing process and shipment, parameters necessary for image processing, and the like. The ROM 104 also stores a control program for executing processing to be described later. The RAM 105 is for temporarily storing image data and the like for processing.

  The flag addition processing unit 106 adds a flag based on defective pixel data stored in the ROM 104. The defective pixel correction unit 107 corrects defective pixels for the pixels to which the flag is added by the flag addition processing unit 106. The DSP 108 is configured to perform various image processes, and performs white balance correction, image format conversion to JPEG, and the like. The display 109 displays the captured image. The recording medium 110 records image data generated by the DSP 108.

Next, the photographing operation of the present embodiment in the above configuration will be described with reference to FIG.
FIG. 2 is a flowchart illustrating an example of a shooting processing procedure performed by the imaging apparatus 100 according to the present embodiment.
First, processing is started when the mode is changed to a photographing mode, and necessary initialization is performed such as supply of a clock to the image sensor 102, setting of a parameter to the defective pixel correction unit 107, setting of a reading start address of defective pixel data ( S201). Then, image data is acquired from the image sensor 102 when a photographer presses a shutter button (not shown) (S202).

  Next, the flag addition processing unit 106 determines whether or not it is the first flag addition processing (S203). As a result of this determination, if it is the first flag addition processing, the flag addition processing unit 106 reads out first defective pixel data, which is information on defective pixels detected in the manufacturing process, from the ROM 104 (S204). Thereafter, a first flag addition process is performed on the acquired image data based on the first defective pixel data (S205). Details of the first flag addition process will be described later. Then, the image data with the flag added is stored in the RAM 105 (S206). Then, the process returns to S203.

  Next, the image data stored in S206 is read from the RAM 105, and a flag addition process is performed based on the second defective pixel data. Here, if the result of the determination in S203 is the second flag addition process, the flag addition processing unit 106 reads out the second defective pixel data newly detected after shipment from the ROM 104 (S207). Then, a second flag addition process is performed on the image data (S208). Details of the second flag addition process will be described later.

  The defective pixel correction unit 107 determines whether or not the value of the pixel of interest is a specific value that means a defective pixel, and corrects the defective pixel if it is a defective pixel. In step S209, valid pixel data is generated for all the pixels. After that, the DSP 108 performs various image processing such as white balance correction on the data corrected in the defective pixel in S209 (S210).

  Next, the DSP 108 displays an image related to the data on which various image processes have been performed on the display 109 (S211). Then, image data is recorded on the recording medium 110 (S212), and the photographing process is terminated.

  Next, the operation of flag addition processing in S205 and S208 will be described in detail with reference to FIG. In the flag addition process, the pixel value is replaced with a specific flag value as a flag indicating that it is a defective pixel, but in this embodiment, the process is performed with the flag value set to 0.

  If there is a pixel with a flag value of 0 in the input image, it will be treated as a defective pixel in the subsequent processing, and such a pixel must be set to a value different from the flag value 0 in the first flag addition processing. . Therefore, in S205, whether or not the input pixel value is the same value as the flag value 0 is confirmed in processing 303, and is replaced with a value different from the flag value 0 (hereinafter referred to as a flag replacement value).

  In consideration of the effect on the image quality, the flag replacement value is preferably a value adjacent to the flag value. For example, even if a pixel value that is 0 in the case of 14 bits per pixel is replaced with 1, it is very difficult to recognize the difference between images that have changed by one of the 14 bit gradations, and the effect on the image quality is hardly affected. I can say no. In the present embodiment, the flag replacement value is assumed to be 1.

  Also, in the first flag addition process of S205, the first defective pixel data 301 is acquired, and if the target pixel is a defective pixel in process 304, the pixel value is replaced with the flag value 0. As described above, when the first flag addition process is completed, a pixel value of 0 indicates a defective pixel.

  Next, the second flag addition process is performed. If the same process as the process 303 is performed, the flag added the first time is cleared. Therefore, the selector 305 is switched so that the process 303 is not performed. That is, only the process of acquiring the second defective pixel data 302 and replacing the pixel value with the flag value 0 is performed. As a result, at the end of the second flag addition process, a flag is added to the image data as a value 0 at the position of the first defective pixel data and the second defective pixel data. In the subsequent defective pixel correction processing, if the pixel value is the flag value 0, processing can be performed as a defective pixel. In this way, it is possible to perform processing without being aware of the existence of two types of defective pixel data.

  As described above, according to the present embodiment, even when there are a plurality of types of defective pixel data, an effect equivalent to that obtained by merging a plurality of defective pixel data can be obtained by sequentially adding flags. Thus, the circuit configuration can be simplified. As in the present embodiment, image data with a defective pixel flag added once is stored in RAM, and the circuit size can be further reduced by using the same circuit as the first flag addition process in the second flag addition process. Is possible.

  In the present embodiment, the flag value is described as 0. However, the present invention is not limited to this, and other values may be used. In the present embodiment, two types of defective pixel data are described. However, the present invention is not limited to this, and it is possible to deal with a case where there are a plurality of types of defective pixel data. If there is more defective pixel data, if the input image has a value to be used as a defective pixel flag, the processing 303 for replacing it with another value is performed only for the first flag addition processing. In the flag addition process, the process 303 may not be performed.

(Second Embodiment)
Hereinafter, the present embodiment will be described with reference to FIGS. 5 and 6. Note that the basic configuration of the imaging apparatus and the imaging processing procedure according to the present embodiment are the same as those in FIGS.

  In the first embodiment, the image data is stored in the RAM 105 after the first flag addition process, but there may be situations where the RAM or memory bandwidth is severe or where it is desired to save capacity. In some cases, defective pixel data may be merged by flag addition processing without being stored in RAM or memory. In such a case, a circuit for the first flag addition process and a circuit for the second flag addition process are prepared, and the data output by the first flag addition process is prepared by the second flag addition process. What is necessary is just to process so that it may input.

  For example, the circuit for the first flag addition processing can be configured as shown in FIG. In the first flag addition process, a process 501 for replacing with a flag replacement value is performed, and a process 503 for replacing the position of a defective pixel with a flag value based on the first defective pixel data 502 read from the ROM 104 is performed.

  Furthermore, the circuit for the second flag addition process can be configured as shown in FIG. 6, and the image data output by the first flag addition process is input. In the first flag addition process, the pixel having the same value as the flag value 0 in the original image data is replaced with another adjacent value, so the second flag addition process corresponds to the process 501. unnecessary. Therefore, in the second flag addition process, it is only necessary to perform the process 602 for replacing the position of the defective pixel with the flag value based on the second defective pixel data 601 read from the ROM 104. By doing so, when the second flag addition process is completed, a flag corresponding to the first defective pixel and a flag corresponding to the second defective pixel are added.

  As described above, according to the present embodiment, by preparing a circuit that performs the first flag addition process and a circuit that performs the second flag addition process, a plurality of image data can be stored without being temporarily stored in a memory or the like. It is possible to obtain the same effect as the merging of defective pixel data.

(Third embodiment)
Hereinafter, the present embodiment will be described with reference to FIGS. 3, 4, and 7. The large configuration is the same as that of the first embodiment, but in this embodiment, defective pixels are detected based on the output result of the first flag addition process, and the second flag addition process is performed based on the detection result. In real time.

FIG. 7 is a block diagram illustrating a configuration example of the imaging apparatus 700 according to the present embodiment.
In addition, the same code | symbol is attached | subjected about the structure same as FIG. 1 demonstrated in 1st Embodiment, and description of the structure is abbreviate | omitted. Note that the ROM 104 also stores defective pixel data detected by a defective pixel detector 701 described later.

  The defective pixel detection unit 701 is for detecting defective pixels that have occurred due to changes over time. The flag addition processing unit 702 adds a flag based on the defective pixel data stored in advance in the ROM 104 and the defective pixel data detected by the defective pixel detection unit 701.

Next, the photographing operation of the present embodiment in the above configuration will be described with reference to FIG.
FIG. 4 is a flowchart illustrating an example of an imaging process procedure performed by the imaging apparatus 700 according to the present embodiment. The first defective pixel flag addition processing in the present embodiment reads out the first defective pixel data detected in the manufacturing process stored in the ROM 104 and adds a flag. In the first embodiment, This is the same as the example shown in FIG.

  First, the processing from S201 to S206 is the same as that in FIG. If the result of the determination in S203 is the second flag addition process, the defective pixel detection unit 701 reads the image data from the RAM 105 and detects defective pixels for the image data to which the flag is added in the first flag addition process. (S401). As a defective pixel detection method, for example, the pixel value of the target pixel is compared with the pixel value of the neighboring pixel, and if there is no neighboring pixel whose difference from the target pixel is within the threshold value, the target pixel is determined as the defective pixel. It is possible to judge that. In the defective pixel detection, the first defective pixel data is supplemented, and defective pixels generated after the creation of the first defective pixel data are detected by detecting defective pixels in real time. In addition, since the flag indicating the defective pixel is already added in S205 to the image data on which the defective pixel detection is performed, for example, it is possible to perform detection while avoiding the defective pixel detected at the time of factory shipment. is there. In other words, if a pixel to which a flag indicating a defective pixel has already been added in S205 exists in the vicinity of the target pixel, this pixel is excluded from the comparison target with the target pixel, and the defective pixel is detected. Just do it.

  Next, the flag addition processing unit 702 adds a flag to a pixel detected as a defective pixel (S402). Since this process is the same as the example shown in FIG. 3 in the first embodiment, a description thereof will be omitted. Thus, when the process of S402 is completed, the first defective pixel data and the result of the defective pixel detection in real time are added as flags. Therefore, it is possible to determine whether or not the value of the pixel of interest is a specific value that means a defective pixel, and if it is a defective pixel, correct the defective pixel and generate effective pixel data. . The processing from S210 to S212 is the same as that in FIG.

  As described above, according to the present embodiment, flags are sequentially added even when there are a plurality of defective pixel data such as defective pixel data detected in advance and defective pixel data detected in real time. As a result, the same effects as those obtained by merging these defective pixel data can be obtained, and can be realized with a simple configuration. As for the flag addition processing, as in the second embodiment, the configuration for performing the first flag addition processing is as shown in FIG. 5, and the configuration for performing the second flag addition processing is shown in FIG. It may be configured.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

101 Image Lens 102 Image Sensor 103 A / D Converter 104 ROM
105 RAM
106 Flag addition processing unit 107 Defective pixel correction unit 108 DSP
109 Display 110 Recording medium

Claims (8)

Storage means for storing first defective pixel data indicating position information of a defective pixel and second defective pixel data detected when the first defective pixel data is different from the first defective pixel data;
First flag addition processing means for replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
Second flag addition processing means for replacing a flag of a pixel corresponding to the second defective pixel data with the specific value for the image data in which the flag is replaced by the first flag addition processing means;
An image processing apparatus comprising: correction means for performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced by the second flag addition processing means. Storage means for storing first defective pixel data indicating position information of the defective pixels;
First flag addition processing means for replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
Detecting means for detecting second defective pixel data different from the first defective pixel data for the image data in which the flag is replaced by the first flag addition processing means;
A second flag that replaces the flag of the pixel corresponding to the second defective pixel data detected by the detection means with the specific value for the image data whose flag is replaced by the first flag addition processing means. Flag addition processing means;
An image processing apparatus comprising: correction means for performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced by the second flag addition processing means.   When the flag of a pixel that does not correspond to the first defective pixel data has the specific value, the first flag addition processing unit sets the flag of the pixel to a value different from the specific value. The image processing apparatus according to claim 1, wherein the image processing apparatus is replaced.   When the flag of a pixel that does not correspond to the first defective pixel data has the specific value, the first flag addition processing unit sets the flag of the pixel to a value adjacent to the specific value. The image processing apparatus according to claim 3, wherein replacement is performed. An image processing method of an image processing apparatus for storing first defective pixel data indicating position information of a defective pixel and second defective pixel data detected when the first defective pixel data is different from the first defective pixel data,
A first flag addition processing step of replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
A second flag addition processing step of replacing a pixel flag corresponding to the second defective pixel data with the specific value for the image data in which the flag is replaced in the first flag addition processing step;
An image processing method comprising: a correction step of performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced in the second flag addition processing step. An image processing method of an image processing apparatus for storing first defective pixel data indicating position information of a defective pixel,
A first flag addition processing step of replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
A detection step of detecting second defective pixel data different from the first defective pixel data for the image data in which the flag is replaced in the first flag addition processing step;
A second flag that replaces the flag of the pixel corresponding to the second defective pixel data detected in the detection step with the specific value for the image data in which the flag is replaced in the first flag addition processing step. A flag addition process,
An image processing method comprising: a correction step of performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced in the second flag addition processing step. A program for controlling an image processing apparatus that stores first defective pixel data indicating position information of a defective pixel and second defective pixel data detected when the first defective pixel data is different from the first defective pixel data. And
A first flag addition processing step of replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
A second flag addition processing step of replacing a pixel flag corresponding to the second defective pixel data with the specific value for the image data in which the flag is replaced in the first flag addition processing step;
A program that causes a computer to execute a correction step of performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced in the second flag addition processing step. A program for controlling an image processing apparatus that stores first defective pixel data indicating position information of a defective pixel,
A first flag addition processing step of replacing a flag of a pixel corresponding to the first defective pixel data with a specific value for the input image data;
A detection step of detecting second defective pixel data different from the first defective pixel data for the image data in which the flag is replaced in the first flag addition processing step;
A second flag that replaces the flag of the pixel corresponding to the second defective pixel data detected in the detection step with the specific value for the image data in which the flag is replaced in the first flag addition processing step. A flag addition process,
A program that causes a computer to execute a correction step of performing defective pixel correction on a pixel whose flag is the specific value in the image data in which the flag is replaced in the second flag addition processing step.

Images