JP2006254388A - Imaging apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of simply correcting defective pixels of its CCD. <P>SOLUTION: A nonvolatile memory 17 built in a camera stores binary compressed defective position data for denoting positions of defective pixels. The data size is small because the data are binary compressed data. When a photographer executes photographing, imaged data by the CCD 3 are stored in a memory 9. On the other hand, the binary compressed defective position data stored in the built-in nonvolatile memory 17 are expanded by a DSP section 5 and thereafter the expanded data are stored in the memory 9. In the expanded binary data, luminance data at addresses of nondefective pixels are zero, and luminance data at addresses of defective pixels are 255. Then defective position (X, Y) is searched from the CCD imaged data in the memory 9 on the basis of the defective correction luminance data and pixel data at X, Y coordinates on the CCD imaged data are corrected. In the correction method, values of coordinates (X+2, Y) of the CCD imaged data are overwritten onto the data at the coordinates (X, Y), for example. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus and a program, and more particularly to an imaging apparatus such as a digital still camera having a pixel defect correction function and a program used in the imaging apparatus.

  Some CCDs used in current digital still cameras have more than 3 million pixels. On the other hand, in the semiconductor manufacturing process, not all pixels of the CCD are formed with completely the same performance, and defective pixels are also formed. However, even if there is a certain number of defective pixels in the product yield, it is generally used as a good CCD for products such as digital still cameras. On the product side, the vertical and horizontal coordinates (address) on the CCD of the defective pixel are stored in advance in a ROM or the like in the product, and the data at the position corresponding to the defective address of the imaging data at the time of shooting is stored in the adjacent pixel of the same color filter Then, correction processing such as still image processing is performed.

  In addition to the above-described defective pixels, there are also pixels with poor dark current characteristics. This is usually not a problem for exposure of about 1 second, but is a pixel with a large increase in dark current as the exposure time increases, and it is superimposed on the image data and appears as a bright spot. This is a star-shaped noise in long exposure photography such as night view photography. There are about 60 defective pixels for a 3 million pixel CCD, and the address of each defective pixel can be stored in the ROM. However, defective pixels due to dark current must be corrected by imaging parameters such as exposure time. In addition, in the case of 4-second exposure, the number of defective pixels is 60,000 pixels even if 2% of 3 million pixels, for example, and the number is enormous.

Conventionally, a technique has been proposed in which a huge amount of data for correcting a defective pixel address or the like is stored in a removable external recording medium (Patent Document 1). In addition, a technique has been proposed in which a CCD screen is divided, and correction information is stored in a storage unit with priority according to the screen position and defect level (Patent Document 2).
Japanese Patent No. 3235204 JP 2000-115644 A

However, there is a problem such as the possibility of losing a removable recording medium, and it is ideal that the CCD and its correction information are originally integrated with the imaging device. Further, the technique using the correction information has a problem that an independent function for correction is required and the configuration is complicated.
An object of the present invention is to obtain an imaging device that can easily correct a defective pixel of a CCD.

  According to the first aspect of the present invention, an image pickup unit that picks up an image of a subject and outputs image pickup data, and defect position binary compressed data that indicates the position where the defective pixel of the image pickup element is located and the position where there is no defect in the image pickup means in binary. Storage means for performing, decompression means for decompressing the stored defect position binary compressed data, and correction means for correcting defective pixel data in the imaging data based on the decompressed defect position binary data. An imaging device characterized by the above.

According to a second aspect of the present invention, in the apparatus according to the first aspect, the correction means detects a position of the defective pixel in the imaging data based on the expanded defect position binary data, and a detected position. The defective pixel data is corrected with pixel data in the vicinity of the pixel.
According to a third aspect of the present invention, in the apparatus according to the first or second aspect, the storage unit stores a plurality of the defect position binary compressed data, and the correction unit is used according to an imaging parameter at the time of imaging. One of the plurality of defect position binary compressed data is used.

According to a fourth aspect of the present invention, in the apparatus according to the third aspect, the plurality of defect position binary compressed data is set in accordance with an exposure time as an imaging parameter at the time of imaging, and the correcting means One of the plurality of defect position binary compressed data used in accordance with an exposure time is used.
According to a fifth aspect of the present invention, in the apparatus according to the fourth aspect, the storage unit stores defect coordinate data indicating coordinates of a defective pixel of the image sensor, and the correction unit stores the defect position binary compressed data. And the defect coordinate data are used according to an imaging parameter at the time of imaging.

  According to a sixth aspect of the present invention, in the apparatus according to the first aspect of the present invention, a creation unit that creates a plane image in which all pixel data is data having zero brightness, and each pixel data of the imaging data is compared with a predetermined threshold value. , Pixel data equal to or greater than the threshold is used as defective pixel data, data at a corresponding position in the plane image is replaced with a predetermined luminance, the replaced plane image data is subjected to binary compression processing, and the processed data is converted into the defect position binary data. Defect position binary compressed data creating means for creating compressed data is provided.

  According to a seventh aspect of the present invention, an image pickup process for picking up a subject image and outputting image pickup data, and defect position binary compressed data indicating in binary the position where the defective pixel of the image pickup element in the image pickup process is present and absent are stored. A reading process to be read from the means, an expansion process to expand the read defect position binary compressed data, and a correction process to correct defective pixel data in the imaging data based on the expanded defect position binary data A program to be executed by a computer.

The invention according to claim 8 is the program according to claim 7, wherein the defect position binary compression data is plural, and the plurality of defect position binary compression used according to the imaging parameters at the time of imaging by the correction process. One of the data is used.
According to a ninth aspect of the present invention, in the program according to the seventh aspect, the computer executes a read process for reading out the defect coordinate data indicating the coordinates of the defective pixel of the image sensor from a storage unit, and the defect process causes the defect to be detected. One of the position binary compressed data and the defect coordinate data is used according to the imaging parameter at the time of imaging.

  According to a tenth aspect of the present invention, in the program according to the seventh aspect, a creation process for creating a plane image in which all pixel data is data having zero brightness, and each pixel data of the imaging data is compared with a predetermined threshold value. A comparison process, a replacement process in which pixel data equal to or greater than the threshold is defective pixel data, the data at a corresponding position in the plane image is replaced with a predetermined luminance, and a compression process in which the replaced plane image data is subjected to a binary compression process; A defect position binary compressed data creation process for storing the compressed data as the defect position binary compressed data in a storage means is executed by a computer.

According to the first or second aspect of the invention, since the defect position binary compressed data indicating the address information of the defective pixel has a small data amount, it can be stored in the memory having a small capacity and held in the imaging apparatus. Further, defect correction can be performed for dark current with a large number of correction pixels. In addition, since the character image reproduction function of the camera can be used, efficient correction can be performed. This is particularly effective when it is necessary to shoot with a long exposure.
According to the third or fourth aspect of the invention, an appropriate number of corrections can be made according to the number of dark current defective pixels.

According to the fifth aspect of the present invention, defective defects can be corrected in a short time during normal imaging, and more dark current defects can be corrected in imaging with a long exposure time, enabling efficient correction. Also, the correction time is performed according to the shooting time, so there is no sense of incongruity.
According to the invention of claim 6, since the character image photographing function of the camera can be used, the defect position binary compressed data can be efficiently created by the imaging apparatus itself. Also, it is possible to create defect position binary compressed image data according to the user's desired exposure time.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a digital camera according to an embodiment of the present invention.
The present embodiment is an imaging apparatus having a binary image capturing and reproducing function, and has defect address information in the apparatus as a binary compressed image so that many defective pixels can be easily corrected. It is.

In FIG. 1, 1 is a lens, 2 is a mechanical shutter, 3 is a CCD as an image sensor,
4 is a CDS / AD unit that performs correlated double sampling and A / D conversion on the image pickup signal of the CCD 3, and 5 is a digital signal that has been A / D converted, converted into a luminance signal Y and color difference signals U and V, and the YUV data is converted. A DSP unit (digital signal processing unit) 6 for JPEG compression and a mechanical driver unit 6 for driving the mechanical unit perform focus driving for driving the lens 1 and opening / closing control of the shutter 2. 7 is a CCD drive unit, 8 is a CPU for controlling the entire camera, 9 is imaged image data, memory for temporarily storing read data from a file, 10 is a communication driver unit for the outside of the camera, and 11 is removable from the camera It is a memory card.

  Reference numeral 12 denotes a display controller that converts the image signal processed by the DSP unit 5 so that it can be displayed on an LCD, a display unit that includes an LCD that displays the converted signal, 13 a switch unit operated by a user such as a shutter button, and 14 a camera. For example, the external device I to be imaged includes a charging circuit for charging a battery in the camera body, for example. If the external device I has a display device, the display unit 12 may not be provided. Reference numeral 15 denotes an external device II connected to the external device I, for example, a PC (personal computer). Reference numeral 16 denotes an audio CODEC. An analog audio signal from an input analog unit 18 (for example, a microphone) is A / D converted and input to the DSP unit 5, or a digital audio signal from the DSP unit 5 is D / A converted and output. It outputs to the analog part 19 (for example, speaker). Reference numeral 17 denotes a non-volatile memory that stores captured image files, recorded audio files, and holds shooting mode information and the like even when the camera is turned off.

FIG. 2 is a flowchart showing the main processing during recording, and FIG. 3 is a block diagram showing the configuration of the main processing block 20 and the parallel processing block 40 in software.
3, each processing unit 21 to 29 of the main processing unit block 20 is a processing unit that executes processing in each step of the flowchart of FIG. For example, the SW determination processing (S1) of FIG. 2 is executed by the SW determination processing portion 21 of FIG. 3, and the operation mode determination processing (S2) of FIG. 2 is executed by the operation mode determination processing portion 22 of FIG. Further, there is a defect position binary compressed data processing unit 30 which is a characteristic part of the present invention, and a plane image creation processing unit 31 and a defect position binary image creation processing unit 32 are included therein. The parallel processing block 40 includes a periodic timer interrupt processing unit 41 that periodically executes interrupt processing.

Next, general imaging operations and image processing will be described.
As shown in FIGS. 2 and 3, in the present embodiment, there are a still image shooting mode for recording a multi-valued image and a character shooting mode for recording a binary image (character image).
In this system, for example, SW (switch) information input by the periodic timer interrupt processing unit 41 every 20 ms is determined by the SW determination processing unit 21 (S1 in FIG. 2), and the information is sent to the operation mode determination processing unit 22. hand over. If there is no particularly valid SW confirmation input, a loop for repeating the SW determination process as an invalid operation is obtained. If there is a valid SW confirmation input, the operation mode determination processing unit 22 assigns the information to the first SW on operation, the second SW on operation, the file reproduction operation, and the display composition operation based on the information (S2 in FIG. 2).

  In the still image shooting mode or the character shooting mode, the imaging unit is driven by the mechanical driver unit 6 and the CCD driving unit 7, and the imaging data is processed and displayed on the display unit 12 (monitoring display). The switch unit 13 in FIG. 1 has a shutter button. When the first SW of the shutter button is turned on in the still image shooting mode, an AE / AF (automatic exposure control / automatic focus control) processing unit 23 performs processing. (S3 in FIG. 2). The CPU 8 moves the lens 1 by the mechanical driver unit 6 to adjust the focus. Further, the imaging data is evaluated by the DSP unit 5 and the exposure time value of the CCD 3 set in the CCD driving unit 7 is determined.

  2 and 3, the SW determination processing unit 21 (S1) and the operation mode determination processing unit 22 (S2) determine the first SW on process, and the AE / AF processing unit 23 processes (S3), and the process ends. After that, the process returns to the SW determination process. If there is no valid SW information, the loop is returned to the SW determination process without doing anything again. Next, when the second SW is turned on (S4), photographing / recording processing is performed. The CCD image data is processed by the DSP unit 5 from image processing to compression and written in the memory 3. Image processing and compression processing are different in the still image shooting mode and the character shooting mode.

  When a multi-value image is shot in the still image shooting mode, a still image recording process (S5) is performed. At the time of image processing, multi-tone luminance data and color data at the image position are generated from the R, G, B data at each imaging position. These data are subjected to still image compression conversion such as JPEG, and a header is added to write this as a still image compressed file in the memory card 11 (S6).

  In the character photographing mode, character recording processing (S7) is performed. The average luminance around each imaging position is used as a threshold value, and the luminance data at that image position is binarized (“1” if greater than the threshold value, “0” if smaller). This binarized data is compressed into binary data such as MMR which is also used in facsimile, and a binary compressed file is attached with a header, and this is written in the memory card 11 in association with the multi-valued image (S6). .

  A normal still picture luminance signal and color signal are 16 bits per pixel. Since a binary image can represent each pixel with 1 bit, it is simply 1/16 of a multi-valued image. In addition, the number of data can be further reduced by assigning codes according to the appearance frequency of the bit string pattern. In binary compression such as the MMR method, the size of the binary compressed image file can be 40 KB or less even when the image is taken with a CCD having a pixel number of 2048 × 1536.

  When a character such as a dial is photographed and attached to the multi-valued image, when the second SW is turned on (S4), the character attachment photographing mode is set and the character recording process (S8) is performed. The processed character data is stored together in the same file as the still image data file in the memory 9 (S9), and is recorded on the memory card 11 as a still image file with characters (character attached file).

  In addition, the camera selects a file to be played back in the memory card in the playback frame number determination process (S10), and reads it into the memory 9 in the camera in the still playback process (S11). Is expanded and displayed on the display unit 12.

Next, features of the present invention will be described.
First, a first embodiment related to claims 1 and 2 will be described.
In the present embodiment, the built-in nonvolatile memory 17 in the camera has defect position binary compressed data indicating the position of the defective pixel. The data size is about 40 KB because it is binary compressed data even if the CCD 3 has 3 million pixels (2048 × 1536 pixels). If the defect address information has coordinates on the CCD, if the number is 60,000 as described above, then 60,000 × (2 bytes (abscissa address, max 2048) +2 bytes (ordinate address, max 1536 )), And requires a large memory capacity of 240 KB.

When the camera photographer performs photographing, the image data of the CCD 3 is stored in the memory 9.
Since only the CCD image data is digitized in the memory, for example, digital data of the R, G, B CCD 3 Bayer array as it is is stored. On the other hand, the defect position binary compressed data for defect correction in the built-in nonvolatile memory 17 is decompressed by the binary processing unit of the DSP unit 5 and then stored in the memory 9. The decompressed binary data is converted into the same luminance and color difference format as the multi-valued image by the binary processing unit in the same manner as the reproduction processing in the character photographing mode described above, but in the luminance information of 2048 × 1536 pixels, The value of the pixel address data corresponding to the pixel having no defect is zero, and the value of the pixel address data corresponding to the defective pixel is 255.

  As described above, in the memory 9, two pieces of CCD image data of 2048 × 1536 pixels and defect correction luminance data of 2048 × 1536 pixels having a defect position value of 255 are created. Next, the defect corresponding position (X, Y) of the defect correction luminance data is searched, and the pixel data of the X, Y coordinates on the CCD image data is corrected. The correction is performed by the defect correction processing unit 26 of FIG. Specifically, the correction is performed by overwriting the data of the coordinates (X, Y) with the coordinates (X + 2, Y) of the CCD imaging data. X and X + 2 are the closest pixels of the same color filter in the CCD Bayer array. The CCD image data corrected in this way is processed by the DSP unit 5 and converted into still image data.

Next, a second embodiment related to claims 3 and 4 will be described.
As described above, the number of defective pixels due to dark current increases when exposed for a long time. Since the defect position binary compressed data in the memory 9 has a small number of data, in the present embodiment, for example, a plurality of defect position binary compressed data for 2 seconds exposure shooting, 4 seconds exposure shooting, and 8 seconds exposure shooting are used. have.
According to the present embodiment, by properly using each of the above data according to the exposure time taken, it is possible to correct an optimal number of pixels without overcorrection.

Next, a third embodiment related to claim 5 will be described.
The method using the defect position binary compressed data requires an initial data expansion process and a search operation for the defect corresponding positions X and Y for each photographing. On the other hand, the number of defective pixels that becomes a problem with a short exposure of about 1 second is at the level of 60, and the coordinate data is sufficiently small. In this embodiment, the short-exposure coordinate data is directly used as the built-in nonvolatile memory 17. Is stored.

  According to the present embodiment, the coordinate formula correction process using the short exposure coordinate data is performed particularly when shooting is not performed at a long time, and the defect image formula correction according to the first embodiment is performed when the exposure is performed at a long time. Processing should be done. Since shooting was originally performed for a long time, there is no disadvantage to the user even if it takes some time due to the defect image type correction processing.

A fourth embodiment relating to claim 6 will be described.
Since the imaging apparatus has a binary image shooting function called a character shooting mode as a shooting mode, the present embodiment provides a user with a defect data update mode and the like. In this defect data update mode, shooting is performed with an exposure time of 4 seconds while the shutter is closed, and the captured data is stored in the memory 9. At the same time, 2048 × 1536 brightness zero data is created in the memory by a program. This is a plain image. This creation processing is performed by the plane image creation processing unit 31 in the defect position binary compressed data processing unit 30 of FIG.

Next, the pixel data of the imaging data is compared with a predetermined threshold value M, and if it is a value equal to or greater than M, it is regarded as a defective pixel and the data at the corresponding address on the plane image side is set to 255. The above processing is performed for all pixel positions captured by the defect position binary image creation processing unit 32. When completed, the plane image is binary-compressed and stored in the built-in nonvolatile memory 17 as defect position binary compressed data for an exposure time of 4 seconds. In this example, it is set to 4 seconds, but this time may be selected by the user. The threshold value M may be set for R, G, and B according to the color filter of the image data.
According to the present embodiment, the defect position binary compressed data can be created in the imaging apparatus itself.

  The program for the CPU 8 to execute the processing in each of the above-described embodiments constitutes a program according to the present invention. As a recording medium for recording the program, a semiconductor storage device, an optical and / or magnetic storage device, or the like can be used. By using such a program and recording medium in a system having a configuration different from that of each embodiment and causing the CPU to execute the program, substantially the same effects as those of the present invention can be obtained.

It is a block diagram which shows the structure of the imaging device which concerns on embodiment of this invention. It is a flowchart which shows the operation | movement and image processing which concern on the form of embodiment of this invention. It is a block diagram which shows the software structure of the imaging device which concerns on embodiment of this invention.

Explanation of symbols

3 CCD
5 DSP section 8 CPU
DESCRIPTION OF SYMBOLS 9 Memory 17 Built-in non-volatile memory 25 Character recording process part 26 Defect correction process part 30 Defect position binary compression data process part 31 Plain image creation process part 32 Defect position binary image creation process part

Claims (10)

Imaging means for capturing a subject image and outputting imaging data;
Storage means for storing defect position binary compressed data indicating in binary the position where the defective pixel of the image sensor in the imaging means is located and where it is not;
Decompression means for decompressing the stored defect position binary compressed data;
An image pickup apparatus comprising: correction means for correcting defective pixel data in the image pickup data based on the extended defect position binary data.   The correction means detects the position of the defective pixel in the imaging data based on the expanded defect position binary data, and corrects the defective pixel data at the detected position with pixel data in the vicinity of the pixel. The imaging apparatus according to claim 1, wherein:   The storage means stores a plurality of the defect position binary compressed data, and the correction means uses one of the plurality of defect position binary compressed data used in accordance with an imaging parameter at the time of imaging. The imaging apparatus according to claim 1, wherein:   The plurality of defect position binary compressed data is set according to an exposure time as an imaging parameter at the time of imaging, and the correction unit is configured to store the plurality of defect position binary compressed data used according to the exposure time. The imaging apparatus according to claim 3, wherein one of them is used.   The storage means stores defect coordinate data indicating coordinates of defective pixels of the image sensor, and the correction means uses either the defect position binary compressed data or the defect coordinate data as an imaging parameter at the time of imaging. The imaging apparatus according to claim 4, wherein the imaging apparatus is used in accordance with the above. Creating means for creating a plain image in which all pixel data has zero brightness data;
Each pixel data of the imaging data is compared with a predetermined threshold, pixel data equal to or higher than the threshold is used as defective pixel data, data at a corresponding position in the plane image is replaced with predetermined luminance, and the replaced plane image data is binary. 2. The imaging apparatus according to claim 1, further comprising defect position binary compressed data creating means for compressing and creating the processed data as the defect position binary compressed data. An imaging process for imaging a subject image and outputting imaging data;
A readout process of reading out the defect position binary compressed data indicating the position where the defective pixel of the image sensor in the imaging process is located and the position where the defective pixel is not present from the storage unit;
A decompression process for decompressing the read defect position binary compressed data;
A program for causing a computer to execute correction processing for correcting defective pixel data in the imaging data based on the expanded defect position binary data.   8. The defect position binary compressed data is plural, and one of the plurality of defect position binary compressed data used according to an imaging parameter at the time of imaging is used by the correction process. The listed program.   The computer executes read processing for reading out defect coordinate data indicating the coordinates of the defective pixel of the image sensor from the storage unit, and the correction processing performs imaging of either the defect position binary compressed data or the defect coordinate data. 8. The program according to claim 7, wherein the program is used according to an imaging parameter at the time. A creation process for creating a plain image in which all pixel data is zero-luminance data;
A comparison process for comparing each pixel data of the imaging data with a predetermined threshold;
A replacement process in which pixel data equal to or greater than the threshold is used as defective pixel data, and data at a corresponding position in the plane image is replaced with a predetermined luminance.
A compression process for performing binary compression on the replaced plain image data;
8. The program according to claim 7, causing a computer to execute defect position binary compressed data creation processing for creating the compressed data as the defect position binary compressed data.

Images