JP2007174124A - Imaging apparatus and correction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology capable of correcting outputs from pixels adjacent to saturated defective pixels by detecting the defective pixels without the need for addition of a special apparatus configuration. <P>SOLUTION: An imaging apparatus disclosed herein including an imaging element (14) having a plurality of pixels each including a light receiving section and a light shield section and a nonvolatile memory (56) for storing at least positional information of a defective pixel of the imaging element acquired in advance and a reference output level, detects a defective pixel existing in the light shield section and whose output level at photographing of an object is less than a preset saturation level (S202), adjusts the reference output level of a defective pixel existing in the light receiving section on the basis of the reference output level of the detected defective pixel and the output level at the photographing (S205), establishes pixels adjacent to the defective pixel the adjusted reference output level of which is a second saturation level or over for correction object pixels (S207), and corrects output levels at the photographing outputted from the defective pixel and the correction object pixels (S209). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a correction method, and more particularly to a technique for correcting an output of a defective pixel of an imaging element in an imaging apparatus using the imaging element.

  Conventionally, solid-state imaging devices such as CCD and CMOS sensors are generally used in imaging devices such as digital cameras and video cameras. In this solid-state imaging device, it is known that defective pixels generated in the manufacturing process are one of the factors that lower the image quality and reduce the manufacturing yield. Since it is difficult to completely eliminate defective pixels, it is generally known to improve image quality by performing interpolation processing using pixels around the defective pixels.

  For example, as a technique for correcting a signal output from a defective pixel, for example, a method described in “Prior Art” in Patent Document 1 is known. First, a defective pixel is determined using a standard charge accumulation time and an output value obtained by exposing the solid-state image sensor under a predetermined condition when the solid-state image sensor is shipped from a factory. Then, information such as the positional information and output level of the defective pixel acquired at that time is stored, and at the time of imaging, it is adjacent to the defective pixel based on the stored positional information and output level information of the defective pixel. Interpolation of the output of the defective pixel is performed using the output level of the pixel to be processed.

  On the other hand, the following techniques have been proposed which are performed during long exposure. First, a dark image is photographed prior to actual photographing, and those having a predetermined output or more are extracted and stored as defective pixels from the dark image, and the extracted defective pixels are compared with the image obtained by actual photographing. to correct. For the pixels other than the extracted defective pixels, so-called blacking is performed in which the output level of the corresponding pixel in the dark image is subtracted from the output level of each pixel obtained by actual photographing. By correcting in this way, even when the output level of a defective pixel that occurs during long-time exposure increases, the system can perform correction with minimal image degradation without being broken (for example, , See Patent Document 2).

  It also has multiple correction data for correcting the output of defective pixels, and selects the most appropriate data according to shooting conditions such as charge accumulation time and shooting sensitivity, and shooting environment such as temperature, and corrects the shot image. A technique for performing the above has also been proposed (see, for example, Patent Document 1).

JP 2003-333435 A JP 2001-028713 A

  Some defective pixels are caused by dark current, and the output level varies greatly depending on temperature and charge accumulation time. The output level of a defective pixel due to such dark current increases as the dark current increases due to high temperature or long exposure (long exposure).

  From the above-described defective pixel caused by dark current, the following phenomenon occurs when the charge accumulated by the increase in dark current reaches the saturation level of the pixel or more. That is, several percent of the pixel exceeding the saturation level of the pixel leaks into the adjacent pixel. As a result, as shown in FIG. 5, the output level of the adjacent pixel becomes unnecessarily high, resulting in a cross-shaped defective pixel region. End up. In FIG. 5, coordinates (n, n) are defective pixels caused by dark current, and coordinates (n-1, n), (n + 1, n), (n, n-1) adjacent to the upper, lower, left, and right sides thereof. , The pixel located at (n, n + 1) also shows a phenomenon in which the output level is increased due to the leakage of charges.

  In the method described in Patent Document 2, since it takes time to capture a dark image with the same charge accumulation time as that for shooting before shooting to extract scratches, shooting is actually performed after shooting is instructed. The release time lag will be longer. In particular, the release time lag becomes very long in long-time shooting that causes leakage of charge into adjacent pixels. However, if the release time lag is to be shortened, there is a problem in that it is not possible to extract adjacent pixels into which charges have leaked, and correction cannot be performed.

  Further, when the correction as shown in Patent Document 1 is performed on the cross-shaped defective pixel region caused by the dark current as shown in FIG. 5, there is the following problem. That is, since the output level changes depending on the temperature and charge accumulation time, it is necessary to have information and address of mobilization for correcting adjacent pixels for each defective pixel for each condition, and the memory capacity to be stored Was also necessary.

  The present invention has been made in view of the above problems, and an object of the present invention is to detect a saturated defective pixel and correct an output from a pixel adjacent thereto without adding a special device configuration. To do.

  In order to achieve the above object, an imaging apparatus according to the present invention includes at least an image sensor including a plurality of pixels having a light receiving unit and a light shielding unit, position information and a reference output level of a defective pixel of the image sensor acquired in advance. Among the defective pixels stored in the storage means and the defective pixels stored in the storage means, the defective pixels in the light-shielding portion, and the output level at the time of photographing the subject is less than a preset first saturation level Based on the detection means for detecting the pixel, the reference output level of the detected defective pixel, and the output level at the time of shooting, the defective pixel stored in the storage means in the light receiving unit And adjusting means for adjusting the reference output level, and determining whether the reference output level adjusted by the adjusting means is equal to or higher than a second saturation level that is a saturation level of the pixel. Setting means for setting a pixel adjacent to a defective pixel whose reference output level is equal to or higher than the second saturation level as a correction target pixel, a defective pixel stored in the storage means, and a correction target set by the setting means Correction means for correcting the output level at the time of shooting output from the pixel.

  In an imaging apparatus comprising: an imaging device having a plurality of pixels each having a light receiving unit and a light shielding unit; and a storage unit that holds at least position information and a reference output level of defective pixels of the imaging device acquired in advance. The correction method for correcting the output level output from the pixel is a defective pixel stored in the storage means, which is the defective pixel in the light-shielding portion, and the output level at the time of photographing the subject is set in advance. Based on the detection step of detecting a defective pixel that is less than one saturation level, the reference output level of the detected defective pixel, and the output level at the time of photographing, the defective pixels stored in the storage means An adjustment step for adjusting a reference output level of a defective pixel in the light receiving unit, and a reference output level adjusted in the adjustment step is a saturation level of the pixel. A setting step for setting a pixel adjacent to a defective pixel whose adjusted reference output level is equal to or higher than the second saturation level as a correction target pixel; and And a correction step of correcting the output level at the time of photographing output from the stored defective pixel and the correction target pixel set in the setting step.

  According to the present invention, it is possible to detect a saturated defective pixel and correct an output from a pixel adjacent thereto without adding a special device configuration.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus having an image processing function according to an embodiment of the present invention. In the present embodiment, a digital camera will be described as an example of the imaging device. As other imaging devices, there are digital video cameras, camera-equipped mobile terminals (including camera-equipped mobile phones), scanners, etc., as long as they can convert an optical image of an object and output an electrical image signal. The present invention can be applied.

  As shown in FIG. 1, the imaging apparatus according to the present embodiment mainly includes a camera body 100 and an interchangeable lens type lens unit 300.

  In the lens unit 300, 310 is an imaging lens composed of a plurality of lenses, 312 is a diaphragm, and 306 is a lens mount that mechanically couples the lens unit 300 to the camera body 100. The lens mount 306 includes various functions for electrically connecting the lens unit 300 to the camera body 100. Reference numeral 320 denotes an interface for connecting the lens unit 300 to the camera body 100 in the lens mount 306, and 322 denotes a connector for electrically connecting the lens unit 300 to the camera body 100.

  The connector 322 transmits a control signal, a status signal, a data signal, and the like between the camera body 100 and the lens unit 300, and also has a function of supplying or supplying a current of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numeral 340 denotes an aperture control unit that controls the aperture 312 in cooperation with a shutter control unit 40 that controls a shutter 12 of the camera body 100, which will be described later, based on photometric information from the photometry unit 46. Reference numeral 342 denotes a focus control unit that controls focusing of the imaging lens 310, and 344 denotes a zoom control unit that controls zooming of the imaging lens 310.

  A lens system control circuit 350 controls the entire lens unit 300. The lens system control circuit 350 includes a memory for storing operation constants, variables, programs, and the like. Further, a non-volatile memory that holds identification information such as a number unique to the lens unit 300, management information, function information such as an open aperture value, a minimum aperture value, and a focal length, and current and past setting values is also provided.

  Next, the configuration of the camera body 100 will be described.

  Reference numeral 106 denotes a lens mount that mechanically couples the camera body 100 and the lens unit 300, and reference numerals 130 and 132 denote mirrors, which guide light beams incident on the imaging lens 310 to the optical viewfinder 104 by a single-lens reflex system. The mirror 130 may be either a quick return mirror or a half mirror. Reference numeral 12 denotes a shutter having a diaphragm function, and reference numeral 14 denotes an image sensor that converts an optical image into an electric signal. In the present embodiment, a Bayer array CMOS area sensor is used as the image sensor 14. The light beam incident on the imaging lens 310 is guided by the single-lens reflex system through the aperture 312, the lens mounts 306 and 106, the mirror 130, and the shutter 12 as light amount limiting means, and forms an optical image on the imaging element 14. The image sensor 14 generally includes a light receiving portion that receives incident light rays and a light shield portion (OB portion) that is shielded from light, and the defective pixel correction in the present invention uses the output of the defective pixel in the OB portion. Then, it is determined whether or not charge leakage into the adjacent pixel as shown in FIG. 5 occurs.

  Reference numeral 15 denotes an AGC (Automatic Gain Controller) circuit including a gain control amplifier that adjusts the gain of a signal output from the image sensor 14 by setting ISO sensitivity. Reference numeral 16 denotes an A / D converter that converts an analog signal output from the AGC circuit 15 into a digital signal. A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 18, and the D / A converter 26, respectively, and is controlled by the memory control circuit 22 and the system control circuit 50.

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs a predetermined calculation process using the image data output from the A / D converter 16. Then, based on the obtained calculation result, the system control circuit 50 controls the shutter control unit 40 and the distance measurement control unit 42, TTL (through the lens) type auto focus (AF) processing, automatic exposure. (AE) processing and flash pre-emission (EF) processing are performed. Further, the image processing circuit 20 performs predetermined arithmetic processing using the image data output from the A / D converter 16, and also performs TTL auto white balance (AWB) processing based on the obtained arithmetic result. ing. Furthermore, as will be described later, based on the information (scratch correction data) of the defective pixel stored in the nonvolatile memory 56, correction processing is performed on the output of the defective pixel.

  In the example shown in FIG. 1 in the present embodiment, since the distance measurement control unit 42 and the photometry control unit 46 are provided exclusively, the system control circuit 50 uses the distance measurement control unit 42 and the photometry control unit 46. Then, AF processing, AE processing, and EF processing may be performed. Further, the image processing circuit 20 may be used so that the AF processing, AE processing, and EF processing are not performed. The AF processing, AE processing, and EF processing are performed using the distance measurement control unit 42 and the photometry control unit 46, and the AF processing, AE processing, and EF processing are performed using the image processing circuit 20. It is good also as a structure to perform.

  A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. Image data output from the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20, the memory control circuit 22, or only through the memory control circuit 22.

  Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, and 28 denotes an image display unit made up of a TFT type LCD or the like. Display image data written in the image display memory 24 is stored in the D / A converter 26. Via the image display unit 28. An electronic viewfinder (EVF) function can be realized by sequentially displaying captured image data using the image display unit 28. Further, the image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50, and when the display is turned off, the power consumption of the camera body 100 can be significantly reduced. it can.

  Reference numeral 30 denotes a memory for storing captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and a moving image for a predetermined time. This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 30 can also be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses / decompresses image data using a known compression method such as adaptive discrete cosine transform (ADCT). The compression / decompression circuit 32 reads an image stored in the memory 30, performs a compression process or an expansion process, and writes the processed data to the memory 30 again.

  Reference numeral 40 denotes a shutter control unit that controls the shutter 12 in cooperation with an aperture control unit 340 that controls the aperture 312 based on photometric information from the photometry control unit 46. Reference numeral 42 denotes a distance measurement control unit for performing AF (autofocus) processing. A light beam incident on the imaging lens 310 in the lens unit 300 is incident as a single-lens reflex system through a diaphragm 312, lens mounts 306 and 106, a mirror 130, and a distance measuring sub-mirror (not shown), thereby forming an optical image. The in-focus state of the imaged image is measured.

  A photometry control unit 46 performs AE (automatic exposure) processing. A light beam incident on the photographing optical system 310 in the lens unit 300 is incident on the single lens reflex system through the stop 312, the lens mounts 306 and 106, the mirror 130 and the photometric submirror (not shown), thereby forming an optical image. The exposure state of the image formed is measured. A flash 48 has an AF auxiliary light projecting function and a flash light control function. The photometry control unit 46 also has an EF (flash dimming) processing function in cooperation with the flash unit 48. In addition, the photometry control unit 46 in the present embodiment performs photometry using a well-known divided photometry method including a plurality of photometry areas, and measures a plurality of points (for example, 35 points) in a shooting scene, and exposure is performed according to the result. Shall be determined.

  As described above, exposure control and AF control may be performed based on the calculation result calculated by the image processing circuit 20 using the image data from the A / D converter 16. In this case, the system control circuit 50 can perform exposure control and AF control using the video TTL method for the shutter control unit 40, the aperture control unit 340, and the focus control unit 342.

  The AF control may be performed using the measurement result obtained by the distance measurement control unit 42 and the calculation result obtained by calculating the image data from the A / D converter 16 by the image processing circuit 20. Furthermore, exposure control may be performed using the measurement result obtained by the photometry control unit 46 and the calculation result obtained by calculating the image data from the A / D converter 16 by the image processing circuit 20.

  Reference numeral 50 denotes a system control circuit that controls the entire camera body 100, and incorporates a known CPU. A memory 52 stores constants, variables, programs, and the like for operating the system control circuit 50.

  Reference numeral 54 denotes a notification unit for notifying the outside of an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50. As the notification unit 54, for example, a display unit that performs visual display using an LCD, an LED, or the like, or a sound generation element that performs voice notification, etc., is used, and the notification unit 54 is configured by a combination of one or more of these. In particular, in the case of the display unit, it is installed at one or a plurality of locations near the operation unit 70 of the camera body 100 that are easy to see. In addition, the notification unit 54 has a part of its function installed in the optical viewfinder 104.

  Among the display contents of the notification unit 54, the following are displayed on the LCD or the like. First, there are displays related to the shooting mode such as single shot / continuous shooting display, self-timer display, and the like. In addition, there are displays relating to recording such as compression rate display, recording pixel number display, recording number display, and remaining image number display. In addition, there are displays relating to shooting conditions such as shutter speed display, aperture value display, exposure correction display, flash display, and red-eye reduction display. In addition, there are a macro shooting display, a buzzer setting display, a clock battery remaining amount display, a battery remaining amount display, an error display, an information display by a multi-digit number, and an attachment / detachment state display of the recording media 200 and 210. Furthermore, the attachment / detachment state display of the lens unit 300, the communication I / F operation display, the date / time display, the display indicating the connection state with the external computer, and the like are also performed.

  In addition, among the display contents of the display unit 54, examples of what is displayed in the optical finder 104 include the following. In-focus display, photographing preparation completion display, camera shake warning display, flash charge display, flash charge completion display, shutter speed display, aperture value display, exposure correction display, recording medium writing operation display, and the like.

  Further, among the display contents of the display unit 54, examples of what is displayed by an LED or the like include the following. In-focus display, shooting preparation completion display, camera shake warning display, flash charge display, flash charge completion display, recording medium writing operation display, macro shooting setting notification display, secondary battery charge display, and the like.

  Among the display contents of the display unit 54, what is displayed on a lamp or the like includes, for example, a self-timer notification lamp. This self-timer notification lamp may be shared with AF auxiliary light.

  Reference numeral 56 denotes an electrically erasable / recordable non-volatile memory in which programs and the like to be described later are stored. For example, an EEPROM or the like is used. The nonvolatile memory 56 stores various parameters, set values such as ISO sensitivity, and defect correction data related to defective pixels used when correction processing is performed on the output of defective pixels.

  Note that the defect correction data relating to the defective pixel stores at least address data indicating coordinates (X, Y) and output level information as position information indicating the position of the defective pixel. In this embodiment, the defect correction data is obtained by extracting defective pixels from image data obtained by exposing the CMOS area sensor at a standard environmental temperature and a standard charge accumulation time at a CMOS area sensor manufacturing factory. Data. Therefore, defect correction data corresponding to each sensor is stored. However, the defect correction data of the present invention is not limited to this, and information relating to defective pixels may be newly created and stored after the CMOS area sensor is incorporated into the camera.

  Reference numerals 60, 62, 64, 66, 68, 69 and 70 are operation means for inputting various operation instructions of the system control circuit 50, such as switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Consists of a single or a plurality of combinations.

  Here, a specific description of these operating means will be given.

  Reference numeral 60 denotes a mode dial switch which can be used to switch and set each function shooting mode such as an automatic shooting mode, a program shooting mode, a shutter speed priority shooting mode, an aperture priority shooting mode, a manual shooting mode, and a depth of focus priority (depth) shooting mode. it can. In addition, each function shooting mode such as portrait shooting mode, landscape shooting mode, close-up shooting mode, sports shooting mode, night view shooting mode, panoramic shooting mode, and the like can be switched and set.

  Reference numeral 62 denotes a shutter switch SW1, which is turned on while a shutter button (not shown) is being operated (for example, half-pressed), and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing.

  A shutter switch SW2 64 is turned on when a shutter button (not shown) is completed (for example, fully pressed), and instructs the start of a series of processes including exposure processing, development processing, and recording processing. First, in the exposure process, the signal read from the image sensor 14 is written into the memory 30 via the A / D converter 16 and the memory control circuit 22, and further the calculation in the image processing circuit 20 and the memory control circuit 22 is performed. Development processing using is performed. Further, in the recording process, image data is read from the memory 30, compressed by the compression / decompression circuit 32, and written to the recording medium 200 or 210.

  Reference numeral 66 denotes a playback switch, which instructs to start a playback operation for reading an image shot in the shooting mode state from the memory 30 or the recording medium 200 or 210 and displaying it on the image display unit 28. Further, the playback switch 66 may be used to set each function mode such as a playback mode, a multi-screen playback / erase mode, and a PC connection mode.

  68 is a single-shot / continuous-shot switch. When the shutter switch SW2 (64) is pressed, a single-shot mode in which one frame is shot and the camera is in a standby state, and while the shutter switch SW2 (64) is pressed, continuous It is possible to set a continuous shooting mode that continues shooting.

  Reference numeral 69 denotes an ISO sensitivity setting switch, and the ISO sensitivity can be set by changing the gain setting in the AGC circuit 15.

  An operation unit 70 includes various buttons and a touch panel. For example, the menu button, set button, macro button, multi-screen playback pagination button, flash setting button, single shooting / continuous shooting / self-timer switching button, menu movement + (plus) button, menu movement-(minus) button Including. Further, a playback image movement + (plus) button, a playback image movement-(minus) button, a shooting image quality selection button, an exposure correction button, a date / time setting button, and the like are included. Each of the functions of the plus button and the minus button can be selected more easily by providing a rotary dial switch.

  In addition, a selection / switch button that sets selection and switching of various functions when shooting and playback in panorama mode, etc., and determination and execution of various functions when shooting and playback in panorama mode and the like are set. There is a decision / execution button. In addition, there is an image display ON / OFF switch for setting ON / OFF of the image display unit 28, and a quick review ON / OFF switch for setting a quick review function for automatically reproducing image data taken immediately after shooting. Further, there is a compression mode switch that is a switch for selecting a compression rate of JPEG compression, or for selecting a CCD RAW mode in which a signal from an image sensor is directly digitized and recorded on a recording medium. In addition, there is an AF mode setting switch that can set a one-shot AF mode and a servo AF mode. In the one-shot AF mode, an autofocus operation is started when the shutter switch SW1 (62) is pressed, and once focused, the focused state is maintained. In the servo AF mode, the autofocus operation is continued while the shutter switch SW1 (62) is being pressed.

  Reference numeral 72 denotes a power switch, which can be switched between power-on and power-off modes of the camera body 100. In addition, the power on and power off settings of various accessory devices such as the lens unit 300, the external strobe, and the recording media 200 and 210 connected to the camera body 100 can be switched.

  A power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like. The power supply control unit 80 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC-DC converter based on the detection result and the instruction of the system control circuit 50, and requires the necessary voltage. It is supplied to each part including the recording medium for a period.

  Reference numerals 82 and 84 denote connectors, 86 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, Li-ion battery or Li polymer battery, an AC adapter, or the like.

  Reference numerals 90 and 94 denote interfaces with recording media such as memory cards and hard disks, and reference numerals 92 and 96 denote connectors for connecting to recording media such as memory cards and hard disks. A recording medium attachment / detachment detection circuit 98 detects whether or not the recording medium 200 or 210 is attached to the connectors 92 and / or 96.

  Although the present embodiment has been described as having two systems of interfaces and connectors for attaching the recording medium, the interface and connectors for attaching the recording medium may have a single or a plurality of systems. . Moreover, it is good also as a structure provided with combining the interface and connector of a different standard.

  As the interface and the connector, it is possible to configure using interfaces that comply with various storage medium standards. For example, a PCMCIA (Personal Computer Memory Card International Association) card, a CF (Compact Flash (registered trademark)) card, an SD card, or the like. When the interfaces 90 and 94 and the connectors 92 and 96 are configured using a standard conforming to a PCMCIA card, a CF (registered trademark) card, or the like, various communication cards can be connected. Communication cards include LAN cards, modem cards, USB (Universal Serial Bus) cards, and IEEE (Institute of Electrical and Electronic Engineers) 1394 cards. In addition, there are P1284 card, SCSI (Small Computer System Interface) card, PHS, and the like. By connecting these various communication cards, image data and management information attached to the image data can be transferred to and from other computers and peripheral devices such as a printer.

  Reference numeral 104 denotes an optical viewfinder, which guides a light beam incident on the imaging lens 310 through an aperture 312, lens mounts 306 and 106, and mirrors 130 and 132 by a single-lens reflex method, and forms an optical image for display. Is possible. Thereby, it is possible to perform imaging using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. In addition, in the optical viewfinder 104, some functions of the notification unit 54, for example, in-focus state, camera shake warning, flash charging, shutter speed, aperture value, exposure correction, and the like are displayed.

  A communication unit 110 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication. Reference numeral 112 denotes a connector for connecting the camera body 100 to another device by the communication unit 110 or an antenna in the case of wireless communication.

  Reference numeral 120 denotes an interface for connecting the camera body 100 to the lens unit 300 in the lens mount 106.

  A connector 122 electrically connects the camera body 100 to the lens unit 300. Whether or not the lens unit 300 is attached to the lens mount 106 and / or the connector 122 is detected by a lens attachment / detachment detection unit (not shown). The connector 122 transmits a control signal, a status signal, a data signal, and the like between the camera body 100 and the lens unit 300, and has a function of supplying currents of various voltages. Further, the connector 122 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numerals 200 and 210 denote recording media such as a memory card and a hard disk. The recording media 200 and 210 include recording units 202 and 212 configured by a semiconductor memory, a magnetic disk, and the like, interfaces 204 and 214 with the camera main body 100, and connectors 206 and 216 for connecting with the camera main body 100, respectively. Yes.

  As the recording media 200 and 210, a PCMCIA card, a memory card such as a compact flash (registered trademark), a hard disk, or the like can be used. Of course, it may be constituted by a micro DAT, a magneto-optical disk, an optical disk such as CD-R or CD-WR, a phase change optical disk such as DVD, or the like.

  Note that although the imaging device in the present embodiment has been described as a single-lens reflex digital camera with interchangeable lenses, a so-called digital compact camera or the like in which a lens or a lens barrel is integrated with the main body may be used. Good.

  Next, with reference to FIGS. 2 to 4, the operation of the imaging apparatus having the above-described configuration in the embodiment of the present invention will be described.

  2 and 3 are flowcharts showing a photographing processing procedure of the imaging apparatus according to the present embodiment. This processing program is stored in a storage medium such as the nonvolatile memory 56, loaded into the memory 52, and executed by the CPU in the system control circuit 50.

  In FIG. 2, the system control circuit 50 initializes flags, control variables, and the like by turning on the power supply such as battery replacement (step S <b> 101), and performs necessary initial settings for each part of the camera body 100. Next, the system control circuit 50 determines the set position of the power switch 72 and determines whether or not the power switch 72 is set to power OFF (step S102). When the power switch 72 is set to power OFF, the system control circuit 50 changes the display of each display unit to the end state, and sets necessary parameters, setting values, and setting modes including flags and control variables to nonvolatile. Record in memory 56. Further, the system control circuit 50 performs predetermined end processing such as shutting off unnecessary power of each part of the camera body 100 including the image display unit 28 by the power control unit 80 (step S103), and then returns to step S102.

  When the power switch 72 is set to ON in step S102, the system control circuit 50 has a problem in the operation of the camera body 100 due to the remaining capacity and operation status of the power source 86 constituted by a battery or the like by the power control unit 80. Determine whether or not. If there is a problem (NO in step S104), a predetermined warning is displayed by image or sound using the notification unit 54 (step S105), and then the process returns to step S102.

  On the other hand, if there is no problem with the power source 86 (YES in step S104), the setting position of the mode dial switch 60 is determined, and it is determined whether or not the mode dial switch 60 is set to the shooting mode (step S106). If the mode dial switch 60 is set to another mode, the system control circuit 50 executes processing according to the selected mode (step S107), and returns to step S102 when the processing is completed. When the mode dial switch 60 is set to the shooting mode, the process proceeds to step S108.

  In step S108, it is determined whether or not the recording medium 200 or 210 is loaded. If it is mounted, the management information of the image data recorded on the recording medium 200 or 210 is acquired. Further, it is determined whether or not the operation state of the recording medium 200 or 210 has a problem in the operation of the camera body 100, particularly the recording / reproducing operation of the image data with respect to the recording medium 200 or 210. If it is determined that there is a problem (NO in step S108), the process proceeds to step S105 to display a predetermined warning by an image or sound using the notification unit 54 (step S105), and then returns to step S102. If it is determined that there is no problem (YES in step S108), the process proceeds to step S109.

  In step S109, the notification unit 54 is used to notify various setting states of the camera body 100 using images and sounds. Here, when the image display switch of the image display unit 28 is ON, various setting states of the camera body 100 may be displayed on the image display unit 28.

  Next, it is determined whether or not the shutter switch SW1 is pressed in step S110. If the shutter switch SW1 is not pressed, the process returns to step S102. If the shutter switch SW1 is pressed, the process proceeds to step S111.

  In step S111, the system control circuit 50 performs a distance measurement process to adjust the focus of the imaging lens 310 to the subject, perform a light measurement process, and determine a diaphragm value and a shutter speed. As a result of photometry, if necessary, the flash flag is set and the flash is set.

  When the distance measurement / photometry processing is completed, the process proceeds to step S112, and it is determined whether or not the shutter switch SW2 is pressed. If the shutter switch SW2 is not pressed (OFF in step S112) and the shutter switch SW1 is also released (OFF in step S113), the process returns to step S102.

  If the shutter switch SW2 is pressed (ON in step S112), the system control circuit 50 determines whether or not the memory 30 has an image storage buffer area capable of storing captured image data (step S114). When it is determined that there is no area where new image data can be stored in the image storage buffer area of the memory 30, the process proceeds to step S 115, and a predetermined warning is displayed by an image or sound using the notification unit 54. The process returns to step S102.

  The case where there is no area where new image data can be stored in the image storage buffer area of the memory 30 can be considered as follows, for example. Immediately after performing the maximum number of continuous shots that can be stored in the image storage buffer area of the memory 30, the first image to be read from the memory 30 and written to the storage medium 200 or 210 is still recorded on the storage medium 200 or 210. There are things that are not. In such a case, one empty area cannot be secured in the image storage buffer area of the memory 30.

  When the captured image data is compressed and stored in the image storage buffer area of the memory 30, it is necessary to consider that the amount of image data after compression differs depending on the compression mode setting. Then, it is determined in the process of step S114 whether or not the storable area is on the image storage buffer area of the memory 30.

  If it is determined that there is an area capable of storing new image data in the memory 30, the process proceeds to step S116.

  In step S116, the system control circuit 50 executes shooting processing. In this photographing process, an image signal accumulated for a predetermined time is stored in memory from the image sensor 14 via the AGC circuit 15, A / D converter 16, image processing circuit 20, memory control circuit 22, or directly from the A / D converter 16. The captured image data is written into a predetermined area of the memory 30 via the control circuit 22.

  In step S117, a defective pixel correction process is performed on the image data written in the memory 30. First, the system control circuit 50 reads address data and an output level as defect correction data relating to defective pixels written in the nonvolatile memory 56. The address data is coordinates (X, Y) indicating position information of defective pixels in all pixels. Then, correction processing is performed on the image data written in the memory 30 by interpolation from adjacent pixels. The details of the correction processing performed here will be described later with reference to FIG.

  In step S118, the system control circuit 50 reads out part of the image data written in a predetermined area of the memory 30 via the memory control circuit 22 and performs WB integration calculation processing, OB (optical Black) Performs integral calculation processing. The calculation result is stored in the internal memory of the system control circuit 50 or the memory 52. The system control circuit 50 reads captured image data written in a predetermined area of the memory 30 using the memory control circuit 22 and, if necessary, the image processing circuit 20. Then, using the calculation results stored in the internal memory of the system control circuit 50 or the memory 52, various development processes including AWB processing, gamma conversion processing, and color conversion processing are performed.

  In step S119, the system control circuit 50 reads the image data written in the predetermined area of the memory 30, and the compression / decompression circuit 32 performs image compression processing according to the set mode. Then, image data that has been shot and finished a series of processing is written into the empty image portion of the image storage buffer area of the memory 30.

  In step S <b> 120, the system control circuit 50 reads processed image data stored in the image storage buffer area of the memory 30. Then, the recording process for writing the read image data to the recording medium 200 or 210 such as a memory card or a compact flash (registered trademark) card via the interface 90 or 94 and the connector 92 or 96 is started. This recording start process is performed on the image data every time new image data that has been shot and finished a series of processes is newly written in the empty image portion of the image storage buffer area of the memory 30.

  Note that while writing image data to the recording media 200 and 201, in order to indicate that the writing operation is being performed, a recording medium writing operation display such as blinking an LED is performed on the display unit 54, for example.

  In step S121, the system control circuit 50 determines whether or not the shutter switch SW1 is pressed. If the shutter switch SW1 has been released, the process returns to step S102. If the shutter switch SW1 has been pressed, the process returns to step S112 to prepare for the next shooting. Thereby, the system control circuit 50 finishes a series of photographing operations and returns to step S102.

  Next, the defective pixel correction process performed in step S117 of FIG. 3 will be described with reference to the flowchart of FIG.

  First, in step S201, the scratch correction data stored in the nonvolatile memory 56 is read. Here, the address data and the output level indicating the coordinates (X, Y) of the defective pixel are read out.

  Next, in step S202, among the defective pixels stored in the defect correction data read out in step S201, a defective pixel in the light shielding portion (OB portion) of the image sensor 14 is selected using the address data. Then, the output level of the selected defective pixel is read out from the image data written in the memory 30 by the photographing process in step S116 of FIG.

  Next, in step S203, it is determined whether the output level at the time of shooting read from the memory 30 in step S202 is less than the saturation level of the A / D converter 16. If it is less than the saturation level, the process proceeds to step S204, and the read output level at the time of photographing is determined to be used for calculation described later. On the other hand, if the read output level has reached the saturation level of the A / D converter 16, the process returns to step S202, and another defective pixel in the OB portion is selected from the defective pixels stored in the defect correction data. Then, the output level at the time of photographing the defective pixel is read from the memory 30. In this way, the processes in step S202 and step S203 are repeated until a defective pixel in the OB portion is found that has an output level less than the saturation level of the A / D converter 16.

  In addition, although it is the order which selects the defective pixel which exists in OB part by step S202, it is good to select in an order from the thing with the largest output level memorize | stored in the defect correction data. By selecting in this way, it is possible to select a defective pixel in which the output level at the time of photographing is less than the saturation level of the A / D converter 16 and the error is less than a predetermined output level.

  Further, the output level is not limited to a defective pixel, but varies depending on the accumulation time and ISO sensitivity. Therefore, the defective pixels that reach the saturation level of the A / D converter 16 also differ depending on the accumulation time and ISO sensitivity at the time of photographing. Accordingly, when a defective pixel is selected in step S202, the defective pixel to be selected is changed according to the setting of the accumulation time and ISO sensitivity at the time of actual photographing, based on the property with respect to the accumulation time and ISO sensitivity of the defective pixel. Of course it doesn't matter. For this purpose, for example, photographing is performed in advance with different accumulation times and ISO sensitivities, and a desired defective pixel is detected under each condition (that is, below the saturation level of the A / D converter 16 and above a predetermined output level). The defective pixels detected for each condition may be stored.

  Further, in step S202, when detecting defective pixels in the OB portion having an output level lower than the saturation level of the A / D converter 16, a limit such as the number of times that step S202 is executed or a minimum level may be provided. . If a defective pixel corresponding to the condition cannot be detected within the limit, the image itself may be broken and some warning may be given and correction may not be performed.

  Next, in step S205, another defect recorded in the defect correction data from the output level of the defective pixel at the time of photographing determined in step S204 and the output level of the same defective pixel read from the defect correction data. The pixel output level is calculated. Here, among defective pixels recorded in the defect correction data, calculation is performed for defective pixels in the light receiving portion of the image sensor 14. This is because the output level of the defective pixel changes depending on the temperature / charge accumulation time, and the output level of the defective pixel in the OB portion at the time of photographing corresponds to the temperature / charge accumulation time at the time of photographing. Because.

  In the calculation performed here, first, for the defective pixel determined in step S204, a ratio K between the output level at the time of shooting and the output level recorded in the defect correction data is obtained, and then this ratio K is calculated. Multiply the output level of each defective pixel recorded in the defect correction data. When the gain at the time of shooting is changed by setting the ISO sensitivity, the output level at the time of shooting is converted according to the gain. For example, if the gain at the time of shooting is twice that when the scratch correction data is acquired, the conversion is made such that it is 1/2 times, and if it is 4 times, it is 1/4 times. By this conversion, the true value of the output level at the time of photographing can be obtained.

  Next, in step S206, it is determined from the calculation result obtained in step S205 whether or not the defective pixel has reached the saturation level of the pixel. When the saturation level has been reached (YES in step S206), there is a high possibility that the charge generated in the defective pixel flows into the adjacent pixel. Accordingly, the upper, lower, left, and right pixels of the defective pixel determined to have reached the saturation level are temporarily stored as the pixels to be subjected to defect correction (step S207).

  In this way, by setting a pixel adjacent to a defective pixel that has reached a saturation level as a correction target pixel, it is possible to correct a pixel that has become an inappropriate output level due to leakage of charge from the defective pixel. It becomes possible.

  If the saturation level has not been reached (NO in step S206), the process directly proceeds to step S208. In step S208, it is determined whether the output level calculation and saturation level determination described above have been performed for all defective pixels recorded in the defect correction data. If not completed, the process returns to step S205 to return to the next defect. Processing is performed on the pixel. If all defective pixels in the light receiving unit of the image sensor 14 stored in the defect correction data have been completed, the process proceeds to step S209.

  In step S209, a defect correction process is performed on the defective pixel stored in the defect correction data and the image data at the time of photographing the pixel temporarily stored as the correction target in step S207. Here, interpolation processing is performed on the defective pixel and the correction target pixel by using image data output from a peripheral pixel portion where normal image data is obtained. In this embodiment, an RGB Bayer array CMOS area sensor is used as an image sensor. Therefore, interpolation processing is performed based on the average value data of pixels from which normal output data is obtained among the pixels in the closest vertical, horizontal, and diagonal directions of the same color as the defective pixel and the correction target pixel.

  Note that the interpolation process is not limited to this, and there is no problem even if it is a known interpolation process using an output level of an arbitrary peripheral pixel.

  Then, when the correction process of the output level at the time of photographing is completed for all defective pixels and correction target pixels in step S209, the defective pixel correction process is terminated, and the process returns to step S117 of FIG.

  As described above, according to the present embodiment, the scratch correction data is stored in the scratch correction data based on the ratio between the output level of the defective pixel in the OB portion and the output level at the time of shooting the same defective pixel. It is determined whether the other stored defective pixels are saturated. When it is determined that the pixel is saturated, correction processing similar to that for the defective pixel is performed on pixels adjacent to the defective pixel in saturation. By controlling in this way, it is possible to easily perform the defect correction including errors as shown in FIG. 5 without adding a special configuration.

  In the present embodiment, correction is performed on the output of the upper, lower, left, and right pixels adjacent to the defective pixel, but the present invention is not limited to this. For example, in an image sensor having a structure in which charge leaks from a saturated defective pixel even in a pixel adjacent in the diagonal direction, the output from the diagonal pixel adjacent to the defective pixel is also corrected. May be.

  In the present embodiment, a CMOS area sensor has been described as an example. However, the present invention may be applied to any area sensor such as a CCD area sensor.

  In the present embodiment, the calculation in step S205 and the determination in S206 are performed based on the output level of one defective pixel in the OB portion. However, the present invention is not limited to this, and a plurality of OB portions are included. You may comprise so that the average value of the output level of a defective pixel, etc. may be used.

It is a block diagram which shows the structure of the imaging device in embodiment of this invention. It is a flowchart which shows the imaging | photography process procedure of the imaging device in embodiment of this invention. It is a flowchart which shows the imaging | photography process procedure of the imaging device in embodiment of this invention. It is a flowchart which shows the defective pixel correction process procedure in embodiment of this invention. It is the figure which showed the example when a defective pixel is saturated and an electric charge leaks into a surrounding pixel.

Explanation of symbols

14 Image sensor 50 System control circuit 56 Non-volatile memory 100 Camera body 300 Lens unit

Claims (14)

An image sensor comprising a plurality of pixels each having a light receiving portion and a light shielding portion;
Storage means for holding at least the position information and the reference output level of the defective pixel of the image sensor acquired in advance;
Detection means for detecting a defective pixel stored in the storage means that is a defective pixel in the light-shielding portion and whose output level at the time of photographing the subject is lower than a preset first saturation level. When,
Adjustment for adjusting the reference output level of the defective pixel in the light receiving unit among the defective pixels stored in the storage unit based on the detected reference output level of the defective pixel and the output level at the time of photographing. Means,
Determining whether the reference output level adjusted by the adjusting means is equal to or higher than a second saturation level that is the saturation level of the pixel, and the adjusted reference output level is equal to or higher than the second saturation level Setting means for setting a pixel adjacent to the pixel as a correction target pixel;
An image pickup apparatus comprising: a correction unit that corrects an output level at the time of photographing output from a defective pixel stored in the storage unit and a correction target pixel set by the setting unit.   The adjusting means obtains a ratio between the detected reference output level of the defective pixel and the output level at the time of photographing, and multiplies the obtained ratio to obtain a reference output level of the defective pixel in the light receiving unit. The imaging apparatus according to claim 1, wherein the imaging apparatus is adjusted. Amplifying means for amplifying the output level of the image sensor;
The imaging apparatus according to claim 1, wherein the first saturation level is a saturation level of the amplification unit.   When the gain of the amplifying unit is different between the time when the position information and the reference output level of the defective pixel stored in the storage unit are acquired and the time of shooting, the adjusting unit sets the output level at the time of shooting. The imaging apparatus according to claim 3, wherein adjustment is performed so as to match the gain at the time of acquisition.   5. The imaging apparatus according to claim 1, wherein the detection unit detects a defective pixel having an output level that is lower than the first saturation level and that is the highest at the time of shooting.   2. When a plurality of defective pixels are detected by the detection unit, the adjustment unit averages the reference output level and the output level at the time of photographing, respectively, of the detected plurality of defective pixels. 5. The imaging device according to 4.   5. The image pickup apparatus according to claim 1, wherein the detection unit changes a defective pixel used for detection based on at least one of charge accumulation time and sensitivity at the time of photographing. In an imaging apparatus comprising: an imaging device having a plurality of pixels each having a light receiving unit and a light shielding unit; and a storage unit that holds at least position information and a reference output level of defective pixels of the imaging device acquired in advance. A correction method for correcting an output level output from a pixel,
A detecting step of detecting defective pixels in the light-shielding portion among the defective pixels stored in the storage unit and having an output level at the time of photographing the subject being lower than a preset first saturation level. When,
Adjustment for adjusting the reference output level of the defective pixel in the light receiving unit among the defective pixels stored in the storage unit based on the detected reference output level of the defective pixel and the output level at the time of photographing. Steps,
It is determined whether the reference output level adjusted in the adjustment step is equal to or higher than a second saturation level that is a saturation level of the pixel, and the adjusted reference output level is higher than or equal to the second saturation level. A setting step for setting a pixel adjacent to the pixel as a correction target pixel;
And a correction step of correcting the output level at the time of photographing output from the defective pixel stored in the storage unit and the correction target pixel set in the setting step.   In the adjustment step, a ratio between a reference output level of the detected defective pixel and an output level at the time of shooting is obtained, and the reference output level of the defective pixel in the light receiving unit is obtained by multiplying the obtained ratio. The correction method according to claim 8, wherein adjustment is performed. The imaging apparatus further includes an amplifying unit that amplifies the output level of the imaging element,
The correction method according to claim 8 or 9, wherein the first saturation level is a saturation level of the amplification means.   When the gain of the amplifying unit is different between the time when the position information and the reference output level of the defective pixel stored in the storage unit are acquired and the time of shooting, the adjustment step sets the output level at the time of shooting. The correction method according to claim 10, wherein adjustment is performed to match the gain at the time of acquisition.   12. The correction method according to claim 8, wherein in the detection step, the highest defective pixel having an output level lower than the first saturation level at the time of photographing is detected.   9. When a plurality of defective pixels are detected in the detection step, the adjustment step averages a reference output level and a photographing output level of the detected plurality of defective pixels, respectively. The correction method as described in thru | or 11.   12. The correction method according to claim 8, wherein in the detection step, a defective pixel used for detection is changed based on at least one of charge accumulation time and sensitivity at the time of photographing.

Images