JP6575926B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging device having an imaging element having a structure in which a plurality of pixels share an FD (Floating Diffusion).

  In a CMOS (Complementary Metal Oxide Semiconductor) image sensor, the number of FETs (Field Effect Transistors) and wiring per unit pixel can be achieved by sharing the FD among multiple pixels in order to achieve downsizing of the image sensor and securing of the light receiving area. A configuration with a reduced number is known. For example, Patent Document 1 describes a specific configuration of this type of CMOS image sensor.

  Also, in CMOS image sensors, it is known that RTS (Random Telegraph Signal) noise is generated in captured images by capturing carriers moving in the MOSFET channel at trap levels present in the gate insulating film or the like. It has been. For example, Patent Document 2 describes a specific configuration of an imaging apparatus that can reduce RTS noise.

JP 2005-198001 A Japanese Patent No. 5561112

  In a CMOS image sensor having a structure in which n (n is a natural number of 2 or more) pixels share an FD (hereinafter referred to as an “n pixel FD sharing structure” for convenience of description), RTS noise is generated in the MOSFET. RTS noise is generated for each pixel group sharing the FD. Hereinafter, for convenience of description, a pixel in which RTS noise occurs is referred to as an “RTS noise generation pixel”.

  Consider a case where RTS noise generating pixels are corrected using a general filter (smoothing filter, median filter, etc.). In this case, for example, when RTS noise occurs between a pair of adjacent pixels sharing the FD, the influence of one adjacent pixel (one adjacent RTS noise generating pixel) is too strong, and the target pixel (the other adjacent RTS) Noise generation pixel) cannot be corrected satisfactorily. In order to satisfactorily correct the target pixel while suppressing the influence of the RTS noise of one adjacent pixel, it is necessary to widen the filter range, for example. However, a problem is pointed out that if pixel correction is performed by expanding the filter range, the resolution of the image is reduced.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to reduce the resolution of an image even when RTS noise occurs between adjacent pixels sharing an FD. An object of the present invention is to provide a photographing apparatus capable of correcting pixels satisfactorily.

  An imaging device according to an embodiment of the present invention includes an imaging device having a structure in which a plurality of pixels share an FD, and a defective pixel that registers a static defective pixel having a static defect in a pixel group included in the imaging device. A registration unit; a defective pixel correction unit that performs defective pixel correction on a static defective pixel registered in the registration unit; and a noise correction unit that performs noise correction on a pixel group after the defective pixel correction is performed. With. The registering unit registers other pixels excluding at least one pixel among the plurality of pixels sharing the FD as static defective pixels.

  In addition, the photographing apparatus according to an embodiment of the present invention may include a detection unit that detects pixels in which noise occurs. In this configuration, the noise correction unit performs noise correction on the pixels detected by the detection unit.

  In addition, the imaging apparatus according to an embodiment of the present invention may include a noise pixel registration unit that registers a pixel that may generate noise. In this configuration, the detection unit detects a pixel registered in the noise pixel registration unit as a noise generation pixel when a predetermined condition is satisfied.

In one embodiment of the present invention, the predetermined condition for detecting a noise-generating pixel is, for example,
(1) The ISO sensitivity is set to a predetermined sensitivity or higher.
(2) The temperature at the time of shooting is a predetermined temperature or higher,
(3) The shooting mode is set to a composite mode for generating a composite image by combining a plurality of images.
(4) The shooting mode is set to live view mode for displaying live view.
(5) The shooting mode is set to a movie mode for shooting a movie,
At least one of the following.

  In one embodiment of the present invention, the image sensor is, for example, a CMOS image sensor.

  In one embodiment of the present invention, the plurality of pixels sharing the FD include at least two same color pixels.

  In one embodiment of the present invention, the noise is RTS noise.

  According to an embodiment of the present invention, even when RTS noise occurs between adjacent pixels sharing an FD, an imaging device capable of satisfactorily correcting a target pixel while suppressing a decrease in image resolution. Is provided.

It is a block diagram which shows the structure of the imaging device which concerns on one Embodiment of this invention. It is a figure which shows typically the pixel arrangement | positioning of the solid-state image sensor with which the imaging device which concerns on one Embodiment of this invention is equipped. It is a figure which shows the flowchart of the pixel correction process performed by the system controller with which an imaging device is equipped in one Embodiment of this invention. It is a figure for demonstrating the effect of the image correction process shown by FIG.

  An imaging apparatus according to an embodiment of the present invention will be described below with reference to the drawings. In the following, a digital single lens reflex camera will be described as an embodiment of the present invention. Note that the photographing apparatus is not limited to a digital single lens reflex camera, but includes, for example, a mirrorless single lens camera, a compact digital camera, a video camera, a camcorder, a tablet terminal, a PHS (Personal Handy phone System), a smartphone, a feature phone, a portable game machine, and the like. The apparatus may be replaced with another type of apparatus having a photographing function.

[Configuration of the photographing apparatus 1]
FIG. 1 is a block diagram showing a configuration of a photographing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the photographing apparatus 1 includes a system controller 100, an operation unit 102, a drive circuit 104, a photographing lens 106, a diaphragm 108, a shutter 110, a solid-state image sensor 112, a signal processing circuit 114, an image processing engine 116, A buffer memory 118, a card interface 120, an LCD (Liquid Crystal Display) control circuit 122, an LCD 124, a ROM (Read Only Memory) 126, and a temperature sensor 128 are provided. Although the photographing lens 106 has a plurality of lenses, it is shown as a single lens for convenience in FIG.

  The operation unit 102 includes various switches necessary for the user to operate the photographing apparatus 1, such as a power switch, a release switch, a photographing mode switch, and a switch for specifying photographing conditions such as ISO sensitivity. When the user operates the power switch, power is supplied from the battery (not shown) to the various circuits of the photographing apparatus 1 through the power line.

  The system controller 100 includes a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). After supplying power, the system controller 100 accesses the ROM 126, reads out a control program, loads it into a work area (not shown), and executes the loaded control program to control the entire photographing apparatus 1.

  When the release switch is operated, the system controller 100 is measured by, for example, a photometric value calculated based on an image captured by the solid-state image sensor 112 or an exposure meter (not shown) built in the photographing apparatus 1. The diaphragm 108 and the shutter 110 are driven and controlled via the drive circuit 104 so that proper exposure can be obtained based on the photometric values. More specifically, drive control of the aperture 108 and the shutter 110 is performed based on an AE function designated by a shooting mode switch such as a program AE (Automatic Exposure), shutter priority AE, aperture priority AE, and the like. Further, the system controller 100 performs AF (Autofocus) control together with AE control. An active method, a phase difference detection method, a contrast detection method, or the like is applied to the AF control. The AF mode includes a central single-point ranging mode using a single central ranging area, a multi-point ranging mode using a plurality of ranging areas, and the like. The system controller 100 controls driving of the photographing lens 106 via the driving circuit 104 based on the AF result, and adjusts the focus of the photographing lens 106. Since the configuration and control of this type of AE and AF are well known, detailed description thereof is omitted here.

  The light flux from the subject passes through the photographing lens 106, the diaphragm 108, and the shutter 110 and is received by the light receiving surface of the solid-state image sensor 112. The solid-state image sensor 112 is a single-plate color CMOS image sensor having a Bayer pixel arrangement.

  FIG. 2 schematically shows the pixel arrangement of the solid-state image sensor 112. In FIG. 2, each pixel is indicated by “color + sign” for the purpose of identifying each pixel. As shown in FIG. 2, in the solid-state imaging device 112, four pixels (R pixel, Gr pixel, Gb pixel, B pixel) are periodically arranged, and these four pixels share the FD. It has an FD sharing structure.

  The solid-state image sensor 112 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, and generates and outputs R, G, and B image signals. The solid-state image sensor 112 is not limited to the one having a Bayer filter mounted thereon, and may be one having another color arrangement filter mounted thereon.

  The signal processing circuit 114 performs predetermined signal processing such as black level correction and defective pixel correction on the image signal input from the solid-state imaging device 112 to generate RAW format image data, and outputs the image data to the image processing engine 116. To do.

  The defective pixel correction by the signal processing circuit 114 is a process of correcting a static defective pixel having a static defect in the effective pixel group of the solid-state imaging device 112. A typical example of a static defective pixel is a so-called white scratched pixel (white point pixel) that generates a false signal when electrons are generated even in the absence of incident light. White scratched pixels (white dot pixels) existing at the time of product shipment are detected on an assembly / adjustment line or the like, and their addresses are registered (stored) in advance in, for example, the internal memory of the system controller 100. For example, a white spot pixel (white dot pixel) that occurs after product shipment is detected at the timing of power-on, for example, and its address is registered in the internal memory of the system controller 100.

  In defective pixel correction by the signal processing circuit 114, correction using peripheral pixels (peripheral pixels excluding defective pixels) such as a smoothing filter and a median filter is performed for static defective pixels registered in the internal memory of the system controller 100. Or correction to replace with a specified pixel value.

  The image processing engine 116 performs predetermined signal processing such as RTS noise generation pixel correction, demosaicing, matrix calculation, white balance, and Y / C separation on the RAW image data input from the signal processing circuit 114. A luminance signal Y and color difference signals Cb and Cr are generated and compressed in a predetermined format such as JPEG (Joint Photographic Experts Group). The buffer memory 118 is used as a temporary storage location for processing data when the image processing engine 116 executes processing. In addition, the captured image is not limited to the captured image data in JPEG format generated by the image processing engine 116, but captured image data in RAW format input from the signal processing circuit 114 may be stored.

  A memory card 200 is detachably inserted into a card slot of the card interface 120.

  The image processing engine 116 can communicate with the memory card 200 via the card interface 120. The image processing engine 116 stores the generated captured image data in the JPEG format in the memory card 200 (or a built-in memory (not shown) provided in the imaging apparatus 1). The memory card 200 may store RAW-format captured image data output from the signal processing circuit 114.

  Further, the image processing engine 116 buffers the generated luminance signal Y and color difference signals Cb and Cr in a frame memory (not shown) in units of frames. The image processing engine 116 sweeps the buffered signal from each frame memory at a predetermined timing, converts it into a video signal of a predetermined format, and outputs it to the LCD control circuit 122. The LCD control circuit 122 modulates and controls the liquid crystal based on the image signal input from the image processing engine 116. Thereby, the photographed image of the subject is displayed on the display screen of the LCD 124. The user can view a real-time through image (live view) photographed with appropriate exposure and focus based on the AE control and AF control through the display screen of the LCD 124.

  When the user performs a reproduction operation of the photographed image, the image processing engine 116 reads the photographed image data designated by the operation from the memory card 200 or the built-in memory, converts it into an image signal of a predetermined format, and the LCD control circuit 122. Output to. The LCD control circuit 122 performs modulation control on the liquid crystal based on the image signal input from the image processing engine 116, so that a captured image of the subject is displayed on the display screen of the LCD 124.

[Pixel correction processing]
Next, pixel correction processing executed by the system controller 100 will be described using the flowchart of FIG. The pixel correction process shown in FIG. 3 is started when the release switch is pressed, for example.

  Note that the RTS noise is a random noise that randomly occurs at a specific pixel every time the image is taken, and does not always have high reproducibility. However, it is possible to identify an RTS noise-generating pixel that generates RTS noise by repeating test imaging on an assembly / adjustment line or the like. In this embodiment, an RTS noise generating pixel is specified on an assembly / adjustment line or the like, and its address is registered (stored) in advance in, for example, an internal memory of the system controller 100.

  Also, some RTS noise generation pixels (non-static defective pixels) registered in the internal memory of the system controller 100 are also registered (stored) as static defective pixels, as will be described in detail later. Hereinafter, for the convenience of explanation, such an RTS noise occurrence pixel is referred to as a “static defect deemed pixel”.

[S11 in FIG. 3 (condition determination)]
In this process step S11, it is determined whether or not a predetermined condition is satisfied. Here, the predetermined condition is a condition in which RTS noise is likely to occur (the amount of noise is not very small and is conspicuous on the image). Illustratively, it is determined that the predetermined condition is satisfied when at least one of the following items (1) to (5) is satisfied.
(1) The ISO sensitivity is set to a predetermined sensitivity or higher.
(2) The photographing temperature detected by the temperature sensor 128 is equal to or higher than a predetermined temperature.
(3) The shooting mode is set to a composite mode for generating a composite image obtained by combining a plurality of images.
(4) The shooting mode is set to a live view mode for displaying a live view.
(5) The shooting mode is set to a moving image mode for shooting a moving image.

[S12 in FIG. 3 (Defective Pixel Correction Not Containing Static Defect Pixels)]
If it is determined in processing step S11 (condition determination) that the predetermined condition is not satisfied (S11: NO), the generation of RTS noise is not detected in the RTS noise generation pixel registered in the internal memory of the system controller 100. This processing step S12 is executed.

  In this processing step S12, only the address of the static defective pixel is read from the internal memory of the system controller 100, and the defective pixel correction is performed by the signal processing circuit 114 on the pixel of the read address. After the defective pixel correction by the signal processing circuit 114, the pixel correction process shown in FIG. 3 ends.

[S13 in FIG. 3 (Defect Pixel Correction Including Static Defect Pixel)]
When it is determined in the processing step S11 (condition determination) that the predetermined condition is satisfied (S11: YES), the occurrence of RTS noise is detected in the RTS noise generation pixel registered in the internal memory of the system controller 100. This processing step S13 is executed as if it were regarded as a thing.

  In this processing step S13, the address of the static defective pixel and the address of the static defective pixel are read from the internal memory of the system controller 100, and the defective pixel correction by the signal processing circuit 114 is performed on the pixel of the read address. Is done.

[S14 in FIG. 3 (correction of RTS noise-generating pixels)]
In this processing step S14, the address of the RTS noise generating pixel excluding the static defect deemed pixel is read from the internal memory of the system controller 100, and the image processing engine 116 generates the RTS noise for the pixel of the read address. Pixels are corrected. For such correction processing, for example, a smoothing filter, a median filter, or the like is used.

  4A and 4B are diagrams for explaining the effect of the image correction processing shown in FIG.

  FIG. 4A is a diagram schematically illustrating RTS noise generation pixels. In the example of FIG. 4A, four pixels (pixel R22, pixel Gr23, pixel Gb32, and pixel B33) sharing the FD are RTS noise generation pixels. In the example of FIG. 4A, a case where the RTS noise generation pixel is corrected using the peripheral pixels of the same color (defective pixel correction) will be described.

  The peripheral pixels of the same color that are closest to the pixel R22 are a pixel R02, a pixel R20, a pixel R24, and a pixel R42. In the example of FIG. 4A, none of the same color pixels are RTS noise generation pixels. Therefore, the pixel R22 is corrected satisfactorily with a minimum filter (specifically, using four pixels (pixel R02, pixel R20, pixel R24, and pixel R42)).

  The peripheral pixels of the same color that are closest to the pixel B33 are the pixel B13, the pixel B31, the pixel B35, and the pixel B53. In this case, as in the example of the pixel R22, none of the same color pixels are RTS noise generation pixels. Therefore, the pixel B33 is also well corrected with the minimum filter (specifically, using four pixels (pixel B13, pixel B31, pixel B35, and pixel B53)).

  The peripheral pixels of the same color that are closest to the pixel Gb32 are a pixel Gr21, a pixel Gr23, a pixel Gr41, and a pixel Gr43. The pixel Gr21, the pixel Gr41, and the pixel Gr43 are not RTS noise generation pixels, but the pixel Gr23 is an RTS noise generation pixel. Therefore, with the minimum filter (specifically, with only four pixels (pixel Gr21, pixel Gr23, pixel Gr41, pixel Gr43)), the influence of the pixel Gr23 that is the RTS noise generation pixel is too strong, and the pixel Gb32 is good. Cannot be corrected.

  As a typical method for satisfactorily correcting the pixel Gb32, for example, in addition to the above four pixels, the filter range includes the pixel Gb10, the pixel Gb12, the pixel Gb14, the pixel Gb30, the pixel Gb34, the pixel Gb50, It is conceivable to expand to the pixel Gb52 and the pixel Gb54. However, expanding the filter range is not desirable because it leads to a decrease in the resolution of the entire captured image.

  Similarly, for the pixel Gr23, the influence of the pixel Gb32, which is an RTS noise generating pixel, is too strong to be corrected satisfactorily, and widening the filter range leads to a decrease in the resolution of the entire captured image. Not desirable.

  Therefore, in the present embodiment, when a neighboring pixel of the same color that is closest to a pixel registered as an RTS noise generation pixel is also registered as an RTS noise generation pixel, one of the pixels is regarded as a static defect-defining pixel. Registered in the internal memory of the system controller 100. In the example of FIG. 4, a peripheral pixel Gr23 of the same color that is closest to the pixel Gb32 is registered in the internal memory of the system controller 100 as a static defect deemed pixel (see the conceptual diagram of FIG. 4B).

  For this reason, the pixel Gr23 is subjected to defective pixel correction by the signal processing circuit 114 in the processing step S13 (defective pixel correction including a static defective pixel). Thereby, the pixel value of the pixel Gr23 becomes a value in which the noise component is suppressed.

  On the other hand, the pixel Gb32 is not registered as a static defect deemed pixel in the internal memory of the system controller 100 and is registered as an RTS noise generation pixel. Therefore, the pixel Gb32 is corrected by the image processing engine 116 in the processing step S14 (correction of RTS noise generation pixel). Specifically, the pixel Gb32 is corrected with a minimum filter (specifically, using four pixels (pixel Gr21, pixel Gr23, pixel Gr41, pixel Gr43)). At this time, since the pixel Gr23 is subjected to the defective pixel correction by the signal processing circuit 114 as a static defective pixel, the pixel Gb32 is corrected well without being affected by the RTS noise generation pixel.

  As described above, according to the present embodiment, even when RTS noise is generated between adjacent pixels sharing the FD, the RTS noise generation pixel can be favorably corrected with the minimum filter. Therefore, it is possible to achieve good correction of RTS noise generating pixels and suppression of reduction in resolution of a captured image at the same time.

  The above is the description of the exemplary embodiments of the present invention. Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention. For example, the embodiment of the present invention also includes contents appropriately combined with embodiments or the like clearly shown in the specification or obvious embodiments.

  In the above embodiment (example in FIG. 4), the solid-state image sensor 112 has a structure in which 2 × 2 pixels share the FD, but the present invention is not limited to this. The solid-state image sensor 112 may have a structure in which n pixels arranged in the vertical direction or the diagonal direction share the FD.

DESCRIPTION OF SYMBOLS 1 Image pick-up apparatus 100 System controller 102 Operation part 104 Drive circuit 106 Shooting lens 108 Aperture 110 Shutter 112 Solid-state image sensor 114 Signal processing circuit 116 Image processing engine 118 Buffer memory 120 Card interface 122 LCD control circuit 124 LCD
126 ROM
128 Temperature sensor 200 Memory card

Claims (7)

An image sensor having a structure in which a plurality of pixels share FD (Floating Diffusion);
Of the group of pixels the image sensor has a registration means for registering the RTS noise pixel static defective pixel and the RTS (Random Telegraph Signal) noise is generated with a static defects,
Defective pixel correction means for correcting defective pixels for static defective pixels registered in the registration means and some RTS noise generation pixels ;
Noise correction means for correcting noise for the pixel group after the defective pixel correction is performed;
With
The registration means includes
Registering at least one RTS noise generating pixel as a static defective pixel among a plurality of pixels sharing the FD and being the RTS noise generating pixel ;
The defective pixel correction means includes
Performing defective pixel correction on the static defective pixels registered in the registration means and the RTS noise generation pixels registered as the static defective pixels ,
Shooting device. Detecting means for detecting whether or not noise is generated in the RTS noise generating pixel registered in the registration means ;
The noise correction means is
When the occurrence of noise in the RTS noise generation pixels registered in the registration unit is detected, the noise correction is performed on the RTS noise generation pixels excluding the part of the RTS noise generation pixels .
The imaging device according to claim 1. The detection means includes
When a predetermined condition is satisfied , detecting the occurrence of noise in the RTS noise generation pixel registered in the registration means ;
The imaging device according to claim 2. The predetermined condition is:
The following items (1) to (5),
(1) The ISO sensitivity is set to a predetermined sensitivity or higher.
(2) The temperature at the time of shooting is a predetermined temperature or higher,
(3) The shooting mode is set to a composite mode for generating a composite image by combining a plurality of images.
(4) The shooting mode is set to live view mode for displaying live view.
(5) The shooting mode is set to a movie mode for shooting a movie,
A condition including at least one of
The imaging device according to claim 3. The image sensor is
A CMOS (Complementary Metal Oxide Semiconductor) image sensor.
The imaging device according to any one of claims 1 to 4. The plurality of pixels sharing the FD are:
Including at least two pixels of the same color,
The imaging device according to any one of claims 1 to 5. The noise is
RTS (Random Telegraph Signal) noise,
The imaging device according to any one of claims 1 to 6.

Images