JP2009089358A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and method capable of extending a dynamic range through one photographing, without synthesizing an image from a plurality of images taken under different exposure amounts. <P>SOLUTION: In this imaging apparatus, first YUV data obtained from a pixel output produced by extrapolating single RAW-RGB data read out of an SDRAM 23 with an extended-D(Dynamic)-range extrapolation section in a YUV converting section 36, and second YUV data obtained from a pixel output to be more than a predetermined saturation level without being extrapolated with an extended-D-range extrapolation section, are loaded into a YUV composition section 37; and then third YUV data is produced in the YUV composition section 37 by combining brightness data taken out of the first YUV data and color difference data taken out of the second YUV data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method such as a digital still camera and a digital video camera, and more particularly to an imaging apparatus and an imaging method that can expand a dynamic range of a captured image.

  Compared to the dynamic range of images taken with a conventional silver salt camera using a silver salt photographic film, the dynamic range of images taken with a digital still camera or a digital video camera having a solid-state image sensor such as a CCD is extremely narrow. When the dynamic range is narrow, a phenomenon called “blackout” occurs in the dark part of the subject, and conversely, a phenomenon called “overexposure” occurs in the bright part of the subject, and the image quality deteriorates.

Therefore, in order to expand the dynamic range of an image captured by a solid-state imaging device such as a CCD, for example, the same subject is shot multiple times with different exposure amounts to obtain a plurality of images with different exposure amounts. A technique for adding these images to generate a composite image with an expanded dynamic range has been known (see, for example, Patent Document 1).
JP 2000-92378 A

  By the way, in order to expand the dynamic range, in the method of performing photographing a plurality of times while changing the exposure amount as in the above-mentioned Patent Document 1, when photographing a moving object, the subject is doubled. Therefore, the image may not be synthesized correctly.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method capable of expanding the dynamic range by one shooting without changing the exposure amount and performing a plurality of shootings to combine images. .

  In order to achieve the above object, an imaging apparatus according to claim 1, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a front side of each pixel A pixel output detector that determines whether or not the output from any of the pixels has reached a predetermined saturation level or more with respect to the output from each pixel. And the pixel output detection means, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, a filter other than the specific color around the filter is arranged Pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from the processed pixels, and the pixel subjected to the interpolation processing by the pixel output correction processing means First YUV data generated from a force and second YUV data generated from a pixel output reaching the predetermined saturation level or higher without performing the interpolation processing by the pixel output correction processing means, and the first YUV data YUV combining means for generating third YUV data obtained by combining the luminance data extracted from the color data and the color difference data extracted from the second YUV data.

  The image pickup apparatus according to claim 2 is characterized in that a processing unit when the output of each pixel is detected by the pixel output detection means is a size of 2 × 2 pixels in the horizontal and vertical directions.

  The image pickup apparatus according to claim 3, in order to select and execute the operation to perform the interpolation when the output of the pixel in which the filter of the specific color is arranged exceeds the predetermined saturation level. It is characterized by comprising the operation selection means.

  The imaging apparatus according to claim 4, wherein the predetermined saturation output from the pixel output correction processing means when an output of a pixel in which the filter of the specific color is arranged has reached or exceeds the predetermined saturation level. Bit compression conversion means for compressing the pixel output data once expanded from the first bit number below the level to the second bit number to the first bit number again, the bit compression conversion means, The compression is performed by reducing the compression rate for the data corresponding to the pixel output at the saturation level or lower than the compression rate for the data corresponding to the pixel output in the region above the saturation level.

  The imaging apparatus according to claim 5, the data corresponding to the pixel output at the low luminance level below the saturation level is substantially the same value before the bit compression by the bit compression conversion unit and after the bit compression. It is characterized by using such a compression rate.

  The imaging apparatus according to claim 6, wherein when there is a defective pixel in the processing unit, a pixel in which a filter of the same color as the defective pixel around the defective pixel is arranged instead of the defective pixel. It is characterized by use.

  The imaging method according to claim 7, wherein a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are provided on the front side of each pixel. In the imaging method of an imaging apparatus including an imaging device in which is arranged, a determination processing step for determining whether or not an output from any pixel reaches or exceeds a predetermined saturation level with respect to an output from each pixel When the determination processing step determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, a filter other than the specific color around it is arranged. An interpolation processing step for interpolating a pixel output determined to be equal to or higher than the predetermined saturation level based on an output from the pixel, and a pixel output subjected to the interpolation processing by the interpolation processing step The first YUV data generated from the first YUV data and the second YUV data generated from the pixel output reaching the predetermined saturation level or higher without performing the interpolation processing in the interpolation processing step are taken out from the first YUV data. YUV composition processing step for synthesizing luminance data and color difference data extracted from the second YUV data to generate third YUV data.

  The imaging method according to an eighth aspect is characterized in that a processing unit when detecting the output of each pixel in the determination processing step is a size of 2 × 2 pixels in the horizontal and vertical directions.

  The imaging method according to claim 9, wherein the operation of interpolating when the output from the pixel in which the filter of the specific color is arranged exceeds the predetermined saturation level is performed by an operation selection unit. It is characterized by being executed by selecting.

  The imaging method according to claim 10, wherein the output of the pixel in which the filter of the specific color is arranged is equal to or lower than the predetermined saturation level that is output from the interpolation processing step when the output reaches the predetermined saturation level or higher. A bit compression conversion step of compressing the pixel output data once expanded from the first bit number to the second bit number into the first bit number, and in the bit compression conversion step, the saturation The compression is performed by reducing the compression rate for the data corresponding to the pixel output below the saturation level, rather than the compression rate for the data corresponding to the pixel output in the region above the level.

  In the imaging method according to claim 11, the data corresponding to the pixel output at the low luminance level below the saturation level has substantially the same value before bit compression and after bit compression in the bit compression conversion step. It compresses using the compression rate which becomes.

  The imaging method according to claim 12, wherein when there is a defective pixel in the processing unit, a pixel in which a filter of the same color as the defective pixel around the defective pixel is arranged instead of the defective pixel. It is characterized by use.

  According to the first and seventh aspects of the present invention, when it is determined that the output from the pixel in which the filter of the specific color is arranged has reached a predetermined saturation level or more, the other surroundings that are not saturated Based on the output from the pixel where the color filter is placed, the pixel output in the region above the saturation level is interpolated to expand the dynamic range, thereby changing the exposure amount and shooting multiple times to compose the image In addition, the dynamic range can be expanded by one shooting.

  Further, according to the first and seventh aspects of the present invention, the first YUV data generated from the interpolated pixel output and the pixel output that has reached a predetermined saturation level or higher without performing the interpolation processing are generated. The second YUV data is taken in, and the luminance data (Y data) taken out from the first YUV data and the color difference data (UV data) taken out from the second YUV data are combined to generate third YUV data. As a result, the dynamic range is expanded only for the luminance data of the first YUV data generated from the interpolated pixel output, and the original color difference data is used as the color, thereby preventing hue shift and accurate color matching. Reproduction can be performed.

  According to the third and ninth aspects of the present invention, an operation for interpolating the output of the pixel in which the filter of the specific color is arranged when the predetermined saturation level or higher is reached is performed by the operation selection unit. This can be done at the photographer's discretion.

  According to the fourth and tenth aspects of the present invention, the compression rate for the data corresponding to the pixel output below the saturation level is made smaller than the compression rate for the data corresponding to the pixel output in the region above the saturation level, and the compression is performed. As a result, it is possible to satisfactorily maintain the gradation property below the saturation level.

  According to the fifth and eleventh aspects of the present invention, the compression corresponding to the pixel output at the low luminance level and below the saturation level is compressed so as to have substantially the same value before the bit compression and after the bit compression. By using the rate, the gradation at a low luminance level can be maintained satisfactorily.

  According to the invention described in claims 6 and 12, when there is a defective pixel in the processing unit, a pixel in which a filter of the same color as the defective pixel around the defective pixel is arranged instead of the defective pixel. By using this, a defective pixel is not used when determining whether or not the output of the pixel in which the filter of the specific color is arranged has reached the saturation level or higher. Therefore, the filter of the specific color is arranged Highly accurate interpolation can be performed when the pixel output reaches or exceeds a predetermined saturation level.

Hereinafter, the present invention will be described based on the illustrated embodiments.
<Embodiment 1>

  FIG. 1A is a front view showing a digital still camera (hereinafter referred to as “digital camera”) as an example of an imaging apparatus according to Embodiment 1 of the present invention, and FIG. 1 (c) is a rear view thereof, and FIG. 2 is a block diagram showing an outline of a system configuration in the digital camera shown in FIGS. 1 (a), (b), and (c).

(Appearance structure of digital camera)
As shown in FIGS. 1A, 1B, and 1C, a release button (shutter button) 2, a power button 3, a shooting / playback switching dial 4 are provided on the upper surface side of the digital camera 1 according to the present embodiment. In the front (front) side of the digital camera 1, a lens barrel unit 6 having a photographing lens system 5, a strobe light emitting unit (flash) 7, and an optical viewfinder 8 are provided.

  On the back side of the digital camera 1, there are a liquid crystal monitor (LCD) 9, an eyepiece 8 a of the optical viewfinder 8, a wide-angle zoom (W) switch 10, a telephoto zoom (T) switch 11, and a menu (MENU) button 12. , A confirmation button (OK button) 13 and the like are provided. Further, a memory card storage unit 15 for storing a memory card 14 (see FIG. 2) for storing captured image data is provided inside the side surface of the digital camera 1.

(Digital camera system configuration)
As shown in FIG. 2, the digital camera 1 includes a CCD 20 serving as a solid-state imaging device on which a subject image incident through a photographing lens system 5 of a lens barrel unit 6 forms an image on a light receiving surface, and electrical signals output from the CCD 20. An analog front-end unit (hereinafter referred to as “AFE unit”) 21 that processes (analog RGB image signal) into a digital signal, a signal processing unit 22 that processes a digital signal output from the AFE unit 21, and temporarily stores data SDRAM 23 for controlling, ROM 24 for storing a control program, a motor driver 25 for driving the lens barrel unit 6 and the like.

  The lens barrel unit 6 includes a photographic lens system 5 having a zoom lens, a focus lens, and the like, an aperture unit 26, and a mechanical shutter unit 27. The drive units of the photographic lens system 5, the aperture unit 26, and the mechanical shutter unit 27 are as follows. It is driven by the motor driver 25. The motor driver 25 is driven and controlled by a drive signal from a control unit (CPU) 28 of the signal processing unit 22.

  In the CCD 20, RGB primary color filters (see FIG. 7: hereinafter referred to as “RGB filters”) are arranged on a plurality of pixels constituting the CCD 20, and electrical signals (analog RGB image signals) corresponding to the RGB three primary colors are output. The

  The AFE unit 21 is sampled by a TG (timing signal generating unit) 30 that drives the CCD 20, a CDS (correlated double sampling unit) 31 that samples an electrical signal (analog RGB image signal) output from the CCD 20, and the CDS 31. An AGC (analog gain controller) 32 that adjusts the gain of the image signal, and an A / D converter 33 that converts the image signal gain-adjusted by the AGC 32 into a digital signal (hereinafter referred to as “RAW-RGB data”) are provided. Yes.

  The signal processing unit 22 outputs a screen horizontal synchronization signal (HD) and a screen vertical synchronization signal (VD) to the TG 30 of the AFE unit 21, and an A / D conversion unit 33 of the AFE unit 21 in accordance with these synchronization signals. CCD interface (hereinafter referred to as “CCD I / F”) 34 that captures RAW-RGB data output from the CPU, a memory controller 35 that controls the SDRAM 23, and a YUV format that can display and record the captured RAW-RGB data. A YUV conversion unit 36 for converting to image data, a YUV composition unit 37 to be described later, a resize processing unit 38 for changing the image size in accordance with the size of image data to be displayed or recorded, and a display output of the image data are controlled. A display output control unit 39, a data compression unit 40 for recording image data by JPEG formation, etc., and image data Based on the operation input information from the media interface (hereinafter referred to as “media I / F”) 41 for reading the image data written to the re-card 14 or the image data written to the memory card 14 and the control stored in the ROM 24. A control unit (CPU) 28 that performs system control and the like of the entire digital camera 1 based on a program is provided.

  The operation unit 42 includes a release button 2, a power button 3, a shooting / playback switching dial 4, and a wide-angle zoom provided on the external surface of the digital camera 1 (see FIGS. 1A, 1B, and 1C). The switch 10, the telephoto zoom switch 11, the menu button 12, the confirm button 13, etc., and a predetermined operation instruction signal is input to the control unit 28 by the photographer's operation.

  In the SDRAM 23, the RAW-RGB data captured by the CCD I / F 34 is stored, and the YUV data (YUV format image data) converted by the YUV converter 36 and the YUV combined by the YUV combiner 37 are combined. Data (details will be described later) are stored, and image data such as JPEG data compressed by the data compression unit 40 is stored.

  The YUV of the YUV data is color information based on luminance data (Y), color difference (difference (U) between luminance data and blue (B) data, and difference (V) between luminance data and red (R)). Is a format that expresses

(Digital camera monitoring and still image shooting)
Next, the monitoring operation and still image shooting operation of the digital camera 1 will be described. In the still image shooting mode, the digital camera 1 performs a still image shooting operation while executing a monitoring operation as described below.

  First, when the photographer turns on the power button 3 and sets the photographing / playback switching dial 4 to the photographing mode, the digital camera 1 is activated in the recording mode. When the control unit 28 detects that the power button 3 is turned on and the shooting / playback switching dial 4 is set to the shooting mode, the control unit 28 outputs a control signal to the motor driver 25 to switch the lens barrel unit 6. The camera 20 is moved to a photographing position and the CCD 20, the AFE unit 21, the signal processing unit 22, the SDRAM 23, the ROM 24, the liquid crystal monitor 9 and the like are activated.

  Then, by directing the photographic lens system 5 of the lens barrel unit 6 toward the subject, a subject image incident through the photographic lens system 5 is formed on the light receiving surface of each pixel of the CCD 20. Then, an electrical signal (analog RGB image signal) corresponding to the subject image output from the CCD 20 is input to the A / D conversion unit 33 via the CDS 31 and the AGC 32, and 12 bits (bit) by the A / D conversion unit 33. To RAW-RGB data.

  This RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22 and stored in the SDRAM 23 via the memory controller 35. The RAW-RGB data read from the SDRAM 23 is converted into YUV data (YUV signal) in a format that can be displayed by the YUV converter 36, and then the YUV data is stored in the SDRAM 23 via the memory controller 35. The

  The YUV data read from the SDRAM 23 via the memory controller 35 is sent to the liquid crystal monitor (LCD) 9 via the display output control unit 39, and a photographed image (moving image) is displayed. At the time of monitoring in which a photographed image is displayed on the liquid crystal monitor (LCD) 9 described above, one frame is read out in a time of 1/30 second by thinning out the number of pixels by the CCD I / F 34.

  In this monitoring operation, the photographed image is only displayed on the liquid crystal monitor (LCD) 9 functioning as an electronic viewfinder, and the release button 2 is not yet pressed (including half-pressed).

  The photographer can confirm the photographed image by displaying the photographed image on the liquid crystal monitor (LCD) 9. In addition, it can output as a TV video signal from the display output control part 39, and a picked-up image (moving image) can also be displayed on external TV (television) via a video cable.

  The CCD I / F 34 of the signal processing unit 22 calculates an AF (automatic focus) evaluation value, an AE (automatic exposure) evaluation value, and an AWB (auto white balance) evaluation value from the captured RAW-RGB data.

  The AF evaluation value is calculated by, for example, the output integrated value of the high frequency component extraction filter or the integrated value of the luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By utilizing this, at the time of AF operation (at the time of focus detection operation), an AF evaluation value at each focus lens position in the photographing lens system 5 is acquired, and the point where the maximum is obtained is used as the focus detection position for AF. The action is executed.

  The AE evaluation value and the AWB evaluation value are calculated from the integrated values of the RGB values in the RAW-RGB data. For example, the screen corresponding to the light receiving surfaces of all the pixels of the CCD 20 is equally divided into 256 areas (16 horizontal divisions and 16 vertical divisions), and the RGB integration of each area is calculated.

  Then, the control unit 28 reads the calculated RGB integrated value, and in the AE process, calculates the luminance of each area of the screen and determines an appropriate exposure amount from the luminance distribution. Based on the determined exposure amount, exposure conditions (the number of electronic shutters of the CCD 20, the aperture value of the aperture unit 26, etc.) are set. In the AWB process, an AWB control value that matches the color of the light source of the subject is determined from the RGB distribution. By this AWB process, white balance is adjusted when the YUV conversion unit 36 performs conversion to YUV data. The AE process and AWB process described above are continuously performed during the monitoring.

  When the still image shooting operation in which the release button 2 is pressed (half-pressed to full-press) is started during the monitoring operation described above, the AF operation and the still image recording process that are the focus position detection operations are performed. .

  That is, when the release button 2 is pressed (half-pressed to full-pressed), the focus lens of the photographing lens system 5 is moved by a drive command from the control unit 28 to the motor driver 25, and is referred to as so-called hill-climbing AF, for example. The contrast evaluation AF operation is executed.

  When the AF (focusing) target range is the entire region from infinity to close, the focus lens of the photographing lens system 5 moves to each focus position from close to infinity, or from infinity to close, and CCDI / The control unit 28 reads out the AF evaluation value at each focus position calculated in F34. Then, the focus lens is moved to the in-focus position with the point where the AF evaluation value at each focus position is maximized as the in-focus position, and in-focus.

  Then, the AE process described above is performed, and when the exposure is completed, the mechanical shutter unit 27 is closed by a drive command from the control unit 28 to the motor driver 25, and an analog RGB image signal for a still image is output from the CCD 20. As in the monitoring, the A / D conversion unit 33 of the AFE unit 21 converts the data into RAW-RGB data.

  The RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22, converted into YUV data by a YUV conversion unit 36 described later, and stored in the SDRAM 23 via the memory controller 35. The YUV data is read from the SDRAM 23, converted into a size corresponding to the number of recorded pixels by the resizing processing unit 38, and compressed to image data in the JPEG format or the like by the data compression unit 40. The compressed image data in JPEG format or the like is written back to the SDRAM 23, then read from the SDRAM 23 via the memory controller 35, and stored in the memory card 14 via the media I / F 41.

(Dynamic range expansion processing by the YUV converter 36)
The YUV conversion unit 36 of the digital camera 1 according to the present embodiment has a dynamic range expansion processing function for expanding the dynamic range.

  A Bayer array RGB filter (see FIG. 7) is arranged on each pixel constituting the CCD 20 of the digital camera 1, but a normal RGB filter for light having a wide wavelength band such as sunlight. Have different luminance sensitivities for each color.

  For example, as shown in FIG. 3, an RGB filter (a, b, c in FIG. 3) has a sensitivity of a G (green) filter that is about twice that of an R (red) filter and a B (blue) filter. In the case of the CCD 20 having the same, when the same amount of light having a wide wavelength band, such as sunlight, is incident on the RGB filter, the G filter (shaded portion in FIG. 3c) is applied to each pixel output of the R and B filters. The pixel output first reaches the saturation level A. In FIG. 3, f is the pixel sensitivity characteristic of the G filter, g is the pixel sensitivity characteristic of the R and B filters, and the pixel sensitivity characteristic of the G filter is about twice the pixel sensitivity characteristics of the R and B filters. It has the sensitivity of.

  By the way, in a digital camera having a solid-state image sensor such as a CCD having a conventional RGB filter, a saturation level A corresponding to the pixel output of a highly sensitive G filter, such as the RGB filters of a, b, and c in FIG. In addition, the dynamic range is set. That is, even when the pixel output of the G filter reaches the saturation level A, the pixel output of the R and B filters is about ½ of the saturation level A.

  On the other hand, in the present invention, even if the pixel output of the G filter exceeds the saturation level A, as in the RGB filters of d and e in FIG. From within the pixel output level of the R and B filters, the pixel sensitivity characteristics of the R and B filters (g in FIG. 3) and the pixel sensitivity characteristics of the G filter (f in FIG. 3) Based on the above, the pixel output level of the G filter is subjected to predictive interpolation (dotted line portion), and the dynamic range is expanded by this predicted interpolation.

  The dynamic range expansion processing operation in this embodiment will be described below.

  In the present embodiment, when the photographer presses the menu (MENU) button 12 (see FIG. 1C), for example, a photographing setting screen as shown in FIG. 4 is displayed on the liquid crystal monitor (LCD) 9. By selecting the item “dynamic range doubling” from this display screen, a control signal is output from the control unit 28 to the YUV conversion unit 36, and a processing operation to double the dynamic range is executed.

  For example, when there is an extremely bright part of the background of the subject, the menu (MENU) button 12 is pressed according to the photographer's judgment and the item “dynamic range double” is selected to Range expansion processing is performed.

  In the present embodiment, as described above, it is assumed that the pixel sensitivity characteristic of the G filter has a sensitivity about twice that of each pixel sensitivity characteristic of the R and B filters. Under the blue light source, the pixel sensitivity of the R and B filters may be saturated rather than the pixel sensitivity of the G filter. If the dynamic range expansion process described above is performed under such circumstances, accurate gradation and color reproduction cannot be obtained. In such a case, the item “dynamic range doubling” described above is determined by the photographer. Do not select.

  The dynamic range expansion process is performed by the YUV conversion unit 36. As shown in FIG. 5, the YUV conversion unit 36 includes a dynamic range expansion prediction interpolation unit (hereinafter referred to as “D range expansion prediction interpolation unit”) 50, a bit compression conversion unit 51, a white balance control unit 52, and a synchronization unit 53. A tone curve converter 54, an RGB-YUV converter 55, an image size converter 56, a luminance histogram generator 57, and an edge enhancer 58.

  In the dynamic range expansion process according to the present embodiment, RAW-RGB data (A in FIG. 5) is input to the D range expansion prediction interpolation unit 50, which will be described later, and RAW-RGB data (FIG. 5). B) is input to the white balance control unit 52 without being input to the D range expansion prediction interpolation unit 50 side. First, the first process will be described. The RAW-RGB data (A in FIG. 5) input to the D range expansion prediction interpolation unit 50 and the RAW− input to the white balance control unit 52 without being input to the D range expansion prediction interpolation unit 50 side. The RGB data (B in FIG. 5) is one RAW-RGB data read from the SDRAM 23.

(First process)
As shown in FIG. 6, the D range expansion prediction interpolation unit 50 detects the pixel output of each pixel provided with an RGB filter from the RAW-RGB data (A in FIG. 5) read from the SDRAM 23, and has the highest sensitivity. A luminance level determination unit 60 that determines whether the pixel output of a pixel provided with a high G filter (hereinafter referred to as “G filter pixel output”) has reached a saturation level or more, and the luminance level determination unit 60 uses the G filter. When the pixel output of the pixel is determined to have exceeded the saturation level, the pixel output of the pixel provided with the surrounding R and B filters (hereinafter referred to as “pixel output of the R and B filters”) has reached the saturation level or more. The pixel output correction processing unit 61 that performs predictive interpolation processing of the pixel output of the G filter and the luminance level determination unit 60 determine that the pixel output of the G filter has not reached the saturation level. A bit expansion processing unit 62 that performs only bit expansion from 12 bits to 14 bits without converting the output level of the G filter pixel output and the R and B filter pixel outputs. ing.

  When the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, in the present embodiment, for each pixel of the CCD 20 having the RGB filter, as shown in FIG. In addition, 2 × 2 pixels in the thick frame A (two G filter pixels, Gr and Gb, one R filter pixel and one R filter pixel) are set as processing units (minimum units). When at least one pixel output of two G filter pixels in the processing unit (thick frame A) has reached the saturation level or higher, the sensitivity of the G filter is the R and B filters as described above. Therefore, the pixel output values (Gr ′, Gb ′) of the G filter are calculated from the following equation (1). In Equation (1), coefficients kr, kb, and kgb are coefficients set based on the white balance gain.

Gr ′ = krR + kbB + kgbGb
Gb ′ = krR + kbB + kgrGr (1)
The pixel output correction processing unit 61 multiplies the pixel outputs of the R, B, and Gb filters by the coefficients (kr, kb, kgb) and adds them together as shown in Expression (1), thereby outputting the pixel output of the Gr filter. A value Gr ′ is calculated. Similarly, for the Gb filter, the pixel output value Gb ′ is calculated as in Expression (1), and the pixel values of Gr and Gb in the processing unit (2 × 2 pixels) are replaced. Since the pixel output values (Gr ′, Gb ′) of the G filter are data exceeding 12 bits, they are replaced with 14-bit data here. Therefore, since the maximum values of the pixel outputs of the R and B filters are both 4095 (12 bits), the maximum value of the pixel output of the G filter is 8190 and can be handled as 14-bit data.

  By the way, when the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, it is necessary not to use the defective pixel for this determination. In other words, if a pixel with a G filter has a defective pixel and a value that is always saturated is output, the pixel with the G filter in the same processing unit is replaced with a large value, so a new defective pixel is generated. Will end up. Further, when a pixel provided with the R and B filters has a defective pixel, the conversion of the pixel provided with the G filter according to the equation (1) becomes an incorrect value.

  For this reason, when the brightness level determination unit 60 determines that there is a defective pixel in the processing unit (2 × 2 pixels), a filter having the same color as the defective pixel around the processing unit is used without using the defective pixel. The pixel having is used instead. As a result, when the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, the defective pixel is not used, so that the output of the pixel in which the G filter is disposed is used. On the other hand, highly accurate predictive interpolation can be performed when a predetermined saturation level or higher is reached.

  The D range expansion prediction interpolation unit 50 outputs the R and B filter pixel output data and the G filter pixel output data that has been subjected to the prediction interpolation processing after reaching the saturation level to the bit compression conversion unit 51. The bit compression conversion unit 51, for example, has a conversion characteristic as shown in FIG. 8A (in FIG. 8A, designates three nodes and approximates them by a straight line between the four sections. ), The pixel output of the G filter among the pixel outputs of the R, G, and B filters expanded to 14 bits is reduced to 12 bits. In FIG. 8A, a is a 12-bit range, and b is a simple linear conversion characteristic (a portion indicated by a one-dot chain line) that halves the data of the maximum value 8190.

  In the conversion characteristics shown in FIG. 8A, since the maximum value of the pixel output of the G filter is 8190, compression is performed so that 8190 becomes 4095. Then, the pixel output values of the R and B filters are also compressed in accordance with the compression magnification of the pixel output of the G filter.

  As described above, this embodiment has three nodes as shown in FIG. 8A as an example of the case where the pixel output of the G filter whose maximum value is expanded to 8190 is compressed to 4095. Conversion characteristics (part indicated by a solid line) were used. In the present embodiment, the following two effects that cannot be obtained with a linear conversion characteristic without a simple node (b in FIG. 8A) are obtained.

  As a first effect, a larger number of bits can be assigned to data with high data reliability. That is, when predictive interpolation processing is performed on the pixel output of the G filter that has reached the saturation level or higher, the prediction interpolation is performed for the range in which the value of the pixel output of the G filter exceeds the specified value near the saturation level as described above. The prediction interpolation is not performed in the range below the specified value. Therefore, the accuracy of the data differs between the range in which predictive interpolation is performed and the range in which it is not performed.

  That is, for example, when predictive interpolation (correction) is performed on the pixel output value of the G filter that is saturated according to the equation (1), the luminance level of the subject is accurately reproduced in the range where the predictive interpolation is performed depending on the color of the main subject. It may not be possible. On the other hand, the range in which the prediction interpolation is not performed is data obtained by A / D conversion of actual data (analog RGB image signal) output from the CCD 20 having the RGB filter, and thus the reliability of this data is high. It becomes.

  That is, in the conversion characteristic (part indicated by the solid line) in this embodiment shown in FIG. 8A, for example, when the input 14-bit data is 1024, the output 12-bit data is 1024, and the original data is It is used as it is. On the other hand, for example, when the input 14-bit data is 3072, the output 12-bit data is 2560. In this range, the bit allocation is smaller than the bit allocation before the prediction interpolation, so that a bit error occurs. .

  In this way, by adopting a conversion characteristic (part indicated by a solid line) having three nodes as in this embodiment, instead of a characteristic for performing linear conversion without a simple node (part indicated by a one-dot chain line). Since the bit allocation can be gradually reduced, a larger number of bits can be allocated to data with high data reliability.

  As a second effect, gradations at low and medium luminance can be accurately stored. That is, when bit compression is performed with a simple linear conversion characteristic (b in FIG. 8A), the bit allocation becomes ¼ in a range where predictive interpolation on the low luminance side is not performed. For this reason, the image has no tone. On the other hand, when bit compression is performed with the conversion characteristics (part indicated by the solid line) in this embodiment as shown in FIG. 8A, it corresponds to the pixel output at the low luminance level below the saturation level. With respect to data, it is possible to satisfactorily maintain gradation at a low luminance level by using a compression rate that is substantially the same value before bit compression by the bit compression conversion unit 51 and after bit compression. it can.

  In this embodiment, when the 14-bit data of the pixel output of the expanded G filter is reduced to 12 bits, three nodes are designated as shown in FIG. 8A and approximated by a straight line between them. Although the bit compression is performed with the broken line approximation characteristic (conversion characteristic) of four sections, the number of sections is not particularly limited. For example, it may be a polygonal line approximate characteristic of two sections designating one node, but the above-mentioned two effects are reduced by changing the bit allocation in the vicinity of the node. Therefore, a broken line approximation characteristic (conversion characteristic) having three or more sections is preferable.

  Also, the conversion characteristic for compressing the 14-bit data of the pixel output of the expanded G filter to 12 bits may be a conversion characteristic by a curve not having a plurality of nodes as shown in FIG. That is, with respect to the conversion characteristic having four sections in FIG. 8A, the conversion characteristic by this curve is obtained by setting the number of sections to 8192. In FIG. 8B, a is a 12-bit range.

  Further, by providing a lookup table having numerical data after compression conversion to 12 bits for 0 to 8192 of input 14-bit data, the pixel output of the expanded G filter with the conversion characteristics by this curve is provided. The 14-bit data can be successfully compressed to 12 bits.

  The pixel output data of the R, G, and B filters that have been compression-converted from 14 bits to 12 bits by the bit compression conversion unit 51 are input to the white balance control unit 52. The white balance control unit 52 amplifies input RAW-RGB data (pixel output data of R, G, and B filters). At this time, the control unit 28 calculates a correction value for adjusting the white balance based on the AWB evaluation value calculated by the CCD I / F 34 and outputs the calculated correction value to the white balance control unit 52. The white balance control unit 52 adjusts the white balance based on the input correction value.

  Then, the pixel output data (12 bits) of the R, G, and B filters with the white balance adjusted from the white balance control unit 52 is input to the synchronization unit 53. The synchronizer 53 performs interpolation calculation processing on RAW data having only one color data per pixel, such as a Bayer array, and generates all RGB data for one pixel.

  All the RGB data (12 bits) generated by the synchronization unit 53 is input to the tone curve conversion unit 54. The tone curve conversion unit 54 performs γ conversion for converting 12-bit RGB data into 8-bit RGB data using a conversion table as shown in FIG. 9 to generate an 8-bit RGB value, and RGB-YUV The data is output to the conversion unit 55.

  The RGB-YUV conversion unit 55 converts input RGB data (8 bits) into YUV data by matrix calculation and outputs the YUV data to the image size converter unit 56. The image size converter unit 56 reduces or enlarges the input YUV data (8 bits) to a desired image size, and outputs it to the luminance histogram generation unit 57 and the edge enhancement unit 57.

  The luminance histogram generation unit 57 generates a luminance histogram based on the input YUV data. The edge enhancement unit 57 performs processing such as edge enhancement according to the image on the input YUV data, and the YUV data (hereinafter referred to as “first YUV”) on which the prediction interpolation processing of the pixel output of the G filter is performed. Data ”) is stored in the SDRAM 23 via the memory controller 35.

(Second process)
Next, the second process will be described. In the second process, the RAW-RGB data (B in FIG. 5) read from the SDRAM 23 is input to the white balance control unit 52 without being input to the D range expansion prediction interpolation unit 50. Hereinafter, it is the same as the first processing.

  That is, the white balance control unit 52 amplifies input RAW-RGB data (pixel output data of R, G, and B filters). At this time, the control unit 28 calculates a correction value for adjusting the white balance based on the AWB evaluation value calculated by the CCD I / F 34 and outputs the calculated correction value to the white balance control unit 52. The white balance control unit 52 adjusts the white balance based on the input correction value.

  Then, the pixel output data (12 bits) of the R, G, and B filters with the white balance adjusted from the white balance control unit 52 is input to the synchronization unit 53. The synchronizer 53 performs interpolation calculation processing on RAW data having only one color data per pixel, such as a Bayer array, and generates all RGB data for one pixel.

  All the RGB data (12 bits) generated by the synchronization unit 53 is input to the tone curve conversion unit 54. The tone curve conversion unit 54 performs γ conversion for converting 12-bit RGB data into 8-bit RGB data using a conversion table as shown in FIG. 9 to generate an 8-bit RGB value, and RGB-YUV The data is output to the conversion unit 55.

  The RGB-YUV conversion unit 55 converts input RGB data (8 bits) into YUV data by matrix calculation and outputs the YUV data to the image size converter unit 56. The image size converter unit 56 reduces or enlarges the input YUV data (8 bits) to a desired image size and outputs it to the edge enhancement unit 57.

  The edge enhancement unit 57 performs processing such as edge enhancement in accordance with the image on the input YUV data, and normal processing YUV data (hereinafter referred to as “first”) in which the prediction interpolation processing of the pixel output of the G filter has not been performed. 2YUV data ”) is stored in the SDRAM 23 via the memory controller 35.

  The YUV synthesis unit 37 extracts only Y data (luminance data) from the first YUV data read from the SDRAM 23, and extracts only UV data (color difference data) from the second YUV data read from the SDRAM 23. YUV data (hereinafter referred to as “third YUV data”) is synthesized. The synthesized third YUV data is stored in the SDRAM 23 via the memory controller 35.

  The third YUV data is read from the SDRAM 23, converted into a size corresponding to the number of recorded pixels by the resizing processing unit 38, and compressed to image data such as JPEG format by the data compression unit 40. The compressed image data in JPEG format or the like is written back to the SDRAM 23, then read from the SDRAM 23 via the memory controller 35, and stored in the memory card 14 via the media I / F 41.

  As described above, in the present embodiment, saturation is performed based on the pixel outputs of the low-sensitivity R and B filters even when the pixel output of the high-sensitivity G filter within the processing unit has reached the saturation level or higher. By performing predictive interpolation processing on the pixel output of the G filter, the expanded region obtained by predictive interpolation of the pixel output of the G filter (d, e in FIG. 3) (d, e in FIG. 3) Based on the pixel output of the G filter, the dynamic range can be doubled by one shooting.

  Therefore, even when there is a high-luminance portion in the background or the like in the captured image, it is possible to prevent the occurrence of overexposure and obtain good gradation.

  Further, in the present embodiment, one RAW-RGB data is read from the SDRAM 23, and the first YUV data generated from the pixel output predicted and interpolated by the pixel output correction processing unit 61 by the first processing, and the second The second YUV data generated from the pixel output that has reached the predetermined saturation level or higher without performing the predictive interpolation process in the pixel output correction processing unit 61 by the process of the above is taken into the YUV synthesis unit 37, and the YUV synthesis unit 37 performs the first YUV synthesis. Y data (luminance data) extracted from the data and UV data (color difference data) extracted from the second YUV data are combined to generate third YUV data.

  Thereby, only the Y data (luminance data) of the first YUV data generated from the pixel output predicted and interpolated by the pixel output correction processing unit 61 is expanded, and the original UV data (color difference data) is used as the color. By using it, it is possible to prevent color shift and accurately reproduce colors.

  For example, when the prediction interpolation process of the pixel output of the G filter that has reached the saturation level by the D range expansion prediction interpolation unit 50 is performed using, for example, the equation (1), and the dynamic range expansion process is performed, for example, When the main subject is magenta, the pixel output of the R and B filters is increased. For this reason, the pixel output of the G filter is calculated to a value larger than the original value accordingly, and as a result, accurate color reproduction may not be possible.

  Even in such a situation, by performing the processing operation according to the above-described embodiment, the dynamic range is expanded only for the Y data (luminance data) of the first YUV data, and the color is the original UV data (color difference data). By using, it is possible to prevent color shift and perform accurate color reproduction.

  FIG. 10A shows a histogram generated by the luminance histogram generation unit 57 when the dynamic range expansion process in the present embodiment described above is performed when the pixel output of the G filter exceeds the saturation level. It is an example. As is apparent from this histogram, by performing the dynamic range expansion processing, whiteout in the maximum luminance portion (255) hardly occurs, and reproduction is performed with good gradation.

  On the other hand, FIG. 10B is generated by the luminance histogram generation unit 57 when the dynamic range expansion process in the present embodiment is not performed when the pixel output of the G filter exceeds the saturation level. It is an example of the done histogram. As is apparent from this histogram, it can be seen that, by not performing the dynamic range expansion process, the maximum luminance portion (255) has a frequency and whiteout occurs.

  In the description of the first embodiment and FIG. 3, the saturation level A in FIG. 3 that is a predetermined saturation level determination value and 4095 that is the maximum value of 12 bits after correction are matched. However, the present invention is not limited to this. For example, in a CCD having an RGB filter whose output linearity (linearity) is not good in a high-luminance part near where the output is completely saturated, for example, 4032 which is a value smaller than 4095 which is completely saturated is set to a predetermined saturation level. As a value (saturation level A in FIG. 3), a pixel output exceeding the value may be a target of the predictive interpolation process.

  Also, depending on the system configuration of the digital camera, even a high-luminance subject may not reach 4095, which is the maximum value of 12 bits. In that case as well, the predetermined saturation level may be set to a value lower than 4095.

  As described above, even when the predetermined saturation level is less than 4095, the output of the bit compression conversion unit 51 can be set to 4095 by switching the conversion curve of FIG. 9 according to the characteristics, and the subsequent processing can be performed. The dynamic range can be expanded without change.

  In the present embodiment, as shown in FIG. 5, 14-bit RAW-RGB data (pixel output data of R, G, and B filters) output from the D range expansion prediction interpolation unit 50 is converted into a bit compression conversion unit. 51, the white balance control unit 52 and the synchronization unit 53 perform 12-bit data processing. However, in addition to this, the bit compression conversion unit 51 is provided after the synchronization unit 53. The 14-bit data output from the synchronization unit 53 may be compressed into 12-bit data.

<Embodiment 2>
In the first embodiment, as shown in FIG. 7, 2 × 2 pixels are used as the processing unit (minimum unit) for the CCD 20 having the RGB primary color filters. However, in this embodiment, as shown in FIG. 11, 5 pixels within the thick frame A (one G filter pixel (Gb) in the center, two R (R1, R2) filter pixels in the vertical direction, and two B (B1, B2) filter pixels in the horizontal direction ) Is a processing unit (minimum unit), and the processing unit is a wider range than in the first embodiment. The configuration of the digital camera, the monitoring operation, the still image shooting operation, and the dynamic range expansion processing operation are the same as those in the first embodiment.

  When the pixel output of the G filter (Gb) in the processing unit of the thick frame A shown in FIG. 11 has reached the saturation level or higher, the sensitivity of the G filter is about the sensitivity of the R and B filters as described above. Since it is twice, the pixel output value (G ′) of the G filter is calculated from the following equation (2). In Equation (2), coefficients kr and kb are coefficients set based on the white balance gain.

G ′ = {kr (R1 + R2) / 2} + {kb (B1 + B2) / 2} (2)
Then, the pixel output correction processing unit 61 of the D range expansion prediction interpolation unit 50 shown in FIG. 6 sets the pixel output value of the G filter calculated by the equation (2) within the processing unit (see FIG. 11). The pixel output value of a certain G filter is replaced, and the same processing as in the first embodiment is performed.

  In this way, by widening the processing unit, it is possible to reduce the influence due to the sensitivity difference of the other R1, R2 filter pixels and B1, B2 filter pixels in the processing unit. More accurate dynamic range expansion prediction interpolation can be performed on the output.

<Embodiment 3>
In the present embodiment, as shown in FIG. 12, for a CCD 20 having RGB filters, 5 × 5 pixels (13 pixels of G filters (Gr , Gb), and six R, B filters) as processing units. The configuration of the digital camera, the monitoring operation, the still image shooting operation, and the dynamic range expansion processing operation are the same as those in the first embodiment.

  If the processing unit is wider than those in the first and second embodiments, processing is performed based on a wide range of luminance information, which is equivalent to applying a low-pass filter. Therefore, the edge portion of the luminance change is lost. Therefore, in the present embodiment, the size of the widened processing unit is partially changed using, for example, the AF evaluation value.

  That is, the CCD I / F 34 of the signal processing unit 22 shown in FIG. 2 calculates an AF evaluation value for performing AF as described above. This is an output of a high-pass filter (HPF), and a large value is output in a portion where there is a change in luminance in the screen of the captured image. Then, the control unit 28 reads out an AF evaluation value at the time of still image shooting, and discriminates between a portion having a luminance change and a portion having no luminance change in the screen. Then, the control unit 28 sets the D range expansion prediction interpolation unit 50 based on this determination data so that the processing unit is narrowed in a portion where there is a luminance change, and in a portion where there is no luminance change, As shown in FIG. 12, setting is performed so that the processing unit is in a wide range.

  Thus, even when the processing unit is further widened, accurate dynamic range expansion prediction interpolation can be performed without reducing the resolution by setting the processing unit to be narrower in a portion where there is a luminance change. .

  In each of the embodiments described above, the RGB three primary color filters are arranged as the color separation filters. However, the present invention is similarly applied to a configuration in which the complementary color filters are arranged as the color separation filters. Can do.

(A) is a front view which shows the digital camera as an example of the imaging device which concerns on Embodiment 1 of this invention, (b) is the top view, (c) is the back view. 1 is a block diagram showing an outline of a system configuration in a digital camera as an example of an imaging apparatus according to Embodiment 1 of the present invention. The figure for demonstrating the principle of the dynamic range expansion in Embodiment 1 of this invention. The figure which shows an example of the imaging | photography setting screen displayed on the liquid crystal monitor. The block diagram which shows the structure of the YUV conversion part in Embodiment 1 of this invention. The block diagram which shows the structure of the D range expansion prediction interpolation part in Embodiment 1 of this invention. The figure which shows the pixel arrangement position and processing unit of CCD which have the RGB filter in Embodiment 1 of this invention. (A) is a figure which shows the conversion characteristic which compresses the 14-bit data of the pixel output of the extended G filter in Embodiment 1 of this invention to 12 bits, (b) is other of Embodiment 1 of this invention The figure which shows the conversion characteristic which compresses 14 bits data of the pixel output of the extended G filter in an example to 12 bits. The figure which shows the conversion table which converts 12-bit RGB data into 8-bit RGB data (gamma conversion). (A) is a figure which shows the histogram at the time of performing the expansion process of the dynamic range in Embodiment 1 of this invention, (b) shows the histogram at the time of not performing the expansion process of the dynamic range in this embodiment. Figure. The figure which shows the pixel arrangement position and processing unit of CCD which have the RGB filter in Embodiment 2 of this invention. The figure which shows the pixel arrangement position and processing unit of CCD which have an RGB filter in Embodiment 3 of this invention.

Explanation of symbols

1 Digital camera (imaging device)
5 Shooting lens system (optical system)
6 Lens unit 9 LCD monitor 12 Menu button (operation selection means)
20 CCD (imaging device)
21 Analog Front End 22 Signal Processor 23 SDRAM
28 Control Unit 34 CCD Interface 35 Memory Controller 36 YUV Conversion Unit 37 YUV Composition Unit (YUV Composition Unit)
50 D range expansion prediction interpolation unit 51 bit compression conversion unit (bit compression conversion means)
60 Luminance level determination unit (pixel output detection means)
61 Pixel output correction processing unit (pixel output correction processing means)
62-bit extension processing section

Claims (12)

An object image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and an image pickup device in which a plurality of color separation filters are arranged on the front side of each pixel is provided.
Pixel output detection means for determining whether the output from any pixel reaches or exceeds a predetermined saturation level with respect to the output from each pixel;
When the pixel output detection means determines that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or higher, the pixel in which the filter other than the specific color is arranged around it Pixel output correction processing means for interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output from
From the first YUV data generated from the pixel output subjected to the interpolation processing by the pixel output correction processing means, and the pixel output reaching the predetermined saturation level or higher without performing the interpolation processing by the pixel output correction processing means. YUV synthesizing means that takes in the generated second YUV data and generates third YUV data by synthesizing the luminance data extracted from the first YUV data and the color difference data extracted from the second YUV data,
An imaging apparatus characterized by that. The processing unit when the output of each pixel is detected by the pixel output detection means is a size of 2 × 2 pixels in the horizontal and vertical directions.
The imaging apparatus according to claim 1. An operation selecting means for selecting and executing the operation to be interpolated when the predetermined saturation level is reached or higher with respect to the output of the pixel in which the filter of the specific color is disposed;
The imaging apparatus according to claim 1 or 2, wherein When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined saturation level, the pixel output correction processing means outputs the first bit number below the predetermined saturation level. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits of 2 to the first number of bits again,
The bit compression conversion means compresses the compression corresponding to the pixel output at the saturation level or lower than the compression corresponding to the data corresponding to the pixel output in the region above the saturation level.
The imaging apparatus according to any one of claims 1 to 3, wherein For the data corresponding to the pixel output at the low luminance level below the saturation level, use a compression ratio that is substantially the same value before and after bit compression by the bit compression conversion means,
The imaging apparatus according to claim 4. When there is a defective pixel in the processing unit, a pixel in which a filter of the same color as the defective pixel around the defective pixel is arranged instead of the defective pixel.
The imaging device according to any one of claims 1 to 5, wherein An imaging device including an imaging device in which a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a plurality of color separation filters are arranged on the front side of each pixel. In the imaging method of the apparatus,
A determination processing step for determining whether the output from any of the pixels has reached a predetermined saturation level or higher with respect to the output from each of the pixels;
When it is determined in the determination processing step that the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more, the surrounding pixels from which the filter other than the specific color is arranged An interpolation processing step of interpolating the pixel output determined to be equal to or higher than the predetermined saturation level based on the output of
First YUV data generated from the pixel output subjected to the interpolation processing in the interpolation processing step, and first output generated from the pixel output that has reached the predetermined saturation level or higher without performing the interpolation processing in the interpolation processing step. YUV composition processing step of taking in 2YUV data and synthesizing luminance data extracted from the first YUV data and color difference data extracted from the second YUV data to generate third YUV data,
An imaging method characterized by the above. The processing unit when detecting the output of each pixel in the determination processing step is a size of 2 × 2 pixels in the horizontal and vertical directions.
The imaging method according to claim 7. The operation of performing the interpolation when the output from the pixel in which the filter of the specific color is arranged has reached the predetermined saturation level or more is performed by selecting an operation with an operation selection unit.
The imaging method according to claim 7 or 8, wherein When the output of the pixel in which the filter of the specific color is disposed has reached the predetermined saturation level or higher, the second output from the first number of bits below the predetermined saturation level is output from the interpolation processing step. A bit compression conversion step of compressing the pixel output data expanded once to the number of bits into the first number of bits again;
In the bit compression conversion step, compression is performed by reducing the compression rate for data corresponding to the pixel output at the saturation level or lower than the compression rate for data corresponding to the pixel output in the region above the saturation level,
The imaging method according to claim 7, wherein: The data corresponding to the pixel output at the low luminance level below the saturation level is compressed using a compression ratio that is substantially the same value before and after bit compression in the bit compression conversion step. ,
The imaging method according to claim 10. When there is a defective pixel in the processing unit, a pixel in which a filter of the same color as the defective pixel around the defective pixel is arranged instead of the defective pixel.
The imaging method according to any one of claims 7 to 11, wherein:

Images