JP5091734B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method such as a digital still camera and a digital video camera, and more particularly to an imaging apparatus and an imaging method that can expand a dynamic range of a captured image.

  Compared to the dynamic range of images taken with a conventional silver salt camera using a silver salt photographic film, the dynamic range of images taken with a digital still camera or a digital video camera having a solid-state image sensor such as a CCD is extremely narrow. When the dynamic range is narrow, a phenomenon called “blackout” occurs in the dark part of the subject, and conversely, a phenomenon called “overexposure” occurs in the bright part of the subject, and the image quality deteriorates.

Therefore, in order to expand the dynamic range of an image captured by a solid-state imaging device such as a CCD, for example, the same subject is shot multiple times with different exposure amounts to obtain a plurality of images with different exposure amounts. A technique for adding these images to generate a composite image with an expanded dynamic range has been known (see, for example, Patent Document 1).
JP 2000-92378 A

  By the way, in order to expand the dynamic range, in the method of performing photographing a plurality of times while changing the exposure amount as in the above-mentioned Patent Document 1, when photographing a moving object, the subject is doubled. Therefore, the image may not be synthesized correctly.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method capable of expanding the dynamic range by one shooting without changing the exposure amount and performing a plurality of shootings to combine images. .

In order to achieve the above object, according to the first aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and the front side of each pixel is provided. An image pickup device having a color separation filter of three primary color filters or a complementary color filter is provided, and a specific color filter of the color separation filters is higher than other color filters for light having a wide wavelength band In the imaging apparatus having luminance sensitivity, the output from each pixel in which the color separation filter is arranged is detected based on an electrical signal output from the imaging element, and any one of the pixels A pixel output detecting means for determining whether or not the output from the pixel has reached a predetermined specified value or more; and the pixel output detecting means determines that there is a pixel having an output exceeding the predetermined specified value. In this case, a pixel output correction unit that corrects the output of the pixel that is equal to or greater than the predetermined specified value based on the output from the surrounding pixels, and a correction coefficient for performing the correction by the pixel output correction unit are set. A correction coefficient setting unit, a shooting scene determination unit that determines a shooting scene for a subject, a face detection unit that detects a human face from an electric signal output from the image sensor, and the filter of the specific color are arranged. Output from the pixel output correcting means when the pixel output reaches or exceeds the predetermined specified value, and is once expanded from the first bit number below the predetermined specified value to the second bit number. Bit compression conversion means for compressing pixel output data again to the first number of bits, wherein the bit compression conversion means converts the pixel output to a pixel output in the region above the predetermined specified value. Than the compression rate for the response data is compressed by reducing the compression ratio for the data corresponding to the pixel output in the following the predetermined specified value, the correction coefficient setting means, photographic scene information determined by the photographic scene determination means The face detection information by the face detection means is input, and a correction coefficient for performing the correction by the pixel output correction means is set based on the input information.

According to a second aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a three primary color filter or a complementary color system is provided on the front side of each pixel. An image pickup device having a color separation filter is disposed, and a specific color filter of the color separation filter has higher luminance sensitivity than light of other colors with respect to light having a wide wavelength band. In the imaging device, based on an electrical signal output from the imaging element, an output from each pixel in which the color separation filter is arranged is detected, and an output from any one of the pixels is a predetermined value. A pixel output detection unit that determines whether or not a predetermined value or more has been reached, and when the pixel output detection unit determines that there is a pixel with an output that is equal to or greater than the predetermined specified value, A pixel output correction unit that corrects an output of a pixel that is equal to or greater than the predetermined specified value based on an output from the element; a correction coefficient setting unit that sets a correction coefficient when performing the correction by the pixel output correction unit; A shooting scene determination unit that determines a shooting scene for the pixel, and an output from the pixel output correction unit when the output of the pixel in which the filter of the specific color has reached the predetermined specified value or more. Bit compression conversion means for compressing the pixel output data once expanded from the first bit number below the prescribed value to the second bit number to the first bit number, and the bit compression conversion means comprises: The pressure on the data corresponding to the pixel output below the predetermined specified value is higher than the compression rate for the data corresponding to the pixel output in the region above the predetermined specified value. Compressed to reduce the rate, the correction coefficient setting means receives the photographic scene information determined by the photographic scene determination means, based on the input photographic scene information, when performing the correction by the pixel output correcting means The correction coefficient is set.

According to a third aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electric signal, and a three primary color filter or a complementary color system is provided on the front side of each pixel. An image pickup device having a color separation filter is disposed, and a specific color filter of the color separation filter has higher luminance sensitivity than light of other colors with respect to light having a wide wavelength band. In the imaging device, based on an electrical signal output from the imaging element, an output from each pixel in which the color separation filter is arranged is detected, and an output from any one of the pixels is a predetermined value. A pixel output detection unit that determines whether or not a predetermined value or more has been reached, and when the pixel output detection unit determines that there is a pixel with an output that is equal to or greater than the predetermined specified value, A pixel output correction unit that corrects an output of a pixel that is equal to or greater than the predetermined specified value based on an output from an element; a correction coefficient setting unit that sets a correction coefficient when performing the correction by the pixel output correction unit; The pixel output when the output of the pixel in which the face detecting means for detecting the face portion of the person from the electrical signal output from the image sensor and the filter of the specific color has reached the predetermined specified value or more Bit compression conversion means for compressing the pixel output data output from the correction means and expanded once from the first bit number below the predetermined prescribed value to the second bit number to the first bit number again And the bit compression conversion means corresponds to the pixel output below the predetermined specified value rather than the compression rate for the data corresponding to the pixel output in the region above the predetermined specified value. The compression ratio for the data to compress small, the correction coefficient setting means inputs the face portion information detected by said face detecting means, based on the input face portion detection information, the correction by the pixel output correcting means It is characterized by setting a correction coefficient when performing.

  According to a fourth aspect of the present invention, there is provided distance detection means for detecting a distance from an electric signal output from the image sensor to a subject, and a color distribution for detecting a color distribution of the subject from the electric signal output from the image sensor. A luminance distribution detection unit that detects a luminance distribution of a subject from an electric signal output from the image sensor; and the photographing scene determination unit includes the distance information detected by the distance detection unit, the color The photographic scene is determined based on one or more of the color distribution information detected by the distribution detection unit and the luminance distribution information detected by the luminance distribution detection unit.

According to a fifth aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a three primary color filter or a complementary color system is provided on the front side of each pixel. An image pickup device having a color separation filter is disposed, and a specific color filter of the color separation filter has higher luminance sensitivity than light of other colors with respect to light having a wide wavelength band. In the imaging method of the imaging apparatus, an output from each pixel in which the color separation filter is arranged is detected based on an electrical signal output from the imaging element, and an output from any one of the pixels A pixel output detection step for determining whether or not the pixel value has reached a predetermined specified value or more, and when it is determined by the pixel output detection step that there is a pixel having an output equal to or higher than the predetermined value, A pixel output correction step for correcting the output of the pixel that is equal to or greater than the predetermined specified value based on an output from a peripheral pixel of the pixel, and a correction coefficient setting step for setting a correction coefficient for performing the correction by the pixel output correction step A shooting scene determination step for determining a shooting scene for a subject, a face detection step for detecting a human face from an electrical signal output from the image sensor, and an output of a pixel in which the filter of the specific color is arranged Pixel output data that is output from the pixel output correction step when the value reaches or exceeds the predetermined specified value and is expanded from the first bit number to the second bit number below the predetermined specified value. A bit compression conversion step that compresses the data again to the first number of bits, wherein the bit compression conversion step includes data corresponding to a pixel output in an area above the predetermined specified value. Than the compression rate for, and compressed to reduce the compression ratio for the data corresponding to the pixel output in the following the predetermined specified value, the correction factor setting step, the photographic scene information and the face that is determined by the photographic scene determination step It is characterized in that face detection information by the detection process is input, and a correction coefficient for performing the correction by the pixel output correction process is set based on the input information.

According to a sixth aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a three primary color filter or a complementary color system is provided on the front side of each pixel. An image pickup device having a color separation filter is disposed, and a specific color filter of the color separation filter has higher luminance sensitivity than light of other colors with respect to light having a wide wavelength band. In the imaging method of the imaging apparatus, an output from each pixel in which the color separation filter is arranged is detected based on an electrical signal output from the imaging element, and an output from any one of the pixels A pixel output detection step for determining whether or not the pixel value has reached a predetermined specified value or more, and when it is determined by the pixel output detection step that there is a pixel having an output equal to or higher than the predetermined value, A pixel output correction step for correcting the output of the pixel that is equal to or greater than the predetermined specified value based on an output from a peripheral pixel of the pixel, and a correction coefficient setting step for setting a correction coefficient for performing the correction by the pixel output correction step And a shooting scene determination step for determining a shooting scene for the subject, and an output from the pixel output correction step when the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value. A bit compression conversion step of compressing the pixel output data once expanded from the first number of bits below the predetermined prescribed value to the second number of bits to the first number of bits again, In the compression conversion step, the data corresponding to the pixel output at the predetermined specified value or less than the compression rate for the data corresponding to the pixel output at the region above the predetermined specified value is used. By reducing the compression ratio and compression of, the correction coefficient setting step, enter the shooting scene information determined by the photographic scene determination step, based on the input photographic scene information, the correction by the pixel output correction step It is characterized by setting a correction coefficient when performing.

According to a seventh aspect of the present invention, a subject image incident through an optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a three primary color filter or a complementary color system is provided on the front side of each pixel. An image pickup device having a color separation filter is disposed, and a specific color filter of the color separation filter has higher luminance sensitivity than light of other colors with respect to light having a wide wavelength band. In the imaging method of the imaging apparatus, an output from each pixel in which the color separation filter is arranged is detected based on an electrical signal output from the imaging element, and an output from any one of the pixels A pixel output detection step for determining whether or not the pixel value has reached a predetermined specified value or more, and when it is determined by the pixel output detection step that there is a pixel having an output equal to or higher than the predetermined value, A pixel output correction step for correcting the output of the pixel that is equal to or greater than the predetermined specified value based on an output from a peripheral pixel of the pixel, and a correction coefficient setting step for setting a correction coefficient for performing the correction by the pixel output correction step And a face detection step of detecting a human face from an electrical signal output from the image sensor, and an output of a pixel in which the filter of the specific color is disposed reaches the predetermined specified value or more. Bits that are output from the pixel output correction step and that are once expanded from the first number of bits below the predetermined specified value to the second number of bits, are compressed again to the first number of bits. A compression conversion process,
In the bit compression conversion step, compression is performed by reducing the compression rate for data corresponding to the pixel output below the predetermined specified value, rather than the compression rate for data corresponding to the pixel output in the region above the predetermined specified value, The correction coefficient setting step inputs face detection information from the face detection step, and sets a correction coefficient for performing the correction by the pixel output correction step based on the input face detection information. It is said.

  The invention according to claim 8 is a distance detection step of detecting a distance from an electrical signal output from the image sensor to a subject, and a color distribution of detecting a color distribution of the subject from the electrical signal output from the image sensor. A luminance distribution detecting step of detecting a luminance distribution of a subject from an electric signal output from the image sensor, wherein the photographing scene determining step includes the distance information detected in the distance detecting step, the color The photographic scene is determined based on one or more of the color distribution information detected in the distribution detection step and the luminance distribution information detected in the luminance distribution detection step.

  According to the present invention, when it is determined that the output from any one of the pixels has reached a predetermined specified value or more, the output of the pixel having a predetermined specified value or more is determined based on the output from the surrounding pixels. By correcting the output and expanding the dynamic range, it is possible to expand the dynamic range by one shooting without changing the exposure amount and performing multiple shootings to synthesize images.

  Further, according to the present invention, the input is performed based on the face detection information by the face detection means (face detection step) and / or the shooting scene determination information by the shooting scene determination means (shooting scene determination step). Based on these information, by setting a correction coefficient when performing correction by the pixel output correction means (pixel output correction step), depending on the situation of the shooting scene and the presence or absence of a human face in the shooting scene, It is possible to set a dynamic range with an appropriate expansion width.

Hereinafter, the present invention will be described based on the illustrated embodiments.
<Embodiment 1>

  FIG. 1A is a front view showing a digital still camera (hereinafter referred to as “digital camera”) as an example of an imaging apparatus according to Embodiment 1 of the present invention, and FIG. 1 (c) is a rear view thereof, and FIG. 2 is a block diagram showing an outline of a system configuration in the digital camera shown in FIGS. 1 (a), (b), and (c).

(Appearance structure of digital camera)
As shown in FIGS. 1A, 1B, and 1C, a release button (shutter button) 2, a power button 3, a shooting / playback switching dial 4 are provided on the upper surface side of the digital camera 1 according to the present embodiment. In the front (front) side of the digital camera 1, a lens barrel unit 6 having a photographing lens system 5, a strobe light emitting unit (flash) 7, and an optical viewfinder 8 are provided.

  On the back side of the digital camera 1, there are a liquid crystal monitor (LCD) 9, an eyepiece 8 a of the optical viewfinder 8, a wide-angle zoom (W) switch 10, a telephoto zoom (T) switch 11, and a menu (MENU) button 12. , A confirmation button (OK button) 13 and the like are provided. Further, a memory card storage unit 15 for storing a memory card 14 (see FIG. 2) for storing captured image data is provided inside the side surface of the digital camera 1.

(Digital camera system configuration)
As shown in FIG. 2, the digital camera 1 includes a CCD 20 serving as a solid-state imaging device on which a subject image incident through a photographing lens system 5 of a lens barrel unit 6 forms an image on a light receiving surface, and electrical signals output from the CCD 20. An analog front-end unit (hereinafter referred to as “AFE unit”) 21 that processes (analog RGB image signal) into a digital signal, a signal processing unit 22 that processes a digital signal output from the AFE unit 21, and temporarily stores data SDRAM 23 for controlling, ROM 24 for storing a control program, a motor driver 25 for driving the lens barrel unit 6 and the like.

  The lens barrel unit 6 includes a photographic lens system 5 having a zoom lens, a focus lens, and the like, an aperture unit 26, and a mechanical shutter unit 27. The drive units of the photographic lens system 5, the aperture unit 26, and the mechanical shutter unit 27 are as follows. It is driven by the motor driver 25. The motor driver 25 is driven and controlled by a drive signal from a control unit (CPU) 28 of the signal processing unit 22.

  As shown in FIG. 3, the CCD 20 has a Bayer array RGB primary color filter (hereinafter referred to as “RGB filter”) arranged on a plurality of pixels 20 a constituting the CCD 20. A signal (analog RGB image signal) is output.

  The AFE unit 21 is sampled by a TG (timing signal generating unit) 30 that drives the CCD 20, a CDS (correlated double sampling unit) 31 that samples an electrical signal (analog RGB image signal) output from the CCD 20, and the CDS 31. An AGC (analog gain control unit) 32 that adjusts the gain of the signal, and an A / D conversion unit 33 that converts the signal gain-adjusted by the AGC 32 into a digital signal (hereinafter referred to as “RAW-RGB data”).

  The signal processing unit 22 outputs a screen horizontal synchronization signal (HD) and a screen vertical synchronization signal (VD) to the TG 30 of the AFE unit 21, and an A / D conversion unit 33 of the AFE unit 21 in accordance with these synchronization signals. CCD interface (hereinafter referred to as “CCD I / F”) 34 that captures RAW-RGB data output from the CPU, a memory controller 35 that controls the SDRAM 23, and a YUV format that can display and record the captured RAW-RGB data. A YUV conversion unit 36 for converting to image data, a resize processing unit 37 for changing the image size in accordance with the size of image data to be displayed or recorded, a display output control unit 38 for performing processing necessary for image display, and an image A data compression unit 39 for recording data by JPEG formation and the like, and writing image data to the memory card 14 or The digital camera 1 is based on a control program stored in the ROM 24 on the basis of a media interface (hereinafter referred to as “media I / F”) 40 that reads image data written on the recard 14 and operation input information from the operation unit 41. A control unit (CPU) 28 that performs overall system control and the like is provided.

  The operation unit 41 includes a release button 2, a power button 3, a shooting / playback switching dial 4, and a wide-angle zoom provided on the external surface of the digital camera 1 (see FIGS. 1A, 1B, and 1C). The switch 10, the telephoto zoom switch 11, the menu button 12, the confirm button 13, and the like, and a predetermined operation instruction signal is input to the control unit 28 by the photographer's operation. The SDRAM 23 stores the RAW-RGB data captured by the CCD I / F 34, and stores the YUV data (YUV format image data) converted by the YUV conversion unit 36. Further, the SDRAM 23 stores the RAW-RGB data. The compressed image data such as JPEG formation is stored. Note that YUV of the YUV data is luminance data (Y), color difference (difference (U) between luminance data and blue (B) component data, and difference (V) between luminance data and red (R) component data). It is a format that expresses color with information.

(Digital camera monitoring and still image shooting)
Next, the monitoring operation and still image shooting operation of the digital camera 1 will be described. In the still image shooting mode, the digital camera 1 performs a still image shooting operation while executing a monitoring operation as described below.

  First, when the photographer turns on the power button 3 and sets the photographing / playback switching dial 4 to the photographing mode, the digital camera 1 is activated in the recording mode. When the control unit 28 detects that the power button 3 is turned on and the shooting / playback switching dial 4 is set to the shooting mode, the control unit 28 outputs a control signal to the motor driver 25 to switch the lens barrel unit 6. The camera 20 is moved to a photographing position and the CCD 20, the AFE unit 21, the signal processing unit 22, the SDRAM 23, the ROM 24, the liquid crystal monitor 9 and the like are activated.

  Then, by directing the photographic lens system 5 of the lens barrel unit 6 toward the subject, a subject image incident through the photographic lens system 5 is formed on the light receiving surface of each pixel of the CCD 20. Then, an electrical signal (analog RGB image signal) corresponding to the subject image output from the CCD 20 is input to the A / D conversion unit 33 via the CDS 31 and the AGC 32, and 12 bits (bit) by the A / D conversion unit 33. To RAW-RGB data.

  This RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22 and stored in the SDRAM 23 via the memory controller 35. The RAW-RGB data read from the SDRAM 23 is converted into YUV data (YUV signal) in a format that can be displayed by the YUV converter 36, and then the YUV data is stored in the SDRAM 23 via the memory controller 35. The

  The YUV data read from the SDRAM 23 via the memory controller 35 is sent to the liquid crystal monitor (LCD) 9 via the display output control unit 38, and a captured image is displayed. At the time of monitoring in which a photographed image is displayed on the liquid crystal monitor (LCD) 9 described above, one frame is read out in a time of 1/30 second by thinning out the number of pixels by the CCD I / F 34.

  In this monitoring operation, the photographed image is only displayed on the liquid crystal monitor (LCD) 9 functioning as an electronic viewfinder, and the release button 2 is not yet pressed.

  By displaying the photographed image on the liquid crystal monitor (LCD) 9, the photographer can confirm the composition for photographing the still image. In addition, it can output as a TV video signal from the display output control part 38, and can display a picked-up image on external TV (television) via a video cable.

  The CCD I / F 34 of the signal processing unit 22 calculates an AF (automatic focus) evaluation value, an AE (automatic exposure) evaluation value, and an AWB (auto white balance) evaluation value from the captured RAW-RGB data.

  The AF evaluation value is calculated by, for example, the output integrated value of the high frequency component extraction filter or the integrated value of the luminance difference between adjacent pixels. When in the in-focus state, the edge portion of the subject is clear, so the high frequency component is the highest. By utilizing this, at the time of AF operation (at the time of focus detection operation), an AF evaluation value at each focus lens position in the photographing lens system 5 is acquired, and the point where the maximum is obtained is used as the focus detection position for AF. The action is executed.

  The AE evaluation value and the AWB evaluation value are calculated from the integrated values of the RGB values in the RAW-RGB data. For example, the screen corresponding to the light receiving surfaces of all the pixels of the CCD 20 is equally divided into 1024 areas (horizontal 32 divisions and vertical 32 divisions), and the RGB integration of each area is calculated.

  Then, the control unit 28 reads the calculated RGB integrated value, and in the AE process, calculates the luminance of each area of the screen and determines an appropriate exposure amount from the luminance distribution. Based on the determined exposure amount, exposure conditions (the number of electronic shutters of the CCD 20, the aperture value of the aperture unit 26, etc.) are set. In the AWB process, the subject color and the light source color are determined from the RGB distribution, and the AWB control value according to the color of the light source is determined. By this AWB process, white balance is adjusted when the YUV conversion unit 36 performs conversion to YUV data. The AE process and AWB process described above are continuously performed during the monitoring.

  When the release button 2 is pressed (half-pressed to full-press) and the still image shooting operation is started during the monitoring operation described above, the AF operation and the still image recording process that are the focus position detection operations are performed. .

  That is, when the release button 2 is pressed (half-pressed to full-pressed), the focus lens of the photographing lens system 5 is moved by a drive command from the control unit 28 to the motor driver 25, and is referred to as so-called hill-climbing AF, for example. The contrast evaluation AF operation is executed.

  When the AF (focusing) target range is the entire region from infinity to close, the focus lens of the photographing lens system 5 moves to each focus position from close to infinity, or from infinity to close, and CCDI / The control unit 28 reads out the AF evaluation value at each focus position calculated in F34. Then, the focus lens is moved to the in-focus position with the point where the AF evaluation value at each focus position is maximized as the in-focus position, and in-focus.

  Then, the AE process described above is performed, and when the exposure is completed, the mechanical shutter unit 27 is closed by a drive command from the control unit 28 to the motor driver 25, and an analog RGB image signal for a still image is output from the CCD 20. As in the monitoring, the A / D conversion unit 33 of the AFE unit 21 converts the data into RAW-RGB data.

  The RAW-RGB data is taken into the CCD I / F 34 of the signal processing unit 22, converted into YUV data by a YUV conversion unit 36 described later, and stored in the SDRAM 23 via the memory controller 35. The YUV data is read from the SDRAM 23, converted into a size corresponding to the number of recorded pixels by the resizing processing unit 37, and compressed into image data such as JPEG format by the data compression unit 39.

  The compressed image data such as JPEG format is written back to the SDRAM 23, read out from the SDRAM 23 through the memory controller 35, and stored in the memory card 14 through the media I / F 40.

  In the present embodiment, as shown in FIG. 4, the control unit 28 includes a face detection unit 42 and a shooting scene determination unit 43.

  For example, the face detection unit 42 reads the luminance data (Y) of the YUV data written in the SDRAM 23 and performs matching between the human face eyes and the nose luminance pattern template stored in advance. Detect faces. Plural types of luminance pattern templates are stored in the ROM 24 in advance, and the face portion of a person is detected by pattern matching at each point in the screen with the luminance pattern templates of the respective sizes. . The face detection operation by the face detection unit 42 is continuously performed during the monitoring operation, and is used as a reference position for determining appropriate luminance in AE (automatic exposure). In AF (automatic focus), it is also used as a peak detection position for determining the focus position.

  The shooting scene determination unit 43 includes distance information of each position in the subject screen calculated from the AF evaluation value acquired during AF, luminance distribution information in the screen based on the AE evaluation value, and in-screen based on the AWB evaluation value. The color distribution information is input, and the shooting scene is determined based on the input information.

  For example, the shooting scene determination unit 43 is based on the input distance information, luminance distribution information, and color distribution information, and the distance to the subject is not close, the luminance of the entire screen is high, and blue is distributed at the top of the screen. If it is determined that the scene is an outdoor daytime landscape scene. Similarly, if the distance to the subject is not close based on the input distance information, luminance distribution information, and color distribution information, the luminance of the entire screen is low, and high luminance is scattered in a part of the screen. If it is determined, it is determined as a night scene.

  Further, as described above, the face detection unit 42 detects the person's face in the screen, and the shooting scene determination unit 43 closes the distance to the person's face, which is the subject, and the other parts are far away. If it is determined that there is a scene, the scene is determined to be a person-captured scene with the landscape as the background. Similarly, the face detection unit 42 detects the face of the person on the screen as described above, and the shooting scene determination unit 43 closes the distance to the face of the person who is the subject. If it is determined that the distance is long, and the shooting scene determination unit 43 determines that the brightness of the entire screen is low and that high brightness is scattered on a part of the screen, the night scene is used as the background. It is determined as a person shooting scene.

(Principle of dynamic range expansion in the present invention)
A Bayer-arranged RGB filter (see FIG. 3) is arranged on each pixel constituting the CCD 20 of the digital camera 1, but a normal RGB filter for light having a wide wavelength band such as sunlight. Have different luminance sensitivity for each color.

  For example, as shown in FIG. 5, an RGB filter (a, b, and c in FIG. 5) has a sensitivity of the G (green) filter that is about twice that of an R (red) filter and a B (blue) filter. In the case of the CCD 20 having the same, when the same amount of light having a wide wavelength band such as sunlight is incident on the RGB filter, the G filter (shaded portion in FIG. 5c) is applied to each pixel output of the R and B filters. The pixel output first reaches the saturation level A. In FIG. 5, f is the pixel sensitivity characteristic of the G filter, g is the pixel sensitivity characteristic of the R and B filters, and the pixel sensitivity characteristic of the G filter is about twice the pixel sensitivity characteristics of the R and B filters. It has the sensitivity of.

  By the way, in a conventional digital camera having a solid-state imaging device such as a CCD in which an RGB filter is arranged, a saturation level corresponding to the pixel output of a G filter with high sensitivity, such as the RGB filters of a, b, and c in FIG. The dynamic range is set according to A. For this reason, even when the pixel output of the G filter reaches the saturation level A, the pixel output of the R and B filters is about ½ of the saturation level A.

  On the other hand, in the present invention, even if the pixel output of the G filter exceeds the saturation level A as in the RGB filters of d and e in FIG. From within the pixel output level of the R and B filters, the pixel sensitivity characteristics of the R and B filters (g in FIG. 5) and the pixel sensitivity characteristics of the G filter (f in FIG. 5) Based on this, the pixel output level of the G filter is subjected to predictive interpolation correction (one-dot chain line portion), and the dynamic range is expanded by this predicted interpolation.

  As described above, in the present embodiment, the pixel sensitivity characteristic of the G filter has a sensitivity about twice that of each pixel sensitivity characteristic of the R and B filters with respect to light having a wide wavelength band such as sunlight. ing. Therefore, the maximum value of the expansion range of the dynamic range in the present embodiment is about twice as large as that in normal shooting without performing the dynamic range expansion processing operation.

  In this embodiment, the pixel sensitivity characteristic of the G filter has a sensitivity that is about twice that of the pixel sensitivity characteristics of the R and B filters, and based on this, the maximum value of the dynamic range expansion width is doubled. However, by changing the pixel sensitivity characteristics of the R, G, and B filters, the maximum value of the expansion range of the dynamic range can be set to a predetermined value that is two times or more, or a predetermined value that is two times or less.

(Dynamic range expansion processing by the YUV converter 36)
The YUV conversion unit 36 of the digital camera 1 according to the present embodiment has a dynamic range expansion processing function for expanding the dynamic range described above.

  As shown in FIG. 6, the YUV conversion unit 36 includes a dynamic range expansion prediction interpolation unit (hereinafter referred to as “D range expansion prediction interpolation unit”) 50, a bit compression conversion unit 51, a white balance control unit 52, and synchronization described later. Unit 53, tone curve conversion unit 54, RGB-YUV conversion unit 55, image size converter unit 56, luminance histogram generation unit 57, and edge enhancement unit 58.

  As shown in FIG. 7, the D range expansion prediction interpolation unit 50 includes a luminance level determination unit 60, a saturated pixel correction processing unit 61, a non-saturated pixel correction processing unit 62, a correction coefficient setting unit 63, and a bit extension processing unit 64. I have.

  The luminance level determination unit 60 detects the pixel output of each pixel provided with the RGB filter from the input RAW-RGB data, and also outputs the pixel output of the pixel provided with the highest sensitivity G filter (hereinafter referred to as “G filter”). It is determined whether or not the “pixel output” is equal to or higher than the saturation level.

  When the luminance level determination unit 60 determines that the pixel output of the G filter has reached the saturation level or higher, the saturation pixel correction processing unit 61 outputs the pixel output of the pixels provided with the surrounding R and B filters (hereinafter, “ Correction of the pixel output of the G filter that has reached the saturation level (predictive interpolation processing) is performed. The saturated pixel correction processing unit 61 performs correction processing based on the correction coefficient input from the correction coefficient setting unit 63 when correcting the pixel output of the G filter that has reached the saturation level (details will be described later). To do). The non-saturated pixel correction processing unit 62 corrects the pixel output of the R and B filters based on the pixel output value of the G filter corrected by the saturated pixel correction processing unit 61.

  The correction coefficient setting unit 63 inputs the shooting scene information determined by the shooting scene determination unit 43 and the face detection information by the face detection unit 42, and based on the input information, the saturated pixel correction processing unit 61 performs G A correction coefficient for correcting the pixel output of the filter is set (details will be described later).

  When the luminance level determination unit 60 determines that the pixel output of the G filter has not reached the saturation level, the bit extension processing unit 64 performs the pixel output of the G filter and the pixel output of the R and B filters. Only bit expansion is performed from 12 bits to 14 bits without correcting the output level.

  The dynamic range expansion processing operation in this embodiment will be described below.

  For example, if there is a part of the background of the subject you want to shoot that is extremely brighter than its surroundings, or the person's face may be overexposed when shooting a person with a flash against a night view In some cases, it is desired to obtain an image having a good gradation with respect to a high luminance part by expanding the dynamic range. Thus, when it is desired to shoot with a larger dynamic range than during normal shooting, the photographer presses the menu button 12 (see FIG. 1C), for example, as shown in FIG. A setting screen is displayed on a liquid crystal monitor (LCD) 9.

  Then, as shown in FIG. 8, with respect to the item “dynamic range doubling” selected by pressing the menu button 12, the confirmation button (see FIG. 1C) 13 is pressed to “double dynamic range”. decide. As a result, the dynamic range expansion processing operation is turned on, and a signal for doubling the dynamic range expansion width is output from the control unit 28 to the luminance level determination unit 60.

  When the signal that doubles the dynamic range expansion width is output from the control unit 28 to the luminance level determination unit 60 and the dynamic range expansion processing operation is executed, the luminance level determination of the D range expansion prediction interpolation unit 50 is performed. The unit 60 determines whether or not the pixel output of the G filter has reached a saturation level or higher based on the input RAW-RGB data.

  In the case where this determination process is performed, in the present embodiment, as shown in FIG. 3, 2 × 2 pixels in the thick frame A (2 pixels for the G filter) for each pixel of the CCD 20 in which the RGB filter is installed. , R and B filters are each one pixel) as a processing unit (minimum unit).

  When at least one pixel output of two G filter pixels in the processing unit (thick frame A) has reached the saturation level or higher, the sensitivity of the G filter is the R and B filters as described above. Therefore, the pixel output value (G) of the G filter is calculated from the following formula (1).

  G = {(R + B) / 2} × 2 Formula (1)

  The saturated pixel correction processing unit 61 calculates the average value of the pixel outputs of the R and B filters and doubles the average value of the pixel outputs of the R and B filters as shown in the equation (1) to calculate the pixel output value of the G filter. The calculated pixel output values of the G filter are replaced with the pixel output values of the two G filters in the processing unit (2 × 2 pixels). Since the pixel output value of the G filter exceeds 12 bits, it is replaced here with 14-bit data. Therefore, since the maximum value of the pixel output of the R and B filters is 4095 (12 bits), the maximum value of the pixel output of the G filter is 8190 (13 bits), so that it can be handled as 14-bit data. .

  In the equation (1), the calculation of the pixel output value of the G filter is performed with 2 × 2 pixels. However, the present invention is not limited to this, and the R and B filters around the pixel of the correction target G filter are not limited thereto. It is also possible to use n (two or more) pixels, calculate the average value in the same manner as in equation (1), and double the average value.

  At this time, the correction coefficient setting unit 63 inputs the shooting scene information determined by the shooting scene determination unit 43 and the face detection information by the face detection unit 42, and based on the input information, the saturated pixel correction processing unit 61. To set a correction coefficient for correcting the pixel output of the G filter. The saturated pixel correction processing unit 61 corrects the pixel output of the G filter that has reached the saturation level or higher based on the correction coefficient set by the correction coefficient setting unit 63.

  Specifically, for example, when the shooting scene determination unit 43 and the face detection unit 42 determine that the scene is a person shooting scene with flash emission against a night scene as described above, the person's face is slightly overexposed due to flash emission. It is easy to become. In normal flash emission, during pre-emission, pre-emission, which is weak emission, is performed, and a necessary emission amount is calculated from the reflected light, and flash emission is performed according to the calculated exposure amount. At this time, when the background is a distant view, the amount of light emission is calculated based on the reflected light from the person, but the reflectance of the torso clothes and hair of a person with a relatively large area affects the calculation of the amount of light emission. When wearing clothes with a lower reflectance than the face part, the face part is often overexposed. In such a case, the face portion is overexposed.

  Therefore, the correction coefficient setting unit 63 has the largest dynamic range expansion width based on the information input from the shooting scene determination unit 43 and the face detection unit 42 (the person shooting scene by flash emission with the night scene as a background). A correction coefficient (in this case, the correction coefficient in this case is 1.0) is set so as to be (in this embodiment, approximately twice that of normal time), and this correction coefficient is output to the saturated pixel correction processing unit 61. The saturated pixel correction processing unit 61 corrects the pixel output of the G filter that has reached the saturation level or higher based on this correction coefficient. In this case, the correction process is such that the expansion range of the dynamic range is maximized.

  In addition, when the shooting scene determination unit 43 and the face detection unit 42 determine, for example, a night scene without a person as described above, in the night scene, the high luminance part in the screen may be a light source such as an electric lamp. Is expensive. Therefore, in the case of a night scene, an image in which this light source (high luminance part) is white and shining is preferred, so that this light source (high luminance part) is preferred to remain above the saturation level. It becomes an image.

  Therefore, the correction coefficient setting unit 63 maximizes the expansion range of the dynamic range based on information (the above-described night view scene) input from the shooting scene determination unit 43 and the face detection unit 42 (in this embodiment, the normal time). A correction coefficient that is more limited than about twice (a correction coefficient in this case is 0.7, for example) is set, and this correction coefficient is output to the saturated pixel correction processing unit 61. The saturated pixel correction processing unit 61 corrects the pixel output of the G filter that has reached the saturation level or higher based on this correction coefficient. In this case, the correction processing is such that the expansion range of the dynamic range is reduced according to a correction coefficient (in this case, the correction coefficient is, for example, 0.7).

  As described above, the correction coefficient setting unit 63 stores a plurality of correction coefficients corresponding to information respectively input from the shooting scene determination unit 43 and the face detection unit 42 in advance, and the shooting scene determination unit 43 and the face detection unit. An appropriate correction coefficient is output to the saturated pixel correction processing unit 61 on the basis of information input from each of 42.

  By the way, before the luminance level determination unit 60 determines whether or not the pixel output of the G filter has reached the saturation level or higher, the correction of the defective pixel needs to be completed. That is, if there is a defective pixel in the pixel provided with the G filter and there is a pixel that always outputs a saturated value, the pixel provided with the G filter in the same processing unit is replaced with a large value. A defective pixel will be generated.

  Further, when a pixel in which the R and B filters are arranged has a defective pixel, the conversion of the pixel provided with the G filter according to the equation (1) becomes an incorrect value. Therefore, in the present embodiment, the CCD I / F 34 includes a defective pixel removal processing unit (not shown) that removes defective pixels.

  By the way, when the pixel output of the G filter that has reached the saturation level or higher is corrected by the saturated pixel correction processing unit 61 based on the correction coefficient input from the correction coefficient setting unit 63 as described above, the surrounding saturation is obtained. There is a case where the balance with the pixel output of the R and B filters that are not performed is lost and the hue (hue) is shifted. Therefore, the non-saturated pixel correction processing unit 62 corrects the R and B filter pixel outputs as follows according to the G filter pixel output corrected by the saturated pixel correction processing unit 61.

  FIG. 9 is a characteristic diagram showing an example of a ratio of each pixel output of the RGB filter with respect to a difference in color temperature of the light source (2000K (Kelvin) or less to 9000K (Kelvin) or more). As is apparent from FIG. 9, the pixel output of the G filter hardly changes even when the color temperature changes, but the pixel output of the R filter increases as the color temperature decreases (reddish light source), and the pixel of the B filter The output increases as the color temperature increases (blue light source).

  The ratio relationship between the color temperature and each pixel output value of the RGB filter as shown in FIG. 9 depends on the CCD in which the RGB filter is arranged. Therefore, when incorporating the CCD 20 with the RGB filter in the digital camera 1, the ratio relationship of the pixel output values of the RGB filter to different color temperatures as shown in FIG. 9 corresponding to the CCD 20 with the RGB filter is shown. The indicated table (hereinafter referred to as “RGB output ratio table”) is stored in the ROM 24.

  Then, the control unit 28 determines the color temperature of the light source of the subject from the RGB distribution that is the AWB evaluation value acquired from the CCD I / F 34, and then reads the RGB output ratio table from the ROM 24, and the unsaturated pixel correction processing unit 62. Output to. The non-saturated pixel correction processing unit 62 corrects the pixel output value of the G filter corrected by the saturated pixel correction processing unit 61 and the pixel output values of the R and B filters around the pixel of the G filter with respect to the color temperature of the light source at the time of shooting. The pixel output values of the R and B filters are corrected so that the ratio is the same as the pixel output ratio in the RGB output ratio table. The corrected R and B filter pixel outputs are expanded to 14 bits and output to the bit compression conversion unit 51.

  Then, the bit compression conversion unit 51 uses, for example, 14 bits by conversion characteristics as shown in FIG. 10A (four-segment broken line approximation characteristics in which three nodes are designated and approximated by a straight line). The pixel output of the G filter among the pixel outputs of the R, G, and B filters expanded to 12 is compressed to 12 bits. In FIG. 10A, a is a 12-bit range, and b is a simple linear conversion characteristic (one-dot chain line portion) for shifting data of the maximum value 8190 by 1 bit (1/2 times).

  In the conversion characteristic shown in FIG. 10A, since the maximum value of the pixel output of the G filter is 8190, when the dynamic range is expanded to the maximum, compression is performed so that 8190 becomes 4095. Then, the pixel output values of the R and B filters are also compressed in accordance with the compression magnification of the pixel output of the G filter.

  In addition, as described above, when the dynamic range is not expanded to the maximum (in the present embodiment, about twice as large as normal), but the expansion range of the dynamic range is limited more than the maximum, FIG. As shown in FIG. 5, the compression is performed so that a value smaller than 8190, for example, 5461 becomes 4095. In the case of setting as shown in FIG. 10B, the expansion range of the dynamic range is about 4/3 times that in the normal state.

  As shown in FIG. 10B, as a case where the expansion range of the dynamic range is limited more than the maximum, for example, there is a night scene without the above-described person. In this case, as described above, by setting a correction coefficient that limits the expansion range of the dynamic range from the maximum, the pixel output value of the G filter that has been corrected (predictive interpolation processing) is reduced, The data is output to the bit compression converter 51. Thus, for example, as shown in FIG. 10B, the compression is performed so that a value smaller than 8190, for example, 5461 becomes 4095.

  As described above, in this embodiment, as an example of the case where the pixel output of the G filter whose maximum value is expanded to 8190 is compressed to 4095, three nodes as shown by the solid line in FIG. A conversion characteristic with In the present embodiment, the following two effects that cannot be obtained with a linear conversion characteristic without a simple node (b in FIG. 10A) are obtained.

  As a first effect, a larger number of bits can be assigned to data with high data reliability. That is, when predictive interpolation processing is performed on the pixel output of the G filter that has reached the saturation level or higher, the prediction interpolation is performed for the range that is equal to or greater than the specified value near the saturation level of the pixel output of the G filter. The prediction interpolation is not performed in the range below the specified value. Therefore, the accuracy of the data is different between the range where the predictive interpolation is performed and the range where the predictive interpolation is not performed.

  That is, for example, when predictive interpolation (correction) is performed on the pixel output value of the G filter that is saturated according to the equation (1), the luminance level of the subject is accurate within the range in which predictive interpolation is performed depending on the color of the main subject. It may not be reproduced. On the other hand, the range in which the predictive interpolation is not performed is data obtained by A / D conversion of actual data (analog RGB image signal) output from the CCD 20 in which the RGB filter is arranged. It will be expensive.

  That is, in the conversion characteristic (part indicated by the solid line) in this embodiment shown in FIG. 10A, for example, when the input 14-bit data is 1024, the output 12-bit data is 1024, and the original data is It is used as it is. On the other hand, for example, when the input 14-bit data is 3072, the output 12-bit data is 2560. In this range, the bit allocation is smaller than the bit allocation before the prediction interpolation, so that a bit error occurs. .

  In this way, by adopting a conversion characteristic (part indicated by a solid line) having three nodes as in this embodiment, instead of a characteristic for performing linear conversion without a simple node (part indicated by a one-dot chain line). Since the bit allocation can be gradually reduced, a larger number of bits can be allocated to data with high data reliability.

  As a second effect, gradations at low and medium luminance can be accurately stored. That is, when bit compression is performed with a simple linear conversion characteristic (b in FIG. 10A), the bit allocation becomes ¼ in a range where predictive interpolation on the low luminance side is not performed. For this reason, the image has no tone. On the other hand, when bit compression is performed with the conversion characteristics (solid line portion) in the present embodiment as shown in FIG. 10A, the data corresponding to the pixel output at the low luminance level below the saturation level is applied. Therefore, by using a compression rate that is substantially the same value before and after bit compression by the bit compression conversion unit 51, it is possible to satisfactorily maintain gradation at low and medium luminance levels. .

  In this embodiment, when the 14-bit data of the pixel output of the expanded G filter is reduced to 12 bits, three nodes are designated as shown in FIG. 10A and approximated by a straight line between them. Although the bit compression is performed with the broken line approximation characteristic (conversion characteristic) of four sections, the number of sections is not particularly limited. For example, it may be a polygonal line approximate characteristic of two sections designating one node, but the above-mentioned two effects are reduced by changing the bit allocation in the vicinity of the node. Therefore, a broken line approximation characteristic (conversion characteristic) having three or more sections is preferable.

  Further, the conversion characteristic for reducing the 14-bit data of the pixel output of the expanded G filter to 12 bits may be a conversion characteristic by a curve not having a plurality of nodes. In other words, if the number of sections is 8192, the resolution of the input 14-bit data is 8192, so that it can be considered equivalent to conversion by a smooth curve. Therefore, if the bit compression conversion unit 51 can have the lookup table of the input 8192, it is possible to perform conversion using a curve, and it is possible to prevent the problem that the bit allocation amount changes near the node.

  The pixel output data of the R, G, and B filters that have been compression-converted from 14 bits to 12 bits by the bit compression conversion unit 51 are input to the white balance control unit 52.

  The white balance control unit 52 multiplies the input pixel output value data of the R, G, and B filters by a correction coefficient for adjusting white balance. This correction coefficient is calculated by the control unit 28 based on the AWB evaluation value generated by the CCD I / F 34. The correction coefficient is a coefficient that makes the light source color white.

  Specifically, based on the ratio of the RGB filter pixel output values to different color temperatures as shown in FIG. 9, the R filter pixel output values include G / R (G filter pixel output as a correction coefficient). Value / R filter pixel output value), and the B filter pixel output value is multiplied by G / B (G filter pixel output value / B filter pixel output value) as a correction coefficient. The levels of the pixel output values of the R, G, and B filters are matched.

  By the way, the ratio of the pixel output values of the R, G, and B filters is determined from the light source color by the correction processing in the saturated pixel correction processing unit 61 and the unsaturated pixel correction processing unit 62 of the D range expansion prediction interpolation unit 50 described above. This is the ratio of the pixel output values of the R, G, and B filters.

  Therefore, the output obtained by multiplying the output from the D range expansion prediction interpolation unit 50 by the correction coefficient in the white balance control unit 52 has substantially the same R, G, and B filter levels, and has a value close to an achromatic color. . Note that an image region in which all the pixel output values of the R, G, and B filters are saturated is overexposed and whitened. In addition, an image region where only the pixel output value of the high-sensitivity G filter is saturated is also likely to have a high luminance and a large amount of incident light, so even if the image region to be corrected for pixel output is an achromatic color There is less discomfort.

  Then, the pixel output value data (12 bits) of the R, G, and B filters with the white balance adjusted from the white balance control unit 52 is input to the synchronization unit 53. The synchronization unit 53 performs an interpolation calculation process on the RAW data having only one color data in one pixel of the Bayer array, and generates all RGB data for one pixel.

  All the RGB data (12 bits) generated by the synchronization unit 53 is input to the tone curve conversion unit 54. For example, the tone curve conversion unit 54 performs γ conversion that converts 12-bit RGB data into 8-bit RGB data using a conversion table as shown in FIG. -Output to the YUV converter 55. The RGB-YUV conversion unit 55 converts input RGB data (8 bits) into YUV data by matrix calculation and outputs the YUV data to the image size converter unit 56. The image size converter unit 56 reduces or enlarges the input YUV data (8 bits) to a desired image size, and outputs it to the luminance histogram generation unit 57 and the edge enhancement unit 58.

  The luminance histogram generation unit 57 generates a luminance histogram based on the input YUV data. The edge enhancement unit 58 performs processing such as edge enhancement according to the image on the input YUV data, and stores the processed data in the SDRAM 23 via the memory controller 35.

  As described above, in the present embodiment, even when the pixel output of the G filter with high sensitivity in the processing unit exceeds the saturation level, the pixel output of the R and B filters that have not reached the surrounding saturation level. As shown in FIG. 5, the pixel output of the G filter (d and e in FIG. 5) is subjected to prediction interpolation expansion by correcting the pixel output of the saturated G filter based on Based on the region (the dot-and-dash line portion of the pixel output of the G filter of d and e in FIG. 5), the dynamic range can be doubled by one shooting.

  Therefore, even when there is a high-luminance portion in the background or the like in the captured image, it is possible to prevent the occurrence of overexposure and obtain good gradation.

  Further, in the present embodiment, as described above, the correction coefficient setting unit 63 inputs the shooting scene information determined by the shooting scene determination unit 43 and the face detection information by the face detection unit 42, and based on these input information. Then, a correction coefficient for correcting the pixel output of the G filter by the saturated pixel correction processing unit 61 is set, and the saturated pixel correction processing unit 61 performs saturation based on the correction coefficient set by the correction coefficient setting unit 63. The pixel output of the G filter that has reached the level is corrected. Accordingly, it is possible to set a dynamic range with an appropriate expansion width according to the situation of the shooting scene and the presence or absence of a human face in the shooting scene.

  FIG. 12A shows an example of a histogram generated by the luminance histogram generation unit 57 when the dynamic range in the present embodiment is expanded to the maximum when the pixel output of the G filter exceeds the saturation level. It is. This histogram is an example of a person-photographed scene by flash emission against the background of the night view described above, and is almost free from whiteout in the vicinity of the person's face (area a in FIG. 12A). Reproduced with gradation.

  On the other hand, FIG. 12B is an example of a histogram generated by the luminance histogram generation unit 57 when the dynamic range expansion process in the present embodiment is not performed in the same shooting scene. As is apparent from this histogram, since the dynamic range expansion process has not been performed, the vicinity of the human face (region a in FIG. 12 (b)) is nearly white.

<Embodiment 2>
In the first embodiment, the ratio between the pixel output value of the corrected G filter and the pixel output values of the R and B filters around the pixel of the G filter according to the color temperature of the light source of the subject is the RGB output ratio. The pixel output values of the R and B filters are corrected so as to be the same as the pixel output ratio of the table. However, as a simple configuration, the sensitivity of the pixel outputs of the R, G, and B filters under a specific light source A ratio corresponding to the difference is measured in advance and stored in the ROM 24 (see FIG. 2), and the pixel output values of the R and B filters are corrected to be the same as the ratio measured in advance. Good. Other configurations are the same as those of the digital camera of the first embodiment.

  For example, FIGS. 13A and 13B show examples of pixel outputs of two R, G, and B filters having different sensitivity characteristics under a specific light source.

  In the digital camera (imaging device) having the pixel output characteristics of the R, G, and B filters in FIG. 13A, the ratio of the pixel outputs of the R, G, and B filters is stored in advance. In the digital camera (imaging device) having the pixel output characteristics of the R, G, and B filters shown in FIG. 13B, the ratio of the pixel outputs of the R, G, and B filters is stored in advance.

  For example, in a digital camera (imaging device) in which the pixel output ratios of the R, G, and B filters as shown in FIG. 13A are stored in advance, the ratio is the same as the ratio measured in advance. As described above, the pixel output values of the R and B filters are corrected.

  In this embodiment, since the normal white balance control range is about 2000 to 8000 K (Kelvin), R, G, and B measured under a light source having a color temperature of about 4000 to 5000 K (Kelvin) near the center. It is desirable to store in advance the pixel output ratio of the filter.

  Thus, in this embodiment, the ratio according to the sensitivity difference of the pixel output of the R, G, and B filters under a specific light source is measured and stored in advance, and is the same as the ratio measured in advance. Since the configuration is such that the pixel output values of the R and B filters are corrected, the configuration can be made simpler than in the first embodiment.

<Embodiment 3>
In the first embodiment, the correction coefficient setting unit 63 inputs the shooting scene information determined by the shooting scene determination unit 43 and the face detection information by the face detection unit 42, and based on the input information, the saturated pixel correction is performed. The correction coefficient when correcting the pixel output of the G filter by the processing unit 61 is set. However, either the shooting scene information determined by the shooting scene determination unit 43 or the face detection information by the face detection unit 42 is used. A configuration may be used in which a correction coefficient for correcting the pixel output of the G filter by the saturated pixel correction processing unit 61 is set based on the information on either side.

<Embodiment 4>
In the first and second embodiments, it is configured to determine whether or not the pixel output of the G filter having the highest sensitivity has reached a saturation level or higher. However, the pixel output of the R filter or the pixel output of the B filter is saturated. It is also possible to determine whether or not the level has been reached.

  When the R filter pixel and the B filter pixel have the characteristics shown in FIG. 9, either the R filter pixel or the B filter pixel is, for example, 70% or more of the saturation level. Then, it can be determined that the pixel of the G filter is saturated. That is, the specified value is set to 70% of the saturation level, and the pixel output of the R filter or the pixel output of the B filter can be the detection target.

  Here, when it is determined that either the pixel output of the R filter or the pixel output of the B filter is greater than the specified value, the color temperature of the light source is determined from the ratio of the pixel output of the R filter and the pixel output of the B filter. 9 RGB output ratio tables are selected, and each pixel output value of the RGB filter is determined based on the selected output ratio and the pixel output of the R filter or the pixel output of the B filter.

  Furthermore, high luminance detection may be performed based on the average value of the pixel output of the R filter and the pixel output of the B filter. Since the pixel output of the R filter and the pixel output of the B filter have a relationship as shown in FIG. As for the correction, the light source color temperature in FIG. 9 is determined from the output ratio of the pixel output of the R filter and the pixel output of the B filter as described above, and the output level of at least one of the pixel output of the R filter or the pixel output of the B filter is determined. The overall output level is determined, and each pixel output of the RGB filter is determined.

  In the first embodiment, the screen corresponding to the light receiving surface of all the pixels of the CCD 20 is equally divided into 1024 areas (horizontal 32 divisions and vertical 32 divisions), and the pixels of the G filter that have reached the saturation level or more are 1024. Although it is determined which area of the area (horizontal 32 divisions and vertical 32 divisions) is included, the screen corresponding to the light receiving surface of all the pixels of the CCD 20 is made into a finer area such as one pixel unit, for example. It is better not to divide.

  This is because the pixel output of the R, G, and B filters has a slight sensitivity difference for each pixel, and there are some pixels that output a specific output regardless of incident light, such as defective pixels. . For this reason, for example, if the integration target at the time of generating the AWB evaluation value is 10 pixels or more, a CCD provided with an RGB filter having a variation that can absorb the variation in characteristics of each pixel by averaging is used. It is desirable to make it.

  In each of the embodiments described above, the RGB three primary color filters are arranged as the color separation filters. However, the present invention is similarly applied to a configuration in which the complementary color filters are arranged as the color separation filters. Can do.

  In each of the above-described embodiments, the description has been made on the imaging device such as a digital camera having a photographing lens system and a solid-state imaging device such as a CCD. However, the present invention can be similarly applied to an image processing device. It is. For example, the present invention is similarly applied to an image processing apparatus in which analog data from a single-plate color image sensor is converted into digital signals and stored, and RAW data is input to output RGB data or YCbCr data. It is possible.

(A) is a front view which shows the digital camera as an example of the imaging device which concerns on Embodiment 1 of this invention, (b) is the top view, (c) is the back view. 1 is a block diagram showing an outline of a system configuration in a digital camera as an example of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 3 is a diagram illustrating a pixel arrangement position and a processing unit of a CCD in which an RGB filter according to Embodiment 1 of the present invention is arranged. The figure which shows the control part which has a face detection part and an imaging | photography scene determination part. The figure for demonstrating the principle of the dynamic range expansion in Embodiment 1 of this invention. The block diagram which shows the structure of the YUV conversion part in Embodiment 1 of this invention. The block diagram which shows the structure of the D range expansion prediction interpolation part in Embodiment 1 of this invention. The figure which shows an example of the imaging | photography setting screen displayed on the liquid crystal monitor. The figure which shows the relationship between color temperature and the pixel output of an RGB filter. (A) is a figure which shows the conversion characteristic which compresses the 14-bit data of the pixel output of the extended G filter in Embodiment 1 of this invention to 12 bits, (b) is the case where the expansion width of a dynamic range is restrict | limited The figure which shows the conversion characteristic which compresses 14-bit data of the pixel output of the extended G filter in 12 bits in 12 bits. The figure which shows the conversion table which converts 12-bit RGB data into 8-bit RGB data (gamma conversion). (A) is a figure which shows the histogram at the time of performing the expansion process of the dynamic range in Embodiment 1 of this invention, (b) shows the histogram at the time of not performing the expansion process of the dynamic range in this embodiment. Figure. (A), (b) is a figure which shows an example of the pixel output of the R, G, B filter from which the sensitivity characteristic each differs under the specific light source in Embodiment 2 of this invention.

Explanation of symbols

1 Digital camera (imaging device)
5 Shooting lens system (optical system)
6 Lens unit 9 LCD monitor 12 Menu button 20 CCD (image sensor)
21 Analog Front End 22 Signal Processor 23 SDRAM
28 Control Unit 34 CCD Interface 35 Memory Controller 36 YUV Conversion Unit 42 Face Detection Unit (Face Detection Unit)
43. Shooting scene determination unit (shooting scene determination means)
50 D range expansion prediction interpolation unit 51 bit compression conversion unit 60 luminance level determination unit (pixel output detection means)
61 Saturated pixel correction processing unit (pixel output correction processing means)
62 Unsaturated pixel correction processing unit 63 Correction coefficient setting unit (correction coefficient setting means)
64-bit extension processing section

Claims (8)

A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging apparatus that includes an imaging device, and a specific color filter of the color separation filter has higher luminance sensitivity than other color filters for light having a wide wavelength band,
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection means for determining whether or not it has reached,
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection means that there is a pixel of the output above the predetermined specified value Means,
Correction coefficient setting means for setting a correction coefficient when performing the correction by the pixel output correction means;
Shooting scene determination means for determining a shooting scene for a subject;
Face detection means for detecting a human face from an electrical signal output from the image sensor;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correcting means. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits to the first number of bits again,
The bit compression conversion means compresses by compressing the data corresponding to the pixel output below the predetermined specified value to be smaller than the data corresponding to the pixel output in the area above the predetermined specified value,
The correction coefficient setting means inputs shooting scene information determined by the shooting scene determination means and face detection information by the face detection means, and based on the input information, the pixel output correction means performs the correction. Set the correction factor when performing,
An imaging apparatus characterized by that. A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging apparatus that includes an imaging device, and a specific color filter of the color separation filter has higher luminance sensitivity than other color filters for light having a wide wavelength band,
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection means for determining whether or not it has reached,
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection means that there is a pixel of the output above the predetermined specified value Means,
Correction coefficient setting means for setting a correction coefficient when performing the correction by the pixel output correction means;
Shooting scene determination means for determining a shooting scene for a subject;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correcting means. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits to the first number of bits again,
The bit compression conversion means compresses by compressing the data corresponding to the pixel output below the predetermined specified value to be smaller than the data corresponding to the pixel output in the area above the predetermined specified value,
The correction coefficient setting means inputs the shooting scene information determined by the shooting scene determination means, and sets a correction coefficient for performing the correction by the pixel output correction means based on the input shooting scene information.
An imaging apparatus characterized by that. A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging apparatus that includes an imaging device, and a specific color filter of the color separation filter has higher luminance sensitivity than other color filters for light having a wide wavelength band,
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection means for determining whether or not it has reached,
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection means that there is a pixel of the output above the predetermined specified value Means,
Correction coefficient setting means for setting a correction coefficient when performing the correction by the pixel output correction means;
Face detection means for detecting a human face from an electrical signal output from the image sensor;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correcting means. Bit compression conversion means for compressing the pixel output data once expanded to the number of bits to the first number of bits again,
The bit compression conversion means compresses by compressing the data corresponding to the pixel output below the predetermined specified value to be smaller than the data corresponding to the pixel output in the area above the predetermined specified value,
The correction coefficient setting means inputs face detection information by the face detection means, and sets a correction coefficient for performing the correction by the pixel output correction means based on the input face detection information.
An imaging apparatus characterized by that. Distance detection means for detecting the distance from the electrical signal output from the image sensor to the subject, color distribution detection means for detecting the color distribution of the subject from the electrical signal output from the image sensor, and output from the image sensor A luminance distribution detecting means for detecting the luminance distribution of the subject from the electrical signal to be
The photographing scene determination unit is any one of distance information detected by the distance detection unit, color distribution information detected by the color distribution detection unit, and luminance distribution information detected by the luminance distribution detection unit. Based on the above information, the shooting scene is determined.
The imaging apparatus according to claim 1 or 2, wherein A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging method of an imaging apparatus comprising an imaging device, a filter of a specific color among the color separation filters with respect to light having a wide wavelength band has higher luminance sensitivity than a filter of another color.
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection step for determining whether or not the threshold has been reached;
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection step that there is a pixel having an output of the predetermined specified value or higher. Process,
A correction coefficient setting step for setting a correction coefficient when performing the correction by the pixel output correction step;
A shooting scene determination process for determining a shooting scene for a subject;
A face detection step of detecting a human face from an electrical signal output from the image sensor;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correction step. A bit compression conversion step of compressing the pixel output data once expanded to the first bit number to the first bit number,
In the bit compression conversion step, compression is performed by reducing the compression rate for data corresponding to the pixel output below the predetermined specified value, rather than the compression rate for data corresponding to the pixel output in the region above the predetermined specified value,
The correction coefficient setting step inputs the shooting scene information determined by the shooting scene determination step and the face detection information by the face detection step, and based on the input information, the correction is performed by the pixel output correction step. Set the correction factor when performing,
An imaging method characterized by the above. A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging method of an imaging apparatus comprising an imaging device, a filter of a specific color among the color separation filters with respect to light having a wide wavelength band has higher luminance sensitivity than a filter of another color.
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection step for determining whether or not the threshold has been reached;
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection step that there is a pixel having an output of the predetermined specified value or higher. Process,
A correction coefficient setting step for setting a correction coefficient when performing the correction by the pixel output correction step;
A shooting scene determination process for determining a shooting scene for a subject;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correction step. A bit compression conversion step of compressing the pixel output data once expanded to the first bit number to the first bit number,
In the bit compression conversion step, compression is performed by reducing the compression rate for data corresponding to the pixel output below the predetermined specified value, rather than the compression rate for data corresponding to the pixel output in the region above the predetermined specified value,
The correction coefficient setting step inputs the shooting scene information determined by the shooting scene determination step, and sets a correction coefficient when performing the correction by the pixel output correction step based on the input shooting scene information.
An imaging method characterized by the above. A subject image incident through the optical system is received by a light receiving surface having a plurality of pixels and converted into an electrical signal, and a color separation filter of a three primary color filter or a complementary color filter is disposed on the front side of each pixel. In an imaging method of an imaging apparatus comprising an imaging device, a filter of a specific color among the color separation filters with respect to light having a wide wavelength band has higher luminance sensitivity than a filter of another color.
Based on the electrical signal output from the image sensor, the output from each pixel in which the color separation filter is arranged is detected, and the output from any one of the pixels is equal to or greater than a predetermined specified value. A pixel output detection step for determining whether or not the threshold has been reached;
Pixel output correction that corrects the output of the pixel above the predetermined specified value based on the output from the surrounding pixels when it is determined by the pixel output detection step that there is a pixel having an output of the predetermined specified value or higher. Process,
A correction coefficient setting step for setting a correction coefficient when performing the correction by the pixel output correction step;
A face detection step of detecting a human face from an electrical signal output from the image sensor;
When the output of the pixel in which the filter of the specific color is arranged reaches or exceeds the predetermined specified value, the second output from the first bit number below the predetermined specified value is output from the pixel output correction step. A bit compression conversion step of compressing the pixel output data once expanded to the first bit number to the first bit number,
In the bit compression conversion step, compression is performed by reducing the compression rate for data corresponding to the pixel output below the predetermined specified value, rather than the compression rate for data corresponding to the pixel output in the region above the predetermined specified value,
The correction coefficient setting step inputs face detection information from the face detection step, and sets a correction coefficient for performing the correction by the pixel output correction step based on the input face detection information.
An imaging method characterized by the above. A distance detection step of detecting a distance from the electrical signal output from the image sensor to a subject, a color distribution detection step of detecting a color distribution of the subject from the electrical signal output from the image sensor, and an output from the image sensor A luminance distribution detecting step of detecting a luminance distribution of the subject from the electrical signal to be
The shooting scene determination step is any one of distance information detected in the distance detection step, color distribution information detected in the color distribution detection step, and luminance distribution information detected in the luminance distribution detection step. Based on the above information, the shooting scene is determined.
The imaging method according to claim 5 or 6, characterized in that:

Images