JP2012119756A - Imaging apparatus and white-balance control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide: an imaging device which can suppress occurence of color failure effectively by enhancing the accuracy of automatic white balance; and a white balance control method.SOLUTION: An imaging apparatus comprises: a feature quantity calculation unit 101 which divides an image frame output from a CMOS sensor into a plurality of blocks and calculates the WB evaluation values of the plurality of blocks thus divided; a white-block detection unit 102 which detects a white block based on the feature quantity thus calculated; a subject area detection unit 103 which detects a subject area from an image frame; and a white balance control unit 104 which controls the white balance of an image frame by calculating a WB control value based on the WB evaluation value of a white block detected from other than the subject when a white block is detected in the subject area of a current image frame and when no white block is detected in the subject area of a preceding frame.

Description

  The present invention relates to an imaging apparatus and a white balance control method.

  In an image pickup apparatus such as a digital camera that picks up an image using an image pickup element, a process called auto white balance is performed to reproduce a white portion that appears white to humans on the image. The auto white balance is, for example, a control value (hereinafter referred to as a control value calculated based on color information of a detected area that detects an area estimated to be white (achromatic) in an image frame output from the image sensor. , And WB control value), and can be realized by adjusting the signal of each pixel of the image frame.

  Specifically, for example, an image frame is divided into 256 blocks of vertical and horizontal 16 × 16, and for each divided block, an integrated value of RGB signals of pixels included in each block is calculated, and each block is calculated. A ratio (G / R, G / B) of the integrated value of the R signal and the B signal to the integrated value of the G signal is obtained as a feature amount (hereinafter referred to as a WB evaluation value) of each block. Then, the obtained WB evaluation values (G / R, G / B) of each block are plotted in a color space (hereinafter referred to as WB color space) with G / R, G / B as axes.

  On the WB color space, a black body radiation locus when the color temperature of a light source (daylight or tungsten light) is changed in advance is prepared as a reference gain. In this WB color space, by matching the WB evaluation values (G / R, G / B) of each plotted block with the reference gain, an area estimated to be white in the image frame is detected. can do.

  The white area detected as described above has a different RGB integration ratio depending on the type of light source, but the RGB integration ratio in this white area is R: G: B = 1: 1: 1. Auto white balance can be achieved by multiplying each pixel by a gain (WB control value). This is based on the mechanism of human chromatic adaptation, and is based on the hypothesis that humans work in such a way that white objects appear white even under different light sources. In other words, auto white balance refers to white balance control that equalizes the output levels of the R signal, G signal, and B signal as shooting signals in a state where a reference subject whose entire screen is white is photographed. It can be said that processing can be performed automatically without taking a picture. In order to make the output level of the R signal and B signal in the area displaying white in the screen equal to the G signal, the gain (coefficient) multiplied by each of the input R signal and B signal is WB. Control value.

  By the way, the auto white balance method as described above cannot detect a white region from an image frame when the subject does not have an achromatic part or when the subject is mostly chromatic (object color). Or, since the accuracy of the processing for detecting the white area is lowered, there is a disadvantage that erroneous white balance control is performed as a result. As a specific example, in a scene where the human skin is up under a light source with a high color temperature, the area that reflects the skin color of the human skin is mistakenly designated as the area that reflects the white color (achromatic color) under the low color temperature light source. A problem has been pointed out that white balance control is performed to detect and correct human skin to pale. In addition, even when the ratio of green is large as in a forest, an object color region may be erroneously detected as a white region illuminated by a fluorescent lamp, and erroneous white balance control may be performed. Such erroneous white balance control is called color failure.

  As a method for improving the color failure as described above, for example, techniques described in Patent Document 1 and Patent Document 2 are known.

  Patent Document 1 discloses a control method using subject tracking information as a method for reducing color failure in moving image shooting. Specifically, an area showing a representative color of a main subject (for example, a person's skin color) is set as a reference area, and the reference means is made to follow the movement of the subject on the screen by the tracking means. The color information included in the reference area is acquired, and exposure and white balance can be appropriately corrected based on image data representing the same portion of the subject (for example, the cheek portion of a person) in each frame. . That is, in the technique described in Patent Document 1, when a reference area set by a user is searched for frames sequentially acquired from the color information, for example, a subject suddenly approaches the imaging apparatus. However, it prevents color failure due to skin color pull-in.

  Further, in Patent Document 2, in a system in which a moving image is tracked by a plurality of imaging systems, when switching from a camera that has captured a subject to a camera with the movement of the subject and taking over imaging of the subject, In addition, it is described that an unnatural image change at the time of camera switching is suppressed by inheriting an imaging condition for capturing an object as an imaging condition for a camera that has succeeded in imaging.

  However, in the technique described in Patent Document 1, it is determined whether the reference area indicating the representative color of the subject set by the user is the light source color (achromatic) area or the object color (chromatic color) area. It is unclear how the color information of the reference area is used for white balance control. Further, in the technique described in Patent Document 1, when the subject moves under a different light source due to movement, the color information obtained from the reference region changes greatly, and it is erroneous that the original reference region must be determined as the original. There is a problem that a color failure occurs.

  Also, in the technique described in Patent Document 2, when the subject moves under a different light source, the imaging condition such as white balance is inherited when the camera is switched, so that a white balance suitable for the shooting environment after the movement can be set. However, there is a problem that color failure occurs.

  The present invention has been made in view of the above, and provides an imaging apparatus and a white balance control method capable of improving the accuracy of auto white balance and effectively suppressing the occurrence of color failure. It is aimed.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention divides an image sensor and an image frame output from the image sensor into a plurality of blocks, and each of the divided blocks. A feature amount calculating means for calculating a feature amount that represents a color feature of each block; and a white block that detects a white block that is estimated to be a white block from the plurality of blocks based on the feature amount. Block detecting means; subject area detecting means for detecting a subject area showing a subject to be tracked in the image frame; the white block is detected from the subject area in the image frame; and the image frame The white block from within the subject area in a preceding frame that is an image frame output from the image sensor before If not, the white balance of the image frame is controlled using a control value calculated based on the feature amount of the white block other than the white block detected in the subject area in the image frame. White balance control means.

  In the white balance control method according to the present invention, the image frame output from the image sensor is divided into a plurality of blocks, and for each of the divided plurality of blocks, a feature amount representing a color feature of each block is calculated. And detecting a white block estimated as a white block from the plurality of blocks based on the feature amount; and a subject area showing a subject to be tracked in the image frame. Detecting the white block from within the subject area in the image frame, and detecting from the subject area in a preceding frame, which is an image frame output from the image sensor before the image frame. When no white block is detected, the image area is within the subject area. Using the control value calculated based on the feature amount of the white blocks other than al detected white blocks, characterized in that it comprises the steps of: controlling the white balance of the image frame.

  According to the present invention, it is possible to improve the accuracy of auto white balance and effectively suppress the occurrence of color failure.

FIG. 1-1 is a top view of the digital camera. FIG. 1-2 is a front view of the digital camera. FIG. 1-3 is a back view of the digital camera. FIG. 2 is a block diagram showing the configuration of the control system of the digital camera. FIG. 3 is a functional block diagram illustrating a functional configuration of a CPU for performing white balance control in the digital camera according to the first embodiment. FIG. 4 is a diagram illustrating a state in which the WB evaluation values of the respective blocks are plotted on the WB color space. FIG. 5 is a diagram for explaining the movement of the subject to be tracked and the positions of the subject areas in the image frame A and the image frame B input in time series. FIG. 6 is a diagram illustrating a state in which the WB evaluation values calculated from the image frame A and the image frame B in FIG. 5 are plotted on the WB color space. FIG. 7 is a diagram for explaining the movement of the subject to be tracked and the position of the subject area in the image frame C, the image frame D, and the image frame E input in time series. FIG. 8 is a diagram illustrating a state in which the WB evaluation values calculated from the image frame C, the image frame D, and the image frame E in FIG. 7 are plotted on the WB color space. FIG. 9 is a flowchart showing a series of processes related to white balance control executed by the CPU when the subject tracking function is on in the digital camera according to the first embodiment. FIG. 10 is a functional block diagram showing a functional configuration of a CPU for performing white balance control in the digital camera according to the second embodiment. FIG. 11 is a flowchart showing a series of processes related to white balance control executed by the CPU when the subject tracking function is on in the digital camera according to the second embodiment. FIG. 12 is a functional block diagram showing a functional configuration of a CPU for performing white balance control in a digital camera according to the third embodiment. FIG. 13 is a flowchart showing a series of processes related to white balance control executed by the CPU when the subject tracking function is on in the digital camera according to the third embodiment. FIG. 14 is a functional block diagram illustrating a functional configuration of a CPU for performing white balance control in a digital camera according to the fourth embodiment. FIG. 15 is a flowchart showing a series of processes related to white balance control executed by the CPU when the subject tracking function is on in the digital camera according to the fourth embodiment.

  Exemplary embodiments of an imaging apparatus and a white balance control method according to the present invention will be explained below in detail with reference to the accompanying drawings. In the following, an example in which the present invention is applied to a digital camera using a CMOS sensor as an image sensor will be described. However, the present invention can be effectively applied to a digital camera using a CCD sensor. The present invention is not limited to digital cameras, and can be widely applied to image pickup apparatuses having a subject tracking function and an auto white balance function.

(First embodiment)
FIG. 1 is a diagram illustrating an appearance of a digital camera according to the present embodiment. FIG. 1-1 is a top view of the digital camera, FIG. 1-2 is a front view of the digital camera, and FIG. It is a reverse view of a digital camera.

  As shown in FIG. 1A, the digital camera according to the present embodiment has a sub LCD 1 that displays the number of still images that can be taken and the like, and a release shutter that is pressed when taking a still image. A button 2 and a mode dial 3 that is operated when switching between various modes such as a recording (shooting) mode for recording an image, a playback mode for reproducing the recorded image, and a SETUP mode for camera setting.

  In addition, as shown in FIG. 1-2, the digital camera according to the present embodiment has a strobe light emitting unit 4 that emits a strobe, a distance measuring unit 5 that measures the distance to the subject, and a not-shown figure. A remote control light receiving unit 6 that receives an infrared signal from a remote control terminal, a lens barrel unit 7 that includes a zoom lens 25, a focus lens 26, a mechanical shutter 27, and a support member that supports these integrally, and an optical finder. (Front) 8 is provided. Further, the side surface of the digital camera according to the present embodiment is provided with a memory card throttle for inserting a memory card 80, which will be described later, and a battery accommodating portion for accommodating a battery. The memory card throttle and the battery accommodating portion are covered with a lid. It is occluded by the body 9.

  In addition, the digital camera according to the present embodiment has an autofocus LED (AFLED) 10 that is turned on when the autofocus is activated, a strobe LED 11 that is turned on when the strobe is lighted, a display of various setting screens, and a display of a playback image. And an LCD monitor 12 used as an electronic viewfinder at the time of photographing, an optical viewfinder (back surface) 13, a zoom button (WIDE) 14 operated during wide-angle zooming, and a zoom button (TELE) 15 operated during telephoto zooming And a power switch 16 and a self-timer / deletion switch 17 operated when the self-timer is activated.

  Furthermore, the digital camera according to the present embodiment has a menu switch 18 that is operated when selecting a menu, an OK switch 19 that is operated when confirming a selected item, and a display on the LCD monitor 12 on the back side. An image confirmation switch (left) 20 and an image confirmation switch (right) 21 that are operated when switching the reproduction image to be performed, a macro switch 22 that is operated when performing macro photography, and a flash light emission mode that are operated. A strobe switch 23 and a display switch 24 operated when switching the display of the LCD monitor 12 are provided.

  FIG. 2 is a block diagram showing a configuration of a control system of the digital camera according to the present embodiment. The digital camera according to the present embodiment includes a motor driver 28 for driving a zoom lens 25, a focus lens 26, and a mechanical shutter 27, a CMOS sensor 30 as an image sensor, and a CMOS sensor 30 as a control system. An analog front end (AFE) 40 that processes the output analog image signal to generate digital image data, a timing generator (TG) 50 that outputs a drive timing signal to the CMOS sensor 30 and the AFE 40, and the AFE 40 A signal processing IC 60 that performs various digital image processing on the output digital image data and controls the operation of the entire digital camera is provided. In addition to the LCD monitor 12 described above, an SDRAM 70, a memory card 80, a ROM 90, an operation unit 100, and the like are connected to the signal processing IC 60.

  The mechanical shutter 27 is a mechanical shutter that allows light to enter the CMOS sensor 30 in the open state and blocks light from entering the CMOS sensor 30 in the closed state. The mechanical shutter 27 is driven by a motor driver 28 together with the zoom lens 25 and the focus lens 26 which are optical lenses. Driving of the zoom lens 25, the focus lens 26, and the mechanical shutter 27 by the motor driver 28 is controlled by a CPU 61 included in the signal processing IC 60.

  The CMOS sensor 30 is a solid-state imaging device that converts an optical image formed by an optical lens into an electrical signal proportional to its brightness and outputs the photoelectric image by a photoelectric conversion function. The CMOS sensor 30 is driven according to a drive timing signal output from the TG 50, and exposure and signal readout of each pixel are controlled. An output electrical signal from the CMOS sensor 30 is input to the AFE 40.

  The AFE 40 samples and holds the output electrical signal (analog pixel signal) of the CMOS sensor 30 (correlated double sampling), an AGC (Auto Gain Control) 42 that adjusts the gain of the signal sampled by the CDS 41, and the AGC 42 And an A / D converter (A / D) 43 for converting the output into a digital signal. Each part constituting the AFE 40 is driven in synchronism with the CMOS sensor 30 in accordance with a drive timing signal output from the TG 50, and gain setting and the like in the AGC 42 are performed in accordance with instructions of the CPU 61 included in the signal processing IC 60. The Digital data (digital image data) output by the AFE 40 is input to the signal processing IC 60.

  The TG 50 is supplied with the vertical synchronization signal VD and the horizontal synchronization signal HD generated by the synchronization signal generation unit 69 included in the signal processing IC 60. The TG 50 generates a drive timing signal for driving the CMOS sensor 30 and the AFE 40 in synchronization based on the vertical synchronization signal VD and the horizontal synchronization signal HD, and supplies the generated drive timing signal to the CMOS sensor 30 and the AFE 40. To do.

  The signal processing IC 60 includes a CPU 61 that comprehensively controls the operation of the entire digital camera, a CMOS I / F unit 62 that captures digital image data from the AFE 40 in accordance with the vertical synchronization signal VD and the horizontal synchronization signal HD, and an SDRAM 70. A memory controller 63 that controls writing and reading of data, a YUV conversion unit 64 that converts digital image data input from the CMOS I / F unit 62 into a YUV data format that can be displayed and recorded, and a predetermined image data. A compression / expansion unit 65 that compresses and expands according to a format (for example, JPEG), a display output control unit 66 that controls output of display data to the LCD monitor 12 and an external display device, and an image size that matches the display and recording size Resize processing unit 67 to be changed and data to the memory card 80 The media I / F unit 68 for controlling writing and reading, and a synchronizing signal generating unit 69 for generating a vertical synchronizing signal VD and horizontal synchronizing signal HD by a command CPU 61.

  Digital image data output from the AFE 40 is taken into the signal processing IC 60 by the CMOS I / F unit 62 as a unit of an image frame, and an image such as black level correction, defect pixel correction, shading correction, etc. is generated inside the signal processing IC 60. After the processing is performed, the memory controller 63 temporarily stores the data in the SDRAM 70 as RGB data (RAW-RGB image data) for each image frame.

  The RAW-RGB image data for each image frame stored in the SDRAM 70 is read from the SDRAM 70 by the memory controller 63 and supplied to the YUV conversion unit 64 through white balance gain multiplication, gamma correction, and RGB interpolation processing. The YUV conversion unit 64 performs various image processing such as edge enhancement and color setting on the image data, and converts the image data into YUV image data that is a luminance / color difference signal. The YUV image data generated by the YUV conversion unit 64 is stored again in the SDRAM 70 by the memory controller 63.

  The YUV image data stored in the SDRAM 70 is read from the SDRAM 70 by the memory controller 63 and supplied to the display output control unit 66 together with OSD (on-screen display) data for displaying shooting mode icons and the like. The display output control unit 66 generates display data by combining the YUV image data and the OSD data, adds a signal such as a synchronization signal, and outputs the display data to the LCD monitor 12. The display output control unit 66 can output the generated display data as a TV video signal and display it on an external display device. For example, if the external display device is an NTSC system TV, a resize processing unit In step 67, horizontal / vertical scaling processing is performed according to the system, and then output to the TV.

  When recording on the memory card 80, the YUV image data stored in the SDRAM 70 is read from the SDRAM 70 by the memory controller 63 according to an instruction from the CPU 61 and supplied to the compression / expansion unit 65. Compressed data is generated. Here, the YUV image data stored in the SDRAM 70 is full-size data, and when the image size to be recorded on the memory card 80 is smaller than the image size of the CMOS sensor 30, the resize processing unit 67 uses the image for recording. Is reduced and supplied to the compression / decompression unit 65. The compressed data (JPEG data) generated by the compression / decompression unit 65 is stored again in the SDRAM 70 by the memory controller 63, and after processing such as header addition is performed, the memory card is passed through the media I / F unit 68. 80.

  The operation of each part in the signal processing IC 60 described above is executed in accordance with instructions from the CPU 61. The CPU 61 loads, for example, a control program and control data for controlling the digital camera stored in the ROM 90 onto the SDRAM 70, for example, and performs overall operation of the digital camera based on the program code of the loaded control program. To control. In other words, the CPU 61 and the AFE 40 can appropriately capture, display, and record an image in accordance with an instruction based on various button keys of the operation unit 100 or an operation instruction from a remote terminal (not shown). , TG50, and each processing unit inside the signal processing IC 60 are controlled. The operation unit 100 is used by the photographer to instruct the operation of the digital camera, and corresponds to various button keys such as the release shutter button 2 and the zoom buttons 14 and 15 shown in FIG.

  Hereinafter, the control by the CPU 61 when shooting a still image in response to pressing of the release shutter button 2 will be described in more detail.

  When the release shutter button 2 is pressed, AF (AUTO FOCUS) processing is performed prior to shooting. As the AF processing, for example, hill-climbing AF for finding the maximum contrast value while moving the focus lens 26 is performed. This is to create an AF evaluation value from the high frequency component of the G signal (or the Y signal that is the luminance value) of the video signal, and calculate the position (focus position) of the focus lens 26 where the AF evaluation value is maximized. In this process, the focus lens 26 is moved.

  As areas for obtaining AF evaluation values, there are multi-area AF in which a plurality of areas are considered for one image frame, spot area AF in which only a part is considered, and the like. Then, a focus position suitable for the shooting scene is calculated based on the AF evaluation value, and the focus lens 26 is moved to that point. The focus lens 26 can be moved via the motor driver 28 according to a command from the CPU 61.

  Next, an electronic shutter for still image shooting, AGC 42 calculation / setting, and aperture calculation / setting are performed (AE (AUTO EXPOUSRE) processing), and recording exposure is performed. The setting of the electronic shutter can be executed by the TG 50 that controls the driving of the CMOS sensor 30 in accordance with an instruction from the CPU 61. The setting of the AGC 42 can be executed by the AFE 40 according to a command from the CPU 61.

  Thereafter, when the exposure for the still image is completed, the mechanical shutter 27 is closed, and the RGB data (RAW-RGB image data) for the still image is output from the CMOS sensor 30. In a camera system equipped with the first release process (half press) and the second release process (full press), AF and AE processes are calculated by executing the first release process, and recording is performed with the second release process. In general, the control is performed such that the exposure is performed.

  RAW-RGB image data for still images is taken into the signal processing IC 60 by the CMOS I / F unit 62, and image processing such as black level correction, missing pixel correction, shading correction, and the like is performed inside the signal processing IC 60. Then, the data is written in the SDRAM 70 by the memory controller 63. In the case of interlaced transfer, all data constituting one image frame is written to the SDRAM 70 as RAW-RGB image data by a plurality of transfers.

  As described above, the RAW-RGB image data for the still image written in the SDRAM 70 is read from the SDRAM 70, subjected to white balance gain multiplication, gamma correction, and RGB interpolation processing. Various image processing such as color setting is performed, converted into YUV data which is a luminance / color difference signal, and written back to the SDRAM 70.

  Thereafter, the YUV image data written in the SDRAM 70 is read out from the SDRAM 70 again, reduced in the resizing processing unit 67 as necessary, and then compressed and expanded by the compression / expansion unit 65, for example, data such as JPEG compression. Compression processing is performed. Then, the compressed data (JPEG data) is written back to the SDRAM 70 again, and after processing such as header addition is performed, the compressed data (JPEG data) is recorded on the memory card 80 via the media I / F unit 68.

  Next, white balance control performed in the digital camera according to the present embodiment will be described in detail.

  As an example, white balance control by the digital camera according to the present embodiment is continuously performed on image frames that are updated as needed. That is, as described above, image data from the CMOS sensor 30 that is an image sensor is input to the signal processing IC 60 as needed by using a frame image as a unit via the AFE 40. White balance control is performed on each image frame input to the signal processing IC 60 as processing by the CPU 61 inside the signal processing IC 60. Note that the image frame subject to white balance control may be an image frame of a monitoring image displayed on the LCD monitor 12 or the like before still image shooting, or continuous still images obtained by continuous shooting or the like, for example. It may be an image frame of an image. In addition, a white balance control value (to be described later) obtained by processing on the image frame of the monitoring image is stored before still image shooting, and the white balance control value stored for the image frame obtained at the time of still image shooting is stored. It may be applied to perform white balance control.

  FIG. 3 is a functional block diagram showing a functional configuration of the CPU 61 for performing white balance control. The CPU 61 calculates a feature amount as shown in FIG. 3, for example, by executing a white balance control program stored in the ROM 90 as one of the control programs described above and loaded on the main memory (eg, SDRAM 70). Each function of the unit 101, the white block detection unit 102, the subject area detection unit 103, and the white balance control unit 104 is realized.

  The feature amount calculation unit 101 divides an image frame output from the CMOS sensor 30 and input to the signal processing IC 60 via the AFE 40 into a plurality of blocks, and calculates a WB evaluation value for each of the divided blocks. It is.

  Based on the WB evaluation value calculated by the feature amount calculation unit 101, the white block detection unit 102 detects a white block that is estimated to be a white block from among the blocks of the image frame divided into a plurality of parts. Processing function.

  The subject area detection unit 103 is a processing function that detects a subject area that is an area in which a subject to be tracked is captured in the current image frame to be processed.

  The digital camera according to the present embodiment has a subject tracking function, and when operating with the subject tracking function turned on, the CPU 61 respectively inputs a plurality of image frames that are continuously input to the signal processing IC 60 in time series. On the other hand, for example, a process of detecting a region (subject area) in which a subject designated as a tracking target by the user is shown. The subject area detection unit 103 performs processing for detecting the subject area from the image frame. When the subject tracking function is on, the subject area detected from the image frame is used, for example, as an area for acquiring the above-described AF evaluation value at the time of still image shooting and as a photometric area for AE processing. Various methods have been proposed for the method of subject tracking itself, and the processing for detecting the subject area from the image frame may be performed using a generally known technique, so the details of the processing will be described. Omitted.

  The white balance control unit 104 calculates a WB control value based on the WB evaluation value of the white block detected by the white block detection unit 102, and controls the white balance of the image frame using the calculated WB control value. It is a function. Here, in particular, in the digital camera according to the present embodiment, the white balance control unit 104 of the CPU 61 uses the information about the subject area detected by the subject area detection unit 103 to compare with the prior art. Thus, a more appropriate WB control value is calculated so that highly accurate white balance control can be performed.

  That is, the white balance control unit 104 detects a white area from the subject area detected by the subject area detection unit 103 in the current image frame to be processed, and detects the CMOS before the current image frame. In the image frame output from the sensor 30 (hereinafter referred to as the preceding frame), white is detected from within the subject area detected by the subject area detection unit 103 (the area where the same subject as the tracking target subject in the current image frame is shown). When no block is detected, the WB control value is calculated based on the WB evaluation value of the other white block without using the white block detected from the subject area in the current image frame. The white balance of the image frame is controlled using the WB control value.The preceding frame is not limited to the image frame output immediately before the current image frame, but is output before the current image frame while tracking the same subject as the current image frame. Including all the image frames.

  Hereinafter, specific examples of white balance control realized by the CPU 61 having the above-described feature amount calculation unit 101, white block detection unit 102, subject area detection unit 103, and white balance control unit 104 will be described in more detail. To do.

  An image frame output from the CMOS sensor 30 via the AFE 40 is taken into the signal processing IC 60 by the CMOS I / F unit 62 of the signal processing IC 60 and stored in the SDRAM 70 (frame memory) by the memory controller 63. At this time, the CMOS I / F unit 62 reads R, G, and B values of a specific part or the entire image frame captured in the signal processing IC 60 and inputs them to the CPU 61.

  The feature amount calculation unit 101 of the CPU 61 divides the image frame taken into the signal processing IC 60 into a plurality of blocks according to a predetermined division method (for example, 256 × 16 × 16), and the CMOS I / F unit 62 Using the read R, G, and B values, the integrated values of the RGB signals of the pixels included in each block are obtained for each divided block. Then, the feature amount calculation unit 101 calculates the ratio (G / R, G / B) of the integrated value of the R signal and the B signal to the integrated value of the G signal for each block. The calculated value is a feature amount representing the color feature of each block, and this is used as the WB evaluation value.

  When the feature amount calculation unit 101 calculates the WB evaluation value (G / R, G / B) of each block in the image frame, the white block detection unit 102 calculates the WB evaluation value (G / R) of each block. , G / B) is plotted on the WB color space with G / R as the horizontal axis and G / B as the vertical axis, and the white block is detected from the distribution of the WB evaluation value of each block in the WB color space. I do.

  FIG. 4 is a diagram showing a state in which the WB evaluation values (G / R, G / B) of each block are plotted on the WB color space. On the WB color space, as shown in FIG. 4, the locus of black body radiation when the color temperature of the light source is changed in advance is prepared as a reference gain. An area (area indicated by an ellipse in FIG. 4) having a certain width around the light source color assumed under the actual shooting environment on the reference gain is set as a white detection area.

  When the white block detection unit 102 plots the WB evaluation value of each block calculated by the feature amount calculation unit 101 on the WB color space, the plotted WB evaluation value (black point in FIG. 4) A block located in the white detection area set in the WB color space is detected as a white block estimated as a block in which white is projected in the image frame. Whether or not the WB evaluation value is within the white detection area can be easily realized by preparing a determination program and executing it by the CPU 61. Since the color characteristics of the same illumination and subject change depending on the spectral sensitivity distribution of the CMOS sensor 30, the reference gain and the white detection area on the white balance color space match the characteristics of the CMOS sensor 30 mounted on the digital camera. It is desirable to decide it.

  When the white block is detected by the white block detection unit 102, the white balance control unit 104 calculates a WB control value based on the WB evaluation value (G / R, G / B) of the detected white block. At this time, the white balance control unit 104 detects a white area from the subject area in the current image frame to be processed, but does not detect a white block from the subject area in the preceding frame. The WB control value is calculated based on the WB evaluation values of the other white blocks without using the white block detected in the subject area in the current image frame. The WB control value is a control value for making the detected white block appear white on the image, so that the RGB integration ratio of each pixel in the white block approaches R: G: B = 1: 1: 1. , The gain value multiplied by the R signal and the B signal.

  Here, since the WB evaluation value is the ratio (G / R, G / B) of the integrated value of the R signal and the B signal with respect to the integrated value of the G signal as described above, attention should be paid to only one white block. The white block G / R value is set as R gain, G / B is set as B gain, R signal is multiplied by R gain, and B signal is multiplied by B gain, thereby integrating RGB. The ratio can be R: G: B = 1: 1: 1. However, actually, a plurality of white blocks are detected from one image frame, and the detected WB evaluation values of the plurality of white blocks often have different values. Therefore, the white balance control unit 104 uses, for example, an average value of G / R values of a plurality of detected white blocks as an R gain, and determines an average value of G / B values of the detected plurality of white blocks as B. As a gain, a WB control value (R gain, B gain) is calculated. The actual shooting environment is often affected by a plurality of light sources, and white blocks may be detected in a plurality of white detection areas as in the example shown in FIG. In such a case, the white balance control unit 104 may calculate the WB control value by, for example, a weighted average weighted according to the number of white blocks detected in each of the plurality of white detection areas. . Alternatively, a different WB control value may be calculated for each region where the influence of each light source is dominant, and the WB control value may be linearly changed near the boundary of the region. Note that the method of calculating the WB control value by the white balance control unit 104 is not limited to the above. For example, there are various methods such as a device for reducing the color failure or a device for producing a hue preferable for the user. You may make it determine the value finally obtained after devising as a WB control value.

  The white balance control unit 104 multiplies the R signal and the B signal of each pixel constituting the frame image by the WB control value (R gain, B gain) calculated as described above, thereby white balance of the image frame. To control. Specifically, as described above, white balance gain multiplication is performed as pre-processing for converting the RAW-RGB image data read from the SDRAM 70 into YUV data by the YUV conversion unit 64. When the white balance control unit 104 sets a WB control value for the YUV conversion unit 64, white balance control based on the WB control value is executed.

  FIGS. 5 and 6 are diagrams for explaining a specific example of white balance control in the present embodiment. FIG. 5 illustrates image frame A and image frame B input in time series with the movement of the subject to be tracked. FIG. 6 is a diagram illustrating a state in which the WB evaluation values calculated from the image frame A and the image frame B in FIG. 5 are plotted on the WB color space.

  The example shown in FIG. 5 assumes a case where the subject to be tracked moves from a position where light from a certain light source is not directly irradiated to a position where it is directly irradiated (under the light source) as shown in FIG. ing. An image frame shot (input) before the tracking target subject moves under the light source is an image frame A shown in FIG. 5B, and is shot (input) after the tracking target subject moves under the light source. The obtained image frame is an image frame B shown in FIG. That is, the image frame A becomes a preceding frame in relation to the image frame B. 5D shows the position of the block in the image frame A corresponding to the subject area detected in the image frame A, and FIG. 5E shows the subject area detected in the image frame B. The position of the block in the image frame B corresponding to is shown. In this example, in order to simplify the description, the number of blocks in one image frame is 54 × 6 × 9, and the size of the subject area is equivalent to two blocks. .

  In the white balance control in the present embodiment, as described above, an appropriate WB control value is calculated using information on the subject area detected from the image frame. Specifically, in the image frame A, since the subject area is detected at the position indicated by the oblique lines in FIG. 5D, (horizontal position, vertical position) = (2, 2), ( A block of (horizontal size, vertical size) = (2, 1) is a block corresponding to the subject area. On the other hand, in the image frame B, since the subject area is detected at the position indicated by the oblique lines in FIG. 5E, (horizontal position, vertical position) = (4, 4), (horizontal size, A block of (vertical size) = (2, 1) is a block corresponding to the subject area. Using information on the position of the block corresponding to the subject area, it is determined whether or not a white block has been detected from within the subject area.

  That is, in the white balance control for the image frame A, it is determined whether or not a block of (horizontal position, vertical position) = (2, 2), (horizontal size, vertical size) = (2, 1) is detected as a white block. judge. Here, the WB evaluation value (G / R, G / B) of the block of (horizontal position, vertical position) = (2, 2), (horizontal size, vertical size) = (2, 1) of the image frame A is As shown in FIG. 6, if it is (0.4, 0.6), this WB evaluation value is out of the white detection area, so in the image frame A, a white block is detected from within the subject area. Judge that it is not.

  On the other hand, in the white balance control for the image frame B, it is determined whether a block of (horizontal position, vertical position) = (4, 4), (horizontal size, vertical size) = (2, 1) is detected as a white block. judge. Here, the WB evaluation values (G / R, G / B) of the block of (horizontal position, vertical position) = (4, 4), (horizontal size, vertical size) = (2, 1) of the image frame B are As shown in FIG. 6, if it is (0.5, 0.9), this WB evaluation value is in the white detection area, so in the image frame B, a white block is detected from within the subject area. It is determined that it has been detected.

  In the image frame A and the image frame B, the WB evaluation value of the block corresponding to the subject detection area changes because the tracking target subject moves under a different light source, resulting in the color (object color) of the tracking target subject itself. This is because the signal output from the CMOS sensor 30 changes due to the influence of the change of the light source even if the same is applied. In the conventional technique, even in such a case, when white balance control is performed on the image frame B, the WB control value is calculated using the WB evaluation values of all the white blocks including the white block detected from within the subject area. Had to do. For this reason, the WB control value in the direction of whitening the subject to be tracked is calculated, which may cause a color failure.

  On the other hand, in the white balance control according to the present embodiment, when white balance control is performed on the image frame B, no white block is detected from the subject area in the image frame A that is the preceding frame. The WB evaluation value of the white block detected from the inside is not used for the calculation of the WB control value, but the WB control value is calculated using the WB evaluation value of the white block other than the white block detected from the subject area. I do. As a result, an appropriate WB control value can be calculated, white balance control can be performed with high accuracy, and the occurrence of color failure can be effectively suppressed.

  7 and 8 are diagrams for explaining another specific example of the white balance control in the present embodiment. FIG. 7 shows the movement of the subject to be tracked and the image frame C and the image frame D input in time series. FIG. 8 is a diagram for explaining the position of the subject area in the image frame E. FIG. 8 shows a state in which the WB evaluation values calculated from the image frame C, the image frame D, and the image frame E in FIG. FIG.

  In the example shown in FIG. 7, as shown in FIG. 7A, after a subject located under a certain light source is set as a tracking target (locked on), the subject to be tracked is light from the light source. It is assumed that the light has moved to a position that is not directly irradiated and then returned to the light source. An image frame captured (input) when the tracking target subject is locked on is an image frame C shown in FIG. 7B, and is captured (input) after the tracking target subject has moved from under the light source. The image frame is an image frame D shown in FIG. 7C, and an image frame taken (input) after the tracking target subject returns under the light source is an image frame E shown in FIG. 7D. It is. That is, the image frame D is a preceding frame in relation to the image frame E. FIG. 7E shows the position of the block in the image frame C corresponding to the subject area detected in the image frame C. FIG. 7F shows the subject area detected in the image frame D. 7 (g) shows the position of the block in the image frame E corresponding to the subject area detected in the image frame E. FIG.

  In the image frame C, since the subject area is detected at the position indicated by the oblique lines in FIG. 7E, (horizontal position, vertical position) = (3,4), (horizontal size, vertical size) with the upper left of the image as a reference ) = (2, 1) is a block corresponding to the subject area. Further, in the image frame D, the subject area is detected at the position indicated by the oblique lines in FIG. 7F, so that the upper left corner of the image is the reference (horizontal position, vertical position) = (7, 5), (horizontal size, A block of (vertical size) = (2, 1) is a block corresponding to the subject area. Further, in the image frame E, the subject area is detected at the position indicated by the oblique lines in FIG. 7G, so that the upper left corner of the image is the reference (horizontal position, vertical position) = (5, 5), (horizontal size, A block of (vertical size) = (2, 1) is a block corresponding to the subject area.

  In this example, the WB evaluation value (G / R, G / B) of the block of (horizontal position, vertical position) = (3, 4), (horizontal size, vertical size) = (2, 1) of the image frame C ) Is (0.5, 0.9) as shown in FIG. 8, and is within the white detection area. Since the image frame C is an image frame shot (input) when the subject to be tracked is locked on, there is no image frame serving as a preceding frame. For this reason, in the white balance control for the image frame C, the WB control value is calculated using the WB evaluation values of all the white blocks including the white block detected in the subject area.

  Next, in the white balance control for the image frame D, the (horizontal position, vertical position) = (7, 5), (horizontal size, vertical size) = (2, 1) block of the image frame D is a white block. Determine whether it was detected. Here, the WB evaluation values (G / R, G / B) of the block of (horizontal position, vertical position) = (7, 5), (horizontal size, vertical size) = (2, 1) of the image frame D are obtained. As shown in FIG. 8, if it is (0.4, 0.6), this WB evaluation value is out of the white detection area, so in the image frame D, a white block is detected from within the subject area. Judge that it is not.

  Next, in the white balance control for the image frame E, a block of (horizontal position, vertical position) = (5, 5), (horizontal size, vertical size) = (2, 1) of the image frame E is set as a white block. Determine whether it was detected. Here, the WB evaluation values (G / R, G / B) of the block of (horizontal position, vertical position) = (5, 5), (horizontal size, vertical size) = (2, 1) of the image frame E are As shown in FIG. 8, if it is (0.5, 0.9), since this WB evaluation value is in the white detection area, in the image frame E, a white block is detected from within the subject area. It is determined that it has been detected.

  In this example, when white balance control is performed on the image frame E, no white block is detected from the subject area in the image frame D, which is the preceding frame. Therefore, the WB evaluation of the white block detected from the subject area is performed. The value is not used for calculating the WB control value, but the WB control value is calculated using the WB evaluation value of the white block other than the white block detected from the subject area. As a result, as in the above-described example, an appropriate WB control value can be calculated, white balance control can be performed with high accuracy, and the occurrence of color failure can be effectively suppressed.

  FIG. 9 is a flowchart showing a series of processes related to white balance control executed by the CPU 61 when the subject tracking function is on. The series of processes shown in the flowchart of FIG. 9 is repeatedly performed every time an image frame is input from the CMOS sensor 30 to the signal processing IC 60 via the AFE 40 as an example.

  When an image frame is input into the signal processing IC 60 (step S101), the feature amount calculation unit 101 of the CPU 61 divides the input image frame into a plurality of blocks, and WB evaluation values ( G / R, G / B) is calculated (step S102).

  Next, the white block detection unit 102 of the CPU 61 is estimated as a block showing white in each of the blocks of the image frame divided into a plurality based on the WB evaluation value calculated by the feature amount calculation unit 101. A white block is detected (step S103).

  Next, it is determined whether or not the tracking target subject has already been set (locked on) (step S104). If the tracking target subject has already been set (step S104: YES), the CPU 61 detects the subject area. The unit 103 detects a subject area showing the subject to be tracked from the image frame input in step S101 (step S105). On the other hand, when the tracking target subject is not set (step S104: NO), the subject area detection unit 103 sets the subject in the image frame input in step S101 as the tracking target in accordance with, for example, designation from the user. Setting is made (step S106), and the position of the subject is set as a subject area.

  Next, based on the position of the white block detected by the white block detection unit 102 and the position of the subject area detected by the subject area detection unit 103, the white balance control unit 104 of the CPU 61 performs white detection from within the subject area. It is determined whether a block is detected (step S107). If no white block is detected from the subject area (step S107: NO), the white balance control unit 104 sets the determination result flag to 1 (step S108), and the process proceeds to step S111. To do. The determination result flag is 1-bit information indicating whether or not a white block is detected in the subject area in the preceding frame, and the determination result flag = 0 indicates that a white block is detected in the subject area. The determination result flag = 1 indicates that no white block is detected in the subject area.

  On the other hand, when a white block is detected from the subject area (step S107: YES), the white balance control unit 104 determines whether or not the determination result flag = 1, that is, from the subject area in the preceding frame. It is confirmed whether or not a white block has been detected (step S109). If the determination result flag = 1 (step S109: YES), the white balance control unit 104 excludes the white block detected from the subject area, and the WB of the white block detected in step S103. Based on the evaluation value, a WB control value is calculated (step S110). On the other hand, if the determination result flag = 0 (step S109: NO), or if no white block is detected from within the subject area (step S107: NO), all the whites detected in step S103 are detected. A WB control value is calculated based on the WB evaluation value of the block (step S111).

  For example, the WB control values calculated in step S110 or step S111 are set in the YUV conversion unit 64 as R gain and B gain. Then, auto white balance is performed by multiplying the R signal of the RAW-RGB image data with the image frame as one unit by the R gain and multiplying the B signal by the B gain.

  As described above in detail with specific examples, in the digital camera according to the present embodiment, the white balance control unit 104 of the CPU 61 detects a white block from within the subject area, and in the preceding frame. When no white block is detected in the subject area, the white balance of the image frame is controlled using the WB control value calculated using the WB evaluation value of the white block other than the white block detected in the subject area. I am doing so. Therefore, according to the digital camera of the present embodiment, it is possible to improve the accuracy of auto white balance and effectively suppress the occurrence of color failure.

(Second Embodiment)
Next, a digital camera according to the second embodiment will be described. The digital camera according to the present embodiment has a function of determining whether or not the subject to be tracked is a person's face, and when the subject to be tracked is a person's face, from within the subject area in the preceding frame Regardless of whether or not a white block is detected, the white block detected from the subject area in the current image frame to be processed is not used, but based on the WB evaluation values of other white blocks. The control value is calculated. Since the basic configuration of the digital camera and the outline of the process are the same as those in the first embodiment described above, the same parts as those in the first embodiment are denoted by the same reference numerals and redundantly described below. The description that has been made will be omitted.

  FIG. 10 is a functional block diagram illustrating a functional configuration of the CPU 61 ′ for performing white balance control in the digital camera according to the present embodiment. As shown in FIG. 10, the CPU 61 ′ in this embodiment further includes a face determination unit 105 and a correction unit 106 in addition to the functional configuration of the CPU 61 in the first embodiment (see FIG. 3).

  The face determination unit 105 is a processing function that determines whether or not the tracking target subject is a human face. As an example, the face determination unit 105 performs a process of detecting a human face by pattern matching or the like with respect to an image frame, and when a human face is detected in the subject area detected by the subject area detection unit 103, It is determined that the subject to be tracked is a human face. Various methods have been proposed in the past for detecting a person's face from an image, and the face determination unit 105 may detect the person's face using these known techniques. The determination by the face determination unit 105 is not necessarily performed for all image frames. If it is determined that the subject to be tracked is a person's face while tracking the same subject, the information is obtained. By memorizing, determination in a later image frame can be omitted.

  In the white balance control in the present embodiment, when the face determination unit 105 determines that the subject to be tracked is a person's face, the white balance control unit 104 ′ detects a white block from the subject area in the preceding frame. WB evaluation of other white blocks detected by the white block detection unit 102 without using the white block detected from the subject area in the current image frame to be processed without determining whether or not it has been detected. A WB control value is calculated based on the value. In other words, if it is determined that the subject to be tracked is a human face, even if a white block is detected from the subject area, it is not a white region, but the WB evaluation value is detected by white due to the influence of light from the light source. It can be determined that the area has entered the area. Therefore, in this case, the white balance control unit 104 ′ calculates the WB control value based on the WB evaluation values of other white blocks excluding the white block in the subject area, thereby enabling more appropriate white balance control. It becomes possible.

  When the face determination unit 105 determines that the tracking target subject is a human face, the correction unit 106 uses the WB evaluation value of the block included in the subject area to calculate the white balance control unit 104 ′. This is a processing function for correcting the WB control value. Specifically, when the face determination unit 105 determines that the tracking target subject is a human face, the correction unit 106 determines the WB evaluation value of the block included in the subject area and the white balance control unit 104 ′. Is compared with the calculated WB control value. When there is a divergence greater than or equal to a predetermined threshold value between the two, the WB control value calculated by the white balance control unit 104 ′ is corrected so as to approach the WB evaluation value of the block included in the subject area. That is, the correction unit 106 determines that the WB control value calculated by the white balance control unit 104 ′ is constant from the WB evaluation value of a block (a block showing a person's face) included in the subject area in the WB color space described above. The feedback control is performed so as to be within the range of the distance. In addition, as a specific correction method by the correction unit 106, the technique disclosed in Japanese Patent Laid-Open No. 2008-236101 proposed by the present inventor can be applied.

  FIG. 11 is a flowchart showing a series of processes related to white balance control executed by the CPU 61 'when the subject tracking function is on in the digital camera according to the present embodiment. The series of processes shown in the flowchart of FIG. 11 is repeatedly performed every time an image frame is input from the CMOS sensor 30 to the signal processing IC 60 via the AFE 40 as an example. In the flowchart of FIG. 11, the processes in steps S201 to S206 are the same as the processes in steps S101 to S106 shown in the flowchart of FIG. 9, and the processes in steps S208 to S212 are the flowchart of FIG. Since this is the same as the processing in steps S107 to S211 shown, description of these processing is omitted.

  In the white balance control in the present embodiment, the face balance determination unit 105 of the CPU 61 ′ before the white balance control unit 104 ′ of the CPU 61 ′ determines whether or not a white block is detected from the subject area in step S208. However, it is determined whether or not the subject to be tracked is a human face (step S207). When it is determined that the subject to be tracked is not a person's face (step S207: NO), the white balance control unit 104 ′ performs the same processing as the first embodiment (the processing from step S208 to step S212). ). On the other hand, when it is determined that the subject to be tracked is a person's face (step S207: YES), the white balance control unit 104 ′ determines the WB based on the WB evaluation value of the white block detected from other than the subject area. A control value is calculated (step S213).

  Next, the correction unit 106 of the CPU 61 ′ compares the WB control value calculated in step S213 with the WB evaluation value of the block in the subject area, and whether or not the difference between them is equal to or greater than a predetermined threshold value. Is determined (step S214). If the difference between the two is equal to or greater than the threshold (step S214: YES), the correction unit 106 brings the WB control value calculated in step S213 closer to the WB evaluation value of the block in the subject area. (Step S215).

  As described above, in the digital camera according to the present embodiment, the face determination unit 105 of the CPU 61 ′ determines whether the subject to be tracked is a human face, and the subject to be tracked is a human face. If it is determined that the white block is present, the white balance control unit 104 ′ determines whether or not the white block is detected from within the subject area in the preceding frame, based on the WB evaluation value of the white block detected from other than the subject area. Thus, the WB control value is calculated. Therefore, compared to the first embodiment described above, the accuracy of auto white balance can be further improved, and the occurrence of color failure can be more effectively suppressed.

  Also, with the digital camera according to the present embodiment, when it is determined that the subject to be tracked is a person's face, the correction unit 106 of the CPU 61 ′ performs the WB calculated by the white balance control unit 104 ′. When the control value is compared with the WB evaluation value of the block in the subject area, and the difference between the two is equal to or greater than the threshold value, the WB control value calculated by the white balance control unit 104 ′ is used as the WB of the block in the subject area. Since correction is performed so as to approach the evaluation value, auto white balance in consideration of the face of a person can be performed, and the occurrence of color failure can be more effectively suppressed.

(Third embodiment)
Next, a digital camera according to the third embodiment will be described. The digital camera according to the present embodiment has a function of determining the size of the subject area. If the size of the subject area is smaller than the reference value, whether or not a white block is detected in the subject area in the preceding frame. Regardless of this, the WB control value is calculated based on the WB evaluation value of other white blocks without using the white block detected from the subject area in the current image frame to be processed. It is a thing. Since the basic configuration of the digital camera and the outline of the process are the same as those in the first embodiment described above, the same parts as those in the first embodiment are denoted by the same reference numerals and redundantly described below. The description that has been made will be omitted.

  FIG. 12 is a functional block diagram showing a functional configuration of the CPU 61 ″ for performing white balance control in the digital camera according to the present embodiment. As shown in FIG. 12, the CPU 61 ″ in the present embodiment further includes a subject area determination unit 107 in addition to the functional configuration of the CPU 61 in the first embodiment (see FIG. 3).

  The subject area determination unit 107 is a processing function that determines the size of the subject area detected by the subject area detection unit 103. When the size of the subject area is sufficiently small relative to the size of the entire image frame, the influence of the WB evaluation value of the block in the subject area on the calculation of the WB control value is small. For example, when the entire image frame is divided into 256 × 16 × 16 blocks, if the size of the subject area corresponds to only one block, the WB evaluation value of the block in the subject area is ignored. be able to. In the present embodiment, a subject area determination unit 107 is provided to determine whether or not the size of the subject area is smaller than a predetermined reference value.

  In the white balance control according to the present embodiment, when the subject area determination unit 107 determines that the size of the subject area is smaller than the reference value, the white balance control unit 104 '' detects from the subject area in the preceding frame. Whether or not a white block has been detected is not determined, and a WB control value is calculated based on the WB evaluation value of another white block detected from outside the subject area in the current image frame to be processed. In other words, if the size of the subject area is smaller than the reference value, the WB control value can be calculated without taking the subject area into account, thereby making the auto white balance unstable due to the effect of subject tracking. It is possible to effectively prevent the power consumption by reducing the processing load on the CPU 61 ″.

  FIG. 13 is a flowchart showing a series of processes related to white balance control executed by the CPU 61 ″ when the subject tracking function is on in the digital camera according to the present embodiment. The series of processes shown in the flowchart of FIG. 13 is repeatedly performed every time an image frame is input from the CMOS sensor 30 to the signal processing IC 60 via the AFE 40 as an example. In the flowchart of FIG. 13, the processing of steps S301 to S308 is the same as the processing of steps S101 to S108 shown in the flowchart of FIG. 9, and the processing of steps S310 to S312 is the flowchart of FIG. 9. Since this is the same as the processing in steps S109 to S111 shown, description of these processing is omitted.

  In the white balance control according to the present embodiment, when it is determined that a white block is detected from within the subject area (step S307: Yes), the subject area determination unit 107 of the CPU 61 ″ determines the size of the subject area. Is greater than or equal to a predetermined reference value (step S309). If it is determined that the size of the subject area is greater than or equal to the reference value (step S309: YES), the white balance control unit 104 ″ determines that the determination result flag = 1 as in the first embodiment. Whether or not a white block is detected from the subject area in the preceding frame (step S310). On the other hand, when it is determined that the size of the subject area is smaller than the reference value (step S309: NO), the white balance control unit 104 '' proceeds to step S311 without performing the determination of step S310. In step S311, the WB control value is calculated based on the WB evaluation value of the white block detected from outside the subject area.

  As described above, in the digital camera according to the present embodiment, when the subject area determination unit 107 of the CPU 61 ″ determines the size of the subject area and the size of the subject area is smaller than a predetermined reference value. The white balance control unit 104 ″ determines the WB control value based on the WB evaluation value of the white block detected from outside the subject area, regardless of whether or not the white block is detected from within the subject area in the preceding frame. I am trying to calculate. Therefore, as in the first embodiment described above, the accuracy of auto white balance can be improved, the occurrence of color failure can be more effectively suppressed, and auto white balance is unstable due to the effect of subject tracking. Can be effectively prevented, and further, the processing load on the CPU 61 ″ can be reduced to reduce the power consumption.

(Fourth embodiment)
Next, a digital camera according to the fourth embodiment will be described. The digital camera according to the present embodiment determines whether the background of the subject to be tracked has changed between a plurality of image frames that are continuously output from the CMOS sensor 30 and input into the signal processing IC 60. When the background of the subject to be tracked has a function, the white balance control is not performed on the image frame to be processed. Since the basic configuration of the digital camera and the outline of the process are the same as those in the first embodiment described above, the same parts as those in the first embodiment are denoted by the same reference numerals and redundantly described below. The description that has been made will be omitted.

  FIG. 14 is a functional block diagram showing a functional configuration of the CPU 61 ″ ″ for performing white balance control in the digital camera according to the present embodiment. As shown in FIG. 14, the CPU 61 ″ in the present embodiment further includes a background change determination unit 108 in addition to the functional configuration of the CPU 61 in the first embodiment (see FIG. 3).

  The background change determination unit 108 is a processing function that determines whether the background of the subject to be tracked has changed between a plurality of image frames that are continuously output from the CMOS sensor 30 and input into the signal processing IC 60. is there.

  In a shooting scene where the subject to be tracked is moving in a fixed background, the light source in the image does not change significantly, but for example, the subject to be tracked may be followed while changing the camera angle as needed. In a photographic scene, the light source may change greatly between a plurality of consecutive image frames. If the background of the subject changes between a plurality of consecutive image frames and the light source changes greatly, auto white balance may become unstable if white balance control is executed in units of image frames. There is.

  Therefore, in the present embodiment, the background change determination unit 108 of the CPU 61 ′ ″ determines whether or not the background of the subject has changed between a plurality of consecutive image frames, and the background of the subject has changed. If it is determined that the image frame is present, the white balance control unit 104 ′ ″ does not perform white balance control on the image frame to be processed. When the white balance control unit 104 ″ ″ does not perform white balance control, a new WB control value corresponding to the current image frame is not calculated. In this case, as an example, the R and B signals of the RAW-RGB image data corresponding to the current image frame are multiplied by the R gain and B gain already set in the YUV conversion unit 64.

  Various methods are conceivable for the background change determination unit 108 to determine whether or not the background of the subject has changed between a plurality of consecutive image frames. For example, it is possible to determine whether or not the background has changed by comparing the AE evaluation value, the WB evaluation value, and the AF evaluation value at the top, bottom, left and right edges of the image frame between successive image frames. If there is no change in the background of the subject between a plurality of consecutive image frames, the change in these evaluation values is small. On the other hand, if the background of the subject changes between a plurality of successive image frames, these evaluation values change greatly. Therefore, the background change determination unit 108 can determine that the background of the subject has changed when these evaluation values change by a predetermined threshold or more between a plurality of consecutive image frames.

  FIG. 15 is a flowchart showing a series of processing related to white balance control executed by the CPU 61 ″ when the subject tracking function is on in the digital camera according to the present embodiment. The series of processing shown in the flowchart of FIG. 15 is repeatedly performed every time an image frame is input from the CMOS sensor 30 to the signal processing IC 60 via the AFE 40 as an example. In the flowchart of FIG. 15, the processing of steps S401 to S406 is the same as the processing of steps S101 to S106 shown in the flowchart of FIG. 9, and the processing of steps S408 to S412 is the flowchart of FIG. 9. Since this is the same as the processing in steps S107 to S111 shown, description of these processing is omitted.

  In the white balance control in the present embodiment, the CPU 61 ″ before the white balance control unit 104 ′ ″ of the CPU 61 ′ ″ determines whether or not a white block is detected from within the subject area in step S408. The background change determination unit 108 determines whether or not the background of the subject to be tracked has changed (step S407). If it is determined that there is no change in the background of the subject to be tracked (step S407: NO), the white balance control unit 104 ″ ′ performs the same processing as in the first embodiment (steps S408 to S412). Process). On the other hand, when it is determined that the background of the subject to be tracked has changed (step S407: YES), the white balance control unit 104 ′ ″ performs WB control corresponding to the current image frame to be processed. The process ends without calculating the value.

  As described above, in the digital camera according to the present embodiment, the background change determination unit 108 of the CPU 61 ′ ″ determines whether the background of the subject to be tracked has changed between a plurality of consecutive image frames. When it is determined that the background has changed, the white balance control unit 104 ′ ″ does not perform white balance control for the current image frame to be processed. Therefore, as in the first embodiment described above, the accuracy of auto white balance can be improved, the occurrence of color failure can be more effectively suppressed, and auto light emission can be caused by a significant change in the light source. It is possible to effectively prevent the white balance from becoming unstable.

  As described above, the first to fourth embodiments have been described in detail as specific embodiments of the present invention. However, the present invention is not limited to the above-described embodiments as they are, and the gist thereof is described in the implementation stage. The constituent elements can be modified and embodied without departing from the above. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the respective embodiments described above. For example, you may delete a some component from all the components shown by each embodiment mentioned above. Furthermore, constituent elements over different embodiments may be appropriately combined.

30 CMOS sensor 60 Signal processing IC
61 (61 ′, 61 ″, 61 ′ ″) CPU
DESCRIPTION OF SYMBOLS 101 Feature-value calculation part 102 White block detection part 103 Subject area detection part 104 (104 ', 104'',104''') White balance control part 105 Face determination part 106 Correction | amendment part 107 Subject area determination part 108 Background change determination part

JP 2007-104200 A JP 2009-65381 A

Claims (6)

An image sensor;
A feature amount calculating unit that divides an image frame output from the image sensor into a plurality of blocks, and calculates a feature amount representing a color feature of each block for each of the plurality of divided blocks;
Based on the feature amount, a white block detection unit that detects a white block that is estimated to be a block that reflects white among the plurality of blocks;
A subject area detecting means for detecting a subject area that shows a subject to be tracked in the image frame;
The white block is detected from within the subject area in the image frame, and the white block is detected from within the subject area in a preceding frame that is an image frame output from the image sensor before the image frame. A white balance for controlling the white balance of the image frame using a control value calculated based on the feature amount of the white block other than the white block detected in the subject area in the image frame. And an imaging device. Face determination means for determining whether or not the subject is a person's face;
If it is determined that the subject is a person's face, the white balance control means determines whether the subject is in the image frame regardless of whether the white block is detected from within the subject area in the preceding frame. The imaging apparatus according to claim 1, wherein the control value is calculated based on the feature amount of the white block other than the white block detected from within the area.   And a correction unit that corrects the control value calculated by the white balance control unit using the feature amount of the block included in the subject area when the subject is determined to be a human face. The imaging apparatus according to claim 2, characterized in that: A subject area determination means for determining the size of the subject area;
When it is determined that the size of the subject area is smaller than a reference value, the white balance control means determines whether the white block is detected from within the subject area in the preceding frame. The imaging apparatus according to claim 1, wherein the control value is calculated based on the feature amount of the white block other than the white block detected from within the subject area in a frame. A background change determination means for determining whether or not the background of the subject has changed between the plurality of image frames continuously output from the image sensor;
The imaging apparatus according to claim 1, wherein the white balance control unit does not control the white balance when it is determined that the background of the subject has changed. Dividing the image frame output from the image sensor into a plurality of blocks, and calculating a feature amount representing a color feature of each block for each of the plurality of divided blocks;
Detecting a white block that is estimated to be a white-colored block among the plurality of blocks based on the feature amount; and
Detecting a subject area showing a subject to be tracked in the image frame;
The white block is detected from within the subject area in the image frame, and the white block is detected from within the subject area in a preceding frame that is an image frame output from the image sensor before the image frame. If not, using the control value calculated based on the feature amount of the white block other than the white block detected in the subject area in the image frame, controlling the white balance of the image frame; And a white balance control method.

Images