JP2013132065A - Imaging apparatus and flash control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image the exposure of both a subject and the background of which is appropriate, even if the top light or backlight phenomenon has occurred.SOLUTION: The imaging apparatus capable of performing flash photography includes a face detection unit (114) for detecting the face region from the image of an image signal obtained from an imaging element (101), control means (115) for determining whether or not flash photography is performed based on the brightness in a face region thus detected, and estimation means (103) for estimating a light source from the image signal. The control means determines to perform flash photography even if a part of the face region is bright. The condition for determining to perform flash photography is either the fact that the brightness of a segmented region including the eye, out of the segmented regions of the face region, indicates underexposure and/or the fact that the brightness difference of two preset regions, out of the segmented regions, reaches or goes above a threshold, and includes a fact that the estimated light source is a preset light source.

Description

  The present invention relates to an imaging apparatus and a flash control method, and more particularly to an imaging apparatus and a flash control method that perform flash emission control according to a face area detected from a captured image.

  2. Description of the Related Art Conventionally, there is an imaging apparatus provided with an automatic exposure control function that automatically determines exposure based on image data obtained by photographing in a digital camera. As photometry methods performed for automatic exposure control, for example, there are photometry performed in consideration of the entire screen, and multi-division metering in which the photometry area in the screen is divided into a plurality of blocks and the photometry of each block is performed. In addition, there are center-weighted metering that performs metering with emphasis on the center of the screen, and spot metering that meteres only an arbitrary range in the center of the screen. However, in these photometry methods, there are cases where a person cannot be properly exposed, which is generally the main subject.

  Therefore, in order to make the luminance of the main subject appropriate, there is a method in which the user designates an area where the main subject exists and adjusts the exposure according to the luminance of the designated area. Furthermore, a camera that automatically detects, for example, a face as a main subject from image data obtained by photographing using a technique such as shape analysis, and performs exposure control so that the detected face has an appropriate exposure. It has been proposed (for example, see Patent Document 1). With this camera, even if the brightness of the entire screen is appropriate, the entire screen is brightened if the face is underexposed, and the entire screen is darkened if the face is overexposed. In this way, in any scene, shooting can be performed so that the luminance of the main subject is appropriate.

  In addition, a camera that detects a face from an image obtained by test light emission and performs exposure control so that the face area is properly exposed even during flash photography has been proposed (see, for example, Patent Document 2). .

  Further, an imaging apparatus that automatically performs flash photography when it is determined whether it is backlit and when it is determined that it is backlit is also commercially available.

JP 2003-107555 A JP 2004-184508 A

  However, in the method disclosed in Patent Document 1, since exposure is controlled so that the detected brightness of the face is appropriate, the background exposure is over or under even if the face brightness is appropriate. May end up. FIG. 9 shows an example in which overexposure of the background is deteriorated by performing exposure control so that the brightness of the face becomes appropriate when the face is underexposed and the background is overexposed. In such a case, there is a problem that the image becomes a failed image when viewed as an entire image.

  In addition, in an imaging device that automatically performs flash photography during backlighting, a part of the face is bright due to a phenomenon in which a shadow is generated in the face area by illumination from above (top light) (hereinafter referred to as “top light phenomenon”). For this reason, it may not be judged as “backlight”. As a result, flash photography is not performed, and the brightness of the face may not be sufficient. For example, when a hat is worn outdoors during the daytime, a large shadow may appear in the face area even if it is not backlit, and the face may appear dark.

  Also, in the invention disclosed in Patent Document 2, it is not determined that flash photography is necessary for normal photography in which people are photographed outdoors during the day, and when backlighting or a toplight phenomenon occurs. My face looks dark.

The present invention has been made in view of the above problems, and an imaging apparatus capable of performing flash photography using a flash according to the present invention is a face that detects a face area from an image of an image signal obtained from an imaging device. Detection means, control means for determining whether to perform flash photography based on the luminance in the face area detected by the face detection means, and estimation means for estimating a light source from the image signal, the control means , Determining to perform the flash shooting even if a part of the face area is bright, and as a condition for determining to perform the flash shooting, the eye includes, among the divided areas obtained by dividing the face area, the eye Either at least one of the brightness of the divided area indicates under-exposure and the brightness difference between two preset areas among the divided areas is equal to or greater than a threshold value. I, and includes the estimated light source is a light source which is set in advance.

  According to the present invention, it is possible to obtain an image in which both the subject and the background are properly exposed even when the backlight or the toplight phenomenon occurs.

It is a schematic block diagram which shows the function structure of the imaging device provided with the face detection function in embodiment of this invention. It is a flowchart explaining the pattern recognition process by template matching. It is a figure explaining the concept of template matching. It is a flowchart which shows the calculation process of the WB correction value in embodiment of this invention. It is a figure which shows the example which divided | segmented the screen into arbitrary several blocks. It is a figure which shows a white detection range. It is a flowchart explaining the light emission control processing of the flash in embodiment of this invention. It is the schematic which shows an example of the luminance distribution in case the backlight and the toplight phenomenon have occurred. It is a conceptual diagram for demonstrating the problem of the exposure control using the conventional face detection.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic block diagram illustrating a functional configuration of an imaging apparatus having a face detection function according to the first embodiment of the present invention as an example of an image processing apparatus.

  In FIG. 1, reference numeral 101 denotes a solid-state image sensor composed of a CCD sensor, a CMOS sensor, or the like. The surface of the solid-state image sensor is covered with, for example, an RGB color filter having a Bayer arrangement, and is capable of color photography. A memory 102 temporarily stores an image signal obtained from the image sensor 101.

  Reference numeral 114 denotes a face detection unit that detects a face area from the image signal stored in the memory 102. Various techniques have been proposed as a technique for detecting a face region, and any technique may be used as long as it can acquire face position and size information, and the present invention is limited by the face detection technique. It is not a thing. For example, a method using learning typified by a neural network and a method of extracting a part having a physical shape such as eyes and nose from an image region by template matching are known. Other methods include detecting image feature quantities such as skin color and eye shape and analyzing them using statistical methods (for example, Japanese Patent Laid-Open Nos. 10-232934 and 2000-48184). reference). In addition, what is currently proposed as a product includes a method of detecting a face using wavelet transform and image feature amounts, a method of combining template matching, and the like.

  Here, template matching, which is one of pattern recognition, will be described as a face detection method. Pattern recognition is a process of matching an observed pattern with one of predetermined concepts (classes). FIG. 2 is a flowchart of the pattern recognition process, which is executed by the face detection unit 114.

  First, image data is read from the memory 102 and preprocessed (step S1), and a pattern of a characteristic portion is extracted from the preprocessed image data (step S2). Then, the extracted pattern is made to correspond to the template (standard pattern) (template matching). For example, when the characteristic part pattern 63 is extracted as shown in FIG. 3, the center point 62 of the template 61 is placed at a certain coordinate point (i, j) of the acquired pattern 63. Then, while shifting the position of the center point 62 in the pattern 63, the similarity of the overlapping portion between the template 61 and the pattern 63 is calculated, and the position where the similarity is maximized is determined. By matching the pattern 63 with, for example, a template 61 including shapes such as eyes and ears, position information of eyes and face areas (face coordinates) can be acquired.

  Thus, a recognition pattern is acquired (step S3), the acquired recognition pattern is output (step S4), and a pattern recognition process is complete | finished.

  Returning to the description of FIG. The CPU 115 calculates the shutter speed Tv and the aperture value Av so that the face becomes optimal brightness if the face is detected based on the signal sent from the face detection unit 114, and focuses on the face. The focus lens drive amount is calculated as follows. Further, it is also determined whether the flash 116 is emitting light or not. On the other hand, if the face is not detected, the CPU 115 calculates the shutter speed Tv and the aperture value Av so that the entire image has the optimum brightness, and focuses on the subject in the preset focus area. The focus lens driving amount is calculated. The exposure value (Tv, Av) calculated by the CPU 115, the focus lens drive amount, and the determination result of light emission / non-light emission of the flash 116 are sent to the control circuit 113. Based on these values, a lens, a diaphragm, a shutter, an image sensor 101, and a flash 116 (not shown) are controlled.

  A white balance (WB) control unit 103 calculates a WB correction value based on the image signal stored in the memory 102 and face information obtained from the face detection unit 114, and uses the calculated WB correction value to store the memory. WB correction is performed on the image signal stored in 102. The WB correction value calculation method used by the WB control unit 103 will be described in detail later.

  Reference numeral 104 denotes a color conversion matrix (MTX) circuit that converts the color difference signals RY and BY into color difference signals so that the image signal corrected by the WB control unit 103 is WB corrected with an optimal color. . Reference numeral 105 denotes a low-pass filter (LPF) circuit that limits the bands of the color difference signals RY and BY. Reference numeral 106 denotes a CSUP (Chroma) that suppresses a false color signal in a saturated portion of the image signal band-limited by the LPF circuit 105. Supress) circuit.

  On the other hand, the WB corrected image signal by the WB control unit 103 is also output to the luminance signal (Y) generation circuit 111 to generate the luminance signal Y, and the edge enhancement circuit 112 performs edge processing on the generated luminance signal Y. Emphasis processing is performed.

  The color difference signals RY and BY output from the CSUP circuit 106 and the luminance signal Y output from the edge emphasis circuit 112 are converted into RGB signals by the RGB conversion circuit 107, and the gamma correction circuit 108 converts the gradation signals Y to B and Y. Tone correction is applied. After that, it is converted into a YUV signal by the color luminance conversion circuit 109, and further compressed by, for example, JPEG by the compression circuit 110 and recorded as an image signal on an external recording medium or an internal recording medium.

  Next, a method for calculating the WB correction value in the present embodiment will be described with reference to FIG.

First, the WB control unit 103 reads an image signal stored in the memory 102, and divides the screen into arbitrary m blocks (divided areas) as shown in FIG. 5 (step S101). Then, for each block (1 to m), the pixel values are added and averaged for each color to calculate a color average value (R [i], G [i], B [i]). The color evaluation values (Cx [i], Cy [i]) are calculated by using them (step S102).
Cx [i] = (R [i]-B [i]) / Y [i] x 1024
Cy [i] = (R [i] + B [i])-2G [i] / Y [i] x 1024 (1)
Where Y [i] = R [i] + 2G [i] + B [i], [i] is the index number of each block

  Next, it is determined whether or not the color evaluation values (Cx [i], Cy [i]) of the i-th block calculated in step S102 are included in the preset white detection range 301 shown in FIG. S103). The white detection range 301 is obtained by photographing white under different light sources in advance and plotting the calculated color evaluation values. In FIG. 6, the negative direction of the x coordinate (Cx) is a color evaluation value when white of a high color temperature subject is photographed, and the positive direction is a color evaluation value when white of a low color temperature subject is photographed. The y coordinate (Cy) means the degree of the green component of the light source, and the G component increases as it goes in the negative direction, that is, the light source is a fluorescent lamp.

  The area A in FIG. 6 is a white distribution during external light (Daylight), the area B is a low color temperature light source distribution such as tungsten, and the area C is a distribution of fluorescent lamps. When the calculated color evaluation values (Cx [i], Cy [i]) are included in the white detection range 301 (YES in step S103), it is determined that the block is white. At this time, illumination can be estimated depending on which area each block is included in. Therefore, the WB control unit 103 constitutes an estimation unit. Then, the color average values (R [i], G [i], B [i]) of the block are accumulated (step S104), and if not included, the process proceeds to step S105 without adding. The processing of step S103 and step S104 can be expressed by equation (2).

Here, in Equation (2), Sw [i] is set to 1 when the color evaluation value (Cx [i], Cy [i]) is included in the white detection range 301, and Sw [i is set when it is not included. ] Is set to 0. In this way, the process of whether or not to add the color average values (R [i], G [i], B [i]) is substantially performed according to the determination in step S103.

  In step S105, it is determined whether or not the above process has been performed for all blocks. If there is an unprocessed block, the process returns to step S102 and the above process is repeated. If all the blocks have been processed, the process proceeds to step S106. .

In step S106, WB correction values (WBCo1_R, WBCo1_G, WBCo1_B) are calculated from the integrated values (sumR, sumG, sumB) of the obtained color evaluation values using the following equation (3).
WBCo1_R = sumY × 1024 / sumR
WBCo1_G = sumY × 1024 / sumG (3)
WBCo1_B = sumY × 1024 / sumB
However, sumY = (sumR + 2 × sumG + sumB) / 4

  Next, the light emission control processing of the flash 116 in the present embodiment will be described with reference to the flowchart of FIG. The processing shown in FIG. 7 is performed by the user operating a shutter button (not shown) when the exposure of the entire image is controlled so as to be appropriate when the electronic viewfinder (EVF) image before photographing is acquired. This is executed under the control of the CPU 115 in accordance with the photographing instruction. That is, the CPU 115 functions as a determination unit, a determination unit, and a control unit.

  First, the image signal stored in the memory 102 is read in step S201, and in step S202, it is determined whether or not a face is detected by the face detection unit 114. If no face is detected, the process proceeds to step S208 and normal shooting is performed.

  On the other hand, when a face is detected, the face detection unit 114 extracts the image signal of the face area from the image signal read out from the memory 102 and sends it to the CPU 115. In the CPU 115, the luminance value of the sent image signal is extracted. Is calculated for each block shown in FIG. The luminance value of the face area is extracted from the calculated luminance signal Y (step S203), and it is determined whether or not the dark portion ratio in the face area exceeds a threshold value (step S204). The determination performed here will be described below.

  In the present embodiment, the face area detected by the face detection unit 114 is calculated for each block shown in FIG. 5, and it is determined whether the calculated brightness value is underexposed. Detect dark areas of. In the backlit state, when the entire screen is properly exposed, the entire face area is underexposed as shown in FIG. 8A, and a shadow is generated on a part of the face due to the top light phenomenon. As shown in (b), a part of the face area is underexposed. In the case of backlighting, the entire face area becomes dark, so it can be easily determined that there is a possibility of backlighting. In the case of top light, the determination is based on the proportion of the dark portion in the face area. That is, when a block having a face area is determined to be a dark part and the ratio of blocks determined to be a dark part in the face area is, for example, 20% or more and 80% or less, there is a possibility that the toplight phenomenon has occurred. It can be judged that there is. The ratio of the dark part block for judging the top light phenomenon may not be 20% or more and 80% or less, and may be set according to the size and composition of the face or a suitable value obtained by experiment. good.

  If the ratio of the dark part does not exceed the threshold value in step S204, the process proceeds to step S208 to perform normal shooting, and if it exceeds, the process proceeds to step S205.

  In step S205, the WB control unit 103 calculates a WB correction value. As mentioned earlier, in normal shooting where people shoot outdoors during the day, the brightness of the entire screen is not high enough to determine that flash shooting is necessary. If you are, your face will be shot under. The result of white balance calculation in the WB control unit 103 is applied as a criterion for determining the backlight and toplight phenomenon. As described above, when detecting a white block, a light source under photographing can be predicted. For example, when a large number of blocks determined to be “white spots” are distributed in the area A in FIG. 6, shooting is performed under daylight such as the sun in the daytime. Judge. Therefore, when the WB control unit 103 determines that the daylight is daylight (Daylight) as a result of the white balance calculation (YES in Step S206), it is determined that the backlight or the toplight phenomenon is occurring. In step S207, flash photography using the flash 116 is performed. If it is not determined that the daylight is during the day, the process proceeds to step S208 to perform normal shooting.

  As described above, according to the present embodiment, the backlight and the toplight phenomenon are discriminated according to whether the ratio of the dark portion in the face area and the white balance calculation result is external light (Daylight), and the flash is turned on. Control light emission. As a result, an image in which both the person and the background are properly exposed can be obtained even when the backlight and the toplight phenomenon occur outdoors in the daytime.

  In the above-described embodiment, the case where the white balance calculation result is used to estimate the light source has been described. However, the present invention is not limited to this, and the light source may be estimated in any manner. For example, a configuration for estimating the light source may be added.

<Modification>
In the above example, in the process in step S204, the backlight or the toplight phenomenon is determined according to the proportion of the dark area in the face area. For example, the following determination conditions may be used.

  First, a block including an eye may be determined from the face detection result, and if the block is a dark part, it may be determined that a toplight phenomenon may occur. This is because when the light source is above the head, the eyes become shadows due to the top light phenomenon.

  Further, a block indicating eyes and mouth may be determined from the face detection result, and it may be determined that the toplight phenomenon may occur when the luminance difference between these blocks is equal to or greater than a threshold value.

Claims (3)

An imaging device capable of flash photography using a flash,
Face detection means for detecting a face region from an image of an image signal obtained from an image sensor;
Control means for determining whether to perform flash photography based on the luminance in the face area detected by the face detection means ;
Estimating means for estimating a light source from the image signal ,
The control means determines to perform the flash shooting even if a part of the face area is bright, and as a condition to determine to perform the flash shooting, among the divided areas into which the face area is divided, The brightness of the divided area including the eyes indicates underexposure , and the brightness difference between two preset areas among the divided areas is at least a threshold value, and the An imaging apparatus comprising: the estimated light source being a preset light source . The imaging apparatus according to claim 1 , wherein the preset light source is daylight outside light. A flash control method in an imaging apparatus capable of performing flash photography using a flash,
A face detection step in which the face detection means detects a face region from the image of the image signal obtained from the image sensor;
A control step in which the control means determines whether to perform flash photography based on the luminance in the face area detected in the face detection step;
An estimating means including an estimating step of estimating a light source from the image signal ,
The control step is to determine to perform the flash shooting even if a part of the face area is bright, and as a condition to determine to perform the flash shooting, among the divided areas into which the face area is divided, The luminance of the divided region including the eyes indicates under-exposure ; and the luminance difference between two predetermined regions of the divided regions is at least a threshold value; and A flash control method characterized in that the estimated light source includes a preset light source .