JP2009239792A - Imaging device, image processing device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device that stably performs a similar brightness correction to a plurality of images obtained by photographing a scene in which a plurality of object persons whose brightness are different from each other exist in relatively proximity to each other, and to provide a technique related to this. <P>SOLUTION: The imaging device extracts a person's face region from each subject area corresponding to each of a plurality of AF areas (step SP11), calculates luminance of the extracted each face region (step SP13) and obtains distance information on the object in the AF area corresponding to each face region (step SP14). Further, the imaging device sets a weighting coefficient of each face region according to the distance information of the AF area corresponding to each face region (step SP15) and corrects brightness of a taken image based on the luminance in each face region and the weighting coefficient of each face region (steps SP19 and SP20). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital camera and a technology related thereto.

  In recent years, with the widespread use of digital cameras and the like, there is a technique for detecting the face area of a person from a shot image and adjusting the brightness of the shot image so that the face area has an appropriate brightness.

  For example, in Patent Document 1, when a face detection area is determined, an AF area having a shorter subject distance among a plurality of AF areas is set to have a higher priority. Then, face detection is performed based on an AF area having a high priority. Further, gradation correction processing is performed according to the luminance distribution of the face area image corresponding to the selected AF area.

JP 2005-215750 A

  However, in the case of shooting a plurality of images regarding a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance from each other, if brightness correction is performed using the above technique, a slight variation in distance information occurs. Accordingly, the brightness correction result may fluctuate between a plurality of images. Specifically, it is determined that the perspective relationship between a plurality of subjects has fluctuated due to the fact that one of the subjects has actually moved slightly or there is an AF distance measurement error. Accordingly, the brightness correction result varies among a plurality of images.

  For example, as shown in FIG. 5, a shooting scene is assumed in which a relatively bright right person in direct sunlight and a relatively dark left person in the shadow of a parasol exist. In this case, in the first image in which the right person is determined to be the closest subject, the overall brightness is relatively suppressed in order to adjust the relatively bright person to the appropriate brightness. On the other hand, in the second image, it is assumed that the left person is determined to be the closest subject due to a distance measurement accuracy error or a minute movement of the person. In the second image, the overall luminance is relatively increased in order to adjust a relatively dark person to an appropriate luminance.

  In this way, different brightness corrections are applied to the first image and the second image. As a result, the brightness varies among a plurality of images in spite of the same scene.

  However, it is preferable that such variations are suppressed. In particular, it is preferable that the same brightness correction is performed between a plurality of consecutive captured images in continuous shooting or the like.

  Therefore, an imaging apparatus capable of stably performing the same brightness correction on a plurality of images capturing a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance, and the related apparatus The issue is to provide technology.

  A first aspect of the present invention is an imaging device or an image processing device, which extracts a human face area from each target area corresponding to each of a plurality of AF areas, and is extracted by the extraction unit Luminance calculation means for calculating the brightness of each face area, distance information acquisition means for obtaining distance information about the subject in the AF area corresponding to each face area, and the distance information for the AF area corresponding to each face area And a correction means for setting a weighting coefficient for each face area in accordance with the brightness of the photographed image based on the luminance of each face area and the weighting coefficient for each face area.

  According to a second aspect of the present invention, there is provided a program, in which a) a person's face area is extracted from each target area corresponding to each of a plurality of AF areas, and the brightness of each extracted face area is set. A procedure for calculating, b) a procedure for obtaining distance information about the subject in the AF area corresponding to each face area, and c) a step for each face area according to the distance information for the AF area corresponding to each face area. A procedure for setting a weighting coefficient and correcting the brightness of the photographed image based on the luminance of each face area and the weighting coefficient of each face area is executed.

  According to the present invention, it is possible to stably perform the same brightness correction on a plurality of images capturing a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. Outline of configuration>
1 and 2 are diagrams showing an external configuration of the imaging apparatus 1 according to the first embodiment of the present invention. Here, FIG. 1 is a front external view of the image pickup apparatus 1, and FIG. 2 is a rear external view of the image pickup apparatus 1. This imaging device 1 is configured as a lens interchangeable single-lens reflex digital camera. The imaging device 1 is also expressed as an image processing device that performs image processing as described below.

  As shown in FIG. 1, the imaging device 1 includes a camera body (camera body) 2. An interchangeable photographic lens unit (interchangeable lens) 3 can be attached to and detached from the camera body 2.

  The photographic lens unit 3 mainly includes a lens barrel 36, a lens group 37 (see FIG. 3) provided in the lens barrel 36, a diaphragm, and the like. The lens group 37 (shooting optical system) includes a focus lens that changes the focal position by moving in the optical axis direction.

  The camera body 2 includes an annular mount Mt to which the photographing lens unit 3 is attached at the front center, and an attach / detach button 89 for attaching / detaching the photographing lens unit 3 near the annular mount Mt. Yes.

  Further, the camera body 2 is provided with a mode setting dial 82 in the upper left part of the front surface and a control value setting dial 86 in the upper right part of the front surface. By operating the mode setting dial 82, various camera modes (such as various shooting modes (portrait shooting mode, landscape shooting mode, full-auto shooting mode, etc.), playback mode for playing back captured images, (Including a communication mode in which data communication is performed) is performed (switching operation). Further, by operating the control value setting dial 86, it is possible to set control values in various shooting modes.

  Further, the camera body 2 includes a grip portion 14 for a photographer to hold at the left end of the front. A release button 11 for instructing the start of exposure is provided on the upper surface of the grip portion 14. A battery storage chamber and a card storage chamber are provided inside the grip portion 14. For example, a lithium ion battery is housed in the battery compartment as a power source for the camera, and a memory card 90 (see FIG. 3) for recording image data of a photographed image is detachably housed in the card compartment. .

  The release button 11 is a two-stage detection button that can detect two states, a half-pressed state (S1 state) and a fully-pressed state (S2 state). When the release button 11 is half-pressed to enter the S1 state, a preparation operation (for example, an AF control operation and an AE control operation) for acquiring a recording still image (main captured image) related to the subject is performed. When the release button 11 is further pushed into the S2 state, an exposure operation related to a subject image (light image of the subject is performed using the imaging device 5 (described later) using the imaging element 5 (described later), and the exposure operation is performed. A series of operations for applying predetermined image processing to the obtained image signal is performed.

  In FIG. 2, a finder window (eyepiece window) 10 is provided at the upper center of the back surface of the camera body 2. The photographer can determine the composition by viewing the viewfinder window 10 and visually recognizing the light image of the subject guided from the photographing lens unit 3. That is, it is possible to determine the composition using an optical finder.

  In FIG. 2, a rear monitor 12 is provided in the approximate center of the rear surface of the camera body 2. The rear monitor 12 is configured as a color liquid crystal display (LCD), for example. The rear monitor 12 can display a menu screen for setting shooting conditions and the like, and can reproduce and display the captured image recorded in the memory card 90 in the reproduction mode.

  A main switch 81 is provided at the upper left of the rear monitor 12. The main switch 81 is a two-point slide switch. When the contact is set to the left “OFF” position, the power is turned off. When the contact is set to the right “ON” position, the power is turned on.

  A direction selection key 84 is provided on the right side of the rear monitor 12. This direction selection key 84 has a circular operation button so that it can detect a pressing operation in four directions of upper, lower, left and right, and a pressing operation in four directions of upper right, upper left, lower right and lower left. It is configured. In addition, the direction selection key 84 can also detect the push operation of the push button in the center, in addition to the push operation in the eight directions.

  A setting button group 83 including a plurality of buttons for setting a menu screen, deleting an image, and the like is provided on the left side of the rear monitor 12.

<1-2. Functional block>
Next, an overview of functions of the imaging apparatus 1 will be described with reference to FIG. FIG. 3 is a block diagram illustrating a functional configuration of the imaging apparatus 1.

  As shown in FIG. 3, the imaging apparatus 1 includes an operation unit 80, an overall control unit 101, a focus control unit 121, a mirror control unit 122, a shutter control unit 123, a timing control circuit 124, a digital signal processing circuit 50, and the like. .

  The operation unit 80 includes various buttons and switches including the release button 11 (see FIG. 1). In response to a user input operation on the operation unit 80, the overall control unit 101 implements various operations.

  The overall control unit 101 is configured as a microcomputer and mainly includes a CPU, a memory, a ROM, and the like. The overall control unit 101 implements various functions by reading a program stored in a ROM (such as an EEPROM) and executing the program on the CPU.

  The overall control unit 101 realizes each processing unit including a distance information acquisition unit 21, a face area extraction unit 22, a luminance calculation unit 23, and a correction control unit 24.

  The distance information acquisition unit 21 relates to each subject in a plurality of AF areas (focus areas) FRi (see FIG. 6) based on information acquired by the AF module 20, the lens position detection unit 39, and the focus control unit 121. Each distance information is obtained.

  The face area extraction unit 22 extracts a human face area HFi (see FIG. 9) from each target area QRi (see FIG. 6) corresponding to each of the plurality of AF areas FRi.

  The luminance calculation unit 23 calculates the luminance of each of the plurality of face regions HFi extracted by the face region extraction unit 22.

  The correction control unit 24 performs gradation conversion of the captured image in cooperation with the γ correction circuit 55 and the like. The correction control unit 24 sets the weighting coefficient Wi of each face area HFi according to the distance information of each AF area FRi corresponding to the plurality of face areas HFi, and determines the brightness of each face area HFi and each face area HFi. The brightness of the captured image is corrected based on the weighting coefficient Wi.

  The overall control unit 101 uses these processing units to implement a brightness adjustment operation (described later) for a captured image.

  The overall control unit 101 performs a focus control operation for controlling the position of the focus lens in cooperation with the AF module 20, the focus control unit 121, and the like. The overall control unit 101 implements an AF operation using the focus control unit 121 according to the focus state of the subject detected by the AF module 20. The AF module 20 can detect the in-focus state of the subject by using the light that has entered through the mirror mechanism 6 by a focus state detection method such as a phase difference method.

  The focus control unit 121 moves a focus lens included in the lens group 37 of the photographing lens unit 3 by generating a control signal based on a signal input from the overall control unit 101 and driving the motor M1. The position of the focus lens is detected by the lens position detection unit 39 of the photographing lens unit 3, and data indicating the position of the focus lens is sent to the overall control unit 101. As described above, the focus control unit 121, the overall control unit 101, and the like control the movement of the focus lens in the optical axis direction.

  The mirror control unit 122 controls state switching between a state in which the mirror mechanism 6 is retracted from the optical path (mirror up state) and a state in which the mirror mechanism 6 blocks the optical path (mirror down state). The mirror control unit 122 switches between the mirror up state and the mirror down state by generating a control signal based on the signal input from the overall control unit 101 and driving the motor M2.

  The shutter control unit 123 controls the opening and closing of the shutter 4 by generating a control signal based on the signal input from the overall control unit 101 and driving the motor M3.

  The timing control circuit 124 performs timing control for the image sensor 5 and the like.

  An imaging device (here, a CCD sensor (also simply referred to as a CCD)) 5 converts an optical image of a subject into an electrical signal by a photoelectric conversion action, and generates an image signal (recording image signal) related to the actual captured image. To do. The image sensor 5 is also expressed as an image sensor for acquiring a recorded image.

  In response to the drive control signals (accumulation start signal and accumulation end signal) input from the timing control circuit 124, the image sensor 5 performs exposure (charge accumulation by photoelectric conversion) of the subject image formed on the light receiving surface. Then, an image signal related to the subject image is generated. Further, the image sensor 5 outputs the image signal to the signal processing unit 51 in response to the read control signal input from the timing control circuit 124. The timing signal (synchronization signal) from the timing control circuit 124 is also input to the signal processing unit 51 and the A / D (analog / digital) conversion circuit 52.

  The image signal acquired by the image sensor 5 is subjected to predetermined analog signal processing in the signal processing unit 51, and the image signal after the analog signal processing is converted into digital image data (image data) by the A / D conversion circuit 52. Is done. This image data is input to the digital signal processing circuit 50.

  The digital signal processing circuit 50 performs digital signal processing on the image data input from the A / D conversion circuit 52 to generate image data related to the captured image. The digital signal processing circuit 50 includes a black level correction circuit 53, a white balance (WB) circuit 54, a γ correction circuit 55, and an image memory 56.

  The black level correction circuit 53 corrects the black level of each pixel data constituting the image data output from the A / D conversion circuit 52 to a reference black level. The WB circuit 54 performs white balance adjustment of the image. The γ correction circuit 55 performs gradation conversion of the captured image. The image memory 56 is a high-speed accessible image memory for temporarily storing generated image data, and has a capacity capable of storing image data for a plurality of frames.

  At the time of actual photographing, the image data temporarily stored in the image memory 56 is appropriately subjected to image processing (including compression processing) in the overall control unit 101 and then stored in the memory card 90 via the card I / F 132. The

  Further, the image data temporarily stored in the image memory 56 is appropriately transferred to the VRAM 131 by the overall control unit 101, and an image based on the image data is displayed on the rear monitor 12. Thereby, confirmation display (after view) for confirming the captured image, reproduction display for reproducing the captured image, and the like are realized.

<1-3. Shooting action>
The operation of the imaging apparatus 1 will be described with reference to FIG. FIG. 4 is a flowchart illustrating an operation immediately after the imaging operation of the imaging apparatus 1.

  When the release button 11 is pressed down to the fully-pressed state S2 by the operator and a shooting command is given, the image pickup apparatus 1 opens the shutter 4 for a predetermined exposure time and uses the image pickup device 5 for image data relating to the subject. Get. Then, the imaging apparatus 1 performs a brightness (brightness) adjustment process (γ correction) and the like as described below on the image data. As a result, the image data subjected to the brightness adjustment operation or the like is generated as a main photographic image and recorded in the memory card 90. Hereinafter, with respect to the operation from when the exposure image (image data) acquired by the image sensor 5 is temporarily read to the image memory 56 until it is recorded on the memory card 90, the luminance adjustment operation (image processing operation). The operation related to the above will be mainly described.

  Here, a shooting scene as shown in FIG. 5 is assumed. In this shooting scene, the person on the right has a relatively bright luminance state under direct sunlight, while the person on the left has a relatively dark luminance state with the shadow of a parasol. Also, the left person and the right person are at a relatively close distance from each other. That is, in this shooting scene, a plurality of subject persons having different brightness exist at a relatively close distance.

  In this embodiment, by performing the following operations, in a situation where a plurality of photographed images relating to the photographing scene as shown in FIG. 5 are continuously photographed (for example, continuous shooting), the plurality of photographed images are processed. Therefore, the same brightness correction can be stably performed.

  In step SP11, the face area HFi of the person is extracted in association with each AF area (focus area) FRi (here, four AF areas FR1 to FR4 (see FIG. 6)) provided in the AF module 20. Processing is performed. Note that the processing of steps SP11 to SP18 is sequentially performed for a plurality of AF areas FRi. Specifically, first, among the plurality of AF areas FRi, the first AF area (for example, FR1) is selected as the AF area to be processed.

  The face area extraction process in step SP11 is executed by setting the target area QRi (that is, the AF area FRi and its surrounding area) obtained by enlarging each AF area FRi as an extraction target area (also referred to as a search target area).

  Each target area QRi is an area having a predetermined size centered on each corresponding AF area FRi. For example, as shown in FIG. 6, the target area QR1 corresponds to the AF area FR1, and the target area QR2 corresponds to the AF area FR2. The target area QR3 corresponds to the AF areas FR3 and FR4.

  As each target area QR (QR1 to QR4), among the plurality of blocks (partition areas) Bj shown in FIG. The area corresponding to the block is selected. FIG. 7 is a diagram illustrating a plurality of blocks Bj in the image G1. The screen G1 is divided into 40 pieces in the horizontal direction and 30 pieces in the vertical direction. As a result, the image G1 is divided into a total of 1200 small regions (blocks) Bj.

  Next, for each target area QR corresponding to the selected AF area, a human face area extraction process is executed. Specifically, the face area is extracted based on the skin color information of each block Bj. More specifically, the skin color block in each target area QR is regarded as a face area (person area) and extracted. Whether each block Bj is a flesh color block (a block whose color is classified into flesh color) is determined as follows.

  Specifically, first, the pixel value of the RGB component of each pixel in the block Bj is converted into a luminance value Y and color components Cr and Cb.

  Then, the color components (Cr, Cb) after the conversion of each pixel in the block Bj are, as shown in FIG. 8, a predetermined range EA (skin color area or color) representing the skin color in the Cr-Cb space (color space). The pixel is determined to be a skin color pixel.

  Further, in the block Bj, when such skin color pixels exceed a predetermined ratio (for example, 80%), it is determined that the color of the block Bj is a skin color, and the block Bj is a “skin color block”. It is determined. Note that the present invention is not limited to this, and the color of the block Bj may be determined by the color of one or more representative pixels in the block Bj. The extracted skin color block Bj is also expressed as a block to which the color belongs to the skin color space EA in the color space.

  Here, the extracted skin color block Bj is regarded as an area belonging to the person's face area (person area), and the aggregate area of the skin color blocks Bj is extracted as the person's face area HF. In other words, the skin color block Bj (skin color region) in each target area QR is regarded as the face region HF and extracted.

  FIG. 9 is a diagram showing the extracted face area HFi (HF1, HF2). In FIG. 9, for example, the face area HF1 is extracted from the target area QR1. Further, the face area HF2 is extracted from the target area QR2.

  As described above, when the human face area HFi is extracted from the target area QRi (for example, QR1) corresponding to a certain AF area FRi (for example, FR1), the process proceeds from step SP12 to step SP13.

  In step SP13, the brightness YSi of the extracted face area HFi is calculated. Specifically, the average brightness (skin color average brightness) of the plurality of skin color blocks Bj included in the face area HFi is calculated as the brightness YSi of the face area HFi.

  In step SP14, distance information about the subject in a certain AF area FRi is acquired. In other words, distance information regarding the subject in the AF area FRi corresponding to each face region HGi is obtained.

  Here, it is assumed that a defocus amount ΔDi (described below) in each AF area FRi is acquired as the distance information. Specifically, a virtual lens position (of the focus lens) (also referred to as a focus lens position) for bringing the subject in each AF area FRi into focus is determined, and the focus lens position of each AF area FRi and the actual focus lens position are actually determined. Is calculated as a defocus amount ΔDi of each AF area FRi. The defocus amount of the in-focus AF area is 0 (zero).

  Note that the defocus amount ΔDi can be converted into the distance L from the imaging device 1 to the subject based on a data table or the like representing the relationship between the lens position x and the subject distance L. Specifically, the subject distance L corresponding to the lens position obtained by adding (or subtracting) the defocus amount ΔDi from the actual lens position can be calculated based on the data table.

  In the next step SP15, the weight of each face area HF is calculated.

  Specifically, the weighting coefficient Wi of each AF area FRi is calculated based on, for example, the equation (1).

  However, when ΔDi> Dmax, Wi = 0.

  Here, the value ΔDi is the defocus amount of the AF area FRi, and the value Dmax is a predetermined value.

  FIG. 10 is a graph showing the relationship of Equation (1). As shown in FIG. 10, the weighting coefficient Wi gradually decreases as the defocus amount ΔDi increases. When the defocus amount ΔDi is equal to or greater than the value Dmax, the weighting coefficient Wi is 0 (zero).

  For example, when the value Dmax is determined to be 2000 (micrometers), the defocus amount ΔD1 of the AF area FR1 is 0 (zero), and the defocus amount ΔD2 of the AF area FR2 is 100 (micrometers). . At this time, the weighting coefficient W1 of the AF area FR1 is calculated as 1.0 based on the formula (1) (W1 = 1.0), and the weighting coefficient W2 of the AF area FR2 is 0.95. (W2 = 0.95).

  Further, the predetermined value Dmax may be determined as follows, for example.

  In general, a subject that exists at a large distance from the subject in the focusing AF area is often not the main subject. And it is preferable that the luminance information regarding the non-main subject is not reflected in the brightness correction.

  Therefore, here, the value Dmax is determined as the maximum value of the defocus amount related to the subject to be reflected in the brightness correction. In other words, luminance information relating to a subject that is farther than the distance Xmax from the focused subject is not reflected in the brightness correction.

  Specifically, the value Dmax may be determined based on Equation (2) using the distance Xmax and the subject magnification (subject magnification) β.

  The equation (2) is calculated based on the equation (3) indicating the relationship among the defocus amount ΔDi, the subject magnification β, and the distance X from the focused subject (see FIG. 11). The subject magnification β is expressed as a ratio (β = f / L) between the focal length f of the imaging device 1 and the distance L from the imaging device 1 to the focused subject.

  For example, if Xmax = 5 (meters) and β = 1/50, the value Dmax is calculated based on the equation (2) as follows: Dmax = 5 × (1/50) × (1/50) = 0.002 ( Meter) = 2000 (micrometers).

  Thereby, for example, when the subject in the AF areas FR3 and FR4 exists at a distance (for example, infinity) farther than the value Xmax, a face area regarding the target areas QR3 and QR4 corresponding to the AF areas FR3 and FR4 temporarily exists. Even then, the brightness of the face area does not affect the brightness correction process.

  On the other hand, if no flesh color block is detected in step SP11, the process proceeds from step SP12 to step SP16.

  In step SP16, unlike step SP13, the average luminance (skin color average luminance) YSi of the face area is set to 0 (zero). In step SP17, unlike step SP15, the weighting coefficient Wi is set to 0 (zero).

  When the operation as described above (steps SP11 to SP17) is finished for a certain AF area FRi (for example, FR1), the process proceeds to step SP18.

  In step SP18, it is determined whether or not the above operation (steps SP11 to SP17) has been completed for all AF areas. When an AF area where the above operation has not been performed still remains, the above-described processing is similarly performed with respect to another AF area FRi (for example, FR2). When the above operation is completed for all the AF areas, the process proceeds to step SP19 in a state where the luminance YSi of the plurality of face areas HFi and the weighting coefficient Wi of the plurality of face areas HFi are calculated.

  In step SP19, a value YS obtained by weighting and adding the luminance YSi of each face area HFi according to the weighting coefficient Wi of each face area HFi is calculated. This value YS is used as the reference luminance in the brightness correction in the next step SP20. By using such a value YS as the reference brightness, not only the brightness of the face area around the focused AF area, but also other AF areas (especially AF whose defocus amount ΔDi is relatively small). Area) Brightness correction can be performed by appropriately reflecting the brightness of the surrounding face area.

  The reference luminance YS is determined based on the equation (4) using, for example, the luminance YSi acquired in step SP13 and the weighting coefficient Wi acquired in step SP15.

  In step SP20, the brightness of the captured image G1 is corrected based on the reference luminance YS.

  Specifically, the brightness of the captured image G1 is corrected so that the reference luminance YS becomes an appropriate luminance. More specifically, a γ (gamma) correction curve that brings the reference luminance YS closer to the target value YT is adopted, and γ correction based on the γ correction curve is performed.

  FIG. 12 is a diagram illustrating a plurality of γ correction curves having different correction amounts GC. “+30”, “+50”, and “+70” are curves that increase the luminance (brightness) of the entire image, and “−30”, “−50”, and “−70” are the luminance (brightness) of the entire image. It is a curve that decreases. In addition, as the numerical value after the positive / negative sign increases, the degree of correction by the curve increases.

  For example, in order to set the reference luminance YS (value 70) to the target value YT (value 120), a γ correction curve (GC = + 50) that increases the average value by about 50 (“+50”) is adopted, and the γ correction is performed. Γ correction based on the curve is performed.

  In this embodiment, the brightness of the captured image is corrected based on the reference luminance YS as described above.

  Here, in the case where a plurality of images are continuously photographed with respect to the photographing scene of FIG. 5 (for example, continuous shooting), the right side of the first image is caused by a distance measurement accuracy error or a minute movement of a person. It is assumed that the person is determined to be the closest subject, and the left person is determined to be the closest subject in the second image.

  In addition, as a comparative example, a technique for simply adjusting the brightness of an image based on the luminance of a face area corresponding to a person who is closest to the shortest distance is assumed.

  In such a comparative example, in the first image, the overall brightness is relatively suppressed in order to adjust a relatively bright person to an appropriate brightness, while in the second image, a relatively dark person In order to adjust the brightness to the proper brightness, the overall brightness is relatively increased. For this reason, different brightness corrections are applied to the first image and the second image. That is, the brightness varies among a plurality of images despite the same scene.

  On the other hand, according to the above embodiment, the brightness YSi of the plurality of face regions is not adjusted based on only the brightness of the face region corresponding to the person at the shortest distance. The brightness of the image G1 is corrected based on the luminance YS reflected based on the weighting coefficient Wi.

  For example, even if it is determined that the left person is the closest subject in the second image, the defocus amount ΔDi (ΔD2) regarding the right person in the second image is the left side in the first image. Is approximately the same as the defocus amount ΔDi (ΔD1) for the person. Therefore, the same reference luminance YS is calculated and the same brightness correction is performed on both the first photographed image and the second photographed image.

  That is, even in the case where a plurality of images are continuously shot (for example, continuous shooting, etc.) regarding a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance, It is possible to stably perform the same brightness correction while suppressing variations in brightness.

<2. Second Embodiment>
The second embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, a value obtained by weighting and adding the luminance YSi of each face area HFi according to the weighting coefficient Wi of each face area HFi is calculated as the reference luminance YS, and the brightness of the captured image is calculated based on the reference luminance YS. The case where the thickness is corrected is illustrated.

  On the other hand, in the second embodiment, each correction amount GCi for calculating the brightness YSi of each face area HFi close to the target brightness YT is first calculated. Then, a correction correction amount GM is calculated by adding the correction amount GCi of each face region HFi according to the weighting coefficient Wi of each face region HFi, and the brightness of the captured image is corrected using the correction correction amount GM.

  FIG. 13 is a flowchart showing an operation according to the second embodiment. As shown in FIG. 13, the processing of steps SP31, SP35, and SP39 is different from that of the first embodiment, but the rest is the same as that of the first embodiment.

  Of these, step SP31 is executed next to step SP13.

  In step SP31, based on the brightness (skin color average brightness) YSi calculated in step SP13 for a certain face area HFi, a brightness correction correction amount GCi for the face area HFi is determined. Specifically, each correction amount GCi for calculating the brightness YSi of the face area HFi to be close to the target brightness YT is calculated. For example, when the target value YT is the value 120 and the luminance YSi is the value 90, the value +30 is determined as the correction amount GCi. Alternatively, when the luminance YSi is a value 170, the value −50 is determined as the correction amount GCi.

  In step SP35, the correction amount GCi is set to 0 (zero). That is, the correction amount GCi is set to 0 when there is no skin color area in the target area QR.

  In step SP39, the correction correction amount GM is calculated. The correction correction amount GM is calculated as a value obtained by adding the correction amounts GCi of the plurality of face regions HFi in accordance with the weightings of the plurality of face regions HFi. Specifically, the correction amount GCi acquired at step SP31 and the weighting coefficient Wi acquired at step SP15 are used to determine the correction based on the equation (5).

  Then, the brightness of the captured image G1 is corrected using the correction correction amount GM. Specifically, a γ (gamma) correction curve corresponding to the correction correction amount GM is adopted, and γ correction based on the γ correction curve is performed.

  In the second embodiment, the brightness of the captured image is corrected based on the correction correction amount GM as described above. Even in such an aspect, similarly to the first embodiment, the same brightness correction is stably performed on a plurality of images that shoot a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance. It can be applied automatically.

<3. Third Embodiment>
In the said 1st Embodiment and 2nd Embodiment, although the case where the face area | region HF was extracted based on skin color information was illustrated, it is not limited to this. For example, the human face area HF may be extracted based on the contour information in the image. In the third embodiment, such a modification will be described. Below, it demonstrates centering on difference with 1st Embodiment.

  FIG. 14 is a flowchart showing an operation according to the third embodiment. As shown in FIG. 14, the processing of steps SP41, SP42, SP43, and SP46 is different from that of the first embodiment, but the rest is the same as that of the first embodiment.

  In step SP41, a human face region HF is extracted based on the contour information in the captured image.

  Specifically, first, an edge extraction process is performed on the image G1 using an edge extraction filter (for example, a Sobel filter) to obtain an image G1b (see FIG. 15). Then, pattern matching processing with a template image prepared in advance is performed for the area QRi around the AF area FRi in the image G1b. In this pattern matching process, template images TM1, TM2, TM3 and the like as shown in FIG. 16 are used. FIG. 16 shows three template images TM1, TM2 and TM3 having different sizes. Each template image is an edge image having a substantially elliptical shape. By using these template images, a human face region is extracted based on contour information in the images.

  Also, here, in order to determine whether or not the face area is more accurate, it corresponds to the eye at a predetermined reference position in the extracted face area (more specifically, a face area candidate). A part having a part corresponding to the mouth and / or the mouth is extracted as a final face area (see FIG. 17). In FIG. 17, the extracted face area is indicated by a hatched area.

  In step SP42, branch processing is performed depending on whether or not a face area exists in the target area QR. When the face area exists in the target area QR, the process proceeds to step SP43, and when the face area does not exist in the target area QR, the process proceeds to step SP46.

  In step SP43, the luminance YSi of the extracted face area HFi is calculated. Specifically, the average brightness of the plurality of blocks Bj included in the outline of the face area HFi is calculated as the brightness YSi of the face area HFi. In step SP43, the average luminance of the plurality of blocks Bj included in the lower half region of the detected substantially elliptical face region may be calculated as the luminance YSi of the face region HFi. According to this, it is possible to reduce the influence of the hair portion.

  On the other hand, in step SP46, the average luminance YSi of the face area is set to 0 (zero).

  In other steps, the same operation as in the first embodiment is executed.

  Even in such an aspect, similarly to the first embodiment, the same brightness correction is stably performed on a plurality of images that shoot a shooting scene in which a plurality of subject persons having different brightness exist at a relatively close distance. It can be applied automatically.

<4. Other>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

  For example, the idea of the third embodiment may be applied to the second embodiment. Specifically, the face area HF of the person is extracted based on the contour information in the image, the correction amount GCi for the extracted face area HF is obtained, and the weights for the respective correction amounts GCi are weighted by using equation (5). The final brightness correction amount (correction correction amount) GM may be obtained.

  Further, in each of the above-described embodiments, the case where the idea of the present invention is applied to the image processing technique for obtaining a recording image (main image) for recording in the imaging apparatus 1 is illustrated, but the present invention is not limited thereto. Not. For example, the idea of the present invention may be applied when image processing is performed on a captured image acquired by the imaging apparatus 1 using a computer such as a personal computer. Specifically, a program that realizes the above function may be executed in a personal computer to correct the brightness of the captured image acquired by the imaging apparatus 1. At this time, information such as the defocus amount ΔDi regarding each AF area is stored along with the photographed image, and the same operation as described above is executed using the information such as the defocus amount ΔDi. That's fine.

  Further, in each of the above-described embodiments, the case where the idea of the present invention is applied to an image processing technique for obtaining a recording still image (captured image) is illustrated, but the present invention is not limited to this. For example, the idea of the present invention may be applied to an image processing technique for obtaining a moving image (captured image) such as a live view image.

It is a front external view of an imaging device. It is a back external view of an imaging device. It is a block diagram which shows the function structure of an imaging device. It is a flowchart which shows the operation | movement immediately after imaging | photography operation | movement of an imaging device. It is a figure which shows a certain imaging | photography scene. It is a figure which shows AF area etc. It is a figure which shows the some block which divided the picked-up image. It is a figure which shows the skin color area in Cr-Cb space. It is a figure which shows the extracted face area | region. It is a figure which shows the relationship between a defocus amount and a weighting coefficient. It is a figure which shows the distance etc. from a focusing object. It is a figure which shows several gamma correction curves. It is a flowchart which shows the operation | movement which concerns on 2nd Embodiment. It is a flowchart which shows the operation | movement which concerns on 3rd Embodiment. It is a figure which shows an edge image. It is a figure which shows a template image. It is a figure which shows the extracted face area | region.

Explanation of symbols

1 imaging device Bj block Cr, Cb color component EA skin color space FR1 to FR4 AF area G1 photographed image G1b edge extracted image GC, GCi correction amount GM correction correction amount HF, HFi face area QR, QRi target area Wi weighting coefficient YS reference luminance YSi average brightness YT Target brightness

Claims (7)

Extracting means for extracting a human face area from each target area corresponding to each of a plurality of AF areas;
Luminance calculating means for calculating the luminance of each face area extracted by the extracting means;
Distance information acquisition means for respectively obtaining distance information about the subject in the AF area corresponding to each face area;
A weighting coefficient for each face area is set according to the distance information of the AF area corresponding to each face area, and the brightness of the captured image is determined based on the brightness of each face area and the weighting coefficient of each face area. Correction means for correcting the thickness;
An imaging apparatus comprising: The imaging device according to claim 1,
The imaging device is characterized in that the extraction means extracts a skin color area in each target area as the face area. The imaging device according to claim 1,
The image pickup apparatus, wherein the extraction unit extracts the face area based on contour information in a picked-up image. The imaging device according to claim 1,
The correction means includes
A value obtained by weighted addition of the brightness of each face area according to the weighting coefficient of each face area is calculated as a reference brightness,
An imaging apparatus that corrects brightness of the captured image based on the reference luminance. The imaging device according to claim 1,
The correction means includes
Calculating a correction amount for bringing the brightness of each face area close to the target brightness,
Calculating a correction correction amount obtained by weighting and adding the correction amount of each face area according to a weighting coefficient of each face area;
An imaging apparatus, wherein the brightness of the captured image is corrected using the correction correction amount. Extracting means for extracting a human face area from each target area corresponding to each of a plurality of AF areas;
Luminance calculating means for calculating the luminance of each face area extracted by the extracting means;
Distance information acquisition means for respectively obtaining distance information about the subject in the AF area corresponding to each face area;
A weighting coefficient for each face area is set according to the distance information of the AF area corresponding to each face area, and the brightness of the captured image is determined based on the brightness of each face area and the weighting coefficient of each face area. Correction means for correcting the thickness;
An image processing apparatus comprising: On the computer,
a) extracting a human face area from each target area corresponding to each of a plurality of AF areas, and calculating the luminance of each extracted face area;
b) a procedure for obtaining distance information about the subject in the AF area corresponding to each face area;
c) A weighting coefficient for each face area is set according to the distance information of the AF area corresponding to each face area, and the captured image is based on the luminance of each face area and the weighting coefficient of each face area. The procedure to correct the brightness of
A program for running

Images