JP2019110402A - Imaging apparatus, control method therefor, and program - Google Patents

Abstract

To prevent an unnatural change in brightness in a moving picture by preventing exposure variation between predetermined frames.SOLUTION: An imaging apparatus comprises: face detection means that detects a face area from image data obtained using an imaging element; photometry means that, in a case where one or more face areas are detected from the image data by the face detection means, increases weighing of an area corresponding to the detected face area, in comparison with the other areas, and thus obtains a photometry result of the entire image data; and exposure control means that, in a case where a first photometry system is selected, exerts exposure control such that a change in exposure between frames in a moving picture obtained on the basis of the first photometry system is gentle.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus capable of exposure control based on a photometric result of a face area, and a control method and program thereof.

  Conventionally, in an imaging apparatus such as a digital camera, as a photometric method for capturing still images and moving images, for example, the acquired image data (hereinafter simply referred to as an image) is divided and the luminance value of each area is calculated. There is known a method of averaging them after they are done.

  In addition, in the case of moving image shooting, in order to suppress frequent changes in the exposure according to the movement of the subject or the composition change at the time of shooting, exposure control with emphasis on stability is desired. For this reason, as a photometric method for moving image shooting, a method called so-called center-weighted average photometry (mode) is often adopted in which a relatively wide range is taken as the photometric range while placing weights in the vicinity of the center of the image.

  In recent years, a method of detecting a face area of a person from an image and tracking the face area frame by frame is used. Then, when a face area is detected at the time of moving image shooting, an imaging device is known that switches the metering method from center-weighted average metering to face evaluation photometry (mode) for increasing weighting of photometry in the face area. There is.

  However, when there is a movement of a person (for example, the face faces other than the front), the detection result of the face area is not stable, and a frame in which the face area can be detected and a frame in which the face area can not be detected exist randomly. At this time, when the face area is detected, the face evaluation photometry is performed, and when the center-weighted average photometry is performed when the face area is not detected, the photometry result changes due to the difference in the photometry method. Unnatural video with flickering in brightness is acquired.

  Patent Document 1 discloses a technique for fixing a predetermined exposure control value for a predetermined time when a face area disappears in exposure control.

JP, 2016-129287, A

  However, in the imaging device of Patent Document 1, although the eyelid region has not disappeared, the case where the direction of the eyelid changes is not considered. For example, since the ratio of the hair portion to the photometric area corresponding to the face area increases in response to the face orientation changing from the front to the side, the photometric result changes due to the brightness of the hair portion, The exposure also changes accordingly.

  Also, if the photometric range and the size of the face area do not match, the ratio of the face area to the photometric range is not constant either. In particular, when the face area is smaller than the photometric range, background portions other than the face area are included in the photometric range. In this case, when the scene in the background changes (for example, when the movement of the person changes from a bright sky background to a dark building background), the photometric result changes, and the exposure changes accordingly.

  An object of the present invention is to suppress variation in exposure between predetermined frames to suppress unnatural change in brightness in a moving image.

  In order to solve the above problems, in an imaging apparatus according to the present invention, an imaging element for imaging a subject and outputting image data, and a face detection unit for detecting a face area from image data obtained using the imaging element When one or more face areas are detected from the image data by the face detection unit, the weight of the area corresponding to the detected face area is set larger than that of the other areas, and the entire image data is Each frame in a moving image acquired based on the first photometric method when the first photometric method is selected by the first photometric method capable of photometric measurement according to the first photometric method for obtaining the photometric result and the photometric means And exposure control means for performing exposure control so that the change in exposure between the two is gradual.

  According to the present invention, it is possible to suppress variation in exposure between predetermined frames, and to suppress unnatural changes in brightness in a moving image.

It is a figure showing roughly an example of composition of an image sensor applied to an imaging device of each embodiment of the present invention. It is a figure which illustrates the subject concerning the present invention illustratively. It is a figure which illustrates in an example the method of computing the target exposure luminosity value which is the target value for exposure control concerning a 1st embodiment of the present invention. It is a figure which illustrates the brightness | luminance change of the to-be-photographed object based on 2nd Embodiment of this invention illustratively. It is a figure which illustrates in an example the method of calculating the target exposure luminosity value which is the target value for exposure control concerning a 2nd embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Example of Configuration of Camera 100)
FIG. 1 is a view schematically showing a configuration example of an imaging device according to each embodiment of the present invention. Hereinafter, with reference to FIG. 1, a basic configuration example of a digital camera (hereinafter simply referred to as a camera) 100 which is an imaging device of the present embodiment will be described.

  As illustrated in FIG. 1, reference numeral 101 denotes a camera microcontroller (camera MPU) for controlling a system such as a photographing sequence, and control means for collectively controlling various operations in the camera 100. Reference numeral 103 denotes a charge storage type imaging device such as a CCD or a CMOS that converts reflected light from a subject into an electrical signal. Reference numeral 102 denotes a timing signal generation circuit that generates a timing signal necessary to operate the imaging device 103. An A / D converter 104 converts analog image data read from the image sensor 102 into digital image data. Reference numeral 105 denotes a memory controller for controlling reading / writing of memory, refresh operation of the memory 106, and the like. An image display unit 107 is an image display unit for displaying the image data stored in the memory 106. An interface unit 108 connects a recording medium for recording image data and the like and an internal block of the camera 100. A recording medium 109 such as a memory card or a hard disk is removable and / or fixed to the camera 100.

  The motor control unit 110 controls a motor (not shown) according to an instruction (signal) from the camera MPU 101 at the time of the exposure operation, thereby performing a mirror up / down operation (not shown) and a shutter charge operation. The shutter control unit 111 operates a front curtain and a rear curtain constituting a shutter (not shown) according to control (signal) from the camera MPU 101 to control an exposure time of the image sensor 103.

  The photometry unit 112 divides the converted digital image data output from the imaging element 103 into a plurality of areas, and calculates the luminance value (photometric result) of each area and the luminance value (photometric result) of the entire image data body. Photometric means to The photometry unit according to the present embodiment can select a face evaluation photometry method (first photometry method) in which the weighting degree of the face area is made larger than that of the other area. The camera MPU 101 executes exposure control at the time of imaging an object based on this luminance value. That is, the camera MPU 101 is an exposure control unit according to the present embodiment. Specifically, with respect to the brightness control of the subject calculated in the exposure control and APEX units of the present embodiment, the exposure of the Av value (aperture value), Tv value (shutter speed), Sv value (shooting sensitivity, ISO sensitivity), etc. Calculate the appropriate combination of parameters. The face detection unit 113 is a face detection unit that detects a face area of a person from the converted digital image data output from the image sensor 103. The face area detection method may be any method, but in the present embodiment, a configuration is employed in which the face area is detected by a pattern matching method with the eyelid information stored in advance in the memory 106. The detection result of the face area is output to the camera MPU 101. When one or more face areas are detected from the image data (that is, when a plurality of face areas exist in the image data), the main face area is detected based on the size, position, and orientation of the eyelid area, Face evaluation photometry to be described later is performed based on the main face area.

  A lens control unit 114 controls driving of various lenses and diaphragms included in a shooting lens group (not shown), and performs focus adjustment of a focus lens included in the shooting lens group, zoom position adjustment of a zoom lens, and lens for controlling the aperture opening. It is a control means. The photographing lens group may be integrally formed with the camera 100, or may be a so-called interchangeable lens type in which the photographing lens group can be attached to and detached from a mount (not shown) provided in the camera 100. Good.

  The focus detection unit 115 is a focus detection unit that detects an amount of defocus with respect to a subject, which is used in an AF (Auto Focus) process. The posture detection unit 116 is, for example, a detection unit that detects the tilt of the camera with respect to the direction of gravity and the rotation direction about the optical axis based on the output of a gyro sensor (not shown) or the like. The switch operation unit 117 is an operation member of the camera. The switch SW1 is turned on by the first stroke of the release button (not shown), and the AF processing and the photometry operation are started. Further, the SW2 is turned on by the second stroke of the release button (not shown), and the exposure control operation and the imaging operation are started. Each operation is controlled by the camera MPU 101.

  The flash control unit 118 is a light emission control unit that controls the light emission of the built-in flash 119 and the external flash 120. The flash control unit 118 also performs control of switching the light emission pattern (pre-flash instruction, main flash instruction), calculation of the main flash amount, and which one of the built-in flash 119 and the external flash 120 is used. Further, communication with the external flash 120 is performed via the flash control unit 118.

First Embodiment
Next, each operation of the camera 100 according to the present embodiment will be described using FIGS. 2 and 3.

  First, with reference to FIG. 2, an outline of the problem according to the present embodiment will be described. FIG. 2 is a view exemplarily explaining the problem according to the present invention. Each of FIG. 2A, FIG. 2B and FIG. 2C shows each frame in which the relative position of the person with respect to two background areas having different brightness is different as time passes when shooting a moving image. Is shown. In each drawing of FIG. 2, the left side in the drawing is a bright background (for example, a range in which light is applied), and the right side of the screen is a dark background (for example, a range in which no light is applied). The person in the figure is assumed to move in the order of photometric frame 1, photometric frame 2, and photometric frame 3.

  The divided areas of each of the photometry frames illustrated in the respective drawings of FIG. 2 indicate photometric frames, and in the present embodiment, each area obtained by dividing an image (frame) into 8 vertical and 10 horizontal is illustrated as an example. Make it a photometric frame. A thick frame portion in each of the drawings of FIGS. 2A and 2B indicates a photometry frame (eyelid partial photometry area) including the detected face area. In the face evaluation photometry, photometric calculation is performed by increasing the weight of the luminance value corresponding to the face partial photometry area.

  As shown in FIG. 2A, at the timing corresponding to the photometry frame 1, only the bright background is included in the face partial photometry area, but as shown in FIG. 2B, the photometry frame 2 is included. At the corresponding timing, a dark background is included in the face partial metering area. At this time, since the dark background is included, the luminance value of the face partial metering area of the metering frame 2 becomes lower (darker) than that of the metering frame 1. Then, the luminance value of the entire photometry frame 2 (photometry result) is also lower (darker) than the photometry frame 1 because the luminance value of the face partial photometry area is lowered, regardless of the change in the brightness of the eyelid area. Exposure control is performed to brighten the entire image.

  Further, as illustrated in FIG. 2C, in the photometry frame 3, when the person moves, again, as in the photometry frame 1, only the bright background is included in the face partial photometry area. At this time, the luminance value of the face partial metering area calculated in the metering frame 3 is higher (brighter) than the metering frame 2. Due to the change in the luminance value of the face partial photometry area, the exposure control is performed so as to darken the entire image again.

  In order to maintain the quality of moving pictures, it is desirable that the change in brightness between frames is not large and that the change in brightness not frequently occur. However, as described above, even if the subject does not have a large luminance change, the exposure is frequently changed in response to the change in the luminance value of the eyelid partial photometry area due to the movement of the face area. The brightness between each image (frame) changes unnaturally. Therefore, in the present embodiment, in order to solve this problem, exposure control is performed so that the change in exposure when acquiring each image (frame) in a moving image is gradual. The details will be described below.

FIG. 3 is a view exemplarily explaining a method of calculating a target exposure luminance value which is a target value for exposure control according to the first embodiment of the present invention. Each photometric frame shown in FIG. 3 represents substantially the same timing as the photometric frame shown in FIG. For example, when calculating the target exposure luminance value of photometric frame 2, the luminance value of the face partial photometric area in the immediately preceding frame (photometric frame 1) and the face portion in the current frame (photometric frame 2) The average value with the luminance value of the photometric area is calculated. That is, the luminance value is smoothed between temporally consecutive frames constituting a moving image. At this time, let Yface1 and Yface2 be the luminance values of the face area photometric areas of the photometry frames 1 and 2, let Yall2 be the luminance values of all the photometric areas, and the luminance values of the face area photometric area and the luminance values of all the photometric areas. Weight ratio is 1: 1. In this case, the target exposure luminance value E2 of the photometric frame 2 can be obtained by the following equation (1).
E2 = (((Yface1 + Yface2) / 2) + Yall2) / 2 (1)

Assuming that the luminance value of the face partial metering area of the metering frame 3 is Yface 3 and the luminance value of all the metering areas is Yall 3 to calculate the target exposure luminance value E3 of the metering frame 3 in the same procedure, the following expression ( It can ask by 2).
E3 = (((Yface2 + Yface3) / 2) + Yall3) / 2 ... (2)

  As shown in FIGS. 2 and 3, in the face partial photometry area of photometry frame 1 and the face partial photometry area of photometry frame 3, the background of the face is only a bright part. Therefore, Yface1 and Yface3 are close. It becomes a value (brightness). Furthermore, since there is no significant change in the brightness of the entire screen between the metering frame 2 and the metering frame 3, the brightness value Yall2 of the entire metering area of the metering frame 2 and the brightness value Yall3 of the entire metering area of the metering frame 3 are also close. It becomes a value (brightness). Since Yface1 ≒ Yface2 and Yall2 ≒ Yall3, E2 ≒ E3, and the change in the target exposure luminance value between the photometric frame 2 and the photometric frame 3 becomes slower than in the conventional face evaluation photometric method.

As described above, in the example described above, the average value of the luminance value of the current face partial light metering area and the luminance value of the face partial light metering area in the immediately preceding frame is calculated, but the face portion between two or more consecutive frames The average value of the luminance values of the photometric area may be calculated. For example, when the target exposure luminance value E2 of the light measurement frame 2 is obtained, in addition to the face partial light measurement area luminance value Yface1 at the previous light measurement time, the face partial light measurement area luminance value Yface0 at the second light measurement time is also You may use it. In this case, when the luminance value of the face partial photometry area is smoothed, the average value of the luminance values can be obtained by the following equation (3).
E2 = (((Yface0 + Yface1 + Yface2) / 3) + Yall2) / 2 ... (3)

Also, in the above example, each of the luminance values Yface0, Yface1 and Yface2 corresponding to the face area may be weighted. Assuming that the weight of Yface0 is W0, the weight of Yface1 is W1, and the weight of Yface2 is W2, the target exposure luminance value E2 of the photometric frame 2 can be obtained by the following equation (4).
E2 = ((((Yface0 * W0) + (Yface1 * W1) + (Yface2 * W2)) / (W0 + W1 + W2)) + Yall2) / 2 (2) (4)

  As described above, in the present embodiment, the average value of the luminance values of the photometric area corresponding to the face area between consecutive frames (second frame and first frame) constituting a moving image is calculated. Then, in the present embodiment, the luminance value of the entire image data of the current frame (first frame) is calculated, and the luminance value of the entire image data and the average value between the frames related to the face area described above are averaged. Thus, the target exposure luminance value of the current frame is calculated. According to this configuration, the degree of change in the luminance value of the entire screen resulting from the change in the luminance value of the partial light metering area due to the movement of the eyelid region is smoothed and becomes gentle, and the change in exposure between frames can be suppressed. In other words, this configuration slows down the change in exposure between successive frames in the video. Therefore, by adopting this configuration, the imaging device according to the present embodiment can suppress variations in exposure between predetermined frames in a moving image, and can suppress unnatural changes in brightness in the moving image. .

Second Embodiment
Hereinafter, a second embodiment of the present invention will be described with reference to FIGS. 4 and 5. The basic configuration of the camera 100 is substantially the same as that of the first embodiment described above, and thus the description thereof is omitted, and the components of the camera 100 will be described with the same reference numerals. In the present embodiment, an example will be described in which the luminance change of the face area in the face partial photometry area is small and the luminance change in the photometric areas other than the face area is large between frames constituting a moving image. In this case, for example, as shown in FIG. 4, when a black car crosses behind the face area and there is no change in luminance of the face area between consecutive frames (photometric frame 1 and photometric frame) 2) Metering frame 2 and metering frame 3 etc.) In such a case, it is desirable that the change in brightness of the face area be gradual in order to prevent the degradation of the quality of the moving image. FIG. 4 is a view exemplarily explaining the luminance change of the subject according to the second embodiment of the present invention, and FIGS. 4 (a), 4 (b), 4 (c) and 4 (d). It shows that time has passed in the order of).

  FIG. 5 is a view exemplarily explaining a method of calculating a target exposure luminance value which is a target value for exposure control according to the second embodiment of the present invention. Each photometry frame illustrated in FIG. 5 represents substantially the same timing as the photometry frame in each diagram illustrated in FIG. 4. As illustrated in FIG. 5, in the present embodiment, when calculating the target exposure luminance value of the photometric frame 2, the luminance values of all the photometric areas in the previous frame (photometric frame 1) and the current frame (photometric Use the luminance values of the entire photometric area of frame 2). The luminance values of all the photometric areas described above are calculated by the eyelid evaluation photometry that increases the weighting of the luminance values of the face area, as in the first embodiment described above.

In FIG. 5B, the luminance value of the entire metering area of the metering frame 1 is Y1, the luminance value of the entire metering area of the metering frame 2 is Y2, and the weight ratio of the metering frames 1 and 2 is 1: 1. I assume. In this case, the target exposure luminance value E2 of the photometric frame 2 can be obtained by the following equation (5).
E2 = ((Y1 + Y2) / 2) (5)

Assuming that the target exposure luminance value E3 of the photometric frame 3 is the luminance value Y3 of the entire photometric area of the photometric frame 3 by the same procedure, it can be obtained by the following equation (6).
E3 = ((Y2 + Y3) / 2) (6)

  That is, in the present embodiment, the change in exposure can be smoothed by performing weighted averaging on the luminance values of all the photometric areas calculated by increasing the weighting of the eyelid area between successive frames. Unnatural changes in brightness can be suppressed.

  In the above description, the current frame is based on the weighted average of the luminance values of all photometric areas of the current frame by the face evaluation photometry and the luminance values of all photometric areas of the previous frame by the face evaluation photometry. Although the target exposure luminance value has been calculated, the present invention is not limited to this. For example, the configuration may be such that the result of the exposure evaluation photometry of each frame is smoothed between two or more consecutive frames.

  Also, in the case as illustrated in FIG. 4, priority is given to suppressing the unnatural change in brightness of the eyelid region, and, for example, the luminance value of only the face portion photometry area of the current frame and the previous one. The weighted averaging with the luminance value of only the face partial photometry area of the frame may be performed.

In addition, the weighting may be larger for the luminance value Y1 of the entire photometric area of the photometric frame 1 and the luminance value Y2 of the entire photometric area of the photometric frame 2 than that of the other photometric areas. For example, when the weight of Y1 is W1 and the weight of Y2 is W2, the target exposure luminance value E2 of the photometric frame 2 can be obtained by the following equation (7).
E2 = ((Yface1 * W1) + (Yface2 * W2)) / (W1 + W2) (7)

  Although the preferred embodiments of the present invention have been described above, the present invention is not limited thereto, and various modifications and changes are possible within the scope of the present invention. For example, in the embodiment described above, the camera MPU 101, the memory controller 105, the memory 106, the other control units, the detection unit, and the like cooperate with one another to control the operation of the camera 100. It is not limited to this. For example, the computer program according to each operation in the camera 100 as described above is stored in advance in the memory 106, and the computer program is executed by the camera MPU 101 or the like to control the operation of the camera 100. It is also good.

  Further, as long as it has the function of a program, the form of the program may be any one, such as an object code, a program executed by an interpreter, script data supplied to an OS, and the like. The recording medium for supplying the program may be, for example, a hard recording medium, a magnetic recording medium such as a magnetic tape, or an optical / optical magnetic recording medium.

  Moreover, although the case where a digital camera was employ | adopted as an example of the imaging device which implements this invention was demonstrated in embodiment mentioned above, it is not limited to this. For example, a configuration may be adopted in which an imaging device other than a digital camera such as a portable device such as a digital video camera or a smartphone, a security camera, or the like is adopted.

(Other embodiments)
Furthermore, the present invention provides a program that implements the functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in the computer of the system or apparatus read and execute the program. But it can be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

101 Camera MPU
103 image sensor 106 memory 107 image display unit 112 photometry unit 113 face detection unit

Claims (7)

An imaging device that captures an image of a subject and outputs image data;
Face detection means for detecting a face area from image data obtained using the image sensor;
When one or more face areas are detected from the image data by the face detection unit, the weighting of the area corresponding to the detected face area is made larger than the other areas, and the photometry of the entire image data is performed. Photometric means capable of photometric measurement by a first photometric method for obtaining a result;
Exposure control that performs exposure control so that the change in exposure between frames in a moving image acquired based on the first photometric method becomes gentle when the first photometric method is selected by the photometric unit An imaging device characterized by:   The exposure control means calculates a target value of exposure control of the first frame constituting the moving image according to the photometric result obtained by the photometry means, and performs exposure control based on the target value. The imaging device according to claim 1. The photometry unit divides image data obtained using the image pickup device into a plurality of areas, and among the plurality of areas, an area corresponding to a face area detected by the face detection unit is photometrically measured. And a second luminance value obtained by measuring the entire image data including the area corresponding to the face area.
In the first photometric method, a luminance value obtained by weighted averaging each of the first luminance values in at least the first frame and a second frame immediately before the first frame; 3. The image pickup apparatus according to claim 2, wherein the light measurement method is to calculate the light measurement result in the first frame by performing weighted averaging of the second luminance values in the frame. The photometry unit divides image data obtained using the image pickup device into a plurality of areas, and among the plurality of areas, weights of areas corresponding to the face area detected by the face detection unit are set to other values The first luminance value obtained by measuring the entire image data larger than the area can be calculated,
The first photometric method performs weighted averaging of each of the first luminance values in at least the first frame and a second frame immediately preceding the first frame. The image pickup apparatus according to claim 2, wherein the image pickup method is a light measurement method for calculating the light measurement result in the above. The photometry unit divides image data obtained using the image pickup device into a plurality of areas, and among the plurality of areas, an area corresponding to a face area detected by the face detection unit is photometrically measured. Can calculate the brightness value of
The first photometric method performs weighted averaging of the first luminance value of each of the first frame and a second frame immediately preceding the first frame, thereby obtaining the first photometry method. The image pickup apparatus according to claim 2, wherein the image pickup method is a light measurement method for calculating the light measurement result. A control method of an image pickup apparatus including an image pickup element which picks up an object and outputs image data,
A face detection step of detecting a face area from image data obtained using the image sensor;
First, when weighting of an area corresponding to the detected face area is larger than that of the other areas when one or more face areas are detected from the image data, the first photometric result of the entire image data is obtained With photometry process that can measure light by photometry method,
An exposure control step of performing exposure control such that a change in exposure between frames in a moving image is gradual when the first photometric method is performed; .   A computer readable program for causing a computer to execute the method of controlling an imaging device according to claim 6.

Images