JP2019184899A - Imaging device - Google Patents

Abstract

To provide a flash unit light emission imaging device that improves face detection accuracy in a subject photographed image upon pre-light emission of a flash unit, and thereby can obtain the photographed image excellent in luminance of the face without increasing a release time lag.SOLUTION: An imaging device which performs pre-light emission prior to main-light emission upon photographing, has: range-finding means that measures a distance to a subject; means that acquires a pre-light emission amount; means that acquires a full-open aperture value of a lens; means that acquires a subject photographed image by the pre-light emission; range-finding means that acquires a change in subject luminance by the pre-light emission; and means that calculates a gain by which the subject photographed image is multiplied on the basis of the pre-light emission amount, a lens full-open aperture value, and a reflection rate of a human face area, creates a face detection image having the gain applied to the pre-light emission image to implement face detection, and computes a main-light emission amount in accordance with a measurement light value by the pre-light emission with respect to the detected face area.SELECTED DRAWING: Figure 3

Description

  The present invention provides an image pickup apparatus that performs pre-flash emission and photometry of a strobe and controls main flash emission using the photometric value, and in particular, has a method of detecting a face from an image and accurately detects the face of a subject in pre-flash. The present invention relates to an imaging apparatus.

  2. Description of the Related Art Conventionally, there has been known an imaging apparatus that performs photometry and face detection by acquiring an image signal with a photometric sensor before shooting and processing the image signal. In an imaging device that performs pre-flash and metering of a strobe and controls the main flash using this photometric value, an imaging device that detects the face of a subject in an image captured during pre-flash and improves photometric accuracy is known Yes.

JP 2005-184508 A Japanese Patent Laid-Open No. 2006-58885

  For example, Patent Document 1 discloses a method for detecting a face from an image at the time of pre-flash of a strobe and improving the accuracy at the time of main flash emission. In this, it is shown that the pre-emission light amount is determined based on distance measurement information performed on the subject in advance, the lens aperture value, and the sensitivity of the image sensor. However, a specific calculation formula is not shown, and a method for appropriately exposing the brightness of the face area in the photometric image obtained by the pre-flash is not specified. Therefore, there is a problem that the accuracy of face detection is deteriorated if the brightness of the face area is not properly exposed in the photometric image by pre-flash.

  For example, in Patent Document 2, in shooting without using a strobe, an image signal is acquired by a photometric sensor before shooting, and gain is applied to the image signal so that the brightness of the in-focus area becomes appropriate exposure. (Photometry without taking a photo) discloses a method for improving the accuracy of face detection by reaching an optimal exposure during repeated photometric operations from the start of the camera to imaging (shutter release). However, if a subject having a brightness different from the brightness of the face area is included in the focus area, the face area is not properly exposed, and the face detection accuracy is not improved. In addition, the photometric operation is repeatedly performed for determining the gain, but there is a problem in performing photometry a plurality of times during pre-flash in strobe shooting.

  That is, the flash light emission time of the xenon tube is, for example, 50 microseconds, and therefore the number of photometric operations performed during one pre-emission is one. Therefore, in order to repeat the photometric operation, it is necessary to perform the pre-light emission a plurality of times, which is not preferable because the electric power stored in the capacitor is consumed for light emission.

  Accordingly, an object of the present invention is to provide a strobe light emitting imaging device that can obtain a captured image with good face brightness without increasing the release time lag by improving face detection accuracy in a subject captured image at the time of pre-flash firing. It is to be.

In order to achieve the above object, the present invention provides:
Ranging means for measuring the distance to the subject;
Means for obtaining a pre-emission amount;
Means for obtaining an open aperture value of the lens;
Means for acquiring a subject captured image by pre-flash;
Ranging means for acquiring changes in subject brightness due to pre-flash,
Gain calculating means for calculating a gain to be applied to the subject captured image based on the pre-emission amount, the lens open aperture value, and the reflectance of the human face area;
Based on a subject captured image gain applying unit that generates a face detection image by applying a gain to a subject captured image by pre-flash, and detects a face area in the subject captured image based on the face detection image generated by applying the gain Face area detection means;
A calculation means for calculating a main light emission amount according to a photometric value obtained by the pre-light emission for the face area detected by the face area detection means;
Control means for controlling the strobe to perform photographing based on the main light emission amount calculated by the calculating means.

  According to the present invention, there is provided a stroboscopic light emitting imaging apparatus capable of obtaining a photographic image with good face brightness without increasing a release time lag by improving face detection accuracy in a subject captured image at the time of flash pre-flash. That is.

Device configuration Shooting flow Shooting flow Face image

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Example 1]
FIG. 1 is a configuration diagram of the imaging apparatus according to the present embodiment. The imaging device according to the present embodiment includes a camera device 10, a lens device 11, and a strobe device 12. The camera device 10 and the lens device 11 are configured such that the lens device is detachable via a mount portion. The camera device 10 and the lens device 11 may be an integrated device. The camera device 10 and the strobe device 12 are configured such that the strobe device is detachable via an accessory shoe. The camera device 10 and the flash device 12 may be an integrated device.

  101 is a camera control unit, 102 is a nonvolatile memory, 103 is a temporary storage memory, 104 is a main mirror, 105 is a focusing plate, 106 is a pentaprism, 107 is a finder, 108 is a photometric sensor, 109 is a sub-mirror, 110 is a focus detection sensor , 111 is a shutter, 112 is an image sensor, 113 is an image processing unit, 114 is a recording medium, 114 is a shutter button, 116 is a setting button, 151 is a strobe control unit, 152 is a xenon tube, 153 is a charging circuit, 154 is a strobe. A light emitting capacitor, 155 is a light emitting circuit, 156 is an external light control sensor, 157 reflector, and 158 is a contact.

  The camera control unit 101 controls the operation of the entire camera. The camera control unit 101 is composed of a microcomputer and is connected to a nonvolatile memory 102 and a temporary storage memory 103. The nonvolatile memory 102 stores programs and setting values, and the temporary storage memory 103 stores images being calculated, calculation results, and control values. The camera control unit 101 communicates with the lens control unit 202 and the strobe control unit 151 through the contact 203 and the contact 158, and transmits instructions and receives information.

  When the main mirror 104 is in the viewfinder observation state shown in FIG. 1, a part of the subject light beam that has passed through the photographing lens 201 is reflected. The light beam reflected by the main mirror 104 forms an image on the focus plate 105, changes the direction of the image by the pentaprism 106, and is guided to the finder 107 to reach the photographer's eyes. Thus, the photographer can confirm the subject image. The subject image formed on the focus plate 105 is guided to a photometric sensor 108 in the vicinity of the viewfinder 107. The subject image formed on the photometric sensor 108 is photoelectrically converted into an electrical image signal and output to the camera control unit 101 as luminance information.

  The photometric sensor 108 is an area type sensor having tens of thousands or more pixels, and can measure light by dividing a screen area. The photometric sensor 108 may output luminance information to the image processing unit 113, perform various calculations in the image processing unit 113, and output the results to the camera control unit 101. In order for the camera control unit 101 or the image processing unit 113 to calculate the subject brightness and analyze the scene, the screen area is divided into, for example, 9 × 7 blocks, and brightness information for each block is obtained based on the pixel signals included in each block. Is calculated. The luminance information of the block is obtained, for example, by averaging the luminance of all the pixels in the block.

  The photometric sensor 108 may have R pixels, G pixels, and B pixels and be able to recognize colors. The image processing unit 113 has a function of detecting a face of a person as a main subject as a main subject by determining feature points of the shape such as eyes, nose, and mouth of the captured image, and the number of detected faces. Output region position / size, etc. to the camera control unit 101.

  The main mirror 104 is a half mirror, and the light beam that has passed through the main mirror 104 without being reflected is reflected by the sub mirror 109 and guided to the focus detection sensor 110. The subject image formed on the focus detection sensor 110 is photoelectrically converted into an electrical image signal and transmitted to the camera control unit 101. The camera control unit 101 performs a focus detection calculation by a phase difference detection method based on the image signal of the subject image from the focus detection sensor 110, and based on the in-focus distance information such as the calculated defocus amount and defocus direction, The position of the focus lens in the imaging lens 201 is changed via the lens control unit 202 and driven to the in-focus position. Further, the camera control unit 101 drives a diaphragm (not shown) via the lens control unit 202.

  At the time of imaging, the camera control unit 101 retracts the main mirror 104 and the sub mirror 109 to change the optical path so that the subject image is formed on the imaging element 112. A shutter 111 is disposed on the front surface of the image sensor 112. The shutter 111 is a focal plane shutter.

  As the image sensor 112, a CMOS image sensor or the like having several million or more imaging pixels is used. The subject image that has passed through the imaging lens 201 forms a subject image on the image sensor 112. The image sensor 112 receives this light and performs photoelectric conversion. The electric charge converted from light shows luminance, and after being stored for each pixel, it is read as a voltage by a transistor in the pixel. The read voltage is converted into a digital signal and transferred to the image processing unit 113.

  The image processing unit 113 is a processor specialized for image processing. The captured image transferred from the image sensor 112 is developed, variously processed, and encoded, and then output to the recording medium 114. In addition, the calculation result of the image is output to the camera control unit 101.

  Reference numeral 115 denotes a shutter button, and SW1 and SW2 are formed of two-stage switches. When SW1 is turned on, shooting preparation is started, and when SW2 is turned on, shooting is started. Reference numeral 116 denotes a setting button, which is a means for inputting a setting for the photographer to select the operation of the camera. It consists of mechanical buttons or touch sensors.

  Reference numeral 151 denotes a strobe control unit that controls each unit of the strobe and communicates with the camera control unit 101 via the contact 158 to receive an operation instruction or transmit various information. The charge charged in the strobe light emission capacitor 154 by the charging circuit 153 is controlled by the light emission circuit 155 to cause the xenon tube 152 to emit light. The emitted light is collected by the reflector 157 and illuminates the subject. Reference numeral 156 denotes an external light control sensor that measures the luminance of the subject without using the light transmitted through the lens 201. As a method for measuring the subject luminance, the image sensor 112 may be used in addition to the method using the external light control sensor 156 and the above-mentioned photometric sensor 108.

  Here, optimal gain calculation for detecting a face will be described. “Determining exposure in a captured image” means that when a main subject region is determined using a face region, the main subject region cannot be determined before the face region is detected. The exposure of the captured image can be adjusted by, for example, the average value of the luminance of the entire screen. However, when there is a large area in the subject image that is significantly different from the reflectance of the face, the exposure of the face area is greatly shifted. End up.

Area of _face area: S _face
Area of _other area: S _other
Luminance of face area: E _face
Luminance in other areas: E _other
When the average brightness of the entire screen: E _avr

It becomes.

For example, in a general bust (bust-up) person image, the size of the face is about 10%, so the chest part with the clothes and the background occupy most of the area. In addition, for example, the difference in brightness between a person standing on a podium and a background such as a dark curtain is EV7 or more.
For example, if E _face = 9 (EV) and E _other = 1 (EV) and this is applied to the formula, the average brightness of the entire screen E _avr = 0.1 * 9 + 0.9 * 1 = 1.8 (EV)
It becomes. Therefore, if the gain is determined according to E _avr and the appropriate exposure is obtained, Eface is 7.2 (EV) over from the appropriate exposure.

In another scene, for example, a person who becomes a shadow in the daytime backlight has a background luminance difference of EV7 or more. E _face = 3 (EV), E _other = 10 (EV), average brightness of the entire screen E _avr = 0.1 * 3 + 0.9 * 10 = 9.3 (EV)

Therefore when the decision to apply expose the gain in accordance with the E _avr, E _fac e becomes 6.3 (EV) under a suitable exposure. In this way, when brightness other than the face area is used when calculating the gain, the brightness of the face area is greatly different, the face contrast is lowered, and the face detection accuracy is lowered.
On the other hand, during flash photography, the brightness of the subject due to flash emission is given by the light emission amount, the reflectance, and the distance from the subject. As the distance to the subject, distance measurement information obtained from the focus distance information at the time of the last stationary light is used. The reflectance of the area that is the face of the subject is known in advance. From these, the luminance of the subject by strobe emission can be obtained.

  The guide number indicating the amount of strobe light of the camera is defined by the distance (m) to the subject, the aperture value (F value), and the imaging ISO sensitivity in order to properly expose the subject with the standard reflectance (18%). As the distance to the subject, distance measurement information obtained from the focus distance information at the time of the last stationary light is used. When the guide number (GN) is appropriate for exposing the subject appropriately

When the pre-emission amount is fixed, the pre-emission amount is fixed to the guide number (GN). In order to generate a face detection image from the pre-flash image, the gain according to the distance is fixed to gain G. Since the face reflectance is around 24-30%, considering the difference from the standard reflectance

The gain G fixed is as follows

For the image used for face detection, the difference between the pre-flash photometric image and the steady-light photometric image is taken in order to extract the luminance due to the flash emission. At the time of face detection, the reflectance of the face is known in advance as described above, and it is not necessary to properly expose subjects other than the face in the image for face detection, so a gain G is applied to the pre-photometric image. , Face area detection is performed.

Face detection image = (Pre-flash metering image-Steady-light metering image) × Gain G fixed (5)
The face detection image obtained by this formula has an appropriate exposure of the face area, and the face can be detected with high accuracy if the face detection is executed once.

  Further, as described in Patent Document 1, the flash light emission amount may be changed from the distance measurement information. Further, the distance to the subject may be obtained by other distance measuring means instead of the focal distance information of the lens.

  Next, photographing using a strobe in the present embodiment will be described with reference to the flowchart of FIG.

  The photographing operation is performed according to each step of FIG. In step S201, the state of SW1 of the shutter button 115 is monitored. If SW1 of the shutter button 115 is turned on, the process returns to step S202, and if it is off, the process returns to step S201.

  In step S202, the focus detection sensor 110 is used to detect the focus state, defocus information is detected, and the process proceeds to step S203. In step S203, the lens 203 is driven to the in-focus position based on the defocus information obtained in step S203, and the process proceeds to step S204.

  In step S204, whether or not to emit the strobe is set according to the setting input by the photographer. Only when the light is emitted, the process proceeds to step S205. If no light is emitted, the process proceeds to step S224. In step S205, external light is measured. A photometric image AE (1) is acquired. In step S206, pre-emission is performed. In step S207, pre-flash photometry is performed. A photometric image AE (2) is acquired. In step S209, the gain G is calculated.

  In step S210, outside light is drawn in order to calculate the subject luminance due to the pre-emission. The photometric image AE (2) -AE (1) is calculated, and a gain G is applied to generate a face detection image. In step S211, face detection is performed on the face detection image. In step S212, it is determined whether a face has been detected. If a face is detected, the process proceeds to step S213. If a face is not detected, the process proceeds to step S215.

  In step S213, the detected face region position is determined. In step S214, the brightness of the detected face area is determined. In step S215, the flash light control region is determined. In step S217, the flash emission amount is determined. In step S218, main photographing / main light emission is performed.

  Data captured using the image sensor 112 in step S219 is subjected to development processing and image processing by the image processing unit 113 to be compressed and encoded. In step S220, the image data is recorded on the recording medium 114. In step S221, shooting is finished.

(When not taking flash photography in step S204)
In step S222, ambient light is measured. In step S223, actual photographing is performed. Proceed to step S219.

  As described above, by improving the face detection accuracy in the subject captured image at the time of pre-flash of the strobe, it is possible to provide a captured image with good face brightness without increasing the release time lag.

[Example 2]
In the first embodiment, the face detection calculation requires about 60 milliseconds, for example. Due to restrictions on the number of pixels of the image sensor and optical resolution, there is a minimum size that can detect a face. When it can be determined that face detection is difficult based on the distance to the subject, the face detection calculation is skipped. When each component such as eyes, nose and mouth of the face is 3 to 4 pixels, the face size is about 20 pixels.

  FIG. 3 shows a face size of about 20 pixels. Each face component of the face eye 301, the face nose 302, and the face mouth 303 is 3 to 4 pixels.

  For example, when the imaging size is 36 mm × 24 mm, the range in which the photometric sensor performs photometry is, for example, 30 mm × 20 mm, and the number of pixels is, for example, 300 × 180 pixels. At this time, 20 pixels of the photometric sensor correspond to 2 millimeters on the focusing plate. If the size of an adult face is 12 centimeters, the distance to form an image on a focusing plate of 2 millimeters is 60 f. (60 times the lens focal length). When the size of the face on the captured image is 20 pixels or less, the constituent elements such as the eyes, nose, and mouth of the face cannot be detected, and the detection accuracy is significantly reduced.

  Further, since the focusing screen diffuses light and lowers the resolution, the minimum size that the face can detect is larger than 20 pixels. As the distance to the subject, distance measurement information obtained from the focus distance information at the time of the last stationary light is used. At this time, when the distance to the subject is 60 times or more of the lens focal length, the size of the face of the subject on the photometric sensor is not more than the minimum size that the above-mentioned face can be detected, making face detection difficult.

  Next, photographing using a strobe in the present embodiment will be described with reference to the flowchart of FIG.

  In step S201, the state of SW1 of the shutter button 115 is monitored. If SW1 of the shutter button 115 is turned on, the process returns to step S202, and if it is off, the process returns to step S201. In step S202, the focus detection sensor 110 is used to detect the focus state, defocus information is detected, and the process proceeds to step S203. In step S203, the lens 203 is driven to the in-focus position based on the defocus information obtained in step S203, and the process proceeds to step S204.

  In step S204, whether or not to emit the strobe is set according to the setting input by the photographer. Only when the light is emitted, the process proceeds to step S205. If no light is emitted, the process proceeds to step S222. In step S205, external light is measured. A photometric image AE (1) is acquired. In step S206, pre-emission is performed. In step S207, pre-flash photometry is performed. A photometric image AE (2) is acquired. In step S208, it is determined whether the in-focus position obtained in step S203 exceeds 60f. When exceeding, it progresses to step S216, and when not exceeding, it progresses to step S209. In step S209, the gain G is calculated.

  In step S210, outside light is drawn in order to calculate the subject luminance due to the pre-emission. The photometric image AE (2) -AE (1) is calculated, and a gain G is applied to generate a face detection image. In step S211, face detection is performed on the face detection image. In step S212, it is determined whether a face has been detected. If a face is detected, the process proceeds to step S213. If a face is not detected, the process proceeds to step S215.

  In step S213, the detected face region position is determined. In step S214, the brightness of the detected face area is determined. In step S215, the flash light control region is determined.

(When the focus position exceeds 60f in step S208)
In step S216, the photometric image AE (2) -AE (1) is calculated. Proceed to step S215.

(From step S215)
In step S217, the flash emission amount is determined. In step S218, main photographing / main light emission is performed. Data captured using the image sensor 112 in step S219 is subjected to development processing and image processing by the image processing unit 113 to be compressed and encoded. In step S220, the image data is recorded on the recording medium 114. In step S221, shooting is finished.

(When not taking flash photography in step S204)
In step S222, ambient light is measured. In step S223, actual photographing is performed. Proceed to step S219.

  As described above, when it is possible to determine that it is difficult to detect the face from the distance information with respect to the subject, it is possible to provide a captured image that suppresses an increase in the release time lag by skipping the face detection step.

[Example 3]
In Example 1, when the distance information with respect to the subject is obtained from the focus position information, the distance to the subject may indicate a range depending on the lens. That is, far object distance: dLfar
Close subject distance: dLnear
And the distance to the subject is indicated to be in the range from dLnear to dLfar. At this time, two gains are obtained by applying the equation (4) in two ways.

  Distant subject gain Gfar

  Gain of near subject Gnear

Face detection image far which calculates two face detection images according to two gains G = pre-photometry image × gain Gfar
Face detection image near = Pre-photometry image × Gain Gnear
Face detection is performed on these two images.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

101 is a camera control unit 108, photometric sensor 113 is an image processing unit

Claims (3)

An imaging device that performs pre-flash before main flash during shooting,
Ranging means for measuring the distance to the subject;
Means for obtaining a pre-emission amount;
Means for obtaining an open aperture value of the lens;
Means for acquiring a subject captured image by pre-flash;
Brightness change acquisition means for acquiring a change in subject brightness due to pre-flash;
Gain calculating means for calculating a gain to be applied to the subject captured image based on the pre-emission amount, the lens open aperture value, and the reflectance of the human face area;
Based on a subject captured image gain applying unit that generates a face detection image by applying a gain to a subject captured image by pre-flash, and detects a face area in the subject captured image based on the face detection image generated by applying the gain Face area detection means;
A calculation means for calculating a main light emission amount according to a photometric value obtained by the pre-light emission for the face area detected by the face area detection means;
Control means for controlling the strobe to perform shooting based on the main light emission amount calculated by the calculating means;
An imaging device comprising: The imaging apparatus according to claim 1, wherein when the distance to the subject is equal to or greater than a threshold value, the face area detection unit is skipped. 2. The imaging apparatus according to claim 1, wherein when the distance information with respect to the subject obtained from the distance measuring means for measuring the distance to the subject indicates a range, the two gains are calculated using the two distances indicated by the distance information. .