JP2013242441A - Imaging apparatus and method for deciding illumination light quantity - Google Patents

Abstract

PROBLEM TO BE SOLVED: To preliminarily decide a flash light emission quantity, without performing pre-light-emission.SOLUTION: An imaging part 11 acquires an actual image. A depth map acquisition part 12 acquires a depth map simultaneously with the actual image. A decision part 26 creates a reduced brightness map in which an average brightness value obtained by averaging the brightness value of each pixel is distributed, for each area obtained by dividing the actual image. The decision part 26 processes the depth map similarly as well, to create a reduced depth map and a virtual illumination brightness map in which the quantity of light reflected from an object, obtained when performing illumination only by a built-in flash 17 is calculated for each area, based on the created reduced brightness map and the information of the light emission quantity of the built-in flash 17. The decision part 26 synthesizes the created illumination brightness map and the created reduced brightness map, to create a virtual brightness map and decide the light emission quantity of the built-in flash 17, so that the brightness value of the area including a target object becomes a predetermined brightness value, based on the created virtual brightness map.

Description

  The present invention relates to an imaging device and an illumination light amount determination method.

  In order to obtain proper exposure using a strobe, it is necessary to control the light emission amount of the strobe. Controls include a flashmatic system that controls the aperture based on the guide number and shooting distance, and an auto strobe that controls the flash to stop when the amount of reflected light returning from the subject reaches a predetermined level. The method is known. In the former method, if there is a measurement error in the shooting distance, it appears as an exposure error as it is. In the latter method, the dimming performance deteriorates when the window of the dimming sensor becomes dirty. Therefore, pre-flash is performed prior to shooting, subject light that reflects the subject by pre-flash is measured by the photometric means, and the strobe flash amount (light amount) during main flash is obtained so that appropriate exposure is obtained based on the measured subject light. And a light emission time) are known (Patent Document 1).

JP 2003-161987

  However, when flash photography is performed on a person, for example, a child, since the main light emission is performed immediately after the pre-light emission, there is a possibility that the main light emission photographing may be performed when the eyes are closed by the glare of the pre-light emission. . Further, when the subject is a living organism, for example, the living organism may move due to the pre-emission. Furthermore, flash photography that performs pre-flash may consume more power than flash photography using only main flash, and may cause the camera to run out of battery.

  In view of the above problems, an object of the present invention is to provide an imaging apparatus and an illumination light quantity determination method that can determine the illumination light quantity for illuminating a subject in advance without performing pre-light emission.

  An aspect of an imaging apparatus illustrating the present invention includes: a real image acquisition unit that captures a subject and acquires a real image; and a three-dimensional information acquisition unit that acquires three-dimensional information including depth information according to the real image. Deciding means for deciding an illuminating light amount of the illuminating means for illuminating the subject based on the real image and the three-dimensional information before photographing; Lighting control means for controlling.

  Further, according to one aspect of the illumination light amount determination method of the present invention, a step of inputting a real image obtained by imaging a subject, a step of inputting three-dimensional information including depth information according to the real image, Based on the real image and the three-dimensional information, virtual subject luminance information considering the depth information obtained when the illumination unit illuminates the subject is generated, and the subject of interest is included based on the virtual subject luminance information. Determining the illumination light quantity of the illumination means so that the luminance value of the region becomes a predetermined luminance value.

  According to the present invention, it is possible to determine the illumination light amount of the illumination unit that illuminates the subject before photographing without performing pre-light emission.

It is a block diagram which shows the outline of the electronic camera which is an example of this invention. It is a block diagram which shows the structure of the determination part which determines light emission amount. It is a flowchart which shows the operation | movement procedure in which a determination part determines the light emission amount. It is explanatory drawing which demonstrated the map which a determination part acquires and produces | generates virtually, (A) is a real image, (B) is a depth map, (C) is a reduced luminance map, (D) is a reduced luminance map, (E) is a virtual proof luminance map, and (F) is an explanatory diagram showing a virtual luminance map as an example. It is explanatory drawing which shows one form of the imaging | photography system containing an external flash and an electronic camera. It is a block diagram which shows the structure of the imaging | photography system demonstrated in FIG. It is a block diagram which shows the structure of the determination part of the electronic camera demonstrated in FIG. It is a flowchart which shows the operation | movement procedure in which the determination part demonstrated in FIG. 7 determines light emission amount.

[First Embodiment]
As shown in FIG. 1, an electronic camera 10 showing an embodiment of an imaging apparatus of the present invention includes an imaging unit 11, a depth map acquisition unit 12, a ROM 13, a RAM 14, a recording unit 15, a flash control unit 16, a built-in flash 17, An operation unit 18, a display control unit 19, a display unit 20, a touch panel 21, and a CPU 22 are provided. The imaging unit 11, depth map acquisition unit 12, ROM 13, RAM 14, recording unit 15, display control unit 19, and CPU 22 are each connected to a bus 23. Further, the flash control unit 16, the operation unit 18, and the touch panel 21 are connected to the CPU 22. The touch panel 21 is provided on the display unit 20. The display unit 20 is for displaying a through image or the like, and the display is controlled by the display control unit 19. The flash controller 16 is connected to the built-in flash 17 and controls the light emission of the built-in flash 17.

  The operation unit 18 includes a power supply operation unit, a release operation unit, and the like. The ROM 13 stores programs to be executed by the CPU 22 such as light emission information (information including guide number and peripheral light amount), a shooting program, and a light emission amount determination program. The RAM 14 is used as a work area for the CPU 22 and a temporary storage area for data. The recording unit 15 records actual image data acquired at the time of actual imaging.

  The imaging unit 11 forms an optical image of the subject formed by the imaging lens on the imaging surface of the imaging device with a predetermined exposure, converts the optical image of the formed subject into an electrical signal by the imaging device, and Generate an image. The imaging unit 11 is an example of a real image acquisition unit.

  The depth map acquisition unit 12 acquires a depth map that is depth information or three-dimensional information including the depth map (hereinafter referred to as “depth map”) so as to be in the same imaging range substantially simultaneously with the actual image. The acquired depth map is stored in the RAM 14 in association with the actual image. The depth map acquisition unit 12 includes the well-known “depth map generation method using a stereo camera”, “TOF (Time of Flight) method using infrared light”, and “method using a reflection pattern of infrared light”. And a depth map generation circuit that generates a depth map using a technique such as “a method of generating a depth map from a motion vector”. For the generated depth map, for example, in the case of a pixel having a deep depth (distance to the subject) based on a certain reference position, the luminance distribution corresponding to the depth in each pixel is obtained, such as increasing the luminance of the pixel. It is a grayscale image. The depth map acquisition unit 12 is an example of a three-dimensional information acquisition unit.

  The CPU 22 includes an AE / AF unit 24, a determination unit 25, and a determination unit 26. The AE / AF unit 24 acquires subject luminance information and subject distance information (AF evaluation value information indicating the contrast of the subject) based on a part of the actual image (ranging region and photometric region), and these Based on the information, the imaging unit 11 is controlled so that the exposure including the ISO sensitivity, the shutter speed, and the aperture value becomes an appropriate exposure, and the photographing lens moves to the in-focus position.

  In response to the half-pressing operation of the release operation unit, the CPU 22 performs pre-shooting for acquiring a real image from the imaging unit 11 in order to determine whether the subject is a low-luminance subject. The actual image acquired at the time of pre-shooting is temporarily stored in the RAM 14. The determination unit 25 determines that the subject is a low-luminance subject when the subject brightness of a part of the real image acquired at the time of pre-shooting is equal to or less than a predetermined threshold, for example.

  When the determination unit 25 determines that the subject is a low-luminance subject, the CPU 22 operates the depth map acquisition unit 12 to acquire a depth map, and uses the acquired depth map and the actual image acquired at the time of pre-shooting to appropriately The amount of light emitted from the built-in flash 17 for obtaining exposure is determined, and in response to a full pressing operation of the release operation unit, the built-in flash 17 is lighted based on the amount of light previously determined to perform the main photographing.

  When performing flash photography, the CPU 22 sets the exposure of the imaging unit 11 to predetermined exposure values (ISO sensitivity, aperture value, and shutter speed). The actual image acquired at the time of the actual photographing is stored in the recording unit 15 as the imaging data. In addition, as a release operation part, you may use the thing of a one-step pushing structure instead of the two-step pushing structure of a half press and a full press. In this case, the main photographing may be performed after a predetermined time has elapsed since the pre-photographing was performed in response to the pressing operation to determine the light emission amount.

  Based on the actual image and the depth map acquired by the pre-shooting, the determination unit 26 is a virtual reflected light amount obtained when the subject corresponding to the distance is illuminated by the built-in flash 17, that is, virtual subject luminance information (hereinafter, “ And the amount of light emitted from the built-in flash 17 is determined so that the luminance value of the area including the subject of interest becomes a predetermined luminance value based on the generated virtual luminance information. The determination unit 26 is an example of a determination unit.

  Specifically, as shown in FIG. 2, the determination unit 26 includes a reduced luminance map creation unit 27, a reduced depth map creation unit 28, an illumination luminance map creation unit 29, a virtual luminance map creation unit 30, and a light emission amount determination unit. 31 is included. In these configurations, when the determination unit 25 determines that the subject is a low-luminance subject, the light emission amount is determined according to the procedure shown in FIG.

  First, when determining that the subject is a moderate subject, the CPU 22 operates the depth map acquisition unit 12 to acquire a depth map, and stores the acquired depth map in the RAM 14 (S-1). At this point, the actual image is stored in the RAM 14 in response to the pre-shooting.

  The reduced luminance map creating unit 27 reads the actual image from the RAM 14, and for each region obtained by dividing the effective image region of the read actual image, the reduced luminance map in which the average luminance value obtained by averaging the luminance values of each pixel is distributed. A map is created based on the actual image (S-2). Here, FIG. 4A shows an actual image, FIG. 4B shows a depth map corresponding to the actual image, and FIG. 4C shows a reduced luminance map created based on the actual image.

  For each area corresponding to each region, the reduced depth map creation unit 28 obtains a reduced depth map in which the average depth value obtained by averaging the depth values of each pixel is distributed for each area (FIG. 4 ( D) (see S). In FIG. 4C and FIG. 4D, the reduced map is shown enlarged.

  The illumination brightness map creating unit 29 illuminates the built-in flash 17 with the basic light emission amount included in the light emission information based on the light emission information (information such as the light emission amount and the irradiation angle) of the built-in flash 17 and the reduced depth map. A virtual illumination luminance map (see FIG. 4E) is created (S-4) in which the amount of reflected light from the subject that is sometimes obtained is calculated for each area. At this time, a table of luminance values using the light emission amount and the distance as parameters is stored in the ROM 13 in advance, and the calculation for creating the illumination luminance map can be simplified by reading and using the table.

  The virtual brightness map creating unit 30 creates a virtual brightness map (see FIG. 4F) by combining the illumination brightness map and the brightness map (S-5). The virtual luminance map is virtual subject luminance information in which the amount of reflected light from the subject obtained when the built-in flash is emitted in an actual environment (under steady light) is calculated for each region.

  Then, the light emission amount determination unit 31 determines the light emission amount of the built-in flash 17 based on the virtual luminance map so that the luminance value of the region 32 (see FIG. 4F) including the subject of interest becomes a predetermined luminance value. . In this operation, the luminance value of the region 32 including the subject of interest is extracted from the virtual luminance map (S-6), and it is determined whether or not the extracted luminance value is a predetermined luminance value determined in advance (S-7). ). If the extracted luminance value is not a predetermined luminance value, the illumination luminance map is recreated by increasing / decreasing the light emission amount to be set again with respect to the reference light emission amount (S-8). A virtual luminance map is re-created by combining the map (S-9), the luminance value of the region 32 including the subject of interest is extracted from the re-created virtual luminance map (S-6), and the extracted luminance value is a predetermined luminance value. It is determined whether or not (S-7). When the extracted luminance value is a predetermined luminance value, the light emission amount set at this time is determined as the light emission amount at the time of actual photographing (S-10).

  The predetermined luminance value may be a value having an allowable range. In this case, it is only necessary to determine whether or not the luminance value of the region 32 including the subject of interest is within the allowable range. Here, it is preferable that the region 32 including the subject of interest is the region having the highest luminance value among the regions of the virtual luminance map.

  It should be noted that the real image is displayed on the display unit 20 after the pre-shooting, and the subject of interest or the region including it is selectively selected using the touch panel 21 on the displayed real image. Also good. The touch panel 21 is an example of a selection unit.

  Further, it is preferable that the reference light emission amount used when the illumination luminance map is first created is set lower than the light emission amount set later. This is because it is more difficult for the brightness value of any area of the virtual brightness map to become high brightness (out-of-brightness) if the light emission amount is gradually increased and the virtual brightness map is re-determined later. This is because the determination can be made quickly. Conversely, the maximum light emission amount of the built-in flash 17 may be set as a reference light emission amount, and the light emission amount may be gradually decreased later.

[Second Embodiment]
As shown in FIG. 5, the second embodiment is an example of an imaging system 48 that performs flash shooting on a person 47 by disposing the flash device 45 at a position away from the electronic camera 46. Reference numeral 49 denotes a reflection surface of an object that reflects flash light. The flash device 45 is an example of illumination means.

  As shown in FIG. 6, the electronic camera 46 includes a transceiver 50 that performs wireless communication with the flash device 45 in addition to the configuration described in FIG. 1. The flash device 45 includes a light emitting unit 51, a transmission / reception unit 52, a position detection unit 53, a direction detection unit 54, a light emission information storage unit 55, and a control unit 56 that comprehensively controls them. The position detection unit 53 includes, for example, a GPS (Global Positioning Satellite) sensor that receives radio waves from a geodetic satellite and obtains position information indicating the measured current position. The control unit 56 transmits the position information obtained from the position detection unit 53 to the electronic camera 46 via the transmission / reception unit 52 in response to a request from the electronic camera 46.

  The direction detection unit 54 includes, for example, an electronic compass and an acceleration sensor. The electronic compass is composed of a magnetic element such as a Hall element, detects geomagnetism to detect the azimuth, and the acceleration sensor detects inclination (elevation angle). In response to a request from the electronic camera 46, the control unit 56 transmits information on the light emission direction of the light emitting unit 51 obtained from the direction detection unit 54 to the electronic camera 46 via the transmission / reception unit 52.

  The light emission information storage unit 55 stores light emission unit information such as a light emission amount (guide number) and an irradiation angle in advance. In response to a request from the electronic camera 46, the control unit 56 reads the light emitting unit information from the light emitting information storage unit 55 and transmits it to the electronic camera 46 via the transmission / reception unit 52. In this embodiment, position information, light emission direction information, and light emission unit information are examples of light emission information.

  The control unit 56 controls the light emission of the light emitting unit 51 based on the transmission of information on the light emission amount determined by the electronic camera 46, a trigger signal, and the like via the transmission / reception unit 52. The flash device 45 and the electronic camera 46 are connected wirelessly, but may be connected by wire instead.

  The CPU 22 of the electronic camera 46 includes a determination unit 57. As shown in FIG. 7, the determining unit 57 includes a reduced luminance map creating unit 27, a reduced depth map creating unit 28, a reflected light luminance map creating unit 60, an illumination luminance map creating unit 61, a virtual luminance map creating unit 30, and a light emission. A quantity determination unit 31 is included. Here, in 2nd Embodiment, the reflected light luminance map preparation part 60 is provided compared with the structure demonstrated in FIG. The reflected light luminance map creating unit 60 obtains the reflectance of the reflecting surface 49 of the object based on the reduced luminance map and the reduced depth map, and creates a reflected light luminance map. The illumination brightness map creation unit 61 creates an illumination brightness map based on the reduced brightness map, the light emission information, and the reflected light brightness map. Note that the same components as those described in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

  If the determination unit 25 determines that the subject is a low-luminance subject by pre-shooting, the determination unit 25 determines the light emission amount according to the procedure shown in FIG. In the second embodiment, compared to the procedure described with reference to FIG. 3, the reflected light luminance map creating unit 60 obtains the reflectance of the reflecting surface 49 of the object based on the reduced luminance map and the reduced depth map, and reflects the reflected light luminance. A procedure for creating a map (S-11) is included. Specifically, the reflective surface 49 is specified by smoothing the reduced depth map according to the similarity of the distance, and for the reduced luminance map, each range of the specified reflective surface 49 is specified. Further, the reflectance of each reflecting surface 49 is specified by performing smoothing according to the similarity of colors.

  The illumination brightness map creation unit 61 transmits a request for light emission information to the flash unit 45, and the light emission information (information such as light emission position, light emission direction, light emission amount, irradiation angle, etc.) received in response to the request is generated first. Based on the reduced depth map and the reflected light luminance map, the amount of light reflected from the subject illuminated by the direct light that directly reaches when the flash device 45 emits light and the reflected light that indirectly reaches the object reflected by surrounding objects is calculated. A virtual illumination brightness map calculated for each region is created (S-12). Thereafter, in the same manner as described in the first embodiment, a virtual luminance map is created, and an illumination luminance map is set so that the luminance value of an area including the subject of interest obtained from the virtual luminance map becomes a predetermined luminance value determined in advance. The amount of light emission to be set when creating the image is obtained.

  In the second embodiment, a reflected light luminance map is created, and a virtual luminance map is created based on three lights of steady light, illumination light, and reflected light, but the reflected light luminance map creating unit is omitted. Alternatively, it may be made of only steady light and illumination light. In addition, the reflected light luminance map creation unit described in the second embodiment is used in the first embodiment, and the virtual luminance map is generated based on the three lights of the steady light, the illumination light, and the reflected light in the first embodiment. You may make it.

  In each of the above embodiments, the electronic camera 10 is described, but a camera-equipped mobile phone, a camera-equipped electronic device, or the like may be used. Further, the internal flash 17 using a flash light source that generates flash light by discharge light emission and the flash device (external flash) 45 are described. However, the illumination means of the present invention is not limited to this, and an LED (Light Emitting Diode). ) May be used. A plurality of illumination means may be used. In this case, a master flash and a single or a plurality of slave flashes that emit light in synchronization with the main light emission from the master flash may be used.

  In each of the above embodiments, the real image (luminance map) and the depth map are reduced to simplify the calculation for creating the illuminance luminance map and the virtual luminance map, but the present invention is not limited to this. , It does not have to be reduced. In each of the above embodiments, the bilinear method of obtaining the reduced map by averaging the luminance values and depth values of the regions is used. However, the bicubic method may be used instead.

  In each of the above embodiments, when the subject brightness is a low brightness subject, the light emission amount for performing flash photography is determined, but the present invention is not limited to this. For example, when the daytime synchro photography mode is selected, It may be configured to perform flash photography by determining the light emission amount regardless of the subject luminance.

  In each of the above embodiments, the actual image and the depth map are always stored in the RAM for one frame before the shutter release, and the actual image and the depth map stored before the shutter release are stored in the RAM. The light emission amount may be determined by reading out from.

  In each of the above embodiments, the electronic camera 10 in which the imaging unit 11 and the depth map acquisition unit 12 are integrally provided is described. However, either or both of the imaging unit 11 and the depth map acquisition unit 12 are separately provided. The provided structure may be sufficient. In this case, a configuration may be adopted in which one or both of the actual image and the depth map are input from the outside, provided separately. Moreover, you may provide the imaging part 11, the depth map acquisition part 12, and an illumination means separately. In this case, as the present invention, the imaging unit 11, the depth map acquisition unit 12, and the illumination control unit (CPU 22) are omitted, and the actual image and the depth map are externally input. It is good also as an illumination light quantity determination method and apparatus which determine an illumination light quantity based on.

  Although the present invention has been specifically described above based on the preferred embodiments, the present invention is not limited to the configurations described in the above embodiments, and various modifications can be made without departing from the scope of the invention. Needless to say.

10, 46 Electronic camera 17 Built-in flash 26, 57 Determining unit 45 Flash device

Claims (8)

Real image acquisition means for capturing a subject and acquiring a real image;
3D information acquisition means for acquiring 3D information including depth information according to the actual image;
A determining means for determining an illumination light amount of an illuminating means for illuminating the subject before photographing based on the real image and the three-dimensional information;
An illumination control means for controlling the illumination of the illumination means at the time of photographing so that the illumination light amount determined by the determination means;
An imaging apparatus comprising: The imaging device according to claim 1,
The determination unit generates virtual subject luminance information in consideration of the depth information, which is obtained when the illumination unit illuminates the subject, and includes an object of interest based on the generated virtual subject luminance information An imaging apparatus characterized in that the illumination light quantity is determined so that the luminance value of the light becomes a predetermined luminance value. The imaging device according to claim 1 or 2,
The illumination means is provided at a position away from the device,
Illumination information acquisition means for acquiring illumination information related to illumination of the illumination means from the illumination means,
The determination means is determined by adding the illumination information to a condition for determining the amount of illumination light. The imaging device according to claim 3.
The illumination information includes information on light quantity, illumination position, and illumination angle,
The determination unit adds the information on the light amount, the position of illumination, and the illumination angle to the condition to determine the illumination light amount. The imaging device according to any one of claims 1 to 4,
An imaging apparatus comprising: selection means for selecting the subject of interest or a range including the subject of interest on the real image. In the imaging device according to any one of claims 1 to 5,
Reflected light information estimation means for estimating reflection information including the reflection position of the object included in the real image and the reflectance based on the real image and the three-dimensional information,
The determination unit adds the reflection information to a condition for determining the illumination light amount and determines the illumination light amount. The imaging apparatus according to any one of claims 2 to 6,
The determining means includes
Means for creating a luminance map having luminance value information for each area obtained by dividing the area of the actual image based on the actual image;
Means for creating a depth map having distance information for each region based on the three-dimensional information;
Means for calculating a virtual illumination luminance map obtained when illuminated only with the illumination means based on the illumination information and the depth map;
Means for generating the virtual luminance information by combining the illumination luminance map and the luminance map;
An imaging apparatus comprising: Inputting a real image obtained by imaging a subject;
Inputting three-dimensional information including depth information according to the actual image;
Based on the real image and the three-dimensional information, virtual object luminance information considering the depth information, which is obtained when the illumination unit illuminates the subject, is generated, and attention is paid based on the virtual subject luminance information. Determining the illumination light amount of the illumination means so that the luminance value of the region including the subject becomes a predetermined luminance value;
A method for determining the amount of illumination light.