JP2010183298A - Imaging device and control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress a color cast generated on a subject that light of an auxiliary light source does not reach sufficiently with simple constitution when the subject is imaged using the auxiliary light source in a state wherein the color temperature of environmental light is low. <P>SOLUTION: The color temperature of the environmental light is acquired during the imaging using the auxiliary light source (S204). Then a quantity of emitted light and a gain are determined such that when the color temperature of the environmental light is low, the quantity of emitted light of the auxiliary light source is decreased to increase imaging sensitivity (S205). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to an imaging apparatus capable of imaging using an auxiliary light source and a control method thereof.

2. Description of the Related Art Conventionally, an imaging apparatus capable of imaging using an auxiliary light source such as a built-in / external flash is known.
In such an imaging apparatus, the flash light amount, the aperture, and the shutter speed are determined so that the main subject that has detected the in-focus position has proper exposure. Also in the white balance adjustment in the captured image, the white balance of the main subject is adjusted so as to be appropriate.

Japanese Patent Laid-Open No. 01-280735 JP 2006-222637 A

  For example, when multiple subjects such as a group photo are included as subjects, if the appropriate flash light amount is set for the main subject, the flash light amount is insufficient for the person or background located farther than the main subject. Yes, underexposed.

  When the same white balance adjustment as that of the main subject is applied to such a subject that is not sufficiently irradiated with flash light, there is a problem that a large error occurs in white balance and color cast occurs. For example, at the time of imaging in the environment of a low color temperature light source, the face of a person located farther than the main subject becomes reddish.

  Patent Document 1 discloses an image pickup apparatus that detects the color temperature of external light and picks up an image with a flash ON when it is not daylight, thereby realizing appropriate white balance adjustment with only a daylight white balance circuit. Has been proposed. However, the color cast of subjects other than the main subject, such as the background, where the flash light does not reach sufficiently cannot be improved.

  Patent Document 2 proposes an imaging apparatus that is provided with a plurality of spatial modulation elements that can change at least the color temperature of light rays from a light source for each part, and that can illuminate an imaging scene with a different color temperature for each part. However, since a plurality of spatial modulation elements and a projection optical system that forms an image at a plurality of subject positions are required, there is a problem that the apparatus is increased in size and cost.

  The present invention has been made in view of the problems of the prior art. The present invention relates to an imaging device capable of suppressing color cast generated in a subject to which light from the auxiliary light source does not reach sufficiently with a simple configuration when imaging is performed using an auxiliary light source in a situation where the color temperature of ambient light is low, and An object is to provide a control method thereof.

  An object of the present invention is to provide an image pickup apparatus that picks up an image using an image pickup element, a color temperature detection unit that detects the color temperature of ambient light, and a determination unit that determines a light emission amount of an auxiliary light source and a gain of the image pickup element at the time of image pickup. And a control means for controlling the gain of the auxiliary light source and the image sensor so as to perform imaging based on the light emission amount of the auxiliary light source determined by the determining means and the gain of the image sensor, and the determining means is detected by the color temperature detecting means When the ambient light color temperature is lower than the predetermined color temperature, the amount of light emitted from the auxiliary light source is reduced and the gain of the image sensor is reduced compared to the case where the ambient light color temperature is equal to or higher than the predetermined color temperature. Achieved by an imaging device characterized by increasing.

  Further, the above-described object is a method for controlling an image pickup apparatus that takes an image using an image pickup device, and includes a color temperature detection step for detecting a color temperature of ambient light, a light emission amount of an auxiliary light source at the time of image pickup, and a gain of the image pickup device. A color temperature detection step, and a control step for controlling the gain of the auxiliary light source and the image sensor so as to perform imaging based on the light emission amount of the auxiliary light source determined in the determination step and the gain of the image sensor. When the color temperature of the ambient light detected in step 1 is lower than the predetermined color temperature, the light emission amount of the auxiliary light source is reduced in the determination step as compared with the case where the color temperature of the ambient light is equal to or higher than the predetermined color temperature. It is also achieved by a method for controlling an imaging apparatus, characterized by increasing the gain of the imaging element.

  With such a configuration, according to the present invention, when an image is captured using an auxiliary light source in a situation where the ambient light has a low color temperature, the color fog generated in a subject to which the light from the auxiliary light source does not reach sufficiently is suppressed with a simple configuration. It becomes possible to do.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment of the present invention. 5 is a flowchart for explaining an imaging operation of the imaging apparatus according to the first embodiment of the present invention. It is a figure which shows an example of the light emission amount and gain correction | amendment operation | movement of the light emission part 110 which the system control part 118 performs in the imaging device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows typically the condition which is imaging the 1st to-be-photographed object 401, the 2nd to-be-photographed object 402, and the background 403. FIG. It is a figure which shows an example of the light emission amount and gain correction | amendment operation | movement of the light emission part 110 which the system control part 118 performs in the imaging device which concerns on the 2nd Embodiment of this invention.

Hereinafter, exemplary and preferred embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the first embodiment of the present invention.
In FIG. 1, an imaging lens unit 101 is a lens provided with an aperture and a shutter. The focus lens 102 moves according to the control of the imaging unit drive circuit 112, and realizes an autofocus function. The image sensor 103, such as a CMOS image sensor or a CCD image sensor, photoelectrically converts an optical image of a subject and converts it into an analog electrical signal. The CDS / AD processing unit 104 performs correlated double sampling, clamping processing, and analog gain adjustment on the analog electric signal output from the image sensor 103, and then performs A / D conversion to convert it into a digital signal.

  The signal processing unit 107 performs signal processing such as digital gain adjustment, white balance processing, pixel interpolation processing, color signal processing, luminance signal processing, and enlargement / reduction on the digital signal output from the CDS / AD processing unit 104 to capture an image. Generate image data. The signal processing unit 107 stores the generated captured image data in the image memory 105. In addition, the signal processing unit 107 functions as a color temperature detection unit that detects the color temperature of ambient light from the captured image data and supplies the detection result to the system control unit 118. Note that the color temperature of the ambient light is not limited to a method of detecting from the image captured by the image sensor 103, and may be detected using a detection element such as an external light sensor.

  A system control unit 118 controls the operation of the entire imaging apparatus. The system control unit 118 is a CPU, for example, and can control the operation of the entire apparatus using constants, variables, programs, and the like stored in the RAM 119 and / or the ROM 120. The system control unit 118 performs automatic exposure control (AE) by driving an aperture and a shutter included in the imaging lens unit 101 through the imaging unit driving circuit 112. Further, the system control unit 118 performs sensitivity control (gain adjustment) of the image sensor 103 through the image capturing unit drive circuit 112 as necessary in the AE process.

  Further, the system control unit 118 detects the contrast value of the captured image data generated by the signal processing unit 107, and controls the position of the focus lens 102 through the imaging unit drive circuit 112 so that the contrast value is maximized. Automatic focus detection (AF) is performed.

  The recording circuit 108 is an interface with a storage medium 109 such as a memory card or a hard disk. The system control unit 118 reads and writes the storage medium 109 using the recording circuit 108.

  The light emitting unit control circuit 111 controls the light emitting unit 110 that is an auxiliary light source such as a built-in flash based on the control of the system control unit 118.

  The system control unit 118 adjusts the light emission amount by controlling the light emitting unit control circuit 111 in accordance with the results of automatic focus detection and automatic exposure control. The photometry unit 115 measures the luminance of the subject, and the light emitting unit control circuit 111 adjusts the light emission amount of the light emitting unit 110 based on the subject luminance so that a predetermined subject luminance is obtained.

Alternatively, the system control unit 118 may calculate the light emission amount of the light emitting unit 110 at the time of main imaging based on image data captured by causing the light emitting unit 110 to perform preliminary light emission.
Note that the light emitting unit 110 and the light emitting unit control circuit 111 may be configured to be removable from the imaging device (an external flash).

  The display circuit 113 displays the captured image data, the menu screen, and the like on the display device 114 by driving the display device 114 such as an LCD based on the display image data. The display device 114 can also function as an electronic viewfinder (EVF) by performing live view display processing in which imaging is performed continuously by the imaging element 103 and the generated captured image data is sequentially displayed on the display device 114.

SW1 (116) is a switch that is turned on when a release button (not shown) is half-pressed, and SW2 (117) is a switch that is turned on when the release button is fully pressed.
When it is detected that the release button is half-pressed by the user by turning on SW1 (116), the system control unit 118 starts AE and AF processing. When it is detected that the release button is fully pressed by the user by turning on SW2 (117), the system control unit 118 executes main imaging processing based on the AE and AF results.

FIG. 2 is a flowchart for explaining the imaging operation of the imaging apparatus of the present embodiment.
Although not specifically described, it is assumed that the imaging device is operating in the imaging mode, and that an operation for causing the display device 114 to function as an electronic viewfinder is performed. Therefore, the signal processing unit 107 sequentially notifies the system control unit 118 of the color temperature of the ambient light detected from the captured image.

  In S201, system control unit 118 waits for SW1 (116) to turn on. When it is detected that SW1 (116) is turned on, the system control unit 118 performs automatic focusing control and automatic exposure control (S202).

  Then, the system control unit 118 determines whether the light emission unit 110 needs to emit light based on the result of the automatic exposure control in S202 (S203). If it is determined that the light emitting unit 110 does not need to emit light, the system control unit 118 moves the process to S206 and waits for detection of an imaging instruction from the user, that is, SW2 (117) being turned on. Although not shown in FIG. 2, if it is detected in step S206 that SW1 (116) is turned off before SW2 (117) is detected, system control unit 118 returns the process to step S201.

  On the other hand, if it is determined that it is necessary to cause the light emitting unit 110 to emit light during the main imaging as a result of the automatic exposure control, the system control unit 118 proceeds to S204 and acquires the color temperature of the external light (ambient light). As described above, the color temperature of ambient light is detected by the signal processing unit 107 and the result is notified to the system control unit 118. Alternatively, a sensor for detecting the color temperature of the external light may be separately provided, and the system control unit 118 may acquire the detection result by this sensor.

  In step S205, the system control unit 118 determines the light emission amount of the light emitting unit 110 and the gain of the image sensor 103 according to the detection result of the color temperature of the ambient light. Specifically, when the color temperature of the ambient light is low, the light emission amount of the light emitting unit 110 is decreased and the gain (ISO sensitivity) of the image sensor 103 is increased. Thereby, it is possible to suppress the influence of the color cast on the subject at a distance where the light of the light emitting unit 110 as the auxiliary light source does not reach sufficiently.

FIG. 3 is a diagram illustrating an example of the light emission amount and gain correction operation of the light emitting unit 110 performed by the system control unit 118 in the present embodiment.
3A shows the relationship between the light emission amount of the light emitting unit 110 and the color temperature, and FIG. 3B shows the relationship between the gain of the image sensor and the color temperature.

  In the example illustrated in FIG. 3, the system control unit 118 sets the gain of the image sensor 103 to the upper limit of the adjustment range (here, gain 3) until the color temperature of the ambient light exceeds the first predetermined color temperature (T1). .0), the light emission amount of the light emitting unit 110 is corrected so as to achieve proper exposure. That is, the system control unit 118 decreases the light emission amount as compared with the case where the gain of the image sensor 103 is not increased.

Further, the system control unit 118 sets the imaging element 103 so as to obtain a proper exposure by maximizing the light emission amount of the light emitting unit 110 when the ambient light color temperature is equal to or higher than a second predetermined color temperature (T2 or higher). Determine the gain. At this time, the system control unit 118 may determine the light emission amount of the light emitting unit 110 so that the gain of the image sensor 103 is set to the minimum value (1.0 in this case) and proper exposure is obtained. In other words, when the color temperature of the ambient light is equal to or higher than the second predetermined color temperature (T2 or higher), the light emission amount and the gain are not corrected.
In addition, the system control unit 118 controls the light emission amount to be decreased stepwise between the first and second color temperatures and the gain to be increased so that proper exposure can be obtained.

  The specific light emission amount and gain according to the color temperature may be stored in a table in advance, or a calculation formula may be stored, and the color temperature may be substituted into the calculation formula and sequentially calculated. Good. When using a table, the stored value may be only a value corresponding to a discrete color temperature, and values not stored may be obtained by interpolation calculation.

  Note that the method for adjusting the light emission amount and the gain is not limited to the method shown in FIG. 3. If the color temperature is low, the gain can be increased more than when the color temperature is high, and the light emission amount of the auxiliary light source can be suppressed. The light emission amount and gain can be determined according to other relationships.

  When it is detected that SW2 (117) is turned on in S206, the system control unit 118 performs the main imaging operation (S207). At this time, if it is determined that light emission from the light emitting unit 110 is necessary as a result of the automatic exposure control, the system control unit 118 controls the light emitting unit control circuit 111 so that the light emitting unit 110 emits light with the light emission amount determined in S205. To do. Further, the system control unit 118 performs the main imaging through the imaging unit driving circuit 112 with the gain of the imaging element 103 as the gain determined in S205.

  As described above, according to this embodiment, when the color temperature of the ambient light is lower than the predetermined color temperature, the light emission amount is higher than when the color temperature is high and the light emission amount of the auxiliary light source is not corrected. Decrease. It also compensates for increasing the imaging sensitivity by increasing the gain of the imaging device and decreasing the light emission amount of the auxiliary light source. This makes it possible to suppress the color cast that occurs in a subject located at a distance where the light from the auxiliary light source does not reach sufficiently with a simple configuration when imaging using the auxiliary light source in a situation where the ambient light color temperature is low. Become.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
FIG. 4 is a block diagram illustrating a configuration example of the imaging apparatus according to the present embodiment. The imaging apparatus according to the present embodiment is the same in configuration as the imaging apparatus according to the first embodiment except that a face detection unit 106 is newly provided.

  The face detection unit 106 applies face detection processing based on a known face detection technique to the image data output from the CDS / AD processing unit 104 to detect a human face included in the image. As a known face detection technique, a method based on learning using a neural network or the like, template matching is used to search a part having a characteristic shape of eyes, nose, mouth, etc. from an image, and if the degree of similarity is high, it is regarded as a face There are methods. In addition, many other methods have been proposed, such as a method that detects image feature amounts such as skin color and eye shape and uses statistical analysis. In general, a plurality of these methods are combined to improve the accuracy of face detection. Specific examples include a face detection method using wavelet transform and image feature amount described in JP-A-2002-251380. The face detection unit 106 can detect a plurality of faces, and outputs at least the position and size of each detected face area (face area) to the system control unit 118 as a detection result.

  FIG. 5 is a diagram schematically illustrating a situation where the first subject 401, the second subject 402 located farther than the first subject, and the background 403 are imaged. FIG. 5A is a diagram illustrating a distance relationship between the imaging apparatus, the first and second subjects 401 and 402, and the background 403, and FIG. 5B is a diagram schematically illustrating an image to be captured. .

  The system control unit 118 of the imaging apparatus according to the present embodiment acquires a contrast value for each of the face regions 404 and 405 obtained from the face detection unit 106 in the automatic focusing control performed in S202 of the flowchart of FIG. . Then, the distances d1 and d2 to the first and second subjects are obtained from the focal length at which the contrast values of the face areas 404 and 405 are highest. As described above, the system control unit 118 according to the present embodiment can perform distance detection for each subject.

  At this time, the distance to the background 403 may be further acquired. In addition to the method based on the contrast value of the face area image, the distance to the first and second subjects may be obtained using a phase difference sensor (not shown).

  In S205, when determining the light emission amount of the light emitting unit 110 and the gain of the image sensor 103, in addition to the color temperature of the ambient light, at least the distance d1 to the first subject and the distance d2 to the second subject. Consider the difference (d2-d1).

FIG. 6 is a diagram illustrating an example of the light emission amount and gain correction operation of the light emitting unit 110 performed by the system control unit 118 in the present embodiment.
6A illustrates the relationship between the light emission amount of the light emitting unit 110 and the color temperature, and FIG. 6B illustrates the relationship between the gain of the image sensor and the color temperature.

  Compared to the first embodiment shown in FIG. 3, the minimum value of the correction amount of the gain and the light emission amount when the color temperature of the ambient light becomes lower than the second predetermined color temperature (T2) is different. The gain and the amount of light emission are different between the first and second color temperatures.

  When the color temperature of the ambient light falls below the second predetermined color temperature (T2), the greater the difference (d2−d1) between the distance d1 to the first subject and the distance d2 to the second subject, the greater the gain. Increase the minimum increase amount. That is, the value of x in FIG. This is because it is considered that the difference between the light amount reaching the second subject from the light emitting unit 110 and the light amount reaching the first subject increases as the distance difference (d2-d1) increases.

  By determining the light emission amount and the gain in this way, it is possible to effectively suppress the influence of the color cast of a subject located far away. Here, in order to simplify the explanation and understanding, only the case where two face regions are detected has been considered. However, the same control can be applied even when three or more face regions are detected. In this case, in consideration of the gain range that can be handled by the image sensor 103, it is possible to determine the subject for which the difference in distance from the first subject located closest (or the main subject) is to be obtained.

(Third embodiment)
In the imaging apparatus according to the third embodiment having the same configuration as that of the second embodiment, the system control unit 118 performs backlighting based on the brightness of the image data in the face areas 404 and 405 and the background 403 area. It can be determined whether or not there is.

  Specifically, when the average luminance of the face region is equal to or lower than a predetermined first luminance and the average luminance of the background region is equal to or higher than the predetermined second luminance, the system control unit 118 is backlit. Can be determined.

  When it is determined that the light is backlit, the system control unit 118 corrects the light emission amount and gain in consideration of the color temperature of the ambient light and the distance of the subject when determining the light emission amount of the light emitting unit 110 and the gain of the image sensor 103. Can be avoided.

  This is because when shooting in backlight, the color temperature of the ambient light is high during the daytime and is less affected by the color cast. In the morning and evening, the background color temperature is low (that is, red). It is because it is thought that it becomes.

If the timing for performing the backlight determination is before the acquisition of the ambient light color temperature, the process of S204 in FIG. 2 can be omitted.
This embodiment can be combined with the light emission amount and gain correction control in the first embodiment and the second embodiment.

(Fourth embodiment)
In the first and second embodiments, it is possible to determine the imaging mode of the imaging apparatus and not to perform the emission amount and gain correction control according to the mode determination result.

  For example, there is a mode (sometimes referred to as a portrait mode) in which the background of the main subject is blurred by setting shooting conditions that reduce the depth of field. In such an imaging mode, it would be desirable to suppress an increase in image noise by increasing the gain, rather than suppressing the influence of color cast on subjects other than the main subject.

  Therefore, the system control unit 118 determines whether or not the imaging mode is a predetermined imaging mode. If the imaging mode is a predetermined imaging mode, the light emission amount and gain correction control is not performed. Can be.

  The present embodiment can be combined with the light emission amount and gain correction control in the first to third embodiments.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer (or CPU, MPU, etc.) provided in the imaging apparatus.
Therefore, in order to realize the above-described embodiment by a computer, a program itself supplied to the computer also realizes the present invention. That is, the program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A program for realizing the above-described embodiment is supplied to a computer by a recording medium or wired / wireless communication. Examples of the recording medium for supplying the program include a magnetic recording medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magnetomagnetic recording medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a recording medium in which a program for realizing the above-described embodiment is encrypted and distributed is distributed, key information for decryption is supplied to a user who satisfies a predetermined condition, and installation on a computer owned by the user May be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the program for realizing the above-described embodiment may use a function of an OS already running on a computer.
Further, a part of the program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. .

Claims (8)

An imaging device for imaging using an imaging element,
Color temperature detection means for detecting the color temperature of ambient light;
Determining means for determining the light emission amount of the auxiliary light source during imaging and the gain of the imaging device;
Control means for controlling the gain of the auxiliary light source and the image sensor so as to perform imaging based on the light emission amount of the auxiliary light source determined by the determining means and the gain of the image sensor;
When the color temperature of the ambient light detected by the color temperature detection unit is lower than a predetermined color temperature, the determination unit is more than the case where the color temperature of the environmental light is equal to or higher than the predetermined color temperature. An image pickup apparatus that reduces the light emission amount of the auxiliary light source and increases the gain of the image pickup element.   The imaging apparatus according to claim 1, wherein the determination unit decreases the light emission amount of the auxiliary light source and increases the gain of the imaging element as the color temperature of the ambient light is lower. Furthermore, it has a distance detection means for detecting the distance to the subject for each subject,
The determining unit increases the amount of light emitted from the auxiliary light source and the gain of the image sensor when the color temperature of the ambient light detected by the color temperature detecting unit is lower than a predetermined color temperature. 3. The minimum amount is determined so as to increase as a difference in distance between a predetermined main subject and a subject located farther than the main subject increases. Imaging device. A backlight determining means for determining whether the backlight is a backlight;
When the backlight determining unit determines that the backlight is backlit, the determining unit assumes that the color temperature of the ambient light is equal to or higher than the predetermined color temperature and the light emission amount of the auxiliary light source at the time of imaging and the imaging The imaging device according to any one of claims 1 to 3, wherein a gain of the element is determined. Mode determining means for determining whether the imaging mode of the imaging apparatus is a predetermined imaging mode;
When the mode determining unit determines that the imaging mode of the imaging apparatus is the predetermined imaging mode, the determining unit has a color temperature of the ambient light that is equal to or higher than the predetermined color temperature. The imaging apparatus according to claim 1, wherein a light emission amount of the auxiliary light source at the time of imaging and a gain of the imaging element are determined. An image pickup apparatus control method for picking up an image using an image pickup element,
A color temperature detection step for detecting the color temperature of the ambient light;
A determination step for determining the light emission amount of the auxiliary light source during imaging and the gain of the imaging device;
A control step of controlling the gain of the auxiliary light source and the image sensor so as to perform imaging based on the light emission amount of the auxiliary light source determined in the determining step and the gain of the image sensor;
When the color temperature of the ambient light detected in the color temperature detection step is lower than a predetermined color temperature, in the determination step, than when the color temperature of the ambient light is equal to or higher than the predetermined color temperature, A control method for an image pickup apparatus, wherein the light emission amount of the auxiliary light source is decreased and the gain of the image pickup element is increased.   The program for making the computer which an imaging device has perform each step of the control method of the imaging device of Claim 6.   A computer-readable recording medium on which the program according to claim 7 is recorded.

Images