JP2016092808A - Imaging apparatus and imaging apparatus control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and an imaging apparatus control method which reduce contrast deterioration due to excessive correction in a digital dodging process.SOLUTION: An imaging apparatus 200 comprises: imaging means 201 which images a subject and outputs image data; calculation means which calculates a gain on the basis of the image data and a result of comparing first image data imaged in a first state by strobe means 208 for projecting light at the time of imaging by the imaging means with second image data imaged in a second state by the strobe means; and image processing means 203 which applies the gain calculated by the calculation means to the image data.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an imaging apparatus that performs digital dodging processing on an image and a method for controlling the imaging apparatus.

  Conventionally, as an analog photographic technique, there is a so-called dodging process in which part of an image is covered during printing to obtain a print with appropriate brightness. An imaging apparatus that captures a digital image can also perform digital dodging processing corresponding to such an analog dodging process to partially correct the contrast of the digital image.

  As a conventional technique related to the digital dodging processing technique, for example, the technique disclosed in Patent Document 1 below can be cited. According to Japanese Patent Laid-Open No. 2004-260688, a digital cover that corrects an area having a low brightness level by referring to the brightness level of the image and, when the brightness level is low, multiplying the brightness signal of the input image signal by a large gain related to the dodging process. Baking process is possible.

  Conventionally, as a technique for performing appropriate exposure control according to the reflectance of a subject, a technique disclosed in Patent Document 2 can be cited. According to Japanese Patent Application Laid-Open No. 2004-260688, the difference between the luminance values of image information captured with a light projecting unit that projects a subject such as a strobe light and image information captured without projecting the light projecting unit. Based on this, the reflectance of the subject is measured, and exposure control according to the reflectance of the subject is possible.

JP 2009-260565 A JP 2008-70611 A

  However, since the gain is calculated only in accordance with the luminance level in the above-mentioned Patent Document 1, the gain is increased even for a subject with originally low reflectance, resulting in a decrease in contrast due to overcorrection of the digital dodging process. It was.

  In the prior art disclosed in Patent Document 2, although exposure control is performed according to the reflectance of the subject, partial contrast correction corresponding to the dodging process cannot be performed.

  In view of the above-described problems, the present invention provides an imaging apparatus and an imaging apparatus that reduce a decrease in contrast due to overcorrection of the digital dodging process by adjusting a gain related to the dodging process according to the reflectance of the subject. It aims to provide a control method.

  In order to achieve the above object, an imaging apparatus according to the present invention includes: an imaging unit that images a subject and outputs image data; the image data; and a light projecting unit that projects light at the time of imaging by the imaging unit. A calculation unit that calculates a gain based on a result of comparing the first image data captured in a state with the second image data captured by the light projecting unit in a second state; and calculated by the calculation unit And image processing means for multiplying the image data by the gain obtained.

  According to the present invention, it is possible to appropriately adjust the gain related to the dodging process according to the reflectance of the subject, and a preferable image can be provided to the user.

It is a block diagram which shows the structural example of the dodging process part which concerns on embodiment. It is a block diagram which shows an example of schematic structure of the imaging device which concerns on embodiment. It is a flowchart which shows an example of the process sequence of a dodging process. It is a block diagram which shows an example of the reflectance factor calculation part which concerns on embodiment. It is a characteristic view which shows an example of the relationship of the input-output characteristic of a gain calculation part. It is a flowchart which shows an example of the process sequence of a reflectance factor calculation process. It is a characteristic view which shows an example of the relationship of the input-output characteristic of a reflection coefficient calculation part.

  The best mode for carrying out the present invention will be described below with reference to the accompanying drawings.

  A first embodiment of the present invention will be described.

  FIG. 2 is a block diagram illustrating an example of a schematic configuration of the imaging apparatus according to the embodiment of the present invention. The imaging device referred to in the present invention can be applied to any electronic device that can capture an image. These electronic devices may include, for example, a mobile phone, a game machine, a tablet terminal, a personal computer, a clock-type or glasses-type information terminal.

  An imaging apparatus 200 illustrated in FIG. 2 includes an imaging element 201, an A / D conversion circuit unit 202, an image signal processing unit 203, a switch unit 204, an image memory unit 205, a dodging processing unit 100, a recording unit 206, and a system control unit 207. The flash unit 208 (light projecting means). Note that the flash unit 208 may be detachable from the imaging apparatus 1.

  Next, an imaging operation by the imaging apparatus having the above configuration will be described.

  First, the image is picked up by the image pickup device 201 while being projected by the flash unit 208 by the system control unit 207. The exposure time of the image sensor at this time is set to the same image capturing condition as that during actual image capturing (when the flash unit 208 does not emit light). The output of the image sensor 201 is converted into digital image data by the A / D conversion circuit unit 202. And input to the image processing unit 203.

  The image processing unit 203 performs processing such as color separation, aperture correction, gamma correction, and white balance correction on the input digital image data, and then calculates the luminance (Y) component and the color difference (U, V) component. The image data is converted into YUV format image data.

  Hereinafter, image data in the YUV format output from the image processing unit 203 when the strobe unit 208 is imaged with light emitted is referred to as strobe image data.

  The strobe image data output from the image processing unit 203 is stored in the image memory unit 205 via the switch unit 204.

  Next, the image pickup device 201 captures an image with the system control unit 207 not emitting light from the flash unit 208 or emitting light with a small amount of light (second light amount). The output of the image sensor 201 is converted into digital image data by the A / D conversion circuit unit 202.

  The image data output from the A / D conversion circuit unit 202 is similarly subjected to processing such as color separation, aperture correction, gamma correction, and white balance correction in the image processing unit 203. Thereafter, the image data is converted into YUV format image data having a luminance (Y) component and a color difference (UV) component, and output.

  Hereinafter, YUV format image data output from the image processing unit 203 when the strobe unit 208 is imaged without emitting light will be referred to as main image data.

  The main image data output from the output of the image processing unit 203 is input to the dodging processing unit 100 via the switch unit 204.

  In the dodging processing unit 100, dodging is performed on the main image data input from the image processing unit 203 via the switch unit 204 based on the main image data and the strobe image data stored in the image memory unit 205. Process. Thereafter, the image data in the YUV format after the dodging process is output to the recording unit 205.

  Details of the process of the dodging processor 100 will be described later.

  The image recording unit 205 compresses the input image data in the YUV format into a format such as JPEG, and then records it on a recording medium such as an SD card.

  Note that one or more of the functional blocks shown in FIG. 2 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or by executing software by a programmable processor such as a CPU or MPU. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the following description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  Next, the flow of processing in the dodging processor 100 will be described with reference to the block diagram of FIG. 1 and the flowchart of FIG.

  In step S300, first, the main image data is input from the image processing unit 203 to the gain calculation unit 101, the reflectance coefficient calculation unit 102, and the correction processing unit 104 via the switch unit 204.

  In step S301, the gain calculation unit 101 calculates gain α, which is a correction parameter related to the dodging process, for each pixel of the input main image data.

  Here, FIG. 5 is a diagram illustrating an example of a conversion table when the gain α is calculated by the gain calculation unit 101 illustrated in FIG. 1.

  In FIG. 5, the horizontal axis represents the luminance level Yh (8 bits) of each pixel of the main image data input, and the vertical axis represents the gain α that is the output of the gain calculation unit 101.

  A boundary value Lth500 shown on the horizontal axis in FIG. 5 is a boundary value of the dodging process. When the luminance level Yh of each pixel of the input main image data is larger than the boundary value Lth500, 0 is output as the gain α. When the luminance level Yh of each pixel of the input main image data is lower than the boundary value Lth500, as shown in FIG. 5, the gain is calculated to be larger as the luminance level is lower. Become.

Here, when the luminance level of the target pixel (i, j) of the input main image data is Yh (i, j) and the conversion table shown in FIG. 5 is ohy_tbl, the target pixel (i, j) The gain of α (i, j) is calculated by the following equation.
α (i, j) = ohy_tbl (Yh (i, j)) (Formula 1)

  As shown in (Equation 1) above, the gain α (i, j) for each pixel calculated by the gain calculation unit 102 depends only on the luminance level Yh (i, j) of each pixel of the main image data. To do. Therefore, for example, depending on the positional relationship between the light source and the subject, the gain is applied not only to the region where the subject is shaded by the light source, but also to the subject that originally has a low reflectance, for example, a black subject and the luminance level Yh is low. α (i, j) is greatly calculated.

  In this embodiment, the gain is calculated for each pixel. However, the main image data is divided into at least two small blocks, and the gain α is calculated for the integrated luminance value for each block. Also good.

  In step S 302, the strobe image data stored in the image memory unit 204 is input to the reflectance coefficient calculation unit 102.

  In step S303, the reflectance coefficient calculation unit 102 calculates the reflectance coefficient β for each pixel based on the input main image data and strobe image data.

  Although details of the reflectance coefficient calculation unit 102 will be described later, the reflectance coefficient β (i, j) for the target pixel (i, j) indicates the intensity of the reflectance of the subject of the target pixel (i, j). When the reflectance intensity of the subject is low, 0.0 is output, and when the reflectance intensity of the subject is high, 1.0 is output. The reflectance coefficient β (i, j) is not high, low, or intermediate between the subject of the pixel of interest (i, j), and depends on the reflectance intensity. And outputs a value between 0.0 and 1.0.

  In step S304, the gain adjustment unit 103 calculates a dodging coefficient γ for each pixel based on the gain α calculated by the gain calculation unit 101 and the reflectance coefficient β calculated by the reflectance coefficient calculation unit 102. Is done.

Here, if the dodging coefficient for the pixel of interest (i, j) is γ (i, j), the dodging gain γ (i, j) is gain α (i, j) and the reflectance coefficient β (i, j). ) And is calculated by the following equation.
γ (i, j) = α (i, j) * β (i, j) (Formula 2)

  As shown in (Equation 2) above, the dodging coefficient γ (i, j) for each pixel calculated by the gain adjustment unit 103 is equal to the gain α (i, j) calculated by the gain calculation unit 101. As a result, the reflectance coefficient β (i, j) calculated by the reflectance coefficient calculation unit 102 is multiplied.

  Therefore, when the luminance level Yh of the subject is high and the gain α is 0, the dodging gain γ is also 0.

  Even when the luminance level Yh of the subject is low and the gain α (i, j) is a large value, the reflectance of the subject is low. For example, when the subject is originally a black subject, the reflectance coefficient β is 0.0 or A value close to that. As a result, the dodging gain γ is also 0 or a value close thereto. For this reason, the dodging gain γ has a large value when the luminance level Yh of the subject is low and the gain α (i, j) has a large value, and the reflectance coefficient β is 1.0 or close to it. The time of value. That is, the reflectance of the subject is not low, but when the subject is behind the light source and the brightness level Yh is low, the dodging gain γ for appropriately performing the contrast correction processing is calculated.

  In step S305, the correction processing unit 104 performs correction processing on the input main image data based on the dodging coefficient γ that is an output of the gain adjustment unit 103.

Here, the luminance level after the correction process by the dodging process of the main image data at the target pixel (i, j) is assumed to be Yo (i, j). Also, the luminance level Yo (i, j) is calculated by the following equation, assuming that the luminance level Yh (i, j) of the main image data and the dodging gain γ (i, j).
Yo (i, j) = (1 + γ (i, j)) * Yh (i, j) (Formula 3)

  The luminance level Yo after the dodging process is calculated by the above (Equation 3). Thus, when the dodging coefficient γ is a large value, the luminance level Yo after the dodging process is higher than the luminance level Yh of the main image data before the dodging process. When the dodging coefficient γ is 0, the luminance level Yo after the dodging process is the same value as the luminance level Yh of the main image data before the dodging process.

  That is, if the subject has a low reflectance, but the subject is behind the light source and the luminance level Yh is low, the luminance level Yo after the dodging process is the luminance level of the main image data before the dodging process. It becomes a value larger than Yh. Therefore, it is possible to perform appropriate contrast correction. For a subject with low reflectance, for example, a black subject, the brightness level Yo after the dodging process is the same as or close to the brightness level Yh of the main image data before the dodging process. Thereby, it is possible to reduce the overcorrection due to the dodging process.

  Next, the detailed processing flow in the reflectance coefficient calculation unit 102 will be described with reference to the block diagram of FIG. 4 and the flowchart of FIG.

  In step S <b> 600, the main image data is input from the image processing unit 203 to the gain adjustment unit 401 via the switch unit 204.

  In step S601, gain processing is performed on the main image data input to the gain adjustment unit 401 in accordance with shooting conditions at the time of main image data shooting and shooting conditions at the time of flash image data shooting. For example, if the exposure time when capturing strobe image data is half the exposure time when capturing main image data, the gain is 0.5 times that of the input main image data. Gain shall be applied. Further, it is assumed that gain processing is appropriately performed even when conditions other than the exposure time are different. When the shooting conditions at the time of actual image data shooting and the shooting conditions at the time of flash image data shooting are the same, gain processing is not performed (gain processing is performed with gain 1), and the main image data is output. The output of the gain adjustment unit 401 is input to the movement vector calculation unit 402 and the strobe reflection component extraction unit 404, respectively.

  In step S <b> 602, the strobe image data stored in the image memory unit 205 is input to the movement vector calculation unit 402 and the coordinate conversion unit 403, respectively.

  In step S603, the movement vector calculation unit 402 calculates a movement vector to the coordinates of the subject position on the strobe image data with respect to the coordinates of the subject position on the main image data after gain adjustment, which is the output of the gain adjustment unit 401. calculate.

  Specifically, first, the edge image of the main image data and the edge of the strobe image data from which high frequency components are extracted by the band pass filter processing on the luminance (Y) signal of the main image data and the strobe image data after the gain adjustment, respectively. Extract images. Next, with respect to the edge image of the main image data, the edge image of the strobe image data is moved one pixel at a time in the horizontal direction and the vertical direction, and the difference absolute value of each pixel is integrated. For example, the horizontal movement amount and the vertical method movement amount that minimize the integrated value of the absolute value of each pixel difference between the edge image of the main image data and the edge image of the strobe image data after movement are output as movement vectors. A well-known method is mentioned.

  In step S604, the coordinate conversion unit 403 uses the movement vector output from the movement vector calculation unit 402 so that the coordinates of the subject position of the strobe image data are the same as the position coordinates of the subject of the main image data. Next, alignment (coordinate conversion) is performed.

  In this embodiment, the movement vector calculation unit 402 calculates the movement vector. However, the present invention is not limited to this, and a known technique may be used to divide the main image data and the strobe image data after gain adjustment into small blocks and calculate projective transformation coefficients from a plurality of movement vectors. Similarly, in the coordinate conversion unit 403, the coordinates of the subject position in the strobe image data are the same as the position coordinates of the subject in the main image data, using the projection conversion coefficient calculated in the movement vector calculation unit 402. As described above, coordinate transformation by oblique transformation may be performed.

  In step S605, the strobe reflection component extraction unit 404 extracts the strobe reflection component Sh based on the input main image data and the coordinate-converted strobe image data output from the coordinate conversion unit 403.

Here, it is assumed that the luminance level of the main image data input at the pixel of interest (i, j) is Yh (i, j), and the luminance level of the strobe image data after coordinate conversion is Ys (i, j). The reflection component of the strobe for the pixel of interest (i, j) is calculated by the following equation.
Sh (i, j) = Ys (i, j) −Yh (i, j) (Formula 4)

  As shown in (Equation 4) above, the luminance level Yh (i, j,) of the main image data is obtained from the luminance level Ys (i, j) of the stroboscopic image data after coordinate conversion composed of the strobe light and the reflection component of external light. j) is subtracted to obtain a difference value and set as a strobe reflection component Sh (i, j). Further, the reflection component is not limited to this, and any method may be used as long as it is a result of comparing two image data, such as taking a ratio.

  In step S606, the reflection coefficient calculation unit 405 calculates a reflectance coefficient β indicating the intensity of the reflectance of the subject based on the strobe reflection component Sh output from the strobe reflection component extraction unit 403.

  Here, FIG. 7 is a diagram illustrating an example of a conversion table when the reflectance coefficient β is calculated by the reflection coefficient calculation unit 405 illustrated in FIG. 4.

  In FIG. 7, the horizontal axis represents the strobe reflection component Sh input, and the vertical axis represents the reflectance coefficient β that is the output of the reflection coefficient calculation unit 405. The reflectance coefficient β indicates the intensity of the reflectance of the subject. When the reflectance intensity of the subject is low, 0.0 is output, and when the reflectance intensity of the subject is high, 1.0 is output. In addition, when the reflectance intensity of the subject is neither high nor low, and in between, a value between 0.0 and 1.0 is output according to the reflectance intensity.

  At this time, the threshold 700 shown in FIG. 7 is set so that the reflectance coefficient β is 0.0 with respect to the strobe reflection component Sh where the contrast intensity of the subject is low and the contrast correction by dodging is not required. It is desirable that The inclination 701 is preferably set so that the reflectance coefficient is 1.0 with respect to the strobe reflection component Sh for which the contrast intensity of the subject is high and contrast correction by dodging is desired.

  As described above, in the present embodiment, the intensity of the reflection component of the subject is calculated based on the strobe image data captured with the strobe unit 208 emitting light and the main image data captured without the strobe unit 208 emitting light. Is possible. Also, the coefficient related to the dodging process is adjusted according to the intensity of the reflection component of the subject. As a result, when the subject is dark behind the light source, the contrast can be corrected appropriately. In addition, overcorrection due to the dodging process can be reduced in the case of a subject that originally has a low reflectance, such as a black subject.

  The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention. For example, in the present embodiment, the dodging process is performed in the dodging processing unit 100 after the image processing unit 203 converts the image data into YUV format image data having a luminance (Y) component and a color difference (UV) component. However, the present invention is not limited to this, and the dodging process may be similarly performed on the RGB signals before the gamma correction in the image processing unit 203.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

  In addition, the present invention is not limited to devices such as digital cameras, but includes built-in or external connection of imaging devices such as mobile phones, personal computers (laptop type, desktop type, tablet type, etc.), game machines, etc. It can be applied to any device. Therefore, the “imaging device” in this specification is intended to include any electronic device having an imaging function.

DESCRIPTION OF SYMBOLS 100 Dodge processing part 101 Gain calculation part 102 Reflectivity coefficient calculation part 103 Gain adjustment part 104 Correction processing part 200 Imaging device 201 Imaging element 202 A / D conversion circuit part 203 Image processing part 204 Switch part 205 Image memory part 206 Unit 207 system control unit 208 strobe unit 401 movement vector calculation unit 402 coordinate conversion unit 403 reflection component extraction unit 404 reflection coefficient calculation unit

Claims (9)

Imaging means for imaging a subject and outputting image data;
First gain calculating means for calculating a gain for the image data for adjusting the brightness of the subject based on the image data output from the imaging means;
Based on a comparison between the first image data captured in the first state by the light projecting unit that projects light during imaging by the imaging unit and the second image data captured in the second state by the light projecting unit, Second gain calculating means for calculating a second gain;
An imaging apparatus, wherein a third gain is calculated from the first gain calculated by the first gain calculating means and the second gain calculated by the second gain calculating means.   The first image data is image data captured by projecting the light projecting means, and the second image data is image data captured without projecting the light projecting means. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized.   The second gain calculating means projects the first image data projected and imaged with the first light quantity by the light projecting means, and the second image projected and imaged with the second light quantity by the light projecting means. The image pickup apparatus according to claim 1, wherein the second gain is calculated after aligning data in advance.   The second gain calculating means adjusts the gain based on a difference value between the first image data and the second image data so that the gain decreases as the difference value increases. The imaging device according to claim 1, claim 2, or claim 3. An imaging process for imaging a subject and outputting image data;
A first gain calculating step of calculating a gain for the image data based on the image data output from the imaging means;
Based on the first image data imaged when the light projecting means is in the first state and the second image data imaged when the light projecting means is in the second state, A second gain calculation step of calculating a second gain based on the difference value of the second image data,
An imaging method comprising: calculating a third gain from the first gain calculated in the first gain calculation step and the second gain calculated in the second gain calculation step in the previous period. Control method of the device. Imaging means for imaging a subject and outputting image data;
The image data, the first image data captured in the first state by the light projecting unit that projects light during imaging by the imaging unit, and the second image data captured in the second state by the light projecting unit. A calculation means for calculating a gain based on the comparison result;
And an image processing unit that multiplies the image data by a gain calculated by the calculation unit. An imaging step in which an imaging means images a subject and outputs image data;
The image data, the first image data captured in the first state by the light projecting unit that projects light during imaging by the imaging unit, and the second image data captured in the second state by the light projecting unit. A calculation step of calculating a gain based on the comparison result;
An image processing step of multiplying the image data by a gain calculated by the calculation step.   A computer-executable program in which a procedure of a control method for an imaging apparatus according to claim 7 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the imaging apparatus according to claim 7.

Images