JP2018098670A - Imaging device, control method of imaging device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to carry out natural gradation correction while restraining deterioration in an S/N ratio.SOLUTION: A system control unit (113) determines each of a first correction amount used in a first gradation correction process and a second correction amount used in a second gradation correction process according to photographing conditions. Then, the system control unit (113) controls the first gradation correction process of combining exposure control and gain control of a pixel signal value of an image and the second gradation correction process of reducing the pixel signal value in a region where the pixel signal value of the image is higher than a prescribed value.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus that captures an image, a control method for the imaging apparatus, and a program.

  2. Description of the Related Art Conventionally, a technique for correcting the gradation of an image in order to obtain an image with favorable brightness and contrast is known. In particular, an image shot in a backlight scene where the brightness of the main subject is significantly darker than the background brightness is prone to so-called “whiteout” in which the pixel signal value is saturated. For example, a technique disclosed in Patent Document 1 is known. In the technique described in Patent Document 1, first, at the stage of exposing the imaging element, imaging is performed with exposure that is darker than appropriate exposure so as to suppress the occurrence of overexposure. Then, output luminance conversion is performed so that the luminance value of an area shot with an exposure darker than the appropriate exposure is increased to an appropriate level and brightened. Hereinafter, such exposure control and level correction processing will be referred to as dynamic range expansion correction (D range expansion correction). Further, as disclosed in Patent Document 2, there is also known a correction technique for detecting a whiteout area generated in an image obtained by photographing and reducing the signal value of the whiteout area. Yes. Hereinafter, such a correction process is referred to as a shine correction.

JP 2010-183461 A Japanese Patent No. 4803178

  However, as in Patent Document 1, when D-range expansion correction processing is performed to brighten an image captured with dark exposure, the noise component of the image is also amplified, and the S / N may be reduced. Further, as in Patent Document 2, when the shine correction process is performed to reduce the signal value of the whiteout area, the gradation of the whiteout area has already been lost as information. In some cases, the imaged area may be an unnatural image that looks like a so-called solid painting.

  Therefore, an object of the present invention is to enable natural gradation correction while suppressing a decrease in S / N.

  The present invention is an imaging apparatus that performs gradation correction that suppresses saturation of pixel signal values of a captured image, and first gradation correction that combines exposure control and gain control for the pixel signal values of the image. A first correction unit that performs the second correction unit that performs second gradation correction that reduces a pixel signal value in a region where the pixel signal value of the image is higher than a predetermined value, and a shooting condition, And determining means for respectively determining a first correction amount used for the first gradation correction and a second correction amount used for the second gradation correction.

  According to the present invention, natural gradation correction can be realized while suppressing a decrease in S / N.

It is a block diagram which shows the structural example of a digital camera. It is a flowchart which shows the flow of a gradation correction | amendment. It is a flowchart which shows the flow of D range expansion correction amount determination processing. It is a flowchart which shows the flow of a shine correction process. It is a figure which shows an example of the image image | photographed in the backlight scene. It is a figure which shows an example of the upper limit of D range expansion correction amount according to imaging | photography conditions. It is a figure which shows the example of a setting of the threshold value area used for correction amount calculation.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. The imaging apparatus of the present embodiment can be applied to digital cameras, digital video cameras, various portable terminals such as smartphones and tablet terminals having camera functions, industrial cameras, in-vehicle cameras, medical cameras, and the like. In the present embodiment, a digital camera will be described as an example.

<Configuration of digital camera>
FIG. 1 is a block diagram illustrating a schematic configuration of a digital camera 100 according to the present embodiment. The digital camera 100 includes an imaging lens 101, an aperture and shutter 102, an automatic exposure (AE) processing unit 103, a focus lens 104, an autofocus (AF) processing unit 105, an image sensor 106, as an imaging mechanism. And an A / D converter 107. The digital camera 100 also includes an image processing unit 108, an image recognition unit 109, a format conversion unit 110, and a DRAM 111. The digital camera 100 further includes an image recording unit 112, a system control unit 113, a VRAM 114, a display unit 115, an operation unit 116, a main switch (main SW) 117, and a shooting switch (shooting SW) 118. I have.

  The taking lens 101 has a zoom mechanism. In accordance with an instruction from the AE processing unit 103, the aperture and shutter 102 adjusts the amount of light incident on the image sensor 106 and the length of time of light incident on the image sensor 106, thereby realizing exposure control. The AE processing unit 103 controls the operation of the aperture and shutter 102. The focus lens 104 focuses on the light receiving surface of the image sensor 106 in accordance with a control signal from the AF processing unit 105 to form an optical image. The AF processing unit 105 can also calculate distance information from the digital camera 100 to the subject.

  The imaging element 106 includes a photoelectric conversion element such as a CCD element or a CMOS element, converts an optical image formed on the light receiving surface of the photoelectric conversion element into an electrical signal, and outputs the electrical signal to the A / D conversion unit 107. The A / D conversion unit 107 converts an electrical signal (analog signal) input from the image sensor 106 into a digital signal. The A / D converter 107 also includes a CDS circuit that removes noise from the electrical signal input from the image sensor 106, a nonlinear amplifier circuit that performs nonlinear amplification before converting an analog signal into a digital signal, and the like.

  The image processing unit 108 performs a so-called development process in which image data is output by performing resizing processing and color conversion processing such as predetermined pixel interpolation and image reduction on the digital signal input from the A / D conversion unit 107. Do.

  The image processing unit 108 also has a function of adjusting the image quality of a captured image by performing gradation correction on the input digital signal by processing such as signal level increase / decrease. For example, as a gradation correction function for adjusting the signal level of the image, the image processing unit 108 can increase or decrease the signal level with a uniform amplification factor for the entire image, or convert the signal level according to the magnitude of the original signal level. So-called tone curve (gamma) function. The digital camera 100 according to the present embodiment can realize processing related to level correction (gain correction) in dynamic range expansion correction processing (D range expansion correction processing) described later by this gradation correction function. Furthermore, the image processing unit 108 also has a gradation correction function for performing processing such as addition, subtraction, multiplication, and replacement on the color luminance signal value for a predetermined region in the image. The digital camera 100 according to the present embodiment can realize a shine correction process to be described later by a gradation correction function for a predetermined region in such an image.

  The image recognition unit 109 receives image data that has been appropriately processed by the image processing unit 108. The image recognizing unit 109 has a function of recognizing the brightness state and the focusing state of the input image. Information representing the brightness status of the image recognized by the image recognition unit 109 is sent to the AE processing unit, and information representing the focusing status is sent to the AF processing unit 105. As a result, the AE processing unit 103 realizes exposure control based on information representing the brightness situation, and the AF processing unit 105 implements AF control based on information representing the focus situation. The image recognition unit 109 also has a function of recognizing, for example, the face of a person in the input image and the facial expression of the face. Further, information indicating the recognition result of the brightness state, the in-focus state, the face, and the like by the image recognition unit 109 is also sent to the system control unit 113, and a shooting scene analysis process described later, a shine correction process described later, and a D range are performed. This is used for gradation correction processing such as correction processing.

The format conversion unit 110 performs format conversion of the image data processed by the image processing unit 108 in order to store the image data in the DRAM 111.
The DRAM 111 is an example of a built-in memory capable of high-speed writing / reading processing, and is used as a high-speed buffer for temporarily storing image data, or as a working memory in image data compression / decompression processing.
The image recording unit 112 includes a recording medium such as a memory card that records captured images (still images, moving images, and the like) and an interface thereof.
The VRAM 114 is an image display memory.

  The display unit 115 is, for example, an LCD (Liquid Crystal Display) or the like, and displays an image, a display for assisting operation, a display of a camera state such as a camera operation mode, and the like. Can also be displayed. The display unit 115 includes a monitor and an electronic viewfinder (EVF) provided on the housing rear surface of the digital camera 100.

The main switch 117 is a switch operated when the user turns on / off the power of the digital camera 100. When the user inputs a power-on instruction via the main switch 117, the digital camera 100 supplies power to each unit.
The photographing switch 118 is a release button, and can detect a two-step operation corresponding to the depth to which the button is pressed. An operation signal of the photographing switch 118 is sent to the system control unit 113. The system control unit 113 causes the AE processing unit 103 to perform AE processing and causes the AF processing unit 105 to perform AF processing as shooting preparation operations when the button of the shooting switch 118 is pressed in shallow. Hereinafter, the state in which the button of the photographing switch 118 is shallowly pressed is referred to as SW1 pressing. When the button of the shooting switch 118 is pressed deeper than the SW1 is pressed, the system control unit 113 controls each unit to execute main shooting. Hereinafter, the state in which the operation of pressing the button of the photographing switch 118 further deeper than pressing the SW1 is referred to as SW2 pressing.

  Here, an example of processing from when the digital camera 100 of the present embodiment is turned on until shooting is briefly described. First, when the power is turned on by turning on the power to the main switch 117, the digital camera 100 periodically performs image capturing by the image sensor 106 with a period of 33 ms or the like, and sequentially captures images captured by the periodic image capturing on the display unit 115. The shooting standby state to be displayed is entered. Thereafter, when a shooting instruction is issued by pressing SW2 through SW1 to the shooting switch 118, the digital camera 100 performs a main shooting process by the image sensor 106. Then, the digital camera 100 performs image processing by the image processing unit 108 on the captured image data obtained by the actual shooting processing, and causes the image recording unit 112 to record the image data after the image processing. Thereafter, the digital camera 100 returns to the main photographing standby state again. When the main switch 117 is turned off, the digital camera 100 stops supplying power to each unit.

The operation unit 116 includes buttons, switches, a touch panel, and the like operated by the user, and an interface circuit that generates a signal corresponding to the operation of the buttons, switches, touch panel, and the like. Specifically, the operation unit 116 includes a menu switch for performing various settings such as setting of an exposure correction value and an aperture value at the time of image shooting, a setting at the time of image reproduction, a zoom lever for instructing a zoom operation of the shooting lens, and an operation mode. Selection switch and the like.
The operation mode selection switch is a switch operated when the user selects switching between the shooting mode and the reproduction mode. The shooting modes include various shooting modes such as a fully automatic mode, a landscape mode, a portrait mode, a macro mode, and a sports mode, and the user can select a desired mode from among these by operating the operation mode selection switch. The shooting mode can be selected. When a desired shooting mode is selected from these various shooting modes, the system control unit 113 sets shooting conditions according to the selected shooting mode. Further, in the fully automatic mode, the system control unit 113 analyzes a shooting scene based on, for example, an image recognition result by the image recognition unit 109, and performs backlight shooting, night scene shooting, landscape shooting, portrait shooting, macro shooting, sports, Automatically sets shooting conditions suitable for shooting.

  The system control unit 113 includes a CPU, a ROM, and a RAM (not shown). The CPU of the system control unit 113 performs overall operation control of the digital camera by expanding and executing a program stored in the ROM in a work area of the RAM. The system control unit 113 also controls which mode is used from a plurality of imaging drive modes of the imaging element 106 and sets imaging conditions according to the various imaging modes described above.

  As an example of setting the shooting conditions according to the shooting mode, for example, when backlight shooting is performed, the system control unit 113, for example, ISO sensitivity suitable for a backlight scene such as shooting a person standing with the light source behind. And so on. For example, when sports photography is performed, the system control unit 113 sets photographing conditions such as a shutter speed suitable for a sports scene such as when photographing a person who moves quickly. In addition, for example, when portrait photography or macro photography is performed, the system control unit 113 shoots an aperture suitable for a portrait scene or a macro scene in which the main subject is focused and the background is blurred. Set conditions. According to the present embodiment, these photographing conditions are also used when determining a correction value for a D range expansion correction process and a correction value for a shine correction process, which will be described later.

  Further, when the system control unit 113 analyzes the shooting scene based on the image recognition result by the image recognition unit 109, for example, as described later, the position or area of the region where the pixel signal value is saturated in the image is determined. A process for obtaining either one is performed. The position or area of the region where the pixel signal value is saturated is used when determining a correction value for a D range expansion correction process and a correction value for a shine correction process, which will be described later.

  Next, the gradation correction processing according to the present embodiment, which is executed in the digital camera 100 having the configuration shown in FIG. 1, will be described. The digital camera 100 according to the present embodiment can execute gradation correction by combining either one of the first gradation correction processing and the second gradation correction processing described below, or a combination of both. Has been made.

<First gradation correction processing>
In the first gradation correction process, when shooting a scene in which a low-brightness subject and a high-brightness subject are mixed, exposure control and tone curve setting control are performed to saturate the high-brightness side. This is gradation correction processing that suppresses so-called whiteout. In the present embodiment, such gradation correction is referred to as dynamic range expansion correction (D range expansion correction or D + correction). An example of a scene in which a low-brightness subject and a high-brightness subject are mixed includes a backlight scene. Hereinafter, the first gradation correction process (D range expansion correction process) will be described in detail.

  Usually, an aperture value, a shutter speed, ISO sensitivity, and the like are determined as shooting conditions before actual shooting. At this time, the digital camera 100 determines the exposure according to the luminance information of the captured image in the actual shooting standby state, the presence or absence of a human face in the image, and the like. However, the dynamic range of the image sensor is limited, and no matter how the aperture, shutter speed, and gain are set, for example, in a backlit scene such as when shooting a person standing with the light source behind, the person is dark. Too much or the background is too bright. In the present embodiment, D range expansion correction processing is performed as processing for reducing the phenomenon that the background is too bright.

  FIG. 5A is a diagram illustrating an example of an image 500 when shooting is performed in a typical backlight scene where a person standing at the window is shot, for example. In the case of the backlight scene image 500 as shown in FIG. 5A, the brightness of the scenery area 501 outside the window is too bright with respect to the person area 502. Whiteout occurs in the outer region 501 (that is, the high luminance portion). For this reason, in the D range enlargement correction process, in order to prevent overexposure in the high-luminance portion, the exposure is set to underexposure, and the main photographing is performed, and the luminance of the low-luminance portion of the obtained captured image data is increased. Gamma processing using the tone curve (gamma) set as described above is performed. The description of the regions 503 to 505 in FIG. 5A and FIG. 5B will be given later.

<Second gradation correction processing>
The second gradation correction process is a shine correction process that performs gradation correction that selectively reduces the pixel signal value with respect to a saturated region (out-of-white region) in the image after the main photographing. Hereinafter, the second gradation correction process (shine correction process) will be described in detail.

  Due to the limitation of the dynamic range of the image sensor as described in the description of the first gradation correction processing, whiteout may occur partially in the subject area of the actually captured image. For example, on the face of a person who has been directly exposed to sunlight or illumination light, so-called shine is generated in which the light is strongly reflected by the cheeks or the forehead, and the part is whitened. It is the shine correction process that reduces such shine phenomenon.

  For example, in the case of the image 500 of FIG. 5A, a state in which shine is generated in cheek regions 504 and 505 and a forehead region 503 in a person region 502 by illumination light or the like is shown. In the shine correction process, first, a correction target area such as a face is selected from the photographed image, and a brightness value area (whiteout area) where whiteout may occur is determined from the brightness value in the correction target area. In the shine correction process, the color component ratio of the whiteout area is estimated based on the color component ratio of the surrounding pixels in the area, and the pixel signal value of the whiteout area is subtracted so as to approach the color estimation ratio. . By such processing, the brightness of the area where whiteout occurs is lowered, the color gradation is restored, and the shine is reduced.

<Flow of gradation correction processing>
Next, in the digital camera 100 of the present embodiment, the overall flow for obtaining an image with a more appropriate gradation using the first and second gradation correction processes described above is shown in FIGS. This will be described with reference to a flowchart.
FIG. 2 is a flowchart showing a flow of the entire process from taking an image in the digital camera 100 to recording (saving). The processing of each step shown in the flowchart of FIG. 2 is realized, for example, when the system control unit 113 controls each unit of the digital camera 100 based on a program. Hereinafter, steps S201 to S209 in the flowchart of FIG. 2 are abbreviated as S201 to S209. The same applies to other flowcharts described later.

  The process of the flowchart of FIG. 2 starts when a power-on instruction is input from the user to the main switch 117, the power is turned on to each part, the digital camera 100 is activated, and various devices are initialized and activated. At this time, the digital camera 100 is in a shooting standby state as described above after performing shooting preparation work such as EVF image display on the display unit 115.

  When the processing of the flowchart of FIG. 2 starts, the system control unit 113 determines in step S201 whether or not the user has pressed SW1 of the photographing switch 118. The system control unit 113 continues the determination of S201 while SW1 is not pressed (No), and proceeds to S202 when it is determined that SW1 is pressed (Yes).

  In step S <b> 202, the system control unit 113 determines a temporary exposure value via the AE processing unit 103 based on the result of the image recognition processing by the image recognition unit 109 (brightness state recognition result). The temporary exposure value at this time is determined as an exposure value not considering the D range expansion correction process. Here, the image recognition unit 109 first divides a captured image obtained by the image sensor 106 into a plurality of blocks with a fineness of about 24 pixels in the vertical direction and about 16 pixels in the horizontal direction. Find the luminance value. Next, the image recognition unit 109 calculates an average value (weighted average value) obtained by weighting the luminance value for each block with a weight set for each block position. Then, the AE processing unit 103 obtains an exposure value whose weighted average value approaches a predetermined value, and determines it as a temporary exposure value. By such processing, an appropriate exposure value is determined for a subject whose brightness is close to the average luminance of the entire screen. However, for example, when shooting a scene where the subject is dark and the background is bright, such as the backlight scene described above, the temporary exposure value determined as described above is an exposure value that is too high for the high-luminance region, Whiteout may occur. After S202, the system control unit 113 advances the process to S203.

  In step S203, the system control unit 113 determines a correction amount for the D range expansion correction process. The details of the processing for determining the D range expansion correction amount will be described later with reference to the flowchart of FIG. After S203, the system control unit 113 advances the process to S204.

  In step S <b> 204, the system control unit 113 determines the main exposure value via the AE processing unit 103. The actual exposure value is determined as an exposure value that is lower than the temporary exposure value determined in S202 by an amount corresponding to the D range expansion correction amount determined in S203. After S204, the system control unit 113 advances the process to S205.

  In step S205, the system control unit 113 determines whether the user has pressed SW2 of the shooting switch 118. The system control unit 113 continues the determination of S205 while the SW2 is not pressed (No), and proceeds to S206 when it is determined that the SW2 is pressed (Yes).

  In step S206, the system control unit 113 controls each unit so as to perform main photographing based on the main exposure value determined in step S204. As a result, an image obtained by actual photographing is acquired from the image sensor 106 via the A / D converter 107. For example, when the above-described backlight scene is shot, a real shot image in which the subject is underexposed is acquired. After S206, the system control unit 113 advances the process to S207.

  In step S207, the system control unit 113 causes the image processing unit 108 to perform gain correction based on the D range expansion correction processing. Specifically, for example, when the above-described backlight scene is shot, the image processing unit 108 sets the luminance of the low-brightness region that is underexposed as described in the explanation of the first gradation correction processing. Gamma processing using a tone curve that lifts is performed. After S207, the system control unit 113 advances the process to S208.

  In step S <b> 208, the system control unit 113 performs a shine correction process via the image processing unit 108. The specific content of the shine correction process will be described later with reference to the flowchart of FIG. After S208, the system control unit 113 advances the process to S209.

  In step S209, the system control unit 113 causes the image recording unit 112 to record the image data obtained by the main shooting finally obtained by the processing in steps S201 to S208. After S209, the system control unit 113 returns the process to S201, and proceeds to a process for the next shooting.

<D range expansion correction amount determination flow>
The D range expansion correction amount determination process performed in S203 of FIG. 2 will be described below. FIG. 3 is a flowchart showing the flow of the D range expansion correction amount determination process performed under the control of the system control unit 113. When the processing proceeds to S203 in FIG. 2, the system control unit 113 starts the processing in the flowchart shown in FIG.

  In step S <b> 301, the system control unit 113 determines whether or not whiteout occurs based on the result of the image recognition process by the image recognition unit 109 (brightness state recognition result). At this time, the image recognizing unit 109 is fine with 24 pixels in the vertical direction and about 16 pixels in the horizontal direction for each image periodically shot with the temporary exposure value determined in S202 in FIG. Now, a process of dividing into a plurality of blocks is performed. Further, the image recognition unit 109 obtains a distribution of luminance values of each pixel in the block (cumulative frequency of each luminance value) for each block. Then, the image recognizing unit 109 determines that a block in which a luminance value with a cumulative frequency greater than a predetermined value is equal to or greater than a predetermined threshold in the distribution of luminance values obtained for each block is a whiteout block. To do. After S301, the system control unit 113 advances the process to S302.

In step S302, the system control unit 113 determines a D range correction amount (D + correction amount). Specifically, the system control unit 113 first obtains an area HL_Area1 that is the sum of the areas of each block determined to be a whiteout block in S301, and compares the total area HL_Area1 with a preset threshold area. . In the present embodiment, the lower limit area TH_DpAreaL and the upper limit area TH_DpAreaH are set in advance as threshold areas used in determining the D range correction amount. The lower limit area TH_DpAreaL is set as an area smaller than the upper limit area TH_DpAreaH. In the present embodiment, the lower limit area TH_DpAreaL and the upper limit area TH_DpAreaH are used and compared with the total area HL_Area1 as shown in the following equation (1).
TH_DpAreaL <HL_Area1 <TH_DpAreaH Formula (1)

  When satisfying the expression (1), the system control unit 113 provisionally determines the D range expansion correction amount based on the average luminance value of each block determined to be a whiteout block in S301. For specific processing for obtaining the D range expansion correction amount based on the average luminance value of each block, a known technique described in Patent Document 1 or the like can be used. Omitted.

  In the present embodiment, the system control unit 113 sets the D range expansion correction amount to zero (0) when, for example, the total area HL_Area1 is equal to or larger than the upper limit area TH_DpAreaH. Similarly, the system control unit 113 sets the D range expansion correction amount to zero (0) even when the total area HL_Area1 is equal to or less than the lower limit area TH_DpAreaL. Note that the D range expansion correction amount of zero (0) means that the D range expansion correction processing is not performed. Specific examples of the lower limit area TH_DpAreaL and the upper limit area TH_DpAreaH will be described later with reference to FIG. After S302, the system control unit 113 advances the process to S303.

  In step S303, the system control unit 113 determines shooting conditions. Specifically, the system control unit 113 determines ISO sensitivity, shutter speed, and aperture values based on the temporary exposure value determined in S202 of FIG. After S303, the system control unit 113 advances the process to S304.

  In step S304, the system control unit 113 determines the D range expansion correction amount based on the shooting condition obtained in step S303. Specifically, when the D range expansion correction amount provisionally determined in S302 exceeds the upper limit value of the D range expansion correction amount provided for each shooting condition shown in FIG. 6, the system control unit 113 sets the upper limit value. The correction amount lowered to is determined as the D range expansion correction amount thus determined. After S304, the system control unit 113 advances the process to S204 in FIG.

  Here, the example of FIG. 6 is a diagram showing the relationship between the ISO sensitivity of the imaging condition and the D range expansion correction amount (D + correction amount) expressed in APEX (Additive System of Photographic Exposure). For example, when the ISO sensitivity is “˜100”, “2.0” of the D + correction amount (APEX stage number) corresponds to the upper limit value, and the D range expansion correction amount temporarily determined in S302 is from “2.0”. If it is larger, the value lowered to “2.0” is determined as the D range expansion correction amount. Therefore, in the case of this example, in the main exposure determination process in S204 of FIG. 2, the ISO sensitivity is set to “˜100”, and at least one of the shutter speed and the aperture value is adjusted so that “2. A correction amount corresponding to “0” is determined. Further, in S207 of FIG. 2, correction processing by gain control using a tone curve determined in advance according to “2.0” of D + enlargement correction amount (APEX stage number) is performed.

<Legacy correction processing flow>
Subsequently, the shine correction process performed in S208 of FIG. 2 will be described. FIG. 4 is a flowchart showing the flow of the shine correction process performed under the control of the system control unit 113. When the process proceeds to S208 in FIG. 2, the system control unit 113 starts the process of the flowchart shown in FIG.

  In step S401, the system control unit 113 sets a partial area corresponding to a human face in the main captured image (hereinafter referred to as a face area) based on the face recognition result of the main captured image by the image recognition unit 109. It is determined whether or not there is. If it is determined in S401 that there is no face area (No), the system control unit 113 ends the shine correction process in FIG. 4 and advances the process to S209 in FIG. On the other hand, if the system control unit 113 determines in S401 that there is a face area (Yes), the system control unit 113 advances the process to S402.

  In step S402, the system control unit 113 controls the image recognition unit 109 to calculate the brightness value of the face area. At this time, the image recognition unit 109 calculates the luminance value of each pixel included in the face area in the actual captured image. After S402, the system control unit 113 advances the process to S403.

  In step S403, the system control unit 113 performs a labeling process based on the luminance value of the face area. Specifically, the system control unit 113 determines a pixel whose luminance value exceeds a predetermined value for each pixel in the face area calculated in S402, and sets the pixel as a whiteout pixel. Furthermore, the system control unit 113 determines whether each pixel adjacent to the whiteout pixel in the vertical and horizontal directions is also a whiteout pixel. Then, the system control unit 113 performs labeling by determining an area where the pixels determined to be whiteout pixels collectively exist as a whiteout connection area. After S403, the system control unit 113 advances the process to S404.

  In step S404, the system control unit 113 calculates the area of each of the plurality of whiteout connected regions labeled in step S403. That is, according to the process of S403 and the process of S404, for example, the total area of each white-out pixel in a positional relationship scattered throughout the entire screen is not calculated, but each has a positional relationship in which each is connected together. The area for each connected region composed of each whiteout pixel is obtained. Then, the system control unit 113 selects a region having the maximum area from among the whiteout connection regions whose areas are respectively determined. Here, the area of the maximum region is HL_Area2. For example, in the case of the image 500 shown in FIG. 5A described above, the cheek regions 504 and 505 and the forehead region 503 in the person region 502 are obtained as labeled over-the-border connected regions (that is, shine areas). It is done. FIG. 5B is a diagram showing a person region 502 extracted from the image 500 of FIG. In FIG. 5B, the region having the largest area among the three regions of the cheek regions 504 and 505 and the forehead region 503, which are the whiteout connection regions (shine areas), is the forehead region 503. . After S404, the system control unit 113 advances the process to S405.

In step S405, the system control unit 113 determines a correction amount. Specifically, the system control unit 113 first compares the area HL_Area2 of the maximum region obtained in S404 with a preset threshold area. In the present embodiment, the lower limit area TH_ShAreaL and the upper limit area TH_ShAreaH are set in advance as threshold areas used in determining the shine correction amount. The lower limit area TH_ShAreaL is smaller than the upper limit area TH_ShAreaH. In this embodiment, the lower limit area TH_ShAreaL and the upper limit area TH_ShAreaH are used and compared with the area HL_Area2 of the maximum region as shown in the following equation (2).
TH_ShAreaL <HL_Area2 <TH_ShAreaH Formula (2)

  Then, when satisfying the expression (2), the system control unit 113 determines the shine correction amount based on the average luminance value of the over-connected region. Note that the shine correction amount based on the brightness value of the overexposed region can be obtained using a known technique described in Patent Document 2 and the like, and detailed description thereof is omitted here.

  In the present embodiment, the system control unit 113 sets the shine correction amount to zero (0) when, for example, the area HL_Area2 of the maximum region is equal to or larger than the upper limit area TH_ShAreaH. Similarly, the system control unit 113 sets the shine correction amount to zero (0) when the area HL_Area2 of the maximum region is equal to or smaller than the lower limit area TH_ShAreaL. If the shine correction amount is zero (0), in other words, the shine correction process is not performed. Specific examples of the lower limit area TH_ShAreaL and the upper limit area TH_ShAreaH will be described later with reference to FIG.

In this embodiment, as described above, when calculating the D range expansion correction amount, the lower limit area TH_DpAreaL and the upper limit threshold TH_DpAreaH are used as the threshold areas. Further, when calculating the shine correction amount, the lower limit area TH_ShAreaL and the upper limit area TH_ShAreaH are used as threshold areas. In the present embodiment, each threshold area used when calculating the D range expansion correction amount and the shine correction amount has a relationship as shown in the following equation (3).
TH_ShAreaL <TH_DpAreaL
TH_ShAreaH <TH_DpAreaH
TH_DpAreaL <TH_ShAreaH Formula (3)

  FIG. 7 is a diagram for explaining the relationship between each threshold area used when calculating the D range expansion correction amount and the shine correction amount. In FIG. 7, each threshold area is expressed as an area ratio in the entire image in order to easily understand the magnitude relationship between the threshold areas. As shown in FIG. 7, the range in which the shine correction process is performed is a range from the lower limit area TH_ShAreaL to the upper limit area TH_ShAreaH, and is converted to an area ratio of the entire image, for example, from 1% to 5%. ing. Further, the range in which the D range expansion correction process is performed is a range from the lower limit area TH_DpAreaL to the upper limit area TH_DpAreaH, which is, for example, a range from 3% to 30% when converted to the area ratio of the entire image. Thus, the lower limit area TH_ShAreaL at the time of calculating the shine correction amount is smaller than the lower limit area TH_DpAreaL at the time of calculating the D range expansion correction amount in terms of the area ratio. On the other hand, the upper limit area TH_ShAreaH at the time of calculating the shine correction amount is larger than the lower limit area TH_DpAreaL at the time of calculating the D range expansion correction amount in terms of an area ratio. Furthermore, the upper limit area TH_ShAreaH at the time of calculating the correction amount is smaller than the upper limit area TH_DpAreaH at the time of calculating the D range expansion correction amount. For these reasons, only the shine correction process is performed in the range from 1% to 3% converted to the area ratio of the entire image, and only the D range expansion correction process is performed in the range from 5% to 30%. On the other hand, in the range from 3% to 5% converted to the area ratio of the entire image, both the D range expansion correction process and the shine correction process are performed. In the range from 0% to 1% or less converted to the area ratio of the entire image, and from 30% to 100%, the correction amount is zero (0) in both the shine correction process and the D range expansion correction process. No correction processing is performed.

Returning to the flowchart of FIG. After S405, the system control unit 113 advances the process to S406.
In step S406, the system control unit 113 controls the image processing unit 108 to perform a shine correction process. In this case, in the image processing unit 108, as described in the description of the second gradation correction process, gradation correction that selectively reduces the pixel signal value with respect to the saturation region (out-of-white region), that is, shine correction. Process. Then, after S406, the system control unit 113 advances the processing to S209 in FIG.

<D-Range Expansion Correction Amount Determination Example According to Shutter Speed Shooting Conditions>
In FIG. 6 described above, the example of determining the D range expansion correction amount according to the ISO sensitivity shooting condition has been described. However, for example, the D range expansion correction amount and the shine correction amount are determined according to the shutter speed shooting condition. Also good. In general, the slower the shutter speed, the more easily the subject image is blurred. When the subject image is blurred, the accuracy of the face recognition result in S401 in the shine correction described with reference to FIG. 4 may be reduced, and a position different from the face may be detected as a face area, or a desired face may not be detected. Yes, there is a possibility that the correction accuracy of the shine correction will decrease. On the other hand, in the D range expansion correction, the entire image is processed, and the area processing as in the case of the shine correction is not performed. Therefore, a desired correction is possible even if the subject image is blurred. Therefore, a determination method may be used in which the correction amount by the D range expansion correction is set to be larger and the correction amount by the shine correction is set to be smaller as the shutter speed of the photographing condition is slower. In the case of this example, the determination of the correction amount for the correction according to the shooting condition of the shutter speed is performed in S405 of FIG. 4, for example. When this method is used, when the shutter speed is slow and there is a possibility that the subject image may be blurred, an erroneous correction such as applying a shine correction process to an area at a position different from the face occurs. It becomes possible to suppress.

<D-Range Expansion Correction Amount Determination Processing Example According to Aperture Shooting Conditions>
Further, the D range expansion correction amount may be determined according to the photographing condition of the aperture value. Generally, as the aperture value is set to the open side, the depth of field becomes shallower, and subjects other than the in-focus position are more easily blurred. When the in-focus position is out of the subject, the accuracy of the face recognition result in S401 in the shine correction described with reference to FIG. 4 is reduced, and a position different from the face is detected as a face area, or a desired face is not detected. There is a possibility that the correction accuracy of the shine correction may be reduced. On the other hand, in the D range expansion correction, since the area processing as in the case of the shine correction is not performed, a desired correction can be performed even if the subject image is blurred. Therefore, a determination method may be used in which the correction amount by the D range expansion correction is set to be larger and the correction amount by the shine correction is set to be smaller as the aperture value of the shooting condition is set to the open side. In the case of this example, the determination of the correction amount of the shine correction according to the shooting condition of the aperture value is performed in S405 of FIG. 4, for example. When this method is used, an area at a position different from the face when the f value of the aperture is small (the aperture value on the open side), the depth of field becomes shallow, and the possibility that the subject is out of focus increases. Therefore, it is possible to suppress the occurrence of erroneous correction that causes the shine correction process.

  As described above, in the digital camera 100 of the present embodiment, the correction amount of at least one of the D range expansion correction and the shine correction is determined according to the shooting conditions. For example, the D range expansion correction amount can be determined as a correction amount in a range that does not exceed the upper limit set according to the ISO sensitivity. In the case of the present embodiment, for example, as shown in FIG. 6, when the ISO sensitivity with a significant decrease in S / N is high, the D range expansion correction amount is set to be low, thereby reducing noise. Increase can be suppressed. On the other hand, when the D range expansion correction amount decreases, the brightness of the captured image increases. In this case, as described above, since the average brightness of the whiteout connection area is high, the correction amount of the shine correction is high, and an image in which whiteout is suppressed is obtained.

  Further, in the present embodiment, as described above, the D range expansion correction amount and the shine correction amount are determined based on the recognition processing result of the photographed image. Processing is performed within a range not exceeding a predetermined area. In the present embodiment, as described above, the area threshold range for the whiteout connected region where the shine correction processing is performed is set to be small, and the area threshold range when performing the D range expansion correction is set to be large. Thus, the following effects can be obtained. In other words, when the area of the whiteout connection region is relatively small, the shine correction process is preferentially performed, so that an increase in noise due to the D range expansion correction can be suppressed. On the other hand, when the area of the whiteout connection area is relatively large, the shine correction process is not performed, and the D range expansion correction process is preferentially performed. Therefore, it is unnatural due to performing the shine correction process on a large area. It is possible to realize natural gradation correction in which the processing trace is not conspicuous.

  A method of determining the D range expansion correction amount and the shine correction amount using conditions other than the above-described example is also conceivable. For example, in the shine correction, as described above, since the process of determining the connection of the overexposed pixels and subtracting the signal value of each pixel is performed, the larger the correction target area, the longer the processing time. For this reason, a method in which only a specific area having a high importance such as a face is set as a processing target is used. According to this method, the processing time can be shortened. On the other hand, since the D range expansion correction process is a process for performing exposure control and uniform gamma processing for the entire image, the size of the correction area does not affect the processing time. Therefore, a method of increasing the D range enlargement correction amount and setting the shine correction amount to a smaller value as more whiteout occurs in other areas except the face is used. By setting the correction amount according to the position of the region where a lot of whiteout occurs in this way, the effect of suppressing an excessive decrease in S / N due to the D range expansion correction while performing the necessary gradation correction Is obtained.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a recording medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. In the present invention, each function of the above-described embodiment can be realized by cooperation of a circuit (for example, ASIC) and a program.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  DESCRIPTION OF SYMBOLS 100 Imaging device, 102 Aperture and shutter, 103 AE processing part, 104 Focus lens, 105 AF processing part, 106 Imaging element, 108 Image processing part, 109 Image recognition part, 113 System control part, 116 Operation part, 118 Shooting switch

Claims (15)

An imaging apparatus that performs gradation correction that suppresses saturation of pixel signal values of a captured image,
First correction means for performing first gradation correction that combines exposure control and gain control for the pixel signal value of the image;
Second correction means for performing second gradation correction for reducing pixel signal values in a region where the pixel signal value of the image is higher than a predetermined value;
Determining means for determining a first correction amount used for the first gradation correction and a second correction amount used for the second gradation correction according to a photographing condition;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the photographing condition includes at least one of ISO sensitivity, shutter speed, and aperture value. The determining means includes
Determine the temporary exposure value based on the image acquired in the shooting standby state before the actual shooting of the image,
Tentatively determining the first correction amount for the main shooting based on the luminance of the image acquired in the shooting standby state of the temporary exposure value;
When the temporarily determined first correction amount exceeds the upper limit value set in accordance with the ISO sensitivity of the photographing condition, the correction amount lowered to the upper limit value is set to the first determined amount. As a correction amount,
3. The imaging apparatus according to claim 1, wherein the first correction unit performs the first gradation correction based on the first correction amount determined in the main.   The imaging apparatus according to claim 1, wherein the determination unit determines the second correction amount to be a smaller correction amount as the shutter speed set as the photographing condition becomes slower.   The imaging apparatus according to claim 4, wherein the determination unit determines the first correction amount to be a larger correction amount as the shutter speed set as the photographing condition becomes slower.   3. The imaging apparatus according to claim 1, wherein the determination unit determines the second correction amount to be a smaller correction amount as the aperture value set as the imaging condition becomes smaller.   The imaging apparatus according to claim 6, wherein the determination unit determines the first correction amount to be a larger correction amount as the aperture value set as the photographing condition becomes smaller. An imaging apparatus that performs gradation correction that suppresses saturation of pixel signal values of a captured image,
First correction means for performing first gradation correction that combines exposure control and gain control for the pixel signal value of the image;
Second correction means for performing second gradation correction for reducing pixel signal values in a region where the pixel signal value of the image is higher than a predetermined value;
An analysis means for analyzing the shooting scene;
Determining means for determining a first correction amount used for the first gradation correction and a second correction amount used for the second gradation correction based on the result of the analysis;
An imaging device comprising:   The imaging apparatus according to claim 8, wherein the analysis by the analysis unit includes a process of obtaining either a position or an area of a region where a pixel signal value is saturated in the shooting scene. The determining means includes
When the area of the region where the pixel signal value is saturated is larger than a predetermined first area, it is determined that the first gradation correction is performed,
When the area of the region where the pixel signal value is saturated is larger than a predetermined second area, it is determined to perform the second gradation correction,
The imaging apparatus according to claim 9, wherein the first area is larger than the second area. The determining unit determines to perform the first gradation correction without performing the second gradation correction when the area of the region where the pixel signal value is saturated is larger than a third area. ,
The imaging apparatus according to claim 10, wherein the third area is larger than the second area. The determining means includes
Determining that the first gradation correction is performed when the area of the region where the pixel signal value is saturated is a predetermined range from the first area to the fourth area;
Determining that the second gradation correction is performed when the area of the region where the pixel signal value is saturated is a predetermined range from the second area to the third area;
The imaging device according to claim 10 or 11, wherein the fourth area is larger than the third area, and the third area is larger than the first area. A method for controlling an imaging apparatus that performs gradation correction that suppresses saturation of pixel signal values of a captured image,
A determination step of determining a first correction amount used for the first gradation correction and a second correction amount used for the second gradation correction according to the shooting conditions;
A first correction step of performing the first gradation correction using exposure control and gain control on the pixel signal value of the image using the first correction amount;
A second correction step in which the second gradation correction for reducing the pixel signal value in a region where the pixel signal value of the image is higher than a predetermined value is performed using the second correction amount;
A control method characterized by comprising: A method for controlling an imaging apparatus that performs gradation correction that suppresses saturation of pixel signal values of a captured image,
An analysis process for analyzing the shooting scene;
A determination step of determining a first correction amount used for the first gradation correction and a second correction amount used for the second gradation correction based on the result of the analysis;
A first correction step of performing the first gradation correction using exposure control and gain control on the pixel signal value of the image using the first correction amount;
A second correction step in which the second gradation correction for reducing the pixel signal value in a region where the pixel signal value of the image is higher than a predetermined value is performed using the second correction amount;
A control method characterized by comprising:   The program for functioning the computer which an imaging device has as each means of the imaging device of any one of Claims 1 thru | or 12.

Images