JP2007081550A - Imaging apparatus, and image processing unit, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately correct the brightness of photographed image data regarding an imaging apparatus, an image processing unit, an image processing method, and an image processing program. <P>SOLUTION: A brightness correction section 71 in the imaging apparatus 100 comprises an index calculation section 74 for calculating a natural light index for expressing the degree of outdoor photographing by the natural light of photographed image data, a luminance ratio index for expressing the level of the dynamic range of the photographed image data, and an exposure index for expressing the degree of exposure photographing caused by exposure setting in photographing; a section 75 for calculating the amount of correction of brightness for setting a plurality of temporary correction quantities to the photographed image data, based on the brightness analysis value of photographed image data and the reproduction target value of brightness, setting a mixture coefficient as a weighting factor to be multiplied to each of the plurality of temporary correction quantities based on the natural light index, and calculating the amount of correction of brightness in the photographed image data, based on the mixture coefficient and the plurality of temporary correction quantities; and a brightness correction curve generation section 76 for generating a brightness correction curve for converting the photographed image data, based on the amount of brightness correction and each index calculated by the index calculation section 74. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, an image processing method, and an image processing program that perform brightness correction processing of captured image data.

  Conventionally, a captured image obtained by photoelectrically reading an image acquired by an imaging device such as a digital camera or an image recorded on a photographic film such as a negative film is output (reproduced) by an output device such as a printer. To be done. In this way, when outputting (reproducing) a captured image using an output device, a so-called “dodging” process in which the low luminance portion and the high luminance portion of the image area are separately corrected and the dynamic range of the entire image is compressed. A method for suppressing the collapse of the bright part and the dark part, which occurs at the limit of the reproduction range of the output device by applying, is known (for example, see Patent Document 1).

On the other hand, the brightness of the captured image data is automatically adjusted so that the brightness of the main subject (for example, a human face area) of the captured image data is appropriately finished. In such a brightness adjustment method, there is a method for setting a correction amount and a correction curve for correcting the brightness of the captured image data based on the brightness obtained by analyzing the main subject of the captured image data. It is known (see, for example, Patent Document 2).
JP 2005-72930 A JP 2002-247393 A

  However, if the brightness correction amount and correction curve for correcting the brightness of the captured image data are set based on the brightness of the main subject of the captured image data, the dynamic range is large, such as in backlight or strobe shooting scenes. In the scene, as the brightness correction amount is larger, there is a problem that a bright part and a dark part are crushed, a high-saturation color is saturated and crushed, and a hue is changed.

  Applying the above-described “Dodge” process as a solution to such a problem makes it difficult to maintain the brightness of the human face area appropriately, and the user's preference for the result of the remarkable “Dodge” process also varies. Because of this, improvements were desired. Furthermore, there is no difference in dynamic range between backlight and flash photography scenes, but in the first place the brightness and color type that should be wary of crushing in each scene is determined whether to make the image brighter or darker, that is, the human face There was an essential problem of depending on the accuracy of how the brightness of the area was detected.

  An object of the present invention is to appropriately correct the brightness of captured image data.

In order to solve the above-described problem, an invention according to claim 1 is an imaging apparatus that performs a process of acquiring a photographed image data by photographing a subject and optimizing brightness of a main subject in the photographed image data. And
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
A correction curve generating means for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the photographed image data in accordance with the correction curve generated by the correction curve generation means.

The invention according to claim 2 is an imaging device that performs processing for capturing a captured image data by capturing a subject and optimizing the brightness of a main subject in the captured image data.
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. Suppression coefficient setting means for setting a suppression coefficient for
Multiplication means for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient;
A correction curve generation means for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means.

The invention according to claim 3 is the imaging apparatus according to claim 1 or 2,
The index calculation means divides the photographed image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire photographed image data for each of the divided regions. The first calculation process and the photographed image data are divided into predetermined areas composed of combinations of the distance from the outer edge of the screen of the photographed image data and the brightness, and the ratio of the divided areas to the entire photographed image data Executing at least one of the second calculation processes for calculating the second occupancy ratio indicating
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, It is characterized by calculating an exposure index.

The invention according to claim 4 is an image processing apparatus that performs processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
A correction curve generating means for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means.

The invention according to claim 5 is an image processing device that performs processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue set based on the natural light index, and the saturation of the captured image data is higher than a predetermined threshold. Suppression coefficient setting means for setting a suppression coefficient for
Multiplication means for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient;
A correction curve generation means for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means.

The invention according to claim 6 is the image processing apparatus according to claim 4 or 5, wherein
The index calculation means divides the photographed image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire photographed image data for each of the divided regions. The first calculation process and the photographed image data are divided into predetermined areas composed of combinations of the distance from the outer edge of the screen of the photographed image data and the brightness, and the ratio of the divided areas to the entire photographed image data Executing at least one of the second calculation processes for calculating the second occupancy ratio indicating
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, It is characterized by calculating an exposure index.

The invention according to claim 7 is an image processing method for performing processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation process to perform,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation step, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting step to be set;
A first mixing coefficient setting step for setting a first mixing coefficient as a weighting coefficient by which each of the plurality of first temporary correction amounts is multiplied based on the natural light index calculated by the index calculation step;
A second provisional correction amount setting step for setting a second provisional correction amount for the captured image data based on the first mixture coefficient set by the first mixture coefficient setting step and the plurality of first provisional correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation step, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weight coefficient. A second mixing coefficient setting step for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation process;
A correction curve generating step for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating step, the natural light index, and the luminance ratio index;
A correction step of correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation step.

The invention according to claim 8 is an image processing method for performing processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation process to perform,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation step, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting step to be set;
A first mixing coefficient setting step for setting a first mixing coefficient as a weighting coefficient by which each of the plurality of first temporary correction amounts is multiplied based on the natural light index calculated by the index calculation step;
A second provisional correction amount setting step for setting a second provisional correction amount for the captured image data based on the first mixture coefficient set by the first mixture coefficient setting step and the plurality of first provisional correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation step, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weight coefficient. A second mixing coefficient setting step for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation process;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. A suppression coefficient setting step for setting a suppression coefficient for
A multiplication step of calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation step by the suppression coefficient;
A correction curve generating step for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication step, the natural light index, and the luminance ratio index;
A correction step of correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation step.

The invention according to claim 9 is the image processing method according to claim 7 or 8,
In the index calculating step, the photographed image data is divided into regions each having a predetermined combination of brightness and hue, and a first occupancy ratio indicating a proportion of the entire photographed image data is calculated for each of the divided regions. The first calculation process and the photographed image data are divided into predetermined areas composed of combinations of the distance from the outer edge of the screen of the photographed image data and the brightness, and the ratio of the divided areas to the entire photographed image data Executing at least one of the second calculation processes for calculating the second occupancy ratio indicating
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, It is characterized by calculating an exposure index.

An image processing program according to claim 10 is stored in a computer.
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation function,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation function, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting function to be set;
A first mixing coefficient setting function for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation function;
A second temporary correction amount setting function for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation function, a first weighting factor as a weighting factor that multiplies the second temporary correction amount by at least one of the plurality of first temporary correction amounts. A second mixing coefficient setting function for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation function;
A correction curve generation function for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculation function, the natural light index, and the luminance ratio index;
And a correction function for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation function.

An image processing program according to claim 11 is stored in a computer.
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation function,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation function, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting function to be set;
A first mixing coefficient setting function for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation function;
A second temporary correction amount setting function for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation function, a first weighting factor as a weighting factor that multiplies the second temporary correction amount by at least one of the plurality of first temporary correction amounts. A second mixing coefficient setting function for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation function;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. A suppression coefficient setting function for setting a suppression coefficient for
A multiplication function for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation function by the suppression coefficient;
A correction curve generation function for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication function, the natural light index, and the luminance ratio index;
And a correction function for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation function.

The invention according to claim 12 is the image processing program according to claim 10 or 11, wherein the index calculation function is realized.
A first calculation process that divides the captured image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire captured image data for each of the divided regions; The captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and a second occupation indicating the ratio of the divided area to the entire captured image data Executing at least one of the second calculation processes for calculating the rate,
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, It is characterized by calculating an exposure index.

Next, terms in the present invention will be described.
“Optimizing the brightness of the main subject in the captured image data” means, for example, an output device (for example, a printer) that outputs the captured image data based on the brightness of the human face area that is the main subject of the captured image data. It is meant to be reproduced with the optimum brightness.

  “Natural light” is light that uses the sun as a light source, not an artificial light source such as steady light or flash light (referred to as “strobe”, “flash”, “speedlight”, etc.). Also called daylight. Daylight is a term that refers to an artificial light source with a color temperature (approximately 5500K) close to that of daylight, and is different from “natural light”.

  “Outdoor shooting degree” (also referred to as “natural light index”) is a numerical value that estimates whether or not the shot image data was taken using “natural light” and quantitatively shows the result. . Scenes that are shot using “natural light” and require brightness correction are often backlit scenes, and there is a high probability that a high-luminance area or high-saturation sky blue area is distributed at the top of the screen, and a low-luminance area or skin color or green is distributed at the bottom of the screen. high. On the other hand, for indoor or strobe shooting scenes that were shot without using “natural light”, a high-brightness area or low-saturation skin color area in the center of the screen, and a low-brightness area or low-brightness skin color or high saturation around the screen. Skin color tends to be distributed. Based on such experience, it is possible to quantitatively indicate whether or not the image is taken using “natural light” as “outdoor shooting degree”. It is desirable to use multivariate analysis as a statistical processing method for deriving one tendency from many variables such as the luminance at the top of the screen and the number of pixels in the sky blue hue, the luminance at the bottom of the screen and the number of pixels in the skin color hue.

  The “brightness ratio index that indicates the size of the dynamic range” is a quantitative measure of the difference in brightness between the main subject area and the background area due to the light source conditions at the time of shooting, such as a backlight scene or a flash shooting scene. It is the value shown in. For example, in a backlight scene, the photographer points the camera in the direction of the sun and photographs a person, so that the person face area is dark and the background area is a bright image. That is, if a difference value of brightness between the human face area and the background area is obtained, the difference value can be used as one of the indices. In general, it is known to obtain a difference value by drawing a histogram of image difference image data.

  The “brightness analysis value” of the captured image data is a numerical value indicating the average brightness of the image obtained by examining the distribution state of the pixel values. The “brightness analysis value” is desirably the brightness of the most important subject (main subject) in the captured image data. The search for the human face area is performed by collation in the color system using hue, saturation, and brightness. In order to obtain higher accuracy, it is preferable to use so-called “face detection” in which parts constituting the face, such as the eyes, nose, mouth, and face contour, are collated. In addition, an exposure index representing the exposure shooting degree (under shooting or over shooting) resulting from the exposure setting at the time of shooting may be calculated, and this exposure index may be used as the brightness analysis value. Further, the analysis method of the brightness analysis value is not limited to the analysis of the captured image data. For example, the brightness obtained by analyzing the automatic exposure information of the imaging apparatus and the additional information recorded of the automatic exposure information. It may be.

  The “reproduction target value” is, for example, a numerical value indicating the brightness necessary for the human face area, which is the main subject of the captured image data, to be optimally reproduced by the output device that outputs the captured image data. In other words, if the “brightness analysis value” of the captured image data is a value that approximates the “reproduction target value”, it means that the probability that the output device is optimally reproduced is high.

  The “first provisional correction amount” is a numerical value that temporarily indicates the correction amount necessary to approximate the “brightness analysis value” to the “reproduction target value”. “Set the first temporary correction amount” is to create a one-dimensional LUT (Look Up Table) in which the relationship between the “brightness analysis value” and the “first temporary correction amount” is defined in advance. This means that the “first provisional correction amount” is determined based on the one-dimensional LUT and the “brightness analysis value”. Furthermore, “setting a plurality of first provisional correction amounts” means “calculating brightness analysis values” under different conditions, or setting different “reproduction target values” (defining a plurality of one-dimensional LUTs. Or at least two “first provisional correction amounts” temporarily.

  In the present invention, two one-dimensional LUTs for setting “first provisional correction amount” are defined, assuming brightness correction of a human face area in a backlight scene and a flash photography scene. In addition, in a scene intermediate between a backlight scene and a flash shooting scene, at least one one-dimensional LUT for setting the “first provisional correction amount” is defined assuming brightness correction of the entire image in the under and over shooting scenes. It is desirable.

  In order to sufficiently exhibit the effects of the present invention, when creating a one-dimensional LUT for setting the “first provisional correction amount”, bright and dark portions are crushed, high-saturated colors are saturated and hushed, Prepare a large number of backlight scenes and flash photography scenes that actually cause phenomena such as changes as “learning images” (also called “teacher data”), and observe the brightness of the human face area and the occurrence of these phenomena. It is desirable to do it.

  Based on “brightness reproduction target value” and “brightness ratio index”, “setting the first temporary correction amount” means that the dynamic range of the captured image data is the dynamic range of the output device that outputs the captured image data. This means that the brightness correction amount set based on the brightness of the main subject is weakened and the brightness correction amount is set based on the luminance ratio index so as to be appropriately within the range.

  “Set the first mixing coefficient” based on the natural light index is a one-dimensional LUT that defines the relationship between the “natural light index” and the “first mixing coefficient” by which each first temporary correction amount is multiplied in advance. This means that the number of first temporary correction amounts is created, and the “first mixing coefficient” to be multiplied by each first temporary correction amount is determined based on the one-dimensional LUT and the “natural light index”. Note that the first mixing coefficient may be adjusted based on the luminance ratio index.

  For example, when the natural light index shows a high outdoor photographing value, a one-dimensional LUT (input: flesh color average luminance) for setting the “first provisional correction amount” assuming brightness correction of a human face area in a backlight scene The one-dimensional LUT is defined so that the “first mixing coefficient” of (output: first provisional correction amount) becomes high.

  On the other hand, when the natural light index indicates a low outdoor photographing value, a one-dimensional LUT (input: flesh color average) for setting the “first provisional correction amount” assuming brightness correction of the human face area in the flash photography scene The one-dimensional LUT is defined so that the “first mixing coefficient” of luminance and output: first provisional correction amount becomes high.

  Further, when the natural light index indicates an ambiguous outdoor shooting degree value, a “first provisional correction amount” setting one-dimensional LUT (assuming brightness correction of the entire image in an under shooting scene and an over shooting scene) The one-dimensional LUT is defined so that the “first mixing coefficient” of input: overall average luminance or under-over shooting index, output: first temporary correction amount) becomes high.

  If the captured image data is known as a backlight image in advance, the brightness correction amount and the correction curve can be corrected based on the “brightness ratio index”, and the bright and dark portions of the captured image data may be crushed, or the high saturation It is possible to prevent the color of the color from being saturated and crushed or the hue from changing. However, since the direction of brightness correction is completely opposite between backlight scenes and flash photography scenes even if the dynamic range is about the same, the process to correct the human face area to the appropriate brightness is automated. To apply the first provisional correction amount based on the luminance ratio index indicating the size of the dynamic range in accordance with a predicted numerical value (natural light index) that quantitatively indicates whether there is a backlight scene or a flash photography scene. Must be defined. Therefore, a “first mixture coefficient” of a one-dimensional LUT (input: luminance ratio index, output: first temporary correction amount) for setting “first temporary correction amount”, assuming brightness correction of a human face area in a backlight scene. ”Is defined so as to define a one-dimensional LUT that defines the relationship between the“ natural light index ”and the“ first mixing coefficient ”.

The “second provisional correction amount” is calculated based on the first provisional correction amount and the first mixing coefficient, and is defined as Expression (1). In Expression (1), among the plurality of first temporary correction amounts set by the first temporary correction amount setting means, m first temporary correction amounts are set as calculation targets, and the i-th first temporary correction amount is set as key_auto. [i], the first mixing coefficient to be multiplied by the first temporary correction amount key_auto [i] is wgt [i].

  “Generate a correction curve for converting captured image data based on the brightness correction amount, the natural light index, and the luminance ratio index” is whether or not the scene was captured using “natural light”. The brightness of the main subject (for example, the human face area) of the captured image data is appropriately finished and brightened according to the predicted numerical value (natural light index) that quantitatively indicates and the luminance ratio index that indicates the size of the dynamic range. By changing the shape of the height correction curve, it is possible to prevent the bright part and the dark part from being crushed, the high-saturation color to be saturated and crushed, and the hue from being changed.

  Scenes that are shot using “natural light” and require brightness correction are often backlit scenes, with a negative brightness correction amount. There is a high probability that the hue will change. On the other hand, for indoor or strobe shooting scenes that were shot without using “natural light”, the brightness correction amount is positive, the dark areas are crushed, the high-saturated skin color is saturated, and the hue is changed to yellow or red. Tend to change. Based on such experience, the shape of the brightness correction curve is changed.

  When the natural light index indicates a numerical value of a high outdoor shooting degree, a soft curve on the highlight side is defined on the assumption that the bright part in the backlight scene is crushed or the sky color of high saturation is saturated and crushed. On the other hand, when the natural light index indicates a numerical value of a low outdoor shooting degree, a shadow-side soft curve is defined on the assumption that a dark part in a flash shooting scene is crushed or a high-saturation skin color is saturated and crushed.

  The “predetermined hue” is a hue of a high-saturation pixel that has a high probability of being saturated and crushed or changing to a hue such as cyan or green, and is a sky blue in a backlight scene, and a flesh color in an indoor shooting or strobe shooting scene. The “predetermined hue set based on the natural light index” is to change the hue of the high saturation pixel for counting the number of pixels based on the natural light index.

  The brightness correction amount calculation means analyzes the additional information recorded in the tag area of the photographed image data, and sets a new brightness correction amount by using the analyzed additional information as a supplement. Also good.

  According to the present invention, while correcting the brightness of the main subject in the photographed image data, it is possible to prevent the bright part and the dark part from being crushed, the color saturation due to high saturation, and the hue from being changed.

Hereinafter, Embodiments 1 and 2 of the present invention will be described in detail with reference to the drawings.
[Embodiment 1]
First, a configuration common to the first and second embodiments will be described.

  FIG. 1 shows a main part configuration of an imaging apparatus 100 according to Embodiments 1 and 2 of the present invention. As shown in FIG. 1, the imaging apparatus 100 includes a lens 1, an aperture 2, a CCD (Charge Coupled Device) 3, an analog processing circuit 4, an A / D converter 5, a temporary storage memory 6, an image processing unit 7, and header information. The processing unit 8, the storage device 9, the CCD drive circuit 10, the control unit 11, the operation unit 12, the display unit 13, the strobe drive circuit 14, the strobe 15, the focal length adjustment circuit 16, the automatic focus drive circuit 17, and the motor 20. The

  The optical system of the imaging apparatus 100 includes a lens 1 that can adjust the focus, a diaphragm 2 that adjusts the amount of light, and a CCD 3 that is a solid-state imaging device that converts subject information into an electrical signal.

  The subject information is imaged on the light receiving surface of the CCD 3 through the lens 1 and the diaphragm 2. The CCD 3 photoelectrically converts subject information into an electrical signal of an amount corresponding to the amount of incident light for each sensor in the CCD 3. A timing pulse is output from the CCD drive circuit 10 based on a control signal input from the control unit 11, and an electrical signal (imaging signal) accumulated in the CCD 3 is sequentially output by this timing pulse, and the analog processing circuit 4. Forwarded to

  The analog processing circuit 4 performs analog image pickup signal amplification and noise reduction processing. The A / D converter 5 converts the analog imaging signal input from the analog processing circuit 4 into a digital imaging signal (R, G, B signal). The digital R, G, B signals output from the A / D converter 5 are temporarily stored in the temporary storage memory 6. Hereinafter, the digital imaging signal is referred to as “photographed image data”.

  The image processing unit 7 performs display on the display unit 13, brightness correction of captured image data used for storage in the recording medium of the storage device 9, spectral sensitivity crosstalk correction, noise suppression, sharpening, white balance adjustment, and coloring. In addition to image quality improvement processing such as degree adjustment, processing such as image size change, trimming, and aspect conversion is performed. In the processing in the image processing unit 7, settings such as ON / OFF or an adaptive amount are switched according to operation information from the operation unit 12. For example, when operation information specifying brightness correction is input from the operation unit 12, the image processing unit 7 starts brightness correction processing (see FIGS. 16 and 30) for correcting the brightness of captured image data. To do. The internal configuration relating to the brightness correction by the image processing unit 7 will be described later with reference to FIG.

  The header information processing unit 8 performs a writing process of header information of the shooting information data generated by the image processing unit 7. The R, G, and B signals that have undergone the header information writing process are stored as image data in a recording medium by the storage device 9. The storage device 9 stores captured image image data and a control program for the imaging apparatus 100.

  The control unit 11 is configured by a CPU (Central Processing Unit) or the like, and performs an automatic focus driving circuit 17, a focal length adjusting circuit 16, and a CCD driving circuit for controlling the motor 20 that adjusts the focal length and focus (focus) of the lens 1. 10, the analog processing circuit 4, the temporary storage memory 6, the image processing unit 7, the display unit 13, and the strobe driving circuit 14 that drives the strobe 15 are controlled.

  The operation unit 12 includes buttons, cursor keys, and the like (not shown), and outputs operation information of the operation unit 12 by the user to the control unit 11. The display unit 13 displays captured image data on a liquid crystal display and displays settings and conditions related to shooting.

  FIG. 2 shows an internal configuration of the image processing unit 7. As illustrated in FIG. 2, the image processing unit 7 includes a preprocessing unit 70, a brightness correction unit 71, and an image quality correction unit 72.

  The preprocessing unit 70 performs spectral sensitivity crosstalk correction, dark current noise suppression, and temporary white balance adjustment on the captured image data input from the temporary storage memory 6, and the brightness correction unit 71 and the image quality correction unit 72. Output to.

  The brightness correction unit 71 includes an image analysis unit 73, an index calculation unit 74, a brightness correction amount calculation unit 75, and a brightness correction curve generation unit 76.

  The image analysis unit 73 calculates a brightness analysis value of the captured image data. The brightness analysis value is a numerical value indicating the average brightness of the photographed image data obtained by examining the distribution state of the pixel values of the photographed image data, and the most important subject (main subject) in the photographed image data. It is preferable that the brightness is. As the brightness analysis value, for example, an average value of luminance values of the entire captured image data (hereinafter referred to as “total average luminance”), an average luminance value in the skin color area of the captured image data (hereinafter referred to as “skin color average luminance”). )), And an exposure index (index 6 described later) indicating the exposure shooting degree (under, over) due to the exposure setting at the time of shooting.

  The index calculation unit 74 includes a natural light index (indicator 4 described later) that represents the degree of outdoor photographing using natural light in the captured image data, a luminance ratio index (indicator 5 that will be described later) that represents the dynamic range of the captured image data, An exposure index (index 6 to be described later) representing the exposure shooting degree (under and over) due to the exposure setting at the time is calculated. The index calculation process executed by the index calculation unit 74 will be described in detail later with reference to FIGS.

  The brightness correction amount calculation unit 75 calculates the brightness correction amount of the captured image data based on the brightness analysis value calculated by the image analysis unit 73 and the calculation result of the index calculation unit 74. The internal configuration of the brightness correction amount calculation unit 75 will be described in detail later with reference to FIG. 3 (Embodiment 1) and FIG. 29 (Embodiment 2).

  The brightness correction curve generation unit 76 generates a brightness correction curve for converting photographed image data based on the brightness correction amount calculated by the brightness correction amount calculation unit 75.

  The image quality correction unit 72 corrects the brightness of the captured image data input from the preprocessing unit 70 in accordance with the brightness correction curve generated by the brightness correction curve generation unit 76. The captured image data whose brightness has been corrected is output to the header information processing unit 8 and the display unit 13.

  FIG. 3 shows an internal configuration of the brightness correction amount calculation unit 75 of the first embodiment. The brightness correction amount calculation unit 75 includes a first temporary correction amount setting unit 101, a first mixing coefficient setting unit 102, a second temporary correction amount calculation unit 103, a second mixing coefficient setting unit 104, and a brightness correction amount calculation unit 105. Consists of.

  The first provisional correction amount setting unit 101 uses the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation unit 74 to determine the captured image data. A first temporary correction amount for is set. When n brightness analysis values are input from the image analysis unit 73, the temporary correction amount setting unit 101 uses a first temporary amount as a correction amount necessary to approximate each brightness analysis value to the reproduction target value. Set n + 4 correction amounts.

  FIG. 4 shows a one-dimensional LUT (Look Up Table) that defines the relationship between the luminance ratio index and the first temporary correction amount key_auto [0], and FIG. 5 shows the exposure index and the first temporary correction amount key_auto [1]. FIG. 6 shows a one-dimensional LUT that defines the relationship between the flesh color average luminance and the first temporary correction amount key_auto [2], and FIG. 7 shows the flesh color average luminance or the entire luminance. A one-dimensional LUT defining the relationship between the average luminance and the first temporary correction amount key_auto [3] is shown. FIG. 8 shows a one-dimensional LUT defining the relationship between the luminance ratio index and the first temporary correction amount key_auto [4]. FIG. 9 shows a one-dimensional LUT that defines the relationship between the exposure index and the first temporary correction amount key_auto [5].

  In the first and second embodiments, the brightness analysis values calculated by the image analysis unit 73 are the skin color average luminance and the overall average luminance (that is, n = 2). Using the one-dimensional LUT shown in FIGS. 4 to 9, six temporary correction amounts (key_auto [0] to [5]) are set as the first temporary correction amounts.

  The first mixing coefficient setting unit 102, based on the natural light index calculated by the index calculation unit 74, out of n + 2 first temporary correction amounts set by the temporary correction amount setting unit 101 (key_auto [0 ] To [3]) are set to n + 2 first mixing coefficients as weighting coefficients to be multiplied. 10 to 13 show one-dimensional LUTs that define the relationship between the natural light index and the first mixing coefficients wgt [0] to wgt [3].

  4 to 9 and the one-dimensional LUTs of FIGS. 10 to 13 are stored in a memory (not shown) of the imaging apparatus 100.

The second temporary correction amount calculation unit 103 includes n + 2 first temporary correction amounts (key_auto [0] to [3]) and n + 2 first mixing coefficients wgt [set by the first mixing coefficient setting unit 102. Based on 0] to wgt [3], the second temporary correction amount for the captured image data is set. Formula (2) shows the formula for calculating the second temporary correction amount.
Second temporary correction amount = key_auto [0] × wgt [0] + key_auto [1] × wgt [1] + key_auto [2] × wgt [2] + key_auto [3] × wgt [3] (2)
Henceforth, the 2nd temporary correction amount of Formula (2) is described with key_auto [6].

  The second mixing coefficient setting unit 104 discriminates the scene type of the captured image data from the values of the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation unit 74, using the discrimination map, and the two first Three second mixing coefficients wgt [4] to [6] are set as weight coefficients to be multiplied to the temporary correction amounts key_auto [4] and key_auto [5] and the second temporary correction amount key_auto [6].

  FIG. 14A shows a discrimination map for discriminating the scene type (forward light, backlight, strobe, under, low accuracy region) of the captured image data based on the natural light index and the luminance ratio index. ) Shows a discrimination map for discriminating the scene type (forward light, backlight, strobe, under, low accuracy region) of the captured image data based on the natural light index and the exposure index. In FIG. 14, the low accuracy region represents a region where the accuracy determined as follow light, backlight, strobe, or under is low (lower than a predetermined value).

  FIG. 15 shows a coefficient calculation table representing the values of the second mixing coefficients wgt [4] to [6] for each scene type. This coefficient calculation table is stored in a memory (not shown) of the imaging apparatus 100. In FIG. 15, each number shown in the scene type item is a number assigned to each scene type in FIG. For example, in FIG. 15, “3” of the scene type item represents the low accuracy region (3) in FIG.

The brightness correction amount calculation unit 105 is set by the two first temporary correction amounts key_auto [4] and key_auto [5], the second temporary correction amount key_auto [6], and the second mixing coefficient setting unit 104 Based on the three second mixing coefficients wgt [4] to [6], the brightness correction amount is calculated as shown in Expression (3).
Brightness correction amount = key_auto [4] × wgt [4] + key_auto [5] × wgt [5] + key_auto [6] × wgt [6] (3)

Next, the operation in the first embodiment will be described.
With reference to the flowchart of FIG. 16, the brightness correction process performed in the image processing unit 7 of the first embodiment will be described.

  First, the index calculation unit 74 performs an index calculation process for calculating a natural light index, a luminance ratio index, and an exposure index (step T1). The index calculation process in step T1 will be described in detail later with reference to FIGS.

  When the index calculation process ends, the image analysis unit 73 calculates n brightness analysis values of the captured image data (step T2). In step T2, two brightness analysis values (skin color average brightness, overall average brightness) are calculated. Next, the brightness analysis value calculated in step T2 and the values corresponding to the natural light index, the luminance ratio index, and the exposure index calculated in the index calculation process in step T1 are extracted from the one-dimensional LUT of FIGS. As a result, n + 4 (six) first temporary correction amounts key_auto [0] to [5] are set (step T3).

  Next, n + 2 (four) first mixing coefficients wgt [0] to [3] are extracted from the one-dimensional LUTs of FIGS. 10 to 13 by extracting values corresponding to the natural light index calculated in step T1. Is set (step T4). Next, using the four first temporary correction amounts key_auto [0] to [3] and the four first mixing coefficients wgt [0] to [3], the second temporary correction amount key_auto is obtained from Equation (2). [6] is calculated (step T5).

  Next, three second mixing coefficients wgt [4] to [6] corresponding to the scene type determined using the determination map (FIG. 14) are extracted from the coefficient calculation table (FIG. 15) and set. (Step T6). Next, using the two first temporary correction amounts key_auto [4] and key_auto [5], the second temporary correction amount key_auto [6], and the three second mixing coefficients wgt [4] to [6] Then, the brightness correction amount is calculated by the equation (3) (step T7), and the captured image data is obtained from the calculated brightness correction amount and the scene type determined based on each index calculated in step T1. A brightness correction curve for conversion is generated (step T8).

  FIG. 17A shows a brightness correction curve when the scene type is normal light according to the brightness correction amount, and FIG. 17B shows a brightness correction curve when the scene type is backlight. Shown by correction amount. FIG. 17 shows a brightness correction curve in a practical range (−6, −4, −2, 0 buttons) of the number of correction buttons operated by the operation unit 12. In step T8, a brightness correction curve corresponding to the brightness correction amount calculated in step T7 is selected from a plurality of brightness correction curves set in advance for each scene type.

  When the brightness correction curve is generated, the brightness of the captured image data is corrected according to the brightness correction curve (step T9), and the brightness correction process is terminated. The captured image data whose brightness has been corrected in this way is output to a designated output destination. Examples of the output destination include the display unit 13 and a printer connected to the imaging apparatus 100.

<Indicator calculation process>
Hereinafter, the index calculation process in step T1 will be described with reference to the flowchart of FIG.

  First, an occupancy ratio calculation process is performed in which the captured image data is divided into predetermined image areas, and an occupancy ratio (first occupancy ratio, second occupancy ratio) indicating the ratio of each divided area to the entire captured image data is calculated. Performed (step S1). Details of the occupation ratio calculation process will be described later with reference to FIGS.

  Next, a bias amount calculation process for calculating a bias amount indicating a bias of the gradation distribution of the photographed image data is performed (step S2). The bias amount calculation process in step S2 will be described later in detail with reference to FIG.

  Next, based on the occupation ratio calculated in step S1 and a coefficient set in advance according to the shooting condition (scene type or the like), a natural light index representing the degree of outdoor shooting with natural light of the shot image data, and the shot image data The brightness ratio index representing the size of the dynamic range and the exposure index representing the exposure shooting degree resulting from the exposure setting at the time of shooting are calculated (step S3), and this index calculation process is completed. The index calculation method in step S3 will be described in detail later.

  Next, the first occupancy rate calculation process will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S10). FIG. 20 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 20, the values of digital image data as input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S11). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation based on the knowledge that the contribution to the determination of the scene type (see FIG. 14) is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue areas (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following formula (4) is defined as the flesh-colored area (H1), and the area that does not satisfy the formula (4) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance (Y) = InR x 0.30 + InG x 0.59 + InB x 0.11 (A)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (4)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formula (4).

When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S12), and the main occupancy ratio calculation is performed. The process ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient required for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting, that is, the lightness state of the face area at the time of strobe shooting, for each divided region. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy rate Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 21 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 21, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 23 shows a first coefficient in the skin color area (H1) and a first coefficient in the other area (green hue area (H3)) as a curve (coefficient curve) that continuously changes over the entire brightness. It is shown. According to Table 2 and FIG. 23, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(5-1)〜式(5-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) …(5-1)；
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) …(5-2)；
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) …(5-3)；
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) …(5-4)． When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (5).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (5-1) to (5-4).
H1 region sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) ... (5-1);
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) ... (5-2);
H3 area sum = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) ... (5-3);
Sum of H4 region = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) ... (5-4).

The index 1 is defined as in Expression (6) using the sum of the H1 to H4 regions shown in Expressions (5-1) to (5-4).
Index 1 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 4.424 (6)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy as backlight photographing, that is, the brightness state of the face region at the time of backlight photographing. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy rate Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 22 shows a lightness (v) -hue (H) plane. According to Table 3, in FIG. 22, a negative (−) coefficient is used for the occupation ratio calculated from the area (r4) distributed in the intermediate brightness of the flesh hue area, and the low brightness (shadow) area ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 24 shows the second coefficient in the skin color area (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 24, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(7-1)〜式(7-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) …(7-1)；
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) …(7-2)；
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 …(7-3)；
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 …(7-4)． When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (7).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (7-1) to (7-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) ... (7-1);
Sum of H2 regions = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (7-2);
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (7-3);
Sum of H4 region = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (7-4).

The index 2 is defined as in Expression (8) using the sum of the H1 to H4 regions shown in Expressions (7-1) to (7-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (8)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, they are effective for determining the scene type when the captured image data is a color image.

  Next, the second occupancy rate calculation process executed to calculate the index 3 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step S20). Next, the photographed image data is divided into regions each composed of a combination of the distance from the outer edge of the photographed image screen and the brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step S21). Hereinafter, the area division of the captured image data will be described in detail.

  26A to 26D show four regions n1 to n4 that are divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 26A is an outer frame, a region n2 shown in FIG. 26B is a region inside the outer frame, and a region n3 shown in FIG. A region n4 shown in FIG. 26D is an inner region, and is a central region of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step S22), and the main occupancy ratio calculation is performed. The process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図２７は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupation ratio Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the shooting conditions.

FIG. 27 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(9-1)〜式(9-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 …(9-1)；
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 …(9-2)；
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 …(9-3)；
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) …(9-4)． If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (9).

Accordingly, the sum of the n1 to n4 regions is expressed by the following formulas (9-1) to (9-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (9-1);
Sum of n2 regions = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (9-2);
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (9-3);
Sum of n4 regions = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) ... (9-4).

The index 3 is defined as in Expression (10) using the sum of the n1 to n4 regions shown in Expressions (9-1) to (9-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (10)
The index 3 is calculated based on a compositional feature (distance from the outer edge of the screen of the captured image data) based on the brightness distribution position of the captured image data, so that it determines the scene type of not only a color image but also a monochrome image. It is also effective.

  Next, the bias amount calculation process (step S2 in FIG. 18) will be described with reference to the flowchart in FIG.

式（１１）では、撮影画像データの各画素について和をとる。式（１１）において、画素輝度値とは、撮影画像データの各画素の輝度であり、全体平均輝度とは、撮影画像データ全体の輝度の平均値である。また、全体画素数とは、撮影画像データ全体の画素数である。 First, the luminance Y (brightness) of each pixel is calculated from the RGB (Red, Green, Blue) values of the photographed image data using Equation (A), and the luminance standard deviation (x1) is calculated (step 1). S23). The standard deviation (x1) of luminance is expressed as shown in Equation (11).

In Expression (11), the sum is calculated for each pixel of the captured image data. In Expression (11), the pixel luminance value is the luminance of each pixel of the captured image data, and the overall average luminance is the average value of the luminance of the entire captured image data. The total number of pixels is the number of pixels of the entire captured image data.

Next, as shown in Expression (12), a luminance difference value (x2) is calculated (step S24).
Difference in luminance value (x2) = (maximum luminance value−total average luminance) / 255 (12)
In Equation (12), the maximum luminance value is the maximum luminance value of the captured image data.

  Next, the average luminance value (x3) of the skin color area in the center of the screen of the captured image data is calculated (step S25), and further, the average luminance value (x4) in the center of the screen is calculated (step S26). Here, the screen center portion is, for example, a region constituted by a region n3 and a region n4 in FIG.

Next, the flesh color luminance distribution value (x5) is calculated (step S27), and this deviation amount calculation processing is completed. When the maximum luminance value of the skin color area of the photographed image data is Yskin_max, the minimum luminance value of the skin color area is Yskin_min, and the average luminance value (skin color average luminance) of the skin color area is Yskin_ave, the skin color luminance distribution value (x5) is expressed by the equation (13). ).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (13)

Let x6 be the average luminance value of the skin color area in the center of the screen of the captured image data. Here, the center of the screen is, for example, an area composed of the area n2, the area n3, and the area n4 in FIG. At this time, the index 4 is defined as in Expression (14) using the index 1, index 3, and x6, and the index 5 is defined as in Expression (15) using the index 2, index 3, and x6. The
Index 4 = 0.46 × Index 1 + 0.61 × Index 3 + 0.01 × x6−0.79 (14)
Index 5 = 0.58 x Index 2 + 0.18 x Index 3 + (-0.03) x x6 + 3.34 (15)
Here, the weighting coefficient multiplied by each index in Expression (14) and Expression (15) is set in advance according to the shooting conditions.

指標６は、式（１６）のように表される。
指標６＝x1×0.02＋x2×1.13＋x3×0.06＋x4×(-0.01)＋x5×0.03−6.49 …（１６）
この指標６は、撮影画像データの画面の構図的な特徴だけでなく、輝度ヒストグラム分布情報を持ち合わせており、特に、ストロボ撮影シーンとアンダー撮影シーンの判別に有効である。 The index 6 is obtained by multiplying the deviation amounts (x1) to (x5) calculated in the deviation amount calculation process by a fourth coefficient set in advance according to the shooting conditions. Table 6 shows a fourth coefficient which is a weighting coefficient by which each deviation amount is multiplied.

The index 6 is expressed as in Expression (16).
Index 6 = x1 x 0.02 + x2 x 1.13 + x3 x 0.06 + x4 x (-0.01) + x5 x 0.03-6.49 (16)
This index 6 has not only the compositional characteristics of the screen of the photographed image data but also the luminance histogram distribution information, and is particularly effective for discriminating between the flash photography scene and the under photography scene.

  The index 4 shown in Expression (14) corresponds to a natural light index, the index 5 shown in Expression (15) corresponds to a luminance ratio index, and the index 6 shown in Expression (16) corresponds to an exposure index.

  As described above, according to the imaging apparatus 100 of the first embodiment, the brightness of the main subject (for example, a human face area) in a backlight scene or a strobe shooting scene is an output device (for example, a printer) that outputs captured image data. It is possible to obtain a brightness correction amount that can be reproduced with an optimal brightness.

  In addition, by generating a brightness correction curve based on the brightness correction amount calculated as described above and the determined scene type, the dynamic range of captured image data such as a backlit scene or a strobe scene is captured. When outputting a large scene, the degree of deviation from the dynamic range of the output device is reduced, so the brightness of the main subject of the captured image data is appropriately finished, and the bright and dark portions of the captured image data are crushed or the high saturation It is possible to suppress the color from being saturated and crushed or the hue from being changed.

[Embodiment 2]
Next, Embodiment 2 of the present invention will be described.
Since the functions of the constituent elements common to the first and second embodiments have been described, the functions unique to the second embodiment will be described below.

  Hereinafter, the internal configuration of the brightness correction amount calculation unit 75 of the second embodiment will be described with reference to FIG. Note that, among the components of the brightness correction amount calculation unit 75 according to the second embodiment, the same components as those of the first embodiment shown in FIG. As shown in FIG. 29, the brightness correction amount calculation unit 75 of the second embodiment includes a first temporary correction amount setting unit 101, a first mixing coefficient setting unit 102, a second temporary correction amount calculation unit 103, and a second mixing coefficient. The setting unit 104, the brightness correction amount calculation unit 105, the suppression coefficient calculation unit 106, and the multiplication unit 107 are configured.

  The suppression coefficient calculation unit 106 calculates the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold, and the total number of pixels of the captured image data A suppression coefficient Re (0.0 <Re ≦ 1.0) for suppressing the brightness correction amount of the photographed image data is calculated (set) based on the ratio of the number of pixels of the high saturation with respect to. A method of calculating the suppression coefficient Re will be described in detail later with reference to FIG.

As shown in Equation (17), the multiplication unit 107 uses the suppression coefficient calculated (set) by the suppression coefficient calculation unit 106 to the brightness correction amount (Equation (3)) calculated by the brightness correction amount calculation unit 105. By multiplying the coefficient Re, a new brightness correction amount is calculated, and this new brightness correction amount is output to the brightness correction curve generation unit 76.
New brightness correction amount = Re × Brightness correction amount (17)

  The brightness correction curve generation unit 76 according to the second embodiment converts captured image data based on the scene type determined by the determination map (see FIG. 14) and the new brightness correction amount input from the multiplication unit 107. To generate a brightness correction curve.

Next, the operation in the second embodiment will be described.
The brightness correction process executed in the image processing unit 7 of the second embodiment will be described with reference to the flowchart of FIG.

  First, the index calculation unit 74 performs an index calculation process (see FIGS. 18 to 28) for calculating a natural light index, a luminance ratio index, and an exposure index (step T20). Next, the image analysis unit 73 calculates n brightness analysis values of the captured image data (step T21). In step T21, two brightness analysis values (skin color average brightness, overall average brightness) are calculated.

  Next, the brightness analysis value calculated in step T21 and the values corresponding to the natural light index, the luminance ratio index, and the exposure index calculated in the index calculation process in step T20 are extracted from the one-dimensional LUT of FIGS. As a result, n + 4 (six) first temporary correction amounts key_auto [0] to [5] are set (step T22).

  Next, n + 2 (four) first mixing coefficients wgt [0] to [3] are extracted from the one-dimensional LUTs shown in FIGS. 10 to 13 by extracting values corresponding to the natural light index calculated in step T20. Is set (step T23). Next, using the four first temporary correction amounts key_auto [0] to [3] and the three first mixing coefficients wgt [0] to [3], the second temporary correction amount key_auto is obtained from Equation (2). [6] is calculated (step T24).

  Next, three second mixing coefficients wgt [4] to [6] corresponding to the scene type determined using the determination map (FIG. 14) are extracted from the coefficient calculation table (FIG. 15) and set. (Step T25). Then, using the two first temporary correction amounts key_auto [4] and key_auto [5], the second temporary correction amount key_auto [6], and the three second mixing coefficients wgt [4] to [6] Then, the brightness correction amount is calculated by the equation (3) (step T26).

  Next, it is determined whether or not the luminance ratio index calculated in step T20 is greater than a threshold value Th1 (step T27). Here, the threshold Th1 = 1.5. If it is determined in step T27 that the luminance ratio index> Th1 (step T27; YES), the hue condition of the pixel to be counted later as a high saturation pixel is set according to the natural light index calculated in step T20. (Step T28).

  FIG. 31 shows the hue condition (hue range) set according to the value of the natural light index in step T28. As shown in FIG. 31, the hue conditions are: hue H = 161 to 250 when −6.0 <natural light index ≦ −1.5, and hue H = 0 to 39, 161 to when −1.5 <natural light index <2.0. 250, 330-359, and when 2.0 ≦ natural light index <6.0, hue H = 0-39, 330-359.

When the hue condition is set, the number of pixels having the hue condition set in step T28 and having a saturation (S) larger than the threshold Th2 among the total number of pixels of the captured image data is calculated. The saturation level is set based on the ratio (%) of the high saturation pixel number satisfying (S)> Th2 to the total pixel number (step T29). Here, Th2 = 130 (HSV color system, maximum value 255). Table 7 shows the ratio (%) of the high saturation pixel number satisfying saturation (S)> Th2 to the total pixel number and the suppression coefficient for each saturation level (saturation 1-4).

  When the saturation level is set, the suppression coefficient Re corresponding to the set saturation level is set according to Table 7 (step T30). Note that the processing of steps T27 to T30 is executed by the suppression coefficient calculation unit 106. Next, as shown in Expression (17), a new brightness correction amount is calculated by multiplying the brightness correction amount calculated in Step T26 by the suppression coefficient Re set in Step T30 (Step T31). ), A brightness correction curve for converting photographed image data is generated from the new brightness correction amount and the scene type determined based on each index calculated in step T20 (step T32).

  When it is determined in step T27 that the luminance ratio index ≦ Th1 (step T27; NO), the scene determined based on the brightness correction amount calculated in step T26 and each index calculated in step T20. From the type, a brightness correction curve for converting the captured image data is generated (step T32).

  When the brightness correction curve is generated, the brightness of the captured image data is corrected according to the brightness correction curve (step T33), and the brightness correction process is terminated. The captured image data whose brightness has been corrected in this way is output to a designated output destination. Examples of the output destination include the display unit 13 and a printer connected to the imaging apparatus 100.

  As described above, according to the imaging apparatus 100 of the second embodiment, the brightness correction amount of the captured image data is calculated using the suppression coefficient calculated based on the ratio of the high saturation pixel number to the total number of pixels of the captured image data. By suppressing the saturation, it is possible to prevent saturation of the high saturation pixels in the captured image data having many high saturation pixels. In particular, the hue of a pixel to be counted as a high saturation pixel is determined based on the value of the natural light index, so that saturation of the high saturation pixel can be prevented with high accuracy.

  In addition, the description content in each said embodiment can be suitably changed in the range which does not deviate from the meaning of this invention.

  For example, although the case where the brightness correction processing shown in each of the above embodiments is performed by an imaging apparatus such as a digital camera has been described, an image processing apparatus (a personal computer) installed in a minilab (small-scale development laboratory) may be used. 2) may be equipped with the function of the image processing unit 7 of FIG.

  FIG. 32 shows a main configuration of an image processing apparatus 200 to which the present invention is applied. The image processing apparatus 200 includes an image input unit 201, an image processing unit 202, and an image output unit 203, as shown in FIG.

  The image input unit 201 acquires photographed image data obtained by photographing a subject and outputs it to the image processing unit 202. The image output unit 203 includes a display such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and displays the captured image data processed by the image processing unit 202. Further, the image output unit 203 may include a printing unit that prints out the captured image data processed by the image processing unit 202.

  As shown in FIG. 32, the image processing unit 202 includes an image analysis unit 204, an index calculation unit 205, a brightness correction amount calculation unit 206, a brightness correction curve generation unit 207, and an image quality correction unit 208. Are equivalent to the image analysis unit 73, index calculation unit 74, brightness correction amount calculation unit 75, brightness correction curve generation unit 76, and image quality correction unit 72 of FIG.

  However, in the case of the image processing apparatus 200 installed in the minilab, the brightness correction curves are not limited to the four types (−6, −4, −2, 0 buttons) shown in FIG. It is possible to generate a correction curve.

1 is a block diagram showing a main part configuration of an imaging apparatus according to an embodiment of the present invention. The block diagram which shows the internal structure of an image process part. The block diagram which shows the internal structure of the brightness correction amount calculation part in Embodiment 1 of this invention. LUT indicating the relationship between the luminance ratio index and the provisional correction amount key_auto [0]. LUT indicating the relationship between the exposure index and the temporary correction amount key_auto [1]. LUT indicating the relationship between the skin color average luminance and the provisional correction amount key_auto [2]. LUT indicating the relationship between the flesh color average luminance or the overall average luminance and the temporary correction amount key_auto [3]. LUT indicating the relationship between the brightness ratio index and the temporary correction amount key_auto [4]. LUT indicating the relationship between the exposure index and the temporary correction amount key_auto [5]. LUT indicating the relationship between the natural light index and the mixing coefficient wgt [0]. LUT indicating the relationship between the natural light index and the mixing coefficient wgt [1]. LUT indicating the relationship between the natural light index and the mixing coefficient wgt [2]. LUT indicating the relationship between the natural light index and the mixing coefficient wgt [3]. The figure which shows the discrimination | determination map for discriminating a scene type from a natural light parameter | index, a luminance ratio parameter | index, and an exposure parameter | index. The figure which shows the coefficient calculation table showing the value of mixing coefficient wgt [4]-[6] according to scene type. 5 is a flowchart illustrating brightness correction processing executed in the image processing unit according to the first embodiment. FIG. 5B is a diagram illustrating a brightness correction curve (a) when the scene type is direct light and a brightness correction curve when the scene type is backlight. The flowchart which shows an index calculation process. The flowchart which shows the 1st occupation rate calculation process which calculates the 1st occupation rate for every area | region of lightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. The figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. The flowchart which shows the 2nd occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient for multiplying the 2nd occupation rate for calculating the parameter | index 3 according to area | region (n1-n4). 7 is a flowchart showing a bias amount calculation process. The block diagram which shows the internal structure of the brightness correction amount calculation part in Embodiment 2 of this invention. 9 is a flowchart illustrating brightness correction processing executed in the image processing unit of the second embodiment. The figure which shows the relationship between a natural light parameter | index and the hue condition of a high saturation pixel. 1 is a block diagram showing a main part configuration of an image processing apparatus to which the present invention is applied.

Explanation of symbols

1 Lens 2 Aperture 3 CCD
4 Analog processing circuit 5 A / D converter 6 Temporary storage memory 7 Image processing unit 70 Pre-processing unit 71 Brightness correction unit 72 Image quality correction unit 73 Image analysis unit 74 Index calculation unit 75 Brightness correction amount calculation unit 76 Brightness correction Curve generation unit 8 Header information processing unit 9 Storage device 10 CCD drive circuit 11 Control unit 12 Operation unit 13 Display unit 14 Strobe drive circuit 15 Strobe 16 Focal length adjustment circuit 17 Automatic focus drive circuit 20 Motor 100 Imaging device 101 First temporary correction Amount setting unit 102 first mixing coefficient setting unit 103 second temporary correction amount calculating unit 104 second mixing coefficient setting unit 105 brightness correction amount calculating unit 106 suppression coefficient calculating unit 107 multiplying unit 200 image processing apparatus 201 image input unit 202 image Processing unit 203 Image output unit 204 Image analysis unit 205 Index calculation unit 206 Brightness compensation Positive amount calculation unit 207 Brightness correction curve generation unit 208 Image quality correction unit

Claims (12)

An imaging apparatus that captures captured image data by capturing a subject and performs processing for optimizing the brightness of a main subject in the captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
A correction curve generating means for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means;
An imaging apparatus comprising: An imaging apparatus that captures captured image data by capturing a subject and performs processing for optimizing the brightness of a main subject in the captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. Suppression coefficient setting means for setting a suppression coefficient for
Multiplication means for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient;
A correction curve generation means for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means;
An imaging apparatus comprising: The index calculating means includes
A first calculation process that divides the captured image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire captured image data for each of the divided regions; The captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and a second occupation indicating the ratio of the divided area to the entire captured image data Executing at least one of the second calculation processes for calculating the rate,
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, The imaging apparatus according to claim 1, wherein an exposure index is calculated. An image processing apparatus that performs processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
A correction curve generating means for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means;
An image processing apparatus comprising: An image processing apparatus that performs processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. Index calculation means to
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the luminance ratio index and the exposure index calculated by the index calculation means, a plurality of first temporary correction amounts for the captured image data are determined. First temporary correction amount setting means for setting;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculating means;
Second temporary correction amount setting means for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation means, at least one of the plurality of first temporary correction amounts is multiplied by a second coefficient as a weighting factor to be multiplied by the second temporary correction amount. A second mixing coefficient setting means for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation means;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. Suppression coefficient setting means for setting a suppression coefficient for
Multiplication means for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation means by the suppression coefficient;
A correction curve generation means for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication means, the natural light index, and the luminance ratio index;
Correction means for correcting the brightness of the captured image data in accordance with the correction curve generated by the correction curve generation means;
An image processing apparatus comprising: The index calculating means includes
A first calculation process that divides the captured image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire captured image data for each of the divided regions; The captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and a second occupation indicating the ratio of the divided area to the entire captured image data Executing at least one of the second calculation processes for calculating the rate,
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, The image processing apparatus according to claim 4, wherein an exposure index is calculated. An image processing method for performing processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation process to perform,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation step, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting step to be set;
A first mixing coefficient setting step for setting a first mixing coefficient as a weighting coefficient by which each of the plurality of first temporary correction amounts is multiplied based on the natural light index calculated by the index calculation step;
A second provisional correction amount setting step for setting a second provisional correction amount for the captured image data based on the first mixture coefficient set by the first mixture coefficient setting step and the plurality of first provisional correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation step, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weight coefficient. A second mixing coefficient setting step for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation process;
A correction curve generating step for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculating step, the natural light index, and the luminance ratio index;
A correction step of correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation step;
An image processing method comprising: An image processing method for performing processing for optimizing the brightness of a main subject in captured image data,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation process to perform,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation step, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting step to be set;
A first mixing coefficient setting step for setting a first mixing coefficient as a weighting coefficient by which each of the plurality of first temporary correction amounts is multiplied based on the natural light index calculated by the index calculation step;
A second provisional correction amount setting step for setting a second provisional correction amount for the captured image data based on the first mixture coefficient set by the first mixture coefficient setting step and the plurality of first provisional correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation step, at least one of the plurality of first temporary correction amounts is multiplied by the second temporary correction amount as a weight coefficient. A second mixing coefficient setting step for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation process;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. A suppression coefficient setting step for setting a suppression coefficient for
A multiplication step of calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation step by the suppression coefficient;
A correction curve generating step for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication step, the natural light index, and the luminance ratio index;
A correction step of correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation step;
An image processing method comprising: In the index calculation step,
A first calculation process that divides the captured image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire captured image data for each of the divided regions; The captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and a second occupation indicating the ratio of the divided area to the entire captured image data Executing at least one of the second calculation processes for calculating the rate,
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, The image processing method according to claim 7, wherein an exposure index is calculated. On the computer,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation function,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation function, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting function to be set;
A first mixing coefficient setting function for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation function;
A second temporary correction amount setting function for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation function, a first weighting factor as a weighting factor that multiplies the second temporary correction amount by at least one of the plurality of first temporary correction amounts. A second mixing coefficient setting function for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation function;
A correction curve generation function for generating a correction curve for converting the captured image data based on the brightness correction amount calculated by the correction amount calculation function, the natural light index, and the luminance ratio index;
A correction function for correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation function;
An image processing program for realizing On the computer,
Calculates a natural light index that represents the degree of outdoor shooting by natural light of the photographed image data, a luminance ratio index that represents the size of the dynamic range of the photographed image data, and an exposure index that represents the exposure photographing degree due to the exposure setting at the time of photographing. An index calculation function,
Based on the brightness analysis value of the captured image data, the brightness reproduction target value, and the brightness ratio index and the exposure index calculated by the index calculation function, a plurality of first temporary correction amounts for the captured image data are determined. A first provisional correction amount setting function to be set;
A first mixing coefficient setting function for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index calculated by the index calculation function;
A second temporary correction amount setting function for setting a second temporary correction amount for the captured image data based on the first mixing coefficient set by the first mixing coefficient setting function and the plurality of first temporary correction amounts; ,
Based on the natural light index, the luminance ratio index, and the exposure index calculated by the index calculation function, a first weighting factor as a weighting factor that multiplies the second temporary correction amount by at least one of the plurality of first temporary correction amounts. A second mixing coefficient setting function for setting two mixing coefficients;
A correction amount for calculating a brightness correction amount of the captured image data based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount. A calculation function;
The brightness correction amount of the captured image data is suppressed based on the number of pixels of high saturation having a predetermined hue that is set based on the natural light index and the saturation of the captured image data is higher than a predetermined threshold. A suppression coefficient setting function for setting a suppression coefficient for
A multiplication function for calculating a new brightness correction amount by multiplying the brightness correction amount calculated by the correction amount calculation function by the suppression coefficient;
A correction curve generation function for generating a correction curve for converting the captured image data based on the new brightness correction amount obtained by the multiplication function, the natural light index, and the luminance ratio index;
A correction function for correcting the brightness of the captured image data according to the correction curve generated by the correction curve generation function;
An image processing program for realizing When realizing the index calculation function,
A first calculation process that divides the captured image data into regions composed of combinations of predetermined brightness and hue, and calculates a first occupancy ratio that indicates a ratio of the entire captured image data for each of the divided regions; The captured image data is divided into a predetermined area composed of a combination of the distance from the outer edge of the screen of the captured image data and the brightness, and a second occupation indicating the ratio of the divided area to the entire captured image data Executing at least one of the second calculation processes for calculating the rate,
Calculating a deviation amount indicating a deviation in gradation distribution of the photographed image data;
By multiplying the calculated first occupancy and / or second occupancy and the deviation amount by a coefficient set in advance according to a shooting condition, the natural light index, the luminance ratio index, The image processing program according to claim 10 or 11, wherein an exposure index is calculated.

Images