JP2007293686A - Imaging apparatus, image processing apparatus, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To optimize brightness of a main photographic subject (for example, a person's face region) without varying an operating-point for existing automatic focus control and imaging element and without varying gradation processing condition from a linear image to a non-linear image. <P>SOLUTION: An imaging apparatus 1 acquires the linear image formed by photographing (S5, S7) and carries out discriminating process for a photographing scene based on the linear image (S8). Then, based on the discriminated result by the scene discriminating process, the gradation processing condition for the linear image is set (S9) and gradation converting process (brightness compensation process and flare compensation process) for the linear image is carried out (S10 and S11) based on the set gradation processing condition. Then, the gradation converting process (a gamma converting process) is carried out (S12) converting the linear image to which the gradation converting process is applied to the non-linear image. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, an image processing method, and an image processing program that perform image processing on a captured image.

When creating a print from photographed image data acquired by a digital camera, the brightness of the photographed image data is adjusted so that the brightness of the main subject (for example, a human face area) is properly finished. . In such a brightness adjustment method, a correction amount and tone conversion LUT for automatically optimizing the brightness of the photographed image data based on information obtained by analyzing the main subject of the photographed image data Is known (see, for example, Patent Document 1).
JP 2002-247393 A

  However, captured image data such as JPEG acquired with a normal digital camera is optimal for viewing on a display medium such as a monitor from image data (hereinafter referred to as “linear image”) in which the luminance value of the subject is recorded. The amount of information (S / N) is considerably lower than the “linear image” immediately after shooting because the image data (hereinafter referred to as “non-linear image”) has undergone gradation conversion processing. Is in a state. For this reason, in scenes with a large luminance range, such as backlighting and overshooting scenes, the larger the correction amount, the more serious the image quality degradation, such as a decrease in the amount of information (S / N) and saturation of bright, dark, and high-saturation parts. Therefore, there has been a problem that the correction amount has to be greatly limited.

  In order to solve this problem, a digital camera is known in which the operating point of the image sensor that defines the luminance range of the subject that falls within the dynamic range of the image sensor is changed. Specifically, by shifting the exposure of the digital camera to the under side, that is, by increasing the setting sensitivity of the image sensor that is the basis for exposure control, the output value of the image sensor for the standard luminance part is reduced, and the shootable range is increased. It is to expand. “Whiteout” that occurs when a high correction amount is applied to a scene with a large luminance range by applying a tone conversion process from this underlined “linear image” to a properly exposed “nonlinear image” This problem has been improved to some extent.

  On the other hand, with ordinary digital cameras, the demands for CPU (Central Processing Unit) performance, high-speed processing, operability, etc., make automatic exposure control algorithms and image processing before recording to storage media more complicated. As a countermeasure against this, “linear image” is recorded as so-called “Raw data” and converted from “Raw data” to “non-linear image” using so-called “development software” by the user himself Provides means based on work. However, the use of "development software" and the management of image data involve a rather complicated operation, so it is not easy for anyone to use, and further improvement of the performance of digital cameras is desired. .

  An object of the present invention is to control the brightness of a main subject (for example, a human face area) without changing the existing automatic exposure control, the operating point of the image sensor, or changing the gradation processing condition from a linear image to a non-linear image. Is to optimize.

In order to solve the above problem, an imaging apparatus according to claim 1 is provided.
Photographing means for photographing the subject;
Image obtaining means for obtaining a linear image formed by photographing by the photographing means;
Scene discrimination processing means for performing a discrimination process of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means;
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means;
And a third gradation conversion processing means for performing a third gradation conversion process for converting the linear image to which the second gradation conversion process is applied into a non-linear image.

According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the image acquisition means acquires a linear image with a reduced image size from the linear image,
The scene discrimination processing means performs a shooting scene discrimination process using a linear image with the image size reduced.

Invention of Claim 3 is provided with the luminance information acquisition means which acquires the luminance information computed at the time of photography of the linear image in the imaging device of Claim 1 or 2,
The scene determination processing unit performs a shooting scene determination process based on the luminance information acquired by the luminance information acquisition unit, and the gradation processing condition setting unit adds the luminance information acquired by the luminance information acquisition unit. Based on this, the first and second gradation processing conditions are set.

  According to a fourth aspect of the present invention, in the imaging apparatus according to any one of the first to third aspects, the scene discrimination processing unit includes a natural light index that represents an outdoor photographing degree of the linear image by natural light, a dynamic range, A luminance ratio index representing the size of the image and an exposure shooting degree index resulting from the exposure setting at the time of shooting are calculated, and a shooting scene is determined based on each of the calculated indexes.

According to a fifth aspect of the present invention, in the imaging device according to the fourth aspect, the scene determination processing unit acquires color information about the linear image, and based on the acquired color information, the linear image of the linear image is acquired. First occupancy rate calculating means for classifying image data into a class composed of a combination of predetermined brightness and hue, and calculating a first occupancy ratio indicating the ratio of the entire image data for each of the classified classes; ,
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating the occupancy rate of
A first index calculating means for calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the photographing condition;
A second index calculating means for calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions;
A third index calculating means for calculating the index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating means for calculating the exposure photographing degree by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to photographing conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. A fifth index calculating means for
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Sixth index calculating means for calculating;
Scene discrimination means for discriminating a shooting scene of the image data based on the exposure shooting degree, the natural light index, and the luminance ratio index.

According to a sixth aspect of the present invention, in the imaging apparatus according to the fourth or fifth aspect, the gradation processing condition setting unit includes a brightness analysis value of the linear image, a brightness reproduction target value, and the luminance. First temporary correction amount setting means for setting a plurality of first temporary correction amounts for the linear image based on the ratio index and the exposure photographing degree;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
Second temporary correction amount setting means for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. Second mixing coefficient setting means for
Gradation adjustment parameter calculation means for calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; With
A first gradation processing condition for the linear image is set based on a determination result by the scene determination processing unit and a gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit.

An image processing apparatus according to claim 7 is provided.
Image acquisition means for acquiring a linear image formed by photographing;
Scene discrimination processing means for performing a discrimination process of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means;
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means;
And third gradation conversion processing means for performing gradation conversion processing for converting the linear image to which the second gradation conversion processing is applied into a non-linear image.

The invention according to claim 8 is the image processing apparatus according to claim 7, wherein the image acquisition means acquires a linear image with a reduced image size from the linear image,
The scene discrimination processing means performs a shooting scene discrimination process using a linear image with the image size reduced.

The invention according to claim 9 is the image processing apparatus according to claim 7 or 8, further comprising luminance information acquisition means for acquiring luminance information calculated at the time of photographing the linear image,
The scene determination processing unit performs a shooting scene determination process based on the luminance information acquired by the luminance information acquisition unit,
The gradation processing condition setting means sets the first and second gradation processing conditions based on the luminance information acquired by the luminance information acquisition means.

According to a tenth aspect of the present invention, in the image processing device according to any one of the seventh to ninth aspects, the scene determination processing unit includes a natural light index that represents an outdoor photographing degree of the linear image by natural light, and a dynamic light. Calculate the brightness ratio index that represents the size of the range and the exposure shooting index that is attributed to the exposure setting at the time of shooting.
A shooting scene is discriminated based on each of the calculated indexes.

According to an eleventh aspect of the present invention, in the image processing apparatus according to the tenth aspect, the scene discrimination processing means includes:
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating the occupancy rate of
A first index calculating means for calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the photographing condition;
A second index calculating means for calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions;
A third index calculating means for calculating the index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating means for calculating the exposure photographing degree by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to photographing conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. A fifth index calculating means for
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Sixth index calculating means for calculating;
Scene discrimination means for discriminating a shooting scene of the image data based on the exposure shooting degree, the natural light index, and the luminance ratio index.

The invention according to claim 12 is the image processing apparatus according to claim 10 or 11, wherein the gradation processing condition setting means includes:
A first temporary correction amount for setting a plurality of first temporary correction amounts for the linear image based on a brightness analysis value of the linear image, a brightness reproduction target value, the luminance ratio index, and the exposure photographing degree. Setting means;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
Second temporary correction amount setting means for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. Second mixing coefficient setting means for
Gradation adjustment parameter calculation means for calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; With
A first gradation processing condition for the linear image is set based on a determination result by the scene determination processing unit and a gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit.

An image processing method according to claim 13 is provided.
An image acquisition step of acquiring a linear image formed by photographing;
A scene determination processing step for performing a determination process of a photographic scene based on the linear image;
Based on the determination result of the scene determination processing step, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting step to perform,
A first gradation conversion processing step for performing a first gradation conversion process on the linear image based on the first gradation processing condition set by the gradation processing condition setting step;
Based on the second gradation processing condition set in the gradation processing condition setting step, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion process,
And a second gradation conversion process step of performing a third gradation conversion process for converting the linear image to which the second gradation conversion process is applied into a non-linear image.

The invention according to claim 14 is the image processing method according to claim 13, wherein in the image acquisition step, a linear image with a reduced image size is acquired from the linear image,
In the scene discrimination processing step, a shooting scene discrimination process is performed using a linear image with a reduced image size.

The invention according to claim 15 includes a luminance information acquisition step of acquiring luminance information calculated at the time of photographing the linear image in the image processing method according to claim 13 or 14,
In the scene determination process step, a shooting scene determination process is performed based on the luminance information acquired in the luminance information acquisition step,
In the gradation processing condition setting step, the first and second gradation processing conditions are set based on the luminance information acquired by the luminance information acquisition step.

The invention according to claim 16 is the image processing method according to any one of claims 13 to 15, wherein in the scene determination processing step,
Calculating a natural light index representing the degree of outdoor photographing with natural light of the linear image, a luminance ratio index representing the size of the dynamic range, and an exposure photographing degree index resulting from the exposure setting at the time of photographing;
A shooting scene is discriminated based on each of the calculated indexes.

The invention according to claim 17 is the image processing method according to claim 16, wherein the scene discrimination processing step includes:
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, A first occupancy ratio calculating step of calculating a first occupancy ratio indicating a ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class A second occupancy rate calculating step for calculating the occupancy rate of
A first index calculation step of calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the imaging condition;
A second index calculating step of calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to shooting conditions;
A third index calculating step of calculating index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating step of calculating the exposure shooting degree by multiplying the average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to shooting conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. And a fifth index calculating step
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. A sixth index calculating step for calculating;
A scene determination step of determining a shooting scene of the image data based on the exposure shooting degree, the natural light index, and the luminance ratio index.

The invention according to claim 18 is the image processing method according to claim 16 or 17, wherein the gradation processing condition setting step includes:
A first temporary correction amount for setting a plurality of first temporary correction amounts for the linear image based on a brightness analysis value of the linear image, a brightness reproduction target value, the luminance ratio index, and the exposure photographing degree. A setting process;
A first mixing coefficient setting step of setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
A second temporary correction amount setting step for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting step and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. A second mixing coefficient setting step,
A gradation adjustment parameter calculating step of calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; Including,
A first gradation processing condition for the linear image is set based on a determination result in the scene determination processing step and a gradation adjustment parameter calculated in the gradation adjustment parameter calculation step.

An image processing program according to claim 19 is provided.
A computer that controls image processing
Image acquisition means for acquiring a linear image formed by photographing;
Scene discrimination processing means for performing discrimination processing of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means,
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means,
It is made to function as a 3rd gradation conversion process means which performs the gradation conversion process which converts the linear image to which the said 2nd gradation conversion process was applied into a non-linear image.

The invention according to claim 20 is the image processing program according to claim 19, wherein the image acquisition means includes:
From the linear image, obtain a linear image reduced in image size,
The scene discrimination processing means includes
It is characterized in that a photographic scene discrimination process is performed using a linear image with a reduced image size.

The invention according to claim 21 is the image processing program according to claim 19 or 20, wherein the computer is
Function as luminance information acquisition means for acquiring luminance information calculated at the time of photographing the linear image;
The scene determination processing unit performs a shooting scene determination process based on the luminance information acquired by the luminance information acquisition unit,
The gradation processing condition setting means sets the first and second gradation processing conditions based on the luminance information acquired by the luminance information acquisition means.

The invention according to claim 22 is the image processing program according to any one of claims 19 to 21, wherein the scene determination processing means includes:
Calculating a natural light index representing an outdoor photographing degree by natural light of the linear image, a luminance ratio index representing a size of a dynamic range, and an exposure photographing degree index resulting from an exposure setting at the time of photographing; It is characterized in that a photographing scene is discriminated based on the above.

The invention according to claim 23 is the image processing program according to claim 22, wherein the scene discrimination processing means includes:
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, Calculating a first occupancy ratio indicating the ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Calculate the share of
By multiplying the first occupancy by a first coefficient set in advance according to the shooting conditions, the index 1 is calculated,
The index 2 is calculated by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions,
The index 3 is calculated by multiplying the second occupation ratio by a third coefficient set in advance according to the shooting conditions,
The exposure photographing degree is calculated by multiplying the average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to the photographing condition,
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. And
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Calculate
The photographing scene of the image data is determined based on the exposure photographing degree, the natural light index, and the luminance ratio index.

The invention according to claim 24 is the image processing program according to claim 22 or 23, wherein the gradation processing condition setting means is:
Based on the brightness analysis value of the linear image, the brightness reproduction target value, the brightness ratio index, and the exposure shooting degree, a plurality of first temporary correction amounts for the linear image are set,
Based on the natural light index, set a first mixing coefficient as a weighting coefficient to be multiplied to each of the plurality of first temporary correction amount,
Based on the first mixing coefficient and the plurality of first temporary correction amounts, a second temporary correction amount for the linear image is set,
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. And
Calculating a gradation adjustment parameter based on the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount;
A first gradation processing condition for the linear image is set based on the determination result by the scene determination processing means and the calculated gradation adjustment parameter.

  Hereinafter, terms used in the present invention will be described.

First, “linear image” and “nonlinear image” will be described in detail.
In general shooting scenes, there are many uneven illuminations, and it is not uncommon for the luminance ratio to exceed 1000 times (for example, see Color Science Handbook 2nd Edition, The University of Tokyo Press, p925-926 (1998)). . On the other hand, the luminance ratio that can be displayed on various media is on the order of 100 times. The photographic gradation will inevitably be different from the scene gradation, and the basics of photographic design are how to properly produce the impression of a scene with a luminance ratio of the order of 1000 times on media with a luminance ratio of the order of 100 times. .

The conversion from the scene gradation to the photographic gradation cannot be determined uniformly because appropriate mapping conditions differ depending on the scene state (luminance ratio in composition, luminance of main subject, etc.). Therefore, in the case of silver halide photography, the following architecture is adopted.
[Design 1] The negative film has a soft design in which the density changes linearly according to the luminance ratio of the order of several thousand times. Thereby, the luminance information of the scene is recorded on the negative film without omission.
[Design 2] The above-mentioned soft negative film image is printed on a high-quality photographic paper to obtain a gradation suitable for viewing. By adjusting the amount of exposure for printing, it is possible to select where in a wide range of scene brightness the photographic gradation is reproduced.

  In the above design 2, an appropriate condition is calculated by the printer automatically analyzing the negative film image. If the calculation result does not match the intention of photographing, an appropriate photograph can be created if the user indicates that fact and performs “reprinting”. For example, there is an example in which it is pointed out that a shaded person should be emphasized for a print that prioritizes landscape depiction.

  On the other hand, in the case of a reversal film, since an image for viewing is directly generated by film development, the soft design as in the above design 1 cannot be performed. Therefore, the reversal film has a narrow range of brightness ratios that can be recorded, and in order to capture appropriate images, it is necessary to carefully set the shooting conditions (lighting, aperture, shutter) at the time of shooting. It cannot be corrected by etc. For this reason, reversal films are sold as products exclusively for professional and high-end amateurs.

  In this way, it can be pointed out that the negative film and the positive film in silver halide photography have different image characteristics in addition to the negative / positive gradation difference.

  Comparing the architecture of DSC (Digital Still Camera) and silver halide photography from the above viewpoint, the general DSC mechanism (which generates an sRGB visible image file) corresponds to the reversal film mechanism. In other words, where to reproduce the photographic gradation centering on a wide range of scene brightness depends on the accuracy of the exposure control program and cannot be corrected after shooting. On the other hand, a professional user uses a DSC that records raw data (raw data received by the CCD) and designates where in the scene luminance the photographic gradation is reproduced with development software after shooting. This method corresponds to a negative film mechanism. Again, it can be pointed out that the characteristics of the image data are different between the sRGB image generated by the general DSC and the Raw data.

Such a difference in image characteristics is caused by a difference in rendering state of image data, and the term “Image State” is used as a concept indicating the “rendering state of image data”. (Detailed definition of Image State is shown in the following document, for example: “Requirements for Unambiguous Specification of a Color Encoding ISO 22028-1”, Kevin Spaulding, in Proc. Tenth Color Imaging Conference: Color Science and Engineering Systems , Technologies, Applications, IS & T, Springfield, VA, p.106-111 (2002)).

  There are scene-referred and output-referred as terms indicating the type of Image State. “Scene-referred” means a state expressing a chromaticity evaluation value of a landscape scene. For example, this corresponds to the state of an image in which only DSC raw data is subjected to calibration such as spectral sensitivity and is not intentionally enhanced. The “linear image” is a relatively expressed chromaticity evaluation value of a scene, but can be converted into an absolute chromaticity evaluation value by referring to additional scale information. Examples of scale information include OECF (photoelectric conversion characteristics, defined by ISO14524), aperture F-number, and exposure time.

The output-referred means a state rendered to an appropriate expression for a specific output device / observation condition. For example, a JPEG image generated by a general DSC is optimized for display display, and thus corresponds to a “nonlinear image”.

  The “linear image” in the present invention is a kind of image data belonging to the above-mentioned scene-referred image state, and the relationship between the recorded pixel luminance value and the scene luminance is substantially a linear relationship. Means things. The “non-linear image” in the present invention means image data belonging to the output-referred image state.

  In other words, the “linear image” has already mapped the signal intensity of each color channel based on at least the spectral sensitivity of the image sensor itself to a color space such as RIMM RGB, ERIMM RGB, or scRGB (“brightness extended color space” described later). This means image data in a state in which tone conversion, sharpness enhancement, saturation enhancement, etc. for modifying data contents are omitted in order to improve the effect at the time of image viewing under a specific output device / observation condition. Linear images are also corrected for the photoelectric conversion characteristics of the imaging device (opto-electronic conversion function defined by ISO1452, for example, Corona “Fine Imaging and Digital Photography” (published by the Japan Photographic Society Publishing Committee, page 479). It is preferable to have performed. The information amount (for example, the number of gradations) of the standardized linear image is equal to or greater than the information amount (for example, the number of gradations) required for the “non-linear image” described later, according to the performance of the A / D converter. It is preferable. For example, when the number of gradations of the non-linear image is 8 bits per channel, the number of gradations of the linear image is preferably 12 bits or more, more preferably 14 bits or more, and further preferably 16 bits or more.

  “Gradation conversion processing to convert linear image to non-linear image” means display devices such as CRT (Cathode Ray Tube), liquid crystal display, plasma display, and output media such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. In the above, “Gamma correction” processing for obtaining an optimal image. For example, when it is assumed that the image is displayed on a CRT display monitor compliant with the sRGB standard, an optimal color reproduction can be obtained within the color gamut of the sRGB standard. Is processed as follows. The processing setting at this time is referred to as “gamma characteristic”.

  If output to silver salt photographic paper is assumed, processing is performed so that optimum color reproduction is obtained within the color gamut of silver salt photographic paper. In addition to the compression of the color gamut, gradation processing from 16 bits to 8 bits, reduction of the number of output pixels, processing corresponding to output characteristics (LUT) of the output device, and the like are also included. It goes without saying that tone compression processing such as noise suppression, sharpening, gray balance adjustment, saturation adjustment, or dodging processing is performed.

“Non-linear image” means digital image data used to generate a hard copy image on an “output medium” such as a CRT, liquid crystal display, plasma display, or silver salt photographic paper, inkjet paper, thermal printer paper or the like. Processing is performed so as to obtain an optimum image on a display device such as a CRT, a liquid crystal display, a plasma display, or an output medium such as a silver salt photographic paper, an inkjet paper, or a thermal printer paper. When the image data formed by photographing is a “linear image”, “image data optimized for viewing on an output medium” corresponds to a “nonlinear image”. Generating a “non-linear image” from the “Raw data” or “linear image” is called “electronic development processing” (or simply “development processing”), and application software having such a processing function is used. It is called “electronic development software” (or simply “development software”).

  “Gradation conversion processing for linear image” uses DSC that records the Raw data (raw data received by the CCD), and where to reproduce the photographic gradation around the scene brightness with development software after shooting This corresponds to a complicated operation in which the user himself designates for each photographing frame. Since this gradation conversion process is equivalent to the action of the exposure control program, the processed image data is not in a state rendered to an appropriate representation for a specific output device / observation condition, but the chromaticity of the scene. Since the evaluation value is in a relatively expressed state, it can be defined as a “linear image” belonging to the scene-referred image state.

“Natural light” refers to light that uses the sun as a light source, not stationary light or flash light (referred to as “flash” or “speed light”), which is an artificial light source. Also called daylight. Daylight is a term that refers to an artificial light source with a color temperature close to daylight (approximately 5500K)
Different from “natural light”.

  “Outdoor shooting degree” (also referred to as “natural light index”) is a numerical value that estimates whether or not the shot image data was taken using “natural light” and quantitatively shows the result. . Scenes that are shot using “natural light” and require brightness correction are often backlit scenes, and there is a probability that a high-luminance area or high-saturation sky blue area is distributed at the top of the screen, and a low-luminance area or skin color or green color is distributed at the bottom of the screen. high. On the other hand, for indoor shooting or overshooting scenes shot without using “natural light”, a high-brightness area or low-saturation skin color area in the center of the screen, and a low-brightness area or low-brightness skin color or high saturation around the screen. Skin color tends to be distributed. Based on such experience, it is possible to quantitatively indicate whether or not the image is taken using “natural light” as “outdoor shooting degree”. It is desirable to use multivariate analysis as a statistical processing method for deriving one tendency from many variables such as the luminance at the top of the screen and the number of pixels of sky blue hue, the luminance at the bottom of the screen and the number of pixels of skin color hue.

  The “brightness ratio index that indicates the size of the dynamic range” is a quantitative measure of the difference in brightness between the main subject area and the background area due to the light source conditions at the time of shooting, such as a backlight scene or overshooting scene. It is the value shown in. For example, in a backlight scene, the photographer points the camera in the direction of the sun and photographs a person, so that the person face area is dark and the background area is a bright image. That is, if a difference value of brightness between the human face area and the background area is obtained, the difference value can be used as one of the indices. In general, it is known to obtain a difference value by drawing a histogram of image difference image data.

  The “brightness analysis value” of a linear image is a numerical value indicating the average brightness of an image obtained by examining the distribution state of pixel values. The “brightness analysis value” is desirably the brightness of the most important subject (main subject) in the captured image data. The search for the human face area is performed by collation in the color system using hue, saturation, and brightness. In order to obtain higher accuracy, it is preferable to use so-called “face detection” in which parts constituting the face, such as the eyes, nose, mouth, and face contour, are collated. Further, an exposure shooting index indicating the exposure shooting degree (under shooting or over shooting) resulting from the exposure setting at the time of shooting may be calculated, and this exposure shooting index may be used as the brightness analysis value. Further, the analysis method of the brightness analysis value is not limited to the analysis of the linear image. For example, the brightness obtained by analyzing the automatic exposure information of the imaging apparatus and the additional information recorded in the automatic exposure information is used. There may be.

  The “reproduction target value” is, for example, a numerical value indicating the brightness necessary for the human face region, which is the main subject of the linear image, to be optimally reproduced by the output device that outputs the captured image data. That is, if the “brightness analysis value” of the linear image is a value that approximates the “reproduction target value”, it means that there is a high probability of optimal reproduction in the output device.

  The “first provisional correction amount” is a numerical value that temporarily indicates the correction amount necessary to approximate the “brightness analysis value” to the “reproduction target value”. “Set the first temporary correction amount” is to create a one-dimensional LUT (Look Up Table) in which the relationship between the “brightness analysis value” and the “first temporary correction amount” is defined in advance. This means that the “first provisional correction amount” is determined based on the one-dimensional LUT and the “brightness analysis value”. Furthermore, “setting a plurality of first provisional correction amounts” means “calculating brightness analysis values” under different conditions, or setting different “reproduction target values” (defining a plurality of one-dimensional LUTs. Or at least two “first provisional correction amounts” temporarily.

  In the present invention, two one-dimensional LUTs for setting “first provisional correction amount” are defined, assuming brightness correction of a human face area in a backlight scene and an over-shooting scene in which the main subject is over. . In addition, in the intermediate scene between the backlight scene and the over shooting scene, at least one one-dimensional LUT for setting the “first provisional correction amount” is defined assuming brightness correction of the entire image in the under and over shooting scenes. It is desirable.

  In order to sufficiently exhibit the effects of the present invention, when creating a one-dimensional LUT for setting the “first provisional correction amount”, bright and dark portions are crushed, high-saturated colors are saturated and hushed, Prepare a number of backlight scenes and overshooting scenes that actually cause phenomena such as changes as “learning images” (also called “teacher data”), and observe the brightness of the human face area and the occurrence of these phenomena. It is desirable to do it.

  Based on “brightness reproduction target value” and “brightness ratio index”, “setting the first temporary correction amount” means that the dynamic range of the captured image data is the dynamic range of the output device that outputs the captured image data. This means that the brightness correction amount set based on the brightness of the main subject is weakened and the brightness correction amount is set based on the luminance ratio index so as to be appropriately within the range.

  “Set the first mixing coefficient” based on the natural light index is a one-dimensional LUT that defines the relationship between the “natural light index” and the “first mixing coefficient” by which each first temporary correction amount is multiplied in advance. This means that the number of first temporary correction amounts is created, and the “first mixing coefficient” to be multiplied by each first temporary correction amount is determined based on the one-dimensional LUT and the “natural light index”. Note that the first mixing coefficient may be adjusted based on the luminance ratio index.

  For example, when the natural light index shows a high outdoor photographing value, a one-dimensional LUT (input: flesh color average luminance) for setting the “first provisional correction amount” assuming brightness correction of a human face area in a backlight scene The one-dimensional LUT is defined so that the “first mixing coefficient” of (output: first provisional correction amount) becomes high.

  On the other hand, when the natural light index indicates a numerical value of a low outdoor photographing degree, a one-dimensional LUT (input: skin color average) for setting the “first provisional correction amount” assuming brightness correction of a human face area in an over photographing scene The one-dimensional LUT is defined such that the “first mixing coefficient” of luminance and output: first provisional correction amount is high.

  Further, when the natural light index indicates an ambiguous outdoor shooting degree value, a “first provisional correction amount” setting one-dimensional LUT (assuming brightness correction of the entire image in an under shooting scene and an over shooting scene) The one-dimensional LUT is defined so that the “first mixing coefficient” of input: overall average luminance or under-over shooting index, output: first temporary correction amount) becomes high.

  When the captured image data is known as a backlight image in advance, the brightness correction amount and the gradation conversion curve can be corrected based on the “brightness ratio index”, and the bright and dark portions of the captured image data may be crushed. Saturation of high-saturation colors is crushed and hue is prevented from changing. However, even if the backlit scene and overshoot scene have the same dynamic range, the direction of brightness correction is completely opposite, so the process of correcting the human face area to the appropriate brightness is automated. To apply the first provisional correction amount based on the luminance ratio index indicating the size of the dynamic range in accordance with a predicted numerical value (natural light index) that quantitatively indicates whether there is a backlight scene or an overshoot scene Must be defined. Therefore, a “first mixture coefficient” of a one-dimensional LUT (input: luminance ratio index, output: first temporary correction amount) for setting “first temporary correction amount”, assuming brightness correction of a human face area in a backlight scene. ”Is defined so as to define a one-dimensional LUT that defines the relationship between the“ natural light index ”and the“ first mixing coefficient ”.

The “second provisional correction amount” is calculated based on the first provisional correction amount and the first mixing coefficient, and is defined as Expression (1). In Expression (1), among the plurality of first temporary correction amounts set by the first temporary correction amount setting means, m first temporary correction amounts are set as calculation targets, and the i-th first temporary correction amount is set as key_auto. [i], the first mixing coefficient to be multiplied by the first temporary correction amount key_auto [i] is wgt [i].

  According to the present invention, the brightness of the main subject (for example, a human face region) is optimized without changing the existing automatic exposure control, the operating point change, and the gradation processing condition from the linear image to the non-linear image. It becomes possible.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, the configuration of the first embodiment (imaging device) according to the present invention will be described.

(Configuration of the imaging device 1)
FIG. 1A shows a front view of the imaging apparatus 1 according to the embodiment of the present invention, and FIG. 1B shows a rear view of the imaging apparatus 1. The imaging device 1 is, for example, a digital camera, and has a cross key 22, a photographing optical system 23, a flash 24, a viewfinder 25, and a power switch 26 on or inside a housing 21 made of a material such as metal or synthetic resin. A display unit 27 and a release button 28 are provided.

  FIG. 2 shows an internal configuration of the imaging apparatus 1. As shown in FIG. 2, the imaging apparatus 1 includes a processor 31, a memory 32, an imaging device 33 such as a CCD (Charge Coupled Device), a photographing optical system 23, a timing generator 41, a shutter unit 42, a diaphragm unit 43, and a focus unit 44. , A display unit 27, an operation unit 38, and an image data output unit 37. The processor 31 includes a photographing processing unit 20 that performs photographing control and processing of a photographed image, and an image processing unit 10 that performs image processing.

  The cross key 22 is made up of buttons in four directions, up, down, left, and right, and is for the user to select or set various modes.

  The photographing optical system 23 includes a plurality of lenses, a lens barrel, and the like, and has a zoom function. The photographing optical system 23 causes the light received by the lens to form an image on the image sensor 33. The flash 24 emits auxiliary light by a control signal from the processor 31 when the subject brightness is low.

  The viewfinder 25 is used for the user to check the shooting target and shooting area by eye contact. The power switch 26 is a switch for operating ON / OFF of the operation in the imaging apparatus 1.

  The display unit 27 includes a liquid crystal panel, and displays an image currently captured on the image sensor 33, an image captured in the past, a menu screen, a setting screen, and the like according to a display control signal input from the processor 31.

  The release button 28 is a two-stage push-in switch that is provided on the upper surface of the housing 21 and can detect a half-pressed state (preliminary shooting) and a fully-pressed state (main shooting) by the user.

  The memory 32 stores (saves) image data obtained by photographing. In addition, the memory 32 stores various processing programs executed in the imaging apparatus 1, data used in the processing programs, and the like.

  The image data output unit 37 transfers and records the image data in the memory 32 to a storage medium for storage (such as an SD memory card or a multimedia card (MMC)), or transfers it to an external device. The image data output unit 37 is controlled by the processor 31.

  The image sensor 33 converts the imaged light into electric charges. Thereby, for example, image data as shown in FIG. This image includes an object in the imaging range (imaging range), that is, an imaging object (imaging target) and other objects (background). The RGB value of each pixel of the entire image is represented by, for example, 256 gradations.

  The shutter unit 42 controls the timing of resetting the image sensor 33 and the timing of charge conversion based on the state (half-pressed state or fully-pressed state) detected by the release button 28. Timing control is performed by the timing generator 41. The exposure amount control by the shutter unit 42 will be described later.

  Adjustment of the amount of light received by the image sensor 33 is performed by the aperture unit 43 and / or the shutter unit 42. The focus unit 44 performs a control operation for driving the photographing optical system 23 to focus on the photographing subject. Focus control by the focus unit 44 will be described later.

(Internal configuration of the imaging processing unit 20)
FIG. 3 shows an internal configuration of the imaging processing unit 20. The shooting processing unit 20 performs control related to shooting conditions during shooting and processing of shot images. As shown in FIG. 3, the imaging processing unit 20 includes an AE control unit 51 and an AF control unit 52 related to imaging conditions, a pixel interpolation unit 53 that performs image processing on the captured image, an AWB control unit 54, and a gamma. It is comprised by the correction | amendment part 55 grade | etc.,.

  The AE control unit 51 automatically controls the exposure amount at the time of image shooting. Normally, the exposure amount control during shooting standby is performed by controlling the shutter speed with the aperture opened, and the exposure amount during shooting is controlled by the aperture and shutter speed.

  For example, the exposure amount control during shooting standby in the AE control unit 51 is performed as follows (see the flowchart in FIG. 4A).

  First, the diaphragm unit 43 sets the diaphragm to an open fixed diaphragm (step S111). The AE control unit 51 reads data of a predetermined photometric area from the image data obtained by the image sensor 33 (step S112), and acquires information corresponding to the luminance value (step S113). Information corresponding to the luminance value is used as information for AE control, and the G value of RGB three-color components is often used simply (hereinafter referred to as luminance information G). The charge accumulation time in the next frame of the image sensor 33 is set according to the luminance information G (step S114), and the charge accumulation time in the next frame is controlled by the timing generator 41 so that the predetermined luminance level is obtained. (Step S117). This is control of the shutter speed, and FIG. 5 shows how charges are accumulated. That is, when the acquired luminance level is large (bright), the charge accumulation time is shortened, and when the luminance level is small (dark), the charge accumulation time is lengthened, thereby stabilizing the exposure amount.

  By performing exposure amount control during shooting standby in this way, a photographer can observe a live view image with exposure amount controlled on the display unit 27 such as a liquid crystal display.

  In the exposure control at the time of actual photographing in the AE control unit 51, in addition to the shutter speed control described above, aperture control is also performed (see the flowchart in FIG. 4B). Similarly to the above (steps S112 and S113), the aperture unit 43 is controlled according to the luminance information G of the photometric area, and an aperture value is set (step S116). That is, the exposure value is stabilized by decreasing the aperture value when the acquired luminance level is high (bright) and increasing the aperture value when the luminance level is low (dark). The adjustment level combined with the shutter speed is controlled based on data of a predetermined program diagram, for example, such that the charge accumulation time of the image sensor 33 is adjusted according to the aperture value (step S115, S117).

  Thus, by performing exposure amount control at the time of photographing, the photographer can automatically perform exposure amount setting for a photographed image with an arbitrary or predetermined combination of aperture value and shutter speed.

  The AF control unit 52 performs automatic control for focusing an image when capturing an image. This in-focus control is performed by, for example, driving the photographing optical system as described below to detect the in-focus position and stopping it according to the position (see the flowchart in FIG. 6).

  When driving of the photographic optical system 23 is started (step S121), the AF control unit 52 sequentially reads data of a predetermined distance measuring area from the image data obtained by the image sensor 33 (step S122). ), Contrast information is acquired according to this data (step S123). This is for detecting whether or not the in-focus position has been reached, and is determined as follows. That is, the contrast information is set and calculated so as to depend on the sharpness of the edge portion by taking the difference between adjacent pixels in the distance measurement area data. The state that has reached the maximum is determined to be the focal point (steps S124 and S125). If it is determined that the focus position is not reached, the movement of the photographing optical system is continued (step S126).

  FIG. 7 shows a change in contrast information accompanying the movement of the photographic optical system 23 and how the focal point position is detected. In the above operation, a distance measurement calculation is performed in which contrast information is sequentially acquired while driving the optical system to obtain a focal point position, and the photographing optical system 23 is stopped in accordance with the focal length ( Step S127).

  With such AF control, it is possible to obtain a captured image that is always in focus automatically during shooting.

  The pixel interpolation unit 53 interpolates pixels for each color component with respect to the CCD array in which the RGB color components are dispersedly arranged in the image sensor 33, and processes the image data so that each color component value is obtained at the same pixel position. (Refer to the flowchart of FIG. 8).

  The pixel interpolation unit 53 masks the RGB image data (step S141) obtained by the image sensor 33 with each of the RGB pixel filter patterns (steps S142, S144, and S146), and then performs average interpolation (also referred to as pixel interpolation). Perform (Steps S143, S145, S147). Among these, the G pixel filter pattern having pixels up to a high band is a median filter that performs average interpolation by substituting the average value of the intermediate binary values of the surrounding four pixels, and the R and B pixel filters The pattern performs average interpolation with respect to the same color from 9 neighboring pixels.

  The AWB control unit 54 automatically adjusts the white balance in the captured image. Assuming that the photographed image has a subject area in which RGB color balance is achieved (white when summed up), each component value of RGB of the image is achieved so as to achieve white balance in that area. Adjust the level for. This white balance processing is performed, for example, as follows (see the flowchart of FIG. 9).

  The AWB control unit 54 estimates a region that is supposed to be white from the luminance and saturation data of the image data obtained by the image sensor 33 (step S131) (step S132). For that region, the average intensity of each RGB component value, the G / R ratio, and the G / B ratio are obtained, and the R value and B value correction gains for the G value are calculated (steps S133 and S134). Based on this, gain correction for each color component in the entire image is performed (step S135).

  By such AWB control, it is possible to automatically correct the collapse of the color balance of the entire screen that occurs during shooting, and an image with a stable color tone can be obtained regardless of the actual illumination state of the subject.

  The gamma correction unit 55 performs processing for converting the gradation of the captured image so as to be suitable for the characteristics of the output medium.

  The gamma characteristic is a gradation characteristic, and indicates how an output gradation is set with respect to an input gradation by a correction value or a correction curve. FIG. 10 shows an example of a correction curve indicating the output value corrected for the input value. The gamma correction is a conversion process for converting an input value into an output value according to the correction value or the correction curve. Since this gradation characteristic varies depending on the output medium, it is necessary to correct this gamma characteristic in order to obtain a gradation suitable for the output medium. As a result, the linear image is converted into a non-linear image.

  A monitor is generally set as the output medium, and gamma correction of the captured image is performed so as to match the gamma characteristic of a general monitor.

  In the present embodiment, the gamma correction unit 55 performs processing for converting the linear image that has been subjected to the tone conversion processing by the tone conversion processing unit 108 in the image processing unit 10 into a non-linear image. Accordingly, the gamma correction unit 55 functions as a third gradation conversion processing unit.

  An example of the gamma characteristic applied to the linear image is shown in FIG. 10, but a plurality of gamma characteristics are set in advance according to a specific output device and observation condition, and selected from the plurality of gamma characteristics. It is preferable to do so.

(Internal configuration of the image processing unit 10)
FIG. 11 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls a gradation correction operation based on scene discrimination for a captured image by the imaging apparatus 1, and after the processing by the shooting processing unit 20 or independently of the processing. A series of processes such as image acquisition, scene discrimination, and gradation processing condition setting are executed.

  As shown in FIG. 11, the image processing unit 10 includes a first image acquisition unit 101, a second image acquisition unit 102, an occupation rate calculation unit 103, an index calculation unit 104, a scene determination unit 105, and a gradation processing condition setting unit 107. , And a gradation conversion processing unit 108.

  The first image acquisition unit 101 acquires the image data of the latest image captured on the image sensor 33 as the first image when the release button 28 is fully pressed. Here, the first image corresponds to the linear image of the present invention. The first image acquisition unit 101 also acquires luminance information calculated at the time of shooting. That is, the first image acquisition unit 101 functions as a luminance information acquisition unit. The luminance information acquired in the present embodiment is a luminance value (Brightness_Value, hereinafter referred to as BV), which is calculated from a photometric value at the time of photographing and is additionally recorded in the image data as Exif (Exchangeable_Image_File_Format) data. The acquired image data and luminance information BV of the first image are held in the memory 32.

The second image acquisition unit 102 divides the acquired first image into N × M rectangular regions (regions divided into M pieces in the vertical direction and N pieces in the horizontal direction). FIG. 12A shows an example of the first image, and FIG. 12B shows an example in which the first image is divided into 22 × 14 regions. The number of divided areas is not particularly limited. In the present embodiment, each divided area is regarded as a “pixel” of image data, and the divided image is handled as an image with a substantially reduced size. This reduced image is the second image, and the second image is acquired by the above operation.
As described above, the first image acquisition unit 101 and the second image acquisition unit 102 function as an image acquisition unit.

  The occupation ratio calculation unit 103 acquires color information for the image data of the second image, that is, for each pixel constituting the image. Based on the acquired color information, each pixel of the image is classified into a predetermined class composed of a combination of brightness and hue (see FIG. 30), and for each classified class, the pixels belonging to the class are included in the entire image. A first occupation ratio indicating the occupation ratio is calculated. That is, the occupation ratio calculation unit 103 functions as a first occupation ratio calculation unit.

  Further, the occupancy rate calculation unit 103 classifies each pixel into a predetermined class composed of a combination of the distance from the outer edge of the screen of the second image and the brightness (see FIG. 31), and for each classified class, A second occupation ratio indicating a ratio of pixels belonging to the class to the entire image is calculated. That is, the occupation ratio calculation unit 103 also functions as a second occupation ratio calculation unit.

  The occupation rate calculation processing executed in the occupation rate calculation unit 103 will be described in detail later with reference to FIG.

  The index calculation unit 104 multiplies the first occupancy calculated by the occupancy rate calculation unit 103 by a coefficient set in advance according to the shooting conditions, thereby specifying an index 1 and an index 2 for specifying a shooting scene. Is calculated. That is, the index calculation unit 104 functions as a first index calculation unit and a second index calculation unit.

  In addition, the index calculation unit 104 multiplies the second occupancy calculated by the occupancy rate calculation unit 103 by a coefficient set in advance according to the shooting condition, thereby obtaining an index 3 for specifying the shooting scene. calculate. That is, the index calculation unit 104 also functions as a third index calculation unit.

  Furthermore, the index calculation unit 104 multiplies each of the average luminance value in the central portion of the screen of the second image and the difference value between the maximum luminance value and the average luminance value by a coefficient set in advance according to the shooting condition. Thus, an exposure shooting degree index for specifying a shooting scene is calculated. That is, the index calculation unit 104 also functions as a fourth index calculation unit.

  In addition, the index calculation unit 104 includes an average brightness value (referred to as index 7) of the skin color area in the center of the screen of the second image, index 1 and index 3, and the above-described brightness information BV at the time of shooting, respectively. Then, a new natural light index is calculated by multiplying a coefficient set in advance according to the photographing condition to obtain a sum. That is, the index calculation unit 104 also functions as a fifth index calculation unit.

  In addition, the index calculation unit 104 multiplies the average brightness value, the index 2 and the index 3, and the above-described brightness information BV at the time of shooting by a coefficient set in advance according to the shooting conditions. Thus, a new luminance ratio index is calculated. That is, the index calculation unit 104 also functions as a sixth index calculation unit.

  The index calculation process executed in the index calculation unit 104 will be described in detail later with reference to FIG.

  The scene determination unit 105 determines the shooting scene of the first image based on each index calculated by the index calculation unit 104. That is, the scene determination unit 105 functions as a scene determination unit. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes over and under degrees. The method for determining the shooting scene will be described later in detail.

  As described above, the occupation rate calculation unit 103, the index calculation unit 104, and the scene determination unit 105 function as a scene determination processing unit.

  The gradation processing condition setting unit 107 sets gradation processing conditions for the first image based on the shooting scene determined by the scene determination unit 105. That is, the gradation processing condition setting unit 107 functions as a gradation processing condition setting unit. The gradation processing condition setting process will be described in detail later with reference to FIG.

  The gradation conversion processing unit 108 executes gradation conversion processing for the first image (linear image) according to the gradation processing conditions set by the gradation processing condition setting unit 107. Therefore, the gradation conversion processing unit 108 functions as first and second gradation conversion processing means.

  In addition to the processing performed in the image processing unit 10, the processor 31 performs processing in the photographing processing unit 20 such as automatic white balance processing and gamma conversion processing, and other image processing, image format conversion, It has a function of controlling processing operations such as recording of image data.

  The processing of each unit in the processor 31 is basically performed by hardware processing, but a part of the processing is performed by software processing by executing a program stored (saved) in the memory 32.

  FIG. 13 shows an internal configuration of the gradation processing condition setting unit 107. The gradation processing condition setting unit 107 includes a brightness correction condition setting unit 107A and a flare correction condition setting unit 107B. The brightness correction condition setting unit 107A includes a first temporary correction amount setting unit 201, a first mixing coefficient setting unit 202, a second temporary correction amount calculation unit 203, a second mixing coefficient setting unit 204, and a brightness correction amount calculation unit 205. Consists of. The flare correction condition setting unit 107B includes a transition coefficient setting unit 206, a weighting coefficient setting unit 207, and a flare correction amount calculation unit 208.

  The brightness correction condition setting unit 107A sets a brightness correction amount for correcting the brightness of the linear image as a gradation processing condition. That is, the brightness correction condition setting unit 107A functions as a gradation processing condition setting unit that sets the first gradation processing condition.

  The first temporary correction amount setting unit 201 uses the linear image brightness analysis value, the brightness reproduction target value, the luminance ratio index calculated by the index calculation unit 104, and the exposure shooting degree index, to determine the linear image. A first temporary correction amount for is set. When n brightness analysis values are input, the first temporary correction amount setting unit 201 sets the first temporary correction amount as a correction amount necessary to approximate each brightness analysis value to the reproduction target value. Set n + 4. That is, the first temporary correction amount setting unit 201 functions as first temporary correction amount setting means.

  FIG. 14 shows a one-dimensional LUT (Look Up Table) that defines the relationship between the luminance ratio index and the first temporary correction amount key_auto [0], and FIG. 15 shows the exposure shooting degree index and the first temporary correction amount key_auto [0]. 1] shows a one-dimensional LUT that defines the relationship with 1], FIG. 16 shows a one-dimensional LUT that defines the relationship between the skin color average luminance and the first temporary correction amount key_auto [2], and FIG. 17 shows the skin color average luminance. Alternatively, a one-dimensional LUT that defines the relationship between the overall average luminance and the first temporary correction amount key_auto [3] is shown, and FIG. 18 illustrates the relationship between the luminance ratio index and the first temporary correction amount key_auto [4]. FIG. 19 shows a one-dimensional LUT that defines the relationship between the exposure shooting index and the first temporary correction amount key_auto [5].

  In the present embodiment, the brightness analysis value input to the gradation processing condition setting unit 107 is the skin color average brightness and the overall average brightness (that is, n = 2). In this case, the first temporary correction amount setting unit 201 uses the one-dimensional LUT shown in FIGS. 14 to 19 as six temporary correction amounts (key_auto [0] to [5]) as the first temporary correction amounts. Is set.

  Based on the natural light index calculated by the index calculation unit 104, the first mixing coefficient setting unit 202 sets n + 2 (key_auto) of the n + 4 first temporary correction amounts set by the first temporary correction amount setting unit 201. N + 2 first mixing coefficients are set as weighting coefficients for multiplying [0] to [3]). That is, the first mixing coefficient setting unit 202 functions as a first mixing coefficient setting unit. 20 to 23 show one-dimensional LUTs that define the relationship between the natural light index and the first mixing coefficients wgt [0] to wgt [3].

  Note that the one-dimensional LUTs of FIGS. 14 to 19 and the one-dimensional LUTs of FIGS. 20 to 23 are stored in the memory 32.

The second temporary correction amount calculation unit 203 includes n + 2 first temporary correction amounts (key_auto [0] to [3]) and n + 2 first mixing coefficients wgt [set by the first mixing coefficient setting unit 202. Based on 0] to wgt [3], the second temporary correction amount for the linear image is set. That is, the second temporary correction amount calculation unit 203 functions as a second temporary correction amount setting unit. Formula (2) shows the formula for calculating the second temporary correction amount.
Second temporary correction amount = key_auto [0] × wgt [0] + key_auto [1] × wgt [1] + key_auto [2] × wgt [2] + key_auto [3] × wgt [3] (2)
Henceforth, the 2nd temporary correction amount of Formula (2) is described with key_auto [6].

  The second mixing coefficient setting unit 204 is based on the values of the natural light index, the luminance ratio index, the exposure shooting degree index calculated by the index calculation unit 104, and the discrimination map for discriminating the shooting scene. Three second mixing coefficients wgt [4] to [6] are set as weight coefficients to be multiplied to the temporary correction amounts key_auto [4] and key_auto [5] and the second temporary correction amount key_auto [6]. That is, the second mixing coefficient setting unit 204 functions as a second mixing coefficient setting unit.

  FIG. 24A shows a discrimination map for discriminating a linear image shooting scene (forward light, backlight, main subject is over, under, low accuracy region) based on the natural light index and the luminance ratio index. FIG. 5B shows a discrimination map for discriminating a linear image shooting scene (forward light, backlight, main subject is over, under, low accuracy region) based on the natural light index and the exposure shooting index. In FIG. 24, the low accuracy region represents a region where the accuracy with which forward light, backlight, main subject is determined to be over or under is low (lower than a predetermined value).

  FIG. 25 shows a coefficient calculation table representing the values of the second mixing coefficients wgt [4] to [6] for each shooting scene. This coefficient calculation table is stored in the memory 32. In FIG. 25, each number shown in the shooting scene item is a number given to each shooting scene in FIG. For example, in FIG. 25, “3” of the shooting scene item represents the low accuracy region (3) of FIG.

The brightness correction amount calculation unit 205 is set by the two first temporary correction amounts key_auto [4] and key_auto [5], the second temporary correction amount key_auto [6], and the second mixing coefficient setting unit 204. From the three second mixing coefficients wgt [4] to [6], a brightness correction amount as a gradation adjustment parameter is calculated as shown in Expression (3A).
Brightness correction amount = key_auto [4] × wgt [4] + key_auto [5] × wgt [5] + key_auto [6] × wgt [6] (3A)
That is, the brightness correction amount calculation unit 205 functions as a gradation adjustment parameter calculation unit.

  The flare correction condition setting unit 107B sets a flare correction amount for adjusting the hardness of the linear image as a gradation condition. That is, the flare correction condition setting unit 107B functions as a gradation processing condition setting unit that sets the second gradation processing condition.

  When the brightness of a linear image is corrected by the brightness correction amount calculated by the brightness correction condition setting unit 107A, if this linear image is a backlight scene, an offset region that is potentially present in the shadow portion of the backlight scene is present. In some cases, the image becomes obvious by correction for brightening the image and becomes a flare image. In order to perform flare correction for preventing the occurrence of such a flare image, the flare correction amount setting unit 107B calculates the flare correction amount.

  The transition coefficient setting unit 206 calculates and sets the transition coefficient w based on the natural light index and the luminance ratio index calculated by the index calculation unit 104. The weighting factor setting unit 207 calculates and sets a weighting factor fw as a luminance ratio index-dependent weight allocation coefficient based on the luminance ratio index calculated by the index calculation unit 104.

The flare correction amount calculation unit 208 is based on the transition coefficient w set by the transition coefficient setting unit 206, the weighting factor fw set by the weighting factor setting unit 207, and the preset flare correction reference value fc. Using the following equation (3B), the flare correction amount flareCorr is calculated and output to the gradation conversion processing unit 108.
flareCorr = fc × fw × w (3B)
For example, the flare correction reference value fc is a fixed value of 32, and is set in advance and stored in the memory 32.

  Here, an example of the transition coefficient w and the weighting coefficient fw will be described with reference to FIG. FIG. 26A shows an example of the transition coefficient w and the weight coefficient fw corresponding to the scene discrimination map. FIG. 26B shows another example of the weighting factor fw. The flare correction is performed on the backlight region (2) and the low accuracy regions (3) and (4) in FIG.

  As shown in FIG. 26 (a), the transition coefficient w is 1 in the backlight region (2), and is a normal value from 0 to 1 in the low accuracy region (3) where the natural light index is −1.5 or less. The normalized transition coefficient w1 is taken, and the normalized transition coefficient w2 from 0 to 1 is taken in the area where the luminance ratio index is 1.5 or more in the low accuracy area (4). The transition coefficient w is a transition in the low accuracy region (3) (4) in the region where the natural light index is from −1.5 to 0.5 and the luminance ratio index is from −0.5 to 1.5. A larger value is adopted among the coefficients w1 and w2.

  Further, as shown in FIG. 26A, the weighting factor fw is 1 when the luminance ratio index is 1.5 or more, and is 0 when the luminance ratio index is −0.5 to 1.5. Takes a normalized value from 1 to 1. The weighting factor fw is not limited to the example of FIG. 26A. For example, as shown in FIG. 26B, the weighting factor fw is normal from 0 to 1 in the region where the luminance ratio index is −0.5 or more. It is also possible to take a normalized value.

(Operation Flow of Imaging Device 1)
Next, the operation of the imaging device 1 in the present embodiment will be described.

  Hereinafter, the photographing object is referred to as “main subject”.

  First, an overall flow of processing executed in the imaging apparatus 1 will be described with reference to a flowchart of FIG.

  First, when the power switch 26 is turned on (when the power is turned on), preprocessing such as resetting of the memory 32 is performed (step S1). The user starts an operation for photographing by directing the imaging device 1 toward the main subject so that the main subject enters the field of the imaging device 1. Next, the release button 28 is pressed to perform shooting (step S2). An image formed on the image sensor 33 by photographing is captured as an electrical signal, and interpolation processing based on the CCD array is performed (step S4). Next, a first image (linear image) is acquired as a captured image (step S <b> 5) and held in the memory 32.

  On the other hand, when photographing is performed, the luminance information BV is captured in step S3, which is a luminance information acquisition step. The luminance information BV acquired in step S3 is calculated from the photometric value at the time of photographing, and is recorded together when the photographed image is recorded in the Exif format later. The acquired luminance information BV is held in the memory 32.

  AWB (automatic white balance) processing is performed on the first image (linear image) acquired by photographing (step S6). This may be processed separately for the first image and the second image after acquisition of a second image (reduced linear image) described below.

  After the AWB process, the image data of the first image is acquired as a divided image composed of a plurality of divided regions, that is, a second image (reduced linear image) (step S7). Steps S5 and S7 correspond to the image acquisition process of the present invention. Each divided region of the divided image is a pixel of the second image, and the second image is an image obtained by reducing the size of the first image. FIG. 12A shows an example of the first image, and FIG. 12B shows an example in which the first image is divided into 22 × 14 cells. Each cell corresponds to one pixel of the second image. As an actual image data size reduction method, a known technique such as a simple average, a bilinear method, or a bicubic method can be used.

  Next, in step S8, which is a scene determination process step, a scene determination process for determining a shooting scene is performed based on the image data of the second image acquired in step S7. The scene determination process in step S8 will be described in detail later with reference to FIG.

  Next, in step S9 which is a gradation processing condition setting step, it is necessary for the gradation conversion processing of the first image based on each index obtained in the scene determination processing in step S8 and the determination result of the shooting scene. Processing for setting conditions is performed. The gradation processing condition setting process in step S9 will be described later with reference to FIG.

  In step S10, which is the first gradation conversion processing step, the image data of the first image that is the photographed image is processed based on the gradation processing condition (brightness correction amount) set based on the second image. A gradation conversion process is performed. The gradation conversion processing in step S10 is to correct the influence on the gradation due to where the photographic gradation is reproduced with respect to the scene luminance or the light source condition of the shooting scene, and the scene determination in step S8. The result is based on the gradation processing condition (brightness correction amount) set in step S9.

  In step S11, which is the second gradation conversion process step, the first gradation conversion process is performed in step S10 based on the gradation processing condition (flare correction amount) set based on the second image. The flare correction process is performed on the image data of the first image. The flare correction process in step S11 is to prevent a flare image by correcting a potential offset region in the shadow part of the backlight scene. The gradation processing condition (set in step S9 based on the scene determination result in step S8 ( (Flare correction amount).

  In step S12, which is the third gradation conversion process, the linear image is converted into a non-linear image by performing a gamma conversion process on the first image (linear image) after the AWB process and the gradation conversion process. The

  In this embodiment, gradation processing conditions are set in step S9 and gradation conversion processing is performed in step S10. In addition to image processing for the first image, for example, an optimal image for the shooting scene is used. It is also possible to perform processing such as calculating an exposure condition for acquiring the image in real time and reflecting it in the photographing operation.

  Next, other image processing (noise processing, sharpness processing, contrast correction processing, dodging processing, etc.) is appropriately performed on the image data after the gamma conversion processing (step S13). Thereafter, the image format is converted for image recording (step S14). Generally, it is converted into an image in JPEG format. Thereafter, the image data in JPEG format is recorded on a storage medium for storage (such as an SD memory card or a multimedia card (MMC)) (step S15). In the image recording in the Exif format, the luminance information BV is additionally recorded as Exif data. When the next shooting is started or the power switch 26 is turned off, the operation of the image pickup apparatus 1 ends.

  Note that the linear image after gradation conversion processing may be stored as Raw data and converted to a non-linear image using development software.

(Scene discrimination process 1 flow)
Next, with reference to the flowchart in FIG. 28, the scene determination process (step S8 in FIG. 27) executed in the image processing unit 10 will be described.

  As shown in FIG. 28, the scene discrimination process includes a color space conversion process (step S20), an occupation rate calculation process (step S21), an index calculation process (step S22), and a scene discrimination (step S23). The Hereinafter, with reference to FIGS. 29 to 36, each process shown in FIG. 28 will be described in detail.

  In the color space conversion process in step S20, first, information indicating the RGB value, the luminance value, and the white balance of each pixel of the second image obtained from the captured first image is acquired. As the luminance value, a value calculated by substituting RGB values into a known conversion formula may be used. Next, the acquired RGB values are converted into the HSV color system, and the color information of the image is acquired. The HSV color system represents image data with three elements: Hue, Saturation, and Lightness (Value or Brightness), and was devised based on the color system proposed by Munsell. It has been done. The conversion to the HSV color system is performed using an HSV conversion program or the like. Usually, the calculated hue value H is defined as 0 to 360 for the input R, G, B, The unit of saturation value S and brightness value V is defined as 0-255.

  In the present embodiment, “brightness” means “brightness” which is generally used unless otherwise noted. In the following description, V (0 to 255) of the HSV color system is used as “brightness”, but a unit system representing the brightness of any other color system may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated.

In the present embodiment, “hue” means “color” that is generally used unless otherwise noted. In the following description, H (0 to 360) of the HSV color system is used as “hue”, but for example, a color represented by a red difference value (Cr) or a blue difference value (Cb) may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated. In step S20, the values of H, S, and V obtained as described above are acquired as color information.
<Occupancy rate calculation process 1>
Next, the occupation rate calculation process in step S21 will be described with reference to the flowchart in FIG.

  First, based on the HSV value calculated by the color space conversion process, each pixel of the second image is classified into a predetermined class composed of a combination of hue and brightness, and the cumulative number of pixels is calculated for each classified class. As a result, a two-dimensional histogram is created (step S30).

  FIG. 30 shows a class composed of combinations of brightness and hue. In step S30, the lightness (V) is 0-5 (v1), 6-12 (v2), 13-24 (v3), 25-76 (v4), 77-109 (v5), 110-110. It is divided into seven classes of 149 (v6) and 150 to 255 (v7). As shown in FIG. 30, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  Hue (H) is a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region having a hue value of 161 to 250. It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the shooting scene is small. The skin color hue region is further divided into a skin color region (H1) and another region (H2). Hereinafter, among the flesh-color hue regions (H = 0 to 39, 330 to 359), the hue ′ (H) that satisfies the following equation (5) is defined as the flesh-color region (H1), and the region that does not satisfy the equation (5) is ( H2).

10 <saturation (S) <175;
Hue '(H) = Hue (H) +60 (when 0 ≦ hue (H) <300);
Hue ′ (H) = Hue (H) −300 (when 300 ≦ hue (H) <360).
Luminance (Y) = R × 0.30 + G × 0.59 + B × 0.11 (4)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (5)
Therefore, the number of classes in the second image is 4 × 7 = 28. Moreover, it has at least three classes (v1, v2, v3) within 10% of the maximum brightness value (255). In addition, it is also possible to use lightness (V) in Formula (4) and Formula (5).

  After step S30, each pixel of the second image is classified into a predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness, and the two-dimensional histogram is calculated by calculating the cumulative number of pixels for each classified class. Is created (step S31).

  FIG. 31A shows three regions n1 to n3 divided in accordance with the distance from the outer edge of the screen of the second image in step S31. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. A region n3 is a central region of the second image. Here, it is preferable that n1 to n3 are divided so as to have substantially the same number of pixels. In the present embodiment, three divisions are used, but the present invention is not limited to this. In step S31, the brightness is divided into seven areas v1 to v7 as described above. FIG. 31B shows a class composed of combinations of three regions n1 to n3 and brightness. As shown in FIG. 31B, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 3 × 7 = 21.

  When a two-dimensional histogram is created in step S30, a first occupancy ratio indicating a ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of brightness and hue to the total number of pixels (N × M) Is calculated (step S32). That is, step S30 and step S32 correspond to the first occupancy rate calculating step.

Assuming that the first occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy in each class is expressed as shown in Table 1.

  When the two-dimensional histogram is created in step S31, the second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness to the total number of pixels is obtained. Calculation is performed (step S33), and the occupancy rate calculation process ends. That is, step S31 and step S33 correspond to the second occupancy rate calculating step.

When the second occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each class is expressed as shown in Table 2.

Note that a three-dimensional histogram may be created by classifying each pixel into a class consisting of a distance from the outer edge of the screen, brightness, and hue, and calculating the cumulative number of pixels for each class. Hereinafter, a method using a two-dimensional histogram is adopted.
<Indicator calculation process 1>
Next, the index calculation process (step S22 in FIG. 28) executed in the index calculation unit 104 will be described with reference to the flowchart in FIG.

  First, in step S40, which is a first index calculation step, the first occupancy calculated for each class in the occupancy ratio calculation process is multiplied by a coefficient (first coefficient) set in advance according to the shooting conditions. As a result, the index 1 for specifying the shooting scene is calculated. The index 1 is an index representing the degree of overshoot of the main subject, and is for separating only the image that should be determined as “the main subject is over” from other shooting scenes.

  Next, in step S41, which is the second index calculation step, a coefficient (second coefficient) different from the first coefficient preset in accordance with the shooting conditions is added to the first occupancy ratio calculated for each class. ) To obtain the sum, the index 2 for specifying the shooting scene is calculated. The index 2 is an index that compositely represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness, and only images that should be determined as “backlight” and “main subject is under” are taken from other shooting scenes. It is for separation.

  Hereinafter, the calculation method of the index 1 and the index 2 will be described in detail.

Table 3 shows the first coefficient necessary for calculating the index 1 by class. The coefficient of each class shown in Table 3 is a weighting coefficient by which the first occupation ratio Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 3, a positive (+) coefficient is used for the first occupancy calculated from the region distributed in the skin color hue region (H1) of high brightness (v6), and the other hue is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region. FIG. 33 shows a curve (coefficient curve) in which the first coefficient in the skin color area (H1) and the first coefficient in the other areas (green hue area (H3)) change continuously over the entire brightness. It is shown. According to Table 3 and FIG. 33, in the region of high brightness (V = 77 to 150), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of both are different.

When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (6).

Therefore, the sum of the H1 to H4 regions is expressed as the following formulas (6-1) to (6-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × (−8) (6-1)
H2 region sum = R12 × (−2) + R22 × (−1) + (omitted)… + R72 × (−10) (6-2)
Sum of H3 regions = R13 × 5 + R23 × (−2) + (omitted)… + R73 × (−12) (6-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)… + R74 × (−12) (6-4)

The index 1 is defined as in Expression (7) using the sum of the H1 to H4 regions shown in Expressions (6-1) to (6-4).
Index 1 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.5 (7)

Table 4 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 4 is a weighting coefficient by which the first occupancy rate Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 4, a negative (-) coefficient is used for the occupancy calculated from the areas (v4, v5) distributed in the intermediate brightness of the flesh-color hue area (H1), and the flesh-color hue range (H1) is low. A coefficient 0 is used for the occupation ratio calculated from the brightness (shadow) region (v2, v3). FIG. 34 shows the second coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 4 and FIG. 34, the sign of the second coefficient of the intermediate lightness region having a lightness value of 25 to 150 in the flesh color hue region is negative (−), and the low lightness (shadow) region having a lightness value of 6 to 24. The second coefficient is 0, and it can be seen that there is a large difference in the coefficients in both regions.

When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (8).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (8-1) to (8-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × 2 (8-1)
Sum of H2 regions = R12 × (−2) + R22 × (−1) + (omitted)… + R72 × 2 (8-2)
Sum of H3 regions = R13 × 2 + R23 × 1 + (omitted) ... + R73 × 3 (8-3)
Sum of H4 regions = R14 × 0 + R24 × (−1) + (omitted)… + R74 × 3 (8-4)

The index 2 is defined as in Expression (9) using the sum of the H1 to H4 regions shown in Expressions (8-1) to (8-4).
Index 2 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.7 (9)

  Since the index 1 and the index 2 are calculated based on the brightness and hue distribution amount of the second image, they are effective in determining a shooting scene when the image is a color image.

  When the index 1 and the index 2 are calculated, the second occupancy ratio calculated for each class in the occupancy ratio calculation process is set in advance according to the shooting conditions in step S42 which is the third index calculation step. By multiplying the third coefficient (coefficient different from the first coefficient and the second coefficient) and taking the sum, an index 3 for specifying the shooting scene is calculated. The index 3 indicates a difference in contrast between the center and the outside of the screen of the image data between the backlight with the main subject under and the image with the main subject over.

  Hereinafter, a method for calculating the index 3 will be described.

Table 5 shows the third coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 35 shows the third coefficient in the screen areas n1 to n3 as a curve (coefficient curve) that continuously changes over the entire brightness.

If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (10).

Accordingly, the sum of the n1 to n3 regions is represented by the following formulas (10-1) to (10-3).
Sum of n1 region = Q11 × 12 + Q21 × 10 + (omitted) ... + Q71 × 0 (10-1)
Sum of n2 regions = Q12 × 5 + Q22 × 3 + (omitted)… + Q72 × 0 (10-2)
Sum of n3 regions = Q13 × (−1) + Q23 × (−4) + (omitted)… + Q73 × (−8) (10-3)

The index 3 is defined as in Expression (11) using the sum of the n1 to n3 regions shown in Expressions (10-1) to (10-3).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + 0.7 (11)

  Since the index 3 is calculated based on the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the second image, the photographing scene of the monochrome image as well as the color image is discriminated. It is also effective.

  Further, for example, according to the focus detection area detected by a known method, the weight of the third coefficient multiplied by the second occupancy calculated from the distance from the outer edge of the screen and the predetermined class of brightness is set. By changing, it is possible to calculate an index for discriminating a scene with higher accuracy.

  When the indices 1 to 3 are calculated, in step S43, which is the fourth index calculation step, the average luminance value of the skin color in the screen center portion of the second image, the maximum luminance plantation and the average luminance value of the second image By multiplying each of the difference values by a coefficient set in advance according to the shooting conditions, an exposure shooting degree index for specifying the shooting scene is calculated.

  Hereinafter, with reference to the flowchart of FIG. 36, the calculation process of the exposure shooting degree index will be described in detail.

  First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the second image using Equation (4). Next, the average luminance value x1 of the skin color area at the center of the screen of the second image is calculated (step S50). Here, the screen center portion is, for example, a region constituted by the region n3 shown in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the second image is calculated (maximum luminance value−average luminance value) (step S51).

Next, the standard deviation x3 of the luminance of the second image is calculated (step S52), and the average luminance value x4 at the center of the screen is calculated (step S53). Next, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the second image and the average luminance value Yskin_ave of the skin color area is calculated (step S54). This comparison value x5 is represented as the following formula (12-1).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (12-1)

Next, an exposure shooting index is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fourth coefficient set in advance according to shooting conditions to obtain a sum. (Step S55), the exposure shooting index calculation processing is completed. The exposure shooting index is defined as the following formula (12-2).
Exposure shooting index = 0.05 × x1 + 1.41 × x2 + (− 0.01) × x3 + (− 0.01) × x4 + 0.01 × x5−5.34 (12-2)
This exposure shooting index includes not only the compositional characteristics of the screen of the second image but also the luminance histogram distribution information, and is particularly effective for distinguishing between a shooting scene in which the main subject is over and an under shooting scene. is there.

  Returning to FIG. When the exposure shooting degree index is calculated, in step S44, which is a fifth index calculation step, the index 1, the index 3, the luminance information BV, and the average luminance value of the skin color area at the center of the screen of the second image are captured. A natural light index is calculated by multiplying a preset weighting factor (fifth factor) according to the condition.

  Further, in step S45, which is the sixth index calculation step, the index 2, index 3, brightness information BV, and the average brightness value of the skin color area in the center of the screen of the second image are preset according to the shooting conditions. By multiplying the weighting coefficient (sixth coefficient), the luminance ratio index is calculated, and this index calculation process ends.

  Hereinafter, the calculation method of the natural light index and the luminance ratio index will be described in detail.

The average luminance value of the skin color area at the center of the screen of the second image is taken as index 7. The center portion of the screen here is, for example, a region composed of the region n2 and the region n3 in FIG. At this time, the natural light index is defined as Expression (13) using the index 1, the index 3, the index 7, and the luminance information BV, and the luminance ratio index is the index 2, the index 3, the index 7, and the luminance information. It is defined as in equation (14) using BV.
Natural light index = 0.45 × index 1 + 0.16 × index 3 + 0.003 × index 7−0.64 × BV + 4.00 (13)
Luminance ratio index = 0.74 × index 2 + 0.08 × index 3 + 0.011 × index 7−0.01 × BV + 1.35 (14)
Here, the coefficients to be multiplied by the respective indexes in the equations (13) and (14) are set in advance according to the photographing conditions.

  Further, instead of calculating the natural light index and the luminance ratio index based on the luminance information BV, the luminance information BV may be used to calculate the index 1, the index 2, and the index 3. In that case, it is not necessary to use the luminance information BV to calculate the natural light index and the luminance ratio index.

36, as a method of calculating the average luminance value (for example, the overall average luminance value), a simple addition average value of individual luminance data obtained from each light receiving unit of the imaging device 1 may be obtained, or imaging may be performed. Similar to the center-weighted average metering that is often used as the metering method of the apparatus 1, the luminance data obtained from the light receiving unit near the center of the screen is highly weighted, and the luminance data obtained from the light receiving unit near the periphery of the screen is weighted. A method of obtaining the average value by lowering may be used. In addition, the luminance data obtained from the vicinity of the light receiving unit corresponding to the focus detection area is increased in weight, and the luminance data obtained from the light receiving unit distant from the focus detection position is reduced in weight to obtain an average value, etc. May be used.
<Scene discrimination 1>
Returning to FIG. 28, scene discrimination will be described.

  When the exposure shooting degree index, the natural light index, and the luminance ratio index are calculated, the shooting scene is determined based on the values of these indexes (step S23). Table 6 shows the details of the shooting scene discrimination based on the values of the exposure shooting index, the natural light index, and the luminance ratio index. The natural light index and the luminance ratio index are calculated based on the luminance information BV at the time of shooting, and the luminance information BV indirectly contributes to the determination of the shooting scene.

表６に示した判別内容を、自然光指標、輝度比指標及び露出撮影度指標の座標系を用いて表したものが、図２４の判別マップである。なお、表６では、図２４に示した低確度領域については省略している。

The discrimination map shown in FIG. 24 represents the discrimination contents shown in Table 6 using the coordinate system of the natural light index, the luminance ratio index, and the exposure shooting index. In Table 6, the low accuracy region shown in FIG. 24 is omitted.

  Next, the gradation processing condition setting process executed in the gradation processing condition setting unit 107 will be described with reference to the flowchart of FIG.

  First, from the one-dimensional LUT of FIGS. 14 to 19, the brightness analysis value of the linear image (n: here, two of skin color average luminance and overall average luminance) and the index calculation processing of FIG. 32 were used. By extracting values corresponding to the natural light index, the luminance ratio index, and the exposure shooting index, n + 4 (six) first temporary correction amounts key_auto [0] to [5] are set (step S61).

  Next, n + 2 (four) first mixing coefficients wgt [0] to [3] are set by extracting a value corresponding to the natural light index from the one-dimensional LUT of FIGS. S62). Next, using the four first temporary correction amounts key_auto [0] to [3] and the four first mixing coefficients wgt [0] to [3], the second temporary correction amount key_auto is obtained from Equation (2). [6] is calculated (step S63).

  Next, from the coefficient calculation table (FIG. 25), three second mixing coefficients wgt [4] to [6] corresponding to the photographic scene determined using the determination map (FIG. 24) are extracted and set. (Step S64). Next, using the two first temporary correction amounts key_auto [4] and key_auto [5], the second temporary correction amount key_auto [6], and the three second mixing coefficients wgt [4] to [6] Then, the brightness correction amount as the tone adjustment parameter is calculated by the equation (3A) (step S65).

  Next, the flare correction condition setting unit 107B calculates the flare correction amount (flareCorr) by the equation (3B) using the natural light index and the luminance ratio index calculated in the index calculation process of FIG. 32 (step S66). .

  Next, gradation processing conditions for the linear image are set from the brightness correction amount calculated in step S65, the photographic scene determined by the scene determination processing, and the flare correction amount calculated in step S66 (step S67). ), The gradation processing condition setting process ends.

  FIG. 38 (a) shows a gradation conversion curve for brightness correction when the shooting scene is in direct light, and FIG. 38 (b) shows brightness correction when the shooting scene is backlight. Are shown for each gradation adjustment parameter. FIG. 39 shows a gradation conversion curve for flare correction. In step S67, the gradation conversion curve corresponding to the gradation adjustment parameter (brightness correction amount) calculated in step S65 and the gradation conversion curve corresponding to the flare correction amount calculated in step S66 are the gradations. Set as processing conditions.

  Each of steps S61 to S65 in FIG. 37 includes a first temporary correction amount setting step, a first mixing coefficient setting step, a second temporary correction amount setting step, a second mixing coefficient setting step, and gradation adjustment in the present invention, respectively. This corresponds to the parameter calculation process.

  As described above, according to the imaging apparatus 1 of the first embodiment, a shooting scene is determined using a linear image, and gradation processing conditions for brightness and hardness adjustment based on the determination result of the shooting scene are set. By defining it to be equivalent to exposure correction at the time of linear image acquisition, without changing the existing automatic exposure control, the operating point of the image sensor, the change of the gradation processing condition from the linear image to the non-linear image, It becomes possible to optimize the brightness of the main subject (for example, the human face area).

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
First, the configuration of the second embodiment (image processing apparatus) will be described. In the following, components having the same functions as those in the first embodiment are denoted by the same reference numerals.

(External configuration of image processing apparatus 2)
FIG. 40 shows an example of the external configuration of the image processing apparatus 2 according to the second embodiment. As shown in FIG. 40, the image processing apparatus 2 includes a loading unit 13 that loads a photosensitive material on one side surface of the housing 11. The housing 11 includes an exposure processing unit 86 that exposes the photosensitive material and a print creation unit 16 that develops the exposed photosensitive material to create a print. On the other side surface of the housing 11, a tray 17 for discharging the print created by the print creation unit 16 is provided.

  At the top of the housing 11, a CRT (Cathode Ray Tube) 85 as a display device, a film scanner unit 81 that is a device for reading a transparent document, a reflection document input device 82, and an operation unit 71 are provided. The housing 11 further includes an image reading unit 14 that can read image information recorded on various digital recording media, and an image writing unit 15 that can write image signals on various digital recording media. Includes a control unit 72 for controlling these units. The image reading unit 14 includes an image transfer unit 83 described later, and the image writing unit 15 includes an image transfer unit 88 described later.

  Not all of these components are necessary, and it is not necessary to have a structure provided integrally. Any one or more may be separated from the image processing apparatus 2 as a separate body, or connected as an external apparatus.

(Configuration of the image processing apparatus 2)
FIG. 41 shows a schematic configuration of the image processing apparatus 2 according to the second embodiment. As shown in FIG. 41, the image processing apparatus 2 includes an operation unit 71, a control unit 72, an image processing unit 10, a film scan data processing unit 73, a reflection original scan data processing unit 74, an image data format decoding processing unit 75, a CRT. A unique processing unit 76, a printer specific processing unit A77, a printer specific processing unit B78, and an image data format creation processing unit 79 are included.

  The film scan data processing unit 73 performs a calibration operation specific to the film scanner unit 81, negative / positive reversal (in the case of a negative document), dust scratch removal, contrast adjustment, granular noise removal, and sharpness for the image data input from the film scanner unit 81. Processing such as conversion enhancement is performed, and processed image data is output to the image processing unit 10. Further, the film size, the negative / positive type, the information relating to the main subject optically or magnetically recorded on the film, the information relating to the photographing conditions (for example, the information content described in APS), and the like are also output to the image processing unit 10.

  The reflection document scan data processing unit 74 performs a calibration operation unique to the reflection document input device 82, negative / positive reversal (in the case of a negative document), dust flaw removal, contrast adjustment, and noise removal for the image data input from the reflection document input device 82. Then, processing such as sharpening enhancement is performed, and processed image data is output to the image processing unit 10.

  The image data format decoding processing unit 75 restores the compression code as necessary according to the data format of the image data input from the image transfer means 83 and / or communication means (input) 84, and color data. Are converted into a data format suitable for computation in the image processing unit 10 and output to the image processing unit 10.

  In addition, when an output image condition is specified from any of the operation unit 71, the communication unit (input) 84, and the image transfer unit 83, the image data format decoding processing unit 75 detects the specified information, and Output to the processing unit 10. Information about the conditions of the output image specified by the image transfer unit 83 is embedded in the header information and tag information of the image data acquired by the image transfer unit 83.

  Based on a command from the operation unit 71 or the control unit 72, the image processing unit 10 applies image data received from any one of the film scanner unit 81, the reflection original input device 82, the image transfer unit 83, and the communication unit (input) 84. On the other hand, image processing described later is performed to generate digital image data for image formation optimized for viewing on the output medium, and a CRT specific processing unit 76, a printer specific processing unit A77, a printer specific processing unit B78, The data is output to any one of the image data format creation processing unit 79.

  The internal configuration of the image processing unit 10 in the image processing apparatus 2 will be described in detail later with reference to FIG.

  The CRT specific processing unit 76 performs processing such as changing the number of pixels and color matching on the image data input from the image processing unit 10 as necessary, and combines it with information that needs to be displayed, such as control information. Are output to the CRT 85.

  The printer-specific processing unit A77 performs printer-specific calibration processing, color matching, and pixel number change processing as necessary, and outputs processed image data to the exposure processing unit 86.

  When an external printer 87 can be connected to the image processing apparatus 2 of the present invention, a printer specific processing unit B78 is provided for each printer apparatus to be connected. The printer-specific processing unit B78 performs printer-specific calibration processing, color matching, pixel number change, and the like, and outputs processed image data to the external printer 87.

  The image data format creation processing unit 79 converts the image data input from the image processing unit 10 into various general-purpose image formats typified by JPEG, TIFF, bitmap, and the like as necessary. The completed image data is output to the image transport unit 88 and the communication means (output) 89.

  41, the film scan data processing unit 73, the reflection original scan data processing unit 74, the image data format decoding processing unit 75, the image processing unit 10, the CRT specific processing unit 76, the printer specific processing unit A77, and the printer specific processing. The section B78 and the image data format creation processing section 79 are sections provided to assist understanding of the function of the image processing apparatus 2, and are not necessarily realized as physically independent devices. It may be realized as a type of software processing by a single CPU.

(Internal configuration of the image processing unit 10)
FIG. 11 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls the gradation correction operation based on scene discrimination for the input image of the image processing apparatus 2, and performs a series of processes such as image acquisition, scene discrimination, and gradation processing condition setting. Execute. In this embodiment, the input image is taken image data, and the internal configuration of the image processing unit 10 will be described with reference to FIG.

  As shown in FIG. 11, the image processing unit 10 includes a first image acquisition unit 101, a second image acquisition unit 102, an occupation rate calculation unit 103, an index calculation unit 104, a scene determination unit 105, and a gradation processing condition setting unit 107. The gradation conversion processing unit 108 is configured.

  The first image acquisition unit 101 acquires input image data from each input data processing unit. Since the input image is photographed image data, for example, the photographed image data input to the image data format decoding processor 75 via the image transfer means 83 or the communication means (input) 84 is necessary according to the data format of the image data. Accordingly, processing such as decompression of the compression code and conversion of the data expression method is performed, and the image processing unit 10 acquires the first image. The photographed image data is converted into a non-linear image such as a JPEG image format and recorded during normal photographing. Here, it is assumed that a linear image (for example, Raw data) before conversion into a non-linear image is acquired as the first image.

  In addition, the first image acquisition unit 101 reads, for example, additional information recorded in the tag area of the linear image at the time of shooting, and also acquires luminance information calculated at the time of shooting. That is, the first image acquisition unit 101 also functions as a luminance information acquisition unit. The luminance information BV acquired in the present embodiment is a luminance value, which is calculated from a photometric value at the time of shooting and is additionally recorded in the tag area of the image data as metadata. The acquired image data and luminance information BV of the first image are held in a memory (not shown) in the image processing unit 10.

Similar to the case of the first embodiment, the second image acquisition unit 102 acquires a substantially reduced second image from the first image.
Similar to the first embodiment, the first image acquisition unit 101 and the second image acquisition unit 102 function as an image acquisition unit.

  Hereinafter, description of components having the same functions as those of the imaging device 1 of the first embodiment will be simplified.

  The occupancy rate calculation unit 103 functions as a first occupancy rate calculation unit, and classifies an image into a predetermined class composed of a combination of brightness and hue based on the color information acquired for the image data of the second image. Then, for each classified class, a first occupation ratio indicating a ratio of pixels belonging to each class to the entire image is calculated.

  The occupancy rate calculation unit 103 also functions as a second occupancy rate calculation unit, and similarly classifies images into a predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness, and a second class is determined for each class. Calculate the occupation rate.

  In the occupancy rate calculation process executed in the occupancy rate calculation unit 103, differences from the first embodiment will be described later with reference to FIG.

  The index calculation unit 104 calculates the index 1 and the index 2 from the first occupation rate calculated by the occupation rate calculation unit 103. That is, the index calculation unit 104 functions as a first index calculation unit and a second index calculation unit.

  In addition, the index calculation unit 104 calculates the index 3 from the second occupancy rate calculated by the occupancy rate calculation unit 103. That is, the index calculation unit 104 also functions as a third index calculation unit.

  Further, the index calculation unit 104 calculates an exposure shooting degree index from the average luminance value in the center of the screen of the second image and the difference value between the maximum luminance value and the average luminance value. That is, the index calculation unit 104 also functions as a fourth index calculation unit.

  In addition, the index calculation unit 104 generates new natural light from the average luminance value (index 7) of the skin color area in the center of the screen of the second image, the index 1 and the index 3, and the luminance information BV at the time of shooting. Calculate the indicator. That is, the index calculation unit 104 also functions as a fifth index calculation unit.

  In addition, the index calculation unit 104 calculates a new brightness ratio index from the average brightness value, the indices 2 and 3, and the above-described brightness information BV at the time of shooting. That is, the index calculation unit 104 also functions as a sixth index calculation unit.

  The difference between the index calculation process executed by the index calculation unit 104 and the case of the first embodiment will be described in detail later with reference to FIG.

  The scene determination unit 105 determines the shooting scene of the first image based on each index calculated by the index calculation unit 104. That is, the scene determination unit 105 functions as a scene determination unit. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes over and under degrees.

  As described above, the occupation rate calculation unit 103, the index calculation unit 104, and the scene determination unit 105 function as a scene determination processing unit, as in the first embodiment.

  The gradation processing condition setting unit 107 sets gradation processing conditions for the first image based on the photographic scene determined by the scene determination unit 105. The gradation processing condition setting method is the same as in the first embodiment. That is, the gradation processing condition setting unit 107 functions as a gradation processing condition setting unit.

  The gradation conversion processing unit 108 executes gradation conversion processing on the first image in accordance with the gradation processing conditions set by the gradation processing condition setting unit 107. In addition, the gradation conversion processing unit 108 performs processing for converting the first image (linear image) subjected to the gradation conversion processing into a non-linear image, like the gamma correction unit 55 of the imaging device 1. Therefore, the gradation conversion processing unit 108 functions as first, second, and third gradation conversion processing means.

(Operation flow of the image processing apparatus 2)
Next, the operation of the image processing apparatus 2 in the present embodiment will be described.

  Hereinafter, a captured image is targeted, and the captured object is referred to as a “main subject”.

  First, the overall flow of processing executed in the image processing apparatus 2 will be described with reference to the flowchart of FIG.

  First, when an input operation to the image processing apparatus 2 is performed, input processing is performed in any of the input processing units 73 to 75 (step S70). The captured image is sent to the image processing unit 10 through the processing in the image data format decoding processing unit 75. Next, in the image processing unit 10, a first image (linear image) is acquired as a captured image (step S71).

  On the other hand, in step S72 which is a luminance information acquisition step, the Exif information added to the acquired first image is read and the luminance information BV is captured. The acquired luminance information BV is calculated from the photometric value at the time of photographing, and is recorded together when the photographed image is recorded in the Exif format. The acquired luminance information BV is held in the memory.

  Thereafter, the image data of the first image is acquired as a divided image composed of a plurality of divided regions, that is, a second image (reduced linear image) (step S73). The second image is an image obtained by reducing the size of the first image, as in the case of the first embodiment. Steps S71 and S73 correspond to the image acquisition process of the present invention.

  Next, in step S74, which is a scene determination processing step, a scene determination process for determining a shooting scene is performed based on the acquired image data of the second image. Regarding the scene determination processing in step S74, differences from the case of the first embodiment will be described later with reference to FIG.

  Next, in step S75, which is a gradation processing condition setting step, as in the case of the first embodiment, based on each index obtained in the scene determination process in step S74 and the shooting scene determination result, the first A gradation processing condition setting process for setting conditions necessary for the gradation conversion process of the image is performed.

  In step S76, which is the first gradation conversion processing step, the image data of the first image that is the photographed image is processed based on the gradation processing condition (brightness correction amount) set based on the second image. A gradation conversion process is performed. The gradation conversion process in step S76 is to correct the influence on the gradation due to where the photographic gradation is reproduced centered on the scene brightness or the light source condition of the shooting scene, and the scene discrimination in step S74. The result is based on the gradation processing condition (brightness correction amount) set in step S75.

  In step S77, which is the second gradation conversion process step, the first gradation conversion process is performed in step S76 based on the gradation processing condition (flare correction amount) set based on the second image. The flare correction process is performed on the image data of the first image. The flare correction process in step S77 corrects a potential offset area in the shadow portion of the backlight scene to prevent a flare image. The gradation processing condition (set in step S75 based on the scene discrimination result in step S74) ( (Flare correction amount).

  In step S78, which is the third gradation conversion processing step, the linear image is converted into a non-linear image by performing gamma conversion processing on the first image (linear image) after the AWB processing and gradation conversion processing. The

  In the present embodiment, gradation processing conditions are set in step S75, and gradation conversion processing is performed in step S76. However, other image characteristics such as color adjustment other than the above gradation processing based on the shooting scene are used. The optimization process may be performed.

  Next, other image processing (noise processing, sharpness processing, contrast correction processing, dodging processing, etc.) is appropriately performed on the image data after the gamma conversion processing (step S79). Thereafter, the image data is processed and output by the output processing units 76 to 79. Alternatively, it is recorded on a storage medium for storage (such as an SD memory card or a multimedia card (MMC)) (step S80). When the output or recording ends, the operation in the image processing apparatus 2 ends.

  Note that the linear image after gradation conversion processing may be stored as Raw data and converted to a non-linear image using development software.

(Flow of scene discrimination process 2)
Next, the scene determination process 2 (step S74 in FIG. 42) in the image processing apparatus 2 will be described with reference to the flowchart in FIG. Below, a different point from 1st Embodiment (imaging device 1) is demonstrated, and the part of the same content is abbreviate | omitted suitably.

  As in the first embodiment, the scene discrimination process includes a color space conversion process (step S20), an occupation rate calculation process (step S21), an index calculation process (step S22), and a scene discrimination (step) as shown in FIG. S23). Each process shown in FIG. 28 will be described below with reference to the above drawings as appropriate.

  First, in step S20 of FIG. 28, color space conversion processing is performed. Since the processing contents are the same as in the case of the first embodiment, a description thereof is omitted here.

<Occupancy rate calculation process 2>
Next, the occupation rate calculation process in step S21 will be described with reference to the flowchart in FIG.

  First, based on the HSV value calculated by the color space conversion process, each pixel of the second image is classified into a predetermined class composed of a combination of hue and brightness, and the cumulative number of pixels is calculated for each classified class. As a result, a two-dimensional histogram is created (step S30).

  The lightness division is different from that in the first embodiment. This is due to the difference between a linear image and a non-linear image. In step S30, the brightness (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200- It is divided into seven classes of 224 (v6) and 225 to 255 (v7).

  Hue (H) is a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region having a hue value of 161 to 250. It is divided into four areas (H4) and a red hue area (H5). These are the same as in the case of the first embodiment. Similarly, the skin color hue region is further divided into a skin color region (H1) and another region (H2).

  The two-dimensional histogram creation in step S31 is the same as in the first embodiment.

  FIG. 43 shows four regions n1 to n4 divided in step S31 according to the distance from the outer edge of the screen of the second image. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. The region n3 is a region further inside the region n2, and the region n4 is a central region of the second image. In step S31, the brightness is divided into seven areas v1 to v7 as described above. Therefore, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 4 × 7 = 28.

  Step S32 corresponds to the first occupancy rate calculation step together with step S30, and the first occupancy rate is calculated as in the case of the first embodiment.

  If the first occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy in each class is expressed as shown in Table 7.

  Step S33 corresponds to the second occupancy ratio calculation step together with step S31, and the second occupancy ratio is calculated as in the case of the first embodiment.

  If the second occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each class is expressed as shown in Table 8.

<Indicator calculation process 2>
Next, the index calculation process (step S22 in FIG. 28) will be described with reference to the flowchart in FIG.

  First, in step S40, which is the first index calculation step, index 1 is calculated by multiplying the first occupancy by a preset first coefficient to obtain the sum, and then the second index is calculated. In step S41, which is a calculation step, the index 2 is calculated by multiplying the first occupancy by a preset second coefficient to obtain the sum, as in the case of the first embodiment. It is the same.

  The index 1 is an index representing the degree of overshoot of the main subject, and is for separating only the image that should be determined as “the main subject is over” from other shooting scenes. The index 2 is an index that compositely represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness, and only images that should be determined as “backlight” and “main subject is under” are taken from other shooting scenes. It is for separation.

  Hereinafter, a method for calculating the index 1 and the index 2 will be described.

  Table 9 shows the first coefficient necessary for calculating the index 1 by class. The coefficient of each class shown in Table 9 is a weighting coefficient by which the first occupancy rate Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 9, a positive (+) coefficient is used for the first occupancy calculated from the region distributed in the skin color hue region (H1) of high brightness (v5, v6), A negative (−) coefficient is used for the first occupation ratio calculated from a certain blue hue region. In FIG. 44, the first coefficient in the skin color region (H1) and the first coefficient in the other region (green hue region (H3)) are represented as a curve (coefficient curve) that continuously changes over the entire brightness. It is shown. According to Table 9 and FIG. 44, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of both are different.

When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (6).

Accordingly, the sum of the H1 to H4 regions is represented by the following formulas (6-5) to (6-8).
H1 region sum = R11 × (−44) + R21 × (−16) + (omitted)… + R71 × (−11.3) (6-5)
Sum of H2 regions = R12 × 0 + R22 × 8.6 + (omitted)… + R72 × (−11.1) (6-6)
Sum of H3 regions = R13 × 0 + R23 × (−6.3) + (omitted)… + R73 × (−10) (6-7)
Sum of H4 region = R14 × 0 + R24 × (−1.8) + (omitted)… + R74 × (−14.6) (6-8)

The index 1 is defined as in Expression (21) using the sum of the H1 to H4 regions shown in Expressions (6-5) to (6-8).
Indicator 1
= Sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 4.424 (21)

  Table 10 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 10 is a weighting coefficient that is multiplied by the first occupancy rate Rij of each class shown in Table 1, and is set in advance according to the shooting conditions.

  According to Table 10, a negative (−) coefficient is used for the occupation ratio calculated from the region (v4) distributed in the intermediate lightness of the flesh color hue region (H1), and the low lightness of the flesh color hue region (H1) ( A positive (+) coefficient is used for the occupation ratio calculated from the shadow area (v2, v3). FIG. 45 shows the second coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 10 and FIG. 45, the sign of the second coefficient of the low brightness area with the brightness value of 26 to 84 in the flesh color hue area is positive (+), the minimum brightness area with the brightness value of 0 to 25, and the brightness. It can be seen that the sign of the second coefficient in the middle and high brightness areas of values 85 to 255 is negative (−), and there is a large difference in the coefficients in both areas.

When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (8).

Accordingly, the sum of the H1 to H4 regions is represented by the following formulas (8-5) to (8-8).
Sum of H1 region = R11 × (−27) + R21 × 4.5 + (omitted)… + R71 × (−24) (8-5)
Sum of H2 regions = R12 × 0 + R22 × 4.7 + (omitted)… + R72 × (−8.5) (8-6)
Sum of H3 regions = R13 × 0 + R23 × 0 + (omitted) ... + R73 × 0 (8-7)
Sum of H4 region = R14 × 0 + R24 × (−5.1) + (omitted)… + R74 × 7.2 (8-8)

The index 2 is defined as in Expression (22) using the sum of the H1 to H4 regions shown in Expressions (8-5) to (8-8).
Indicator 2
= Sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.554 (22)

  Since the index 1 and the index 2 are calculated based on the brightness and hue distribution amount of the second image, they are effective in determining a shooting scene when the image is a color image.

  When the index 1 and the index 2 are calculated, the index 3 is obtained by multiplying the second occupancy by a preset third coefficient in step S42, which is a third index calculating step, to obtain a sum. Is calculated. The index 3 indicates a difference in contrast between the center and the outside of the screen of the image data between the backlight with the main subject under and the image with the main subject over.

  Hereinafter, a method for calculating the index 3 will be described.

  Table 11 shows the third coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 11 is a weighting coefficient by which the second occupancy Qij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 46 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (10).

Accordingly, the sum of the n1 to n4 regions is represented by the following formulas (10-5) to (10-8).
Sum of n1 region = Q11 × 40.1 + Q21 × 37 + (omitted)… + Q71 × 22 (10−5)
Sum of n2 regions = Q12 × (−14.8) + Q22 × (−10.5) + (omitted)… + Q72 × 0 (10−6)
Sum of n3 regions = Q13 × 24.6 + Q23 × 12.1 + (omitted)… + Q73 × 10.1 (10−7)
Sum of n4 regions = Q14 × 1.5 + Q24 × (−32.9) + (omitted)… + Q74 × (−52.2) (10−8)

The index 3 is defined as in Expression (23) using the sum of the n1 to n4 regions shown in Expressions (10-5) to (10-8).
Indicator 3
= Sum of n1 region + sum of n2 region + sum of n3 region + sum of n4 region + 12.6201 (23)

  Since the index 3 is calculated based on the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the second image, the photographing scene of the monochrome image as well as the color image is discriminated. It is also effective.

  When the indices 1 to 3 are calculated, the difference value between the average luminance value of the skin color at the center of the screen of the second image and the maximum luminance value and the average luminance value is obtained in step S43, which is the fourth index calculating step. The exposure shooting index is calculated by multiplying each by a preset coefficient.

  Hereinafter, with reference to the flowchart of FIG. 36, the processing for calculating the exposure shooting degree index will be described.

  First, from step S50 to step S54, the average luminance value x1 of the skin color region in the center of the screen of the second image, the difference value x2 between the maximum luminance value and the average luminance value of the second image = maximum luminance value-average The luminance value, the standard deviation x3 of the luminance of the second image, the average luminance value x4 in the center of the screen, the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the second image, and the average of the skin color area The process of calculating the comparison value x5 with the luminance value Yskin_ave is the same as in the case of the first embodiment, and thus the description thereof is omitted here.

  Next, an exposure shooting index is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fourth coefficient set in advance according to the shooting conditions and taking the sum. (Step S55), the exposure shooting index calculation processing is completed. The exposure shooting index is defined as the following equation (12-3).

Exposure shooting index = 0.06 × x1 + 1.13 × x2 + 0.02 × x3 + (− 0.01) × x4 + 0.03 × x5-6.50 (12-3)
This exposure shooting index includes not only the compositional characteristics of the screen of the second image but also the luminance histogram distribution information, and is particularly effective for distinguishing between a shooting scene in which the main subject is over and an under shooting scene. is there.

  When the exposure shooting degree index is calculated, in step S44 which is a fifth index calculation step, the index 1, the index 3, the luminance information BV, and the average luminance value of the skin color region in the center of the screen of the second image, A natural light index is calculated by multiplying a weighting coefficient (fifth coefficient) set in advance according to the photographing condition.

  Further, in step S45, which is the sixth index calculation step, the index 2, index 3, brightness information BV, and the average brightness value of the skin color area in the center of the screen of the second image are preset according to the shooting conditions. By multiplying the weighting coefficient (sixth coefficient), the luminance ratio index is calculated, and this index calculation process ends.

  Hereinafter, a method for calculating the natural light index and the luminance ratio index will be described.

The average luminance value of the skin color area at the center of the screen of the second image is taken as index 7. Here, the center of the screen is, for example, an area composed of the area n2, the area n3, and the area n4 in FIG. At this time, the natural light index is defined as Expression (24) using the index 1, the index 3, the index 7, and the luminance information BV, and the luminance ratio index is the index 2, the index 3, the index 7, and the luminance information. It is defined as in equation (25) using BV.
Natural light index = 0.18 × index 1 + 0.19 × index 3 + 0.004 × index 7−0.68 × BV + 4.39 (24)
Luminance ratio index = 0.24 × index2 + 0.16 × index 3 + 0.034 × index 7−0.05 × BV + 4.61 (25)
Here, the coefficients to be multiplied by the respective indices in the expressions (24) and (25) are set in advance according to the photographing conditions.

Further, instead of calculating the natural light index and the luminance ratio index based on the luminance information BV, the luminance information BV may be used to calculate the index 1, the index 2, and the index 3. In that case, it is not necessary to use the luminance information BV to calculate the natural light index and the luminance ratio index.
<Scene discrimination 2>
Returning to step S23 (scene discrimination) in FIG.

  When the exposure shooting degree index, the natural light index, and the luminance ratio index are calculated, the shooting scene is determined based on the values of these indexes, but the scene determination method is the same as in the first embodiment. The description here is omitted.

(Flow of gradation processing condition setting)
The gradation processing condition setting process for the first image (step S75 in FIG. 42) is the same as that in the first embodiment and is as described with reference to the flowchart in FIG. Is omitted.

  As described above, according to the image processing apparatus 2 of the second embodiment, it is possible to optimize the brightness of the main subject (for example, a human face area) as in the first embodiment. Become.

  In addition, the description content in each said embodiment can be suitably changed in the range which does not deviate from the meaning of this invention.

It is a figure showing an example of appearance composition of an imaging device concerning an embodiment of the present invention. It is a block diagram which shows the internal structure of an imaging device. It is a block diagram which shows the internal structure of an imaging | photography process part. It is a flowchart which shows exposure amount control in an AE control part. It is a figure which shows a mode that an electric charge accumulate | stores in an image pick-up element in control of shutter speed. It is a flowchart which shows the control which adjusts the focus of the image in AF control part. It is a figure which shows the change of the contrast information accompanying the movement of an imaging optical system, and the mode of a focal point position detection. It is a flowchart which shows the process of the image data which performs the interpolation between pixels for every RGB color component in a pixel interpolation part. It is a flowchart which shows the process which automatically adjusts the white balance in a picked-up image in an AWB control part. It is a figure which shows an example of a gamma correction curve. It is a block diagram which shows the internal structure of an image process part. FIG. 4A is a diagram illustrating an example of a captured first image, and FIG. 4B is a diagram illustrating an example of a second image obtained by dividing the first image into a reduced image of M × N pixels. It is a block diagram which shows the internal structure of a gradation process condition setting part. It is an LUT showing the relationship between the luminance ratio index and the provisional correction amount key_auto [0]. It is a LUT showing the relationship between the exposure shooting index and the temporary correction amount key_auto [1]. It is an LUT showing the relationship between the flesh color average luminance and the provisional correction amount key_auto [2]. It is an LUT indicating the relationship between the skin color average luminance or the overall average luminance and the temporary correction amount key_auto [3]. It is a LUT showing the relationship between the luminance ratio index and the provisional correction amount key_auto [4]. It is an LUT showing the relationship between the exposure shooting index and the temporary correction amount key_auto [5]. It is a LUT showing the relationship between the natural light index and the mixing coefficient wgt [0]. It is a LUT showing the relationship between the natural light index and the mixing coefficient wgt [1]. It is a LUT showing the relationship between the natural light index and the mixing coefficient wgt [2]. It is a LUT showing the relationship between the natural light index and the mixing coefficient wgt [3]. It is a figure which shows the discrimination | determination map for discriminating a photography scene from a natural light parameter | index, a luminance ratio parameter | index, and an exposure photography degree parameter | index. It is a figure which shows the coefficient calculation table showing the value of mixing coefficient wgt [4]-[6] for every imaging | photography scene. (A) is a figure which shows an example of the transition coefficient w and the weighting coefficient fw corresponding to a scene discrimination | determination map. (B) is a figure which shows another example of the weighting coefficient fw. 3 is a flowchart showing an overall flow of processing executed in the imaging apparatus 1. It is a flowchart which shows the scene discrimination | determination process performed in an image process part. It is a flowchart which shows an occupation rate calculation process. It is a figure which shows the class which consists of brightness and hue. It is the figure (a) which shows the area | regions n1-n3 determined according to the distance from the outer edge of a screen, and the figure (b) which shows the class which consists of the area | regions n1-n3 and brightness. It is a flowchart which shows an index calculation process. It is a figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied for every area | region (n1-n3). It is a flowchart which shows exposure photography degree parameter | index calculation processing. It is a flowchart which shows the gradation process condition setting process performed in a gradation process condition setting part. It is a figure which shows the gradation conversion curve (a) of the brightness correction | amendment when a picked-up scene is a follow light, and the gradation conversion curve (b) of the brightness correction | amendment when a picked-up scene is a backlight. It is a figure which shows the gradation conversion curve of flare correction. It is a figure which shows the example of an external appearance structure of the image processing apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the internal structure of an image processing apparatus. It is a flowchart which shows the flow of the whole process performed in an image processing apparatus. It is a figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of a screen. It is a figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied for every area | region (n1-n4).

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Image processing apparatus 10 Image processing part 11 Housing | casing 13 Loading part 14 Image reading part 15 Image writing part 16 Print preparation part 17 Tray 20 Shooting process part 21 Case 22 Cross key 23 Shooting optical system 24 Flash 25 Finder 26 Power switch 27 Display unit 28 Release button 31 Processor 32 Memory 33 Image sensor 37 Image data output unit 38 Operation unit 41 Timing generator 42 Shutter unit 43 Aperture unit 44 Focus unit 51 AE control unit 52 AF control unit 53 Pixel interpolation unit 54 AWB control 55 Gamma correction unit 71 Operation unit 72 Control unit 73 Film scan data processing unit 74 Reflected original scan data processing unit 75 Image data format decoding processing unit 76 CRT specific processing unit 77 Printer specific processing unit A
78 Printer Specific Processing Unit B
79 Image data format creation processing section 81 Film scanner section 82 Reflected document input device 83 Image transfer means 84 Communication means (input)
85 CRT
86 Exposure processing unit 87 External printer 88 Image transport unit 89 Communication means (output)
101 First image acquisition unit 102 Second image acquisition unit 103 Occupancy rate calculation unit 104 Index calculation unit 105 Scene determination unit 107 Tone processing condition setting unit 107A Brightness correction condition setting unit 108 Tone conversion processing unit 201 First temporary correction Amount setting unit 202 first mixing coefficient setting unit 203 second temporary correction amount calculating unit 204 second mixing coefficient setting unit 205 brightness correction amount calculating unit 107B flare correction condition setting unit 206 transition coefficient setting unit 207 weight coefficient setting unit 208 flare Correction amount calculator

Claims (24)

Photographing means for photographing the subject;
Image obtaining means for obtaining a linear image formed by photographing by the photographing means;
Scene discrimination processing means for performing a discrimination process of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means;
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means;
Third gradation conversion processing means for performing third gradation conversion processing for converting a linear image to which the second gradation conversion processing is applied into a non-linear image;
An imaging apparatus comprising: The image acquisition means includes
From the linear image, obtain a linear image reduced in image size,
The scene discrimination processing means includes
The imaging apparatus according to claim 1, wherein a shooting scene discrimination process is performed using a linear image with a reduced image size. Luminance information acquisition means for acquiring luminance information calculated at the time of photographing the linear image,
The scene discrimination processing means includes
Based on the luminance information acquired by the luminance information acquisition means, to determine the shooting scene,
The gradation processing condition setting means includes:
The imaging apparatus according to claim 1, wherein the first and second gradation processing conditions are set based on the luminance information acquired by the luminance information acquisition unit. The scene discrimination processing means includes
Calculating a natural light index representing the degree of outdoor photographing with natural light of the linear image, a luminance ratio index representing the size of the dynamic range, and an exposure photographing degree index resulting from the exposure setting at the time of photographing;
The imaging apparatus according to claim 1, wherein a shooting scene is determined based on each of the calculated indexes. The scene discrimination processing means includes
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating the occupancy rate of
A first index calculating means for calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the photographing condition;
A second index calculating means for calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions;
A third index calculating means for calculating the index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating means for calculating the exposure photographing degree by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to photographing conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. A fifth index calculating means for
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Sixth index calculating means for calculating;
Scene discriminating means for discriminating a shooting scene of the image data based on the exposure shooting degree, the natural light index and the luminance ratio index;
The imaging apparatus according to claim 4, further comprising: The gradation processing condition setting means includes:
A first temporary correction amount for setting a plurality of first temporary correction amounts for the linear image based on a brightness analysis value of the linear image, a brightness reproduction target value, the luminance ratio index, and the exposure photographing degree. Setting means;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
Second temporary correction amount setting means for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. Second mixing coefficient setting means for
Gradation adjustment parameter calculation means for calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; With
The first gradation processing condition for the linear image is set based on a determination result by the scene determination processing unit and a gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit. Item 6. The imaging device according to Item 4 or 5. Image acquisition means for acquiring a linear image formed by photographing;
Scene discrimination processing means for performing a discrimination process of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means;
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means;
Third gradation conversion processing means for performing gradation conversion processing for converting the linear image to which the second gradation conversion processing is applied into a non-linear image;
An image processing apparatus comprising: The image acquisition means includes
From the linear image, obtain a linear image reduced in image size,
The scene discrimination processing means includes
The image processing apparatus according to claim 7, wherein a shooting scene determination process is performed using a linear image with the image size reduced. Luminance information acquisition means for acquiring luminance information calculated at the time of photographing the linear image,
The scene discrimination processing means includes
Based on the luminance information acquired by the luminance information acquisition means, to determine the shooting scene,
The gradation processing condition setting means includes:
9. The image processing apparatus according to claim 7, wherein the first and second gradation processing conditions are set based on the luminance information acquired by the luminance information acquisition unit. The scene discrimination processing means includes
Calculating a natural light index representing the degree of outdoor photographing with natural light of the linear image, a luminance ratio index representing the size of the dynamic range, and an exposure photographing degree index resulting from the exposure setting at the time of photographing;
The image processing apparatus according to claim 7, wherein a shooting scene is determined based on each of the calculated indexes. The scene discrimination processing means includes
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating the occupancy rate of
A first index calculating means for calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the photographing condition;
A second index calculating means for calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions;
A third index calculating means for calculating the index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating means for calculating the exposure photographing degree by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to photographing conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. A fifth index calculating means for
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Sixth index calculating means for calculating;
Scene discriminating means for discriminating a shooting scene of the image data based on the exposure shooting degree, the natural light index and the luminance ratio index;
The image processing apparatus according to claim 10, further comprising: The gradation processing condition setting means includes:
A first temporary correction amount for setting a plurality of first temporary correction amounts for the linear image based on a brightness analysis value of the linear image, a brightness reproduction target value, the luminance ratio index, and the exposure photographing degree. Setting means;
First mixing coefficient setting means for setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
Second temporary correction amount setting means for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting means and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. Second mixing coefficient setting means for
Gradation adjustment parameter calculation means for calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; With
The first gradation processing condition for the linear image is set based on a determination result by the scene determination processing unit and a gradation adjustment parameter calculated by the gradation adjustment parameter calculation unit. Item 12. The image processing apparatus according to Item 10 or 11. An image acquisition step of acquiring a linear image formed by photographing;
A scene determination processing step for performing a determination process of a photographic scene based on the linear image;
Based on the determination result of the scene determination processing step, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. A gradation processing condition setting step;
A first gradation conversion processing step for performing a first gradation conversion process on the linear image based on the first gradation processing condition set by the gradation processing condition setting step;
Based on the second gradation processing condition set in the gradation processing condition setting step, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion process,
A second gradation conversion process step of performing a third gradation conversion process for converting the linear image to which the second gradation conversion process is applied into a non-linear image;
An image processing method comprising: In the image acquisition step,
From the linear image, obtain a linear image reduced in image size,
In the scene discrimination processing step,
14. The image processing method according to claim 13, wherein a scene determination process is performed using a linear image with a reduced image size. Including a luminance information acquisition step of acquiring luminance information calculated at the time of capturing the linear image;
In the scene discrimination processing step,
Based on the luminance information acquired by the luminance information acquisition step, to determine the shooting scene,
In the gradation processing condition setting step,
15. The image processing method according to claim 13, wherein the first and second gradation processing conditions are set based on the luminance information acquired by the luminance information acquisition step. In the scene discrimination processing step,
Calculating a natural light index representing the degree of outdoor photographing with natural light of the linear image, a luminance ratio index representing the size of the dynamic range, and an exposure photographing degree index resulting from the exposure setting at the time of photographing;
The image processing method according to any one of claims 13 to 15, wherein a shooting scene is determined based on each of the calculated indexes. The scene discrimination processing step includes
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, A first occupancy ratio calculating step of calculating a first occupancy ratio indicating a ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class A second occupancy rate calculating step for calculating the occupancy rate of
A first index calculation step of calculating index 1 by multiplying the first occupancy by a first coefficient set in advance according to the imaging condition;
A second index calculating step of calculating index 2 by multiplying the first occupancy by a second coefficient set in advance according to shooting conditions;
A third index calculating step of calculating index 3 by multiplying the second occupancy by a third coefficient set in advance according to the shooting conditions;
A fourth index calculating step of calculating the exposure shooting degree by multiplying the average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to shooting conditions;
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. And a fifth index calculating step
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. A sixth index calculating step for calculating;
A scene determination step of determining a shooting scene of the image data based on the exposure shooting degree, the natural light index, and the luminance ratio index;
The image processing method according to claim 16, further comprising: The gradation processing condition setting step includes
A first temporary correction amount for setting a plurality of first temporary correction amounts for the linear image based on a brightness analysis value of the linear image, a brightness reproduction target value, the luminance ratio index, and the exposure photographing degree. A setting process;
A first mixing coefficient setting step of setting a first mixing coefficient as a weighting coefficient for multiplying each of the plurality of first temporary correction amounts based on the natural light index;
A second temporary correction amount setting step for setting a second temporary correction amount for the linear image based on the first mixing coefficient set by the first mixing coefficient setting step and the plurality of first temporary correction amounts;
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. A second mixing coefficient setting step,
A gradation adjustment parameter calculating step of calculating a gradation adjustment parameter based on the second mixing coefficient set by the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount; Including,
The first gradation processing condition for the linear image is set based on a determination result obtained by the scene determination processing step and a gradation adjustment parameter calculated by the gradation adjustment parameter calculation step. Item 18. The image processing method according to Item 16 or 17. A computer that controls image processing
Image acquisition means for acquiring a linear image formed by photographing;
Scene discrimination processing means for performing discrimination processing of a photographic scene based on the linear image;
Based on the determination result by the scene determination processing means, a first gradation processing condition for adjusting the brightness of the linear image and a second gradation processing condition for adjusting the hardness are set. Gradation processing condition setting means,
First gradation conversion processing means for performing a first gradation conversion process on the linear image based on a first gradation processing condition set by the gradation processing condition setting means;
Based on the second gradation processing condition set by the gradation processing condition setting means, a second floor that performs a second gradation conversion process on the linear image to which the first gradation conversion process is applied. Key conversion processing means,
Third gradation conversion processing means for performing gradation conversion processing for converting a linear image to which the second gradation conversion processing is applied into a non-linear image;
Image processing program to function as The image acquisition means includes
From the linear image, obtain a linear image reduced in image size,
The scene discrimination processing means includes
The image processing program according to claim 19, wherein a scene determination process is performed using a linear image with a reduced image size. The computer,
Function as luminance information acquisition means for acquiring luminance information calculated at the time of photographing the linear image;
The scene discrimination processing means includes
Based on the luminance information acquired by the luminance information acquisition means, to determine the shooting scene,
The gradation processing condition setting means includes:
The image processing program according to claim 19 or 20, wherein the first and second gradation processing conditions are set based on the luminance information acquired by the luminance information acquisition means. The scene discrimination processing means includes
Calculating a natural light index representing the degree of outdoor photographing with natural light of the linear image, a luminance ratio index representing the size of the dynamic range, and an exposure photographing degree index resulting from the exposure setting at the time of photographing;
The image processing program according to any one of claims 19 to 21, wherein a shooting scene is determined based on each of the calculated indices. The scene discrimination processing means includes
Color information is acquired for the linear image, and based on the acquired color information, the image data of the linear image is classified into a class composed of a combination of predetermined brightness and hue, and for each of the classified classes, Calculating a first occupancy ratio indicating the ratio of the entire image data;
Based on the acquired color information, the image data is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a second ratio indicating the proportion of the entire image data for each classified class Calculate the share of
By multiplying the first occupancy by a first coefficient set in advance according to the shooting conditions, the index 1 is calculated,
The index 2 is calculated by multiplying the first occupancy by a second coefficient set in advance according to the shooting conditions,
The index 3 is calculated by multiplying the second occupation ratio by a third coefficient set in advance according to the shooting conditions,
The exposure photographing degree is calculated by multiplying the average brightness of the skin color at least in the center of the screen of the image data by a fourth coefficient set in advance according to the photographing condition,
The natural light index is calculated by multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a fifth coefficient that is set in advance according to shooting conditions. And
By multiplying at least one of the index 1, the index 2, and the index 3 and the luminance information by a sixth coefficient that is set in advance according to shooting conditions, the luminance ratio index is obtained. Calculate
23. The image processing program according to claim 22, wherein a shooting scene of the image data is determined based on the exposure shooting degree, the natural light index, and the luminance ratio index. The gradation processing condition setting means includes:
Based on the brightness analysis value of the linear image, the brightness reproduction target value, the brightness ratio index, and the exposure shooting degree, a plurality of first temporary correction amounts for the linear image are set,
Based on the natural light index, set a first mixing coefficient as a weighting coefficient to be multiplied to each of the plurality of first temporary correction amount,
Based on the first mixing coefficient and the plurality of first temporary correction amounts, a second temporary correction amount for the linear image is set,
Based on the exposure photographing degree, the natural light index, and the luminance ratio index, at least one of the plurality of first temporary correction amounts and a second mixing coefficient as a weighting coefficient for multiplying the second temporary correction amount are set. And
Calculating a gradation adjustment parameter based on the second mixing coefficient, the at least one first temporary correction amount, and the second temporary correction amount;
24. The first gradation processing condition for the linear image is set based on a determination result by the scene determination processing unit and the calculated gradation adjustment parameter. Image processing program.

Images