JP2007228132A - Imaging apparatus, image processing apparatus, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of automatically setting correction processing conditions by discriminating picked-up images per scene, and applying more properly corrected images by applying correction considering the condition where halation occurs, and to provide an image processing apparatus, an image processing method and an image processing program. <P>SOLUTION: An index for predicting the condition where the halation occurs in correction processing, on the basis of distribution of a high-intensity portion in positions in a screen, and the set correction processing conditions are re-corrected, depending on the index. In this way, even in a situation where halation may occur, correction can be added in the direction of suppressing the halation, thereby acquiring a more properly corrected image. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, an image processing method, and an image processing program that perform image processing of a captured image.

  In recent years, digital still cameras (including those incorporated in devices such as mobile phones and laptop computers, hereinafter abbreviated as DSC) have become widespread, and, as with conventional color photographic film systems, output as hard copy images. In addition, systems that display on a medium such as a CRT or record on a recording medium such as a CD-R (CD-Recordable) are widely used.

  However, when an image taken with a DSC or the like is viewed as an image through various media as described above, an image suitable for viewing is generally obtained as it is due to inadequate exposure adjustment at the time of shooting. There are many cases where this is not possible. That is, in order to obtain an appropriate appreciation image, it is desirable that an appropriate image gradation correction process is performed in order to compensate for overexposure or underexposure during shooting.

  In addition, if there is a large luminance bias in the image due to the influence of the light source state at the time of shooting, for example, shooting in a backlit state or close-up shooting using a flash, the image can be viewed as it is. It cannot be said that the image is suitable for the purpose, and it is desirable to perform some correction processing.

  In order to solve such problems, correction processing such as gradation conversion processing has been performed conventionally, and various proposals have been made regarding methods for determining the correction processing (for example, Patent Document 1 and 2).

  For example, Patent Document 1 describes a method of discriminating an image scene by analyzing image data when a DSC photographed image is output by a printer, and automatically performing optimum image processing according to the image scene. Yes.

  Further, Patent Document 2 obtains subject information such as in-screen contrast strength and luminance distribution state from the divided photometry result at the time of DSC photographing, and determines gradation correction of photographed image data in the DSC according to the subject information. A method is described.

Japanese Patent Application Laid-Open No. 2005-228561 describes a method for improving the brightness of an image by detecting skin color pixels from a digital image and applying a gradation conversion amount calculated based on the statistics of the skin color pixels to the image. .
JP 2000-134467 A JP 2001-54014 A JP 2003-108988 A

  The methods described in Patent Literature 1 and Patent Literature 2 automatically correct a captured image by performing processing such as scene discrimination from the brightness and distribution of the captured image or obtaining a direct correction method. be able to. Further, the method described in Patent Document 3 focuses on the person and skin elements as subjects and attempts to improve the brightness. However, in either case, depending on the situation of the photographic scene, the brightness balance in the image may be lost due to the correction.

  For example, in a general image, a person's face is most noticed during viewing, and it is important that the brightness of the main subject such as the person's face and the brightness of the entire image are properly balanced.

  However, with only the method described above, the area of whiteout in the corrected image may become large, or whiteout may occur in the main subject portion, resulting in the intention of the photographer. In some cases, an image reflecting the above cannot be obtained, and an unfavorable image may be obtained.

  An object of the present invention is to automatically set a correction processing condition by determining a photographing scene of a photographed image, and to add correction considering the occurrence of overexposure based on image data, To provide an imaging apparatus, an image processing apparatus, an image processing method, and an image processing program capable of obtaining a more appropriate corrected image.

  The present invention has the following features in order to solve the above problems.

  1. A scene discrimination processing unit that performs a scene determination process based on image data of an image obtained by shooting, and a gradation setting condition for the image based on the scene determination result obtained by the scene determination processing unit. A first occupancy ratio that classifies the image data into a class composed of a combination of a distance from the outer edge of the screen and a brightness, and indicates a ratio of the entire image data for each classified class Of the first occupancy ratio calculated by the first occupancy ratio calculation means and the first occupancy ratio calculation means, the occupancy ratio for the class consisting of a combination of at least the center portion of the screen and high brightness, A first index calculation unit that calculates index 1 by multiplying a preset first coefficient, wherein the gradation processing condition setting unit is configured to perform the scene determination. The setting of the gradation processing condition based on results, corrected based on the indices 1 calculated by the first index calculating means, the image pickup apparatus, characterized in that.

  2. A second image acquisition unit configured to acquire a second image obtained by reducing the image size from the first image acquired by photographing, wherein the scene determination processing unit is acquired by the second image acquisition unit; 2. The imaging apparatus according to 1, wherein a shooting scene discrimination process is performed based on the image data of the second image.

  3. The imaging apparatus according to 2, wherein the first occupancy rate calculating unit calculates the first occupancy rate from image data of the second image acquired by the second image acquisition unit. .

  4). The scene determination processing unit acquires color information for the image data of the second image, and classifies the image data into a class composed of a combination of predetermined brightness and hue based on the acquired color information, For each classified class, a second occupancy ratio calculating unit that calculates a second occupancy ratio indicating a ratio of the entire image data, and a first occupancy ratio calculated by the first occupancy ratio calculator By multiplying the rate by a second coefficient set in advance, second index calculation means for calculating index 2 and by multiplying the second occupancy by a preset third coefficient A third index calculating means for calculating the index 3, a fourth index calculating means for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient, and at least the In the center of the image data screen A fifth index calculating means for calculating the index 5 by multiplying the average luminance of the skin color by a fifth coefficient set in advance, and at least one of the index 2, the index 3, and the index 4 Sixth index calculation means for calculating index 6 by multiplying one or more indices by a preset sixth coefficient, and at least one or more of index 2, index 3, and index 4 A seventh index calculating means for calculating the index 7 by multiplying the index of the index by a preset seventh coefficient, and the scene of the image data based on the index 5, the index 6 and the index 7 The imaging apparatus according to 2 or 3, further comprising: a scene determination unit that identifies

  5. The first index calculation means is set in advance to an occupancy ratio for a class composed of a combination of a screen outer edge portion and high brightness among the first occupancy ratio calculated by the first occupancy ratio calculation means. The imaging apparatus according to any one of 1 to 4, wherein the index 1 is calculated by multiplying the eighth coefficient.

  6). A scene discrimination processing unit that performs a scene determination process based on image data of a photographed image, and a gradation process that sets a gradation processing condition for the image based on a scene discrimination result obtained by the scene discrimination processing unit The condition setting means and the image data are classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a first occupancy ratio indicating a ratio of the entire image data is calculated for each classified class. The first occupancy ratio calculating means and the first occupancy ratio calculated by the first occupancy ratio calculating means are set in advance to an occupancy ratio for at least a class consisting of a combination of the screen center and high brightness. A first index calculation means for calculating index 1 by multiplying the first coefficient, and the gradation processing condition setting means is based on the scene discrimination result. The setting of the gradation processing condition is corrected based on the calculated index 1 by the first index calculating means, that the image processing apparatus according to claim.

  7). A second image acquisition unit configured to acquire a second image obtained by reducing the image size from the first image acquired by photographing, wherein the scene determination processing unit is acquired by the second image acquisition unit; 7. The image processing apparatus according to 6, wherein a scene determination process is performed based on the image data of the second image.

  8). 8. The image processing according to claim 7, wherein the first occupancy rate calculating unit calculates the first occupancy rate from image data of the second image acquired by the second image acquisition unit. apparatus.

  9. The scene determination processing unit acquires color information for the image data of the second image, and classifies the image data into a class composed of a combination of predetermined brightness and hue based on the acquired color information, For each classified class, a second occupancy ratio calculating unit that calculates a second occupancy ratio indicating a ratio of the entire image data, and a first occupancy ratio calculated by the first occupancy ratio calculator By multiplying the rate by a second coefficient set in advance, second index calculation means for calculating index 2 and by multiplying the second occupancy by a preset third coefficient A third index calculating means for calculating the index 3, a fourth index calculating means for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient, and at least the In the center of the image data screen A fifth index calculating means for calculating the index 5 by multiplying the average luminance of the skin color by a fifth coefficient set in advance, and at least one of the index 2, the index 3, and the index 4 Sixth index calculation means for calculating index 6 by multiplying one or more indices by a preset sixth coefficient, and at least one or more of index 2, index 3, and index 4 A seventh index calculating means for calculating the index 7 by multiplying the index of the index by a preset seventh coefficient, and the scene of the image data based on the index 5, the index 6 and the index 7 The image processing apparatus according to 7 or 8, further comprising: a scene determination unit that identifies

  10. The first index calculation means is set in advance to an occupancy ratio for a class composed of a combination of a screen outer edge portion and high brightness among the first occupancy ratio calculated by the first occupancy ratio calculation means. The image processing apparatus according to any one of 6 to 9, wherein the index 1 is calculated by multiplying the eighth coefficient.

  11. A scene determination processing step for performing a shooting scene determination process based on image data of a photographed image, and a gradation process for setting a gradation processing condition for the image based on a scene determination result obtained by the scene determination processing step The condition setting step and the image data are classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness, and a first occupancy ratio indicating a ratio of the entire image data is calculated for each classified class. Among the first occupancy ratios calculated by the first occupancy ratio calculating step and the first occupancy ratio calculating means, the occupancy ratio is set in advance for an occupancy ratio for a class composed of a combination of at least the screen center and high brightness. A first index calculation step of calculating the index 1 by multiplying the first coefficient that has been performed, and in the gradation processing condition setting step, the scene determination result The setting of the brute gradation processing condition is corrected based on the indices 1 calculated in the first index calculating step, an image processing method, characterized in that.

  12 A second image acquisition step of acquiring a second image obtained by reducing the image size from the first image acquired by photographing, wherein the scene discrimination processing step is acquired in the second image acquisition step; 12. The image processing method according to 11, wherein a photographic scene discrimination process is performed based on the image data of the second image.

  13. 13. The image processing according to 12, wherein the first occupancy ratio calculating step calculates the first occupancy ratio from image data of the second image acquired in the second image acquisition step. Method.

  14 The scene determination processing step acquires color information for the image data of the second image, and classifies the image data into a class composed of a combination of predetermined brightness and hue based on the acquired color information, For each classified class, a second occupancy ratio calculating step for calculating a second occupancy ratio indicating a ratio of the entire image data, and a first occupancy calculated in the first occupancy ratio calculation step By multiplying the rate by a second coefficient set in advance, a second index calculation step for calculating index 2 and multiplying the second occupancy by a preset third coefficient, A third index calculating step for calculating the index 3, a fourth index calculating step for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient, and at least the image In the center of the data screen A fifth index calculation step of calculating the index 5 by multiplying the average brightness of the skin color by a fifth coefficient set in advance, and at least one of the index 2, the index 3, and the index 4 A sixth index calculating step of calculating index 6 by multiplying at least one index by a preset sixth coefficient, and at least one of index 2, index 3, and index 4 A seventh index calculating step of calculating the index 7 by multiplying the index of the index by a preset seventh coefficient, and the scene of the image data based on the index 5, the index 6 and the index 7 The image processing method according to 12 or 13, further comprising: a scene determination step for specifying

  15. The first index calculation step is preset to an occupancy rate for a class composed of a combination of the outer edge of the screen and high brightness among the first occupancy rates calculated in the first occupancy rate calculation step. 15. The image processing method according to any one of 11 to 14, wherein the index 1 is calculated by multiplying the eighth coefficient.

  16. An image processing program for causing a computer to execute image processing, a scene determination processing function for performing a scene determination processing process based on image data of a captured image, and a scene determination result obtained by the scene determination processing function And a gradation processing condition setting function for setting gradation processing conditions for the image, and classifying the image data into a class composed of a combination of a distance from the outer edge of the screen and brightness, and for each of the classified classes, Of the first occupancy ratio calculating function for calculating a first occupancy ratio indicating the ratio of the entire image data, and the first occupancy ratio calculated by the first occupancy ratio calculation function, at least the center portion of the screen The first index calculation for calculating index 1 by multiplying the occupancy ratio for the class consisting of the combination of the brightness and the high brightness by a preset first coefficient The gradation processing condition setting function corrects the setting of the gradation processing condition based on the scene discrimination result based on the index 1 calculated by the first index calculation function. A characteristic image processing program.

  17. A second image acquisition function for acquiring a second image obtained by reducing the image size from the first image acquired by shooting, wherein the scene determination processing function is acquired by the second image acquisition function; 17. The image processing program according to 16, wherein a shooting scene discrimination process is performed based on the image data of the second image.

  18. 18. The image processing according to 17, wherein the first occupancy rate calculating function calculates the first occupancy rate from image data of the second image acquired by the second image acquisition function. program.

  19. The scene determination processing function acquires color information for the image data of the second image, and classifies the image data into a class composed of a combination of predetermined brightness and hue based on the acquired color information, For each classified class, a second occupancy ratio calculation function for calculating a second occupancy ratio indicating a ratio of the entire image data, and a first occupancy ratio calculated by the first occupancy ratio calculation function By multiplying the rate by a second coefficient preset, a second index calculation function for calculating index 2 and by multiplying the second occupancy by a preset third coefficient, A third index calculation function for calculating the index 3, a fourth index calculation function for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient, and at least the image In the center of the data screen At least one of the fifth index calculation function for calculating the index 5 by multiplying the average brightness of the skin color by a fifth coefficient set in advance, the index 2, the index 3, and the index 4 A sixth index calculation function for calculating index 6 by multiplying the above index by a preset sixth coefficient, and at least one of index 2, index 3, and index 4 Based on the seventh index calculation function for calculating the index 7 by multiplying the index by a preset seventh coefficient, the index 5, the index 6, and the index 7, the scene of the image data is calculated. The image processing program according to 17 or 18, further comprising a scene discrimination function to be specified.

  20. The first index calculation function is set in advance to an occupancy ratio for a class including a combination of a screen outer edge portion and high brightness among the first occupancy ratios calculated by the first occupancy ratio calculation function. The image processing program according to any one of 16 to 19, wherein the index 1 is calculated by multiplying the eighth coefficient.

  According to the present invention, when automatically setting correction processing conditions by determining a shooting scene of a shot image, high brightness at a position in the screen is used as an element for predicting the occurrence of whiteout at the time of image correction. Based on the distribution of the degree part, an index that indicates the occurrence of whiteout is calculated, and the correction processing conditions that have been set are corrected in accordance with the index, so that even if whiteout is likely to occur, it will be suppressed. An imaging apparatus, an image processing apparatus, an image processing method, and an image processing program that can add correction and obtain a more appropriate corrected image can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, the configuration of the first embodiment (imaging device) according to the present invention will be described.

(Configuration of the imaging device 1)
FIG. 1A shows a front view of the imaging apparatus 1 according to the embodiment of the present invention, and FIG. 1B shows a rear view of the imaging apparatus 1. The imaging device 1 is a digital camera, for example, and has a cross key 22, a photographing optical system 23, a flash 24, a finder 25, and a power switch 26 on or inside a housing 21 made of a material such as metal or synthetic resin. A display unit 27 and a release button 28 are provided.

  FIG. 2 shows an internal configuration of the imaging apparatus 1. As shown in FIG. 2, the image pickup apparatus 1 includes a processor 31, a memory 32, an image pickup device 33 such as a CCD (Charge Coupled Device), a shooting optical system 23, a timing generator 41, a shutter unit 42, a diaphragm unit 43, and a focus unit 44. , A display unit 27, an operation unit 38, and an image data output unit 37. The processor 31 includes a photographing processing unit 20 that performs photographing control and photographed image processing, and an image processing unit 10 that performs image processing.

  The cross key 22 is made up of buttons in four directions, up, down, left and right, and is used by the user to select or set various modes.

  The photographing optical system 23 includes a plurality of lenses, a lens barrel, and the like, and has a zoom function. The photographing optical system 23 causes the light received by the lens to form an image on the image sensor 33. The flash 24 emits auxiliary light by a control signal from the processor 31 when the subject brightness is low.

  The viewfinder 25 is used for the user to check the shooting target and shooting area by eye contact. The power switch 26 is a switch for operating ON / OFF of the operation in the imaging apparatus 1.

  The display unit 27 includes a liquid crystal panel, and displays an image currently captured on the image sensor 33, an image captured in the past, a menu screen, a setting screen, and the like according to a display control signal input from the processor 31.

  The release button 28 is a two-stage push-in switch that is provided on the upper surface of the housing 21 and can detect a half-pressed state (preliminary shooting) and a fully-pressed state (main shooting) by the user.

  The memory 32 stores (saves) image data obtained by photographing. In addition, the memory 32 stores various processing programs executed in the imaging apparatus 1, data used in the processing programs, and the like.

  The image data output unit 37 transfers and records the image data in the memory 32 to a recording medium for storage (such as an SD memory card or a multimedia card (MMC)), or transfers it to an external device. The image data output unit 37 is controlled by the processor 31.

  The image sensor 33 converts the imaged light into electric charges. Thereby, for example, image data as shown in FIG. This image includes an object in the imaging range (imaging range), that is, an imaging object (imaging target) and other objects (background). The RGB value of each pixel of the entire image is represented by, for example, 256 gradations.

  The shutter unit 42 controls the timing of resetting the image sensor 33 and the timing of charge conversion based on the state (half-pressed state or fully-pressed state) detected by the release button 28. Timing control is performed by the timing generator 41. The exposure amount control by the shutter unit 42 will be described later.

  Adjustment of the amount of light received by the image sensor 33 is performed by the aperture unit 43 and / or the shutter unit 42. The focus unit 44 performs a control operation for driving the photographing optical system 23 to focus on the photographing subject. Focus control by the focus unit 44 will be described later.

(Internal configuration of the imaging processing unit 20)
FIG. 3 shows an internal configuration of the imaging processing unit 20. The shooting processing unit 20 performs control related to shooting conditions during shooting and processing of shot images. The internal configuration of the imaging processing unit 20 will be described with reference to FIG.

  As shown in FIG. 3, the imaging processing unit 20 includes an AE control unit 51, an AF control unit 52, and a pixel interpolation unit 53, an AWB control unit 54, and a gamma that perform image processing on a captured image. The correction unit 55 is configured.

  The AE control unit 51 automatically controls the exposure amount at the time of image shooting. Normally, the exposure amount control during shooting standby is performed by controlling the shutter speed with the aperture opened, and the exposure amount during shooting is controlled by the aperture and shutter speed.

  For example, the exposure amount control during shooting standby in the AE control unit 51 is performed as follows (see the flowchart in FIG. 4A).

  First, the diaphragm unit 43 sets the diaphragm to an open fixed diaphragm (step S111). The AE control unit 51 reads data of a predetermined photometric area from the image data obtained by the image sensor 33 (step S112), and acquires information corresponding to the luminance value (step S113). Information corresponding to the luminance value is used as information for AE control, and the G value of RGB three-color components is often used simply (hereinafter referred to as luminance information G). The charge accumulation time in the next frame of the image sensor 33 is set according to the luminance information G (step S114), and the charge accumulation time in the next frame is controlled by the timing generator 41 so that the predetermined luminance level is obtained. (Step S117). This is control of the shutter speed, and FIG. 5 shows how charges are accumulated. That is, when the acquired luminance level is large (bright), the charge accumulation time is shortened, and when the acquired luminance level is small (dark), the charge accumulation time is lengthened, thereby stabilizing the exposure amount.

  By performing exposure amount control during shooting standby in this way, a photographer can observe a live view image whose exposure amount is controlled on the display unit 27 such as a liquid crystal display.

  In the exposure control at the time of actual photographing in the AE control unit 51, in addition to the shutter speed control described above, aperture control is also performed (see the flowchart in FIG. 4B). Similarly to the above (steps S112 and S113), the aperture unit 43 is controlled according to the luminance information G of the photometric area, and an aperture value is set (step S116). That is, the exposure value is stabilized by decreasing the aperture value when the acquired brightness level is large (bright) and increasing the aperture value when the brightness level is low (dark). The adjustment level combined with the shutter speed is controlled based on data of a predetermined program diagram, for example, such that the charge accumulation time of the image sensor 33 is adjusted according to the aperture value (step S115, S117).

  As described above, the exposure amount control is performed at the time of photographing, so that the photographer can automatically set the exposure amount with respect to the photographed image by using an arbitrary or predetermined combination of the aperture value and the shutter speed. If the subject brightness is low and a predetermined exposure value cannot be obtained with a combination of a predetermined aperture value and shutter speed, the flash 24 emits auxiliary light according to a control signal from the processor 31 according to the flash mode.

  The AF control unit 52 performs automatic control for focusing an image when capturing an image. This in-focus control is performed by, for example, driving the photographing optical system as described below to detect the in-focus position and stopping it according to the position (see the flowchart in FIG. 6).

  When driving of the photographic optical system 23 is started (step S121), the AF control unit 52 sequentially reads data of a predetermined distance measuring area from the image data obtained by the image sensor 33 (step S122). ), Contrast information is acquired according to this data (step S123). This is for detecting whether or not the in-focus position has been reached, and is determined as follows. That is, the contrast information is set and calculated so as to depend on the sharpness of the edge portion by taking the difference between adjacent pixels in the distance measurement area data. The state that has reached the maximum is determined to be the focal point (steps S124 and S125). If it is determined that the focus position is not reached, the movement of the photographing optical system is continued (step S126).

  FIG. 7 shows a change in contrast information accompanying the movement of the photographic optical system 23 and how the focal point position is detected. In the above operation, a distance measurement calculation is performed in which contrast information is sequentially acquired while driving the optical system to obtain a focal point position, and the photographing optical system 23 is stopped in accordance with the focal length ( Step S127).

  With such AF control, it is possible to obtain a captured image that is always in focus automatically during shooting.

  The pixel interpolation unit 53 interpolates pixels for each color component with respect to the CCD array in which the RGB color components are dispersedly arranged in the image sensor 33, and processes the image data so that each color component value is obtained at the same pixel position. (Refer to the flowchart of FIG. 8).

  The pixel interpolation unit 53 masks the RGB image data (step S141) obtained by the image sensor 33 with each of the RGB pixel filter patterns (steps S142, S144, and S146), and then performs average interpolation (also referred to as pixel interpolation). Perform (Steps S143, S145, S147). Among these, the G pixel filter pattern having pixels up to a high band is a median filter that performs average interpolation by substituting the average value of the intermediate binary values of the surrounding four pixels, and the R and B pixel filters The pattern performs average interpolation with respect to the same color from 9 neighboring pixels.

  The AWB control unit 54 automatically adjusts the white balance in the captured image. Assuming that the photographed image has a subject area in which RGB color balance is achieved (white when summed up), each component value of RGB of the image is achieved so as to achieve white balance in that area. Adjust the level for. This white balance processing is performed, for example, as follows (see the flowchart of FIG. 9).

  The AWB control unit 54 estimates a region that is supposed to be white from the luminance and saturation data of the image data obtained by the image sensor 33 (step S131) (step S132). For that region, the average intensity of each RGB component value, the G / R ratio, and the G / B ratio are obtained, and the R value and B value correction gains for the G value are calculated (steps S133 and S134). Based on this, gain correction for each color component in the entire image is performed (step S135).

  By such AWB control, it is possible to automatically correct the collapse of the color balance of the entire screen that occurs during shooting, and an image with a stable color tone can be obtained regardless of the actual illumination state of the subject.

  The gamma correction unit 55 performs processing for converting the gradation of the captured image so as to be suitable for the characteristics of the output device.

  The gamma characteristic is a gradation characteristic and indicates how to set an output gradation with respect to an input gradation by a correction value or a correction curve. FIG. 10 shows an example of a correction curve indicating an output value corrected for the input value. The gamma correction is a conversion process for converting an input value into an output value according to the correction value or the correction curve. Since this gradation characteristic varies depending on the output device, it is necessary to correct this gamma characteristic in order to obtain a gradation suitable for the output device. As a result, the captured linear image is converted into a non-linear image.

  A monitor is generally set as the output device, and gamma correction of the captured image is performed so as to match the gamma characteristic of a general monitor.

(Internal configuration of the image processing unit 10)
FIG. 11 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls a gradation correction operation based on scene discrimination for a captured image in the imaging apparatus 1, and after the processing in the imaging processing unit 20 or independently of the processing, A series of processes such as image acquisition, scene discrimination, and gradation processing condition setting are executed. The internal configuration of the image processing unit 10 will be described with reference to FIG.

  As shown in FIG. 11, the image processing unit 10 includes a first image acquisition unit 101, a second image acquisition unit 102, a first occupancy rate calculation unit 103a, a second occupancy rate calculation unit 103b, an index calculation unit 104, and a scene determination. A unit 105, a first index calculation unit 106, a gradation processing condition setting unit 107, and a gradation conversion processing unit 108.

  The first image acquisition unit 101 acquires the image data of the latest image captured on the image sensor 33 as the first image when the release button 28 is fully pressed. The acquired image data of the first image is held in the memory 32.

  The second image acquisition unit 102 divides the acquired first image into N × M rectangular regions (regions divided into M pieces in the vertical direction and N pieces in the horizontal direction). FIG. 12A shows an example of the first image, and FIG. 12B shows an example in which the first image is divided into 22 × 14 regions. The number of divided areas is not particularly limited. In the present embodiment, each divided area is regarded as a “pixel” of image data, and the divided image is handled as an image with a substantially reduced size. This reduced image is the second image, and the second image is acquired by the above operation. Therefore, the second image acquisition unit 102 functions as a second image acquisition unit.

  The first occupancy rate calculation unit 103a acquires color information for each piece of image data, that is, each pixel constituting the image. Based on the acquired color information, each pixel of the image is classified into a predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness (see FIG. 16), and each classified class belongs to the class. A first occupation ratio indicating a ratio of pixels to the entire image is calculated. That is, the first occupancy rate calculation unit 103a functions as a first occupancy rate calculation unit.

  In addition, the second occupancy rate calculation unit 103b classifies each pixel into a predetermined class composed of a combination of brightness and hue (see FIG. 17), and for each classified class, the pixels belonging to the class are included in the entire image. A second occupation ratio indicating the occupation ratio is calculated. That is, the second occupancy rate calculation unit 103b functions as a second occupancy rate calculation unit.

  The occupation rate calculation processing executed in the first occupation rate calculation unit 103a and the second occupation rate calculation unit 103b will be described in detail later with reference to FIG.

  The index calculation unit 104 multiplies the first occupancy calculated by the first occupancy rate calculation unit 103a by a coefficient set in advance according to the shooting conditions, thereby obtaining an index 2 for specifying the shooting scene. calculate. That is, the index calculation unit 104 functions as a second index calculation unit.

  In addition, the index calculation unit 104 multiplies the second occupancy calculated by the first occupancy calculation unit 103b by a coefficient set in advance according to the shooting conditions, thereby specifying an photographic scene. 3 and index 4 are calculated. That is, the index calculation unit 104 also functions as a third index calculation unit and a fourth index calculation unit.

  Furthermore, the index calculation unit 104 multiplies the average luminance value in the center of the screen of the image and the difference value between the maximum luminance value and the average luminance value by a coefficient set in advance according to the shooting condition, An index 5 for specifying the shooting scene is calculated. That is, the index calculation unit 104 functions as a fifth index calculation unit.

  In addition, the index calculation unit 104 multiplies the average luminance value of the skin color area (referred to as index 9) in the center of the screen of the image, and the index 2 and the index 3, respectively, by coefficients set in advance according to the shooting conditions. The new index 6 is calculated by taking the sum. That is, the index calculation unit 104 functions as a sixth index calculation unit.

  Further, the index calculation unit 104 calculates a new index 7 by multiplying the average luminance value, the index 2 and the index 4 by a coefficient set in advance according to the shooting conditions, and taking the sum. . That is, the index calculation unit 104 functions as a seventh index calculation unit.

  The index calculation process executed in the index calculation unit 104 will be described in detail later with reference to FIG.

  The scene determination unit 105 determines the shooting scene of the first image based on each index calculated by the index calculation unit 104. That is, the scene determination unit 105 functions as a scene determination unit. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes over and under degrees. The method for determining the shooting scene will be described later in detail.

  As described above, the second occupancy ratio calculation unit, the index calculation unit, and the scene determination unit function as a scene determination processing unit.

  The first index calculator 106 corrects the gradation processing condition by multiplying the first occupancy calculated by the first occupancy calculator 103a by a coefficient set in advance according to the shooting condition. The index 1 is calculated. That is, the first index calculation unit 106 functions as a first index calculation unit. The first index calculation process will be described later with reference to FIG.

  The gradation processing condition setting unit 107 functions as a gradation processing condition setting unit, and sets gradation processing conditions (see FIG. 25) for the first image based on the shooting scene determined by the scene determination unit 105. .

  Further, the gradation processing condition setting unit 107 calculates a gradation adjustment parameter for gradation adjustment for the first image based on each index calculated by the index calculation unit 104, and further calculates a gradation conversion amount. decide. The setting of the gradation processing conditions will be described in detail later with reference to FIG.

  The gradation conversion processing unit 108 executes gradation conversion processing for the first image in accordance with the gradation processing conditions set in the gradation processing condition setting unit 107.

  In addition to the processing performed in the image processing unit 10, the processor 31 performs processing in the photographing processing unit 20 such as automatic white balance processing and gamma conversion processing, and other image processing, image format conversion, It has a function of controlling processing operations such as recording of image data.

  The processing of each unit in the processor 31 is basically performed by hardware processing, but a part of the processing is performed by software processing by executing a program stored (saved) in the memory 32.

(Operation Flow of Imaging Device 1)
Next, the operation of the imaging device 1 in the present embodiment will be described.

  Hereinafter, the photographing object is referred to as “main subject”.

  First, the overall flow of processing executed by the imaging apparatus 1 will be described with reference to the flowchart of FIG. These processes can be executed by a computer using a program. Each process described below can be similarly described as each function of the image processing program.

  First, when the power switch 26 is turned on (when the power is turned on), preprocessing such as resetting of the memory 32 is performed (step S1). The user directs the imaging device 1 toward the main subject so that the main subject enters the field of the imaging device 1, and starts an operation for photographing. The release button 28 is pressed to take a picture (step S2). The image formed on the image sensor 33 is captured as an electrical signal, and interpolation processing based on the CCD array is performed (step S4). In step S <b> 5, the first image is acquired as a captured image and held in the memory 32. The first image is a linear image and is called a RAW image.

  The first image acquired by shooting is subjected to AWB (automatic white balance) processing (step S6). This may be processed separately for the first image and the second image after the second image described below is acquired.

  After the AWB process, the image data of the first image is acquired as a divided image composed of a plurality of divided regions, that is, a second image (step S7). Each divided region of the divided image is a pixel of the second image, and the second image is an image obtained by reducing the size of the first image. FIG. 12A shows an example of the first image, and FIG. 12B shows an example in which the first image is divided into 22 × 14 cells. Each cell corresponds to one pixel of the second image. As an actual image data size reduction method, a known technique such as a simple average, a bilinear method, or a bicubic method can be used. Step S7 functions as a second image acquisition process.

  In step S8, which is the first occupancy rate calculation step, the first occupancy rate is calculated for the second image. However, the image data does not necessarily have to be the second image. You may use the image data of the 1st image acquired by imaging | photography. The first occupation rate calculation process will be described later with reference to FIG.

  In step S10, index 1 is calculated based on the first occupation ratio calculated in step S8. That is, step S10 functions as a first index calculation step. The index 1 is used later to correct the gradation processing condition set by the scene discrimination result. A method for calculating the index 1 will be described later with reference to FIG.

  Next, in step S9, which is a scene determination process step, a scene determination process for determining a shooting scene is performed based on the acquired image data of the second image. The shooting scene determination process in step S9 will be described later with reference to FIG.

  Next, in step S11 which is a gradation processing condition setting step, it is necessary for the gradation conversion processing of the first image based on each index obtained in the scene determination processing in step S9 and the determination result of the shooting scene. Processing for setting conditions is performed. Further, the gradation conversion amount is adjusted by the index 1 calculated in the first index calculation step. The gradation processing condition setting process in step S7 will be described later with reference to FIG.

  On the other hand, the first image after the AWB processing is subjected to gamma conversion processing and converted into a non-linear image (step S12). However, the gamma conversion process is a gradation conversion process, and may be performed together with the gradation conversion process in step S13 described below.

  In step S13, gradation conversion processing is performed on the image data of the first image, which is a captured image, based on the gradation processing conditions set in step S11. The gamma conversion is converted into a non-linear gradation, but the gradation conversion in step S13 is to correct the influence on the gradation due to the light source condition of the shooting scene, and is set in step S11 based on the scene discrimination result in step S9. A gradation conversion process is performed based on the processed conditions.

  Next, the image format is converted for image recording (step S14). Generally, it is converted into an image in JPEG format. Thereafter, the image data in the JPEG format is recorded on a storage recording medium (such as an SD memory card or a multimedia card (MMC)) (step S15). When the next shooting is started or the power switch 26 is turned off, the operation of the image pickup apparatus 1 ends.

(Scene discrimination process 1 flow)
Next, referring to the flowchart of FIG. 14 and FIGS. 15 to 23, the first occupancy rate calculation process (step S8 in FIG. 13) and the scene determination process 1 (step S9 in FIG. 13) in the imaging apparatus 1 will be described. explain. In the following description, the scene determination process including the first occupancy rate calculation process will be referred to.

As shown in FIG. 14, the scene determination process includes a first occupancy ratio calculation process (step S20), a second occupancy ratio calculation process (step S21), an index calculation process (step S22), and a scene determination (step S23). It consists of each process. Hereinafter, with reference to FIGS. 15 to 23, each process illustrated in FIG. 14 will be described in detail.
<First Occupancy Calculation Processing 1>
First, in step S20, a first occupancy rate calculation process is performed. The first occupancy rate calculation process will be described with reference to the flowchart in FIG.

  In step S30 of FIG. 15A, color space conversion processing is first performed.

  Information indicating the RGB value, luminance value, and white balance of each pixel of the second image obtained from the photographed first image is acquired. As the luminance value, a value calculated by substituting RGB values into a known conversion formula may be used. Next, the acquired RGB values are converted into the HSV color system, and the color information of the image is acquired. The HSV color system represents image data with three elements of hue (Hue), saturation (Saturation), and lightness (Value or Brightness), and was devised based on the color system proposed by Munsell. It has been done. The conversion to the HSV color system is performed using an HSV conversion program or the like. Usually, the calculated hue value H is defined as 0 to 360 for the input R, G, B, The unit of saturation value S and brightness value V is defined as 0-255.

  In this embodiment, “brightness” means “brightness” generally used unless otherwise noted. In the following description, V (0 to 255) of the HSV color system is used as “brightness”, but a unit system representing the brightness of any other color system may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated.

  In the present embodiment, “hue” means “color” that is generally used unless otherwise noted. In the following description, H (0 to 360) of the HSV color system is used as “hue”, but for example, a color represented by a red difference value (Cr) or a blue difference value (Cb) may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated. In step S30, the values of H, S, and V obtained as described above are acquired as color information.

  In step S31, first, each pixel of the second image is classified into a predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness based on the HSV value calculated in the color sky conversion process. A two-dimensional histogram is created by calculating the cumulative number of pixels for each class.

  FIG. 16A shows three regions n1 to n3 divided in accordance with the distance from the outer edge of the screen of the second image in step S31. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. A region n3 is a central region of the second image. Here, it is preferable that n1 to n3 are divided so as to have substantially the same number of pixels. In the present embodiment, three divisions are used, but the present invention is not limited to this. In step S31, the brightness (V) is 0 to 5 (v1), 6 to 12 (v2), 13 to 24 (v3), 25 to 76 (v4), 77 to 109 (v5), It is divided into seven classes of 110 to 149 (v6) and 150 to 255 (v7). As shown in FIG. 16, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  FIG. 16B shows a class composed of combinations of three regions n1 to n3 and brightness. As shown in FIG. 16B, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 3 × 7 = 21.

  When the two-dimensional histogram is created in step S31, the first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness to the total number of pixels is obtained. It is calculated (step S32), and the first occupancy rate calculation process ends. That is, step S31 and step S32 function as a first occupation rate calculation step.

  When the first occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the first occupancy in each class is expressed as shown in Table 1.

<Second Occupancy Calculation Processing 1>
Next, in step S21 of FIG. 14, a second occupancy rate calculation process is performed. The second occupancy rate calculation process will be described with reference to the flowchart in FIG.

  In step S35 of FIG. 15B, each pixel of the second image is classified into a predetermined class composed of a combination of hue and brightness, and a cumulative pixel count is calculated for each classified class, thereby obtaining a two-dimensional histogram. Is created.

  FIG. 17 shows a class composed of a combination of brightness and hue. In step S35, the lightness is divided into seven regions v1 to v7 as described above. As shown in FIG. 17, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  Hue (H) is a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region having a hue value of 161 to 250. It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the shooting scene is small. The skin color hue region is further divided into a skin color region (H1) and another region (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue ′ (H) that satisfies the following formula (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the formula (1) is ( H2).

10 <saturation (S) <175;
Hue '(H) = Hue (H) +60 (when 0 ≦ hue (H) <300);
Hue ′ (H) = Hue (H) −300 (when 300 ≦ Hue (H) <360)
Luminance (Y) = R × 0.30 + G × 0.59 + B × 0.11 (A)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)
Therefore, the number of classes in the second image is 4 × 7 = 28. Moreover, it has at least three classes (v1, v2, v3) within 10% of the maximum brightness value (255). In addition, it is also possible to use lightness (V) in Formula (A) and Formula (1).

  When a two-dimensional histogram is created in step S35, the first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of brightness and hue to the total number of pixels (N × M) Is calculated (step S36), and the second occupancy rate calculation process ends. That is, step S35 and step S36 function as a second occupation rate calculation step.

  Assuming that the second occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the second occupancy in each class is expressed as shown in Table 2.

Note that a three-dimensional histogram may be created by classifying each pixel into a class consisting of a distance from the outer edge of the screen, brightness, and hue, and calculating the cumulative number of pixels for each class. Hereinafter, a method using a two-dimensional histogram is adopted.
<Indicator calculation process 1>
Next, the index calculation process (step S22 in FIG. 14) will be described with reference to the flowchart in FIG.

  First, in step S40, which is the second index calculation step, the first occupancy calculated for each class in the occupancy calculation processing is multiplied by a coefficient (second coefficient) set in advance according to the shooting conditions. Then, by taking the sum, the index 2 for specifying the shooting scene is calculated. The index 2 indicates a difference in contrast between the center and the outside of the screen of the image data between the backlight with the main subject under and the image with the main subject over.

  Hereinafter, a method for calculating the index 2 will be described.

  Table 3 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 3 is a weighting coefficient that is multiplied by the first occupation ratio Qij of each class shown in Table 1, and is set in advance according to the shooting conditions.

  FIG. 19 shows the second coefficient in the screen areas n1 to n3 as a curve (coefficient curve) that continuously changes over the entire brightness.

  If the second coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 2 is defined as shown in Expression (2).

  Accordingly, the sum of the n1 to n3 regions is expressed by the following formulas (2-1) to (2-3).

Sum of n1 region = Q11 × 12 + Q21 × 10 + (omitted)... + Q71 × 0 (2-1)
Sum of n2 regions = Q12 × 5 + Q22 × 3 + (omitted)... + Q72 × 0 (2-2)
Sum of n3 regions = Q13 × (−1) + Q23 × (−4) + (omitted)... + Q73 × (−8)
(2-3)
The index 2 is defined as in Expression (3) using the sum of the n1 to n3 regions shown in Expressions (2-1) to (2-3).

Index 2 = sum of n1 regions + sum of n2 regions + sum of n3 regions + 0.7 (3)
The index 2 is calculated on the basis of the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the second image, so that the shooting scene of the monochrome image as well as the color image is discriminated. It is also effective.

  Further, for example, according to the focus detection area detected by a known method, the weight of the second coefficient multiplied by the second occupancy calculated from the distance from the outer edge of the screen and the predetermined class of brightness is set. By changing, it is possible to calculate an index for discriminating a scene with higher accuracy.

  When the index 2 is calculated, the third occupancy calculated in step S41, which is the third index calculation step, is set to the second occupancy calculated for each class in the occupancy ratio calculation process according to the shooting conditions. By multiplying the coefficient (coefficient different from the second coefficient) and taking the sum, the index 3 for specifying the shooting scene is calculated. The index 3 is an index representing the degree of overshoot of the main subject, and is for separating only the image that should be determined as “the main subject is over” from other shooting scenes.

  Next, in step S42, which is a fourth index calculation step, a coefficient (fourth coefficient) different from the third coefficient preset in accordance with the shooting conditions is added to the second occupancy rate calculated for each class. ) To obtain the sum, the index 4 for specifying the shooting scene is calculated. The index 4 is an index that collectively represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness. Only images that should be determined as “backlight” and “main subject is under” are taken from other shooting scenes. It is for separation.

  Hereinafter, the calculation method of the index 3 and the index 4 will be described in detail.

  Table 4 shows the third coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 4 is a weighting coefficient by which the second occupation ratio Rij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 4, a positive (+) coefficient is used for the second occupancy calculated from the region distributed in the skin color hue region (H1) of high brightness (v6), and the other hue is blue. A negative (−) coefficient is used for the second occupation ratio calculated from the hue region. FIG. 20 shows a curve (coefficient curve) in which the third coefficient in the skin color region (H1) and the third coefficient in the other region (green hue region (H3)) continuously change over the entire brightness. It is shown. According to Table 4 and FIG. 20, in the region of high brightness (V = 77 to 150), the sign of the third coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the third coefficient in (H3)) is negative (−), and it can be seen that the signs of both are different.

  When the third coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 3 is defined as in Expression (4).

  Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).

Sum of H1 regions = R11 × 0 + R21 × 0 + (omitted)... + R71 × (−8) (4-1)
Sum of H2 regions = R12 × (−2) + R22 × (−1) + (omitted)
... + R72 x (-10) (4-2)
Sum of H3 regions = R13 × 5 + R23 × (−2) + (omitted)... + R73 × (−12)
(4-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)... + R74 × (−12)
(4-4)
The index 3 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).

Index 3 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.5 (5)
Table 5 shows the fourth coefficient necessary for calculating the index 4 by class. The coefficient of each class shown in Table 5 is a weighting coefficient that is multiplied by the second occupancy rate Rij of each class shown in Table 2, and is set in advance according to the shooting conditions.

  According to Table 5, a negative (−) coefficient is used for the occupancy calculated from the regions (v4, v5) distributed in the intermediate brightness of the flesh color hue region (H1), and the flesh color hue range (H1) is low. A coefficient 0 is used for the occupation ratio calculated from the brightness (shadow) region (v2, v3). FIG. 21 shows the fourth coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 5 and FIG. 21, the sign of the fourth coefficient of the intermediate lightness region having a lightness value of 25 to 150 in the flesh color hue region is negative (−), and the low lightness (shadow) region having a lightness value of 6 to 24. The fourth coefficient is 0, and it can be seen that there is a large difference between the coefficients in both regions.

  When the fourth coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 4 is defined as in Expression (6).

  Therefore, the sum of the H1 to H4 regions is expressed as the following formulas (6-1) to (6-4).

Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)... + R71 × 2 (6-1)
Sum of H2 region = R12 × (−2) + R22 × (−1) + (omitted)... + R72 × 2
(6-2)
Sum of H3 regions = R13 × 2 + R23 × 1 + (omitted)... + R73 × 3 (6-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)... + R74 × 3 (6-4)
The index 4 is defined as in Expression (7) using the sum of the H1 to H4 regions shown in Expressions (6-1) to (6-4).

Index 4 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.7 (7)
Since the index 3 and the index 4 are calculated on the basis of the brightness and hue distribution amount of the second image, they are effective in determining a shooting scene when the image is a color image.

  When the indices 2 to 4 are calculated, the average brightness value of the skin color at the screen center of the second image, the maximum brightness value, and the average brightness value of the second image are obtained in step S43, which is the fifth index calculation step. The index 5 for specifying the shooting scene is calculated by multiplying each of the difference values by a coefficient preset in accordance with the shooting conditions.

  Hereinafter, the calculation process of the index 5 will be described in detail with reference to the flowchart of FIG.

  First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the second image using Expression (A). Next, the average luminance value x1 of the skin color area at the center of the screen of the second image is calculated (step S50). Here, the center of the screen is an area constituted by, for example, the area n3 shown in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the second image is calculated (maximum luminance value−average luminance value) (step S51).

  Next, the standard deviation x3 of the luminance of the second image is calculated (step S52), and the average luminance value x4 at the center of the screen is calculated (step S53). Next, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the second image and the average luminance value Yskin_ave of the skin color area is calculated (step S54). This comparison value x5 is expressed as the following formula (8-1).

x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave
(8-1)
Next, the index 5 is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fifth coefficient set in advance according to the shooting conditions to obtain a sum (step 5). S55), the index 5 calculation process ends. The index 5 is defined as the following formula (8-2).

Index 5 = 0.05 × x1 + 1.41 × x2 + (− 0.01) × x3 + (− 0.01)
× x4 + 0.01 × x5−5.34 (8-2)
This index 5 has not only the compositional characteristics of the screen of the second image but also the luminance histogram distribution information, and is particularly effective for discriminating a shooting scene from which the main subject is over and an under shooting scene.

  When the index 5 is calculated, in step S44 (see FIG. 18), which is the sixth index calculation step, the average luminance value of the skin color area in the center of the screen of the indexes 2, 3 and the second image is set to the shooting condition. Accordingly, the index 6 is calculated by multiplying the weighting factor set in advance.

  Further, in step S45, which is a seventh index calculation step, the average luminance value of the skin color area in the center of the screen of the indices 2, 4 and the second image is multiplied by a weighting factor set in advance according to the shooting conditions. Thus, the index 7 is calculated, and this index calculation process ends.

  Hereinafter, the calculation method of the index 6 and the index 7 will be described in detail.

  The average luminance value of the skin color area in the center of the screen of the second image is set as an index 9. Here, the center of the screen is, for example, an area composed of the areas n2 and n3 in FIG. At this time, the index 6 is defined as in Expression (9) using the index 3, index 2, and index 9, and the index 7 is defined as in Expression (10) using the index 4, index 2, and index 9. Defined.

Index 6 = 0.54 × Index 3 + 0.50 × Index 2 + 0.01 × Index 9−0.65 (9)
Index 7 = 0.83 × Index 4 + 0.23 × Index 2 + 0.01 × Index 9-1.17 (10)
Note that, as a method of calculating the average luminance value (for example, the overall average luminance value) in FIG. 22, a simple addition average value of the individual luminance data obtained from each light receiving unit of the imaging device 1 may be obtained, or imaging is performed. Similar to the center-weighted average metering that is often used as the metering method of the apparatus 1, the luminance data obtained from the light receiving unit near the center of the screen is highly weighted, and the luminance data obtained from the light receiving unit near the periphery of the screen is weighted. A method of obtaining the average value by lowering may be used. In addition, the luminance data obtained from the vicinity of the light receiving unit corresponding to the focus detection area is increased in weight, and the luminance data obtained from the light receiving unit distant from the focus detection position is reduced in weight to obtain an average value, etc. May be used.
<Scene discrimination 1>
Returning to FIG. 14, the scene discrimination will be described.

  When the indices 5 to 7 are calculated, the shooting scene is determined based on the values of these indices (step S23). Table 6 shows the contents of the shooting scene discrimination according to the values of the index 5, the index 6, and the index 7.

  FIG. 23 is a discrimination map representing the discrimination contents shown in Table 6 using the coordinate system of the indices 5-7.

(First index calculation process flow)
Next, the first index calculation process (step S10 in FIG. 13) in the imaging apparatus 1 will be described with reference to the flowchart in FIG.

  As shown in FIG. 13, the first index calculation process is performed following the first occupancy ratio calculation process (step S8 in FIG. 13). That is, index 1 is calculated based on the first occupancy rate.

  In the first occupancy ratio calculating step, as described in the description of the scene discrimination process, each pixel is first classified into a predetermined class consisting of a combination of the distance from the outer edge of the screen and the brightness, and for each classified class. A two-dimensional histogram is created by calculating the cumulative number of pixels.

  FIG. 16A shows three regions n1 to n3 divided according to the distance from the outer edge of the image screen. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. A region n3 is a central region of the second image. Here, it is preferable that n1 to n3 are divided so as to have substantially the same number of pixels. In the present embodiment, three divisions are used, but the present invention is not limited to this. In addition, the lightness (V) is 0-5 (v1), 6-12 (v2), 13-24 (v3), 25-76 (v4), 77-109 (v5), 110-149 ( v6), and is divided into seven classes of 150 to 255 (v7). As shown in FIG. 16, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  FIG. 16B shows a class composed of combinations of three regions n1 to n3 and brightness. As shown in FIG. 16B, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 3 × 7 = 21.

  As described above, when the two-dimensional histogram is created, the first occupation indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of the combination of the distance from the outer edge of the screen and the brightness to the total number of pixels. A rate is calculated.

  Therefore, returning to FIG. 24A, the subsequent first index calculation step (step S10 in FIG. 13) will be described.

  First, in FIG. 16B, the central portion of the screen is set as n3 and n2, and the high brightness portion is set as v7. Then, the class combining the combination of n2 in v7 or n3 in v7 is a class composed of a combination of the center of the screen and high brightness, and the first occupancy ratio for the class is Z1 as shown in the figure. And Z1 is acquired (step S60).

  In FIG. 16B, the outer edge of the screen is the n1 region, and the high brightness portion is the v7 region. Then, the class consisting of the combination of n1 in v7 is a class consisting of a combination of the outer edge of the screen and high brightness, and the first occupancy for that class is set to Z2 as shown in the figure, and Z2 is acquired. (Step S61).

  Z1 and Z2 are factors that indicate that as Z1 is larger and Z2 is smaller, there are more high-lightness portions in the center of the screen (that is, whiteout tends to occur) with respect to the entire screen. The index 1 may be calculated using only Z1, but by calculating the index 1 based on these two factors, it is possible to increase the prediction accuracy of occurrence of whiteout. Further, it is desirable to obtain the following factors of Z3 and Z4.

  Z3 is acquired by setting the maximum brightness of the flesh color pixels at the center (n3 and n2) of the screen as Z3 (step S62). Further, Z4 is acquired by setting the skin color pixel occupancy rate in the center of the screen and the class (the first occupancy rate is indicated by Z1) consisting of a combination of high brightness as Z4 (step S63). As the skin color information, the skin color condition used in the description of the occupation rate calculation processing of the flow of the scene discrimination processing, that is, the one satisfying the formulas (A) and (1) is also regarded as the skin color here. Description is omitted.

  Z4 indicates the size of the possibility that there is a person (particularly a face) in the center of the screen, and Z3 indicates how bright the face of the person is (that is, whether whiteout is likely to occur).

  Finally, in step S64, the whiteout degree (index 1) is calculated based on the factors of Z1, Z2, Z3, and Z4 acquired in steps S60 to S63. Thus, steps S60 to S64 function as a first index calculation process.

  The degree of overexposure predicts that when the gradation correction processing is performed depending on the shooting scene, the overexposure area on the screen will increase, or overexposure will occur in the main subject area such as a human face. , Which is an index for suppressing gradation processing, and is used to correct the setting of gradation processing conditions described later. As a result, the gradation processing conditions are set based only on the discrimination result of the scene discrimination processing step, and in particular, the occurrence of overexposure such as a human face is prevented and the gradation processing setting is corrected in a more appropriate direction. can do.

  The index 1, that is, the whiteness (W) is the factor obtained above, that is, the first occupancy Z1 in the center of the screen and the high brightness class, the first occupancy in the outer edge of the screen and the high brightness class. Using the rate Z2, the maximum brightness Z3 of the flesh color pixel in the class at the center of the screen, and the flesh color pixel occupancy rate Z4 in the Z1 occupancy rate, numerical values are obtained by linear combination of each factor. For example, the following formula is used.

Whiteness (W) = Z1 × a + Z2 × b + Z3 × c + Z4 × d + constant × e
“a”, “b”, “c”, “d”, “e”, and “f” are predetermined coefficients and may be experimentally determined in advance according to the numerical value of each factor. However, the first coefficient “a” and the eighth coefficient “b” have opposite signs. This is because Z1 and Z2 are factors that indicate that as Z1 is larger and Z2 is smaller, there are more high brightness portions in the center of the screen (that is, whiteout tends to occur) with respect to the entire screen.

  The correction of the gradation processing condition using the index 1 (the degree of whiteout (W)) thus obtained will be described in the following gradation processing condition setting flow.

(Flow of gradation processing condition setting)
Next, the gradation processing condition setting (step S11 in FIG. 13) for the image after the gamma conversion processing of the first image will be described with reference to the flowchart in FIG.

First, in accordance with the shooting scene determined in the scene determination processing step, a gradation adjustment method for the first image data after the gamma conversion process is determined in step S70, and a gradation adjustment parameter is calculated in step S71. In step S72, the gradation conversion amount corrected in accordance with these parameters is calculated. In step S73, the gradation processing condition (gradation conversion curve) is determined. In this embodiment, the case where both the gradation adjustment method and the gradation conversion amount are determined in steps S70 and S73 is shown, but only the gradation conversion amount may be used.
<Tone adjustment method selection>
When determining the gradation adjustment method in step S70, as shown in FIG. 25, the gradation adjustment method A (FIG. 25 (a)) is selected when the shooting scene is front light, and when it is back light, the floor is adjusted. When the tone adjustment method B (FIG. 25B) is selected and the main subject is over, the tone adjustment method C (FIG. 25C) is selected, and when it is under, the tone adjustment method B is selected. (FIG. 25B) is selected.

In the present embodiment, the method for determining the gradation conversion curve to be applied to the image after the gamma conversion of the first image has been described. However, when the gradation conversion process is performed before the gamma conversion, FIG. The gradation conversion is performed on a curve obtained by performing inverse conversion of gamma conversion on the 25 gradation conversion curves.
<Calculation of gradation adjustment parameters>
When the gradation adjustment method is determined, in step S71, parameters necessary for gradation adjustment are calculated based on each index calculated in the index calculation process. Hereinafter, a method for calculating the gradation adjustment parameter will be described. In the following description, it is assumed that the 8-bit captured image data is converted into 16 bits in advance, and the unit of the value of the captured image data is 16 bits.

  As parameters necessary for gradation adjustment (gradation adjustment parameters), the following parameters P1 to P9 are calculated.

P1: Average luminance of the entire photographing screen P2: Block division average luminance P3: Average luminance of the skin color area (H1) P4: Luminance correction value 1 = P1-P2
P5: Reproduction target correction value = luminance reproduction target value (30360) -P4
P6: Offset value 1 = P5-P1
P7: Key correction value P7 ': Key correction value 2
P8: Brightness correction value 2
P9: Offset value 2 = P5-P8-P1
Here, a method for calculating the parameter P2 (block division average luminance) will be described with reference to FIGS.

  First, in order to normalize image data, a CDF (cumulative density function) is created. Next, the maximum value and the minimum value are determined from the obtained CDF. The maximum value and the minimum value are obtained for each RGB. Here, the maximum value and the minimum value obtained for each RGB are Rmax, Rmin, Gmax, Gmin, Bmax, and Bmin, respectively.

  Next, normalized image data for any pixel (Rx, Gx, Bx) of the image data is calculated. Assuming that Rx normalization data in the R plane is Rpoint, Gx normalization data in the G plane is Gpoint, and Bx normalization data in the B plane is Bpoint, the normalization data Rpoint, Gpoint, and Bpoint are expressed by the following equation (11), respectively. ) To Expression (13).

Rpoint = {(Rx−Rmin) / (Rmax−Rmin)} × 65535
(11)
Gpoint = {(Gx−Gmin) / (Gmax−Gmin)} × 65535
(12)
Bpoint = {(Bx−Bmin) / (Bmax−Bmin)} × 65535
(13)
Next, the luminance Npoint of the pixel (Rx, Gx, Bx) is calculated by Expression (14).

Npoint = (Bpoint + Gpoint + Rpoint) / 3 (14)
FIG. 26A shows a frequency distribution (histogram) of luminance of RGB pixels before normalization. In FIG. 26A, the horizontal axis represents luminance, and the vertical axis represents pixel frequency. This histogram is created for each RGB. When the luminance histogram is created, normalization is performed for each plane with respect to the image data by Expressions (11) to (13). FIG. 26B shows a histogram of luminance calculated by the equation (14). Since the captured image data is normalized by 65535, each pixel takes an arbitrary value between the maximum value of 65535 and the minimum value of 0.

  When the luminance histogram shown in FIG. 26B is divided into blocks divided by a predetermined range, a frequency distribution as shown in FIG. 26C is obtained. In FIG. 26C, the horizontal axis is the block number (luminance), and the vertical axis is the frequency.

  Next, a process of deleting highlight and shadow areas is performed from the luminance histogram shown in FIG. This is because the average brightness is very high in white walls and snow scenes, and the average brightness is very low in dark scenes, so highlights and shadow areas adversely affect average brightness control. . Therefore, by limiting the highlight area and the shadow area of the luminance histogram shown in FIG. 26C, the influence of both areas is reduced. When the high luminance region (highlight region) and the low luminance region (shadow region) are deleted from the luminance histogram shown in FIG. 27A (or FIG. 26C), the result is as shown in FIG.

  Next, as shown in FIG. 27C, an area having a frequency greater than a predetermined threshold is deleted from the luminance histogram. This is because if there is a part having an extremely high frequency, the data in this part strongly affects the average luminance of the entire captured image, and thus erroneous correction is likely to occur. Therefore, as shown in FIG. 27C, the number of pixels equal to or greater than the threshold is limited in the luminance histogram. FIG. 27D is a luminance histogram after the pixel number limiting process is performed.

  Each block number of the luminance histogram (FIG. 27 (d)) obtained by deleting the high luminance region and the low luminance region from the normalized luminance histogram and further limiting the number of accumulated pixels, and the respective frequencies The parameter P2 is obtained by calculating the average luminance based on the above.

  The parameter P1 is an average value of luminance of the entire image data, and the parameter P3 is an average value of luminance of the skin color area (H1) in the image data. The key correction value of the parameter P7, the key correction value 2 of the parameter P7 ', and the luminance correction value 2 of the parameter P8 are defined as shown in Expression (15), Expression (16), and Expression (17), respectively.

P7 (key correction value) = {P3-((index 7/6) × 18000 + 22000)}
/24.78 (15)
P7 ′ (key correction value 2) = {P3-((index 5/6) × 10000 + 30000)}
/24.78 (16)
P8 (Luminance correction value 2) = (Index 6/6) × 17500 (17)
<Calculation of gradation conversion amount>
Returning to FIG. 24B, when the gradation adjustment parameter is calculated, in step S73, determination of the gradation conversion curve for the photographed image data based on the calculated gradation adjustment parameter, that is, gradation processing conditions. Is set. However, before determining the gradation conversion curve in step S73, the gradation conversion amount based on the index 1 is calculated in step S72, and the gradation adjustment is corrected.

  Specifically, the first gradation conversion amount X is selected from the gradation adjustment parameters according to the already determined scene, and the second gradation conversion amount Y is calculated based on the index 1 (step S72). The calculation of the second gradation conversion amount Y based on the index 1 will be described later.

  Next, a gradation conversion curve corresponding to the calculated second gradation conversion amount Y is selected (determined) from a plurality of gradation conversion curves set in advance corresponding to the already determined gradation adjustment method. (Step S73). Alternatively, the gradation conversion curve may be calculated based on the second gradation conversion amount Y.

When the gradation conversion curve is determined, the gradation processing condition setting ends.
<Tone conversion amount calculation method based on index 1>
Next, the gradation conversion amount correction processing corresponding to the index 1 (calculation of the gradation conversion amount in step S72) will be described. The selection of the first gradation conversion amount X from the gradation adjustment parameter corresponding to the shooting scene will be described later. Here, it is assumed that the first gradation conversion amount X has already been selected.

  As an index for correction, index 1 (out-of-whiteness (W)) has already been calculated in the first index calculation step. If the gradation conversion process is determined based only on the scene discrimination process result of the image in the scene discrimination process, an image that does not conform to the user's intention may be obtained.

  For example, in general image appreciation, the most noticeable part is a person's face, and it is necessary to make the person's face appropriate brightness in order to obtain image quality appropriate for appreciation. Therefore, it is necessary to correct brightly an image in which a person is photographed under. However, depending on the situation of the shooting scene, only the correction as described above may increase the area of the image that is whitened in the corrected image, or whiteout may occur in the main subject such as the face. In some cases, the brightness balance in the image may be lost.

  In such a case, it is better to conceal the automatic gradation correction by predicting the occurrence of whiteout.

  As a specific gradation conversion correction method, the second gradation conversion amount Y is calculated by the following equation (18) with respect to the selected first gradation conversion amount X.

When the gradation conversion amount X is in the direction of correcting brightly and W (index 1)> constant g (predetermined threshold), h is defined as a predetermined constant.
Y = Xh * (Wg) (when Xh * (Wg)> 0)
Y = 0 (when Xh × (Wg) ≦ 0) (18)
g is a threshold value of W (index 1), h is a coefficient representing the degree of reflecting W (index 1), and each is a predetermined constant.

  By appropriately determining the values of g and h, the degree of suppression of gradation conversion processing can be controlled. That is, the smaller the gradation conversion amount Y, the smaller the correction amount, and zero means no correction.

  For example, if the gradation conversion amount X is a direction to be corrected dark, the whiteout does not occur, so the gradation conversion amount X is not corrected and Y = X. Further, even when W (index 1) does not exceed the threshold ((W−g) <0), correction is not suppressed.

  Further, even when the gradation conversion amount X is in the direction of correcting brightly and W> g, all Y = 0 are set to Y = 0, that is, the gradation process is not performed.

  The values of g and h in the above equation (18) may be set so that an appropriate correction amount is obtained depending on when an actual whiteout occurrence scene occurs or does not occur.

As described above, the second gradation conversion amount Y suppressed from the selected first gradation conversion amount X is calculated based on each condition, and the next gradation processing condition is determined.
<Setting gradation processing conditions for each scene>
Hereinafter, a method for determining a gradation conversion curve (determination of gradation processing conditions in step S73) for each shooting scene (light source condition and exposure condition) based on the scene determination processing result will be described.

<For direct light>
If the photographic scene is front light, the parameter P6 is selected as the first gradation conversion amount X. Since the parameter P6 = P1-P5, X = P6 means that if the second gradation conversion amount Y = X, offset correction (parallel shift of 8-bit value) for matching P1 with P5 is performed as follows: (19).

RGB value of output image = RGB value of input image + Y (19)
However, if Y is not X but is controlled by the shooting conditions, the correction amount is smaller than that when Y = P6.

  Therefore, when the photographic scene is front light, a gradation conversion curve corresponding to Expression (19) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (19).

<Backlight>
When the shooting scene is backlit, the parameter P7 (key correction value) is selected as the first gradation conversion amount X. Therefore, based on the second gradation conversion amount Y calculated from X, the gradation conversion curve corresponding to Y shown in Expression (18) is selected from the plurality of gradation conversion curves shown in FIG. Is done. A specific example of the gradation conversion curve in FIG. 25B is shown in FIG. The correspondence between the Y value and the selected gradation conversion curve is shown below.

When −50 <Y <+ 50 → L3
When + 50 ≦ Y <+ 150 → L4
When + 150 ≦ Y <+ 250 → L5
When −150 <Y ≦ −50 → L2
When −250 <Y ≦ −150 → L1
When the shooting scene is backlit, it is preferable to perform a dodging process together with the gradation conversion process. In this case, it is desirable to adjust the degree of dodging processing according to the index 6 indicating the backlight intensity.

<In case of under>
When the shooting scene is under, the parameter P7 ′ (key correction value 2) is selected as the first gradation conversion amount X. Therefore, based on the second gradation conversion amount Y calculated from X, the gradation conversion curve corresponding to Y shown in Expression (18) is selected from the plurality of gradation conversion curves shown in FIG. Is done. Specifically, the gradation conversion curve corresponding to the Y value is selected from the gradation conversion curves shown in FIG. 28 in the same manner as the method for selecting the gradation conversion curve when the shooting scene is backlit. When the shooting scene is under, the dodging process as shown in the case of backlight is not performed.

<When the main subject is over>
If the main subject is over, the parameter P9 is selected as the first gradation conversion amount X. Therefore, based on the second gradation conversion amount Y calculated from X, offset correction (parallel shift of 8-bit value) is performed by Expression (20).

RGB value of output image = RGB value of input image + Y (20)
Therefore, when the main subject is over, a gradation conversion curve corresponding to Expression (20) is selected from the plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (20). When the value Y in Expression (20) exceeds a preset value α, a curve corresponding to the key correction value Y−α is selected from the curves L1 to L5 shown in FIG. The

  In the present embodiment bear, when the gradation conversion process is actually performed on the captured image data, the above-described gradation conversion process condition is changed from 16 bits to 8 bits.

(Second Embodiment)
Next, the configuration of the second embodiment (image processing apparatus) according to the present invention will be described.

  FIG. 29 shows an example of the external configuration of the image processing apparatus 2 according to the present embodiment. FIG. 30 is a block diagram illustrating a schematic configuration of the image processing apparatus 2.

(External configuration of image processing apparatus 2)
In FIG. 29, the image processing apparatus 2 includes a loading unit 13 that loads a photosensitive material on one side surface of the housing 11. The housing 11 includes an exposure processing unit 86 that exposes the photosensitive material and a print creation unit 16 that develops the exposed photosensitive material to create a print. On the other side surface of the housing 11, a tray 17 for discharging the print created by the print creation unit 16 is provided.

  At the top of the housing 11, a CRT (Cathode Ray Tube) 85 as a display device, a film scanner unit 81 that is a device for reading a transparent document, a reflective document input device 82, and an operation unit 71 are provided. The housing 11 further includes an image reading unit 14 that can read image information recorded on various digital recording media, and an image writing unit 15 that can write image signals on various digital recording media. Includes a control unit 72 for controlling these units. The image reading unit 14 includes an image transfer unit 83 described later, and the image writing unit 15 includes an image transfer unit 88 described later.

  Not all of these components are required, and it is not necessary to have an integrally provided structure. Any one or more may be separated from the image processing apparatus 2 or connected as an external apparatus.

(Configuration of the image processing apparatus 2)
FIG. 30 shows a schematic configuration of the image processing apparatus 2 according to the embodiment of the present invention. As shown in FIG. 30, the image processing apparatus 2 includes an operation unit 71, a control unit 72, an image processing unit 10, a film scan data processing unit 73, a reflection original scan data processing unit 74, an image data format decoding processing unit 75, a CRT. The unique processing unit 76, the printer specific processing unit A 77, the printer specific processing unit B 78, and the image data format creation processing unit 79 are configured.

  The film scan data processing unit 73 performs a calibration operation specific to the film scanner unit 81, negative / positive reversal (in the case of a negative document), dust scratch removal, contrast adjustment, granular noise removal, and sharpness for the image data input from the film scanner unit 81. Processing such as conversion enhancement is performed, and processed image data is output to the image processing unit 10. Further, the film size, the negative / positive type, the information relating to the main subject optically or magnetically recorded on the film, the information relating to the photographing conditions (for example, the description information content of APS) and the like are also output to the image processing unit 10.

  The reflection document scan data processing unit 74 performs a calibration operation unique to the reflection document input device 82, negative / positive reversal (in the case of a negative document), dust flaw removal, contrast adjustment, and noise removal for the image data input from the reflection document input device 82. Then, processing such as sharpening enhancement is performed, and processed image data is output to the image processing unit 10.

  The image data format decoding processing unit 75 restores the compression code as necessary according to the data format of the image data input from the image transfer means 83 and / or the communication means (input) 84, and the color data. Are converted into a data format suitable for computation in the image processing unit 10 and output to the image processing unit 10.

  In addition, when an output image condition is specified from any of the operation unit 71, the communication unit (input) 84, and the image transfer unit 83, the image data format decoding processing unit 75 detects the specified information, and Output to the processing unit 10. Information about the conditions of the output image specified by the image transfer unit 83 is embedded in the header information and tag information of the image data acquired by the image transfer unit 83.

  Based on an instruction from the operation unit 71 or the control unit 72, the image processing unit 10 applies image data received from any one of the film scanner unit 81, the reflection original input device 82, the image transfer unit 83, and the communication unit (input) 84. On the other hand, image processing described later (see FIGS. 14, 15, 18 and 24) is performed to generate digital image data for image formation optimized for viewing on the output medium, and a CRT specific processing unit 76, output to any one of the printer specific processing unit A77, the printer specific processing unit B78, and the image data format creation processing unit 79.

  The internal configuration of the image processing unit 10 will be described later with reference to FIG.

  The CRT specific processing unit 76 performs processing such as changing the number of pixels and color matching on the image data input from the image processing unit 10 as necessary, and combines it with information that needs to be displayed, such as control information. Are output to the CRT 85.

  The printer-specific processing unit A77 performs printer-specific calibration processing, color matching, and pixel number change processing as necessary, and outputs processed image data to the exposure processing unit 86.

  When an external printer 87 can be connected to the image processing apparatus 2 of the present invention, a printer specific processing unit B78 is provided for each printer apparatus to be connected. The printer-specific processing unit B78 performs printer-specific calibration processing, color matching, pixel number change, and the like, and outputs processed image data to the external printer 87.

  The image data format creation processing unit 79 converts the image data input from the image processing unit 10 into various general-purpose image formats typified by JPEG, TIFF, bitmap, and the like as necessary. The completed image data is output to the image transport unit 88 and the communication means (output) 89.

  Note that the film scan data processing unit 73, the reflection original scan data processing unit 74, the image data format decoding processing unit 75, the image processing unit 10, the CRT specific processing unit 76, the printer specific processing unit A77, and the printer specific processing shown in FIG. The section B78 and the image data format creation processing section 79 are sections provided to assist understanding of the function of the image processing apparatus 2, and are not necessarily realized as physically independent devices. It may be realized as a type of software processing performed by a single CPU.

(Internal configuration of the image processing unit 10)
FIG. 11 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls the gradation correction operation based on scene discrimination for the input image of the image processing apparatus 2, and performs a series of processes such as image acquisition, scene discrimination, and gradation processing condition setting. Execute. In this embodiment, the input image is taken image data, and the internal configuration of the image processing unit 10 will be described with reference to FIG.

  As shown in FIG. 11, the image processing unit 10 includes a first image acquisition unit 101, a second image acquisition unit 102, a first occupancy rate calculation unit 103a, a second occupancy rate calculation unit 103b, an index calculation unit 104, and a scene determination. A unit 105, a first index calculation unit 106, a gradation processing condition setting unit 107, and a gradation conversion processing unit 108.

  The first image acquisition unit 101 acquires input image data from each input data processing unit. Since the input image is photographed image data, for example, the photographed image data input to the image data format decoding processing unit 75 via the image transfer means 83 or the communication means 84 is compressed as necessary according to the data format of the image data. Processing such as code restoration and conversion of the data representation method is performed, and the image processing unit 10 acquires the first image. The photographed image data is converted into a non-linear image such as a JPEG image format and recorded during normal photographing. Here, it is assumed that the image converted into the non-linear image is acquired as the first image. Of course, a linear image (RAW image) at the time of shooting may be acquired.

  Further, the first image acquisition unit 101 reads, for example, Exif information recorded at the time of shooting from the input image, and also acquires the shooting conditions at the time of shooting. The acquired image data of the first image and shooting conditions are held in a memory (not shown) in the image processing unit 10.

  The second image acquisition unit 102 functions as a second image acquisition unit as in the case of the first embodiment. The same applies to acquiring a substantially reduced second image from the acquired first image, and a description thereof will be omitted.

  Hereinafter, description of the components having the same functions as those of the first embodiment (in the case of the imaging apparatus) will be simplified.

  The first occupancy rate calculation unit 103a functions as a first occupancy rate calculation unit, and based on the color information acquired for the image data, the image is a predetermined class comprising a combination of the distance from the outer edge of the screen and the brightness. And a first occupation ratio indicating a ratio of pixels belonging to each class to the entire image is calculated for each classified class.

  The second occupancy rate calculation unit 103b functions as a second occupancy rate calculation unit, similarly classifies images into a predetermined class composed of combinations of brightness and hue, and calculates a second occupancy ratio for each class. To do.

  The occupancy rate calculation processing executed in the first occupancy rate calculation unit 103a and the second occupancy rate calculation unit 103b will be described later with reference to FIG.

  The index calculation unit 104 functions as a second index calculation unit. That is, the index 2 is calculated from the calculated first occupation rate.

  The index calculation unit 104 also functions as a third index calculation unit and a fourth index calculation unit. That is, index 3 and index 4 are calculated from the calculated second occupancy rate.

  Furthermore, the index calculation unit 104 also functions as a fifth index calculation unit. The index 5 is calculated from the average luminance value at the center of the screen of the second image and the difference value between the maximum luminance value and the average luminance value.

  In addition, the index calculation unit 104 functions as a sixth index calculation unit. That is, a new index 6 is calculated from the average luminance value (index 9) of the skin color area in the center of the screen of the second image and the indices 2 and 3.

  The index calculation unit 104 functions as a seventh index calculation unit. A new index 7 is calculated from the average luminance value and the indices 2 and 4.

  The difference between the index calculation process executed in the index calculation unit 104 and the case of the first embodiment will be described in detail later with reference to FIG.

  The scene determination unit 105 determines the shooting scene of the first image based on each index calculated by the index calculation unit 104. That is, it functions as a scene discrimination means. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes over and under degrees.

  As described above, the second occupancy ratio calculation unit, the index calculation unit, and the scene determination unit function as the scene determination processing unit as in the case of the first embodiment.

  The first index calculation unit 106 functions as a first index calculation unit. The index 1 is calculated by multiplying the first occupancy calculated by the first occupancy calculator 103a by a preset coefficient as in the case of the first embodiment.

  The gradation processing condition setting unit 107 functions as a gradation processing condition setting unit. The gradation processing conditions are set based on the shooting scene determined by the scene determination unit 105. The setting method is the same as in the case of the first embodiment.

  The gradation conversion processing unit 108 executes gradation conversion processing for the first image in accordance with the gradation processing conditions set in the gradation processing condition setting unit 107.

(Operation flow of the image processing apparatus 2)
Next, the operation of the image processing apparatus 2 in the present embodiment will be described.

  Hereinafter, a captured image is targeted, and the captured object is referred to as a “main subject”.

  First, the overall flow of processing executed by the image processing apparatus 2 will be described with reference to the flowchart of FIG. These processes can be executed by a computer using a program. Each process described below can be similarly described as each function of the image processing program.

  First, when an input operation to the image processing apparatus 2 is performed, input processing is performed in any of the input processing units 73 to 75 (step S90). The captured image is sent to the image processing unit 10 through the processing in the image data format decoding processing unit 75. In step S91, the image processing unit 10 acquires a first image as a captured image. It is assumed that the first image is a non-linear image and is a JPEG image.

  Also, the Exif information added to the acquired first image is read, and the shooting conditions are captured. The acquired shooting conditions are held in the memory.

  Thereafter, the image data of the first image is acquired as a divided image composed of a plurality of divided regions, that is, a second image (step S92). The second image is an image obtained by reducing the size of the first image as in the first embodiment, and step S92 functions as a second image acquisition step.

  In step S93, which is the first occupancy rate calculation step, the first occupancy rate is calculated for the second image. However, the image data does not necessarily have to be the second image. The image data of the first image acquired by shooting may be used as in the case of the first embodiment.

  In step S95, the index 1 is calculated based on the first occupation ratio calculated in step S93 as in the case of the first embodiment. That is, step S95 functions as a first index calculation step. The index 1 is used later to correct the gradation processing condition set by the scene discrimination result. The first index calculation process has already been described in the first embodiment with reference to FIG.

  Next, in step S94, which is a scene determination processing step, a scene determination process for determining a shooting scene is performed based on the acquired image data of the second image. The scene discrimination process in step S94 will be described later with reference to FIG. 14 where it differs from the case of the first embodiment.

  Next, in step S96 which is a gradation processing condition setting step, it is necessary for the gradation conversion processing of the first image based on each index obtained in the photographing scene discrimination processing in step S94 and the judgment result of the photographing scene. A gradation processing condition setting process for setting various conditions is performed. Further, the adjustment of the gradation conversion amount by the index 1 calculated in the first index calculation step is performed here as in the case of the first embodiment.

  In step S97, gradation conversion processing is performed on the image data of the first image, which is a captured image, based on the gradation processing conditions set in step S96. The gradation conversion in step S97 corrects the influence on the gradation due to the light source condition of the photographic scene, and the gradation conversion processing is performed based on the condition set in step S96 based on the scene discrimination result in step S94.

  Next, other image processing (noise processing, sharpness processing, etc.) is appropriately performed (step S98). Thereafter, the image data is processed and output by the output processing units 76 to 79. Alternatively, it is recorded on a storage medium for storage (such as an SD memory card or a multimedia card (MMC)) (step S99). When the output or recording ends, the operation in the image processing apparatus 2 ends.

(Flow of scene discrimination process 2)
Next, referring to the flowchart in FIG. 14 and FIGS. 15, 18, 22, and 32 to 35, the first occupancy rate calculation process (step S93 in FIG. 31) and the scene determination process 2 (in FIG. 31) in the image processing apparatus 2. The 31 step S94) will be described. In the following description, the scene determination process including the first occupancy rate calculation process will be referred to. Further, differences from the first embodiment (imaging device 1) will be described, and the same portions will be omitted as appropriate.

  As in the first embodiment, the scene determination process includes a first occupancy rate calculation process (step S20), a second occupancy rate calculation process (step S21), and an index calculation process (step S22), as shown in FIG. ) And scene discrimination (step S23). Each process shown in FIG. 14 will be described below with reference to the above drawings as appropriate.

<First occupation rate calculation process 2>
First, in step S20, a first occupancy rate calculation process is performed. The first occupancy rate calculation process will be described with reference to the flowchart in FIG.

  First, in step S30 in FIG. 15A, color space conversion processing is performed. The processing contents are the same as those in the first embodiment, and a description thereof will be omitted.

  The two-dimensional histogram creation in step S31 is the same as in the first embodiment. However, the division according to the distance from the outer edge is not three areas but four areas.

  FIG. 32A shows four regions n1 to n4 divided according to the distance from the outer edge of the screen of the second image. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. The region n3 is a region further inside the region n2, and the region n4 is a central region of the second image. In step S31, the brightness is divided into seven areas v1 to v7 as described above. Therefore, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 4 × 7 = 28.

  Step S32, together with step S31, functions as a first occupancy rate calculation step, and the first occupancy rate is calculated as in the case of the first embodiment.

  If the first occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the first occupancy in each class is expressed as shown in Table 7.

<Second Occupancy Calculation Processing 2>
Next, in step S21 of FIG. 14, a second occupancy rate calculation process is performed. The second occupancy rate calculation process will be described with reference to the flowchart in FIG.

  In step S35 of FIG. 15B, each pixel of the second image is classified into a predetermined class composed of a combination of hue and brightness, and a cumulative pixel count is calculated for each classified class, thereby obtaining a two-dimensional histogram. Is created.

  The lightness division is different from that in the first embodiment. This is due to the difference between a linear image and a non-linear image. In step S35, the lightness (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200- It is divided into seven classes of 224 (v6) and 225 to 255 (v7).

  Hue (H) is a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region having a hue value of 161 to 250. It is divided into four areas (H4) and a red hue area (H5). These are the same as in the case of the first embodiment. Similarly, the skin color hue region is further divided into a skin color region (H1) and another region (H2).

  Step S36, together with step S35, functions as a second occupancy rate calculation step, and the second occupancy rate is calculated as in the case of the first embodiment.

  If the second occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the second occupancy in each class is expressed as shown in Table 8.

<Indicator calculation process 2>
Next, the index calculation process (step S22 in FIG. 14) will be described with reference to the flowchart in FIG.

  First, in step S40, which is the second index calculation step, index 2 is calculated by multiplying the first occupancy by a preset coefficient (second coefficient) and taking the sum. The index 2 indicates a difference in contrast between the center and the outside of the screen of the image data between the backlight with the main subject under and the image with the main subject over.

  Hereinafter, a method for calculating the index 2 will be described.

  Table 9 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 9 is a weighting coefficient by which the first occupancy rate Qij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 33 shows the second coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

  If the second coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 2 is defined as shown in Expression (2).

  Accordingly, the sum of the n1 to n4 regions is represented by the following formulas (2-5) to (2-8).

Sum of n1 region = Q11 × 40.1 + Q21 × 37 + (omitted)... + Q71 × 22
(2-5)
Sum of n2 regions = Q12 × (−14.8) + Q22 × (−10.5) + (omitted)
... + Q72 × 0 (2-6)
Sum of n3 regions = Q13 × 24.6 + Q23 × 12.1 + (omitted) ... + Q73 × 10.1
(2-7)
Sum of n4 regions = Q14 × 1.55 + Q24 × (−32.9) + (omitted)
... + Q74 × (−52.2) (2-8)
The index 2 is defined as in Expression (21) using the sum of the n1 to n4 regions shown in Expressions (2-5) to (2-8).

Index 2 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions + 12.6201
(21)
The index 2 is calculated on the basis of the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the second image, so that the shooting scene of the monochrome image as well as the color image is discriminated. It is also effective.

  When the index 2 is calculated, the index 3 is calculated by multiplying the second occupancy by a preset third coefficient in step S41, which is the third index calculating step, and taking the sum. Next, in step S42, which is a fourth index calculating step, the index 4 is calculated by multiplying the second occupancy by a preset fourth coefficient to obtain the sum. This is the same as the case of the embodiment.

  The index 3 is an index representing the degree of overshoot of the main subject, and is for separating only the image that should be determined as “the main subject is over” from other shooting scenes. The index 4 is an index that collectively represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness. Only images that should be determined as “backlight” and “main subject is under” are taken from other shooting scenes. It is for separation.

  Hereinafter, the calculation method of the index 3 and the index 4 will be described.

  Table 10 shows the first coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 10 is a weighting coefficient that is multiplied by the first occupation ratio Rij of each class shown in Table 2, and is set in advance according to the shooting conditions.

  According to Table 10, a positive (+) coefficient is used for the second occupancy calculated from the area distributed in the flesh color hue area (H1) of high brightness (v5, v6), and other hues are used. A negative (−) coefficient is used for the second occupation ratio calculated from a certain blue hue region. FIG. 34 shows a curve (coefficient curve) in which the third coefficient in the skin color area (H1) and the third coefficient in the other areas (green hue area (H3)) continuously change over the entire brightness. It is shown. According to Table 10 and FIG. 34, in the region of high brightness (V = 170 to 224), the sign of the third coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the third coefficient in (H3)) is negative (−), and it can be seen that the signs of both are different.

  When the third coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 3 is defined as in Expression (4).

  Accordingly, the sum of the H1 to H4 regions is represented by the following formulas (4-5) to (4-8).

Sum of H1 region = R11 × (−44) + R21 × (−16) + (omitted)
... + R71 × (-11.3) (4-5)
Sum of H2 regions = R12 × 0 + R22 × 8.6 + (omitted)... + R72 × (−11.1)
(4-6)
Sum of H3 regions = R13 × 0 + R23 × (−6.3) + (omitted)... + R73 × (−10)
(4-7)
H4 region sum = R14 × 0 + R24 × (−1.8) + (omitted)
... + R74 × (-14.6) (4-8)
The index 3 is defined as in Expression (22) using the sum of the H1 to H4 regions shown in Expressions (4-5) to (4-8).

Index 3 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 4.424
(22)
Table 11 shows the fourth coefficient necessary for calculating the index 4 by class. The coefficient of each class shown in Table 11 is a weighting coefficient by which the second occupation ratio Rij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 11, a negative (−) coefficient is used for the occupation ratio calculated from the region (v4) distributed in the intermediate brightness of the flesh color hue region (H1), and the low lightness of the flesh color hue range (H1) ( A positive (+) coefficient is used for the occupation ratio calculated from the shadow area (v2, v3). FIG. 35 shows the fourth coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 11 and FIG. 35, the sign of the fourth coefficient of the low brightness (shadow) area of the flesh color hue area having a brightness value of 26 to 84 is positive (+), and the intermediate brightness of the brightness value of 85 to 169 is shown. It can be seen that the sign of the fourth coefficient in the area and the other areas is negative (−), and the sign of the coefficient in both areas is different.

  When the fourth coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 4 is defined as in Expression (6).

  Accordingly, the sum of the H1 to H4 regions is represented by the following formulas (6-5) to (6-8).

H1 region sum = R11 × (−27) + R21 × 4.5 + (omitted)
... + R71 × (−24) (6-5)
Sum of H2 regions = R12 × 0 + R22 × 4.7 + (omitted)... + R72 × (−8.5)
(6-6)
Sum of H3 regions = R13 × 0 + R23 × 0 + (omitted)... + R73 × 0 (6-7)
H4 region sum = R14 × 0 + R24 × (−5.1) + (omitted)
... + R74 × 7.2 (6-8)
The index 4 is defined as in Expression (23) using the sum of the H1 to H4 regions shown in Expressions (6-5) to (6-8).

Index 4 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.554
(23)
Since the index 3 and the index 4 are calculated on the basis of the brightness and hue distribution amount of the second image, they are effective in determining a shooting scene when the image is a color image.

  When the indices 3 to 4 are calculated, in step S43, which is a fifth index calculating step, the average luminance value of the skin color at the screen center portion of the second image and the difference value between the maximum luminance value and the average luminance value are calculated. The index 5 is calculated by multiplying each by a preset coefficient.

  Hereinafter, the calculation process of the index 5 will be described with reference to the flowchart of FIG.

  First, from step S50 to step S54, the average luminance value x1 of the skin color region in the center of the screen of the second image, the difference value x2 between the maximum luminance value and the average luminance value of the second image = maximum luminance value-average The luminance value, the standard deviation x3 of the luminance of the second image, the average luminance value x4 in the center of the screen, the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color region in the second image, and the average of the skin color region The process of calculating the comparison value x5 with the luminance value Yskin_ave is the same as in the case of the first embodiment, and a description thereof will be omitted.

  Next, the index 5 is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fifth coefficient set in advance according to the shooting conditions to obtain a sum (step 5). S55), the index 5 calculation process ends. The index 5 is defined as the following formula (8-3).

Index 5 = 0.06 × x1 + 1.13 × x2 + 0.02 × x3 + (− 0.01)
* X4 + 0.03 * x5-6.50 (8-3)
This index 5 has not only the compositional characteristics of the screen of the second image but also the luminance histogram distribution information, and is particularly effective for discriminating a shooting scene in which the main subject is over and an under shooting scene.

  When the index 5 is calculated, in step S44, which is the sixth index calculation step, the average brightness value of the skin color area in the center of the screen of the indexes 2, 3 and the second image is preset according to the shooting conditions. The index 6 is calculated by multiplying the weighting factors.

  Further, in step S45, which is a seventh index calculation step, the average luminance value of the skin color area in the center of the screen of the indices 2, 4 and the second image is multiplied by a weighting factor set in advance according to the shooting conditions. Thus, the index 7 is calculated, and this index calculation process ends.

  Hereinafter, a method for calculating the index 6 and the index 7 will be described.

  The average luminance value of the skin color area in the center of the screen of the second image is set as an index 9. The center portion of the screen here is, for example, a region composed of the region n2, the region n3, and the region n4 in FIG. At this time, the index 6 is defined as in Expression (24) using the index 3, index 2, and index 9, and the index 7 is defined as in Expression (25) using the index 4, index 2, and index 9. Defined.

Index 6 = 0.46 × index 3 + 0.61 × index 2 + 0.01 × index 9−0.79 (24)
Index 7 = 0.58 × Index 4 + 0.18 × Index 2 + (− 0.03) × Index 9 + 3.34
(25)
<Scene discrimination 2>
Returning to step S23 (scene discrimination) in FIG.

  When the indices 5 to 7 are calculated, the photographic scene is determined based on the values of these indices. The scene determination method is the same as in the first embodiment, and the description thereof is omitted.

(First index calculation process flow)
Next, the first index calculation process (step S95 in FIG. 31) in the image processing apparatus 2 will be described with reference to the flowchart in FIG.

  As shown in FIG. 31, the first index calculation process is performed subsequent to the first occupancy ratio calculation process (step S93 in FIG. 31). That is, index 1 is calculated based on the first occupancy rate. Since the description of the first occupancy rate calculation step has been described above, it will be omitted. As described above, in the first occupancy ratio calculation, the division based on the distance from the outer edge of the screen is not four areas but four areas.

  A subsequent first index calculation step (step S95 in FIG. 31) will be described with reference to the flowchart in FIG.

  FIG. 32B shows a class composed of combinations of four regions n1 to n4 and brightness. First, in FIG. 32 (b), the central portion of the screen is the n4, n3, and n2 regions, and the high brightness portion is the v7 region. Then, the class combining the combination of n2 in v7, n3 in v7, or n4 in v7 is a class composed of a combination of the center of the screen and high brightness, and the class is the same as in the first embodiment. The first occupancy ratio is set to Z1 as shown in the figure, and Z1 is acquired (step S60).

  In FIG. 32B, the outer edge of the screen is the n1 region, and the high brightness portion is the v7 region. Then, the class consisting of the combination of n1 in v7 is a class consisting of a combination of the outer edge of the screen and high brightness, and the first occupancy rate for the class is shown in the figure as in the first embodiment. Z2 is acquired as described above (step S61).

  Z1 and Z2 are factors that indicate that, as Z1 is larger and Z2 is smaller, there are more high brightness portions in the center of the screen (that is, whiteout is likely to occur) with respect to the entire screen. It is desirable to acquire the following factors of Z3 and Z4 as in the case of the first embodiment.

  Z3 is obtained by setting the maximum brightness of the flesh color pixels in the center (n4, n3 and n2) of the screen as Z3 (step S62). Further, Z4 is acquired by setting the skin color pixel occupancy rate in the center of the screen and the class (the first occupancy rate is indicated by Z1) consisting of a combination of high brightness as Z4 (step S63).

  Finally, in step S64, the whiteout degree (index 1) is calculated based on the factors of Z1, Z2, Z3, and Z4 acquired in steps S60 to S63. Thus, steps S60 to S64 function as a first index calculation process.

  The degree of overexposure predicts that when the gradation correction processing is performed depending on the shooting scene, the overexposure area on the screen will increase, or overexposure will occur in the main subject area such as a human face. This is an index for suppressing gradation processing, and is used to correct the setting of gradation processing conditions described later, as in the case of the first embodiment.

  The index 1, that is, the whiteout degree (W), is configured as shown in the following equation, for example, using the factors acquired above.

Whiteness (W) = Z1 × a + Z2 × b + Z3 × c + Z4 × d + constant × e
“a”, “b”, “c”, “d”, “e”, and “f” are predetermined coefficients and may be experimentally determined in advance according to the numerical value of each factor. However, the first coefficient “a” and the eighth coefficient “b” have opposite signs.

  The correction of the gradation processing condition using the index 1 (the degree of whiteout (W)) thus obtained will be described in the following gradation processing condition setting flow.

(Flow of gradation processing condition setting)
Next, the gradation processing condition setting for the first image (step S96 in FIG. 31) is the same as that in the first embodiment, and is as described with reference to the flowchart in FIG. Therefore, explanation is omitted.

  As described above, according to the first embodiment and the second embodiment, when automatically setting the correction processing condition by determining the shooting scene of the shot image, the white at the time of image correction is set. As an element that predicts the occurrence of skipping, by calculating an index that indicates the occurrence of whiteout based on the distribution of the high brightness part at a position in the screen, and correcting the set correction processing conditions according to that index Further, it is possible to provide an imaging apparatus, an image processing apparatus, an image processing method, and an image processing program that can correct a direction in which whiteout is likely to occur, and that can obtain a more appropriate corrected image. .

  The embodiment of the present invention is not limited to the above-described embodiment, and various modified forms are also included in the scope thereof as long as the gist of the present invention is met.

It is a figure showing an example of appearance composition of imaging device 1 concerning an embodiment of the present invention. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 1. FIG. 2 is a block diagram illustrating an internal configuration of an imaging processing unit 20. FIG. It is a flow church which shows exposure amount control in an AE control part. It is a figure which shows a mode that an electric charge accumulate | stores in an image pick-up element in control of shutter speed. It is a flowchart which shows the control which adjusts the focus of the image in AF control part. It is a figure which shows the change of the contrast information accompanying the movement of an imaging optical system, and the mode of a focal point position detection. It is a flowchart which shows the process of the image data which performs the interpolation between pixels for every RGB color component in a pixel interpolation part. It is a flowchart which shows the process which automatically adjusts the white balance in a picked-up image in an AWB control part. It is a figure which shows the example of a gamma correction curve. 2 is a block diagram showing an internal configuration of an image processing unit 10. FIG. FIG. 4A is a diagram illustrating an example of a captured first image, and FIG. 4B is a diagram illustrating an example of a second image obtained by dividing the first image into a reduced image of M × N pixels. 3 is a flowchart showing an overall flow of processing executed in the imaging apparatus 1. 4 is a flowchart illustrating a scene determination process executed in the image processing unit 10. It is a flowchart which shows an occupation rate calculation process. It is the figure (a) which shows the area | regions n1-n3 determined according to the distance from the outer edge of a screen, and the figure (b) which shows the class which consists of the area | regions n1-n3 and brightness. It is a figure which shows the class which consists of brightness and hue. It is a flowchart which shows an index calculation process. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied for every area | region (n1-n3). It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied. It is a figure which shows the curve showing the 4th coefficient by which the 2nd occupation rate for calculating the parameter | index 4 is multiplied. It is a flowchart which shows the parameter | index 5 calculation process. It is a plot figure which shows the relationship between an imaging | photography scene (forward light, strobe, backlight, under) and the indexes 5-7. (A) is a flowchart which shows a 1st parameter | index calculation process. (B) is a flowchart showing gradation processing condition setting. It is a figure which shows the gradation conversion curve corresponding to each gradation adjustment method. It is a figure which shows the frequency distribution (histogram) (a) of a brightness | luminance, the normalized histogram (b), and the block-divided histogram (c). It is a figure ((a) and (b)) explaining deletion of the low-intensity area | region and high-intensity area | region from a brightness | luminance histogram, and a figure ((c) and (d)) explaining the restriction | limiting of a luminance frequency. It is a figure which shows the gradation conversion curve showing the gradation processing conditions in case a imaging | photography scene is backlight or under. It is a figure which shows the example of an external appearance structure of the image processing apparatus 2 which concerns on embodiment of this invention. 3 is a block diagram illustrating a configuration example of an image processing device 2. FIG. 3 is a flowchart showing an overall flow of processing executed in the image processing apparatus 2. It is the figure (a) which shows the area | region n1-n4 determined according to the distance from the outer edge of a screen, and the figure (b) which shows the class which consists of area | region n1-n4 and a brightness. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied for every area | region (n1-n4). It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied. It is a figure which shows the curve showing the 4th coefficient by which the 2nd occupation rate for calculating the parameter | index 4 is multiplied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging apparatus 2 Image processing apparatus 10 Image processing part 11 Case 13 Loading part 14 Image reading part 15 Image writing part 16 Print creation part 17 Tray 20 Imaging processing part 21 Case 22 Cross key 23 Shooting optical system 24 Flash 25 Finder 26 Power switch 27 Display unit 28 Release button 31 Processor 32 Memory 33 Image sensor 37 Image data output unit 38 Operation unit 41 Timing generator 42 Shutter unit 43 Aperture unit 44 Focus unit 51 AE control unit 52 AF control unit 53 Pixel interpolation unit 54 AWB control 55 Gamma Correction Unit 71 Operation Unit 72 Control Unit 73 Film Scan Data Processing Unit 74 Reflected Original Scan Data Processing Unit 75 Image Data Format Decoding Processing Unit 76 CRT Specific Processing Unit 77 Printer Specific Processing Part A
78 Printer Specific Processing Unit B
79 Image data format creation processing section 81 Film scanner section 82 Reflected document input device 83 Image transfer means 84 Communication means (input)
85 CRT
86 Exposure processing unit 87 External printer 88 Image transport unit 89 Communication means (output)
DESCRIPTION OF SYMBOLS 101 1st image acquisition part 102 2nd image acquisition part 103a 1st occupation rate calculation part 103b 2nd occupation rate calculation part 104 Index calculation part 105 Scene discrimination | determination part 106 1st index calculation part 107 Gradation processing condition setting part 108 Gradation Conversion processing section

Claims (20)

Scene discrimination processing means for performing discrimination processing of a shooting scene based on image data of an image acquired by shooting;
Gradation processing condition setting means for setting gradation processing conditions for the image based on the scene determination result obtained by the scene determination processing means;
A first occupation for classifying the image data into a class composed of a combination of a distance from the outer edge of the screen and a brightness, and calculating a first occupation ratio indicating a ratio of the entire image data for each classified class Rate calculation means;
Of the first occupancy ratios calculated by the first occupancy ratio calculating means, multiplying the occupancy ratio for the class composed of at least the center portion of the screen and high brightness by a preset first coefficient. And a first index calculating means for calculating index 1;
The gradation processing condition setting means includes:
Correcting the setting of the gradation processing condition based on the scene discrimination result based on the index 1 calculated by the first index calculating means;
An imaging apparatus characterized by that. A second image acquisition means for acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
The scene determination processing unit performs a shooting scene determination process based on the image data of the second image acquired by the second image acquisition unit.
The imaging apparatus according to claim 1. The first occupancy ratio calculating means calculates the first occupancy ratio from the image data of the second image acquired by the second image acquisition means;
The imaging apparatus according to claim 2. The scene discrimination processing means includes
Obtaining color information for the image data of the second image;
Based on the acquired color information, the image data is classified into a class composed of a combination of predetermined brightness and hue, and a second occupancy ratio indicating a proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating
A second index calculating means for calculating index 2 by multiplying the first occupancy calculated by the first occupancy calculating means by a preset second coefficient;
A third index calculating means for calculating index 3 by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating means for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculating means for calculating index 5 by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a preset fifth coefficient;
A sixth index calculating means for calculating the index 6 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset sixth coefficient;
A seventh index calculating means for calculating the index 7 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset seventh coefficient;
Scene discriminating means for identifying the scene of the image data based on the index 5, the index 6 and the index 7.
The imaging apparatus according to claim 2 or 3, wherein The first index calculation means includes:
Of the first occupancy ratio calculated by the first occupancy ratio calculating means, the occupancy ratio for the class composed of a combination of the outer edge of the screen and high brightness is multiplied by a preset eighth coefficient. To calculate index 1
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. Scene discrimination processing means for performing discrimination processing of a shooting scene based on image data of a shot image;
Gradation processing condition setting means for setting gradation processing conditions for the image based on the scene determination result obtained by the scene determination processing means;
A first occupation for classifying the image data into a class composed of a combination of a distance from the outer edge of the screen and a brightness, and calculating a first occupation ratio indicating a ratio of the entire image data for each classified class Rate calculation means;
Of the first occupancy ratios calculated by the first occupancy ratio calculating means, multiplying the occupancy ratio for the class composed of at least the center portion of the screen and high brightness by a preset first coefficient. And a first index calculating means for calculating index 1;
The gradation processing condition setting means includes:
Correcting the setting of the gradation processing condition based on the scene discrimination result based on the index 1 calculated by the first index calculating means;
An image processing apparatus. A second image acquisition means for acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
The scene determination processing unit performs a shooting scene determination process based on the image data of the second image acquired by the second image acquisition unit.
The image processing apparatus according to claim 6. The first occupancy ratio calculating means calculates the first occupancy ratio from the image data of the second image acquired by the second image acquisition means;
The image processing apparatus according to claim 7. The scene discrimination processing means includes
Obtaining color information for the image data of the second image;
Based on the acquired color information, the image data is classified into a class composed of a combination of predetermined brightness and hue, and a second occupancy ratio indicating a proportion of the entire image data for each classified class Second occupancy rate calculating means for calculating
A second index calculating means for calculating index 2 by multiplying the first occupancy calculated by the first occupancy calculating means by a preset second coefficient;
A third index calculating means for calculating index 3 by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating means for calculating the index 4 by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculating means for calculating index 5 by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a preset fifth coefficient;
A sixth index calculating means for calculating the index 6 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset sixth coefficient;
A seventh index calculating means for calculating the index 7 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset seventh coefficient;
Scene discriminating means for identifying the scene of the image data based on the index 5, the index 6 and the index 7.
The image processing apparatus according to claim 7, wherein the image processing apparatus is an image processing apparatus. The first index calculation means includes:
Of the first occupancy ratio calculated by the first occupancy ratio calculating means, the occupancy ratio for the class composed of a combination of the outer edge of the screen and high brightness is multiplied by a preset eighth coefficient. To calculate index 1
The image processing apparatus according to claim 6, wherein the image processing apparatus is an image processing apparatus. A scene discrimination processing step for performing a discrimination process of a shooting scene based on the image data of the shot image;
A gradation processing condition setting step for setting gradation processing conditions for the image based on the scene determination result obtained by the scene determination processing step;
A first occupation for classifying the image data into a class composed of a combination of a distance from the outer edge of the screen and a brightness, and calculating a first occupation ratio indicating a ratio of the entire image data for each classified class Rate calculation process;
Of the first occupancy ratios calculated by the first occupancy ratio calculating means, multiplying the occupancy ratio for the class composed of at least the center of the screen and high brightness by a preset first coefficient. A first index calculation step of calculating index 1 by
In the gradation processing condition setting step,
Correcting the setting of the gradation processing condition based on the scene discrimination result based on the index 1 calculated in the first index calculation step;
An image processing method. A second image acquisition step of acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
The scene determination processing step performs a shooting scene determination process based on the image data of the second image acquired in the second image acquisition step.
The image processing method according to claim 11. In the first occupancy ratio calculating step, the first occupancy ratio is calculated from image data of the second image acquired in the second image acquisition step.
The image processing method according to claim 12. The scene discrimination processing step includes
Obtaining color information for the image data of the second image;
Based on the acquired color information, the image data is classified into a class composed of a combination of predetermined brightness and hue, and a second occupancy ratio indicating a proportion of the entire image data for each classified class A second occupancy ratio calculating step for calculating
A second index calculating step for calculating index 2 by multiplying the first occupancy calculated in the first occupancy calculating step by a preset second coefficient;
A third index calculating step of calculating index 3 by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating step of calculating index 4 by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculation step of calculating index 5 by multiplying the average brightness of the skin color at least in the center of the screen of the image data by a preset fifth coefficient;
A sixth index calculating step of calculating index 6 by multiplying at least one of the index 2, index 3 and index 4 by a preset sixth coefficient;
A seventh index calculation step of calculating the index 7 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset seventh coefficient;
A scene determination step for identifying a scene of the image data based on the index 5, the index 6, and the index 7.
The image processing method according to claim 12 or 13, The first index calculation step includes
Of the first occupancy ratio calculated in the first occupancy ratio calculation step, an occupancy ratio for a class composed of a combination of the outer edge of the screen and high brightness is multiplied by a preset eighth coefficient. To calculate index 1
The image processing method according to claim 11, wherein: An image processing program for causing a computer to execute image processing,
A scene discrimination processing function for performing a scene discrimination process based on image data of a shot image;
A gradation processing condition setting function for setting gradation processing conditions for the image based on a scene determination result obtained by the scene determination processing function;
A first occupation for classifying the image data into a class composed of a combination of a distance from the outer edge of the screen and a brightness, and calculating a first occupation ratio indicating a ratio of the entire image data for each classified class Rate calculation function,
Of the first occupancy ratio calculated by the first occupancy ratio calculation function, multiplying the occupancy ratio for the class composed of at least the center portion of the screen and high brightness by a preset first coefficient. And a first index calculation function for calculating index 1;
The gradation processing condition setting function is
Correcting the setting of the gradation processing condition based on the scene discrimination result based on the index 1 calculated by the first index calculation function;
An image processing program characterized by that. A second image acquisition function for acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
The scene discrimination processing function performs a shooting scene discrimination process based on the image data of the second image acquired by the second image acquisition function.
The image processing program according to claim 16. The first occupancy rate calculation function calculates the first occupancy rate from image data of the second image acquired by the second image acquisition function.
The image processing program according to claim 17. The scene discrimination processing function is
Obtaining color information for the image data of the second image;
Based on the acquired color information, the image data is classified into a class composed of a combination of predetermined brightness and hue, and a second occupancy ratio indicating a proportion of the entire image data for each classified class A second occupancy ratio calculating function for calculating
A second index calculating function for calculating index 2 by multiplying the first occupancy calculated by the first occupancy calculating function by a preset second coefficient;
A third index calculation function for calculating index 3 by multiplying the second occupancy by a preset third coefficient;
A fourth index calculation function for calculating index 4 by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculation function for calculating index 5 by multiplying an average brightness of the skin color at least in the center of the screen of the image data by a fifth coefficient set in advance;
A sixth index calculation function for calculating the index 6 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset sixth coefficient;
A seventh index calculation function for calculating the index 7 by multiplying at least one of the index 2, the index 3 and the index 4 by a preset seventh coefficient;
A scene discrimination function for specifying a scene of the image data based on the index 5, the index 6, and the index 7.
The image processing program according to claim 17 or 18, characterized in that: The first index calculation function is:
Of the first occupancy ratio calculated by the first occupancy ratio calculation function, the occupancy ratio for the class composed of a combination of the outer edge of the screen and high brightness is multiplied by a preset eighth coefficient. To calculate index 1
20. The image processing program according to claim 16, wherein the image processing program is any one of claims 16 to 19.

Images