JP2007300348A - Imaging apparatus, and image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a more suitable image by automatically setting gray scale processing conditions by deciding a photographic scene and also making corrections based upon whether a vivid blue sky is included in an image. <P>SOLUTION: The photographic scene is decided on the basis of image data of an image obtained by photographing (step S6). Further, blue pixel information which is within a predetermined lightness value, on the upper part of a picture of the image data is acquired and skin color pixel information of the image data is acquired. On the basis of the blue pixel information, skin color pixel information, and absolute luminance information, an index 1 is calculated (step S9). Gray scale processing conditions for the image data are calculated on the basis of the decision result of the photographic scene, and the gray scale processing conditions are corrected on the basis of the index 1 (step S10). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that acquires a color image, an image processing method, and an image processing program.

  In recent years, digital still cameras (including those incorporated in devices such as mobile phones and laptop computers; hereinafter abbreviated as DSC) have become widespread and, like conventional color photographic film systems, hard copy images Are widely used. For example, a system such as a CD-R (CD-Recordable) or the like is widely used.

  When an image captured by DSC or the like is viewed as an image through various media, it is often not suitable as an image for viewing as it is due to inadequate exposure adjustment at the time of shooting. In other words, it is desirable to perform an appropriate gradation correction process on the image in order to compensate for overexposure or underexposure during shooting. In addition to this, for example, if there is a large luminance bias in the image due to the influence of the light source state at the time of shooting, such as shooting in a backlight state or close-up shooting using a flash, some correction It is desirable to perform processing.

In order to solve such a problem, for example, when an image captured by DSC is output by a printer, an image scene is discriminated by analyzing a luminance histogram of image data, and according to the image scene. When performing image processing automatically (see Patent Document 1) or DSC image capturing, subject information such as contrast intensity in the screen and brightness distribution state is obtained from the divided photometry results, and the subject information is captured within the DSC. A method for correcting the gradation of image data (see Patent Document 2) has been proposed. In addition, paying attention to the elements of human skin, the skin color pixels are detected from the digital image, and the brightness adjustment amount calculated based on the statistics of the skin color pixels is applied to the digital image, thereby improving the brightness of the image. A method has been proposed (see Patent Document 3).
JP 2000-134467 A JP 2001-54014 A JP 2003-108988 A

  However, in the above-described prior art, depending on the situation of the shooting scene, the vividness may be reduced by the correction, which may result in an undesirable image. For example, in a general image, a person's face is most noticed during viewing, and it is important that the balance between the brightness of the main subject such as the person's face and the brightness of the entire image is appropriate. However, when a colorful blue sky appears in the image, the vividness of the blue sky is reduced by the correction, resulting in an image that does not reflect the photographer's intention.

  The present invention has been made in view of the above-described problems in the prior art, and by automatically determining gradation processing conditions by determining the shooting scene of a shot image, the image is brightly colored. It is an object to obtain a more appropriate image by adding correction based on whether or not a clear blue sky is reflected.

  In order to solve the above problem, an imaging apparatus according to claim 1 is based on a scene determination processing unit that determines a shooting scene based on image data of an image acquired by shooting, and a determination result of the shooting scene. A gradation processing condition setting means for setting a gradation processing condition for the image data, and posture information indicating a degree of inclination of the imaging device at the time of photographing the image, and acquiring the image information based on the posture information. Image up / down determining means for determining up and down, blue pixel information acquiring means for acquiring blue pixel information within the predetermined brightness value located at the upper part of the screen of the image data, and at least preset in the blue pixel information First gradation calculation means for calculating a first index by multiplying by a first coefficient, and the gradation processing condition setting means based on the first index, And correcting the gradation processing condition based on the shadow scene discrimination result.

  Invention of Claim 2 is provided with the 2nd image acquisition means which acquires the 2nd image which reduced the image size from the 1st image acquired by photography in the imaging device of Claim 1, The scene discrimination processing means discriminates a photographic scene based on image data of the second image.

  According to a third aspect of the present invention, in the imaging apparatus according to the second aspect, the blue pixel information acquisition unit acquires the blue pixel information from image data of the second image.

  According to a fourth aspect of the present invention, in the imaging device according to the second or third aspect, the scene determination processing unit includes a color information acquisition unit that acquires color information from image data of the second image, and the acquisition Based on the obtained color information, the image data of the second image is classified into a class composed of a combination of predetermined brightness and hue, and occupies the entire image data of the second image for each classified class. First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio, and based on the acquired color information, the image data of the second image is obtained from a combination of the distance from the outer edge of the screen and the brightness. A second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the second image to the entire image data for each of the classified classes, and the first occupancy ratio Multiply the rate by a preset second factor A second index calculating means for calculating a second index, and a third index for calculating the third index by multiplying the first occupancy by a preset third coefficient. Index calculation means, fourth index calculation means for calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient, and image data of at least the second image The fifth index calculating means for calculating the fifth index by multiplying the average brightness of the skin color in the center of the screen by a preset fifth coefficient, the second index, the third index Sixth index calculating means for calculating a sixth index by multiplying at least one of the index and the fourth index by a preset sixth coefficient; and the second index , At least of the third index and the fourth index Seventh index calculating means for calculating a seventh index by multiplying one or more indices by a preset seventh coefficient, the fifth index, the sixth index, and the seventh index Scene discriminating means for identifying a photographic scene of the image data of the first image based on the index.

  Invention of Claim 5 is equipped with the skin color pixel information acquisition means which acquires skin color pixel information from the image data of a said 2nd image in the imaging device as described in any one of Claims 2-4, The first index calculating means further calculates the first index by multiplying the skin color pixel information by a preset eighth coefficient.

  The image processing method according to claim 6 is a scene determination processing step of determining a shooting scene based on image data of an image acquired by shooting, and a step for the image data based on the determination result of the shooting scene. Gradation processing condition setting step for setting a tone processing condition, and posture information indicating the degree of inclination of the imaging device at the time of shooting the image, and determining image up and down based on the posture information A blue pixel information acquisition step of acquiring blue pixel information located at the top of the screen of the image data and within a predetermined brightness value, and multiplying at least a first coefficient set in advance to the blue pixel information A first index calculation step of calculating a first index, and in the gradation processing condition setting step, the discrimination result of the shooting scene based on the first index And correcting the gradation processing condition based.

  The invention described in claim 7 includes a second image acquisition step of acquiring a second image obtained by reducing the image size from the first image acquired by photographing in the image processing method according to claim 6. The scene discrimination processing step discriminates a photographic scene based on image data of the second image.

  The invention according to claim 8 is the image processing method according to claim 7, wherein in the blue pixel information acquisition step, the blue pixel information is acquired from image data of the second image.

  The invention according to claim 9 is the image processing method according to claim 7 or 8, wherein the scene determination processing step includes a color information acquisition step of acquiring color information from image data of the second image, and Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of a predetermined lightness and hue, and the entire image data of the second image is classified into the classified class. A first occupancy ratio calculating step for calculating a first occupancy ratio indicating a ratio, and a combination of the distance from the outer edge of the screen and the brightness of the image data of the second image based on the acquired color information And a second occupancy ratio calculating step for calculating a second occupancy ratio indicating the ratio of the second image to the entire image data for each of the classified classes, and the first 2nd coefficient preset in occupancy A second index calculating step for calculating a second index by multiplication and a third index for calculating a third index by multiplying the first occupancy by a preset third coefficient. Index calculation step, a fourth index calculation step of calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient, and an image of at least the second image A fifth index calculating step of calculating a fifth index by multiplying the average brightness of the skin color in the center of the screen of data by a preset fifth coefficient, the second index, and the third index A sixth index calculating step of calculating a sixth index by multiplying at least one of the index and the fourth index by a preset sixth coefficient; and the second index Less than the index, the third index and the fourth index A seventh index calculating step of calculating a seventh index by multiplying one or more indices by a preset seventh coefficient; the fifth index; the sixth index; And a scene determination step of specifying a shooting scene of the image data of the first image based on the index of 7.

  Invention of Claim 10 in the image processing method as described in any one of Claims 7-9 includes the skin color pixel information acquisition process which acquires skin color pixel information from the image data of a said 2nd image, In the first index calculation step, the first index is further calculated by multiplying the flesh color pixel information by a preset eighth coefficient.

  According to an eleventh aspect of the present invention, a scene discrimination processing function for discriminating a shooting scene based on image data of an image acquired by shooting on a computer, and the image data based on the discrimination result of the shooting scene. A gradation processing condition setting function for setting gradation processing conditions, and posture information indicating the degree of inclination of the imaging device at the time of capturing the image, and determining the upper and lower of the image based on the posture information Multiplying a determination function, a blue pixel information acquisition function that is located at the top of the screen of the image data and acquires blue pixel information within a predetermined brightness value, and at least a first coefficient set in advance to the blue pixel information By doing so, an image processing program for realizing a first index calculation function for calculating a first index, wherein the gradation processing condition setting function is realized. When that, based on the first index, which is an image processing program for causing a correction gradation processing condition based on the determination result of the photographic scene.

  According to a twelfth aspect of the present invention, in the image processing program according to the eleventh aspect, the computer further acquires a second image obtained by reducing the image size from the first image acquired by photographing. When the image acquisition function is realized, and the scene determination processing function is realized, the photographic scene is determined based on the image data of the second image.

  According to a thirteenth aspect of the present invention, in the image processing program according to the twelfth aspect, the blue pixel information is acquired from the image data of the second image when the blue pixel information acquisition function is realized. Features.

  According to a fourteenth aspect of the present invention, in the image processing program according to the twelfth or thirteenth aspect, the scene determination processing function includes a color information acquisition function for acquiring color information from image data of the second image, Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of a predetermined lightness and hue, and the entire image data of the second image is classified into the classified class. A first occupancy ratio calculating function for calculating a first occupancy ratio indicating a occupancy ratio, and a combination of the distance from the outer edge of the screen and the brightness of the image data of the second image based on the acquired color information A second occupancy ratio calculation function for calculating a second occupancy ratio indicating the ratio of the second image to the entire image data for each of the classified classes, and the first Pre-set to occupancy A second index calculation function for calculating a second index by multiplying by the second coefficient, and a third index by multiplying the first occupancy by a preset third coefficient. A third index calculation function for calculating the fourth index, a fourth index calculation function for calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient, and at least the second A fifth index calculating function for calculating a fifth index by multiplying the average brightness of the skin color of the image data of the image of the second image by a preset fifth coefficient; A sixth index calculation function for calculating a sixth index by multiplying at least one of the index, the third index, and the fourth index by a preset sixth coefficient; , The second index, the third index, and the fourth index A seventh index calculating function for calculating a seventh index by multiplying at least one of the indices by a preset seventh coefficient; the fifth index; the sixth index; And a scene discrimination function for specifying a shooting scene of the image data of the first image based on a seventh index.

  The invention according to claim 15 is the image processing program according to any one of claims 12 to 14, wherein the computer further acquires skin color pixel information from image data of the second image. When realizing the information acquisition function and realizing the first index calculation function, the first index is calculated by multiplying the skin color pixel information by a preset eighth coefficient. It is characterized by.

  According to the first, sixth, and eleventh aspects, the gradation processing condition is automatically set by determining the photographing scene of the photographed image, and a colorful blue sky appears in the image. A more appropriate image can be obtained by applying correction based on whether or not there is.

  According to the second, seventh, and twelfth aspects of the present invention, since the photographic scene is determined based on the image data of the image whose image size is reduced, the processing speed can be increased.

  According to the third, eighth, and thirteenth aspects of the present invention, the blue pixel information is acquired from the image data of an image with a reduced image size, so that the processing speed can be increased.

  According to the invention described in claims 4, 9, and 14, the photographing scene can be specified accurately.

  According to the fifth, tenth, and fifteenth aspects of the present invention, correction based on skin color pixel information can be added to the gradation processing conditions.

(Configuration of Imaging Device 100)
First, the configuration of the imaging apparatus 100 in the embodiment of the present invention will be described.
1A is a front view of the imaging apparatus 100, and FIG. 1B is a rear view of the imaging apparatus 100. The imaging apparatus 100 is, for example, a digital camera, and has a cross key 2, a photographing optical system 3, a flash 4, a finder 5, and a power switch 6 on or inside a housing 1 made of a material such as metal or synthetic resin. A display unit 7 and a release button 8 are provided.

  FIG. 2 shows an internal configuration of the imaging apparatus 100. As shown in FIG. 2, the imaging apparatus 100 includes a processor 31, a memory 32, an imaging device 33 such as a CCD (Charge Coupled Device), a photographing optical system 3, an image data output unit 34, an operation unit 35, a timing generator 36, a shutter. The unit 37, the aperture unit 38, the focus unit 39, the posture information detection unit 40, and the display unit 7 are configured. The processor 31 includes an image processing unit 10 that performs image processing, and a photographing processing unit 20 that performs photographing control and processing of photographed images.

  The cross key 2 is composed of buttons in four directions, up, down, left, and right, and is used by the user to select or set various modes.

  The photographing optical system 3 includes a plurality of lenses, a lens barrel, and the like, and has a zoom function. The imaging optical system 3 forms an image on the image sensor 33 with the light received by the lens. The flash 4 emits auxiliary light according to a control signal from the processor 31 when the subject brightness is low.

  The viewfinder 5 is for the user to check the shooting target and shooting area by eye contact. The power switch 6 is a switch for operating ON / OFF of the operation in the imaging apparatus 100.

  The display unit 7 includes a liquid crystal panel, and displays an image currently captured on the image sensor 33, an image captured in the past, a menu screen, a setting screen, and the like according to a display control signal input from the processor 31.

  The release button 8 is a two-stage push-in switch that is provided on the upper surface of the housing 1 and can detect a half-pressed state (preliminary shooting) and a fully-pressed state (main shooting) by the user.

  The memory 32 stores (saves) image data obtained by photographing. Further, the memory 32 stores various processing programs executed in the imaging apparatus 100, data used by the processing programs, and the like.

  The image data output unit 34 transfers and records the image data in the memory 32 to a storage medium for storage (such as an SD memory card or a multimedia card (MMC)). Alternatively, it is transferred to an external device. The image data output unit 34 is controlled by the processor 31.

  The image sensor 33 converts the imaged light into electric charges. Thereby, for example, image data as shown in FIG. 13A is obtained. This image includes an object in the imaging range (imaging range), that is, an imaging object (imaging target) and other objects (background). The RGB value of each pixel of the entire image is represented by, for example, 256 gradations.

  The shutter unit 37 controls the timing of resetting the image sensor 33 and the timing of charge conversion based on the state (half-pressed state or fully-pressed state) detected by the release button 8. Timing control is performed by the timing generator 36.

  Adjustment of the amount of light received by the image sensor 33 is performed by the aperture unit 38 and / or the shutter unit 37. The focus unit 39 drives the photographic optical system 3 to execute a control operation for focusing on the photographic subject.

  The posture information detection unit 40 is composed of an angle sensor such as a gyro-type horizontal sensor, detects the tilt angle of the imaging device 100 at the time of shooting when the release button 8 is pressed (full press), and indicates the degree of the tilt. Posture information is output to the processor 31. In the present embodiment, as the posture information, the normal position (see FIG. 3A) that is a reference for photographing is set to 0 degree, and the image is rotated 90 degrees clockwise from the normal position around the optical axis of the imaging apparatus. In the case of the first vertical position (see FIG. 3B), the second vertical position (FIG. 3) rotated by 90 degrees counterclockwise around the optical axis of the imaging device from +90 degrees. In the case of (c)), −90 degrees is obtained, and in the case of the upside down position (see FIG. 3D) rotated 180 degrees from the normal position around the optical axis of the imaging device, 180 degrees are obtained. Shall. Specifically, when the tilt angle of the imaging apparatus 100 is −45 degrees to 45 degrees, the normal position (0 degrees), and when 45 degrees to 135 degrees, the first vertical position (+90 degrees), from −45 degrees. When the angle is −135 degrees, the second vertical position (−90 degrees) is set, and when the angle is 135 degrees to 225 degrees, the upside down position (180 degrees) is set.

(Internal configuration of the imaging processing unit 20)
FIG. 4 shows an internal configuration of the imaging processing unit 20. The shooting processing unit 20 performs control related to shooting conditions during shooting and processing of shot images.

  As shown in FIG. 4, the imaging processing unit 20 includes an AE control unit 21, an AF control unit 22, a pixel interpolation unit 23, an AWB control unit 24, a gamma correction unit 25, and the like related to imaging conditions.

  The AE control unit 21 automatically controls the exposure amount at the time of image capturing. Normally, the exposure amount control during shooting standby is performed by controlling the shutter speed with the aperture opened, and the exposure amount during shooting is controlled by the aperture and shutter speed.

  With reference to the flowchart of FIG. 5A, an example of exposure amount control during shooting standby in the AE control unit 21 will be described.

  First, the diaphragm unit 38 sets the diaphragm to an open fixed diaphragm (step S111). The AE control unit 21 reads data of a predetermined photometric area from the image data obtained by the image sensor 33 (step S112), and acquires information corresponding to the luminance value (step S113). Information corresponding to this luminance value is used as information for AE control, and the G value of RGB three-color components is often used simply (hereinafter referred to as luminance information G). The charge accumulation time in the next frame of the image sensor 33 is set according to the luminance information G (step S114), and the charge accumulation time in the next frame is controlled by the timing generator 36 so that the predetermined luminance level is obtained. (Step S117). This is control of the shutter speed, and FIG. 6 shows how charges are accumulated. That is, when the acquired luminance level is large (bright), the charge accumulation time is shortened, and when the acquired luminance level is small (dark), the charge accumulation time is lengthened, thereby stabilizing the exposure amount.

  By performing exposure amount control during shooting standby in this way, a photographer can view a live view image whose exposure amount is controlled on the display unit 7 such as a liquid crystal display.

  During actual shooting, in addition to the shutter speed control described above, aperture control is also performed. FIG. 5B is a flowchart illustrating an example of exposure amount control at the time of actual photographing in the AE control unit 21. Similarly to the above (steps S112 and S113), the aperture unit 38 is controlled in accordance with the luminance information G of the photometry area obtained (step S116). That is, the exposure value is stabilized by decreasing the aperture value when the acquired brightness level is large (bright) and increasing the aperture value when the brightness level is low (dark). The adjustment level in combination with the shutter speed is controlled based on data of a predetermined program diagram, for example, the charge accumulation time of the image sensor 33 is adjusted according to the aperture value (step S115, S117).

  As described above, the exposure amount control is performed at the time of photographing, so that the photographer can automatically set the exposure amount with respect to the photographed image by using an arbitrary or predetermined combination of the aperture value and the shutter speed.

  The AF control unit 22 performs automatic control for focusing an image when taking an image. For example, as shown in the flowchart of FIG. 7, this in-focus control is performed by detecting the in-focus position by driving the photographing optical system 3 and stopping it according to the position.

  When driving of the photographic optical system 3 is started (step S121), data of a predetermined distance measuring area is sequentially read from the image data obtained by the image sensor 33 by the AF control unit 22 along with the driving (step S121). In step S122, contrast information is acquired according to this data (step S123). This is for detecting whether or not the in-focus position has been reached, and is determined as follows. That is, the contrast information is set and calculated so as to depend on the sharpness of the edge portion by taking the difference between adjacent pixels in the distance measurement area data. The state that has reached the maximum is determined to be the focal point (steps S124 and S125). If it is determined that the focus position is not reached (step S125; No), the driving of the photographing optical system 3 is continued (step S126).

  FIG. 8 shows changes in contrast information accompanying the movement of the photographic optical system 3 and how the in-focus position is detected. In the above operation, a distance measurement calculation is performed in which contrast information is sequentially acquired while driving the optical system to obtain a focal position, and the photographing optical system 3 is stopped in accordance with the focal length ( Step S127).

  With such AF control, it is possible to automatically obtain a captured image that is always in focus during shooting.

  The pixel interpolation unit 23 interpolates pixels for each color component with respect to the CCD array in which the RGB color components are dispersedly arranged in the image sensor 33, and processes the image data so that each color component value is obtained at the same pixel position. To do.

  As shown in the flowchart of FIG. 9, the pixel interpolation unit 23 masks the RGB image data (step S131) obtained by the image sensor 33 with the RGB pixel filter patterns (steps S132, S134, and S136), and thereafter. Average interpolation (also referred to as pixel interpolation) is performed (steps S133, S135, and S137). Among these, the G pixel filter pattern having pixels up to a high band is a median filter that performs average interpolation by substituting the average value of the intermediate binary values of the surrounding four pixels, and the R and B pixel filters The pattern performs average interpolation with respect to the same color from 9 neighboring pixels.

  The AWB control unit 24 automatically adjusts the white balance in the captured image. Assuming that the photographed image has a subject area in which RGB color balance is achieved (white when summed up), each component value of RGB of the image is achieved so as to achieve white balance in that area. Adjust the level for. This white balance processing is performed as follows, for example.

  As shown in the flowchart of FIG. 10, the AWB control unit 24 estimates an area that is supposed to be white from the luminance and saturation data of the image data obtained by the image sensor 33 (step S141) (step S141). S142). For that region, the average intensity, G / R ratio, and G / B ratio of each RGB component value are obtained, and the R value and B value correction gains for the G value are calculated (steps S143 and S144). Based on this, gain correction for each color component in the entire image is performed (step S145).

  By such AWB control, it is possible to automatically correct the collapse of the color balance of the entire screen that occurs during shooting, and an image with a stable color tone can be obtained regardless of the actual illumination state of the subject.

  The gamma correction unit 25 performs processing for converting the gradation of the captured image so as to be suitable for the gamma characteristic of the output device.

  The gamma characteristic is a gradation characteristic, and indicates how an output gradation is set with respect to an input gradation by a correction value or a correction curve. FIG. 11 shows an example of a correction curve indicating an output value corrected for an input value. The gamma correction is a conversion process for converting an input value into an output value according to the correction value or the correction curve. Since this gradation characteristic varies depending on the output device, it is necessary to correct this gamma characteristic in order to obtain a gradation suitable for the output device. As a result, the captured linear image is converted into a non-linear image.

  A monitor is generally set as the output device, and gamma correction of the captured image is performed so as to match the gamma characteristic of a general monitor.

(Internal configuration of the image processing unit 10)
FIG. 12 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls a gradation correction operation based on scene discrimination for a captured image in the imaging apparatus 100. After the processing in the imaging processing unit 20, or independently of the processing, A series of processes such as image acquisition, scene discrimination, and gradation processing condition setting are executed.

  As shown in FIG. 12, the image processing unit 10 includes a first image acquisition unit 11, a second image acquisition unit 12, an occupation rate calculation unit 13, an index calculation unit 14, a scene determination unit 15, an image up / down determination unit 16a, a blue color The pixel information acquisition unit 16b, the skin color pixel information acquisition unit 16c, the index 1 calculation unit 16d, the gradation processing condition setting unit 17, and the gradation conversion processing unit 18 are configured.

  The first image acquisition unit 11 acquires the image data of the latest image captured on the image sensor 33 as the first image when the release button 8 is fully pressed. Further, absolute brightness information BV (Brightness Value) calculated from the brightness information G of the image sensor 33 at the time of shooting is also acquired. The absolute luminance information BV is calculated from a photometric value at the time of shooting, and is added and recorded in the image data as Exif (Exchangeable Image File Format) data. The acquired image data of the first image and absolute luminance information BV are held in the memory 32.

  The second image acquisition unit 12 divides the acquired first image into N × M rectangular regions (regions divided into M pieces in the vertical direction and N pieces in the horizontal direction). FIG. 13A shows an example of the first image, and FIG. 13B shows an example in which the first image is divided into 22 × 14 regions. The number of divided areas is not particularly limited. Each divided area corresponds to one pixel of image data of the second image. That is, the second image is an image in which the image size is substantially reduced. This reduced image is the second image, and the second image is acquired by the above operation.

  The occupation ratio calculation unit 13 acquires color information from the image data of the second image, that is, each pixel constituting the image. Based on the acquired color information, each pixel of the image is classified into a predetermined class composed of a combination of lightness and hue (see FIG. 17), and for each classified class, the pixels belonging to the class are included in the entire image. A first occupation ratio indicating the occupation ratio is calculated.

  Further, the occupancy rate calculation unit 13 classifies each pixel into a predetermined class composed of a combination of the distance from the outer edge of the screen of the second image and the brightness (see FIG. 18), and for each classified class, A second occupation ratio indicating a ratio of pixels belonging to the class to the entire image is calculated.

  The occupation rate calculation process executed in the occupation rate calculation unit 13 will be described in detail later with reference to FIG.

  The index calculation unit 14 multiplies the first occupancy calculated by the occupancy calculation unit 13 by a coefficient set in advance according to the shooting conditions, thereby specifying an index 2 and an index 3 for specifying a shooting scene. Is calculated. That is, the index calculation unit 14 functions as a second index calculation unit and a third index calculation unit.

  In addition, the index calculation unit 14 multiplies the second occupancy calculated by the occupancy ratio calculation unit 13 by a coefficient set in advance according to the shooting conditions, thereby obtaining an index 4 for specifying the shooting scene. calculate. That is, the index calculation unit 14 functions as a fourth index calculation unit.

  Further, the index calculation unit 14 multiplies the average luminance value of the flesh color region in the central portion of the screen of the second image, the difference value between the maximum luminance value and the average luminance value, and the like by a coefficient set in advance according to the shooting conditions. By doing this, the index 5 for specifying the shooting scene is calculated. That is, the index calculation unit 14 functions as a fifth index calculation unit.

  In addition, the index calculation unit 14 multiplies the average luminance value of the skin color area in the center of the screen of the second image, the index 2 and the index 4 by a coefficient set in advance according to the shooting conditions. As a result, a new index 6 is calculated. That is, the index calculation unit 14 functions as a sixth index calculation unit.

  In addition, the index calculation unit 14 multiplies the average luminance value of the skin color area in the center of the screen of the second image, the index 3 and the index 4 by a coefficient set in advance according to the shooting conditions. Thus, a new index 7 is calculated. That is, the index calculation unit 14 functions as a seventh index calculation unit.

  The index calculation process executed in the index calculation unit 14 will be described in detail later with reference to FIG.

  The scene determination unit 15 identifies the shooting scene of the first image based on each index calculated by the index calculation unit 14. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes the over and under degrees. The method for determining the shooting scene will be described in detail later.

  As described above, the occupation rate calculation unit 13, the index calculation unit 14, and the scene determination unit 15 function as a scene determination processing unit.

  The image up / down determination unit 16a determines the up / down direction of the second image based on the posture information acquired by the posture information detection unit 40. That is, the posture information detection unit 40 and the image up / down determination unit 16a realize an image up / down determination unit.

  The blue pixel information acquisition unit 16b is located at the top of the screen of the image data of the second image and acquires blue pixel information within a predetermined brightness value. That is, the blue pixel information acquisition unit 16b functions as a blue pixel information acquisition unit.

  The skin color pixel information acquisition unit 16c acquires skin color pixel information from the image data of the second image. That is, the skin color pixel information acquisition unit 16c functions as a skin color pixel information acquisition unit.

  The index 1 calculation unit 16d multiplies the skin color pixel information acquired by the skin color pixel information acquisition unit 16c in advance by multiplying at least the blue pixel information acquired by the blue pixel information acquisition unit 16b by a preset first coefficient. The index 1 is calculated by multiplying the set eighth coefficient.

  The gradation processing condition setting unit 17 sets gradation processing conditions for the first image based on the shooting scene determined by the scene determination unit 15. At that time, the gradation processing condition setting unit 17 corrects the gradation processing condition based on the determination result of the shooting scene based on the index 1 calculated by the index 1 calculation unit 16d.

  The gradation processing condition setting unit 17 first calculates a gradation adjustment parameter for gradation adjustment for the first image based on each index calculated by the index calculation unit 14. Then, the gradation conversion amount is calculated based on the gradation adjustment parameter and the index 1, and the gradation processing condition for the first image is set based on the gradation conversion amount. The setting of the gradation processing conditions will be described in detail later with reference to FIG.

  The gradation conversion processing unit 18 performs gradation conversion processing on the first image based on the gradation processing conditions set by the gradation processing condition setting unit 17.

  In addition to the processing performed in the image processing unit 10, the processor 31 performs processing in the photographing processing unit 20 such as automatic white balance processing and gamma conversion processing, other image processing, image format conversion, image processing, and the like based on known techniques. It has a function of controlling processing operations such as data recording.

  The processing of each unit in the processor 31 is basically performed by hardware processing, but a part thereof is performed by software processing such as executing a program stored (saved) in the memory 32.

(Operation of Imaging Device 100)
Next, the operation of the imaging device 100 in the present embodiment will be described. Hereinafter, the photographing object is referred to as “main subject”.

  First, an overall flow of processing executed by the imaging apparatus 100 will be described with reference to a flowchart of FIG.

  First, when the power switch 6 is turned on (when the power is turned on), preprocessing such as resetting of the memory 32 is performed. The user starts an operation for shooting by directing the imaging device 100 toward the main subject so that the main subject enters the field of the imaging device 100. When the release button 8 is pressed and shooting is performed (step S1), the image formed on the image sensor 33 is captured as an electrical signal, and interpolation processing based on the CCD array is performed (step S2). Then, the first image acquisition unit 11 acquires a first image as a captured image and holds it in the memory 32 (step S3). The first image is a linear image (RAW image).

  AWB (automatic white balance) processing is performed on the first image acquired by shooting (step S4). The AWB process may be performed separately on the first image and the second image after the acquisition of the second image.

  After the AWB process, the image size of the image data of the first image is reduced by the second image acquisition unit 12 and acquired as the second image (step S5). As an actual image data size reduction method, a known technique such as a simple average, a bilinear method, or a bicubic method can be used.

  Next, scene discrimination processing for discriminating a shooting scene is performed based on the acquired image data of the second image (step S6). The scene discrimination process will be described later with reference to FIG.

  On the other hand, when shooting is performed, the absolute luminance information BV is acquired by the first image acquisition unit 11 (step S7). The acquired absolute luminance information BV is held in the memory 32.

  Simultaneously with the photographing, the posture information detection unit 40 detects the tilt angle of the imaging device 100 and acquires posture information (step S8).

  After the second image is acquired, the index 1 calculation process is performed (step S9). The index 1 calculation process will be described later with reference to FIG.

  Next, conditions necessary for the tone conversion processing of the first image are set based on each index obtained by the scene discrimination process and the discrimination result of the shooting scene and the index 1 obtained by the index 1 calculation process. A gradation processing condition is set (step S10). The gradation processing condition setting will be described later with reference to FIG.

  On the other hand, the first image after the AWB process is subjected to a gamma conversion process and converted to a non-linear image (step S11). However, the gamma conversion process is a gradation conversion process, and may be performed together with the gradation conversion process in step S12 described below.

  Next, gradation conversion processing is performed on the image data of the first image, which is a photographed image, based on the gradation processing conditions set based on the second image (step S12). The gamma conversion process in step S11 is to convert to a non-linear gradation in accordance with the vision, but the gradation conversion process in step S12 is to correct the influence on the gradation due to the light source condition of the shooting scene.

  Next, other image processing (noise processing, sharpness processing, etc.) is performed on the image data after the gradation conversion processing (step S13).

  Next, the image format is converted into an image such as a JPEG format, a TIFF format, or a BMP format for image recording (step S14). Thereafter, the image data after the image format conversion is recorded on a storage recording medium (such as an SD memory card or a multimedia card (MMC)) (step S15). When the next shooting is started or the power switch 6 is turned off, the operation of the image pickup apparatus 100 ends.

(Scene discrimination processing)
Next, the scene discrimination process (step S6 in FIG. 14) in the imaging device 100 will be described with reference to FIGS.

  As shown in FIG. 15, the scene determination process includes a color space conversion process (step S20), an occupation ratio calculation process (step S21), an index calculation process (step S22), and a scene determination (step S23). The Hereinafter, each process shown in FIG. 15 will be described in detail.

<Color space conversion processing>
In the color space conversion process in step S20 of FIG. 15, first, information indicating the RGB value, the luminance value, and the white balance of each pixel of the second image obtained from the photographed first image is acquired. As the luminance value, a value calculated by substituting RGB values into a known conversion formula may be used. Next, the acquired RGB values are converted into the HSV color system, and the color information of the image is acquired. The HSV color system represents image data with three elements: Hue, Saturation, and Lightness (Value or Brightness), and was devised based on the color system proposed by Munsell. It has been done. The conversion to the HSV color system is performed using an HSV conversion program or the like. Usually, the calculated hue value H is defined as 0 to 360 for the input R, G, B, The unit of saturation value S and brightness value V is defined as 0-255.

  In the present embodiment, “brightness” means “brightness” that is generally used unless otherwise noted. In the following description, V (0 to 255) of the HSV color system is used as “brightness”, but a unit system representing the brightness of any other color system may be used. In that case, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated.

  In the present embodiment, “hue” means “color” generally used unless otherwise noted. In the following description, H (0 to 360) of the HSV color system is used as “hue”, but for example, a color represented by a red difference value (Cr) or a blue difference value (Cb) may be used. In that case, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated. In step S20, the values of H, S, and V obtained as described above are acquired as color information.

<Occupancy rate calculation process>
Next, the occupation rate calculation process (step S21 in FIG. 15) will be described with reference to the flowchart in FIG.

  First, based on the HSV value calculated by the color space conversion process, each pixel of the second image is classified into a predetermined class composed of a combination of hue and brightness, and the cumulative number of pixels is calculated for each classified class. As a result, a two-dimensional histogram is created (step S30).

  FIG. 17 shows a class composed of a combination of brightness and hue. In step S30, the lightness (V) is 0-5 (v1), 6-12 (v2), 13-24 (v3), 25-76 (v4), 77-109 (v5), 110-110. It is divided into seven classes of 149 (v6) and 150 to 255 (v7). As shown in FIG. 17, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  Hue H includes a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region (H4) having a hue value of 161 to 250. ), And is divided into four regions of a red hue region (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the shooting scene is small. The skin color hue region is further divided into a skin color region (H1) and another region (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue H ′ that satisfies the following equation (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the equation (1) is (H2). And

10 <saturation S <175;
Hue H ′ = Hue H + 60 (when 0 ≦ Hue H <300);
Hue H ′ = Hue H−300 (when 300 ≦ Hue H <360).
Luminance Y = R × 0.30 + G × 0.59 + B × 0.11 (A)
As
Hue H ′ / Luminance Y <3.0 × (Saturation S / 255) +0.7 (1)

  Therefore, the number of classes when the second image is classified into classes composed of combinations of brightness and hue is 4 × 7 = 28. Moreover, it has at least three classes (v1, v2, v3) within 10% of the maximum brightness value (255). In addition, it is also possible to use lightness (V) in Formula (A) and Formula (1).

  After step S30, each pixel of the second image is classified into a predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness, and the two-dimensional histogram is calculated by calculating the cumulative number of pixels for each classified class. Is created (step S31).

  FIG. 18A shows three regions n1 to n3 divided in step S31 according to the distance from the outer edge of the screen of the second image. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. A region n3 is a central region of the second image. Here, it is preferable to divide n1 to n3 so as to have substantially the same number of pixels. In the present embodiment, three divisions are used, but the present invention is not limited to this. In step S31, the brightness is divided into seven areas v1 to v7 as described above. FIG. 18B shows a class composed of combinations of three regions n1 to n3 and brightness. As shown in FIG. 18B, the number of classes when the second image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 3 × 7 = 21.

  When a two-dimensional histogram is created in step S30, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of hue and brightness to the total number of pixels (N × M) Is calculated (step S32). That is, step S30 and step S32 correspond to a first occupation rate calculation step.

  Assuming that the first occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy in each class is expressed as shown in Table 1.

  When the two-dimensional histogram is created in step S31, the second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness to the total number of pixels is obtained. Calculation is performed (step S33), and the occupancy rate calculation process ends. That is, step S31 and step S33 correspond to a second occupancy rate calculating step.

  When the second occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each class is expressed as shown in Table 2.

  Note that a three-dimensional histogram may be created by classifying each pixel into a class consisting of a distance from the outer edge of the screen, brightness, and hue, and calculating the cumulative number of pixels for each class. Hereinafter, a method using a two-dimensional histogram is adopted.

<Indicator calculation process>
Next, the index calculation process (step S22 in FIG. 15) will be described with reference to the flowchart in FIG.

  First, the first occupancy calculated for each class in the occupancy calculation processing is multiplied by a second coefficient set in advance according to the shooting conditions to obtain a sum, thereby specifying a shooting scene. Index 2 is calculated (step S40). The index 2 is an index representing the degree of overshoot of the main subject, and is for separating only an image that should be determined as “main subject is over” from other shooting scenes.

  Next, the first occupancy rate calculated for each class is multiplied by a third coefficient (a coefficient different from the second coefficient) set in advance according to the shooting conditions to obtain a sum, thereby taking a picture. An index 3 for specifying the scene is calculated (step S41). The index 3 is an index that collectively represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness. Only images that should be determined as “backlight” and “main subject is under” are taken from other shooting scenes. It is for separation.

Hereinafter, the calculation method of the index 2 and the index 3 will be described in detail.
Table 3 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 3 is a weighting coefficient by which the first occupation ratio Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 3, a positive (+) coefficient is used for the first occupancy calculated from the region distributed in the skin color hue region (H1) of high brightness (v6), and the other hue is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region. FIG. 20 shows a curve (coefficient curve) in which the second coefficient in the skin color region (H1) and the second coefficient in the other region (green hue region (H3)) continuously change over the entire brightness. It is shown. According to Table 3 and FIG. 20, in the region of high brightness (V = 77 to 150), the sign of the second coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the second coefficient in (H3)) is negative (-), and it can be seen that the signs of both are different.

  When the second coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (2-1) to (2-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × (−8) (2-1)
Sum of H2 regions = R12 x (-2) + R22 x (-1) + (omitted) ... + R72 x (-10) (2-2)
Sum of H3 regions = R13 × 5 + R23 × (−2) + (omitted)… + R73 × (-12) (2-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)… + R74 × (-12) (2-4)

The index 2 is defined as in Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 2 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.5
(3)

  Table 4 shows the third coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 4 is a weighting coefficient by which the first occupation ratio Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 4, a negative (-) coefficient is used for the occupancy calculated from the areas (v4, v5) distributed in the intermediate brightness of the flesh-color hue area (H1), and the flesh-color hue range (H1) is low. A coefficient 0 is used for the occupation ratio calculated from the brightness (shadow) region (v2, v3). FIG. 21 shows the third coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 4 and FIG. 21, the sign of the third coefficient of the intermediate lightness region of lightness value 25 to 150 in the flesh color hue region is negative (−), and the low lightness (shadow) region of lightness value 6 to 24 The third coefficient is 0, and it can be seen that there is a large difference in the coefficients in both regions.

  When the third coefficient in the lightness area vi and the hue area Hj is Dij, the sum of the Hk areas for calculating the index 3 is defined as in Expression (4).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × 2 (4-1)
Sum of H2 regions = R12 x (-2) + R22 x (-1) + (omitted) ... + R72 x 2 (4-2)
Sum of H3 region = R13 × 2 + R23 × 1 + (omitted)… + R73 × 3 (4-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)… + R74 × 3 (4-4)

The index 3 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 3 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.7
(5)

  Since the index 2 and the index 3 are calculated based on the brightness and hue distribution amount of the second image, the index 2 and the index 3 are effective in determining a shooting scene when the image is a color image.

  When the index 2 and the index 3 are calculated, the fourth factor (second factor, third factor) set in advance according to the shooting condition is added to the second occupancy rate calculated for each class in the occupancy rate calculation process. The index 4 for specifying the photographic scene is calculated by multiplying by a coefficient different from the coefficient of () and taking the sum (step S42). The index 4 indicates the difference in the contrast between the center and the outside of the screen of the image data between the backlight in which the main subject is under and the image in which the main subject is over.

Hereinafter, a method for calculating the index 4 will be described.
Table 5 shows the fourth coefficient necessary for calculating the index 4 by class. The coefficient of each class shown in Table 5 is a weighting coefficient by which the second occupation ratio Qij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 22 shows the fourth coefficient in the screen areas n1 to n3 as a curve (coefficient curve) that continuously changes over the entire brightness.

  When the fourth coefficient in the lightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 4 is defined as in Expression (6).

Accordingly, the sum of the n1 to n3 regions is represented by the following formulas (6-1) to (6-3).
Sum of n1 region = Q11 × 12 + Q21 × 10 + (omitted) ... + Q71 × 0 (6-1)
Sum of n2 regions = Q12 × 5 + Q22 × 3 + (omitted) ... + Q72 × 0 (6-2)
Sum of n3 regions = Q13 × (-1) + Q23 × (-4) + (omitted) ... + Q73 × (-8) (6-3)

The index 4 is defined as in Expression (7) using the sum of the n1 to n3 regions shown in Expressions (6-1) to (6-3).
Index 4 = sum of n1 region + sum of n2 region + sum of n3 region + 0.7 (7)

  The index 4 is calculated on the basis of the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the second image, and therefore, the photographing scene of the monochrome image as well as the color image is discriminated. It is also effective.

  Further, for example, the weight of the fourth coefficient multiplied by the second occupancy calculated from a predetermined class of distance and brightness from the outer edge of the screen in accordance with the focus detection area detected by a known method By changing, it is possible to calculate an index for discriminating a scene with higher accuracy.

  When the indices 2 to 4 are calculated, the average brightness value of the skin color area in the center of the screen of the second image, the difference value between the maximum brightness plant and the average brightness value of the second image, etc., according to the shooting conditions By multiplying a preset fifth coefficient, an index 5 for specifying a shooting scene is calculated (step S43).

Hereinafter, the index 5 calculation process will be described in detail with reference to the flowchart of FIG.
First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the second image using the equation (A). Next, the average luminance value x1 of the skin color area at the center of the screen of the second image is calculated (step S50). Here, the screen center portion is, for example, a region constituted by the region n3 shown in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the second image is calculated (maximum luminance value−average luminance value) (step S51).

Next, the standard deviation x3 of the luminance of the second image is calculated (step S52), and the average luminance value x4 at the center of the screen is calculated (step S53). Next, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the second image and the average luminance value Yskin_ave of the skin color area is calculated (step S54). This comparison value x5 is expressed as the following formula (8-1).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (8-1)

Next, the index 5 is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fifth coefficient set in advance according to the shooting conditions to obtain a sum (step 5). S55), the index 5 calculation process ends. The index 5 is defined as the following formula (8-2).
Index 5 = 0.05 × x1 + 1.41 × x2 + (− 0.01) × x3
+ (-0.01) × x4 + 0.01 × x5-5.34 (8-2)

  This index 5 has not only the compositional characteristics of the screen of the second image but also the luminance histogram distribution information, and is particularly effective for discriminating a shooting scene from which the main subject is over and an under shooting scene.

  Returning to FIG. 19, when the index 5 is calculated, a sixth coefficient that is set in advance according to the shooting condition is added to the average luminance value of the skin color area in the center of the screen of the index 2, the index 4, and the second image. By multiplying, the index 6 is calculated (step S44).

  Further, the index 7 is calculated by multiplying the average luminance value of the skin color area in the center of the screen of the index 3, the index 4 and the second image by a seventh coefficient set in advance according to the shooting conditions ( Step S45), the present index calculation process ends.

Hereinafter, the calculation method of the index 6 and the index 7 will be described in detail.
The average luminance value of the skin color area in the center of the screen of the second image is set as an index 9. Here, the central portion of the screen is, for example, a region composed of the region n2 and the region n3 in FIG. At this time, the index 6 is defined as in Expression (9) using the indices 2, 4, and 9, and the index 7 is defined as in Expression (10) using the indices 3, 4, and 9. Defined.
Indicator 6 = 0.54 x Indicator 2 + 0.50 x Indicator 4 + 0.01 x Indicator 9-0.65 (9)
Indicator 7 = 0.83 x Indicator 3 + 0.23 x Indicator 4 + 0.01 x Indicator 9-1.17 (10)

  As a method of calculating the average luminance value (for example, the overall average luminance value) in FIG. 23, a simple addition average value of individual luminance data obtained from each light receiving unit of the imaging device 100 may be obtained, or imaging may be performed. Similar to the center-weighted average metering, which is often used as the metering method of the apparatus 100, the luminance data obtained from the light receiving unit near the center of the screen is highly weighted, and the luminance data obtained from the light receiving unit near the periphery of the screen is weighted. A method of obtaining the average value by lowering may be used. In addition, the luminance data obtained from the vicinity of the light receiving unit corresponding to the focus detection area is increased in weight, and the luminance data obtained from the light receiving unit distant from the focus detection position is reduced in weight to obtain an average value, etc. May be used.

<Scene discrimination>
Returning to FIG. 15, the scene discrimination (step S23) will be described.
When the indices 5 to 7 are calculated, the shooting scene is determined based on the values of these indices. Table 6 shows the contents of the shooting scene discrimination according to the values of the index 5, the index 6, and the index 7.

  FIG. 24 is a discrimination map representing the discrimination contents shown in Table 6 using the coordinate system of the indices 5-7.

(Indicator 1 calculation process)
Next, the index 1 calculation process (step S9 in FIG. 14) will be described with reference to FIG.
First, the up / down direction of the second image is determined by the image up / down determination unit 16a based on the posture information acquired by the posture information detection unit 40 (step S60).

  Next, the blue pixel information acquisition unit 16b acquires blue pixel information located at the upper part of the screen of the image data of the second image and within a predetermined brightness value (step S61). Here, blue is the range of the blue hue region (H4) described above. The upper part of the screen refers to the upper part of the screen according to the vertical direction of the second image determined in step S60. 26A to 26D show the positional relationship between the posture information (0 degrees, +90 degrees, −90 degrees, 180 degrees) acquired by the posture information detection unit 40 and the upper portion F of the screen. In FIGS. 26A to 26D, a portion above the area n3 shown in FIG. In addition, the predetermined brightness value range of the blue pixel information is preferably set to a brightness range close to the blue sky. For example, the luminance Y is in the range of 70 to 225 (in the range of 0 to 255 of 8 bits). .

Specifically, a blue pixel rate, a blue high saturation pixel rate, and a blue average saturation value are acquired as blue pixel information. The blue pixel ratio and the blue high saturation pixel ratio are defined by the following equations (11-1) and (11-2), respectively.
Blue pixel ratio = number of blue pixels within the specified brightness value at the top of the screen / total number of pixels at the top of the screen
(11-1)
Blue high saturation pixel ratio = the number of blue pixels within a predetermined lightness value at the top of the screen and higher than the predetermined saturation (saturation S> 75) / the total number of pixels at the top of the screen (11-2)
The blue average saturation value is an average value of the saturation of blue pixels within a predetermined brightness value at the top of the screen.

  Next, the skin color pixel information is acquired from the image data of the second image by the skin color pixel information acquisition unit 16c (step S62). Here, the skin color is a range of the hue H1 that satisfies the formula (1).

Specifically, the skin color average luminance value, the low luminance skin color pixel rate for all the pixels, and the low luminance skin color pixel rate for the skin color pixels are acquired as the skin color pixel information. The average flesh color luminance value at the center of the screen is the average value of the luminance Y of the flesh color pixels in the area composed of the areas n2 and n3 in FIG. The low luminance skin color pixel rate for all pixels and the low luminance skin color pixel rate for skin color pixels are defined by the following equations (12-1) and (12-2), respectively.
Low luminance skin color pixel ratio for all pixels = (number of skin color pixels with luminance Y ≦ 84) / total number of pixels
(12-1)
Low luminance skin color pixel ratio to skin color pixels = (number of skin color pixels with luminance Y ≦ 84) / total number of skin color pixels
(12-2)

Next, the index 1 calculation unit 16d calculates the index 1 based on the blue pixel information, the skin color pixel information, and the absolute luminance information BV (step S63). The index 1 is defined by the following formula (13).
Index 1 = Blue pixel ratio × a1
+ Blue high saturation pixel rate xa2
+ Blue average saturation value xa3
+ Skin color average luminance value in the center of the screen × b1
+ Low luminance skin color pixel ratio for all pixels × b2
+ Low luminance skin color pixel ratio for skin color pixels × b3
+ Absolute luminance information BV x c
+ D (13)
Here, a1, a2, and a3 are preset coefficients and correspond to the first coefficient recited in the claims. In addition, b1, b2, and b3 are preset coefficients and correspond to the eighth coefficient described in the claims. Also, c is a preset coefficient indicating the contribution of the absolute luminance information BV, and d is a preset constant. The index 1 is an index indicating the likelihood of blue sky.

(Tone processing condition setting)
Next, the gradation processing condition setting (step S10 in FIG. 14) for the first image after the gamma conversion processing will be described with reference to the flowchart in FIG.

  First, a gradation adjustment method for the first image data after the gamma conversion process is determined according to the determined shooting scene (step S70), and a gradation adjustment parameter is calculated (step S71). Then, a gradation conversion amount is calculated based on the gradation adjustment parameter and index 1 (step S72), and a gradation processing condition (gradation conversion curve) is set based on the gradation conversion amount (step S73). In this embodiment, the case where both the gradation adjustment method and the gradation conversion amount are determined is shown, but only the gradation conversion amount may be used.

<Determination of gradation adjustment method>
When the gradation adjustment method is determined in step S70, as shown in FIG. 28, the gradation adjustment method A (FIG. 28 (a)) is selected when the shooting scene is in the direct light, and the floor is in the backlight. When the tone adjustment method B (FIG. 28B) is selected and the main subject is over, the tone adjustment method C (FIG. 28C) is selected, and when it is under, the tone adjustment method B is selected. (FIG. 28B) is selected.

  In the present embodiment, the method of determining the gradation conversion curve to be applied to the image after the gamma conversion of the first image has been described. However, when the gradation conversion processing is performed before the gamma conversion, 28, gradation conversion is performed using a curve obtained by performing inverse conversion of gamma conversion on the gradation conversion curve of FIG.

<Calculation of gradation adjustment parameters>
When the gradation adjustment method is determined, in step S71, parameters necessary for gradation adjustment are calculated based on each index calculated in the index calculation process. Hereinafter, a method for calculating the gradation adjustment parameter will be described. In the following description, it is assumed that 8-bit captured image data is converted to 16 bits in advance, and the unit of the value of the captured image data is 16 bits.

As parameters necessary for gradation adjustment (gradation adjustment parameters), the following parameters P1 to P9 are calculated.
P1: Average luminance of the entire photographing screen P2: Block division average luminance P3: Average luminance of the skin color area (H1) P4: Luminance correction value 1 = P1-P2
P5: Reproduction target correction value = luminance reproduction target value (30360) −P4
P6: Offset value 1 = P5-P1
P7: Key correction value P7 ': Key correction value 2
P8: Brightness correction value 2
P9: Offset value 2 = P5-P8-P1

Here, a method for calculating the parameter P2 (block division average luminance) will be described with reference to FIGS.
First, in order to normalize image data, a CDF (cumulative density function) is created. Next, the maximum value and the minimum value are determined from the obtained CDF. The maximum value and the minimum value are obtained for each RGB. Here, the maximum value and the minimum value obtained for each RGB are Rmax, Rmin, Gmax, Gmin, Bmax, and Bmin, respectively.

  Next, normalized image data for any pixel (Rx, Gx, Bx) of the image data is calculated. Assuming that Rx normalization data in the R plane is Rpoint, Gx normalization data in the G plane is Gpoint, and Bx normalization data in the B plane is Bpoint, the normalization data Rpoint, Gpoint, and Bpoint respectively ) To Expression (16).

Rpoint = {(Rx−Rmin) / (Rmax−Rmin)} × 65535 (14);
Gpoint = {(Gx−Gmin) / (Gmax−Gmin)} × 65535 (15);
Bpoint = {(Bx−Bmin) / (Bmax−Bmin)} × 65535 (16).

Next, the luminance Npoint of the pixel (Rx, Gx, Bx) is calculated by Expression (17).
Npoint = (Bpoint + Gpoint + Rpoint) / 3 (17)

  FIG. 29A is a luminance frequency distribution (histogram) of RGB pixels before normalization. In FIG. 29A, the horizontal axis represents luminance, and the vertical axis represents pixel frequency. This histogram is created for each RGB. When the luminance histogram is created, normalization is performed for each plane with respect to the image data by Expressions (14) to (16). FIG. 29B shows a histogram of luminance calculated by the equation (17). Since the captured image data is normalized by 65535, each pixel takes an arbitrary value between the maximum value of 65535 and the minimum value of 0.

  When the luminance histogram shown in FIG. 29B is divided into blocks divided by a predetermined range, a frequency distribution as shown in FIG. 29C is obtained. In FIG. 29C, the horizontal axis is the block number (luminance), and the vertical axis is the frequency.

  Next, a process of deleting highlight and shadow areas is performed from the luminance histogram shown in FIG. This is because the average brightness is very high in white walls and snow scenes, and the average brightness is very low in dark scenes, so highlights and shadow areas adversely affect average brightness control. . Therefore, by limiting the highlight area and the shadow area of the luminance histogram shown in FIG. 29C, the influence of both areas is reduced. When the high luminance region (highlight region) and the low luminance region (shadow region) are deleted from the luminance histogram shown in FIG. 30A (or FIG. 29C), the result is as shown in FIG.

  Next, as shown in FIG. 30C, an area having a frequency greater than a predetermined threshold is deleted from the luminance histogram. This is because if there is a part having an extremely high frequency, the data in this part strongly affects the average luminance of the entire captured image, and thus erroneous correction is likely to occur. Therefore, as shown in FIG. 30C, the number of pixels equal to or larger than the threshold is limited in the luminance histogram. FIG. 30D is a luminance histogram after the pixel number limiting process is performed.

  Each block number of the luminance histogram (FIG. 30 (d)) obtained by deleting the high luminance region and the low luminance region from the normalized luminance histogram, and further limiting the cumulative number of pixels, and the respective frequencies The parameter P2 is obtained by calculating the average luminance based on the above.

  The parameter P1 is an average value of luminance of the entire image data, and the parameter P3 is an average value of luminance of the skin color area (H1) in the image data. The key correction value of parameter P7, the key correction value 2 of parameter P7 ', and the luminance correction value 2 of parameter P8 are defined as shown in equations (18), (19), and (20), respectively.

P7 (key correction value) = {P3-((index 7/6) × 18000 + 22000)} / 24.78 (18)
P7 ′ (key correction value 2) = {P3-((index 5/6) × 10000 + 30000)} / 24.78
(19)
P8 (Luminance correction value 2) = (Index 6/6) × 17500 (20)

<Calculation of gradation conversion amount>
Returning to FIG. 27, when the gradation adjustment parameter is calculated, the gradation conversion amount is calculated based on the gradation adjustment parameter and the index 1 in step S72. Specifically, the provisional gradation conversion amount X is selected from the gradation adjustment parameters according to the already determined shooting scene, and the provisional gradation conversion amount X is corrected based on the index 1 to convert the gradation. The quantity Z is calculated.

  Here, the calculation process of the gradation conversion amount Z based on the index 1 will be described assuming that the provisional gradation conversion amount X has already been selected. The selection of the provisional gradation conversion amount X from the gradation adjustment parameter corresponding to the shooting scene will be described later.

The gradation conversion amount Z is calculated by the following equations (21-1) to (21-3), where W is an index 1 and t is a predetermined threshold value.
When W (index 1) ≦ threshold value t Z = X (21-1)
When W (index 1)> threshold value t and X> 0, Z = X−u × (W−t) (when X−u × (W−t)> 0)
Z = 0 (when X−u × (W−t) ≦ 0) (21-2)
When W (index 1)> threshold value t and X <0, Z = X + u × (W−t) (when X + u × (W−t) <0)
Z = 0 (when X + u × (W−t) ≧ 0) (21-3)
Here, u (u> 0) is a coefficient representing the degree to which the index 1 is reflected, and is predetermined.

For example, in the case of the provisional gradation conversion amount X> 0, when W (index 1) exceeds the threshold value t, the provisional gradation conversion amount X is corrected in a direction approaching 0 according to the degree (W However, minimum 0). Further, in the case where the provisional gradation conversion amount X <0, when W (index 1) exceeds the threshold value t, the provisional gradation conversion amount X is corrected in a direction approaching 0 according to the degree (W However, maximum 0). That is, when W (index 1) is larger than the threshold value t, the gradation conversion amount Z is a value obtained by correcting the temporary gradation conversion amount X so as to suppress the effect of gradation conversion.
In this way, the gradation conversion amount Z is calculated from the provisional gradation conversion amount X, and the next gradation processing condition setting is started.

<Setting gradation processing conditions for each scene>
In step S73, a gradation conversion curve corresponding to the gradation conversion amount Z is selected (determined) from a plurality of gradation conversion curves set in advance corresponding to the already determined gradation adjustment method. Alternatively, the gradation conversion curve may be calculated based on the gradation conversion amount Z.
Hereinafter, a method for determining a gradation conversion curve for each shooting scene (light source condition and exposure condition) will be described.

<< In the case of front light >>
If the photographic scene is front light, the parameter P6 is selected as the provisional gradation conversion amount X. Since the parameter P6 = P5-P1, the provisional gradation conversion amount X = P6 means that if the gradation conversion amount Z = the provisional gradation conversion amount X, offset correction (8 bits) is performed to match P1 with P5. (Parallel shift of values) is performed by the following equation (22).
RGB value of output image = RGB value of input image + Z (22)
However, when W (index 1)> threshold value t, the correction amount is smaller than when Z = P6.

  Therefore, when the photographic scene is front light, a gradation conversion curve corresponding to Expression (22) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (22).

<< In case of backlight >>
When the shooting scene is backlit, the parameter P7 (key correction value) is selected as the provisional gradation conversion amount X. Therefore, based on the gradation conversion amount Z calculated from the provisional gradation conversion amount X, the gradation conversion curve corresponding to the gradation conversion amount Z is selected from the plurality of gradation conversion curves shown in FIG. Is selected. A specific example of the gradation conversion curve of FIG. 28B is shown in FIG. The correspondence relationship between the value of the gradation conversion amount Z and the selected gradation conversion curve is shown below.

When -50 <Z <+ 50 → L3
When + 50 ≦ Z <+150 → L4
When + 150 ≦ Z <+ 250 → L5
If -150 <Z ≤ -50 → L2
If -250 <Z ≤ -150 → L1

  When the shooting scene is backlit, it is preferable to perform a dodging process together with the gradation conversion process. In this case, it is desirable to adjust the degree of the dodging process according to the index 7 indicating the backlight intensity.

<< In case of under >>>>
If the shooting scene is under, the parameter P7 ′ (key correction value 2) is selected as the provisional gradation conversion amount X. Therefore, based on the gradation conversion amount Z calculated from the provisional gradation conversion amount X, the gradation conversion curve corresponding to the gradation conversion amount Z is selected from the plurality of gradation conversion curves shown in FIG. Is selected. Specifically, similar to the method of selecting a gradation conversion curve when the shooting scene is backlit, a gradation conversion curve corresponding to the value of the gradation conversion amount Z is selected from the gradation conversion curves shown in FIG. Selected. When the shooting scene is under, the dodging process as shown in the case of backlight is not performed.

<< When main subject is over >>
If the main subject is over, the parameter P9 is selected as the provisional gradation conversion amount X. Accordingly, based on the gradation conversion amount Z calculated from the provisional gradation conversion amount X, offset correction (parallel shift of 8-bit value) is performed by Expression (23).
RGB value of output image = RGB value of input image plus Z (23)

  Therefore, when the main subject is over, the gradation conversion curve corresponding to the gradation conversion amount Z is selected from the plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (23). If the value of the gradation conversion amount Z in Expression (23) exceeds a preset predetermined value α, the key correction value corresponds to Z−α from the curves L1 to L5 shown in FIG. A curve is selected.

  In the present embodiment bear, when the gradation conversion process is actually performed on the captured image data, the above-described gradation conversion process condition is changed from 16 bits to 8 bits.

  As described above, according to the imaging apparatus 100, the gradation processing condition is automatically set by determining the photographing scene of the photographed image, and whether or not a colorful blue sky is reflected in the image. A more appropriate image can be obtained by adding correction based on the color correction and correction based on skin color pixel information.

  Further, since the shooting scene is discriminated and the blue pixel information and skin color pixel information are acquired based on the image data of the image with the reduced image size, the processing speed can be increased.

  Note that the description in the above embodiment is an example of the imaging apparatus according to the present invention, and the present invention is not limited to this. The detailed configuration and detailed operation of each part constituting the imaging apparatus can be changed as appropriate without departing from the spirit of the present invention.

It is an external appearance block diagram of the imaging device 100 in embodiment of this invention. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 100. FIG. It is a figure for demonstrating the attitude | position information which shows the degree of inclination of an imaging device. 3 is a block diagram illustrating an internal configuration of a photographing processing unit 20. FIG. 4 is a flowchart showing exposure amount control in an AE control unit 21. It is a figure which shows a mode that an electric charge is accumulate | stored in an image pick-up element in control of shutter speed. 4 is a flowchart showing control for focusing an image in an AF control unit 22. It is a figure which shows the change of the contrast information accompanying the movement of the imaging optical system 3, and the mode of a focus position detection. 7 is a flowchart illustrating image data processing in which interpolation between pixels is performed for each RGB color component in the pixel interpolation unit 23. It is a flowchart which shows the process which automatically adjusts the white balance in a picked-up image in the AWB control part 24. It is a figure which shows the example of a gamma correction curve. 2 is a block diagram showing an internal configuration of an image processing unit 10. FIG. (A) is a figure which shows an example of the image | photographed 1st image, (b) shows an example of the 2nd image which divided | segmented the 1st image and made it the reduced image of a NxM pixel. FIG. 3 is a flowchart illustrating an overall flow of processing executed in the imaging apparatus 100. It is a flowchart which shows a scene discrimination | determination process. It is a flowchart which shows an occupation rate calculation process. It is a figure which shows the class which consists of brightness and hue. (A) is a figure which shows the area | regions n1-n3 determined according to the distance from the outer edge of a screen, (b) is a figure which shows the class which consists of area | regions n1-n3 and brightness. It is a flowchart which shows an index calculation process. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. It is a figure which shows the curve showing the 3rd coefficient by which the 1st occupation rate for calculating the parameter | index 3 is multiplied. It is a figure which shows the curve showing the 4th coefficient by which the 2nd occupation rate for calculating the parameter | index 4 is multiplied for every area | region (n1-n3). It is a flowchart which shows the parameter | index 5 calculation process. It is a plot figure which shows the relationship between an imaging | photography scene (forward light, backlight, over, under) and the indexes 5-7. It is a flowchart which shows the parameter | index 1 calculation process. It is a figure for demonstrating the screen upper part in the image data of a 2nd image. It is a flowchart which shows gradation processing condition setting. It is a figure which shows the gradation conversion curve corresponding to each gradation adjustment method. (A) is a luminance frequency distribution (histogram), (b) is a normalized histogram, and (c) is a block-divided histogram. (A) And (b) is a figure explaining the deletion of the low-intensity area | region and high-intensity area | region from the brightness | luminance histogram, (c) And (d) is a figure explaining the restriction | limiting of the frequency of a brightness | luminance. . It is a figure which shows the gradation conversion curve showing the gradation processing conditions in case a imaging | photography scene is backlight or under.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 1 Housing | casing 2 Cross key 3 Shooting optical system 4 Flash 5 Finder 6 Power switch 7 Display part 8 Release button 10 Image processing part 11 1st image acquisition part 12 2nd image acquisition part 13 Occupancy rate calculation part 14 Index calculation Unit 15 scene determination unit 16a image up / down determination unit 16b blue pixel information acquisition unit 16c skin color pixel information acquisition unit 16d index 1 calculation unit 17 gradation processing condition setting unit 18 gradation conversion processing unit 20 shooting processing unit 21 AE control unit 22 AF Control unit 23 Pixel interpolation unit 24 AWB control unit 25 Gamma correction unit 31 Processor 32 Memory 33 Image sensor 34 Image data output unit 35 Operation unit 36 Timing generator 37 Shutter unit 38 Aperture unit 39 Focus unit 40 Attitude information detection unit

Claims (15)

Scene discrimination processing means for discriminating a shooting scene based on image data of an image acquired by shooting;
A gradation processing condition setting means for setting a gradation processing condition for the image data based on the determination result of the shooting scene;
Image up-and-down determining means for acquiring posture information indicating a degree of inclination of the imaging device at the time of photographing the image and determining up and down of the image based on the posture information;
Blue pixel information acquisition means for acquiring blue pixel information located at the top of the screen of the image data and within a predetermined brightness value;
A first index calculating means for calculating a first index by multiplying at least the blue pixel information by a preset first coefficient;
With
The gradation processing condition setting unit corrects a gradation processing condition based on a result of discrimination of the shooting scene based on the first index. A second image acquisition means for acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
The imaging apparatus according to claim 1, wherein the scene determination processing unit determines a shooting scene based on image data of the second image.   The imaging apparatus according to claim 2, wherein the blue pixel information acquisition unit acquires the blue pixel information from image data of the second image. The scene discrimination processing means includes
Color information acquisition means for acquiring color information from the image data of the second image;
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the second image is classified for each classified class. First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and for each classified class, the image data of the second image is classified. Second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the entire image data;
A second index calculating means for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating means for calculating a third index by multiplying the first occupancy by a preset third coefficient;
A fourth index calculating means for calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculating means for calculating a fifth index by multiplying an average brightness of the skin color at least in the center of the screen of the image data of the second image by a preset fifth coefficient;
A sixth index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset sixth coefficient. Index calculation means;
A seventh index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset seventh coefficient. Index calculation means;
Scene discriminating means for specifying a shooting scene of the image data of the first image based on the fifth index, the sixth index, and the seventh index;
The imaging apparatus according to claim 2, further comprising: Skin color pixel information acquisition means for acquiring skin color pixel information from the image data of the second image,
The first index calculation means further calculates the first index by multiplying the flesh color pixel information by an eighth coefficient set in advance. The imaging device according to claim 1. A scene determination processing step for determining a shooting scene based on image data of an image acquired by shooting;
A gradation processing condition setting step for setting a gradation processing condition for the image data based on the determination result of the shooting scene;
An image up / down determination step of obtaining posture information indicating a degree of inclination of the imaging device at the time of capturing the image, and determining up / down of the image based on the posture information;
A blue pixel information acquisition step for acquiring blue pixel information located at the top of the screen of the image data and within a predetermined brightness value;
A first index calculation step of calculating a first index by multiplying at least the blue pixel information by a preset first coefficient;
Including
In the gradation processing condition setting step, the gradation processing condition based on the shooting scene discrimination result is corrected based on the first index. Including a second image acquisition step of acquiring a second image obtained by reducing the image size from the first image acquired by photographing,
The image processing method according to claim 6, wherein in the scene determination processing step, a shooting scene is determined based on image data of the second image.   The image processing method according to claim 7, wherein in the blue pixel information acquisition step, the blue pixel information is acquired from image data of the second image. The scene discrimination processing step includes
A color information acquisition step of acquiring color information from the image data of the second image;
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the second image is classified for each classified class. A first occupancy ratio calculating step of calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and for each classified class, the image data of the second image is classified. A second occupancy ratio calculating step of calculating a second occupancy ratio indicating the ratio of the entire image data;
A second index calculating step of calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating step of calculating a third index by multiplying the first occupancy by a preset third coefficient;
A fourth index calculating step of calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculating step of calculating a fifth index by multiplying an average brightness of the skin color at least in the center of the screen of the image data of the second image by a preset fifth coefficient;
A sixth index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset sixth coefficient. An index calculation process;
A seventh index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset seventh coefficient. An index calculation process;
A scene determination step for specifying a shooting scene of the image data of the first image based on the fifth index, the sixth index, and the seventh index;
The image processing method according to claim 7 or 8, comprising: Including a skin color pixel information acquisition step of acquiring skin color pixel information from the image data of the second image,
The said 1st parameter | index calculation process WHEREIN: Furthermore, the said 1st parameter | index is calculated by multiplying the 8th coefficient preset to the said skin color pixel information, The any one of Claim 7-9 characterized by the above-mentioned. An image processing method according to claim 1. On the computer,
A scene discrimination processing function for discriminating a shooting scene based on image data of an image acquired by shooting;
A gradation processing condition setting function for setting a gradation processing condition for the image data, based on the determination result of the shooting scene;
An image up / down determination function that acquires posture information indicating a degree of inclination of the imaging device at the time of capturing the image, and determines the up / down of the image based on the posture information;
A blue pixel information acquisition function for acquiring blue pixel information located at the top of the screen of the image data and within a predetermined brightness value;
A first index calculation function for calculating a first index by multiplying at least the blue pixel information by a preset first coefficient;
An image processing program for realizing
An image processing program that, when realizing the gradation processing condition setting function, corrects a gradation processing condition based on a determination result of the shooting scene based on the first index. In addition to the computer,
Realizing a second image acquisition function for acquiring a second image obtained by reducing the image size from the first image acquired by photographing;
12. The image processing program according to claim 11, wherein when the scene discrimination processing function is realized, a shooting scene is discriminated based on image data of the second image.   13. The image processing program according to claim 12, wherein the blue pixel information is acquired from the image data of the second image when the blue pixel information acquisition function is realized. The scene discrimination processing function is
A color information acquisition function for acquiring color information from the image data of the second image;
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the second image is classified for each classified class. A first occupancy ratio calculating function for calculating a first occupancy ratio indicating a proportion of
Based on the acquired color information, the image data of the second image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and for each classified class, the image data of the second image is classified. A second occupancy ratio calculation function for calculating a second occupancy ratio indicating a ratio of the entire image data;
A second index calculating function for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculation function for calculating a third index by multiplying the first occupancy by a preset third coefficient;
A fourth index calculating function for calculating a fourth index by multiplying the second occupancy by a preset fourth coefficient;
A fifth index calculation function for calculating a fifth index by multiplying the average brightness of the skin color at the center of the screen of the image data of at least the second image by a preset fifth coefficient;
A sixth index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset sixth coefficient. An index calculation function,
A seventh index is calculated by multiplying at least one of the second index, the third index, and the fourth index by a preset seventh coefficient. An index calculation function,
A scene determination function for specifying a shooting scene of image data of the first image based on the fifth index, the sixth index, and the seventh index;
The image processing program according to claim 12 or 13, characterized by comprising: In addition to the computer,
Realizing a skin color pixel information acquisition function for acquiring skin color pixel information from the image data of the second image;
13. When realizing the first index calculation function, the first index is calculated by multiplying the flesh color pixel information by a preset eighth coefficient. The image processing program as described in any one of -14.

Images