JP2007288245A - Imaging apparatus, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of applying proper image processing respectively to a live view display image and a recording image. <P>SOLUTION: An imaging apparatus 100 disclosed herein wherein an imaging element 33a acquires a first image for live view display and an imaging element 33b acquires a second image for recording, discriminates a photographing scene on the basis of image data of the first image and applies gradation conversion processing in response to the photographing scene to the first image, and discriminates a photographing scene on the basis of image data of the second image and applies gradation conversion processing in response to the photographing scene to the second image. Moreover, the imaging apparatus 100 calculates an exposure adjustment amount on the basis of a result of the discrimination of the photographing scene of the first image and controls the exposure when acquiring the first image for the live view display and acquiring the second image for the recording on the basis of the exposure adjustment amount. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that acquires a color image, an image processing method, and an image processing program.

  In recent years, with the widespread use of digital still cameras (including those incorporated in devices such as mobile phones and laptop computers; hereinafter abbreviated as DSC), images taken with DSC are output as hard copy images, Opportunities for viewing on a medium such as a CRT or a display device attached to a DSC are increasing.

  In general, an image photographed by DSC is often not suitable as an image for viewing due to inadequate exposure adjustment at the time of photographing.

Therefore, for example, the shooting range is divided into a plurality of areas, and brightness information for each divided area is averaged in at least one of the row direction and the column direction to obtain a plurality of average brightness information, and the brightness change in the shooting range Calculating the exposure compensation amount based on the gradient of the average luminance value within the shooting range (see Patent Document 1), or when taking a DSC image, the in-screen contrast is calculated from the divided photometry results. There has been proposed a method (see Patent Document 2) in which subject information such as intensity and luminance distribution state is obtained and tone conversion characteristics of image data are optimized from the subject information.
JP 2002-296635 A JP 2001-54014 A

  However, in the method described in Patent Document 1, it is possible to correctly determine backlight in the case of typical backlight photography. However, since only luminance information is used, when the typical composition does not apply. There was a problem that the shooting scene could not be correctly identified. In Patent Document 1, the exposure correction amount is calculated from the slope of the average luminance value, but it does not necessarily take into account the brightness of the main subject and does not quantitatively represent the shooting scene. there were. In Patent Document 2, the gradation conversion characteristics are changed depending on the contrast level. However, since the shooting scene cannot be determined with high accuracy, an appropriate gradation cannot be selected according to the shooting scene. There was a problem.

  In recent years, a single-lens reflex type DSC having an image sensor for live view display and an image sensor for recording has been proposed. Even in such a DSC, due to inadequate exposure adjustment at the time of shooting, it is often impossible to obtain an optimal image as an image to be viewed in live view and an image for recording.

  In general, such a DSC uses a recording image sensor with high resolution and low noise, and an image sensor for live view display uses a low-resolution image sensor for cost reduction. Even when an image is captured, different image data is obtained. In order to optimize the image for recording and live view display, it is important to appropriately perform appropriate image processing on each image data.

  The present invention has been made in view of the above problems in the prior art, and an object of the present invention is to perform appropriate image processing on each of the live view display image and the recording image.

  In order to solve the above-described problem, an imaging apparatus according to claim 1 includes a first imaging unit that acquires a first image for live view display using a first imaging element, and a second imaging element. A second imaging means for acquiring a second image for recording; and a scene of the first image is determined based on the image data of the first image; and the image data of the second image And a scene discrimination processing means for discriminating a shooting scene of the second image.

  According to a second aspect of the present invention, in the imaging device according to the first aspect, a gradation processing condition for the first image is based on a determination result of a shooting scene of the first image by the scene determination processing unit. The first gradation processing condition setting means for setting is provided.

  According to a third aspect of the present invention, in the imaging device according to the first or second aspect, a gradation for the second image is determined based on a determination result of a shooting scene of the second image by the scene determination processing unit. A second gradation processing condition setting means for setting a processing condition is provided.

  According to a fourth aspect of the present invention, in the imaging device according to any one of the first to third aspects, the first scene is based on a discrimination result of a shooting scene of the first image by the scene discrimination processing unit. An exposure adjustment amount calculating means for calculating an exposure adjustment amount of the imaging means, and a first exposure control for controlling the exposure of the first imaging means based on the exposure adjustment amount calculated by the exposure adjustment amount calculating means. Means.

  According to a fifth aspect of the present invention, in the imaging apparatus according to the fourth aspect, the second imaging unit is configured to perform the second adjustment based on the exposure adjustment amount calculated immediately before the second imaging unit acquires the second image. The second exposure control means for controlling the exposure of the imaging means is provided.

  According to a sixth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, a third image obtained by reducing the image size from the first image acquired by the first imaging means. And a scene discrimination processing unit for discriminating a shooting scene of the first image based on image data of the third image.

  A seventh aspect of the present invention is the imaging apparatus according to any one of the first to sixth aspects, wherein the image size is reduced from the second image acquired by the second imaging unit. And a scene determination processing unit for determining a shooting scene of the second image based on image data of the fourth image.

  According to an eighth aspect of the present invention, in the imaging apparatus according to the sixth aspect, the scene determination processing unit includes a color information acquisition unit that acquires color information from image data of the third image, and the acquired Based on the color information, the image data of the third image is classified into a class composed of a combination of predetermined brightness and hue, and the ratio of the third image to the entire image data is classified for each classified class. A first occupancy ratio calculating means for calculating a first occupancy ratio to be displayed, and a class composed of a combination of the distance from the outer edge of the screen and the brightness based on the acquired color information. Second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the third image to the entire image data for each of the classified classes, and the first occupancy ratio Multiply by preset first coefficient The first index calculating means for calculating the first index and the second index for calculating the second index by multiplying the first occupancy by a second coefficient set in advance. A calculation means; a third index calculation means for calculating a third index by multiplying the second occupancy by a preset third coefficient; and at least the image data of the third image. A fourth index calculating means for calculating a fourth index by multiplying the average luminance of the skin color in the center of the screen by a preset fourth coefficient; the first index; and the second index. And a fifth index calculating means for calculating a fifth index by multiplying at least one of the third indices by a preset fifth coefficient, the first index, At least one of the second index and the third index The sixth index calculating means for calculating the sixth index by multiplying the index by a preset sixth coefficient, the fourth index, the fifth index, and the sixth index. And a scene discriminating unit for identifying a shooting scene of the first image.

  According to a ninth aspect of the present invention, in the imaging device according to the seventh aspect, the scene discrimination processing means includes color information acquisition means for acquiring color information from image data of the fourth image, and the acquired Based on the color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined brightness and hue, and the ratio of the fourth image to the entire image data for each classified class. A first occupancy ratio calculating means for calculating a first occupancy ratio to be displayed, and a class composed of a combination of the distance from the outer edge of the screen and the brightness based on the acquired color information. Second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the fourth image to the entire image data for each of the classified classes, and the first occupancy ratio Multiply with a preset first coefficient The first index calculating means for calculating the first index and the second index for calculating the second index by multiplying the first occupancy by a second coefficient set in advance. A calculating means; a third index calculating means for calculating a third index by multiplying the second occupancy by a preset third coefficient; and at least image data of the fourth image. A fourth index calculating means for calculating a fourth index by multiplying the average luminance of the skin color in the center of the screen by a preset fourth coefficient; the first index; and the second index. And a fifth index calculating means for calculating a fifth index by multiplying at least one of the third indices by a preset fifth coefficient, the first index, At least one of the second index and the third index The sixth index calculating means for calculating the sixth index by multiplying the index by a preset sixth coefficient, the fourth index, the fifth index, and the sixth index. And a scene discriminating unit for specifying a shooting scene of the second image.

  The image processing method according to claim 10 is a first imaging step of acquiring a first image for live view display by the first imaging element, and a second recording element by the second imaging element. A shooting scene of the first image is determined based on a second imaging step of acquiring an image and image data of the first image, and the second image processing is performed based on the image data of the second image. And a scene discrimination processing step for discriminating a shooting scene of the image.

  According to an eleventh aspect of the present invention, in the image processing method according to the tenth aspect, a gradation process for the first image is performed based on a determination result of a shooting scene of the first image in the scene determination processing step. A first gradation processing condition setting step for setting conditions is included.

  According to a twelfth aspect of the present invention, in the image processing method according to the tenth or eleventh aspect of the present invention, a floor for the second image is determined based on a determination result of a shooting scene of the second image in the scene determination processing step. A second gradation processing condition setting step for setting the tone processing conditions is included.

  According to a thirteenth aspect of the present invention, in the image processing method according to any one of the tenth to twelfth aspects of the present invention, the first scene is based on a determination result of a shooting scene of the first image in the scene determination processing step. An exposure adjustment amount calculating step for calculating an exposure adjustment amount in one imaging step, and a first exposure for controlling exposure in the first imaging step based on the exposure adjustment amount calculated in the exposure adjustment amount calculating step. And a control step.

  According to a fourteenth aspect of the present invention, in the image processing method according to the thirteenth aspect, based on the exposure adjustment amount calculated immediately before obtaining the second image in the second imaging step. And a second exposure control step for controlling exposure in the second imaging step.

  A fifteenth aspect of the present invention is the image processing method according to any one of the tenth to fourteenth aspects, wherein a third image size is reduced from the first image acquired in the first imaging step. A third image acquisition step of acquiring an image, wherein the scene determination processing step determines a shooting scene of the first image based on image data of the third image;

  According to a sixteenth aspect of the present invention, in the image processing method according to any one of the tenth to fifteenth aspects, the fourth image size is reduced from the second image acquired in the second imaging step. Including a fourth image acquisition step of acquiring an image, wherein the scene determination processing step determines a shooting scene of the second image based on image data of the fourth image.

  According to a seventeenth aspect of the present invention, in the image processing method according to the fifteenth aspect, the scene determination processing step includes a color information acquisition step of acquiring color information from image data of the third image, and the acquired information. Based on the color information, the image data of the third image is classified into a class composed of a combination of a predetermined brightness and hue, and the ratio of the classified image to the entire image data of the third image A first occupancy ratio calculating step for calculating a first occupancy ratio indicating the image data of the third image based on the acquired color information and a combination of the distance from the outer edge of the screen and the brightness. A second occupancy ratio calculating step for classifying into a class and calculating a second occupancy ratio indicating the ratio of the third image to the entire image data for each of the classified classes; and the first occupancy ratio The first coefficient preset in A first index calculating step for calculating the first index by calculating, and a second index for calculating the second index by multiplying the first occupancy by a preset second coefficient. Index calculation step, a third index calculation step of calculating a third index by multiplying the second occupancy by a preset third coefficient, and an image of at least the third image A fourth index calculating step of calculating a fourth index by multiplying the average brightness of the skin color in the central portion of the screen of data by a preset fourth coefficient; the first index; and the second index A fifth index calculating step of calculating a fifth index by multiplying at least one of the third index and the third index by a preset fifth coefficient; and the first index At least of the indicator, the second indicator, and the third indicator A sixth index calculation step of calculating a sixth index by multiplying one or more indices by a preset sixth coefficient, the fourth index, the fifth index, and the sixth index And a scene determination step for specifying a shooting scene of the first image based on the index.

  According to an eighteenth aspect of the present invention, in the image processing method according to the sixteenth aspect, the scene determination processing step includes a color information acquisition step of acquiring color information from image data of the fourth image, and the acquired information. Based on the obtained color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined lightness and hue, and a ratio of the classified image class to the entire image data of the fourth image A first occupancy ratio calculating step for calculating a first occupancy ratio indicating the image data of the fourth image based on the acquired color information and a combination of the distance from the outer edge of the screen and the brightness. A second occupancy ratio calculating step for classifying into a class and calculating a second occupancy ratio indicating the ratio of the fourth image to the entire image data for each of the classified classes; and the first occupancy ratio The first coefficient preset in A first index calculating step for calculating the first index by calculating, and a second index for calculating the second index by multiplying the first occupancy by a preset second coefficient. Index calculation step, a third index calculation step of calculating a third index by multiplying the second occupancy by a preset third coefficient, and an image of at least the fourth image A fourth index calculating step of calculating a fourth index by multiplying the average brightness of the skin color in the central portion of the screen of data by a preset fourth coefficient; the first index; and the second index A fifth index calculating step of calculating a fifth index by multiplying at least one of the third index and the third index by a preset fifth coefficient; and the first index At least of the indicator, the second indicator, and the third indicator A sixth index calculation step of calculating a sixth index by multiplying one or more indices by a preset sixth coefficient, the fourth index, the fifth index, and the sixth index And a scene determination step for specifying a shooting scene of the second image based on the index.

  According to a nineteenth aspect of the present invention, a first image capturing function for acquiring a first image for live view display by a first image sensor and a second image sensor for recording by a second image sensor. A shooting scene of the first image is determined based on the second imaging function for acquiring the image of the first image and the image data of the first image, and the first scene is determined based on the image data of the second image. 2 is an image processing program for realizing a scene discrimination processing function for discriminating a shooting scene of the second image.

  According to a twentieth aspect of the present invention, in the image processing program according to the nineteenth aspect of the present invention, the computer further includes the first image based on a determination result of a shooting scene of the first image by the scene determination processing function. A first gradation processing condition setting function for setting gradation processing conditions for the image is realized.

  According to a twenty-first aspect of the present invention, in the image processing program according to the nineteenth or twentieth aspect, the computer further includes a determination result of a shooting scene of the second image by the scene determination processing function. A second gradation processing condition setting function for setting a gradation processing condition for the second image is realized.

  According to a twenty-second aspect of the present invention, in the image processing program according to any one of the nineteenth to twenty-first aspects, the computer further includes a discrimination result of the shooting scene of the first image by the scene discrimination processing function. Based on the exposure adjustment amount calculation function for calculating the exposure adjustment amount in the first imaging function, and the exposure adjustment amount calculated by the exposure adjustment amount calculation function, the exposure in the first imaging function is calculated. And a first exposure control function to be controlled.

  According to a twenty-third aspect of the present invention, in the image processing program according to the twenty-second aspect, the exposure adjustment amount calculated immediately before the computer further acquires the second image by the second imaging function. And a second exposure control function for controlling exposure in the second imaging function.

  According to a twenty-fourth aspect of the present invention, in the image processing program according to any one of the nineteenth to twenty-third aspects, an image size is further calculated from the first image acquired by the first imaging function in the computer. A third image acquisition function for acquiring a reduced third image, and the scene determination processing function determines a shooting scene of the first image based on image data of the third image It is characterized by that.

  The invention according to claim 25 is the image processing program according to any one of claims 19 to 24, wherein the image size is further calculated from the second image acquired by the second imaging function in the computer. A fourth image acquisition function for acquiring a reduced fourth image, and the scene determination processing function determines a shooting scene of the second image based on image data of the fourth image It is characterized by that.

  According to a twenty-sixth aspect of the present invention, in the image processing program according to the twenty-fourth aspect, the scene determination processing function includes a color information acquisition function for acquiring color information from image data of the third image, and the acquired information. Based on the color information, the image data of the third image is classified into a class composed of a combination of a predetermined brightness and hue, and the ratio of the classified image to the entire image data of the third image A first occupancy ratio calculation function for calculating a first occupancy ratio indicating the image data of the third image based on the acquired color information and a combination of the distance from the outer edge of the screen and the brightness. A second occupancy ratio calculation function for classifying the class into a class and calculating a second occupancy ratio indicating the ratio of the third image to the entire image data for each of the classified classes; and the first occupancy ratio The first preset to A second index is calculated by multiplying the first index calculation function for calculating the first index by multiplying the number and a second coefficient set in advance to the first occupation ratio. A second index calculating function; a third index calculating function for calculating a third index by multiplying the second occupancy by a preset third coefficient; and at least the third image. A fourth index calculation function for calculating a fourth index by multiplying the average brightness of the skin color in the center of the screen of the image data by a preset fourth coefficient, the first index, A fifth index calculation function for calculating a fifth index by multiplying at least one of the second index and the third index by a preset fifth coefficient; Less than 1 index, 2nd index and 3rd index A sixth index calculation function for calculating a sixth index by multiplying at least one index by a preset sixth coefficient, the fourth index, the fifth index, and the And a scene discrimination function for specifying a shooting scene of the first image based on a sixth index.

  According to a twenty-seventh aspect of the present invention, in the image processing program according to the twenty-fifth aspect, the scene determination processing function includes a color information acquisition function for acquiring color information from image data of the fourth image, and the acquired information. Based on the obtained color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined lightness and hue, and a ratio of the classified image class to the entire image data of the fourth image A first occupancy ratio calculating function for calculating a first occupancy ratio indicating the image data of the fourth image based on the acquired color information and a combination of the distance from the outer edge of the screen and the brightness. A second occupancy ratio calculation function that classifies the class and calculates a second occupancy ratio indicating the ratio of the fourth image to the entire image data for each of the classified classes; and the first occupancy ratio The first preset to A second index is calculated by multiplying the first index calculation function for calculating the first index by multiplying the number and a second coefficient set in advance to the first occupation ratio. A second index calculating function; a third index calculating function for calculating a third index by multiplying the second occupancy by a preset third coefficient; and at least the fourth image. A fourth index calculation function for calculating a fourth index by multiplying the average brightness of the skin color in the center of the screen of the image data by a preset fourth coefficient, the first index, A fifth index calculation function for calculating a fifth index by multiplying at least one of the second index and the third index by a preset fifth coefficient; Less than 1 index, 2nd index and 3rd index A sixth index calculation function for calculating a sixth index by multiplying at least one index by a preset sixth coefficient, the fourth index, the fifth index, and the And a scene determination function for specifying a shooting scene of the second image based on a sixth index.

  According to the first, tenth, and nineteenth aspects, the first scene and the second image are determined because the shooting scenes of the first image for live view display and the second image for recording are determined. Appropriate image processing can be performed on each of them.

  According to the second, eleventh and twentieth aspects of the present invention, the gradation processing conditions corresponding to the shooting scene of the first image are set, so that appropriate gradation conversion processing is performed on the first image. be able to.

  According to the third, twelfth, and twenty-first aspects of the present invention, since the gradation processing condition is set according to the shooting scene of the second image, an appropriate gradation conversion process is performed on the second image. be able to.

  According to the invention described in claims 4, 13 and 22, the exposure at the time of acquiring the first image is controlled based on the exposure adjustment amount according to the shooting scene of the first image. Control can be performed.

  According to the invention described in claims 5, 14, and 23, since the exposure at the time of acquiring the second image is controlled based on the exposure adjustment amount according to the shooting scene of the first image, the appropriate exposure is achieved. Control can be performed.

  According to the sixth, seventh, fifteenth, sixteenth, twenty-fourth and twenty-fifth aspects of the present invention, the shooting scene is determined based on the image data of the image whose image size is reduced, so that the processing speed can be increased. .

  According to the invention described in claims 8, 9, 17, 18, 26, and 27, it is possible to accurately specify the shooting scene.

(Configuration of Imaging Device 100)
First, the configuration of the imaging apparatus 100 according to the embodiment of the present invention will be described.
1A is a front view of the imaging apparatus 100, and FIG. 1B is a rear view of the imaging apparatus 100. The imaging device 100 is a digital camera, and has a direction selection key 2, a photographing optical system 3, a flash 4, a finder 5, a power switch 6, on the inside or the surface of a housing 1 made of a material such as metal or synthetic resin. A display unit 7 and a release button 8 are provided.

  FIG. 2 shows an internal configuration of the imaging apparatus 100. As shown in FIG. 2, the imaging apparatus 100 includes a processor 31, a memory 32, an image sensor 33a for live view display, an image sensor 33b for recording, an imaging optical system 3, an image data output unit 34, an operation unit 35, timing. The generator 36, the shutter unit 37, the aperture unit 38, the focus unit 39, and the display unit 7 are configured. The processor 31 includes an image processing unit 10 that performs image processing, and a photographing processing unit 20 that performs photographing control and processing of photographed images.

  The direction selection key 2 is a key that can select four directions, up, down, left, and right, and is used by the user to select or set various modes.

  The photographing optical system 3 includes a plurality of lenses, a lens barrel, and the like, and has a zoom function. The photographing optical system 3 forms an image of light received by the lens on imaging elements 33a and 33b such as a CCD (Charge Coupled Device). The flash 4 emits auxiliary light according to a control signal from the processor 31 when the subject brightness is low.

  The viewfinder 5 is an eyepiece type viewfinder for allowing the user to check his / her shooting target and shooting area by eyepiece. The light (subject image) incident through the lens of the photographing optical system 3 reaches the finder 5 through a half mirror and a prism (not shown). That is, the finder 5 is an optical finder.

  The power switch 6 is a switch for operating ON / OFF of the operation in the imaging apparatus 100.

  The display unit 7 is configured by a liquid crystal panel, and displays an image currently captured on the image sensor 33a, an image recorded by the image sensor 33b, a menu screen, a setting screen, and the like according to a display control signal input from the processor 31.

  The display unit 7 functions as a view finder that displays an image (live view display) acquired every predetermined time by the image sensor 33a in a shooting standby state and allows the user to check the subject image. That is, the display unit 7 is an electronic finder. Further, unlike the finder 5, the display unit 7 is a non-ocular finder for visually recognizing the display content while keeping the eyes away.

  The release button 8 is a two-step push switch that can be detected by distinguishing between a half-pressed state (preliminary shooting) and a fully-pressed state (main shooting) by the user.

  The memory 32 stores (saves) image data obtained by photographing. Further, the memory 32 stores various processing programs executed in the imaging apparatus 100, data used by the processing programs, and the like.

  The image data output unit 34 transfers and records the image data in the memory 32 to a storage medium for storage (such as an SD memory card or a multimedia card (MMC)). Alternatively, it is transferred to an external device. The image data output unit 34 is controlled by the processor 31.

  The imaging elements 33a and 33b convert the imaged light into electric charges. Thereby, for example, image data as shown in FIG. This image includes an object in the imaging range (imaging range), that is, an imaging object (imaging target) and other objects (background). The RGB value of each pixel of the entire image is represented by, for example, 256 gradations. The image sensor 33a is an image sensor for acquiring an image for live view display (first image), and the image sensor 33b is an image sensor for acquiring an image for recording (second image). is there.

  The shutter unit 37 controls the timing of resetting the image sensor 33b and the timing of charge conversion based on the state (half-pressed state or fully-pressed state) detected by the release button 8. Timing control is performed by the timing generator 36.

  The diaphragm unit 38 and / or the shutter unit 37 adjust the amount of light received by the imaging elements 33a and 33b. The focus unit 39 drives the photographic optical system 3 to execute a control operation for focusing on the photographic subject.

(Internal configuration of the imaging processing unit 20)
FIG. 3 shows an internal configuration of the imaging processing unit 20. The shooting processing unit 20 performs control related to shooting conditions during shooting and processing of shot images.

  As shown in FIG. 3, the imaging processing unit 20 includes an AE control unit 21, an AF control unit 22, a pixel interpolation unit 23, an AWB control unit 24, a gamma correction unit 25, and the like related to imaging conditions.

  The AE control unit 21 automatically controls the exposure amount at the time of image shooting. Normally, the exposure amount control during shooting standby is performed by controlling the shutter speed with the aperture open, and the exposure amount during shooting is controlled by the aperture and shutter speed.

  With reference to the flowchart of FIG. 4A, an example of exposure amount control when the AE control unit 21 acquires a live view image will be described.

  First, the diaphragm unit 38 sets the diaphragm to an open fixed diaphragm (step S111). The AE control unit 21 reads data in a predetermined photometric area from the image data obtained by the image sensor 33a (step S112), and acquires information corresponding to the luminance value (step S113). As information corresponding to the luminance value, the G value of the RGB three-color components is often used simply (hereinafter referred to as luminance value data G).

  Next, the exposure adjustment amount calculated from the scene discrimination result is acquired (step S114). The exposure adjustment amount acquired here is an exposure adjustment amount calculated based on the previous image data. In order to reduce the processing load, instead of calculating the exposure amount every time image data is acquired, the exposure adjustment amount may be calculated once every plural times for acquiring image data. The calculation of the exposure adjustment amount will be described later.

  Next, the charge accumulation time in the next frame of the image sensor 33a is set according to the luminance value data G and the exposure adjustment amount (step S115), and the next frame is set by the timing generator 36 so that the predetermined luminance level is obtained. The charge accumulation time at is controlled (step S118). This is control of the shutter speed, and FIG. 5 shows how charges are accumulated. Specifically, when the acquired luminance level is large (bright), the charge accumulation time is shortened, when the luminance level is small (dark), the charge accumulation time is lengthened. Stabilize exposure.

  As described above, the exposure amount control for the live view image is performed, and further, the gradation conversion process described later is performed, so that the photographer can view the optimum live view image on the display unit 7.

  In addition to the shutter speed control described above, the aperture control is also performed in the exposure amount control during recording image recording (see the flowchart in FIG. 4B). Similarly to the above, the luminance value data G of the photometric area is acquired (steps S112 and S113), and the exposure adjustment amount calculated from the scene discrimination result is acquired (step S114). The exposure adjustment amount acquired here is an exposure adjustment amount calculated based on the image data acquired by the image sensor 33a immediately before the recording image is captured.

  Next, a combination of the aperture value and the shutter speed (charge accumulation time) is set according to the luminance value data G and the exposure adjustment amount based on data of a predetermined program diagram (step S116). Then, the aperture unit 38 is controlled in accordance with the aperture value (step S117), and the charge accumulation time in the next frame is controlled by the timing generator 36 so as to obtain a predetermined luminance level (step S118). Specifically, when the acquired brightness level is large (bright), the aperture value is decreased, when the acquired brightness level is small (dark), the aperture value is increased. To stabilize.

  As described above, the exposure amount control is performed at the time of shooting the recording image, and further, the gradation conversion process described later is performed, so that the photographer can arbitrarily or defaultly combine the aperture value and the shutter speed. It is possible to automatically set the exposure amount for the photographed image and obtain an optimum recording image.

  The AF control unit 22 performs automatic control for focusing an image when taking an image. For example, as shown in the flowchart of FIG. 6, this in-focus control is performed by detecting the in-focus position by driving the photographing optical system 3 and stopping it according to the position.

  When driving of the photographic optical system 3 is started (step S121), data of a predetermined distance measuring area is sequentially read out from the image data obtained by the image sensor 33a by the AF control unit 22 along with the driving (step S121). In step S122, contrast information is acquired according to this data (step S123). This is for detecting whether or not the in-focus position has been reached, and is determined as follows. That is, the contrast information is set and calculated so as to depend on the sharpness of the edge portion by taking the difference between adjacent pixels in the distance measurement area data. The state that has reached the maximum is determined to be the focal point (steps S124 and S125). If it is determined that the focus position is not reached (step S125; No), the driving of the photographing optical system 3 is continued (step S126).

  FIG. 7 shows changes in contrast information accompanying the movement of the photographic optical system 3 and how the in-focus position is detected. In the above operation, a distance measurement calculation is performed in which contrast information is sequentially acquired while driving the optical system to obtain a focal position, and the photographing optical system 3 is stopped in accordance with the focal length ( Step S127).

  With such AF control, it is possible to automatically obtain a captured image that is always in focus during shooting.

  The pixel interpolation unit 23 performs inter-pixel interpolation for each color component with respect to the CCD array in which the RGB color components in the image pickup devices 33a and 33b are dispersedly arranged, and image data so that each color component value is obtained at the same pixel position. Process.

  As shown in the flowchart of FIG. 8, the pixel interpolation unit 23 masks the RGB image data (step S131) obtained by the image sensors 33a and 33b with RGB pixel filter patterns (steps S132, S134, and S136). Thereafter, average interpolation (also referred to as pixel interpolation) is performed (steps S133, S135, and S137). Among these, the G pixel filter pattern having pixels up to a high band is a median filter that performs average interpolation by substituting the average value of the intermediate binary values of the surrounding four pixels, and the R and B pixel filters The pattern performs average interpolation with respect to the same color from 9 neighboring pixels.

  The AWB control unit 24 automatically adjusts the white balance in the captured image. Assuming that the photographed image has a subject area in which RGB color balance is achieved (white when summed up), each component value of RGB of the image is achieved so as to achieve white balance in that area. Adjust the level for. This white balance processing is performed as follows, for example.

  As shown in the flowchart of FIG. 9, the AWB control unit 24 estimates a region that is supposed to be white originally from the luminance and saturation data of the image data obtained by the imaging elements 33a and 33b (step S141). (Step S142). For that region, the average intensity, G / R ratio, and G / B ratio of each RGB component value are obtained, and the R value and B value correction gains for the G value are calculated (steps S143 and S144). Based on this, gain correction for each color component in the entire image is performed (step S145).

  By such AWB control, it is possible to automatically correct the collapse of the color balance of the entire screen that occurs during shooting, and an image with a stable color tone can be obtained regardless of the actual illumination state of the subject.

  The gamma correction unit 25 performs processing for converting the gradation of the captured image so as to be suitable for the gamma characteristic of the output device.

  The gamma characteristic is a gradation characteristic, and indicates how an output gradation is set with respect to an input gradation by a correction value or a correction curve. FIG. 10 shows an example of a correction curve indicating the output value corrected for the input value. The gamma correction is a conversion process for converting an input value into an output value according to the correction value or the correction curve. Since this gradation characteristic varies depending on the output device, it is necessary to correct this gamma characteristic in order to obtain a gradation suitable for the output device. As a result, the captured linear image is converted into a non-linear image.

(Internal configuration of the image processing unit 10)
FIG. 11 shows an internal configuration of the image processing unit 10. The image processing unit 10 controls a gradation correction operation based on scene discrimination for a captured image in the imaging apparatus 100. After the processing in the imaging processing unit 20, or independently of the processing, A series of processes such as image acquisition, scene discrimination, and gradation processing condition setting are executed.

  As shown in FIG. 11, the image processing unit 10 includes a first image acquisition unit 11a, a second image acquisition unit 11b, a third image acquisition unit 12a, a fourth image acquisition unit 12b, an occupancy rate calculation unit 13, and an index calculation unit. 14, a scene determination unit 15, a gradation processing condition setting unit 16, a gradation conversion processing unit 17, and an exposure adjustment amount calculation unit 18.

  The first image acquisition unit 11a acquires the image data of the latest image captured on the image sensor 33a as the first image. In addition, the shooting conditions at the time of shooting are also acquired. The acquired image data and shooting conditions of the first image are held in the memory 32.

  The third image acquisition unit 12a divides the acquired first image into N × M rectangular regions (regions divided into M pieces in the vertical direction and N pieces in the horizontal direction). FIG. 12A shows an example of the first image, and FIG. 12B shows an example in which the first image is divided into 22 × 14 regions. The number of divided areas is not particularly limited. Each divided area corresponds to one pixel of image data of the third image. That is, the third image is an image obtained by substantially reducing the image size of the first image. This reduced image is the third image, and the third image is acquired by the above operation.

  The second image acquisition unit 11b acquires the image data of the image captured on the image sensor 33b as the second image when the release button 8 is fully pressed. In addition, the shooting conditions at the time of shooting are also acquired. The acquired image data and shooting conditions of the second image are held in the memory 32.

  The fourth image acquisition unit 12b divides the acquired second image into N × M rectangular regions (regions divided into M pieces in the vertical direction and N pieces in the horizontal direction). Each divided area corresponds to one pixel of the image data of the fourth image. That is, the fourth image is an image obtained by substantially reducing the image size of the second image. This reduced image is the fourth image, and the fourth image is acquired by the above operation.

  The occupation ratio calculation unit 13 acquires color information from the image data of the third image or the image data of the fourth image, that is, each pixel constituting the reduced image. Based on the acquired color information, each pixel of the image is classified into a predetermined class composed of a combination of lightness and hue (see FIG. 17), and for each classified class, the pixels belonging to the class are included in the entire image. A first occupation ratio indicating the occupation ratio is calculated.

  Further, the occupancy rate calculation unit 13 classifies each pixel into a predetermined class composed of a combination of the distance from the outer edge of the screen of the third image or the fourth image and the brightness (see FIG. 18), and is classified. For each class, a second occupancy ratio indicating the ratio of pixels belonging to the class to the entire image is calculated.

  The occupation rate calculation process executed in the occupation rate calculation unit 13 will be described in detail later with reference to FIG.

  The index calculation unit 14 multiplies the first occupancy calculated by the occupancy calculation unit 13 by a coefficient set in advance according to the shooting conditions, thereby specifying an index 1 and an index 2 for specifying a shooting scene. Is calculated. That is, the index calculation unit 14 functions as a first index calculation unit and a second index calculation unit.

  Also, the index calculation unit 14 multiplies the second occupancy calculated by the occupancy rate calculation unit 13 by a coefficient set in advance according to the shooting conditions, thereby obtaining an index 3 for specifying the shooting scene. calculate. That is, the index calculation unit 14 functions as a third index calculation unit.

  Further, the index calculation unit 14 captures the average brightness value (referred to as index 7) of the skin color area in the center of the screen of the third image or the fourth image, the difference value between the maximum brightness value and the average brightness value, and the like. By multiplying a coefficient set in advance according to the condition, an index 4 for specifying the shooting scene is calculated. That is, the index calculation unit 14 functions as a fourth index calculation unit.

  In addition, the index calculation unit 14 sets a coefficient set in advance for each of the average luminance value of the skin color area in the center of the screen of the third image or the fourth image, and the index 1 and the index 3, depending on the shooting conditions. A new index 5 is calculated by multiplying and taking the sum. That is, the index calculation unit 14 functions as a fifth index calculation unit.

  In addition, the index calculation unit 14 sets coefficients set in advance for the average luminance value of the skin color area in the center of the screen of the third image or the fourth image, and the index 2 and the index 3 according to the shooting conditions. A new index 6 is calculated by multiplying and taking the sum. That is, the index calculation unit 14 functions as a sixth index calculation unit.

  The index calculation process executed in the index calculation unit 14 will be described in detail later with reference to FIG.

  The scene determination unit 15 identifies the shooting scene of the first image based on each index calculated based on the image data of the third image, and calculates each of the calculated scenes based on the image data of the fourth image. Based on the index, the shooting scene of the second image is specified. Here, the shooting scene indicates a light source condition when shooting a subject such as a front light, a backlight, a proximity flash, etc., and is a main subject (mainly a person, but is not limited to this). This includes the over and under degrees. The method for determining the shooting scene will be described in detail later.

  As described above, the occupation rate calculation unit 13, the index calculation unit 14, and the scene determination unit 15 function as a scene determination processing unit.

  The gradation processing condition setting unit 16 sets gradation processing conditions for the first image based on the shooting scene of the first image determined by the scene determination unit 15, and the first determination determined by the scene determination unit 15. The gradation processing condition for the second image is set based on the shooting scene of the second image.

  The gradation processing condition setting unit 16 first calculates a gradation adjustment parameter for gradation adjustment for the first image or the second image based on each index calculated by the index calculation unit 14. A gradation processing condition for the first image or the second image is set based on the tone adjustment parameter. The setting of the gradation processing conditions will be described in detail later with reference to FIG.

  The gradation conversion processing unit 17 performs gradation conversion processing on the first image or the second image based on the gradation processing conditions set by the gradation processing condition setting unit 16.

  The exposure adjustment amount calculation unit 18 calculates an exposure adjustment amount when acquiring the first image based on the result of determining the shooting scene of the first image by the scene determination unit 15.

  In addition to the processing performed in the image processing unit 10, the processor 31 performs processing in the photographing processing unit 20 such as automatic white balance processing and gamma conversion processing, other image processing, image format conversion, image processing, and the like based on known techniques. It has a function of controlling processing operations such as data recording.

  The processing of each unit in the processor 31 is basically performed by hardware processing, but a part thereof is performed by software processing such as executing a program stored (saved) in the memory 32.

(Operation of Imaging Device 100)
Next, the operation of the imaging apparatus 100 in this embodiment will be described. Hereinafter, the photographing object is referred to as “main subject”.

  First, a live view display process executed in the imaging apparatus 100 will be described with reference to a flowchart of FIG.

  First, when the power switch 6 is turned on (when the power is turned on), preprocessing such as resetting of the memory 32 is performed. The user starts an operation for shooting by directing the imaging device 100 toward the main subject so that the main subject enters the field of the imaging device 100.

  As shown in FIG. 13, first, the AE control unit 21 calculates the AE exposure amount based on the luminance value data G of the image data obtained by the image sensor 33a (step S151).

  Next, the exposure adjustment amount calculated in step S156 is acquired (step S152). The acquisition of the exposure adjustment amount corresponds to step S114 in FIG. Then, the exposure is controlled based on the luminance value data G and the exposure adjustment amount (step S118 in FIG. 4A), and the first image acquisition unit 11a converts the image data from the image sensor 33a into the live view display first. 1 is acquired and stored in the memory 32 (step S153). The first image is a linear image (RAW image).

  Further, the first image acquisition unit 11 a acquires the shooting conditions together, and the acquired shooting conditions are held in the memory 32.

  Next, the image size of the image data of the first image is reduced and acquired as a third image by the third image acquisition unit 12a (step S154). As an actual image data size reduction method, a known technique such as a simple average, a bilinear method, or a bicubic method can be used.

  Next, based on the acquired image data of the third image, a scene discrimination process for discriminating a shooting scene is performed (step S155). The scene discrimination process will be described later with reference to FIG.

  Next, an exposure adjustment amount calculation process is performed based on each index obtained in the scene determination process and the result of shooting scene determination (step S156). The exposure adjustment amount calculation process will be described later with reference to FIG.

  Next, gradation processing condition setting for setting conditions necessary for gradation conversion processing of the first image is performed on the basis of each index obtained in the scene determination processing and the determination result of the photographic scene (step) S157). The gradation processing condition setting will be described later with reference to FIG.

  On the other hand, a gamma conversion process is performed on the first image to convert it into a non-linear image (step S158). However, the gamma conversion process is a gradation conversion process, and may be performed together with the gradation conversion process in step S159 described below.

  Next, gradation conversion processing is performed on the image data of the first image based on the gradation processing conditions set based on the third image (step S159). The gamma conversion process in step S158 is to convert to a non-linear gradation in accordance with vision, but the gradation conversion process in step S159 is to correct the influence on the gradation due to the light source condition of the shooting scene.

  Next, other image processing (noise processing, sharpness processing, etc.) is performed on the image data after the gradation conversion processing (step S160).

  Then, based on the image data after image processing, a live view image is displayed on the display unit 7 (step S161).

  Here, it is determined whether or not the power switch 6 has been turned off (step S162). If the power switch 6 has been turned off (step S162; Yes), this processing ends.

  If the power switch 6 is not turned off (step S162; No), it is determined whether or not the release button 8 has been pressed (step S163). When the release button 8 is pressed (step S163; Yes), a photographing process is performed (step S164).

  If the release button 8 is not pressed in step S163 (step S163; No) or after the photographing process (step S164), the process returns to step S151 and the process is repeated.

  Next, with reference to the flowchart of FIG. 14, the imaging process (step S164 of FIG. 13) executed in the imaging apparatus 100 will be described.

  When the release button 8 is pressed, the AE control unit 21 calculates the AE exposure amount based on the luminance value data G of the image data obtained by the image sensor 33a (step S171).

  Next, the exposure adjustment amount calculated in step S156 immediately before the photographing process is acquired (step S172). The acquisition of the exposure adjustment amount corresponds to step S114 in FIG. Then, the exposure is controlled based on the brightness value data G and the exposure adjustment amount (step S118 in FIG. 4B), and the second image acquisition unit 11b converts the image data from the image sensor 33b into a second recording image. It is acquired as an image and stored in the memory 32 (step S173). The second image is a linear image (RAW image).

  The second image acquisition unit 11b also acquires the shooting conditions. The shooting conditions are recorded together when the shot image is recorded in the Exif format (Exchangeable Image File Format) later. The acquired shooting conditions are held in the memory 32.

  Next, the image size of the image data of the second image is reduced and acquired as a fourth image by the fourth image acquisition unit 12b (step S174). As an actual image data size reduction method, a known technique such as a simple average, a bilinear method, or a bicubic method can be used.

  Next, a scene discrimination process for discriminating a shooting scene is performed based on the acquired image data of the fourth image (step S175). The scene discrimination process will be described later with reference to FIG.

  Next, gradation processing condition setting for setting conditions necessary for gradation conversion processing of the second image is performed on the basis of each index obtained in the scene determination processing and the determination result of the photographic scene (step) S176). The gradation processing condition setting will be described later with reference to FIG.

  On the other hand, a gamma conversion process is performed on the second image to convert it into a non-linear image (step S177). However, the gamma conversion process is a gradation conversion process, and may be performed together with the gradation conversion process in step S178 described below.

  Next, gradation conversion processing is performed on the image data of the second image based on the gradation processing conditions set based on the fourth image (step S178).

  Next, other image processing (noise processing, sharpness processing, etc.) is performed on the image data after the gradation conversion processing (step S179).

  Next, the image format is converted into an image such as a JPEG format, a TIFF format, or a BMP format for image recording (step S180). Then, the image data after the image format conversion is recorded on a recording medium for storage (such as an SD memory card or a multimedia card (MMC)) (step S181), and the process returns to step S151 in FIG.

(Scene discrimination processing)
Next, with reference to FIGS. 15 to 24, the scene determination process (step S155 in FIG. 13 and step S175 in FIG. 14) in the imaging apparatus 100 will be described. The scene determination process is a process of determining the shooting scene of the first image based on the image data of the third image, or a process of determining the shooting scene of the second image based on the image data of the fourth image. It is. Hereinafter, a case where the shooting scene of the first image is determined will be described as an example. The determination of the shooting scene of the second image is the same process, and thus the description thereof is omitted.

  As shown in FIG. 15, the scene determination process includes a color space conversion process (step S20), an occupation ratio calculation process (step S21), an index calculation process (step S22), and a scene determination (step S23). The Hereinafter, each process shown in FIG. 15 will be described in detail.

<Color space conversion processing>
In the color space conversion process of step S20 in FIG. 15, first, information indicating the RGB value, the luminance value, and the white balance of each pixel of the third image obtained from the captured first image is acquired. As the luminance value, a value calculated by substituting RGB values into a known conversion formula may be used. Next, the acquired RGB values are converted into the HSV color system, and the color information of the image is acquired. The HSV color system represents image data with three elements: Hue, Saturation, and Lightness (Value or Brightness), and was devised based on the color system proposed by Munsell. It has been done. The conversion to the HSV color system is performed using an HSV conversion program or the like. Usually, the calculated hue value H is defined as 0 to 360 for the input R, G, B, The unit of saturation value S and brightness value V is defined as 0-255.

  In the present embodiment, “brightness” means “brightness” which is generally used unless otherwise noted. In the following description, V (0 to 255) of the HSV color system is used as “brightness”, but a unit system representing the brightness of any other color system may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated.

  In the present embodiment, “hue” means “color” that is generally used unless otherwise noted. In the following description, H (0 to 360) of the HSV color system is used as “hue”, but for example, a color represented by a red difference value (Cr) or a blue difference value (Cb) may be used. At that time, it goes without saying that numerical values such as various coefficients described in the present embodiment are recalculated. In step S20, the values of H, S, and V obtained as described above are acquired as color information.

<Occupancy rate calculation process>
Next, the occupation rate calculation process (step S21 in FIG. 15) will be described with reference to the flowchart in FIG.

  First, based on the HSV value calculated by the color space conversion process, each pixel of the third image is classified into a predetermined class composed of a combination of hue and brightness, and the cumulative number of pixels is calculated for each classified class. As a result, a two-dimensional histogram is created (step S30).

  FIG. 17 shows a class composed of a combination of brightness and hue. In step S30, the lightness (V) is 0-5 (v1), 6-12 (v2), 13-24 (v3), 25-76 (v4), 77-109 (v5), 110-110. It is divided into seven classes of 149 (v6) and 150 to 255 (v7). As shown in FIG. 17, the brightness range in the highest brightness class is wider than the brightness range in the lowest brightness class.

  Hue (H) is a flesh color hue region (H1 and H2) having a hue value of 0 to 39 and 330 to 359, a green hue region (H3) having a hue value of 40 to 160, and a blue hue region having a hue value of 161 to 250. It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the shooting scene is small. The skin color hue region is further divided into a skin color region (H1) and another region (H2). Hereinafter, among the flesh-colored hue regions (H = 0 to 39, 330 to 359), the hue ′ (H) that satisfies the following formula (1) is defined as the flesh-colored region (H1), and the region that does not satisfy the formula (1) is ( H2).

10 <saturation (S) <175;
Hue '(H) = Hue (H) +60 (when 0 ≦ hue (H) <300);
Hue ′ (H) = Hue (H) −300 (when 300 ≦ hue (H) <360).
Luminance (Y) = R x 0.30 + G x 0.59 + B x 0.11 (A)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)

  Therefore, the number of classes when the third image is classified into classes composed of combinations of brightness and hue is 4 × 7 = 28. Moreover, it has at least three classes (v1, v2, v3) within 10% of the maximum brightness value (255). In addition, it is also possible to use lightness (V) in Formula (A) and Formula (1).

  After step S30, each pixel of the third image is classified into a predetermined class consisting of a combination of the distance from the outer edge of the screen and the brightness, and a two-dimensional histogram is calculated by calculating the cumulative number of pixels for each classified class. Is created (step S31).

  FIG. 18A shows three regions n1 to n3 divided in step S31 according to the distance from the outer edge of the screen of the third image. Region n1 is the outer frame, region n2 is the inner region of the outer frame,. A region n3 is a central region of the third image. Here, it is preferable to divide n1 to n3 so as to have substantially the same number of pixels. In the present embodiment, three divisions are used, but the present invention is not limited to this. In step S31, the brightness is divided into seven areas v1 to v7 as described above. FIG. 18B shows a class composed of combinations of three regions n1 to n3 and brightness. As shown in FIG. 18B, the number of classes when the third image is classified into classes composed of combinations of the distance from the outer edge of the screen and the brightness is 3 × 7 = 21.

  When a two-dimensional histogram is created in step S30, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of hue and brightness to the total number of pixels (N × M) Is calculated (step S32). That is, step S30 and step S32 correspond to a first occupation rate calculation step.

  Assuming that the first occupancy calculated in the class composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy in each class is expressed as shown in Table 1.

  When the two-dimensional histogram is created in step S31, the second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each predetermined class composed of a combination of the distance from the outer edge of the screen and the brightness to the total number of pixels is obtained. Calculation is performed (step S33), and the occupancy rate calculation process ends. That is, step S31 and step S33 correspond to a second occupancy rate calculating step.

  When the second occupancy calculated in each class composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each class is expressed as shown in Table 2.

  Note that a three-dimensional histogram may be created by classifying each pixel into a class consisting of a distance from the outer edge of the screen, brightness, and hue, and calculating the cumulative number of pixels for each class. Hereinafter, a method using a two-dimensional histogram is adopted.

<Indicator calculation process>
Next, the index calculation process (step S22 in FIG. 15) will be described with reference to the flowchart in FIG.

  First, the shooting scene is specified by multiplying the first occupancy calculated for each class in the occupancy calculation processing by a coefficient (first coefficient) set in advance according to the shooting conditions and taking the sum. An index 1 is calculated (step S40). The index 1 is an index representing the degree of overshoot of the main subject, and is for separating only the image that should be determined as “the main subject is over” from other shooting scenes.

Next, the first occupancy calculated for each class is multiplied by a coefficient (second coefficient) different from the first coefficient set in advance according to the shooting conditions to obtain the sum, thereby taking a picture. The index 2 for specifying the scene is calculated (step S41). The index 2 is an index that compositely represents the characteristics at the time of backlight shooting such as sky blue high brightness and low face color brightness. It is for separation.

Hereinafter, the calculation method of the index 1 and the index 2 will be described in detail.
Table 3 shows the first coefficient necessary for calculating the index 1 by class. The coefficient of each class shown in Table 3 is a weighting coefficient by which the first occupation ratio Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 3, a positive (+) coefficient is used for the first occupancy calculated from the region distributed in the skin color hue region (H1) of high brightness (v6), and the other hue is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region. FIG. 20 shows a first coefficient in the skin color area (H1) and a first coefficient in the other area (green hue area (H3)) as a curve (coefficient curve) that continuously changes over the entire brightness. It is shown. According to Table 3 and FIG. 20, in the region of high brightness (V = 77 to 150), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of both are different.

  When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (2-1) to (2-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × (−8) (2-1)
Sum of H2 regions = R12 x (-2) + R22 x (-1) + (omitted) ... + R72 x (-10) (2-2)
Sum of H3 region = R13 × 5 + R23 × (-2) + (omitted)... + R73 × (-12) (2-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)... + R74 × (-12) (2-4)

The index 1 is defined as in Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 1 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.5
(3)

  Table 4 shows the second coefficient necessary for calculating the index 2 by class. The coefficient of each class shown in Table 4 is a weighting coefficient by which the first occupation ratio Rij of each class shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  According to Table 4, a negative (-) coefficient is used for the occupancy calculated from the areas (v4, v5) distributed in the intermediate brightness of the flesh-color hue area (H1), and the flesh-color hue range (H1) is low. A coefficient 0 is used for the occupation ratio calculated from the brightness (shadow) region (v2, v3). FIG. 21 shows the second coefficient in the skin color region (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 4 and FIG. 21, the sign of the second coefficient of the intermediate lightness region having a lightness value of 25 to 150 in the flesh color hue region is negative (−), and the low lightness (shadow) region having a lightness value of 6 to 24 The second coefficient is 0, and it can be seen that there is a large difference in the coefficients in both regions.

  When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index 2 is defined as in Expression (4).

Accordingly, the sum of the H1 to H4 regions is expressed by the following formulas (4-1) to (4-4).
Sum of H1 region = R11 × 0 + R21 × 0 + (omitted)… + R71 × 2 (4-1)
Sum of H2 regions = R12 x (-2) + R22 x (-1) + (omitted) ... + R72 x 2 (4-2)
Sum of H3 region = R13 × 2 + R23 × 1 + (omitted)… + R73 × 3 (4-3)
Sum of H4 region = R14 × 0 + R24 × (−1) + (omitted)… + R74 × 3 (4-4)

The index 2 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 2 = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.7
(5)

  Since the index 1 and the index 2 are calculated based on the lightness and hue distribution amount of the third image, the index 1 and the index 2 are effective in determining a shooting scene when the image is a color image.

  When the index 1 and the index 2 are calculated, the third factor (first factor, second factor) set in advance according to the shooting condition is added to the second occupancy rate calculated for each class in the occupancy rate calculation process. The index 3 for specifying the photographic scene is calculated by multiplying the coefficient by a coefficient different from the coefficient (2) (step S42). The index 3 indicates a difference in contrast between the center and the outside of the screen of the image data between the backlight with the main subject under and the image with the main subject over.

Hereinafter, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 by class. The coefficient of each class shown in Table 5 is a weighting coefficient by which the second occupation ratio Qij of each class shown in Table 2 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 22 shows the third coefficient in the screen areas n1 to n3 as a curve (coefficient curve) that continuously changes over the entire brightness.

  If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (6).

Accordingly, the sum of the n1 to n3 regions is represented by the following formulas (6-1) to (6-3).
Sum of n1 region = Q11 × 12 + Q21 × 10 + (omitted) ... + Q71 × 0 (6-1)
Sum of n2 regions = Q12 × 5 + Q22 × 3 + (omitted) ... + Q72 × 0 (6-2)
Sum of n3 regions = Q13 × (-1) + Q23 × (-4) + (omitted) ... + Q73 × (-8) (6-3)

The index 3 is defined as in Expression (7) using the sum of the n1 to n3 regions shown in Expressions (6-1) to (6-3).
Index 3 = sum of n1 region + sum of n2 region + sum of n3 region + 0.7 (7)

  Since the index 3 is calculated based on the compositional feature (distance from the outer edge of the screen of the entire image) based on the lightness distribution position of the third image, the photographing scene of the monochrome image as well as the color image is discriminated. It is also effective.

  Further, for example, according to the focus detection area detected by a known method, the weight of the third coefficient multiplied by the second occupancy calculated from the distance from the outer edge of the screen and the predetermined class of brightness is set. By changing, it is possible to calculate an index for discriminating a scene with higher accuracy.

  When the indices 1 to 3 are calculated, the average brightness value of the flesh color region in the center of the screen of the third image, the difference value between the maximum brightness plant and the average brightness value of the third image, etc., according to the shooting conditions By multiplying a preset coefficient (fourth coefficient), an index 4 for specifying the shooting scene is calculated (step S43).

Hereinafter, the index 4 calculation process will be described in detail with reference to the flowchart of FIG.
First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the third image using the equation (A). Next, the average luminance value x1 of the skin color area at the screen center of the third image is calculated (step S50). Here, the center of the screen is an area constituted by, for example, the area n3 shown in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the third image = maximum luminance value−average luminance value is calculated (step S51).

Next, the standard deviation x3 of the luminance of the third image is calculated (step S52), and the average luminance value x4 at the center of the screen is calculated (step S53). Next, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the third image and the average luminance value Yskin_ave of the skin color area is calculated (step S54). This comparison value x5 is expressed as the following formula (8-1).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (8-1)

Next, the index 4 is calculated by multiplying each of the values x1 to x5 calculated in steps S50 to S54 by a fourth coefficient that is set in advance according to the shooting conditions to obtain a sum (step 4). S55), the index 4 calculation process ends. The index 4 is defined as the following formula (8-2).
Index 4 = 0.05 × x1 + 1.41 × x2 + (− 0.01) × x3
+ (-0.01) × x4 + 0.01 × x5-5.34 (8-2)

  This index 4 has not only the compositional characteristics of the screen of the third image but also the luminance histogram distribution information, and is particularly effective for discriminating a shooting scene in which the main subject is over and an under shooting scene.

  Returning to FIG. 19, when the index 4 is calculated, the weight coefficients (fifth) set in advance in the average luminance value of the skin color area in the center of the screen of the index 1, the index 3, and the third image are set. The index 5 is calculated by multiplying the coefficient (step S44).

  Furthermore, the index 6 is obtained by multiplying the average luminance value of the skin color area in the center of the screen of the index 2, the index 3, and the third image by a weighting coefficient (sixth coefficient) set in advance according to the shooting conditions. Is calculated (step S45), and the index calculation process is terminated.

Hereinafter, the calculation method of the index 5 and the index 6 will be described in detail.
The average luminance value of the skin color area in the center of the screen of the third image is taken as index 7. Here, the central portion of the screen is, for example, a region composed of the region n2 and the region n3 in FIG. At this time, the index 5 is defined as in Expression (9) using the index 1, index 3, and index 7, and the index 6 is defined as in Expression (10) using the index 2, index 3, and index 7. Defined.
Index 5 = 0.54 × Index 1 + 0.50 × Index 3 + 0.01 × Index 7−0.65 (9)
Indicator 6 = 0.83 x Indicator 2 + 0.23 x Indicator 3 + 0.01 x Indicator 7-1.17 (10)

  As a method of calculating the average luminance value (for example, the overall average luminance value) in FIG. 23, a simple addition average value of individual luminance data obtained from each light receiving unit of the imaging device 100 may be obtained, or imaging may be performed. Similar to the center-weighted average metering, which is often used as the metering method of the apparatus 100, the luminance data obtained from the light receiving unit near the center of the screen is highly weighted, and the luminance data obtained from the light receiving unit near the periphery of the screen is weighted. A method of obtaining the average value by lowering may be used. In addition, the luminance data obtained from the vicinity of the light receiving unit corresponding to the focus detection area is increased in weight, and the luminance data obtained from the light receiving unit distant from the focus detection position is reduced in weight to obtain an average value, etc. May be used.

<Scene discrimination>
Returning to FIG. 15, the scene discrimination (step S23) will be described.
When the indices 4 to 6 are calculated, the shooting scene is determined based on the values of these indices. Table 6 shows the details of the shooting scene discrimination based on the values of index 4, index 5, and index 6.

  FIG. 24 is a discrimination map representing the discrimination contents shown in Table 6 using the coordinate system of the indices 4-6.

(Exposure adjustment amount calculation processing)
Next, the exposure adjustment amount calculation process (step S156 in FIG. 13) will be described with reference to FIG. The exposure adjustment amount calculation process is a process of calculating the exposure adjustment amount based on the shooting scene of the first image.

  First, indices 4 to 6 calculated based on the image data of the third image obtained by reducing the image size of the first image are acquired (step S60) and determined based on the image data of the third image. A shooting scene of the first image is acquired (step S61).

Next, an exposure adjustment amount is calculated based on the indices 4 to 6 and the determined shooting scene so as to achieve appropriate exposure (step S62). The exposure adjustment amount is defined as the following formula (11).
Exposure adjustment amount = adjustment coefficient × {(index 4/6) × weight of index 4+ (index 5/6) × weight of index 5+ (index 6/6) × weight of index 6} (11)

  Table 7 shows the adjustment coefficient, the weight of the index 4, the weight of the index 5, and the weight of the index 6 in Expression (11). As shown in Table 7, the adjustment coefficient and the weight of each index are set according to the determined shooting scene. Note that the method for calculating the exposure adjustment amount is not particularly limited to the expression (11). For example, the exposure adjustment amount may be calculated based on the average luminance value and the index value of the divided area determined as the skin color in the reduced image.

  Based on the calculated exposure adjustment amount, the aperture unit 38 (or the shutter unit 37) is controlled.

(Tone processing condition setting)
Next, the gradation processing condition setting (step S157 in FIG. 13 and step S176 in FIG. 14) for the first image or the second image will be described with reference to the flowchart in FIG. The gradation processing condition setting is based on the process of setting the gradation processing condition for the first image based on the determination result of the shooting scene of the first image or the determination result of the shooting scene of the second image. This is a process for setting the gradation processing conditions for the second image. Hereinafter, a case where the gradation processing condition for the first image is set based on the determination result of the shooting scene of the first image will be described as an example. The setting of the gradation processing condition for the second image is the same process, and thus the description thereof is omitted.

  First, a gradation adjustment method for the image data of the first image after the gamma conversion process is determined according to the determined shooting scene (step S70), and a gradation adjustment parameter is calculated (step S71). Then, a gradation processing condition (gradation conversion curve) is set based on the gradation adjustment parameter (step S72).

<Determination of gradation adjustment method>
When determining the gradation adjustment method in step S70, as shown in FIG. 27, the gradation adjustment method A (FIG. 27 (a)) is selected when the shooting scene is front light, and when it is back light, the floor is adjusted. When the tone adjustment method B (FIG. 27B) is selected and the main subject is over, the tone adjustment method C (FIG. 27C) is selected, and when it is under, the tone adjustment method B is selected. (FIG. 27B) is selected.

  In the present embodiment, the method for determining the gradation conversion curve to be applied to the first image after the gamma conversion process has been described. However, when the gradation conversion process is performed before the gamma conversion process, The gradation conversion is performed with a curve obtained by performing inverse conversion of gamma conversion on the gradation conversion curve of FIG.

<Calculation of gradation adjustment parameters>
When the gradation adjustment method is determined, in step S71, parameters necessary for gradation adjustment are calculated based on each index calculated in the index calculation process. Hereinafter, a method for calculating the gradation adjustment parameter will be described. In the following description, it is assumed that 8-bit captured image data is converted to 16 bits in advance, and the unit of the value of the captured image data is 16 bits.

As parameters necessary for gradation adjustment (gradation adjustment parameters), the following parameters P1 to P9 are calculated.
P1: Average luminance of the entire photographing screen P2: Block division average luminance P3: Average luminance of the skin color area (H1) P4: Luminance correction value 1 = P1-P2
P5: Reproduction target correction value = luminance reproduction target value (30360) −P4
P6: Offset value 1 = P5-P1
P7: Key correction value P7 ': Key correction value 2
P8: Brightness correction value 2
P9: Offset value 2 = P5-P8-P1

Here, with reference to FIG. 28 and FIG. 29, a calculation method of the parameter P2 (block division average luminance) will be described.
First, in order to normalize image data, a CDF (cumulative density function) is created. Next, the maximum value and the minimum value are determined from the obtained CDF. The maximum value and the minimum value are obtained for each RGB. Here, the maximum value and the minimum value obtained for each RGB are Rmax, Rmin, Gmax, Gmin, Bmax, and Bmin, respectively.

  Next, normalized image data for any pixel (Rx, Gx, Bx) of the image data is calculated. Assuming that Rx normalization data in the R plane is Rpoint, Gx normalization data in the G plane is Gpoint, and Bx normalization data in the B plane is Bpoint, the normalization data Rpoint, Gpoint, and Bpoint are expressed by the formula (12), respectively. ) To Expression (14).

Rpoint = {(Rx−Rmin) / (Rmax−Rmin)} × 65535 (12);
Gpoint = {(Gx−Gmin) / (Gmax−Gmin)} × 65535 (13);
Bpoint = {(Bx−Bmin) / (Bmax−Bmin)} × 65535 (14).

Next, the luminance Npoint of the pixel (Rx, Gx, Bx) is calculated by Expression (15).
Npoint = (Bpoint + Gpoint + Rpoint) / 3 (15)

  FIG. 28A shows a frequency distribution (histogram) of luminance of RGB pixels before normalization. In FIG. 28A, the horizontal axis represents luminance, and the vertical axis represents pixel frequency. This histogram is created for each RGB. When the luminance histogram is created, normalization is performed for each plane with respect to the image data by Expressions (12) to (14). FIG. 28B shows a histogram of the brightness calculated by the equation (15). Since the captured image data is normalized by 65535, each pixel takes an arbitrary value between the maximum value of 65535 and the minimum value of 0.

  When the luminance histogram shown in FIG. 28B is divided into blocks divided by a predetermined range, a frequency distribution as shown in FIG. 28C is obtained. In FIG. 28C, the horizontal axis represents the block number (luminance), and the vertical axis represents the frequency.

  Next, processing for deleting highlight and shadow areas is performed from the luminance histogram shown in FIG. This is because the average brightness is very high in white walls and snow scenes, and the average brightness is very low in dark scenes, so highlights and shadow areas adversely affect average brightness control. . Therefore, by limiting the highlight area and shadow area of the luminance histogram shown in FIG. 28C, the influence of both areas is reduced. If the high brightness area (highlight area) and the low brightness area (shadow area) are deleted from the brightness histogram shown in FIG. 29A (or FIG. 28C), the result is as shown in FIG.

  Next, as shown in FIG. 29 (c), an area having a frequency greater than a predetermined threshold is deleted from the luminance histogram. This is because if there is a part having an extremely high frequency, the data in this part strongly affects the average luminance of the entire captured image, and thus erroneous correction is likely to occur. Therefore, as shown in FIG. 29C, the number of pixels equal to or greater than the threshold is limited in the luminance histogram. FIG. 29D is a luminance histogram after the pixel number limiting process is performed.

  Each block number of the luminance histogram (FIG. 29 (d)) obtained by deleting the high luminance region and the low luminance region from the normalized luminance histogram and limiting the cumulative number of pixels and the respective frequencies The parameter P2 is obtained by calculating the average luminance based on the above.

  The parameter P1 is an average value of luminance of the entire image data, and the parameter P3 is an average value of luminance of the skin color area (H1) in the image data. The key correction value of the parameter P7, the key correction value 2 of the parameter P7 ', and the luminance correction value 2 of the parameter P8 are defined as shown in Expression (16), Expression (17), and Expression (18), respectively.

P7 (key correction value) = {P3-((index 6/6) × 18000 + 22000)} / 24.78 (16)
P7 ′ (key correction value 2) = {P3-((index 4/6) × 10000 + 30000)} / 24.78
(17)
P8 (Luminance correction value 2) = (Index 5/6) × 17500 (18)

<Setting gradation processing conditions for each scene>
Returning to FIG. 26, when the tone adjustment parameters are calculated, in step S72, tone processing conditions for the photographic image data are set based on the calculated tone adjustment parameters. Specifically, a gradation conversion curve corresponding to the value of a parameter selected according to the shooting scene is selected from a plurality of gradation conversion curves set in advance corresponding to the already determined gradation adjustment method. Selected (determined). Alternatively, the gradation conversion curve may be calculated based on the parameter value.

  Hereinafter, a method for determining a gradation conversion curve (setting of gradation processing conditions in step S72) will be described for each shooting scene (light source condition and exposure condition).

<< In the case of front light >>
When the photographic scene is front light, using the parameter P6, offset correction (parallel shift of an 8-bit value) that matches P1 with P5 is performed by the following equation (19).
RGB value of output image = RGB value of input image + P6 (19)

  Therefore, when the photographic scene is front light, a gradation conversion curve corresponding to Expression (19) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (19).

<< In case of backlight >>
When the shooting scene is backlit, a gradation conversion curve corresponding to the parameter P7 (key correction value) is selected from a plurality of gradation conversion curves shown in FIG. FIG. 30 shows a specific example of the gradation conversion curve of FIG. The correspondence between the value of parameter P7 and the selected gradation conversion curve is shown below.

If -50 <P7 <+ 50 → L3
When + 50 ≦ P7 <+150 → L4
When + 150 ≦ P7 <+ 250 → L5
If -150 <P7 ≤ -50 → L2
If -250 <P7 ≤ -150 → L1

  When the shooting scene is backlit, it is preferable to perform a dodging process together with the gradation conversion process. In this case, it is desirable to adjust the degree of dodging processing according to the index 6 indicating the backlight intensity.

<< In case of under >>>>
When the shooting scene is under, a gradation conversion curve corresponding to the parameter P7 ′ (key correction value 2) is selected from the plurality of gradation conversion curves shown in FIG. Specifically, the gradation conversion curve corresponding to the value of the parameter P7 ′ is selected from the gradation conversion curves shown in FIG. 30 in the same manner as the selection method of the gradation conversion curve when the shooting scene is backlit. The When the shooting scene is under, the dodging process as shown in the case of backlight is not performed.

<< When main subject is over >>
When the main subject is over, offset correction (parallel shift of 8-bit value) is performed by the equation (20) using the parameter P9.
RGB value of output image = RGB value of input image plus P9 (20)

  Therefore, when the main subject is over, the gradation conversion curve corresponding to the parameter P9 is selected from the plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (20). When the value of parameter P9 in equation (20) exceeds a predetermined value α, a curve corresponding to the key correction value P9-α is selected from curves L1 to L5 shown in FIG. Is done.

  In the present embodiment bear, when the gradation conversion process is actually performed on the captured image data, the above-described gradation conversion process condition is changed from 16 bits to 8 bits.

  As described above, according to the imaging apparatus 100, since the shooting scenes of the first image for live view display and the second image for recording are determined, respectively, the first image and the second image are respectively determined. On the other hand, appropriate image processing can be performed.

  Specifically, since gradation processing conditions are set for the first image according to the shooting scene of the first image, appropriate gradation is set for the first image for live view display. Conversion processing can be performed. In addition, since gradation processing conditions are set for the second image according to the shooting scene of the second image, appropriate gradation conversion processing is performed on the second image for recording. Can do.

  In addition, since the exposure at the time of acquiring the live view image and the exposure at the time of shooting the recording image are controlled based on the exposure adjustment amount according to the shooting scene of the first image, appropriate exposure control is performed. be able to.

  In addition, since the photographic scene is determined based on the image data of the third image or the fourth image whose image size is reduced, the processing speed can be increased.

The description in the above embodiment is an example of the imaging apparatus according to the present invention, and the present invention is not limited to this. The detailed configuration and detailed operation of each part constituting the imaging apparatus can be changed as appropriate without departing from the spirit of the present invention.

1 is an external configuration diagram of an imaging apparatus 100 according to an embodiment of the present invention. 2 is a block diagram illustrating an internal configuration of the imaging apparatus 100. FIG. 3 is a block diagram illustrating an internal configuration of a photographing processing unit 20. FIG. 4 is a flowchart showing exposure amount control in an AE control unit 21. It is a figure which shows a mode that an electric charge is accumulate | stored in an image pick-up element in control of shutter speed. 4 is a flowchart showing control for focusing an image in an AF control unit 22. It is a figure which shows the change of the contrast information accompanying the movement of the imaging optical system 3, and the mode of a focus position detection. 7 is a flowchart illustrating image data processing in which interpolation between pixels is performed for each RGB color component in the pixel interpolation unit 23. It is a flowchart which shows the process which automatically adjusts the white balance in a picked-up image in the AWB control part 24. It is a figure which shows the example of a gamma correction curve. 2 is a block diagram showing an internal configuration of an image processing unit 10. FIG. (A) is a figure which shows an example of the image | photographed 1st image, (b) shows an example of the 3rd image which divided | segmented the 1st image and made it the reduced image of a NxM pixel. FIG. 12 is a flowchart showing live view display processing executed in the imaging apparatus 100. 3 is a flowchart illustrating imaging processing executed in the imaging apparatus 100. It is a flowchart which shows a scene discrimination | determination process. It is a flowchart which shows an occupation rate calculation process. It is a figure which shows the class which consists of brightness and hue. (A) is a figure which shows the area | regions n1-n3 determined according to the distance from the outer edge of a screen, (b) is a figure which shows the class which consists of area | regions n1-n3 and brightness. It is a flowchart which shows an index calculation process. It is a figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. It is a figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. It is a figure which shows the curve showing the 3rd coefficient by which the 2nd occupation rate for calculating the parameter | index 3 is multiplied for every area | region (n1-n3). It is a flowchart which shows the parameter | index 4 calculation process. It is a plot figure which shows the relationship between an imaging | photography scene (front light, backlight, over, under) and the indices 4-6. It is a flowchart which shows exposure adjustment amount calculation processing. It is a flowchart which shows gradation processing condition setting. It is a figure which shows the gradation conversion curve corresponding to each gradation adjustment method. (A) is a luminance frequency distribution (histogram), (b) is a normalized histogram, and (c) is a block-divided histogram. (A) And (b) is a figure explaining the deletion of the low-intensity area | region and high-intensity area | region from the brightness | luminance histogram, (c) And (d) is a figure explaining the restriction | limiting of the frequency of a brightness | luminance. . It is a figure which shows the gradation conversion curve showing the gradation processing conditions in case a imaging | photography scene is backlight or under.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Image pick-up device 1 Housing | casing 2 Direction selection key 3 Shooting optical system 4 Flash 5 Finder 6 Power switch 7 Display part 8 Release button 10 Image processing part 11a 1st image acquisition part 11b 2nd image acquisition part 12a 3rd image acquisition part 12b Fourth image acquisition unit 13 Occupancy rate calculation unit 14 Index calculation unit 15 Scene determination unit 16 Gradation processing condition setting unit 17 Gradation conversion processing unit 18 Exposure adjustment amount calculation unit 20 Imaging processing unit 21 AE control unit 22 AF control unit 23 Pixel interpolation unit 24 AWB control unit 25 Gamma correction unit 31 Processor 32 Memory 33a, 33b Image sensor 34 Image data output unit 35 Operation unit 36 Timing generator 37 Shutter unit 38 Aperture unit 39 Focus unit

Claims (27)

First imaging means for acquiring a first image for live view display by a first imaging element;
A second imaging means for acquiring a second image for recording by the second imaging element;
A scene determination process for determining a shooting scene of the first image based on the image data of the first image and determining a shooting scene of the second image based on the image data of the second image. Means,
An imaging apparatus comprising:   The image processing apparatus includes first gradation processing condition setting means for setting gradation processing conditions for the first image based on a determination result of the shooting scene of the first image by the scene determination processing means. The imaging device according to claim 1.   And a second gradation processing condition setting means for setting a gradation processing condition for the second image based on a result of determination of a shooting scene of the second image by the scene determination processing means. The imaging apparatus according to claim 1 or 2. An exposure adjustment amount calculating unit that calculates an exposure adjustment amount of the first imaging unit based on a determination result of the shooting scene of the first image by the scene determination processing unit;
First exposure control means for controlling exposure of the first imaging means based on the exposure adjustment amount calculated by the exposure adjustment amount calculation means;
The imaging apparatus according to claim 1, further comprising:   A second exposure control unit configured to control exposure of the second imaging unit based on the exposure adjustment amount calculated immediately before obtaining the second image by the second imaging unit; The imaging apparatus according to claim 4, wherein the imaging apparatus is characterized. A third image acquisition unit that acquires a third image obtained by reducing the image size from the first image acquired by the first imaging unit;
The imaging apparatus according to claim 1, wherein the scene determination processing unit determines a shooting scene of the first image based on image data of the third image. . A fourth image acquisition means for acquiring a fourth image obtained by reducing the image size from the second image acquired by the second imaging means;
The imaging apparatus according to claim 1, wherein the scene determination processing unit determines a shooting scene of the second image based on image data of the fourth image. . The scene discrimination processing means includes
Color information acquisition means for acquiring color information from image data of the third image;
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the third image is classified for each classified class. First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the third image of the third image is classified for each classified class. Second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the entire image data;
A first index calculating means for calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating means for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating means for calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating means for calculating a fourth index by multiplying the average brightness of the skin color at least in the center of the screen of the image data of the third image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. Index calculation means;
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. Index calculation means;
Scene discriminating means for specifying a shooting scene of the first image based on the fourth index, the fifth index, and the sixth index;
The imaging apparatus according to claim 6, further comprising: The scene discrimination processing means includes
Color information acquisition means for acquiring color information from image data of the fourth image;
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the fourth image is classified for each classified class. First occupancy ratio calculating means for calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the fourth image of the fourth image is classified for each classified class. Second occupancy ratio calculating means for calculating a second occupancy ratio indicating the ratio of the entire image data;
A first index calculating means for calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating means for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating means for calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating means for calculating a fourth index by multiplying the average brightness of the skin color at least in the center of the screen of the image data of the fourth image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. Index calculation means;
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. Index calculation means;
Scene discriminating means for specifying a shooting scene of the second image based on the fourth index, the fifth index, and the sixth index;
The imaging apparatus according to claim 7, further comprising: A first imaging step of acquiring a first image for live view display by the first imaging element;
A second imaging step of acquiring a second image for recording by the second imaging element;
A scene determination process for determining a shooting scene of the first image based on the image data of the first image and determining a shooting scene of the second image based on the image data of the second image. Process,
An image processing method comprising:   A first gradation processing condition setting step of setting a gradation processing condition for the first image based on a determination result of a shooting scene of the first image in the scene determination processing step; The image processing method according to claim 10.   And a second gradation processing condition setting step of setting a gradation processing condition for the second image based on a determination result of a shooting scene of the second image in the scene determination processing step. The image processing method according to claim 10 or 11. An exposure adjustment amount calculation step for calculating an exposure adjustment amount in the first imaging step based on a determination result of a shooting scene of the first image in the scene determination processing step;
A first exposure control step of controlling exposure in the first imaging step based on the exposure adjustment amount calculated in the exposure adjustment amount calculation step;
The image processing method according to claim 10, further comprising:   And a second exposure control step of controlling exposure in the second imaging step based on the exposure adjustment amount calculated immediately before obtaining the second image in the second imaging step. The image processing method according to claim 13. Including a third image acquisition step of acquiring a third image obtained by reducing the image size from the first image acquired in the first imaging step;
The image processing according to any one of claims 10 to 14, wherein, in the scene determination processing step, a shooting scene of the first image is determined based on image data of the third image. Method. Including a fourth image acquisition step of acquiring a fourth image reduced in image size from the second image acquired in the second imaging step;
The image processing according to any one of claims 10 to 15, wherein in the scene determination processing step, a shooting scene of the second image is determined based on image data of the fourth image. Method. The scene discrimination processing step includes
A color information acquisition step of acquiring color information from the image data of the third image;
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the third image is classified for each classified class. A first occupancy ratio calculating step of calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the third image of the third image is classified for each classified class. A second occupancy ratio calculating step of calculating a second occupancy ratio indicating the ratio of the entire image data;
A first index calculating step of calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating step of calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating step of calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating step of calculating a fourth index by multiplying the average brightness of the skin color at the center of the screen of the image data of the third image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. An index calculation process;
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. An index calculation process;
A scene determination step for specifying a shooting scene of the first image based on the fourth index, the fifth index, and the sixth index;
The image processing method according to claim 15, further comprising: The scene discrimination processing step includes
A color information acquisition step of acquiring color information from the image data of the fourth image;
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the fourth image is classified for each classified class. A first occupancy ratio calculating step of calculating a first occupancy ratio indicating a ratio of
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the fourth image of the fourth image is classified for each classified class. A second occupancy ratio calculating step of calculating a second occupancy ratio indicating the ratio of the entire image data;
A first index calculating step of calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating step of calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating step of calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculating step of calculating a fourth index by multiplying the average brightness of the skin color at least in the center of the screen of the image data of the fourth image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. An index calculation process;
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. An index calculation process;
A scene determination step for identifying a shooting scene of the second image based on the fourth index, the fifth index, and the sixth index;
The image processing method according to claim 16, further comprising: On the computer,
A first imaging function for acquiring a first image for live view display by the first imaging element;
A second imaging function for acquiring a second image for recording by the second imaging element;
A scene determination process for determining a shooting scene of the first image based on the image data of the first image and determining a shooting scene of the second image based on the image data of the second image. Function and
An image processing program for realizing In addition to the computer,
A first gradation processing condition setting function for setting a gradation processing condition for the first image is realized based on a determination result of a shooting scene of the first image by the scene determination processing function. The image processing program according to claim 19. In addition to the computer,
A second gradation processing condition setting function for setting a gradation processing condition for the second image is realized based on a determination result of a shooting scene of the second image by the scene determination processing function. The image processing program according to claim 19 or 20. In addition to the computer,
An exposure adjustment amount calculation function for calculating an exposure adjustment amount in the first imaging function based on a determination result of a shooting scene of the first image by the scene determination processing function;
A first exposure control function for controlling exposure in the first imaging function based on the exposure adjustment amount calculated by the exposure adjustment amount calculation function;
The image processing program according to any one of claims 19 to 21, wherein the image processing program is implemented. In addition to the computer,
Realizing a second exposure control function for controlling exposure in the second imaging function based on the exposure adjustment amount calculated immediately before obtaining the second image by the second imaging function. 23. An image processing program according to claim 22, wherein In addition to the computer,
Realizing a third image acquisition function for acquiring a third image obtained by reducing the image size from the first image acquired by the first imaging function;
The image processing according to any one of claims 19 to 23, wherein the scene determination processing function determines a shooting scene of the first image based on image data of the third image. program. In addition to the computer,
Realizing a fourth image acquisition function for acquiring a fourth image obtained by reducing the image size from the second image acquired by the second imaging function;
The image processing according to any one of claims 19 to 24, wherein the scene determination processing function determines a shooting scene of the second image based on image data of the fourth image. program. The scene discrimination processing function is
A color information acquisition function for acquiring color information from the image data of the third image;
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the third image is classified for each classified class. A first occupancy ratio calculating function for calculating a first occupancy ratio indicating a proportion of
Based on the acquired color information, the image data of the third image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the third image of the third image is classified for each classified class. A second occupancy ratio calculation function for calculating a second occupancy ratio indicating a ratio of the entire image data;
A first index calculating function for calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating function for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating function for calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculation function for calculating a fourth index by multiplying the average brightness of the skin color at least in the center of the screen of the image data of the third image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. An index calculation function,
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. An index calculation function,
A scene determination function for specifying a shooting scene of the first image based on the fourth index, the fifth index, and the sixth index;
The image processing program according to claim 24, further comprising: The scene discrimination processing function is
A color information acquisition function for acquiring color information from the image data of the fourth image;
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of a predetermined brightness and hue, and the entire image data of the fourth image is classified for each classified class. A first occupancy ratio calculating function for calculating a first occupancy ratio indicating a proportion of
Based on the acquired color information, the image data of the fourth image is classified into a class composed of a combination of the distance from the outer edge of the screen and the brightness, and the fourth image of the fourth image is classified for each classified class. A second occupancy ratio calculation function for calculating a second occupancy ratio indicating a ratio of the entire image data;
A first index calculating function for calculating a first index by multiplying the first occupancy by a preset first coefficient;
A second index calculating function for calculating a second index by multiplying the first occupancy by a preset second coefficient;
A third index calculating function for calculating a third index by multiplying the second occupancy by a preset third coefficient;
A fourth index calculation function for calculating a fourth index by multiplying the average brightness of the skin color at least in the center of the screen of the image data of the fourth image by a preset fourth coefficient;
A fifth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset fifth coefficient. An index calculation function,
A sixth index is calculated by multiplying at least one of the first index, the second index, and the third index by a preset sixth coefficient. An index calculation function,
A scene determination function for specifying a shooting scene of the second image based on the fourth index, the fifth index, and the sixth index;
The image processing program according to claim 25, further comprising:

Images