JP2010041288A - Image pickup device, image processing apparatus, imaging method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to distinguish imaged scenes with accuracy so as to capture an image subjected to more preferable chroma correction. <P>SOLUTION: A feature amount calculation section 21 separates a shot image captured in an image pickup section 12 into a plurality of areas, and calculates color ratios (R/G) and (B/G) and a luminance for each area as feature amounts. An image identification section 22 identifies the feature amounts as an area indicative of a sky image if the feature amounts are in a range of color ratios indicating the sky image and correspond to a luminance equal to a threshold or higher. The image identification section 22 sets a scene identification area on an upper portion of the shot image, and determines that the imaged scene is a scene obtained by shooting scenery, depending on the fact that the scene identification area includes an area indicative of a sky image. A correction factor setting section 25 sets a correction factor for chroma correction according to a color level of the sky when it is determined that it is a scene obtained by shooting scenery. An image processing section 30 performs image processing suitable for an image of the scenery based on this correction factor. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, an imaging method, and a program. Specifically, it is possible to correctly determine the shooting mode suitable for the captured image from the captured image.

  An imaging device such as a digital camera is provided with shooting modes (scene modes) suitable for shooting portraits, landscapes, night scenes, etc., and each shooting subject can be switched according to the shooting subject. On the other hand, an optimum image processing is performed.

  For example, in Patent Document 1, the presence / absence of a person is determined from a captured image, and when there is no person, the scene is determined to be a portrait or a commemorative photograph of the person based on the number of people, the area of the face, and the like. It is automatically performed, and optimal image creation and color creation are performed using this discrimination result.

JP 2005-173932 A

  By the way, when it is determined that the scene is captured when there is no person, if the person is not included even when the image is captured indoors, the captured scene is determined to be a landscape scene. Here, the parameters used when performing image processing on a landscape scene are set assuming outdoor imaging, so that a landscape image is obtained with respect to a captured image obtained by imaging indoors. If the image processing is performed as the scene, the optimum image processing cannot be performed.

  Therefore, the present invention provides an imaging apparatus, an image processing apparatus, an imaging method, and a program that can accurately determine an imaging scene.

The first aspect of the present invention is:
An imaging unit;
A feature amount calculation unit that divides a captured image obtained by the imaging unit into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. An image discriminating unit that discriminates from a predetermined imaging scene according to being included;
And an image processing unit that executes image processing according to a scene determination result of the image determination unit.

  In the present invention, the captured image obtained by the imaging unit is divided into a plurality of regions, and the ratio of each color, for example, the ratio (R / G) (B / G) and the luminance are calculated as feature amounts for each region. . When the calculated feature value indicates that the brightness is equal to or higher than a threshold within the range of the ratio of each color indicating the sky image that is the specific image, it is determined that the region is an area indicating the sky image. Is done. In addition, a scene determination area is set in an upper area of the captured image, and a scene that captures a predetermined captured scene, that is, a landscape is determined according to the fact that the scene determination area includes an area indicating an empty image. . In addition, the imaging scene is discriminated by weighting the scene judgment area according to the position of the area and / or weighting the area showing the sky image according to the color variation, and is included in the scene judgment area. An evaluation value is calculated according to the area indicating the empty image to be determined, and is performed according to a comparison result between the evaluation value and the threshold value.

  When the captured scene is determined to be a scene capturing a landscape, the image processing unit performs image processing suitable for the landscape image. Furthermore, when it is determined that the scene is a scene captured, a correction rate for performing saturation correction according to the sky color level is set, and saturation correction according to the correction rate is performed by the image processing unit. .

The second aspect of the present invention is
A feature amount calculation unit that divides a captured image into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. The image processing apparatus includes an image determination unit that determines a predetermined image processing mode according to the inclusion.

  In the present invention, for example, a captured image that is a RAW image is obtained. When image processing is performed on the captured image, a feature amount is calculated from the captured image, and a specific image based on the feature amount It is automatically determined whether or not the image is included in the captured image. For this reason, when a sky image is included in the captured image, it is possible to automatically perform image processing suitable for the captured image of the landscape.

The third aspect of the present invention is
A feature amount calculating unit that divides a captured image obtained by the imaging unit into a plurality of regions, and calculates a ratio and luminance of each color as a feature amount for each region;
An image determining unit determines whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is preset on the captured image. Determining a predetermined imaging scene according to being included in a scene determination area,
And an image processing unit that performs image processing on the captured image in accordance with the scene determination result.

Furthermore, the fourth aspect of the present invention is
Computer
A feature amount calculation unit that divides a captured image into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. Image discriminating means for discriminating a predetermined imaging scene according to being included;
A computer program for causing an image processing unit to perform image processing according to a scene determination result of the image determining unit.

  The computer program of the present invention is a program that can be provided by, for example, a storage medium or a communication medium provided in a computer-readable format for a general-purpose system capable of executing various program codes. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  According to the present invention, the captured image is divided into a plurality of regions, and the ratio and luminance of each color are calculated as feature amounts for each region. Based on this feature quantity, it is determined whether or not each of the plurality of regions is a region indicating a specific image, and the region indicating the specific image is included in the scene determination region set in advance on the captured image Is determined as a predetermined imaging scene. For this reason, since the captured scene is determined according to the fact that the area indicating the sky image as the specific image is included in the scene determination area set in advance on the captured image, the captured image indoors It is not discriminated as a captured image, and the imaging scene can be discriminated accurately.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a part of the configuration of the imaging apparatus 10. In the imaging device 10, the subject light is incident on the imaging unit 12 via the lens unit 11.

  The imaging unit 12 is configured using, for example, a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor of the imaging unit 12 accumulates charges according to the amount of incident light (light amount), generates a voltage signal according to the accumulated amount of charge, and generates a digital signal 13 as an image signal. To supply.

  The digital signal converting unit 13 converts the image signal supplied from the imaging unit 12 into a digital image signal and supplies the digital image signal to the feature amount calculating unit 21, the image processing unit 30, and the focus control unit 40.

  The digital image signal generated by the digital signal converting unit 13 is a signal in accordance with the color arrangement of the color filter provided on the light receiving surface of the image sensor, and is a signal (mosaic that is obtained by arranging specific color signals on the mosaic. Signal). For example, when the color filter is a Bayer array, the image signal is a signal indicating a RAW image having an R, G (Gr, Gb), B data array as shown in FIG. Become. Further, the signal of one pixel of the RAW image is composed of R, G (Gr, Gb), B image signals as shown in FIG. The color filter is not limited to the Bayer arrangement, and a color filter having a color arrangement different from the Bayer arrangement may be used.

  In the RAW image shown in FIG. 2, each pixel has a signal value of an R signal, a G (Gr, Gb) signal, and a B signal. When an output image is generated, a demosaic based on the values of individual color signals is generated. The output value for each color of each pixel is determined by the processing.

  The feature amount calculation unit 21 divides the captured image into a plurality of regions, calculates the ratio of each color as a feature amount for each region, and supplies the calculated feature amount to the image determination unit 22. In this feature amount calculation unit 21, for example, ratios (R / G) and (B / G) are used as feature amounts in order to be able to determine whether or not the segmented region is a specific image, for example, a region indicating an empty image. calculate.

  The image discriminating unit 22 discriminates whether or not each of the plurality of areas is an area showing an empty image based on the feature quantity calculated by the feature quantity calculating unit 21, and the area showing the empty image is a captured image. Based on being included in the preset scene determination area above, it is determined as a predetermined imaging scene, that is, a scene capturing a landscape. Further, in the determination of the imaging scene, it is determined whether or not it is a predetermined imaging scene using distance information supplied from a focus control unit 40 described later. Further, when the image determination unit 22 determines that the captured scene is a scene obtained by capturing a landscape, the image processing unit 30 (to be described later) performs saturation correction to improve the impression of the landscape when capturing the landscape. Then, a saturation correction rate discriminant value serving as a determination criterion for setting a saturation correction correction rate is calculated and supplied to the correction rate setting unit 25.

  The correction rate setting unit 25 sets the saturation correction rate based on the saturation correction rate determination value supplied from the image determination unit 22 and notifies the image processing unit 30 of it.

  The image processing unit 30 performs saturation correction on the image signal at the correction rate set by the correction rate setting unit 25, and supplies the image signal after the saturation correction to the storage unit or the display unit.

  The focus control unit 40 generates a lens control signal based on the image signal and supplies the lens control unit 41 with a lens control signal so that the captured image becomes clear. The lens driving unit 41 drives the lens unit 11 based on the lens control signal. Therefore, the lens unit 11 is driven by the lens driving unit 41 so that the subject image is formed on the imaging surface of the imaging unit 12. The focus control unit 40 also determines the distance to the subject based on the lens position where the captured image is clear, and supplies distance information indicating the determined distance to the image determination unit 22.

Next, a scene determination operation in the imaging apparatus 10 will be described. As shown in FIG. 3, the feature amount calculation unit 21 divides the captured image into, for example, seven regions in the horizontal direction and the vertical direction, respectively, and divides the image into 49 regions. In addition, the feature amount calculation unit 21 can determine whether or not the segmented region is a region indicating an empty image, so that the integrated value of the R signal, the G (Gr, Gb) signal, and the B signal for each region. And the ratios (R / G) and (B / G) are calculated for each region from the integrated value, and used as the feature amount. In addition, the feature amount calculation unit 21 calculates the luminance Y in order to correctly determine whether the imaging operation is performed indoors or outdoors. Formula (1) is a calculation formula for the ratio (R / G), Formula (2) is a calculation formula for the ratio (B / G), and Formula (3) is a calculation formula for the luminance Y. In equations (1) to (3), “NR” represents the number of red pixels in the region, “NGr + Gb” represents the number of green pixels in the region, and “NB” represents the number of green pixels in the region. .

  The image discriminating unit 22 discriminates whether or not each segmented area is an area showing an empty image based on the ratio (R / G) and the ratio (B / G). In addition, the image determination unit 22 sets in advance a region that is likely to include the sky when capturing a landscape as a scene determination region on the captured image, and the region indicating the sky image is the scene determination region. Is determined to be a predetermined image capturing scene, that is, a scene capturing a landscape. Further, the image determination unit 22 determines whether the imaging operation is performed indoors or outdoors based on the luminance Y calculated by the feature amount calculation unit 21 and the distance information supplied from the focus control unit 40. When an imaging operation is being performed outdoors, it is determined whether or not the imaging scene is a scene that captures a landscape.

  FIG. 4 is a flowchart showing the operation of the image discrimination unit 22. In step ST1, the image determination unit 22 determines whether or not the overall luminance of the captured image is equal to or greater than a preset threshold value TH1. The image discriminating unit 22 calculates the average luminance value of the entire captured image. If the average value is equal to or greater than the threshold value TH1, the image determination unit 22 proceeds to step ST2 assuming that the imaging operation is being performed outdoors. Completes the scene discrimination operation in the predetermined mode, assuming that an imaging operation is being performed indoors. When calculating the average value of the brightness of the entire captured image, the image determination unit 22 calculates the average value of the brightness of all regions using, for example, the brightness calculated for each region, and calculates the average value of the entire image. Let it be luminance.

  In step ST2, the image determination unit 22 determines whether or not the distance indicated by the distance information from the focus control unit 40 is equal to or greater than the threshold value TH2. When the distance is greater than or equal to the threshold value TH2, the image discriminating unit 22 proceeds to step ST3, assuming that a distant view image is being captured. This completes the scene discrimination operation.

  In step ST3, the image determination unit 22 selects one area from the areas into which the captured images are divided. The image discriminating unit 22 selects one unselected region from the 49 regions obtained by dividing the captured image into “7 × 7”, and proceeds to step ST4.

  In step ST4, the image determination unit 22 determines whether or not the selected region shows a sky color. The image discriminating unit 22 proceeds to step ST5 when the selected area shows a sky color, and proceeds to step ST7 when the selected area does not show the sky color.

  FIG. 5 is a diagram for explaining an operation for determining whether or not a selected region shows a sky color. The image discriminating unit 22 presets a color area ARsky indicating the sky color, and the coordinate position indicated by the ratio (R / G), (B / G) of the selected area is within the range of the color area ARsky. It is determined whether or not there is, and the area selected when it is within the range is set as an area indicating an empty image. In FIG. 5, black circles exemplify coordinate values for each region. For the color area ARsky, for example, a plurality of ratios (R / G) and (B / G) of images indicating the sky are obtained from a plurality of captured images obtained by capturing a landscape, and the plurality of ratios (R / G) and (B / G) are obtained. G) is statistically processed and determined.

  When determining whether or not the color of the selected area indicates the sky color, the image determination unit 22 provides an LUT (Look Up Table) indicating the color area ARsky and sets the ratio (R / G), ( When B / G) is designated, a discrimination result indicating whether or not the color is the sky is obtained. In this way, it is possible to easily determine whether or not the color of the selected area indicates a sky color regardless of the area shape of the color area ARsky.

  In step ST5, the image determination unit 22 determines whether or not the luminance of the selected region is equal to or higher than the threshold value TH3. The image discriminating unit 22 obtains a brightness range of a sky image from a plurality of sky images, and sets a threshold value TH3 in advance so that the brightness range and a brightness lower than the brightness range can be distinguished. The image discriminating unit 22 proceeds to step ST6 when the luminance of the selected area is equal to or higher than the preset threshold TH3, and proceeds to step ST7 when not higher than the threshold TH3.

  In step ST6, the image determination unit 22 counts the number of areas indicating an empty image. The image determination unit 22 determines that the selected area is an area indicating an empty image, and counts up the number of areas indicating an empty image. In other words, the image discriminating unit 22 discriminates an area indicating an empty image from the color and brightness of the selected area by the processing of step ST4 and step ST5, and the selected area is converted to an empty image by the process of step ST6. As the area to be shown, the number of areas showing this empty image is counted.

  In step ST <b> 7, the image determination unit 22 determines whether or not all areas have been selected. The image discriminating unit 22 returns to step ST3 when an unselected region remains, newly selects an unselected region, and performs the processing from step ST4. Further, the image discriminating unit 22 proceeds to step ST8 when selection of all regions is completed.

  In step ST8, the image determination unit 22 determines whether or not the scene determination area includes an area indicating an empty image. The image discriminating unit 22 proceeds to step ST9 when determining that the area indicating the empty image is included in the scene determination area, and ends the scene determining operation when determining that the area indicating the empty image is not included. To do.

  Here, when the imaging device 10 is set to the vertical position and the captured image having the vertically long composition or the imaging device 10 is set to the horizontal position and the captured image having the horizontally long composition is generated, the image determination unit 22 In order to determine whether or not an area indicating an empty image is included in the upper part of the captured image even when a captured image having the composition of FIG. 6 is generated, the scene determination area is formed in a U-shape as shown in FIG. Provide. If the scene determination area is provided in this way, when the imaging device 10 is in the vertical position, either the scene determination area Asa or the scene determination area ASc is the upper part of the captured image. When the imaging device 10 is in the horizontal position, the scene determination area ASb is the upper part of the captured image. Therefore, if the scene determination area AS is provided in this way, the scene determination area indicates the upper part of the captured image regardless of whether the imaging apparatus 10 is in the vertical position or the horizontal position.

  FIG. 7 illustrates the determination result of the scene determination area and the area indicating the sky image. When the processing shown in FIG. 4 is performed by the image discrimination unit 22, the image discrimination unit 22 discriminates an area corresponding to an empty image in the captured image as an area showing an empty image indicated by hatching.

  In step ST <b> 9, the image determination unit 22 determines that the scene determination area is a scene capturing a landscape because the scene determination area includes an area indicating an empty image, and proceeds to step ST <b> 10.

In step ST10, the image determination unit 22 calculates a correction rate determination value. The image determination unit 22 calculates a correction rate determination value for determining a correction rate for saturation correction so that the blueness of the sky and the green of the trees are more vivid and the impression of the landscape is good. The image discriminating unit 22 calculates an average value of the ratio (B / G) that is the blue level of the area discriminated as the area showing the sky image as the correction rate discrimination value. In addition, Formula (4) has shown the calculation formula of ratio (B / G) avg which is an average value of ratio (B / G). In Expression (4), “Nsky” indicates the number of areas determined to be areas indicating an empty image.

  As described above, when the scene determination operation is performed and the captured image satisfies the determination condition, the image determination unit 22 determines that the scene that is the predetermined imaging scene is captured. Further, when it is determined that the scene is an imaged scene, a correction rate determination value is calculated and supplied to the correction rate setting unit 25.

  The correction rate setting unit 25 sets a correction rate based on the correction rate discrimination value calculated by the image discrimination unit 22 and notifies the image processing unit 30 of the correction rate.

  FIG. 8 shows the relationship between the ratio (B / G) avg and the correction rate. The correction rate setting unit 25 sets the correction rate to 0% when the ratio (B / G) avg is less than the threshold value TH4, and sets the correction rate to 100% when the ratio (B / G) avg is equal to or greater than the threshold value TH5. Further, when the ratio (B / G) avg is equal to or greater than the threshold value TH4 and less than the threshold value TH5, the correction factor setting unit 25 sets the correction factor Kcs to 0% to 100% according to the value of the ratio (B / G) avg. Variable in range.

  The image processing unit 30 performs saturation correction using the correction rate Kcs set by the correction rate setting unit 25. FIG. 9 is a diagram for explaining saturation correction. When the distance from the origin 0 is increased with respect to the coordinates (r1, b1) and moved to the coordinates (r2, b2), the saturation increases. Further, when the coordinates (r1, b1) are rotated by the angle θ and moved to the coordinates (r3, b3), the hue changes. For this reason, the image processing unit 30 corrects only the saturation without changing the hue when it is determined that the scene is a scene captured.

Equation (5) represents a saturation correction equation. Since only saturation is corrected, “θ = 0” is set in equation (5). “Α” represents a preset correction coefficient.

  The image processing unit 30 corrects the correction coefficient α according to the correction rate Kcs set by the correction rate setting unit 25, and performs saturation correction using the corrected correction coefficient. Assuming that saturation correction is performed in this way, saturation correction is not performed when the ratio (B / G) avg is small, and saturation correction is performed with the correction coefficient α when the ratio (B / G) avg is large. Is done.

  As described above, the captured image is divided into a plurality of regions, and the ratio and luminance of each color is calculated as a feature amount for each region, and based on this feature amount, the region indicates an empty image for each region. If it is determined whether it is a predetermined shooting scene according to the fact that the area showing the sky image is included in the scene determination area set in advance on the picked-up image, the image taken indoors Can be correctly identified without being identified as a landscape image.

  In addition, since saturation correction is performed according to the color ratio of the area indicating the sky image, for example, when a landscape is photographed under a blue sky, the impression of the landscape is made by making the blueness of the sky and the green of the trees more vivid. Image processing can be performed so as to be favorable. In addition, when a landscape is photographed under a cloudy sky, it is possible to prevent clouds and achromatic subjects (buildings, etc.) from being unnaturally emphasized.

  By the way, in the above-described imaging device, the imaging device 10 is set to the vertical position and the captured image having the vertically long composition or the captured image having the horizontal position and the imaging device 10 being the horizontal position is generated. The case has been described where the scene determination area is provided in a U-shape so that it can be determined whether or not an area indicating an empty image is included in the upper part of the image. However, when it is possible to determine whether a captured image having a vertically long composition or a captured image having a horizontally long composition is to be generated by the imaging device, the imaging device should provide a scene determination area according to the determination result. For example, the scene determination area can be set corresponding to the captured image.

  FIG. 10 illustrates a part of another configuration of the imaging apparatus. In the imaging apparatus 10a, it is possible to determine which of a captured image having a vertically long composition or a captured image having a horizontally long composition is to be generated.

  In the imaging device 10a, the subject light is incident on the imaging unit 12 via the lens unit 11 as described above. The imaging unit 12 generates an image signal and supplies it to the digital signal converting unit 13. The digital signal converting unit 13 converts the image signal supplied from the imaging unit 12 into a digital image signal and supplies the digital image signal to the feature amount calculating unit 21, the image processing unit 30, and the focus control unit 40. The feature amount calculation unit 21 divides the captured image into a plurality of regions, calculates a feature amount for each region, and supplies the calculated feature amount to the image determination unit 22a.

  The image discriminating unit 22a discriminates an area indicating an empty image by using the feature amount calculated by the feature amount calculating unit 21, and the discrimination result, distance information supplied from the focus control unit 40, vertical and horizontal directions to be described later. Based on the determination signal supplied from the detection unit 45, it is determined whether or not it is a predetermined imaging scene. Further, the image discriminating unit 22a calculates a correction rate discriminating value and determines that it is a predetermined imaging scene, and supplies it to the correction rate setting unit 25.

  The correction rate setting unit 25 sets a correction rate based on the correction rate discrimination value supplied from the image discrimination unit 22a and notifies the image processing unit 30 of the correction rate.

  The image processing unit 30 performs saturation correction on the image signal based on the correction rate set by the correction rate setting unit 25, and supplies the image signal after the saturation correction to the storage unit or the display unit.

  The focus control unit 40 generates a lens control signal based on the image signal and supplies the lens control signal to the lens driving unit 41 so that the lens driving unit 41 drives the lens unit 11 to perform an autofocus operation. The focus control unit 40 also determines the distance to the subject based on the lens position where the captured image is clear, and supplies distance information indicating the determined distance to the image determination unit 22.

  The vertical / horizontal detection unit 45 detects whether a captured image having a vertically long composition or a captured image having a horizontally long composition is generated, and supplies a detection signal to the image determination unit 22a. The vertical / horizontal detection unit 45 is configured using an acceleration sensor, an inclination sensor, a gyro sensor, or the like, and detects whether the imaging device 10a is in the vertical position or the horizontal position. As the acceleration sensor, an acceleration sensor capable of detecting gravitational acceleration is used. If such an acceleration sensor is used, the direction of gravity can be discriminated, so that it is possible to detect whether the imaging device 10a is in the vertical position or the horizontal position. When the tilt sensor is used, if the tilt of the image pickup device 10a is detected by the tilt sensor, it is possible to detect whether the image pickup device 10a is in the vertical position or the horizontal position. In addition, even if a gyro sensor is used, the inclination of the imaging device 10a can be detected. Therefore, based on the detected inclination, it is possible to detect whether the imaging device 10a is in the vertical position or the horizontal position. Become. The vertical / horizontal detection unit 45 is not limited to generating a detection signal using a sensor, and performs image processing of a captured image to detect whether the captured image has a vertically long composition or a horizontally long composition. Thus, the detection signal may be generated.

  Next, the operation of the other image determination unit 22a will be described with reference to FIG. In step ST <b> 21, the image determination unit 22 a determines whether or not the overall luminance of the captured image is equal to or greater than a preset luminance threshold TH <b> 1. The image discriminating unit 22a calculates the average luminance value of the entire captured image, and proceeds to step ST22 when the average value is greater than or equal to the threshold value TH1, and ends the scene discrimination operation when not greater than or equal to the threshold value TH1.

  In step ST22, the image determination unit 22a determines whether or not the distance indicated by the distance information from the focus control unit 40 is greater than or equal to the threshold value TH2. The image discriminating unit 22a proceeds to step ST23 when the distance is greater than or equal to the threshold value TH2, and ends the scene discrimination operation when not greater than or equal to the threshold value TH2.

  In step ST23, the image determination unit 22a selects one area from the areas obtained by dividing the captured image, and proceeds to step ST24.

  In step ST24, the image determination unit 22a determines whether or not the selected area shows a sky color. The image discriminating unit 22a proceeds to step ST25 when the selected area indicates a sky color, and proceeds to step ST27 when the selected area does not indicate the sky color.

  In step ST25, the image determination unit 22a determines whether or not the luminance of the selected region is equal to or higher than the threshold value TH3. The image discriminating unit 22a proceeds to step ST26 when the luminance of the selected area is equal to or higher than a preset threshold TH3, and proceeds to step ST27 when not higher than the threshold TH3.

  In step ST26, the image determination unit 22a counts the number of blank image areas. The image determination unit 22a determines that the selected area is an area indicating an empty image, and counts up the number of areas indicating an empty image.

  In step ST <b> 27, the image determination unit 22 a determines whether or not all areas have been selected. The image discriminating unit 22a returns to step ST23 when an unselected region remains, newly selects an unselected region, and performs the processing from step ST24. Further, the image discriminating unit 22a proceeds to step ST28 when selection of all regions is completed.

  In step ST28, the image determination unit 22a sets a scene determination area. Based on the detection signal supplied from the vertical / horizontal detection unit 45, the image determination unit 22a sets a scene determination region above the captured image, and proceeds to step ST29. Here, when the detection signal from the vertical / horizontal detection unit 45 indicates that the imaging device 10a is in the horizontal position, the captured image has a horizontally long composition as shown in FIG. Therefore, the image determination unit 22a sets a scene determination area in the upper area of the captured image having a horizontally long composition. When the detection signal from the vertical / horizontal detection unit 45 indicates that the imaging device 10a is in the vertical position, the captured image is an image having a vertically long composition as shown in FIG. Therefore, the image determination unit 22a sets a scene determination area in the upper area of the captured image having a vertically long composition.

  In step ST29, the image determination unit 22a determines whether or not the scene determination area includes an area indicating an empty image. When the image determination unit 22a determines that the scene determination area includes an area indicating an empty image, the image determination unit 22a proceeds to step ST30, and determines that the scene determination area does not include an area indicating an empty image. The discrimination operation is terminated.

  In step ST30, the image determination unit 22a determines that the scene determination area is a scene capturing a landscape since the area indicating the sky image is included in the scene determination area, and proceeds to step ST31.

  In step ST31, the image determination unit 22a calculates a correction rate determination value. The image discriminating unit 22a calculates an average value of the ratio (B / G) which is the blue level of the area discriminated as the area showing the sky image as the correction rate discrimination value.

  If such processing is performed by the imaging apparatus 10a, the scene determination area is set according to the vertical position or the horizontal position of the imaging apparatus 10a. Therefore, the scene determination area is set in a U-shape. Compared to the above, it is possible to improve the discrimination accuracy of the empty area.

  Further, in the above-described embodiment, the case where the width of one area from the edge of the image is set as the scene determination area as the scene determination area is described. However, when the captured image is divided into a plurality of areas, the area size is reduced. If the width of one area from the edge of the image is used as the scene determination area, the area size of the scene determination area is reduced, and there is a possibility that the determination accuracy of the sky in the captured image is lowered. In addition, when the width of a plurality of areas from the edge of the image is set as the scene determination area, if the width is increased, not only the sky but also a lot of ground images are included. Therefore, by setting the width of a plurality of areas from the edge of the image as the scene determination area, the area on the upper end side of the captured image is likely to be empty, and the area located on the lower end side is less likely to be empty. For this reason, the upper weighting of the scene determination area is increased so that the empty portion in the captured image can be more reliably determined.

  Next, the operation of the image determination unit 22a when performing weighting will be described with reference to FIG. In step ST41, the image determination unit 22a determines whether or not the overall luminance of the captured image is equal to or greater than a preset luminance threshold TH1. The image discriminating unit 22a calculates the average luminance value of the entire captured image, and proceeds to step ST42 when the average value is greater than or equal to the threshold value TH1, and ends the scene discrimination operation when not greater than or equal to the threshold value TH1.

  In step ST42, the image determination unit 22a determines whether or not the distance indicated by the distance information from the focus control unit 40 is equal to or greater than the threshold value TH2. The image discriminating unit 22a proceeds to step ST43 when the distance is greater than or equal to the threshold value TH2, and ends the scene discrimination operation when it is not greater than or equal to the threshold value TH2.

  In step ST43, the image determination unit 22a selects one area from the areas obtained by dividing the captured image, and proceeds to step ST44.

  In step ST44, the image determination unit 22a determines whether or not the selected region shows a sky color. The image discriminating unit 22a proceeds to step ST45 when the selected area indicates a sky color, and proceeds to step ST47 when the selected area does not indicate the sky color.

  In step ST45, the image determination unit 22a determines whether or not the luminance of the selected region is equal to or higher than the threshold value TH3. The image discriminating unit 22a proceeds to step ST46 when the luminance of the selected area is equal to or higher than a preset threshold TH3, and proceeds to step ST47 when it is not equal to or higher than the threshold TH3.

  In step ST46, the image determination unit 22a counts the number of areas indicating an empty image. The image determination unit 22a determines that the selected area is an area indicating an empty image, and counts up the number of areas indicating an empty image.

  In step ST <b> 47, the image determination unit 22 a determines whether selection of all areas has been completed. The image discriminating unit 22a returns to step ST43 when an unselected area remains, newly selects an area that has not been selected, and performs the processing from step ST44. Further, the image discriminating unit 22a proceeds to step ST48 when selection of all the regions is completed.

  In step ST48, the image determination unit 22a sets a scene determination area. Based on the detection signal supplied from the vertical / horizontal detection unit 45, the image determination unit 22a sets a scene determination region having a plurality of widths above the captured image, and proceeds to step ST49. Here, when the detection signal from the vertical / horizontal detection unit 45 indicates that the imaging device 10a is in the horizontal position, the captured image has a horizontally long composition as shown in FIG. Therefore, as shown in FIG. 14B, the image determination unit 22a increases the weight of the upper end region of the captured image having a horizontally long composition and decreases the weight of each region in the lower end direction. Further, when the detection signal from the vertical / horizontal detection unit 45 indicates that the imaging device 10a is in the vertical position, the captured image is an image having a vertically long composition as shown in FIG. . Therefore, as shown in FIG. 15B, the image determination unit 22a increases the weight of the upper end region of the captured image having a vertically long composition and decreases the weight of each region in the lower end direction. As described above, if the scene determination area is weighted, an area that is highly likely to contain sky is highly weighted, and an area that is unlikely to contain sky is low in weight.

  If the vertical / horizontal detection unit 45 is not provided and the above-described image determination unit 22 sets a scene determination region in a U-shape, the image determination unit 22 is located on the end side of the captured image as shown in FIG. What is necessary is just to give weighting so that weighting may be made high and weighting may become low at the center part side of a captured image. In addition, when the scene determination area is set in a U-shape, the lower end area of the captured image having a horizontally long composition is less likely to include sky, and thus the weight is reduced.

In step ST49, the image determination unit 22a calculates an evaluation value SC. The image discriminating unit 22a calculates the evaluation value SC using Expression (6). In this equation (6), “Ki” indicates a determination result as to whether or not the i-th area is an empty image area. For example, when it is determined as an empty area, “1” When it is not determined as an area, it is set to “0”. “WPi” indicates the weight of the i-th region.

  In step ST50, the image determination unit 22a determines whether or not the evaluation value SC is greater than or equal to a threshold value TH6. When it is determined that the evaluation value SC is equal to or greater than the threshold value TH6, the image determination unit 22a proceeds to step ST51, and when it is determined that the evaluation value is not equal to or greater than the threshold value TH6, the scene determination operation is terminated.

  In step ST51, since the evaluation value for scene determination exceeds the threshold value, the image determination unit 22a determines that the scene is a scene captured and proceeds to step ST52.

  In step ST52, the image determination unit 22a calculates a correction rate determination value. The image discriminating unit 22a calculates an average value of the ratio (B / G) which is the blue level of the area discriminated as the area showing the sky image as the correction rate discrimination value.

  In this way, the scene determination area is weighted, and it is determined whether or not the shooting mode is set to the landscape mode based on the empty area determination result and the evaluation value SC calculated based on the weight of the scene determination area. By doing so, even if the area width of the scene determination area is widened, it is possible to accurately determine according to the captured image.

  Furthermore, the weighting is not limited to the case where it is performed on the scene determination area, and the weighting can be performed for each area where the captured image is divided. Here, when the image of the area is only an empty image, the color variation in the area is small. In addition, when the image of the region includes not only a sky image but also an image of the ground or the like, the color variation in the determination region is larger than that in the case of only the sky image. That is, it is considered that there is a high possibility that the image is an empty image when the color variation in the region is small in the region determined to be an empty image. Therefore, the image discriminating unit 22a assigns a higher weight to a region with small color variation and lowers a weight for a region with large color variation.

Expressions (7) to (9) are expressions for calculating the variation for each color, and Expression (10) is an expression for calculating the weight WC. Equations (7) to (9) show calculation formulas for obtaining dispersion for each color as color variations, and “Ravg” in Equation (7) is an average value of R signals in the region, and Equation (8). “Gavg” in FIG. 7 is an average value of G signals in the region, and “Bavg” in Equation (9) is an average value of B signals in the region. “A” in equation (10) is a coefficient. In calculating the variance of the G signal, for example, an average value of Gr and Gb in one pixel is used.

As described above, the image determination unit 22a calculates the weight for each region, and calculates the evaluation value SC from Expression (11) using this weight.

  Further, when the scene scene determination area is also weighted, the image determination unit 22a is based on color variation if the weight WC set in the area is used instead of “Ki” in Expression (6). On the basis of the evaluation value SC calculated by using the weighting and the weighting of the scene determination area, it is possible to discriminate the captured scene more accurately.

  In addition, it is considered that an area determined to indicate an empty image is likely to be an empty image when it is determined that the surrounding area indicates an empty image. Therefore, the image discriminating unit 22a can also weight the weight for each area according to the weighting result of the surrounding area.

Expression (12) represents a calculation expression when the weight for each area is further weighted according to the weighting result of the surrounding area.

  For example, as shown in FIG. 17, when the weights of the determination regions located on the top, bottom, left, and right are “w1” to “w4”, the weight Wi of the region APi is calculated based on Expression (12). In Equation (11), “wn” represents the weight set for the nth neighboring region, “N” represents the number of regions located in the vicinity, and in the case shown in FIG. 17, Wn is “w1”. ] To "w4", N is 4.

  In this way, if the weighting is further performed using the weighting result of the surrounding area, the weight of the area indicating the empty image can be further increased. Therefore, the evaluation value SC is calculated using the weight set for each region in this way in place of “Ki” in Expression (6). As described above, if weighting is performed according to the weighting result of the surrounding area, it is possible to more accurately determine whether the scene is a scene captured.

  By the way, when weighting is performed using the surrounding area, it is not limited to the case where the weight set in the surrounding area is used. For example, weighting can be performed using the determination result of whether or not the surrounding area is an area indicating an empty image.

Expression (13) shows a case where weighting is performed based on the determination result of the surrounding area.

  For example, as shown in FIG. 18, when the determination result is obtained and the determination results of the four determination regions located in the upper, lower, left, and right directions are used, the region APn1 is determined that the left and lower regions are not empty images (determination result). In the case where it is determined that the upper and right areas are empty images (determination result “1”), the determination result of the area APn1 is weighted based on the determination result of the surrounding area. And “1 × (2/4) = 0.5”. In addition, since the area APn2 is determined to be an area in which the upper, lower, left, and right areas indicate an empty image (determination result “1”), the determination result of the area APn2 is weighted and “1 × (4/4) = 1 ”.

  In this manner, weighting is performed based on the determination result of the surrounding area, and the evaluation value SC is calculated from the equation (12) using the weighted determination result of the area. When the scene determination area is also weighted, the image determination unit 22a uses the determination result after weighting instead of “Ki” in Expression (6). In this way, it is possible to calculate the evaluation value SC that is weighted using the determination result of the surrounding area, even if the weight is not set based on the color variation in the area.

  As described above, the image determination unit 22a calculates the evaluation value SC by setting the scene determination region, weighting the region, and the like based on the detection signal indicating that the imaging device is in the vertical position, the horizontal position, or the like. Further, the image determination unit 22a determines whether or not the shooting scene is a scene capturing a landscape based on the evaluation value SC, and calculates a saturation correction rate determination value when determining that the scene is a scene capturing a landscape. , And supplied to the correction rate setting unit 25.

  The correction rate setting unit 25 determines the correction rate as described above based on the saturation correction rate determination value calculated by the image determination unit 22a and supplies the correction rate to the image processing unit 30. The image processing unit 30 performs saturation correction using the correction rate Kcs set by the correction rate setting unit 25.

  As described above, in the imaging apparatus 10a, the setting of the scene determination area based on the detection signal indicating that the imaging apparatus is in the vertical position, the horizontal position, etc., the weighting of the scene determination area, and the color of each area obtained by dividing the captured image Since either weighting based on variation, weighting using adjacent areas, or a combination of multiple processes is performed, it is possible to accurately detect an empty area from a captured image and capture a landscape. The scene can be correctly identified.

  Also, when it is determined that the scene is a scene captured, the saturation correction correction rate is controlled according to the color of the area indicating the sky image, so that the optimum saturation correction can be performed as described above. become able to.

  The above-described embodiment has been described with respect to the case where the shooting mode is set and the saturation correction is performed by the imaging device. . For example, when a RAW image is recorded on a recording medium and image processing of the RAW image is performed by the image processing device, the image processing device divides the captured image into a plurality of regions as described above, and the feature amount for each region. Is determined based on the feature amount to determine whether or not it is a region indicating a sky image, and is determined as a predetermined imaging scene according to the fact that a region indicating a sky image is included in the scene determination region. . Furthermore, the image processing apparatus automatically performs, for example, the above-described saturation correction as an image processing mode suitable for the image of the determined imaging scene. In this way, even if the image recorded on the recording medium is a RAW image, the image supplied to the display unit or the like is subjected to image processing in an image processing mode suitable for a scene image capturing a landscape. It becomes a broken image. Therefore, it is not necessary for the user to perform an image processing setting operation or the like for each captured image, and a good landscape image can be easily displayed.

  In addition, the above-described processing, that is, the processing for determining the shooting mode and the image processing mode from the captured image, and the processing for further setting the correction factor can be executed not only by hardware but also by software or a combined configuration of both. When executing processing by software, a computer program recording a processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or a general-purpose computer capable of executing various types of processing is used. A computer program can be installed and executed.

  The computer program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the computer program can be temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. Stored (recorded).

  The computer program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. Then, the computer can receive the computer program transferred in this manner and install it in a recording medium such as a built-in hard disk.

  The present invention is not limited to the above-described embodiment, and it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention. That is, in order to determine the gist of the present invention, the claims should be taken into consideration.

  In the present invention, a captured image is divided into a plurality of regions, and the ratio and luminance of each color is calculated as a feature amount for each region, and an empty image is shown for each of the plurality of regions based on the feature amount. It is determined whether or not there is an image, and a predetermined shooting mode is set in accordance with the fact that an area showing an empty image is included in a scene determination area set in advance on the captured image. For this reason, it becomes possible to correctly determine the shooting mode suitable for the captured image from the captured image, and for example, an indoor captured image is not determined to be an outdoor captured image. Therefore, the present invention is suitable for an imaging device that obtains a captured image of a still image or a moving image or an image processing device that performs image processing of a captured image.

It is a figure which shows a part of structure of an imaging device. It is a figure which shows a RAW image. It is a figure for demonstrating operation | movement of the feature-value calculation part. It is a flowchart which shows operation | movement of an image discrimination | determination part. It is a figure for demonstrating the discrimination | determination operation | movement of whether the selected area | region has shown the sky color. It is the figure which illustrated the scene judgment area. It is the figure which illustrated the determination result of the area | region which shows a scene determination area | region and an empty image. It is a figure which shows the relationship between ratio (B / G) avg and a correction factor. It is a figure for demonstrating saturation correction. It is a figure which shows a part of other structure of an imaging device. It is a flowchart which shows operation | movement of another image discrimination | determination part. It is a figure which shows the scene judgment area | region set according to the vertical / horizontal detection result. It is a flowchart which shows operation | movement of the image discrimination | determination part in the case of performing weighting. FIG. It is a figure which shows the weighting of the scene determination area | region in the captured image which is a landscape composition. It is a figure which shows the weighting of the scene judgment area | region in the captured image which is a vertically long composition. It is a figure which shows the weighting with respect to a U-shaped scene determination area | region. It is a figure for demonstrating the weighting using the weight of an adjacent area | region. It is a figure for demonstrating the weighting using the discrimination | determination result of an adjacent area | region.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10,10a ... Imaging device, 11 ... Lens part, 12 ... Imaging part, 13 ... Digital signal conversion part, 21 ... Feature-value calculation part, 22, 22a ... Image discrimination | determination part 25 ... Correction rate setting unit, 30 ... Image processing unit, 40 ... Focus control unit, 41 ... Lens drive unit, 45 ... Vertical / horizontal detection unit

Claims (15)

An imaging unit;
A feature amount calculation unit that divides a captured image obtained by the imaging unit into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. An image discriminating unit that discriminates from a predetermined imaging scene according to being included;
And an image processing unit that executes image processing according to a scene determination result of the image determination unit. The image discriminating unit weights the scene determination area according to the position of the area and / or weights the area indicating the specific image according to color variation, and then applies the weight to the scene determination area. The imaging apparatus according to claim 1, wherein an evaluation value is calculated according to an area indicating the specific image included, and the predetermined imaging scene is determined when the evaluation value is higher than a preset threshold value. The area of the specific image is an area indicating an empty image;
The imaging apparatus according to claim 2, wherein the image determination unit sets the scene determination area above the captured image. The imaging apparatus according to claim 3, wherein the image determination unit increases the weight of the upper part of the captured image with respect to the determination region. The imaging apparatus according to claim 3, wherein the image determination unit increases the weighting of an area where color dispersion is small in an area indicating the sky image. The image discriminating unit determines whether the area indicating the sky image is an area indicating the sky image in an area adjacent to the area or a weighting result according to the color variation. The imaging device according to claim 3, wherein weighting based on the imaging device is performed. A vertical / horizontal detection unit that detects whether the imaging device is vertical or horizontal and generates a detection signal,
The imaging apparatus according to claim 1, wherein the image determination unit sets the scene determination area based on the detection signal. The image determination unit predetermines a ratio range of each color indicating the sky image, and indicates the sky image when the ratio is included in the range and the luminance is equal to or greater than a preset threshold value. The imaging device according to claim 3, wherein the imaging device is determined to be a region. The imaging apparatus according to claim 3, wherein the image discrimination unit discriminates the predetermined imaging scene when the luminance of the captured image and the distance to the subject are equal to or greater than a predetermined threshold. The imaging apparatus according to claim 3, wherein the predetermined imaging scene is a scene obtained by imaging a landscape. A correction rate setting unit that sets a correction rate when performing saturation correction on the captured image in the image processing unit;
The image processing unit calculates the sky color level from a color of an area determined to represent the sky image;
The imaging apparatus according to claim 3, wherein the correction rate setting unit sets the correction rate so that saturation is high when the sky color level is high. The image pickup apparatus according to claim 11, wherein the sky color level is an average value of blue levels of a determination area determined to indicate the sky image. A feature amount calculation unit that divides a captured image into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. An image processing apparatus having an image determination unit that determines a predetermined image processing mode according to being included. A feature amount calculating unit that divides a captured image obtained by the imaging unit into a plurality of regions, and calculates a ratio and luminance of each color as a feature amount for each region;
An image determining unit determines whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is preset on the captured image. Determining a predetermined imaging scene according to being included in a scene determination area,
An imaging method comprising: an image processing unit performing image processing of the captured image in accordance with the scene determination result. Computer
A feature amount calculation unit that divides a captured image into a plurality of regions and calculates a ratio and luminance of each color as a feature amount for each region;
It is determined whether or not each of the plurality of regions is a region indicating a specific image based on the feature amount, and the region indicating the specific image is a scene determination region set in advance on the captured image. Image discriminating means for discriminating a predetermined imaging scene according to being included;
A computer program for causing an image processing unit to perform image processing according to a scene determination result of the image determining unit.