JP2006236181A - Image identification device and method therefor, and image identification program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To identify image classification with high accuracy to which an image to be represented by image data belongs. <P>SOLUTION: An image file 10, including image data 11 and tag information 12, is received. Based on the image data 11, a degree of certainty showing the probability, in which an image represented by the image data belongs to an image classification, is computed for each of a plurality of preliminarily determined image classifications (object type and lighting type). Further, the probability where the image data 11 belongs to object type "Evening scenes" and a probability in which the image data belongs to lighting type "tungsten" are computed, based on the tag information 12 included in the image file 10. When the probabilities are smaller than a prescribed value, the degree of certainty which obtains such probabilities (certainty degree of evening scene or tungsten degree of certainty) is decreased. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an apparatus and method for identifying an image classification of an image represented by image data, and a program for causing a computer to execute an image identification process.

  Patent Document 1 describes a digital still camera including an image data processing unit 24 including a scene determination unit 31. Based on whether an image representing a person is included in the image and the color histogram, the scene determination unit 31 converts the image into a snapshot, a portrait, a commemorative photo, a landscape shot, an evening scene, a night scene, a night scene portrait, an ocean scene. , It is determined whether the light source is a fluorescent light source or a tungsten light source. When it is determined in which shooting scene the image is, a correction process set for each shooting scene is performed on the image data based on the determined shooting scene.

However, since the shooting scene of “Evening scene” and the shooting scene of “Tungsten light source” are both images with an orange color as a whole, they are accurately obtained based only on the information obtained from the image data. It is not easy to determine (identify).
JP 2002-218480 A

  An object of the present invention is to improve the accuracy of classification of images represented by image data (classification of shooting scene, subject type, lighting type, etc.).

  An image identification device according to the present invention has an image file receiving means for receiving an input of an image file including tag information and image data, and is predetermined based on the image data included in the image file received by the image file receiving means. For each of a plurality of image classifications, a certainty factor calculating means for calculating a certainty factor indicating to what degree the image represented by the image data belongs to those image classifications is received by the image file receiving means. Reliability calculation means for calculating the reliability of at least one of the plurality of certainty degrees calculated by the certainty degree calculation means based on the tag information contained in the image file, and the reliability calculation Such an evaluation value when the reliability calculated by the means is lower than a predetermined value. Further comprising a confidence correction means for reducing the resulting confidence characterized.

  The image identification method according to the present invention receives an input of an image file including tag information and image data, and receives the image data for each of a plurality of predetermined image classifications based on the image data included in the received image file. The degree of certainty representing the degree of accuracy of the image represented by the image classification is calculated, and based on the tag information included in the received image file, the plurality of degrees of certainty calculated by the certainty degree calculating means are calculated. And calculating the reliability of at least one of the certainty degrees, and when the calculated reliability is lower than a predetermined value, reducing the certainty of obtaining such reliability. To do.

  The present invention also provides a program for causing a computer to execute the above-described image identification method (a program for causing a computer to function as the above-described image identification device).

  Based on the image data included in the given image file, the certainty level indicating the degree of accuracy that the image represented by the image data belongs to each of a plurality of predetermined image classifications, Calculated for each. Image classification is a classification that can be identified based on image data, and should be corrected differently in image correction processing (it is empirically known that different correction processing is more appropriate). is there.

  In one embodiment, the certainty factor calculating means indicates a subject type that indicates to what degree each image representing a plurality of predetermined subject types is included in the image represented by the image data. Based on the feature amount obtained from the image data, a certain type of certainty factor calculating means for calculating the certain type of subject, and an image represented by the image data includes a plurality of predetermined types. For each lighting type, a lighting confidence level that indicates the degree of accuracy with which the shooting was performed under the lighting type is calculated for each predetermined lighting type based on the feature amount obtained from the image data. Another certainty factor calculation means is included. Examples of the image classification related to the subject type include “person face”, “underwater”, “high saturation”, “evening scene”, “night scene”, “blue sky”, “high key”, and the like. Image classifications related to lighting types include “backlight”, “proximity strobe”, and “tungsten”.

  For example, an image of the subject type “underwater” naturally includes pixels having a so-called underwater color (a color belonging to a blue to green color range) in a wide range in the image. In the simplest case, if a pixel with underwater color is included in a predetermined area or more, it can be identified that the image is “underwater” image classification, and the certainty factor is calculated according to the area. be able to. Of course, the certainty factor for the image classification of “underwater” can be calculated using a plurality of types of feature quantities.

  Preferably, for each of the image classifications, there is provided a memory storing one or a plurality of types of feature quantities suitable for calculating the certainty factor and point data for calculating the certainty factor according to the feature quantity. Is referred to to calculate a feature amount based on the image data, and a certainty factor is calculated according to point data obtained according to the calculated feature amount. The point data may be used as the certainty as it is, or when a plurality of feature quantities are used for calculating the certainty, the added value of each point data may be used as the certainty. A value obtained by dividing the added point data by the number of feature values may be used as the certainty factor. Preferably, the certainty factor is normalized to a value ranging from 0 to 1.

  The certainty factor obtained for each image classification represents the certainty of the image classification to which the image represented by the image data contained in the given image file belongs, so it was determined for each image classification based on this certainty factor If correction processing is performed, image correction processing (density correction, white balance correction, etc.) suitable for the image data can be performed.

  When the image correction processing is performed according to the image classification, more appropriate image correction processing can be performed on the image data to be processed if finer (more) image classification is set.

  However, when fine (many) image classifications are set, there is a risk that accuracy of certainty obtained based on image data may deteriorate.

  For example, an image obtained by photographing a sunset scene (an evening scene image) and an image obtained by photographing a subject (for example, a person) under a tungsten light source (tungsten image). Since both the sunset scene image and the tungsten image are images with an orange color as a whole, if the certainty factor is calculated based only on the image data, the certainty factor for the tungsten image is obtained even though it is an evening scene image. A large value may be obtained, or conversely, although the image is a tungsten image, a large value may be obtained as a certainty factor that the image is an evening scene image. However, the correction process is different, for example, the sunset image is weakly white-balanced to maintain the atmosphere of the evening scene that appears in the image, and the tungsten image is strongly white-balanced to make the color more natural. The necessity is high. For this reason, it is preferable to use separate image classifications rather than combining the sunset scene image and the tungsten image as one image classification.

  Therefore, tag information stored together with image data in an image file in many digital cameras is used to prevent the accuracy of certainty from deteriorating. The tag information is information stored together with image data in an image file obtained by a digital camera or the like in accordance with, for example, the Exif (Exchangeble image file format) standard. Based on the tag information included in the image file, the reliability for at least one of the calculated certainty levels is calculated.

  The tag information stores image data acquisition conditions (photographing conditions at the time of photographing a subject) and the like when an image file is created (at the time of photographing). Evaluation according to the reliability can be added to the certainty factor obtained based on the image data (that is, the certainty factor obtained from the image feature) based on the tag information.

  The reliability obtained based on the tag information is preferably calculated for one of the plurality of subject types and one of the plurality of lighting types having similar image characteristics. If image classifications that do not approximate image characteristics are used, a relatively reliable value is calculated as the certainty factor obtained from the image data. However, image classifications that approximate image characteristics (for example, the above-described sunset image and tungsten image). This is because the accuracy of the certainty factor obtained from the image data may be lower than that of image categories that are not approximated.

  When the reliability obtained based on the tag information is lower than a predetermined value, the certainty that such reliability (evaluation value) is obtained is reduced.

  According to the present invention, since the certainty factor for the image classification determined to be low in reliability based on the tag information is reduced, the correction effect of the correction process corresponding to the image classification can be weakened. It is possible to prevent (reduce) an adverse effect caused by the image correction process based on an erroneous certainty factor.

  In one embodiment, the certainty factor correcting unit varies the degree of decrease in the certainty factor according to the magnitude of the reliability calculated by the reliability calculating unit. The lower the reliability, the smaller the certainty factor can be corrected.

  FIG. 1 shows a functional block diagram of the image discrimination device.

  Based on the image file to be processed, the image discriminating apparatus determines the image type (scene, situation) of the image represented by the image data included in the image file, and the lighting type (at the time of shooting the subject) ( This is a device for discriminating the type of light source at the time of shooting. The image file to be processed is acquired using, for example, a digital camera. The image data acquired by the digital camera is given (inputted) to the image discrimination device via the memory card or the network.

  Rather than determining that the image represented by the image file to be processed is one of a plurality of predetermined image types (in other words, not another image type), It is more convenient in terms of image processing to calculate how close each image type is to those image types (probability for each image type). For this reason, as will be described later, the image discriminating apparatus determines whether the image represented by the image file belongs to each of a plurality of preset image types with certainty. It is expressed by the numerical value “degree”. The same applies to the lighting type.

  The certainty factor for each subject type and each lighting type obtained by the image discriminating apparatus is used for image correction processing of image data included in the image file to be processed. In the image correction processing, correction processing corresponding to each of the subject type and the lighting type is prepared. Depending on the degree of certainty obtained for each of the subject type and the lighting type, the degree of the correction effect of the correction processing on the image data (the degree of the correction effect of the correction processing prepared for each of the subject type and the lighting type) ) Is determined.

  Referring to FIG. 1, the image discriminating device calculates subject type certainty factor calculation function 1, lighting type certainty factor calculation function 2, evening scene / tungsten certainty factor recalculation function 3, and subject type “non-detection target” certainty factor calculation. Configured to accomplish function 4. Each of these functions may be configured by hardware, or part or all of the functions may be realized by programming processing (software). An image discriminating device may be incorporated as part of an image correcting device that performs image correction processing on image data. An image discriminating apparatus (image discriminating apparatus) that achieves the above functions by programming processing includes a CPU, a memory, a storage device, a CD-ROM drive, a communication device, a display device, and the like.

  The subject type certainty factor calculation function 1 determines the subject type (scene, situation) appearing in the image represented by the image data 11 based on the image data 11 included in the image file 10 to be processed, The certainty (a numerical value that defines the certainty) is calculated. In this embodiment, the subject type certainty factor calculation function 1 has seven types of subject types: “person face”, “underwater”, “high saturation”, “evening scene”, “night scene”, “blue sky”, and “high key”. Calculate confidence for each.

  The writing type certainty factor calculation function 2 is a light source when the image file 10 is obtained based on the image data 11 included in the image file 10 to be processed (for example, when the image is taken by a digital camera). Is determined, and its certainty (a numerical value defining the certainty) is calculated. In this embodiment, the writing type certainty factor calculation function 2 calculates the certainty factor for each of the three types of lighting, “backlight”, “proximity strobe”, and “tungsten”.

  One of the subject types for which the certainty factor is calculated by the subject type certainty factor calculation function 1 and one of the lighting types for which the certainty factor is calculated by the certainty type certainty factor calculation function 2 As for “tungsten”, when these certainty factors (“evening scene certainty factor” and “tungsten certainty factor”) are calculated based only on the image data 11, the tungsten certainty factor is actually an evening scene image. A large value may be obtained as a degree, or conversely, a large value may be obtained as a certainty of the evening scene despite the fact that it is actually a tungsten image. This is because both the sunset scene image and the tungsten image are orange-colored as a whole and are relatively difficult to distinguish on the image data.

  The evening scene / tungsten certainty recalculation function 3 includes the certainty factor for the subject type “evening scene” and the certainty factor for the lighting type “tungsten” obtained based on the image data 11 in the image file 10 to be processed. The tag information 12 is used for recalculation. That is, the sunset / tungsten certainty recalculation function 3 is a process for accurately discriminating whether the overall orange image is an evening image or a tungsten image using the tag information 12. (Recalculation of evening scene certainty and tungsten certainty, calculation process of corrected evening scene certainty and modified tungsten certainty).

  The subject type “non-detection target” certainty factor calculation function 4 has the above seven subject types (“person face”, “underwater”, “high saturation”, “evening scene”, “night scene”, “blue sky”, and “high key”). ) To calculate the certainty factor that the subject type is other than (the subject type is “non-detection target”).

  Hereinafter, each of the subject type certainty factor calculation function 1, the lighting type certainty factor calculation function 2, the evening scene / tungsten certainty factor recalculation function 3, and the subject type “non-detection target” certainty factor calculation function 4 will be described.

(1) Subject type certainty factor calculation function 1
The subject type certainty factor calculation function 1 can be configured by a plurality of identification devices (identification programs) corresponding to a plurality of predetermined subject types. In this embodiment, the subject type certainty factor calculation function 1 has seven functions corresponding to “person face”, “underwater”, “high saturation”, “evening scene”, “night scene”, “blue sky”, and “high key”. Includes identification device.

  For example, an identification device corresponding to the subject type “evening scene” (hereinafter referred to as an evening scene identification device) is represented by image data 11 included in the image file 10 to be processed, based on learning data obtained by a learning process performed in advance. Whether the image to be displayed is an evening scene image (the certainty level that is an evening scene image) is calculated.

  As a representative of a plurality of subject types, a certainty factor calculation process for the subject type “evening scene” will be described.

  FIG. 2 shows how learning data used in the certainty factor calculation process for the subject type “evening scene” is created.

  In the learning data creation process, first, a large number of evening scene sample image data and a large number of non-evening scene sample image data are prepared. As the evening scene sample image data, data representing evening scene images taken at various places is used. For the non-evening scene sample image data, an image that is not an evening scene image (non-evening scene image) (which may be completely different from or similar to the evening scene image) may be used (left side in FIG. 2).

  One feature quantity type, for example, an R value of 70% (a value corresponding to the 70% point in the cumulative histogram of R values for each pixel constituting the image represented by the image data) is selected. The selected feature quantity type (R value 70% value) is calculated using a large number of sunset scene sample image data, and its frequency histogram is created. Similarly, the selected feature quantity type (R value 70% value) is calculated using a large number of non-evening scene sample image data, and a frequency histogram is created (center of FIG. 2).

  A cumulative histogram created using a large number of evening scene sample image data (upper center in FIG. 2) and a cumulative histogram created using a large number of non-evening scene sample image data (lower center in FIG. 2). If it is biased, it can be said that the selected feature quantity type (R value 70% in the case of the above example) is a feature quantity type suitable for discrimination (identification) of an evening scene image and a non-evening scene image.

  A logarithmic value of the ratio of the frequency values for each feature value corresponding to the two accumulated histograms is calculated. This logarithmic value is called an identification point. The identification points form learning data together with the types of feature quantities.

  FIG. 3 shows an example of learning data for the subject type “evening scene”. The learning data for the subject type “evening scene” includes types of feature quantities suitable for identification of the subject type “evening scene” (here, R value 70% value, Y value 90% value, R value 90% value). And the Cr average value) and the identification points are stored. A plurality of identification points are stored corresponding to each feature value.

  The evening scene identification device stores learning data (FIG. 3) on the subject type “evening scene”, and based on this learning data, the image represented by the image data 11 is compared with the image data 11 to be processed. A certainty factor that defines the certainty of the subject type “evening scene” is calculated. The total value of the identification points may be used as it is as the certainty for the subject type “evening scene”, or the total value of the identification points is normalized to a numerical value ranging from 0 to 1, and the normalized value is confirmed. It may be a degree. The total value of the identification points may be divided by the number of feature quantity types, the divided value may be normalized, and the normalized value may be used as the certainty factor.

  The identification devices corresponding to each of the other subject types “person face”, “underwater”, “high saturation”, “night view”, “blue sky”, and “high key” also store learning data, and based on the learning data, A certainty factor that defines the certainty that the image represented by the image data 11 to be processed is the corresponding subject type is calculated. That is, the image represented by the image data to be processed is assigned to each of the seven subject types of “person face”, “underwater”, “high saturation”, “evening scene”, “night scene”, “blue sky”, and “high key”. Seven corresponding degrees of confidence are calculated (including that the degree of confidence is “0”).

(2) Certainty calculation function 2 by writing type
Like the subject type certainty factor calculation function 1, the writing type certainty factor calculation function 2 can also be configured by a plurality of identification devices corresponding to a plurality of predetermined lighting types. In this embodiment, the writing type certainty factor calculation function 2 includes three identification devices corresponding to “backlight”, “proximity strobe”, and “tungsten”, respectively. Each of the three identification devices stores learning data in the same manner as the identification device for the subject type described above. Based on the learning data, the certainty factor for each of “backlight”, “proximity strobe” and “tungsten” is calculated for the image represented by the image data 11 to be processed (the certainty factor is “0”). Including that).

(3) Evening scene / tungsten confidence recalculation function 3
The sunset / tungsten certainty recalculation function 3 performs processing to increase the accuracy of the certainty of the subject type “evening scene” and the certainty of the lighting type “tungsten” calculated based on the image data 11.

  The image file 10 includes tag information 12 together with image data. This tag information 12 is, for example, compliant with the “digital still camera image file format standard” (hereinafter referred to as the Exif standard). In the Exif standard, a tag related to version, a tag related to image data characteristics, a tag related to structure, a tag related to user information, a tag related to related files, a tag related to date and time, a tag related to shooting conditions, and the like are defined. Among these tags, in the processing in the evening scene / tungsten certainty recalculation function 3, a tag related to the shooting condition is used.

  FIG. 4 shows the contents of the information included in the tag information related to the shooting conditions defined in the Exif standard.

  The Exif standard does not stipulate that all information specified in the standard is recorded in an image file. For this reason, the information content recorded in the tag information in the image file differs depending on the type of digital camera (manufacturer, model) used to acquire the image file. The sunset / tungsten certainty recalculation function 3 assumes that the tag information 12 is included in the image file 10 to be processed, and that the tag information 12 has various contents. Operates as described below.

  5, FIG. 6 and FIG. 7 are flowcharts showing the flow of processing in the evening scene / tungsten certainty recalculation function 3.

  In the sunset / tungsten certainty recalculation function 3, first, the probability that the image represented by the image data 11 included in the image file 10 to be processed is an evening scene image using the tag information 12 (hereinafter, sunset scene probability). ) And the probability of being a tungsten image (hereinafter referred to as tungsten probability) is calculated (FIGS. 5 and 6). The evening scene / tungsten certainty recalculation function 3 determines whether the image represented by the image data 11 included in the image file to be processed is an evening scene image or a tungsten image (which is likely to be) Therefore, the sum of the evening scene probability and the tungsten probability is “1” (100%).

  As will be apparent from the description below, the sunset / tungsten certainty recalculation function 3 uses two pieces of information: “flash information (flash on / off information)” and “subject luminance value”. This is because these two pieces of information are stored in the tag information 12 at a high rate, are effective in distinguishing (identifying) the sunset scene image from the tungsten image, and are not easily affected by the performance and settings of the camera used for shooting. .

  Referring to FIG. 5, first, tag information 12 is read from image file 10 to be processed (step 21).

  It is determined whether flash information is stored in the tag information 12 (step 22).

  When flash information is not stored (NO in step 22), both the evening scene probability and the tungsten probability are set to 0.5 (step 34). In this case, as will be described later, the evening scene certainty and the tungsten certainty calculated based on the image data 11 are output as they are (the same value as the certainty calculated in the subject type certainty calculation function 1). Considering that even if the luminance value of the subject is calculated and the sunset probability and tungsten probability are calculated while it is unknown whether the flash is on or off, the sunset probability and tungsten probability are extremely low reliability. It is.

  If flash information (on / off information) is stored (YES in step 22), it is determined whether or not the subject luminance value information BV is stored in the tag information 12 (step 23).

  If the subject brightness value information BV is not stored in the tag information 12 (NO in step 23), whether the subject brightness value can be calculated based on other information stored in the tag information 12 Is determined (step 28, FIG. 6) (details will be described later). Even when the subject luminance value cannot be calculated based on other information stored in the tag information 12, both the evening scene probability and the tungsten probability are set to 0.5 (NO in step 29, step 34).

  When the subject brightness information BV is stored (YES in step 23), the evening scene probability and the tungsten probability are calculated based on the recorded subject brightness value BV. Here, the evening scene probability and the tungsten probability calculated based on the subject luminance value BV are different depending on whether the flash information is on-information or off-information (step 24).

  FIG. 8A is a graph showing the relationship between the subject luminance value and the sunset scene probability (range from 0 to 1) when the flash information is on-information. FIG. 8B is a graph showing the relationship between the subject luminance value and the tungsten probability (range from 0 to 1) when the flash information is on-information. Data (tables) representing the graphs shown in FIGS. 8A and 8B are stored in advance in a storage device (this will be described later with reference to FIGS. 9A, 9B, and 10). The same applies to the graphs shown in A) and FIG. 10 (B).

  The relationship between the subject luminance value and the evening scene probability when the flash information shown in FIG. 8A is on-information, and the subject luminance value and the tungsten probability when the flash information shown in FIG. 8B is on-information. The relationship is obtained by the learning process. For example, a lot of sunset sample image data and a lot of tungsten sample image data in which flash-on information is stored in the tag information and the subject luminance value is stored are prepared. Using these sample image data, a histogram relating to the subject luminance value is calculated for each of the sunset scene image and the tungsten image. Using the number of sunset scene images and the number of tongue images at a certain subject brightness value, the sunset scene probability for the subject brightness value is obtained by dividing the number of sunset scene images by the sum of the number of sunset scene images and the number of tungsten images. If the number of tungsten images is divided by the sum of the number of sunset scene images and the number of tungsten images, the tungsten probability for the subject luminance value can be obtained.

When the flash information shown in FIG. 8A is on-information, the relationship between the subject luminance value and the evening scene probability (P _ON (evening scene | BV)) is first obtained, and the flash information shown in FIG. The relationship (P _ON (tungsten | BV)) between the subject luminance value and the tungsten probability at a certain time may be obtained by the following equation.

P _ON (Tungsten | BV) = 1−P _ON (Evening scene | BV) Formula 1

Of course, the relationship (P _ON (tungsten | BV)) between the subject luminance value and the tungsten probability when the flash information is on-information is first obtained, and the subject luminance value and tungsten when the obtained flash information is on-information. Based on the relationship with the probability, the relationship (P _ON (evening scene | BV)) between the subject brightness value and the evening scene probability when the flash information is on-information may be obtained (similar to Equation 1).

  FIG. 9A is a graph showing the relationship between the subject luminance value when the flash information is off information and the evening scene probability, and FIG. 9B shows the subject luminance value when the flash information is off information. It is a graph which shows the relationship with a tungsten probability.

The relationship between the subject luminance value and the sunset probability and the relationship between the subject luminance value and the tungsten probability when the flash information shown in FIG. 9A and FIG. It was obtained by. The relationship between the subject luminance value when the flash information shown in FIG. 9B is off information and the tungsten probability (P _OFF (tungsten | BV)) is similar to the subject luminance value when the flash information is off information. Using the relationship with the evening scene probability (P _OFF (evening scene | BV)), it can be obtained by the following equation.

P _OFF (Tungsten | BV) = 1−P _OFF (Evening scene | BV) Expression 2

First, the relationship between the subject luminance value and the tungsten probability (P _OFF (tungsten | BV)) when the flash information is off information is obtained, and the subject luminance value and the tungsten probability when the obtained flash information is off information. Based on this relationship, the relationship (P _OFF (evening scene | BV)) between the subject luminance value and the evening scene probability when the flash information is off information may be obtained (similar to Expression 2).

  As described above, in the case of image data captured by flash emission, based on the subject luminance value BV in the tag information 12 based on the graphs of FIGS. 8A and 8B. , Sunset scene probability and tungsten probability are calculated (step 25), and in the case of image data taken without flash emission, tag information is based on the graphs of FIG. 9 (A) and FIG. 9 (B). Based on the subject brightness value BV in 12, the evening scene probability and the tungsten probability are calculated (step 26).

  Referring to FIG. 7, based on the calculated evening scene probability and tungsten probability, a weighting coefficient β to be multiplied by the evening scene certainty factor and a weighting coefficient γ to be multiplied by the tungsten certainty factor are calculated (step 61). ).

  FIG. 11 is a graph showing the relationship between the evening scene probability and the weighting factor β, and FIG. 12 is a graph showing the relationship between the tungsten probability and the weighting factor γ.

  The weighting factor β and the weighting factor γ calculated according to the graphs shown in FIGS. 11 and 12 are used, and the evening scene certainty factor and the tungsten certainty factor are corrected according to the following equations (step 62).

Corrected evening scene certainty = β · Evening scene certainty Modified tungsten certainty = γ · Tungsten certainty ・ ・ ・ Equation 3

  As is clear from FIGS. 11, 12, and Equation 3, the corrected evening scene certainty and tungsten certainty are obtained when both the evening scene probability and the tungsten probability calculated based on the tag information 12 are 0.5. Is the same as the evening scene certainty factor and the tungsten certainty factor calculated by the subject type certainty factor calculation function 1. If either of the evening scene probability and the tungsten probability calculated based on the tag information 12 is larger (or smaller) than 0.5, the probability is smaller (either the evening scene confidence level or the tungsten certainty level). Is smaller than the value before correction. The corrected evening scene certainty and tungsten certainty, or the evening scene certainty and the modified tungsten certainty are output (step 63).

  In fact, even if the image is a sunset image, the tungsten certainty factor is decreased when a large value is obtained as the tungsten certainty factor. Conversely, a large value is obtained as the evening scene certainty factor despite being a tungsten image. If so, the evening scene certainty is reduced. The certainty level determines the degree (degree) of application of the image correction processing that will be performed later, so that the correction effect of either the correction process for the sunset image or the correction process for the tungsten image is weakened by reducing the certainty level. It is done. Inappropriate image correction processing can be prevented.

  The determination of the subject luminance value calculation possibility (step 28, FIG. 6) will be described.

  The subject brightness value can be calculated by the following equation.

  CalBV = TV + AV + EBV-SV ・ ・ ・ Formula 4

  In Equation 4, CalBV is the subject brightness value calculated by calculation, TV is the shutter speed, AV is the aperture value, SV is the ISO sensitivity, and EBV is the exposure correction value. If these values can be obtained, the subject luminance value can be obtained by calculation according to Equation 4.

(I) Check whether the formula for calculating the subject luminance value (Formula 4) is feasible (Steps 41, 42, 47, 48)
Expression 4 described above assumes that the exposure setting (which means aperture value setting and shutter speed setting) at the time of shooting is not manual setting but automatic setting. Whether the exposure setting at the time of shooting is a manual setting or an auto setting can be detected by the exposure program information or the exposure mode information in the tag information 12.

  It is determined whether or not the exposure information or exposure mode information is included in the tag information 12 (steps 41 and 47), and if it is included, it is determined whether shooting has been performed by manual setting. (Steps 42 and 48).

  The exposure program information is defined by nine numbers from 0 to 8 in the Exif standard. When the tag information 12 includes exposure program information and “1” is stored as the exposure program information, it indicates that the image file 10 has been obtained by setting the exposure manually. In this case, the subject luminance value cannot be calculated based on the above equation 4 (the relationship premised on equation 4 does not hold). Data indicating that the subject brightness value CalBV cannot be calculated is output (YES in step 41, NO in step 42, step 52).

  If the exposure program information is not included in the tag information 12, it is determined whether exposure mode information is stored in the tag information 12 (NO in step 41, step 47). The exposure mode information is defined by three numbers 0, 1 or 2 in the Exif standard. When “1” is stored as the exposure mode information, it indicates that the image file 10 is obtained by manually setting the exposure. Also in this case, data indicating that the subject luminance value CalBV cannot be calculated is output (YES in step 47, NO in step 48, step 52).

  If both the exposure program information and the exposure mode information are not included in the tag information 12, it is unclear whether or not the above equation 4 is satisfied, and in this case as well, the subject luminance value CalBV cannot be calculated. Data indicating that is output (NO in step 47, step 52).

  If it is determined that the exposure setting is not a manual setting based on the exposure program information or exposure mode information (YES in step 42 or YES in step 48), the shutter speed required for calculating the subject luminance value CalBV (TV ), Aperture value (AV), ISO sensitivity (SV), and exposure correction value (EBV) can be determined.

(ii) Check whether shutter speed (TV) can be acquired (steps 43 and 49)
When the shutter speed information is stored in the tag information 12 or the exposure time information is stored in the tag information 12, the shutter speed (TV) can be acquired (shutter speed (TV) = − log ₂ (Exposure time) is established). If neither the shutter speed information nor the exposure time information is stored in the tag information 12, data indicating that the subject brightness value CalBV cannot be calculated is output (NO in step 43, NO in step 49, Step 52). If the shutter speed information is stored in the tag information 12 or the exposure time information is stored in other cases, the process proceeds to the next process (YES in step 43 or YES in step 49).

(iii) Checking whether or not aperture value (AV) can be acquired (steps 44 and 50)
If the aperture information is stored in the tag information 12 or the F number is stored in the tag information 12, the aperture value (AV) can be obtained (aperture value (AV) = 2log ₂ (F Number))). If neither aperture value information nor F number information is stored in the tag information 12, data indicating that the subject luminance value CalBV cannot be calculated is output (NO in step 44, and in step 50). NO, step 52). If the aperture value information is stored in the tag information 12 or if the F number information is stored, the process proceeds to the next process (YES in step 44 or YES in step 50).

(Iv) Check whether ISO sensitivity (SV) can be acquired (step 45)
The ISO sensitivity can be calculated based on the ISO speed rate (the relational expression ISO sensitivity = log 2 (ISO speed rate / 3.125) is established). Therefore, it is determined whether ISO speed rate information is stored in the tag information 12 (step 45). If the ISO speed rate information is not stored in the tag information 12, data indicating that the subject brightness value CalBV cannot be calculated is output (NO in step 45, step 52). If the ISO speed rate information is stored in the tag information 12, the process proceeds to the next process (YES in step 45).

(V) Checking whether exposure correction value information (EBV) can be acquired (step 46)
It is determined whether exposure correction value information (EBV) is stored in the tag information 12 (step 46). If the exposure correction value information (EBV) is not stored in the tag information 12, data indicating that the subject brightness value CalBV cannot be calculated is output (NO in step 46, step 52).

  If the exposure setting is not a manual setting and the shutter speed (TV), aperture value (AV), ISO sensitivity (SV), and exposure correction value (EBV) can be acquired (YES in step 46), the above equation 4 Based on the above, the subject brightness value CalBV can be calculated. In this case, data indicating that the subject luminance value CalBV can be calculated is output (step 51).

  Returning to FIG. 5, if the subject brightness value CalBV cannot be calculated, both the evening scene probability and the tungsten probability are set to 0.5 (NO in step 29, step 34). As described above, the evening scene certainty and the tungsten certainty calculated based on the image data 11 are output as they are (same values as the certainty calculated in the subject type certainty calculation function 1).

  If the subject luminance value CalBV can be calculated, the subject luminance value CalBV is calculated based on the above-described equation 4 (YES in step 29, step 30). Based on the subject brightness value CalBV obtained by the calculation, the evening scene probability and the tungsten probability are calculated. The evening scene probability and tungsten probability calculated based on the subject brightness value CalBV obtained by the calculation are also different depending on whether the flash information is on-information or off-information (step 31).

  FIG. 10A is a graph showing the relationship between the subject luminance value (calculated value) and the evening scene probability when the flash information is on-information. FIG. 10B is a graph showing the relationship between the subject luminance value (calculated value) and the tungsten probability when the flash information is on-information. The relationship between the subject luminance value (calculated value) and the evening scene probability shown in FIGS. 10A and 10B is also obtained by the learning process.

  When the flash information is off information, the relationship between the subject brightness value CalBV and the sunset scene probability obtained by the calculation is almost equal to the subject brightness value BV and the sunset scene probability stored in the tag information 12, and obtained by the calculation. The relationship between the obtained subject luminance value CalBV and the tungsten probability is almost equal to the relationship between the subject luminance value BV and the tungsten probability stored in the tag information 12 (the same characteristics were obtained when the learning process was performed). Is). Therefore, when the flash information is off information, the subject brightness value CalBV obtained by the calculation is handled as the subject brightness value BV stored in the tag information 12 (NO in step 31 and step 32). The evening scene probability and the tungsten probability are calculated from the graphs shown in FIGS. 9A and 9B (step 26).

  Based on the calculated evening scene probability and tungsten probability, the weighting coefficient β to be multiplied by the evening scene certainty and the weighting coefficient γ to be multiplied by the tungsten certainty are calculated, and the evening scene belief is calculated using the weighting coefficients β and γ. As described above, either the degree of tungsten or the confidence of tungsten is corrected.

(4) Subject type “not subject to detection” certainty factor calculation function 4
Certainty for human face, underwater, high saturation, night view, blue sky and high key calculated by subject type certainty factor calculation function 1, and certainty factor for backlight and proximity strobe calculated by lighting type certainty factor calculation function 2 Then, the evening scene / tungsten certainty recalculation function 3 corrects the evening scene certainty and tungsten certainty to calculate the certainty for the subject type “not detected”. The certainty factor Pot of the subject type “not subject to detection” is calculated by the following equation.

Confidence level Pot =
(1-person face certainty)
* (1-Blue sky certainty * (1-Tungsten certainty) * (1-Backlight certainty))
* (1-Evening confidence * (1-Tungsten certainty))
* (1-Night view certainty) * (1-Tungsten certainty))
* (1-High chroma certainty) * (1-Tungsten certainty))
* (1-Underwater certainty) * (1-Tungsten certainty))
* (1-high key certainty) * (1-tungsten certainty) * (1-backlight certainty))
... Formula 5

  In Formula 5, when the evening scene certainty and the tungsten certainty are corrected by the evening scene / tungsten certainty recalculation function 3, the corrected values are used.

  Certainty of human face, underwater, high saturation, evening scene, night scene, blue sky, high key, and out of detection target for the image represented by the image data 11 included in the processing target image file 10 obtained by the above processing. , And certainty for backlighting, proximity strobe and tungsten. Based on these certainty factors, image correction processing (white balance correction, density correction, etc.) is performed on the image data 11 included in the image file 10. As described above, the image correction processing is set for each subject type and each lighting type, and the image represented by the image data 11 is based on the confidence level obtained for each subject type and each lighting type. (The correction effect is strengthened as the certainty factor increases, and the correction effect is weakened as the certainty factor is small).

  As described above, for evening scene images and tungsten images having features similar to each other, not only the image data 11 but also the tag information 12 is used to calculate the certainty factor. As described above, when a large value is calculated as the tungsten certainty factor in spite of the sunset image, the tungsten certainty factor is reduced, so that the correction effect of the correction process for the tungsten image is weakened. Inappropriate image correction processing can be prevented (at least the correction effect can be weakened). The same applies to the case where a large value is calculated as the evening scene certainty in spite of the tungsten image.

  FIG. 13 shows experimental results when the processing (processing using Exif tag information) by the evening scene / tungsten certainty recalculation function 3 is performed and when it is not performed. In FIG. 13, hatched portions indicate erroneous detection. The false detection can be eliminated by the sunset scene / tungsten certainty recalculation function 3.

  In the embodiment described above, the evening scene / tungsten reliability recalculation function 3 always operates together with the subject type certainty factor calculation function 1 and the lighting type certainty factor calculation function 2, but the subject type certainty factor calculation. When at least one of the evening scene certainty obtained by the function 1 and the tungsten certainty obtained by the writing type certainty calculating function 2 is equal to or greater than a predetermined threshold (for example, 0.5), If both the evening scene certainty and the writing certainty are values smaller than a predetermined threshold value, the operation may not be performed.

The functional block diagram of an image discrimination device is shown. The state of learning data generation is shown. An example of learning data about the subject type “evening scene” is shown. The contents of the tag information related to the shooting conditions are shown. It is a flowchart which shows the flow of a process in the evening scene / tungsten certainty recalculation function. It is a flowchart which shows the flow of a process in the evening scene / tungsten certainty recalculation function. It is a flowchart which shows the flow of a process in the evening scene / tungsten reliability recalculation function. (A) shows the relationship between the subject luminance value and the evening scene probability when the flash information is on-information, and (B) shows the relationship between the subject luminance value and the tungsten probability when the flash information is on-information. (A) shows the relationship between the subject luminance value and the sunset scene probability when the flash information is off information, and (B) shows the relationship between the subject luminance value and the tungsten probability when the flash information is off information. (A) shows the relationship between the subject luminance value (calculated value) and the evening scene probability when the flash information is on-information, and (B) shows the subject luminance value (calculated value) and tungsten when the flash information is on-information. The relationship with probability is shown. The relationship between the evening scene probability and the weighting factor β is shown. The relationship between tungsten probability and weighting coefficient γ is shown. The experiment results when the processing (processing using the Exif tag) with the sunset / tungsten confidence recalculation function is performed and when it is not performed are shown.

Explanation of symbols

1 Subject type certainty factor calculation function 2 Lighting type certainty factor calculation function 3 Tungsten certainty factor recalculation function 4 Subject type “non-detection target” certainty factor calculation function
10 Image files
11 Image data
12 Tag information

Claims (7)

Image file receiving means for receiving input of an image file including tag information and image data;
Based on the image data included in the image file received by the image file receiving means, for each of a plurality of predetermined image classifications, the degree of accuracy of the image represented by the image data A certainty factor calculating means for calculating a certainty factor indicating whether it belongs to a classification;
A reliability level for calculating a reliability level for at least one of the multiple reliability levels calculated by the reliability level calculation unit based on the tag information included in the image file received by the image file reception unit. Calculation means, and confidence correction means for reducing the confidence that such reliability is obtained when the reliability calculated by the reliability calculation means is lower than a predetermined value;
An image identification apparatus comprising: The certainty factor calculation means
A feature amount obtained from the image data is a subject type certainty factor indicating how much each of the images representing a plurality of predetermined subject types is included in the image represented by the image data. Based on the above, the subject type certainty factor calculation means for each subject type, and the accuracy of the image represented by the image data for each of a plurality of predetermined lighting types Including a lighting type certainty factor calculating means for calculating for each predetermined lighting type, based on the feature amount obtained from the image data, the writing certainty level indicating whether the image was taken under the lighting type,
The image identification device according to claim 1. The confidence correction means is
The degree of decrease in the certainty is varied according to the degree of reliability calculated by the reliability calculating means.
The image identification device according to claim 1. The reliability calculation means is:
The reliability is calculated for one of the image classifications related to the plurality of subject types and one of the image classifications related to the plurality of lighting types having similar image characteristics.
The image identification device according to claim 2. The image classification relating to the subject type for which the reliability is calculated is an image classification to which the sunset scene image belongs, and the image classification relating to the lighting type for which the reliability is calculated is an image classification to which an image photographed under a tungsten light source belongs. ,
The image identification device according to claim 4. Receives input of image file including tag information and image data,
Based on the image data included in the received image file, for each of a plurality of predetermined image classifications, a certainty indicating to what degree the image represented by the image data belongs to those image classifications Calculate the degree,
Based on the tag information included in the received image file, the reliability of at least one of the plurality of certainty calculated by the certainty calculating means is calculated,
An image identification method for reducing the certainty of obtaining such reliability when the calculated reliability is lower than a predetermined value. Image file acceptance processing that accepts input of image files including tag information and image data,
Based on the image data included in the image file received by the image file reception process, for each of a plurality of predetermined image classifications, the degree of accuracy of the image represented by the image data A certainty factor calculation process for calculating a certainty factor indicating whether it belongs to a classification,
A reliability level for calculating a reliability level for at least one of the multiple reliability levels calculated by the reliability level calculation process based on tag information included in the image file received by the image file reception process. A calculation process, and a certainty factor correction process for reducing the certainty factor when the reliability calculated by the reliability calculation process is lower than a predetermined value,
A program that causes a computer to execute.

Images