JP2008242638A - Image discrimination method, image discrimination device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the speed of the discrimination processing of image data. <P>SOLUTION: This image identification method includes evaluating an image shown by image data, determining the order of first discrimination processing for discriminating whether or not an image shown by the image data is belonging to a first category base on the evaluation result and second discrimination processing for judging whether or not the image shown by the image data is belonging to a second category different from the first category, and when it is judged that the image belongs to the specific category in the discrimination processing performed first between the first discrimination processing and the second discrimination processing, omitting the discrimination processing performed later between the first discrimination processing and the second discrimination processing. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image identification method, an image identification apparatus, and a program.

  Some digital still cameras have a mode setting dial for setting a shooting mode. When the user sets the shooting mode with the dial, the digital still camera determines shooting conditions (such as exposure time) according to the shooting mode and performs shooting. When shooting is performed, the digital still camera generates an image file. In this image file, additional data such as shooting conditions at the time of shooting is added to the image data of the shot image.

  Image processing is performed on the image data according to the additional data. For example, when a printer performs printing based on an image file, the image data is corrected according to the shooting conditions indicated by the additional data, and printing is performed according to the corrected image data (see Patent Document 1).

In addition, image data is analyzed to classify image categories indicated by the image data (see Patent Documents 2 and 3).
JP 2001-238177 A Japanese Patent Laid-Open No. 10-302067 JP 2006-511000 Gazette

The image data is classified into a plurality of categories by combining a plurality of discriminators for identifying whether or not the image indicated by the image data belongs to a specific category. When using a plurality of discriminators as described above, if all the discriminators are always used, the speed of the discrimination process is reduced.
An object of the present invention is to improve the speed of image data identification processing.

A main invention for achieving the above object is to evaluate an image indicated by image data, and based on the evaluation result, identify whether the image indicated by the image data belongs to a first category, and Determining an order of second identification processing for identifying whether or not an image indicated by the image data belongs to a second category different from the first category, and wherein, among the first identification processing and the second identification processing, When it is possible to identify that the image belongs to a specific category in the identification process performed first, the identification process performed after the first identification process and the second identification process is omitted.
Other features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

  At least the following matters will become clear from the description of the present specification and the accompanying drawings.

(A) Evaluating the image indicated by the image data, (B) First identification processing for identifying whether the image indicated by the image data belongs to the first category based on the evaluation result, and indicating the image data Determining an order of second identification processing for identifying whether an image belongs to a second category different from the first category; and (C) first of the first identification processing and the second identification processing. An image identification method characterized by omitting an identification process performed after the first identification process and the second identification process when it can be identified that the image belongs to a specific category in the identification process performed. Becomes clear.
According to such an image identification method, the processing speed is increased.

  Further, the image identification method includes a first overall identification process for identifying whether the image belongs to the first category based on an overall feature amount indicating an overall feature of the image, and the overall feature amount. The image belongs to the first category based on a second overall identification process for identifying whether or not the image belongs to the second category and a partial feature amount indicating a partial feature of the image. A first partial identification process for identifying whether the image belongs to the second category based on the partial feature amount, and a first partial identification process for identifying whether the image belongs to the second category. The identification process corresponds to the first partial identification process, the second identification process corresponds to the second partial identification process, and the result of the first overall identification process and the result of the second overall identification process are evaluated. As a result, the first identification process and the second identification It is desirable that the management of the order is determined. This shortens the processing time.

  In the first overall identification process, first data corresponding to a probability that the image belongs to the first category is calculated based on the overall feature amount, and the image is converted to the first data based on the first data. Whether the image belongs to a category is identified, and in the second overall identification process, second data corresponding to the probability that the image belongs to the second category is calculated based on the overall feature amount, and the second data It is preferable that whether or not the image belongs to the second category is identified based on the first data and the first data and the second data are used as the evaluation result. This increases the processing speed.

  In the first partial identification process and the second partial identification process, whether each partial image belongs to a specific category based on each partial feature amount of a plurality of partial images obtained by dividing the image. It is preferable that whether or not the image belongs to a specific category is identified based on the number of partial images that are identified and belong to the specific category. This is particularly effective in such a case.

  In addition, when it is possible to identify that the image belongs to a specific category in the first overall identification process or the second overall identification process, it is preferable that the first identification process and the second identification process are omitted. This increases the processing speed.

  The processing time for performing the first identification processing and the second identification processing is preferably slower than the processing time for evaluating the image indicated by the image data and acquiring the evaluation result. This is particularly effective in such a case.

  An image identification apparatus including a controller, wherein the controller (A) evaluates an image indicated by image data, and (B) whether the image indicated by the image data belongs to a first category based on an evaluation result. Determining an order of a first identification process for identifying whether or not an image indicated by the image data belongs to a second category different from the first category, and (C) When it can be identified that the image belongs to a specific category in the first identification process and the second identification process, after the first identification process and the second identification process. An image identification device characterized by omitting the identification process to be performed is also clarified.

  (A) causing the image identification device to evaluate the image indicated by the image data, and (B) a first identification process for identifying whether the image indicated by the image data belongs to the first category based on the evaluation result; and An order of second identification processing for identifying whether or not an image indicated by the image data belongs to a second category different from the first category; and (C) the first identification processing and the second identification processing If the image can be identified as belonging to a specific category in the identification process performed earlier, the identification process performed after the first identification process and the second identification process is omitted. The program will be revealed.

=== Overall structure ===
FIG. 1 is an explanatory diagram of an image processing system. This image processing system includes a digital still camera 2 and a printer 4.

  The digital still camera 2 is a camera that acquires a digital image by forming an image of a subject on a digital device (CCD or the like). The digital still camera 2 is provided with a mode setting dial 2A. The user can set the shooting mode according to the shooting conditions by using the dial 2A. For example, when the “night scene” mode is set by the dial 2A, the digital still camera 2 performs shooting under shooting conditions suitable for night scene shooting by decreasing the shutter speed or increasing the ISO sensitivity.

  The digital still camera 2 stores an image file generated by photographing in the memory card 6 in accordance with the file format standard. In the image file, not only digital data (image data) of a captured image but also additional data such as a shooting condition (shooting data) at the time of shooting is stored.

  The printer 4 is a printing device that prints an image indicated by image data on paper. The printer 4 is provided with a slot 21 into which the memory card 6 is inserted. The user can take a picture with the digital still camera 2, remove the memory card 6 from the digital still camera 2, and insert the memory card 6 into the slot 21.

  FIG. 2 is an explanatory diagram of the configuration of the printer 4. The printer 4 includes a printing mechanism 10 and a printer-side controller 20 that controls the printing mechanism 10. The printing mechanism 10 includes a head 11 that ejects ink, a head control unit 12 that controls the head 11, a motor 13 for conveying paper, and a sensor 14. The printer-side controller 20 includes a memory slot 21 for transmitting and receiving data from the memory card 6, a CPU 22, a memory 23, a control unit 24 for controlling the motor 13, and a drive signal for generating a drive signal (drive waveform). And a generation unit 25.

  When the memory card 6 is inserted into the slot 21, the printer-side controller 20 reads out the image file stored in the memory card 6 and stores it in the memory 23. Then, the printer-side controller 20 converts the image data of the image file into print data for printing by the printing mechanism 10, controls the printing mechanism 10 based on the printing data, and prints an image on paper. This series of operations is called “direct printing”.

  “Direct printing” is not only performed by inserting the memory card 6 into the slot 21, but also by connecting the digital still camera 2 and the printer 4 with a cable (not shown).

  The image file stored in the memory card 6 is composed of image data and additional data. The image data is composed of a plurality of pixel data. The pixel data is data indicating pixel color information (gradation value). An image is formed by arranging the pixels in a matrix. Therefore, the image data is data indicating an image. The additional data includes data indicating the characteristics of image data, shooting data, thumbnail image data, and the like.

=== Outline of automatic correction function ===
When printing a “person” photo, there is a demand to clean the skin tone. In addition, when printing a “landscape” photograph, there is a demand for emphasizing the blue of the sky and the green of trees and grass. Therefore, the printer 4 of the present embodiment includes an automatic correction function that analyzes an image file and automatically performs a suitable correction process.

  FIG. 3 is an explanatory diagram of the automatic correction function of the printer 4. Each element of the printer-side controller 20 in the figure is realized by software and hardware.

  The storage unit 31 is realized by a partial area of the memory 23 and the CPU 22. All or part of the image file read from the memory card 6 is developed in the image storage unit 31 </ b> A of the storage unit 31. In addition, the calculation result of each element of the printer-side controller 20 is stored in the result storage unit 31B of the storage unit 31.

  The face identification unit 32 is realized by the CPU 22 and a face identification program stored in the memory 23. The face identification unit 32 analyzes the image data stored in the image storage unit 31A and identifies the presence or absence of a face. When the face identifying unit 32 identifies that there is a face, the image to be identified is identified as belonging to the “person” scene. In this case, the scene identification process by the scene identification unit 33 is not performed. Since the face identification process by the face identification unit 32 is the same as a process that has already been widely performed, detailed description thereof is omitted.

  The scene identification unit 33 is realized by the CPU 22 and a scene identification program stored in the memory 23. The scene identification unit 33 analyzes the image file stored in the image storage unit 31A and identifies the scene of the image indicated by the image data. When the face identifying unit 32 identifies that there is no face, a scene identifying process by the scene identifying unit 33 is performed. As will be described later, the scene identifying unit 33 identifies whether the image to be identified is a “landscape”, “evening scene”, “night scene”, “flower”, “autumn leaves”, or “other” image. .

FIG. 4 is an explanatory diagram of a relationship between an image scene and correction contents.
The image correction unit 34 is realized by the CPU 22 and an image correction program stored in the memory 23. The image correction unit 34 converts the image data of the image storage unit 31A based on the identification results (identification results of the face identification unit 32 and the scene identification unit 33) stored in the result storage unit 31B (described later) of the storage unit 31. to correct. For example, when the identification result of the scene identification unit 33 is “landscape”, correction is performed so that blue is emphasized and green is emphasized. The image correction unit 34 may correct the image data by reflecting not only the scene identification result but also the contents of the image data of the image file. For example, when the exposure correction is negative, the image data may be corrected so as not to brighten the dark atmosphere image.
The printer control unit 35 is realized by a printer control program stored in the CPU 22, the drive signal generation unit 25, the control unit 24, and the memory 23. The printer control unit 35 converts the corrected image data into print data, and causes the printing mechanism 10 to print the image.

=== Scene Identification Processing ===
FIG. 5 is a flowchart of the scene identification process performed by the scene identification unit 33. FIG. 6 is an explanatory diagram of the function of the scene identification unit 33. Each element of the scene identification unit 33 in the figure is realized by software and hardware. The scene identification unit 33 includes a feature amount acquisition unit 40, an overall classifier 50, a partial classifier 60, and an integrated classifier 70 shown in FIG.

First, the feature amount acquisition unit 40 analyzes the image data developed in the image storage unit 31A of the storage unit 31 and acquires partial feature amounts (S101). Specifically, the feature amount acquisition unit 40 divides the image data into 8 × 8 64 blocks, calculates the color average and variance of each block, and acquires the color average and variance as partial feature amounts. Here, each pixel has gradation value data in the YCC color space, and the average value of Y, the average value of Cb, and the average value of Cr are calculated for each block, and the variance of Y and the variance of Cb are calculated. And the variance of Cr are calculated respectively. That is, three color averages and three variances are calculated as partial feature amounts for each block. These color averages and variances indicate the characteristics of the partial images in each block. Note that an average value or variance in the RGB color space may be calculated.
Since the color average and variance are calculated for each block, the feature amount acquisition unit 40 expands the image data for the blocks in the block order without expanding all the image data in the image storage unit 31A. For this reason, the image storage unit 31A does not necessarily have a capacity sufficient to expand all of the image files.

  Next, the feature amount acquisition unit 40 acquires the entire feature amount (S102). Specifically, the feature quantity acquisition unit 40 acquires the overall color average, variance, center of gravity, and shooting information of the image data as the overall feature quantity. Note that these color averages and variances indicate the overall characteristics of the image. The color average, variance, and center of gravity of the entire image data are calculated using the partial feature values calculated previously. For this reason, it is not necessary to re-expand the image data when calculating the entire feature amount, and the calculation speed of the entire feature amount is increased. Although the overall identification process (described later) is performed prior to the partial identification process (described later), the overall feature value is obtained after the partial feature value in order to increase the calculation speed. The shooting information is extracted from the shooting data of the image file. Specifically, information such as the aperture value, shutter speed, and the presence or absence of flash emission is used as the overall feature amount. However, not all shooting data of the image file is used as the entire feature amount.

  Next, the overall classifier 50 performs overall identification processing (S103). The overall identification process is a process for identifying (estimating) an image scene indicated by image data based on the overall feature amount. Details of the overall identification process will be described later.

  If the scene can be identified by the overall identification process (YES in S104), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  If the scene cannot be identified by the overall identification process (NO in S104), the partial classifier 60 performs the partial identification process (S105). The partial identification process is a process for identifying the scene of the entire image indicated by the image data based on the partial feature amount. Details of the partial identification processing will be described later.

When the scene can be identified by the partial identification process (YES in S106), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.
If the scene cannot be identified by the partial identification process (NO in S106), the integrated discriminator 70 performs the integrated identification process (S107). Details of the integrated identification process will be described later.

  When the scene can be identified by the integrated identification process (YES in S108), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. On the other hand, if the scene cannot be identified by the integrated identification process (NO in S108), the image indicated by the image data is “other” (other than “landscape”, “evening scene”, “night scene”, “flower” or “autumn leaves”. Is stored in the result storage unit 31B (S110).

=== Overall identification processing ===
FIG. 7 is a flowchart of the overall identification process. Here, the overall identification process will be described with reference to FIG.

  First, the overall classifier 50 selects one sub-classifier 51 from the plurality of sub-classifiers 51 (S201). The overall classifier 50 is provided with five sub-classifiers 51 for identifying whether an image to be identified (identification target image) belongs to a specific scene. The five sub classifiers 51 identify scenes of scenery, evening scene, night scene, flowers, and autumn leaves, respectively. Here, the overall classifier 50 selects the sub classifier 51 in the order of landscape → evening scene → night scene → flower → autumn leaves (the selection order of the sub classifier 51 will be described later). For this reason, first, the sub classifier 51 (landscape classifier 51L) for identifying whether or not the classification target image belongs to a landscape scene is selected.

Next, the overall classifier 50 refers to the classification target table and determines whether or not a scene should be identified using the selected sub-classifier 51 (S202).
FIG. 8 is an explanatory diagram of the identification target table. This identification target table is stored in the result storage unit 31B of the storage unit 31. In the identification target table, all fields are set to zero in the first stage. In the process of S202, the “No” column is referred to, and if it is zero, it is determined as YES, and if it is 1, it is determined as NO. Here, the overall classifier 50 refers to the “No” column in the “Scenery” column in the identification target table, and determines “YES” because it is zero.

  Next, the sub classifier 51 calculates the probability (confidence) that the classification target image belongs to a specific scene based on the entire feature amount (S203). For the sub classifier 51 of this embodiment, a classification method using a support vector machine (SVM) is used. The support vector machine will be described later. When the classification target image belongs to a specific scene, the discriminant of the sub classifier 51 tends to be a positive value. When the classification target image does not belong to a specific scene, the discriminant of the sub classifier 51 tends to be a negative value. Further, the discriminant becomes a larger value as the certainty that the identification target image belongs to the specific scene is higher. For this reason, if the discriminant value is large, the probability (confidence) that the identification target image belongs to a specific scene is high, and if the discriminant value is small, the probability that the classification target image belongs to a specific scene is low. .

  Next, the sub discriminator 51 determines whether or not the value of the discriminant is larger than the positive threshold (S204). If the value of the discriminant is larger than the positive threshold, the sub discriminator 51 determines that the classification target image belongs to a specific scene.

  FIG. 9 is an explanatory diagram of an affirmative threshold value of the overall identification process. In the figure, the horizontal axis indicates an affirmative threshold, and the vertical axis indicates the probability of recall or precision. FIG. 10 is an explanatory diagram of Recall and Precision. If the discriminant value is greater than or equal to the positive threshold, the identification result is Positive. If the discriminant value is not greater than or equal to the positive threshold, the identification result is Negative.

  Recall indicates the recall rate and detection rate. Recall is the ratio of the number of images identified as belonging to a specific scene to the total number of images of the specific scene. In other words, Recall is the probability that the sub-identifier 51 identifies the image as a positive when the image of the specific scene is identified by the sub-identifier 51 (the probability that the image of the specific scene belongs to the specific scene. ). For example, when a landscape image is identified by the landscape classifier 51L, it indicates the probability that the landscape classifier 51L identifies it as belonging to a landscape scene.

  Precision indicates the correct answer rate and the correct answer rate. Precision is the ratio of the number of images in a particular scene to the total number of images identified as Positive. In other words, Precision indicates the probability that the image to be identified is a specific scene when the sub-classifier 51 that identifies the specific scene identifies it as Positive. For example, when the landscape classifier 51L identifies that it belongs to a landscape scene, it indicates the probability that the identified image is really a landscape image.

As can be seen from FIG. 9, the greater the positive threshold, the greater the Precision. For this reason, the larger the positive threshold value, the higher the probability that an image identified as belonging to a landscape scene, for example, is a landscape image. That is, the greater the positive threshold, the lower the probability of misidentification.
On the other hand, the larger the positive threshold, the smaller the Recall. As a result, for example, even when a landscape image is identified by the landscape classifier 51L, it is difficult to correctly identify it as belonging to a landscape scene. By the way, if the image to be identified can be identified as belonging to a landscape scene (YES in S204), the speed of the overall identification process is increased so as not to identify other remaining scenes (such as sunsets). For this reason, the larger the positive threshold, the lower the overall identification processing speed. Further, if the scene can be identified by the overall identification process, the partial identification process is not performed and the speed of the scene identification process is increased (S104). Therefore, as the positive threshold is increased, the scene identification process speed decreases. Become.
That is, if the positive threshold is too small, the probability of misidentification increases, and if it is too large, the processing speed decreases. In this embodiment, since the correct answer rate (Precision) is set to 97.5%, the landscape affirmation threshold is set to 1.27.

If the discriminant value is greater than the affirmative threshold value (YES in S204), the sub-classifier 51 determines that the classification target image belongs to a specific scene and sets an affirmative flag (S205). “Set an affirmative flag” means that the “affirmation” column in FIG. In this case, the overall discriminator 50 ends the overall discrimination process without performing discrimination by the next sub discriminator 51. For example, if the image can be identified as a landscape image, the entire identification process is terminated without identifying the sunset scene or the like. In this case, since the identification by the next sub-identifier 51 is omitted, the speed of the overall identification process can be increased.
If the value of the discriminant is not greater than the positive threshold (NO in S204), the sub discriminator 51 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S206.

  Next, the sub discriminator 51 compares the discriminant value with a negative threshold value (S206). Thereby, the sub classifier 51 determines whether the classification target image does not belong to a predetermined scene. There are two types of such determinations. First, if the value of the discriminant of the sub-identifier 51 of a specific scene is smaller than the first negative threshold, it is determined that the classification target image does not belong to the specific scene. For example, if the discriminant value of the landscape classifier 51L is smaller than the first negative threshold, it is determined that the classification target image does not belong to a landscape scene. Second, if the value of the discriminant of the sub-identifier 51 of a specific scene is larger than the second negative threshold, it is determined that the classification target image does not belong to a scene different from the specific scene. . For example, if the discriminant value of the landscape classifier 51L is larger than the second negative threshold, it is determined that the classification target image does not belong to the night scene.

  FIG. 11 is an explanatory diagram of the first negative threshold. In the figure, the horizontal axis indicates the first negative threshold, and the vertical axis indicates the probability. The bold line in the graph is a True Negative Recall graph, and indicates the probability of correctly identifying an image other than a landscape image as not a landscape image. The thin line in the graph is a False Negative Recall graph, which indicates the probability of erroneously identifying a landscape image that is not a landscape image.

As can be seen from FIG. 11, the smaller the first negative threshold is, the smaller False Negative Recall is. For this reason, the smaller the first negative threshold, the lower the probability that an image identified as not belonging to a landscape scene is a landscape image, for example. That is, the probability of misidentification is reduced.
On the other hand, the True Negative Recall decreases as the first negative threshold decreases. As a result, it is difficult to identify an image other than a landscape image unless it is a landscape image. On the other hand, if it is possible to identify that the identification target image is not a specific scene, the process by the sub partial classifier 61 for the specific scene is omitted during the partial identification process to increase the scene identification processing speed (described later in FIG. 14). S302). For this reason, the scene identification processing speed decreases as the first negative threshold is decreased.
That is, if the first negative threshold is too large, the probability of misidentification increases, and if it is too small, the processing speed decreases. In the present embodiment, in order to set False Negative Recall to 2.5%, the first negative threshold is set to -1.10.

  By the way, if the probability that an image belongs to a landscape scene is high, the probability that the image belongs to a night scene is inevitably low. For this reason, when the discriminant value of the landscape discriminator 51L is large, it may be identified that the scene is not a night scene. In order to perform such identification, a second negative threshold is provided.

  FIG. 12 is an explanatory diagram of the second negative threshold. In the figure, the horizontal axis indicates the value of the landscape discriminant, and the vertical axis indicates the probability. In the same figure, the Recall graph of the night view is drawn with a dotted line together with the Recall and Precision graph of FIG. If attention is paid to this dotted line graph, if the value of the discriminant of the landscape is larger than −0.45, the probability that the image is a night scene image is 2.5%. In other words, if the landscape discriminant value is greater than −0.45, even if the image is identified as not a night scene image, the probability of misidentification is only 2.5%. Therefore, in the present embodiment, the second negative threshold is set to −0.45.

  When the discriminant value is smaller than the first negative threshold value, or when the discriminant value is larger than the second negative threshold value (YES in S206), the sub-classifier 51 determines that the classification target image belongs to a predetermined scene. It is determined not to do so, and a negative flag is set (S207). “Set a negative flag” means to set the “No” column in FIG. For example, when it is determined that the image to be identified does not belong to a landscape scene based on the first negative threshold, the “denial” column in the “landscape” column is 1. Further, when it is determined that the identification target image does not belong to the night scene based on the second negative threshold, the “Negation” field in the “Night scene” field is “1”.

  FIG. 13A is an explanatory diagram of a threshold table. The threshold value table may be stored in the storage unit 31 or may be incorporated in a part of a program for executing the overall identification process. The threshold table stores data related to the affirmative threshold and the negative threshold described above.

  FIG. 13B is an explanatory diagram of threshold values in the landscape classifier 51L described above. An affirmative threshold value and a negative threshold value are preset in the landscape discriminator 51L. 1.27 is set as the positive threshold. The negative threshold includes a first negative threshold and a second negative threshold. -1.10 is set as the first negative threshold. In addition, a value is set for each scene other than the landscape as the second negative threshold.

  FIG. 13C is an explanatory diagram outlining the processing of the landscape classifier 51L described above. Here, for simplification of description, only the night view will be described for the second negative threshold. If the discriminant value is greater than 1.27 (YES in S204), the landscape classifier 51L determines that the classification target image belongs to a landscape scene. If the discriminant value is 1.27 or less (NO in S204) and is greater than −0.45 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to the night scene. To do. If the value of the discriminant is smaller than −1.10 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to a landscape scene. Note that the landscape discriminator 51L also determines whether the image to be identified does not belong to the scene based on the second negative threshold for sunset scenes, flowers, and autumn leaves. However, since these second negative threshold values are larger than the positive threshold values, the landscape discriminator 51L does not determine that the classification target image does not belong to these scenes.

  In the case of NO in S202, in the case of NO in S206, or when the processing in S207 is completed, the overall discriminator 50 determines the presence or absence of the next sub discriminator 51 (S208). Here, since the process by the landscape classifier 51L is finished, the overall classifier 50 determines in S208 that there is a next sub-classifier 51 (evening scene classifier 51S).

  Then, when the process of S205 is finished (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S208 that there is no next sub-classifier 51 (the identification target image is a specific image). When it cannot be determined that the scene belongs to the scene), the overall discriminator 50 ends the overall discrimination process.

As already described, when the overall identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the overall identification process (S104 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.
If the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  By the way, although not described above, the overall discriminator 50, when the sub discriminator 51 calculates the discriminant value, the Precision corresponding to the discriminant value is stored in the result storage unit 31B as information on the certainty factor. Store (see also FIG. 19 described later). Of course, the discriminant value itself may be stored as information on the certainty factor.

=== Partial identification processing ===
FIG. 14 is a flowchart of the partial identification process. The partial identification process is performed when the scene cannot be identified by the overall identification process (NO in S104 of FIG. 5). As will be described below, the partial identification process is a process for identifying the scene of the entire image by identifying each scene of the divided partial images. Here, the partial identification process will be described with reference to FIG.

  First, the partial classifier 60 selects one sub partial classifier 61 from the plurality of sub partial classifiers 61 (S301). The partial discriminator 60 is provided with three sub partial discriminators 61. Each sub partial discriminator 61 discriminates whether or not each partial image divided into 8 × 8 64 blocks belongs to a specific scene. Here, the three sub partial classifiers 61 identify the scenes of sunset, flowers, and autumn leaves, respectively. Here, the partial discriminator 60 selects the sub partial discriminator 61 in the order of sunset scene → flower → autumn leaves (the selection order of the sub partial discriminator 61 will be described later). Therefore, first, the sub partial classifier 61 (evening scene partial classifier 61S) for identifying whether or not the partial image belongs to the sunset scene is selected.

  Next, the partial discriminator 60 refers to the discrimination target table (FIG. 8) and determines whether or not the scene should be discriminated using the selected sub partial discriminator 61 (S302). Here, the partial discriminator 60 refers to the “No” column of the “Evening Scene” column in the classification target table, and determines YES if it is zero, and NO if it is 1. When the evening scene classifier 51S sets a negative flag with the first negative threshold during the overall identification process or when another sub-classifier 51 sets a negative flag with the second negative threshold, in S302 It is judged as NO. If it is determined NO, the sunset partial identification process is omitted, and the partial identification process speed increases. However, for the convenience of explanation, it is assumed that YES is determined here.

Next, the sub partial discriminator 61 selects one partial image from the partial images divided into 8 × 8 64 blocks (S303).
FIG. 15 is an explanatory diagram of the order of partial images selected by the evening scene partial classifier 61S. When a scene of the entire image is identified from the partial image, it is desirable that the partial image used for identification is a portion where the subject exists. Therefore, in this embodiment, thousands of samples of sunset scene images are prepared, each sunset scene image is divided into 64 blocks of 8 × 8, and blocks including sunset scene partial images (sun and sky partial images of the sunset scene) are included. The presence probability of the sunset partial image in each block was calculated based on the extracted block position. And in this embodiment, a partial image is selected in an order from a block with a high existence probability. Note that the selection order information shown in the figure is stored in the memory 23 as part of the program.

  In the case of an evening scene image, since the sky of the evening scene often spreads from the vicinity of the center to the upper half, the existence probability increases in the upper half block from the vicinity of the center. In the case of an evening scene image, the lower 1/3 of the image is shaded by backlight, and the partial image alone often cannot be distinguished from the evening scene or the night scene, so the existence probability is lower in the lower 1/3 block. In the case of a flower image, since the composition is often such that a flower is arranged near the center, the probability of existence of a flower partial image near the center increases.

  Next, the sub partial classifier 61 determines whether or not the partial image belongs to a specific scene based on the partial feature amount of the selected partial image (S304). Similar to the sub classifier 51 of the overall classifier 50, the sub partial classifier 61 uses a discrimination method using a support vector machine (SVM). The support vector machine will be described later. If the discriminant value is a positive value, it is determined that the partial image belongs to a specific scene, and the sub partial classifier 61 increments the positive count value. If the discriminant value is a negative value, it is determined that the partial image does not belong to a specific scene, and the sub partial discriminator 61 increments the negative count value.

  Next, the sub partial discriminator 61 determines whether or not the positive count value is larger than the positive threshold value (S305). The positive count value indicates the number of partial images determined to belong to a specific scene. If the positive count value is larger than the affirmative threshold (YES in S305), the sub partial classifier 61 determines that the classification target image belongs to a specific scene, and sets an affirmative flag (S306). In this case, the partial discriminator 60 ends the partial discriminating process without performing discrimination by the next sub partial discriminator 61. For example, if the image can be identified as an evening scene image, the partial identification process is terminated without identifying flowers and autumn leaves. In this case, since the identification by the next sub partial classifier 61 is omitted, the speed of the partial classification process can be increased.

  If the positive count value is not greater than the positive threshold value (NO in S305), the sub partial classifier 61 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S307.

  If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold (YES in S307), the sub partial discriminator 61 proceeds to the process of S309. If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold value, the positive count value does not become larger than the positive threshold value even if the positive count value is incremented by all the remaining partial images. By proceeding with the process, the remaining partial images are not identified by the support vector machine. Thereby, the speed of the partial identification process can be increased.

If the sub partial discriminator 61 determines NO in S307, the sub partial discriminator 61 determines whether there is a next partial image (S308). In the present embodiment, not all of the partial images divided into 64 are selected in order. In FIG. 15, only the top 10 partial images indicated by thick frames are selected in order. Therefore, when the identification of the tenth partial image is completed, the sub partial classifier 61 determines in S308 that there is no next partial image. (In consideration of this point, the “number of remaining partial images” in S307 is also determined.)
FIG. 16 is a Recall and Precision graph when an evening scene image is identified using only the top 10 partial images. If an affirmative threshold as shown in the figure is set, the accuracy rate (Precision) can be set to about 80%, the recall rate (Recall) can be set to about 90%, and identification with high accuracy is possible.

In this embodiment, the evening scene image is identified using only 10 partial images. For this reason, in the present embodiment, it is possible to increase the speed of the partial identification process compared to the case where the evening scene image is identified using all 64 partial images.
In this embodiment, the sunset scene image is identified using the top tenth partial image having a high existence probability of the sunset scene partial image. For this reason, in the present embodiment, it is possible to set both Recall and Precision higher than the identification of an evening scene image using 10 partial images extracted by ignoring the existence probability.
In this embodiment, the partial images are selected in descending order of the existence probability of the sunset partial image. As a result, the determination in S305 is likely to be YES at an early stage. For this reason, in the present embodiment, the speed of the partial identification process can be increased as compared with the case where the partial images are selected in the order in which the presence probability level is ignored.

  When it is determined YES in S307, or when it is determined that there is no next partial image in S308, the sub partial discriminator 61 determines whether or not the negative count value is larger than the negative threshold (S309). . Since this negative threshold performs substantially the same function as the negative threshold (S206 in FIG. 7) in the above-described overall identification process, detailed description thereof is omitted. When YES is determined in S309, a negative flag is set as in S207 of FIG.

  In the case of NO in S302, in the case of NO in S309, or when the process of S310 is completed, the partial discriminator 60 determines whether or not there is a next sub partial discriminator 61 (S311). In the case after the processing by the evening scene partial classifier 61S is finished, since the flower partial classifier 61F and the autumnal leaves partial classifier 61R are still present as the sub partial classifier 61, the partial classifier 60 determines the next sub partial classifier in S311. It is determined that there is a container 61.

  Then, when the process of S306 is completed (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S311 that there is no next sub partial classifier 61 (the identification target image is specified). If it cannot be determined that the scene belongs to the scene), the partial discriminator 60 ends the partial discrimination processing.

As already described, when the partial identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the partial identification process (S106 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.
When the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

  In the above description, the evening scene partial classifier 61S identifies evening scene images using ten partial images. However, the number of partial images used for identification is not limited to ten. Further, the other sub partial classifier 61 may identify an image using a different number of partial images from the sunset scene partial classifier 61S. In the present embodiment, it is assumed that the flower partial discriminator 61F identifies a flower image using 20 partial images, and the autumnal foliage partial discriminator 61R identifies a autumnal foliage image using 15 partial images.

=== Support vector machine ===
Before describing the integrated identification process, the support vector machine (SVM) used in the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process will be described.

FIG. 17A is an explanatory diagram of determination by the linear support vector machine. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. The learning sample is divided into two classes A and B. In the figure, samples belonging to class A are indicated by circles, and samples belonging to class B are indicated by squares.
A boundary that divides the two-dimensional space into two is defined by learning using the learning sample. The boundary is defined by <w · x> + b = 0 (where x = (x1, x2), w is a weight vector, and <w · x> is an inner product of w and x). However, the boundary is defined by learning using a learning sample so that the margin is maximized. That is, in the case of the figure, the boundary is not a thick dotted line but a thick solid line.
The discrimination is performed using the discriminant f (x) = <w · x> + b. It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined.

  Here, the description is made using a two-dimensional space, but the present invention is not limited to this (that is, the feature amount may be two or more). In this case, the boundary is defined by a hyperplane.

  By the way, there are cases where the two classes cannot be separated by a linear function. In such a case, if the determination is performed by the linear support vector machine, the accuracy of the determination result is lowered. Therefore, if the feature quantity of the input space is nonlinearly transformed, that is, if the input space is nonlinearly mapped to a certain feature space, it can be separated by a linear function in the feature space. This is used in the nonlinear support vector machine.

  FIG. 17B is an explanatory diagram of discrimination using a kernel function. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. If the nonlinear mapping from the input space of FIG. 17B becomes a feature space as shown in FIG. 17A, it can be separated into two classes by a linear function. If the boundary is defined so that the margin is maximized in this feature space, the inverse mapping of the boundary in the feature space becomes the boundary shown in FIG. 17B. As a result, the boundary becomes nonlinear as shown in FIG. 17B.

In this embodiment, by using a Gaussian kernel, the discriminant f (x) becomes as follows (where M is the number of features and N is the number of learning samples (or contributes to the boundary): The number of learning samples), w _i is a weighting factor, y _j is the feature quantity of the learning sample, and x _j is the feature quantity of the input x).

  It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined. Further, the larger the value of the discriminant f (x), the higher the probability that the input x (this input x is different from the learning sample) belongs to the class A. On the contrary, the smaller the value of the discriminant f (x), the lower the probability that the input x (this input x is different from the learning sample) belongs to the class A.

  In the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process, the value of the discriminant f (x) of the support vector machine is used. The calculation of the value of the discriminant f (x) by the support vector machine takes time when the number of learning samples (in this embodiment, tens of thousands) increases. For this reason, the sub partial classifier 61 that needs to calculate the value of the discriminant f (x) a plurality of times requires more processing time than the sub classifier 51 that only needs to calculate the value of the discriminant f (x) once. It takes.

  An evaluation sample is prepared separately from the learning sample. The above Recall and Precision graphs are based on the identification results for the evaluation samples.

=== Integrated identification processing ===
In the above-described overall identification process and partial identification process, the positive threshold value in the sub-classifier 51 and the sub-classifier 61 is set relatively high, and the Precision (correct answer rate) is set high. This is because, for example, if the accuracy rate of the landscape classifier 51L of the overall classifier is set low, the scene classifier 51L erroneously identifies the autumnal image as a landscape image, and before the autumnal classifier 51R performs the classification. This is because a situation occurs in which the entire identification process ends. In the present embodiment, by setting the Precision (accuracy rate) high, an image belonging to a specific scene is identified by the sub-classifier 51 (or sub-partial classifier 61) of the specific scene ( For example, the autumnal leaves image is identified by the autumnal leaves discriminator 51R (or the autumnal leaf partial discriminator 61R).

  However, if the Precision (accuracy rate) of the overall identification process or the partial identification process is set to be high, there is a high possibility that the scene cannot be identified by the overall identification process or the partial identification process. Therefore, in this embodiment, when the scene cannot be identified by the overall identification process and the partial identification process, the integrated identification process described below is performed.

  FIG. 18 is a flowchart of the integrated identification process. As will be described below, the integrated identification process is a process of selecting a scene with the highest certainty factor based on the discriminant value of each sub-classifier 51 in the overall identification process.

  First, the integrated discriminator 70 extracts a positive scene based on the discriminant values of the five sub discriminators 51 (S401). At this time, the value of the discriminant calculated by each sub classifier 51 during the overall identification process is used.

Next, the integrated discriminator 70 determines whether or not a scene having a positive discriminant value exists (S402).
If there is a scene with a positive discriminant value (YES in S402), an affirmative flag is set in the maximum value scene column (S403), and the integrated identification process is terminated. Accordingly, it is determined that the identification target image belongs to the maximum value scene.
On the other hand, if there is no scene having a positive discriminant value (NO in S402), the integrated identification process is terminated without setting an affirmative flag. As a result, there is no scene in the affirmative column of the identification target table in FIG. That is, it cannot be identified to which scene the identification target image belongs.

  As already described, when the integrated identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the integrated identification process (S108 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column. If it is determined NO in S402, the determination in S108 is also NO.

=== Selection Order of Sub Classifier / Sub Partial Classifier ===
<Overview>
In the above description, the overall classifier 50 selects the sub-classifier 51 in the order of landscape → evening scene → night scene → flower → autumn leaves in S201 in FIG. Further, the partial discriminator 60 selects the sub partial discriminator 61 in the order of evening scene → flower → autumn leaves in S301 of FIG.
However, if the order of the sub classifier 51 (or sub partial classifier 61) is fixed, the processing speed of the scene classification process cannot be increased. For example, when there is a high possibility that the image to be identified is an autumnal image, the autumnal color classifier 51R is selected last in the overall classification process, or the autumnal color partial classifier 61R is selected last in the partial identification process. The expected value of the processing speed of the scene identification process is slow.
Therefore, in this embodiment, the selection order of the sub classifier 51 and the sub partial classifier 61 is changed according to the evaluation result of the classification target image.

<First embodiment>
In the first embodiment, the certainty factor calculated in the overall identification process is used as the evaluation result of the identification target image, and the selection order of the sub partial classifiers 61 in the partial identification process is changed. The first embodiment will be described in detail below.

  The partial identification process described above is performed when the scene cannot be identified (NO in S104) by the overall identification process (S103 in FIG. 5) (S105). That is, the entire identification process is always performed before the partial identification process is performed.

  FIG. 19 is an explanatory diagram of the certainty factor calculated in the overall identification process. As described above, when the discriminant value is calculated by the sub discriminator 51, the overall discriminator 50 stores the Precision corresponding to the discriminant value in the result storage unit 31B as information on the certainty factor. Here, the certainty of the landscape is 0%, the certainty of the evening scene is 30%, the certainty of the night view is 0%, the certainty of the flower is 40%, and the certainty of the autumn leaves is 30%. Suppose that Here, the total certainty of all scenes is exactly 100%, but the certainty of each scene is calculated based on the discriminant value, so the total certainty of all scenes is It may not be 100%.

  In S301 of FIG. 14, the partial discriminator 60 first extracts the certainty of evening scene / flower / autumn among the certainty of FIG. Then, the partial discriminator 60 selects the sub partial discriminators 61 in order from the scene with the highest certainty factor. For this reason, the flower part discriminator 61F is selected first.

  If the flower part classifier 61F can identify that the identification target image is a flower image (YES in S305 in FIG. 14), the other sub part classifier 61 (in this case, the sunset scene part classifier 61S or the autumnal leaves part). Since the processing of S303 to S311 by the discriminator 61R) can be omitted, the speed of the partial discrimination processing is increased. In the first embodiment, since the sub partial classifiers 61 are selected in order from the scene with the highest certainty factor, the probability that the processing by the other sub partial classifiers 61 is omitted increases, and the speed of the partial classification process increases. The probability of becoming higher. In other words, in the first embodiment, the expected processing time of the partial identification process (S105 in FIG. 5) is shorter than when the order of the sub partial classifiers 61 is fixed.

  When the flower part classifier 61F cannot identify the flower image (NO in S305), in the next S301, one of the remaining sub partial classifiers 61 (the sunset scene part classifier 61S and the autumn leaves part classifier 61R) is selected. Is done. Here, since the certainty of the evening scene and the autumn leaves is the same, it becomes a question which of the evening scene partial classifier 61S and the autumnal scene partial classifier 61R should be selected next.

  In the first embodiment, when the certainty factor is the same, the partial discriminator 60 selects the shorter processing time of each sub partial discriminator 61. Thereby, the expected value of the processing time of the whole partial identification process can be shortened.

  As described above, the evening scene partial classifier 61S identifies the evening scene image using the ten partial images. On the other hand, the autumnal leaf partial classifier 61R identifies the autumnal leaf image using 15 partial images. For this reason, in the first embodiment, the evening scene partial classifier 61S is considered to have a shorter processing time than the autumnal leaves partial classifier 61R. Therefore, when the certainty of the evening scene and the autumn leaves is the same, the evening scene partial classifier 61S is selected with priority over the autumnal scene partial classifier 61R.

  In the first embodiment, the comparison of the processing time of the sub partial classifier 61 is performed based on the number of partial images used for identification, but is not limited thereto. For example, the processing time of each sub partial classifier 61 may be stored in advance, and the length of the processing time of the sub partial classifier 61 may be compared based on the stored processing time.

  As described above, if the certainty factor calculated in the overall identification process is as shown in FIG. 19, the partial classifier 60 selects the sub partial classifier 61 in the order of flower → evening scene → autumn leaves. Thereby, the expected value of the processing time of the partial identification processing is shortened.

<Second embodiment>
In the second embodiment, a part of the entire feature amount is used as the evaluation result of the identification target image, and the selection order of the sub classifiers 51 in the entire identification process is changed.

  In general, since a night scene image is dark as a whole, the average luminance value of the entire image is a dark value. On the other hand, since the entire image of a landscape image or the like is bright, the average value of the luminance of the entire image shows a bright value. For this reason, if the average luminance value of the identification target image is a dark value, the probability of being a night scene image is high, and the probability of being a landscape image is low. Using this fact, in the second embodiment, the average value of the luminance (Y) of the identification target image is used to change the selection order of the sub-identifiers 51 in the overall identification process.

  FIG. 20 is an explanatory diagram of the selection order of the sub-identifier 51 in the second embodiment. Luminance data corresponding to each scene is stored in the storage unit 31 in advance, and the overall discriminator 50 refers to this data in S201 of FIG. Alternatively, luminance data of each scene is incorporated in advance in a program for executing the overall identification process.

  Then, in S201 of FIG. 7, the overall classifier 50 calculates the difference between each luminance data and the average value of Y (luminance) of the classification target image, and selects the sub-classifier 51 in order from the scene with the smallest difference. . In the case of FIG. 20, the overall classifier 50 selects the sub classifier 51 in the order of night view → evening scene → autumn leaves → flower → landscape.

  If the night scene classifier 51N can identify that the classification target image is a night scene image (YES in S204 in FIG. 7), the processing of S203 to S208 by the other sub classifiers 51 can be omitted. The speed of the entire identification process is increased. In the second embodiment, since the sub discriminators 51 are selected in descending order of the possibility of being YES in S204, the probability that the processing by the other sub discriminators 51 is omitted increases, and the overall discrimination processing is performed. The probability that the speed of will increase. In other words, in the second embodiment, the expected value of the processing time of the overall identification process is shorter than when the order of the sub-identifiers 51 is fixed.

  In the second embodiment, the selection order of the sub-identifier 51 is changed based on the average luminance value of the identification target image, but the present invention is not limited to this. For example, the selection order of the sub-identifier 51 may be changed based on the hue of the identification target image (H data in the HSB color space). In this case, if the hue of the identification target image is close to red, the sub-classifier 51 for evening scenes and autumn leaves is selected with priority. If the hue of the classification target image is close to blue or green, the sub-classifier 51 for landscape is prioritized. It should be selected.

  In the second embodiment, the selection order of the sub-identifier 51 is changed based on one value related to the image to be identified, but two or more values (for example, an average value of luminance and an average value of color difference). The selection order of the sub classifiers 51 may be changed based on the above. In the second embodiment, the selection order of the sub discriminators 51 is changed based on a part of the entire feature amount (here, the average value of luminance). However, information other than the entire feature amount may be used. good.

  As described above, if the average luminance value of the identification target image is as shown in FIG. 20, the overall classifier 50 selects the sub classifier 51 in the order of night view → evening scene → autumn leaves → flower → landscape. Thereby, the expected value of the processing time of the entire identification process is shortened.

=== Other Embodiments ===
Although a printer or the like as one embodiment has been described, the above embodiment is for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and it is needless to say that the present invention includes equivalents thereof. In particular, the embodiments described below are also included in the present invention.

<About the printer>
In the above-described embodiment, the printer 4 performs the scene identification process, but the digital still camera 2 may perform the scene identification process. Further, the image identification device that performs the above-described scene identification processing is not limited to the printer 4 or the digital still camera 2. For example, an image identification device such as a photo storage that stores a large amount of image files may perform the scene identification process described above. Of course, a personal computer or a server installed on the Internet may perform the scene identification process.

<About image files>
The image file described above is in the Exif format, but the image file format is not limited to this. Further, the above-described image file is a still image, but may be a moving image. In short, if the image file includes image data and additional data, the scene identification process as described above can be performed.

<About Support Vector Machine>
For the above-described sub classifier 51 and sub partial classifier 61, a classification method using a support vector machine (SVM) is used. However, the method for identifying whether or not the identification target image belongs to a specific scene is not limited to using a support vector machine. For example, pattern recognition such as a neural network may be employed.

<About scene identification>
In the above-described embodiment, the sub classifier 51 and the sub partial classifier 61 identify whether or not the image indicated by the image data belongs to a specific scene. However, the sub discriminator 51 and the sub partial discriminator 61 are not limited to identifying whether or not they belong to a specific scene, as long as they can classify whether or not an image indicated by image data belongs to a specific category. For this reason, the sub classifier 51 and the sub partial classifier 61 may identify whether the image indicated by the image data has a specific pattern shape, for example.

<Selection order of sub classifier / sub partial classifier>
In the above-described embodiment, after the overall identification process by the overall classifier 50, the partial identification process by the partial classifier 60 is performed (see FIGS. 6 and 7). However, the present invention is not limited to this.
For example, in the case of FIG. 20, the scene discriminating unit 33 performs “identification processing by the night scene discriminator 51N” → “identification processing by the evening scene discriminator 51S” → “discrimination processing by the evening scene partial discriminator 61S” → “autumn leaf discriminator 51R. "Identification process by" and "Identification process by autumn color partial discriminator 61R"-> ..., the sub-classifier 51 of the overall discriminator 50 and the sub-partial discriminator 61 of the partial classifier 60 are mixed and selected. May be.
Even in this case, for example, when the night scene classifier 51N can identify that the classification target image is a night scene image, the classification process speed increases if the identification process by the other sub classifier 51 is omitted.

=== Summary ===
(1) In the above-described embodiment, the overall classifier 50 has a plurality of sub-classifiers 51, and the partial classifier 60 has a plurality of sub-partial classifiers 61. The sub classifier 51 and the sub partial classifier 61 perform identification processing for identifying whether or not the image indicated by the image data belongs to a specific scene. And in the above-mentioned embodiment, the scene (an example of a category) of the image which image data shows is classified by combining several such sub discriminators.

  Here, if the identification process is performed by all the sub classifiers 51 (or all the sub partial classifiers 61) every time the scene identification process is performed, the processing speed of the scene identification process becomes slow. Therefore, in the above-described embodiment, when the sub discriminator 51 (or the sub partial discriminator 61) can identify that the image indicated by the image data belongs to a specific scene, the processing by another sub discriminator 51 is omitted. The processing speed of scene identification processing is increased. For example, if the landscape classifier 51L of the overall classifier 50 can identify a landscape image (YES in S204 of FIG. 7), the process by the evening scene classifier 51S of the overall classifier 50 or the evening scene partial classifier 61S of the partial classifier 60 will be described. The processing is omitted, and the processing speed of the scene identification processing is increased.

  However, if the order of the sub classifier 51 and the sub partial classifier 61 is fixed, the processing speed of the scene classification process cannot be increased. For example, when the identification target image is highly likely to be an autumnal image, the autumnal color identifier 51R is selected last in the overall identification process, or the autumnal color partial identifier 61R is selected last in the partial identification process. The expected value of the processing speed of the scene identification process becomes slow.

Therefore, in the above-described embodiment, the printer-side controller 20 evaluates the identification target image, and determines the selection order of the sub classifier 51 (or sub partial classifier 61) according to the evaluation result. For example, in the above-described first embodiment, the certainty factor calculated in the overall identification process is used as the evaluation result of the identification target image, and the selection order of the sub partial classifier 61 in the partial identification process is determined. Further, in the above-described second embodiment, a part of the entire feature amount is used as the evaluation result of the identification target image, and the selection order of the sub classifiers 51 in the entire identification process is determined.
As a result, the probability that the identification process of the sub-classifier with the later priority is omitted is increased, and the processing speed of the scene identification process is increased.

(2) The above-described overall classifier 50 is based on the overall feature quantity and the evening scene classifier 51S that identifies whether or not the classification target image belongs to an evening scene (corresponding to the first category). And a flower discriminator 51F for discriminating whether or not the classification target image belongs to a flower scene (corresponding to the second category). Further, the partial discriminator 60 is an evening scene partial discriminator 61S for discriminating whether or not the classification target image belongs to an evening scene based on the partial feature amount, and the classification target image is a flower scene based on the partial feature amount. It includes a flower part identifier 61F that identifies whether it belongs or not.

In the first embodiment described above, the printer-side controller 20 uses the certainty factor calculated by the evening scene discriminator 51S of the overall discriminator 50 and the certainty factor calculated by the flower discriminator 51F. The order of the evening scene partial classifier 61S and the flower partial classifier 61F is determined. For example, as shown in FIG. 19, when the certainty factor calculated by the evening scene classifier 51S of the overall classifier 50 is 30% and the certainty factor calculated by the flower classifier 51F is 40%, the flower partial classifier 61F has the sunset scene. It is selected before the partial discriminator 61S.
This increases the probability that processing by the evening scene partial discriminator 61S selected later is omitted, so that the expected value of the processing time of the partial discriminating processing (S105 in FIG. 5) fixes the order of the sub partial discriminators 61. It becomes shorter than the case.

(3) The evening scene discriminator 51S of the overall discriminator 50 described above calculates a discriminant value (corresponding to the first data) based on the overall feature quantity, and compares the discriminant value with an affirmative threshold value. , It is determined whether or not the image to be identified belongs to a sunset scene. Similarly, the flower discriminator 51F of the overall discriminator 50 calculates a discriminant value (corresponding to the second data) based on the overall feature amount, and compares the discriminant value with an affirmative threshold value, thereby identifying the discriminant. Whether or not the target image belongs to a flower scene is identified. Note that the value of the discriminant is related to the probability of belonging to a specific scene (see Precision in FIG. 9), and the greater the value of the discriminant, the higher the probability that the image to be identified belongs to the specific scene.

  In the first embodiment described above, the certainty factor (%) obtained from the discriminant value is used as the evaluation result of the classification target image, and the selection of the evening scene partial classifier 61S and the flower partial classifier 61F in the partial classification processing is performed. Determine the order. As a result, since the data calculated in the overall identification process is diverted without newly calculating data for determining the order of the sub partial classifiers 61 of the partial classifier 60, the processing speed is increased.

(4) The sub partial classifier 61 of the partial classifier 60 described above identifies whether each partial image belongs to a specific scene based on the partial feature amount of each of the 64 divided partial images. Then, based on the number of partial images identified as belonging to a specific scene, it is identified whether the identification target image belongs to a specific scene. For this reason, the sub partial discriminator 61 of the partial discriminator 60 has to calculate the discriminant value a plurality of times, and the processing speed is slower than that of the sub discriminator 51 of the overall discriminator 50.
For this reason, if processing by the sub partial classifier 61 selected later can be omitted by changing the order of the sub partial classifiers 61 of the partial classifier 60, the processing speed is significantly improved.

(5) If the scene can be identified by the above-described overall identification process (YES in S104), the partial identification process is omitted. This increases the speed of the scene identification process.

(6) In the first embodiment described above, the processing time for the sub partial discriminator 61 of the partial discriminator 60 to calculate a plurality of discriminant values and discriminate whether or not the classification target image belongs to a specific scene. Is slower than the processing time for the sub classifier 51 of the overall classifier 50 to calculate the value of the discriminant (corresponding to the evaluation result). That is, in the first embodiment described above, the order of the sub partial classifiers 61 of the partial classifier 60 having a low processing speed is changed.

  In the second embodiment described above, the processing time for the sub-classifier 51 of the overall classifier 50 to calculate the discriminant value and identify whether or not the classification target image belongs to a specific scene is It is slower than the processing time for calculating the average value of the luminance (corresponding to the evaluation result) used for determining the order of the discriminator 51 (the calculation of the discriminant value by the support vector machine includes the number of learning samples (this number (In the embodiment, it takes time when tens of thousands) increase).

In this way, by changing the order of the sub classifiers that take a long time to process, if the process by the sub classifier to be selected later can be omitted, the processing speed is significantly improved.
Note that even if the order of the overall identification process and the partial identification process is changed and the order of the sub-classifiers 51 of the overall classifier 50 is changed based on the processing result of the partial classifier 60, the processing as in the above embodiment is performed. Speed is not expected to improve.

(7) The printer described above (corresponding to an image identification device) includes a printer-side controller 20 (see FIG. 2). The printer-side controller 20 evaluates the identification target image and determines the selection order of the sub-identifiers according to the evaluation result. For example, in the first embodiment described above, the printer-side controller 20 uses the certainty factor calculated in the overall identification process as the evaluation result of the identification target image, and determines the selection order of the sub partial classifier 61 in the partial identification process. is doing.
As a result, the probability that the identification process of the sub-classifier with the later priority is omitted is increased, and the processing speed of the scene identification process is increased.

(8) The above-described memory 23 stores a program for causing the printer 4 to execute the processes of FIGS. 5, 7, and 14. That is, the program includes a code for evaluating the image indicated by the image data, a first identification process for identifying whether the image indicated by the image data belongs to a first category based on the evaluation result, and the image A code for determining an order of a second identification process for identifying whether an image indicated by data belongs to a second category different from the first category, the first identification process, and the second identification process; And a code for omitting the identification process performed after the first identification process and the second identification process when the image can be identified as belonging to a specific category in the identification process performed first. Yes.

It is explanatory drawing of an image processing system. 2 is an explanatory diagram of a configuration of a printer. FIG. It is explanatory drawing of the automatic correction function of a printer. It is explanatory drawing of the relationship between the scene of an image, and the correction content. It is a flowchart of the scene identification process by a scene identification part. It is explanatory drawing of the function of a scene identification part. It is a flowchart of a whole identification process. It is explanatory drawing of an identification object table. It is explanatory drawing of the affirmation threshold value of the whole identification process. It is explanatory drawing of Recall and Precision. It is explanatory drawing of a 1st negative threshold value. It is explanatory drawing of a 2nd negative threshold value. FIG. 13A is an explanatory diagram of a threshold table. FIG. 13B is an explanatory diagram of threshold values in the landscape classifier. FIG. 13C is an explanatory diagram of an outline of the process of the landscape classifier. It is a flowchart of a partial identification process. It is explanatory drawing of the order of the partial image which an evening scene partial identifier selects. It is a Recall and Precision graph when the evening scene image is identified only by the top 10 partial images. FIG. 17A is an explanatory diagram of determination by the linear support vector machine. FIG. 17B is an explanatory diagram of discrimination using a kernel function. It is a flowchart of an integrated identification process. It is explanatory drawing of the certainty degree calculated in the case of a whole identification process. It is explanatory drawing of the selection order of the sub discrimination device 51 in 2nd Example.

Explanation of symbols

2 Digital still camera, 2A mode setting dial,
4 Printer, 6 Memory card,
10 printing mechanism, 11 head, 12 head control unit, 13 motor, 14 sensor,
20 printer-side controller, 21 slots, 22 CPU, 23 memory,
24 control unit, 25 drive signal generator,
31 storage unit, 31A image storage unit, 31B result storage unit,
32 face identification unit, 33 scene identification unit, 34 image correction unit, 35 printer control unit,
40 feature quantity acquisition unit, 50 global classifier, 51 sub classifier, 51L landscape classifier,
51S evening scene classifier, 51N night scene classifier, 51F flower classifier, 51R autumn leaves classifier,
60 partial classifiers, 61 sub partial classifiers, 61S evening scene partial classifiers,
61F Flower partial classifier, 61R Colored leaves partial classifier, 70 Integrated classifier

Claims (8)

Evaluate the image indicated by the image data,
A first identification process for identifying whether the image indicated by the image data belongs to the first category based on the evaluation result, and the image indicated by the image data belongs to a second category different from the first category Determining the order of the second identification process for identifying whether or not
If the image can be identified as belonging to a specific category in the first identification process and the second identification process, the first identification process and the second identification process. An image identification method characterized by omitting a subsequent identification process. The image identification method according to claim 1,
The image identification method includes:
First overall identification processing for identifying whether or not the image belongs to the first category based on an overall feature amount indicating an overall feature of the image;
A second overall identification process for identifying whether the image belongs to the second category based on the overall feature amount;
A first partial identification process for identifying whether or not the image belongs to the first category based on a partial feature amount indicating a partial feature of the image;
A second partial identification process for identifying whether the image belongs to the second category based on the partial feature amount;
Have
The first identification process corresponds to the first partial identification process, the second identification process corresponds to the second partial identification process,
An image identification method, wherein an order of the first identification process and the second identification process is determined by using the result of the first overall identification process and the result of the second overall identification process as the evaluation result. The image identification method according to claim 2,
In the first overall identification process, first data corresponding to a probability that the image belongs to the first category is calculated based on the overall feature amount, and the image is assigned to the first category based on the first data. Whether it belongs or not,
In the second overall identification process, second data corresponding to the probability that the image belongs to the second category is calculated based on the overall feature amount, and the image is assigned to the second category based on the second data. Whether it belongs or not,
The image identification method, wherein the first data and the second data are used as the evaluation result. The image identification method according to claim 2 or claim 3, wherein
In the first partial identification process and the second partial identification process,
Whether each of the partial images belongs to a specific category is identified based on each partial feature amount of each of the partial images obtained by dividing the image,
An image identification method, wherein whether or not the image belongs to a specific category is identified based on the number of the partial images identified as belonging to the specific category. The image identification method according to any one of claims 2 to 4,
The image in which the first identification process and the second identification process are omitted when the image can be identified as belonging to a specific category in the first overall identification process or the second overall identification process. Identification method. The image identification method according to any one of claims 1 to 5,
An image identification method characterized in that a processing time for performing the first identification process and the second identification process is slower than a processing time for evaluating an image indicated by the image data and acquiring the evaluation result. . An image identification device including a controller,
The controller is
Evaluate the image indicated by the image data,
A first identification process for identifying whether the image indicated by the image data belongs to the first category based on the evaluation result, and the image indicated by the image data belongs to a second category different from the first category Determining the order of the second identification process for identifying whether or not
If the image can be identified as belonging to a specific category in the first identification process and the second identification process, the first identification process and the second identification process. An image identification apparatus characterized by omitting a subsequent identification process. In the image identification device,
Evaluate the image indicated by the image data,
A first identification process for identifying whether the image indicated by the image data belongs to the first category based on the evaluation result, and the image indicated by the image data belongs to a second category different from the first category Determining the order of the second identification processing for identifying whether or not
If the image can be identified as belonging to a specific category in the first identification process and the second identification process, the first identification process and the second identification process. A program characterized in that identification processing performed later is omitted.

Images