JP2008242751A - Image discrimination method, image discrimination device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the processing speed of the discrimination processing of image data. <P>SOLUTION: This image discrimination method is provided to perform discrimination processing for successively and selectively discriminating whether or not an image shown by image data is belonging to a specific category for every category (A), to omit the discrimination processing which has not be performed according to the result of certain discrimination processing (B), and to discriminate the category of the image based on the result of the discrimination processing (C). According to the result of the previously performed discrimination processing, the order of selection of the discrimination processing to be performed later is determined. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image identification method, an image identification apparatus, and a program.

  Some digital still cameras have a mode setting dial for setting a shooting mode. When the user sets the shooting mode with the dial, the digital still camera determines shooting conditions (such as exposure time) according to the shooting mode and performs shooting. When shooting is performed, the digital still camera generates an image file. In this image file, additional data such as shooting conditions at the time of shooting is added to the image data of the shot image.

  Image processing is performed on the image data according to the additional data. For example, when a printer performs printing based on an image file, the image data is corrected according to the shooting conditions indicated by the additional data, and printing is performed according to the corrected image data (see Patent Document 1).

In addition, image data is analyzed to classify image categories indicated by the image data (see Patent Documents 2 and 3).
JP 2001-238177 A Japanese Patent Laid-Open No. 10-302067 JP 2006-511000 Gazette

  The image data is classified into a plurality of categories by combining a plurality of discriminators for identifying whether or not the image indicated by the image data belongs to a specific category. When a plurality of discriminators are used in this way, the speed of the discrimination process cannot be increased if the selection order of the discriminators is always the same.

  An object of the present invention is to improve the speed of image data identification processing.

The main invention for achieving the above object is to perform identification processing for identifying whether or not an image indicated by image data belongs to a specific category in order for each of the plurality of categories. The identification process that has not yet been performed is omitted, and in the image identification method for identifying the category of the image based on the result of the identification process, The selection order of identification processing to be performed is determined.
Other features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

At least the following matters will become clear from the description of the present specification and the accompanying drawings.
Identification processing for identifying whether the image indicated by the image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
In the image identification method for identifying the category of the image based on the result of the identification process,
An image identification method is characterized in that the order of selection of identification processes to be performed later is determined according to the result of the identification process performed first.
According to such an image identification method, the processing time can be shortened.

  In addition, when it is identified that the image belongs to a specific category corresponding to the identification process in the identification process, it is preferable that the identification process that has not been performed is omitted. In addition, when it is identified that the image does not belong to a specific category corresponding to an identification process different from the identification process in the identification process, it is preferable that the another identification process is omitted. As a result, the processing time can be increased.

  In the identification process, a value corresponding to the probability that the image belongs to the specific category is calculated, and based on the value, it is identified whether the image belongs to the specific category. It is preferable that the selection order of the identification process performed later is determined according to the value calculated in the identification process. This simplifies determination of the selection order of identification processing performed later.

  In addition, it is preferable that the selection order is determined using data in which the result of the identification processing performed first and the identification processing performed next are associated with each other. By using such data, it is possible to increase the processing speed.

  In addition, for a plurality of candidates having different selection orders, until the category of the image is identified based on the processing time of the identification process performed later and the occurrence probability of the result of the identification process performed earlier. It is preferable that an expected value of processing time is calculated, and the selection order is determined by selecting a candidate with a short expected value from the plurality of candidates. As a result, the processing time can be increased.

  In addition, there is a determination process for determining a candidate with a short expected value when performing n-1 identification processes, and the determination process is performed first when determining a candidate with a short expected value when performing n identification processes. The expectation value is calculated using the determination process for each of n candidates having different identification processes, and the selection order is determined by selecting a candidate with a short expectation value from the n candidates. It is desirable. Thereby, the optimal selection order can be determined by a simple procedure.

An image identification device comprising a controller,
The controller is
Identification processing for identifying whether the image indicated by the image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
Based on the result of the identification process, the category of the image is identified,
An image identification apparatus characterized by determining the selection order of the identification process performed later according to the value calculated in the identification process performed earlier.

In the image identification device,
Identification processing for identifying whether an image indicated by image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
In a program for identifying the category of the image based on the result of the identification process,
A program that causes the image identification apparatus to determine the selection order of identification processing to be performed later according to the result of identification processing to be performed first becomes clear.

=== Overall structure ===
FIG. 1 is an explanatory diagram of an image processing system. This image processing system includes a digital still camera 2 and a printer 4.

The digital still camera 2 is a camera that acquires a digital image by forming an image of a subject on a digital device (CCD or the like). The digital still camera 2 is provided with a mode setting dial 2A. The user can set the shooting mode according to the shooting conditions by using the dial 2A. For example, when the “night scene” mode is set by the dial 2A, the digital still camera 2 performs shooting under shooting conditions suitable for night scene shooting by decreasing the shutter speed or increasing the ISO sensitivity.
The digital still camera 2 stores an image file generated by photographing in the memory card 6 in accordance with the file format standard. In the image file, not only digital data (image data) of a captured image but also additional data such as a shooting condition (shooting data) at the time of shooting is stored.

The printer 4 is a printing device that prints an image indicated by image data on paper. The printer 4 is provided with a slot 21 into which the memory card 6 is inserted. The user can take a picture with the digital still camera 2, remove the memory card 6 from the digital still camera 2, and insert the memory card 6 into the slot 21.
FIG. 2 is an explanatory diagram of the configuration of the printer 4. The printer 4 includes a printing mechanism 10 and a printer-side controller 20 that controls the printing mechanism 10. The printing mechanism 10 includes a head 11 that ejects ink, a head control unit 12 that controls the head 11, a motor 13 for conveying paper, and a sensor 14. The printer-side controller 20 includes a memory slot 21 for transmitting and receiving data from the memory card 6, a CPU 22, a memory 23, a control unit 24 for controlling the motor 13, and a drive signal for generating a drive signal (drive waveform). And a generation unit 25.

When the memory card 6 is inserted into the slot 21, the printer-side controller 20 reads out the image file stored in the memory card 6 and stores it in the memory 23. Then, the printer-side controller 20 converts the image data of the image file into print data for printing by the printing mechanism 10, controls the printing mechanism 10 based on the printing data, and prints an image on paper. This series of operations is called “direct printing”.
“Direct printing” is not only performed by inserting the memory card 6 into the slot 21, but also by connecting the digital still camera 2 and the printer 4 with a cable (not shown).

  The image file stored in the memory card 6 is composed of image data and additional data. The image data is composed of a plurality of pixel data. The pixel data is data indicating pixel color information (gradation value). An image is formed by arranging the pixels in a matrix. Therefore, the image data is data indicating an image. The additional data includes data indicating the characteristics of image data, shooting data, thumbnail image data, and the like.

=== Outline of automatic correction function ===
When printing a “person” photo, there is a demand to clean the skin tone. In addition, when printing a “landscape” photograph, there is a demand for emphasizing the blue of the sky and the green of trees and grass. Therefore, the printer 4 of the present embodiment includes an automatic correction function that analyzes an image file and automatically performs a suitable correction process.

FIG. 3 is an explanatory diagram of the automatic correction function of the printer 4. Each element of the printer-side controller 20 in the figure is realized by software and hardware.
The storage unit 31 is realized by a partial area of the memory 23 and the CPU 22. All or part of the image file read from the memory card 6 is developed in the image storage unit 31 </ b> A of the storage unit 31. In addition, the calculation result of each element of the printer-side controller 20 is stored in the result storage unit 31B of the storage unit 31.

  The face identification unit 32 is realized by the CPU 22 and a face identification program stored in the memory 23. The face identification unit 32 analyzes the image data stored in the image storage unit 31A and identifies the presence or absence of a face. When the face identifying unit 32 identifies that there is a face, the image to be identified is identified as belonging to the “person” scene. In this case, the scene identification process by the scene identification unit 33 is not performed. Since the face identification process by the face identification unit 32 is the same as a process that has already been widely performed, detailed description thereof is omitted.

  The scene identification unit 33 is realized by the CPU 22 and a scene identification program stored in the memory 23. The scene identification unit 33 analyzes the image file stored in the image storage unit 31A and identifies the scene of the image indicated by the image data. When the face identifying unit 32 identifies that there is no face, a scene identifying process by the scene identifying unit 33 is performed. As will be described later, the scene identifying unit 33 identifies whether the image to be identified is a “landscape”, “evening scene”, “night scene”, “flower”, “autumn leaves”, or “other” image. .

FIG. 4 is an explanatory diagram of a relationship between an image scene and correction contents.
The image correction unit 34 is realized by the CPU 22 and an image correction program stored in the memory 23. The image correction unit 34 converts the image data of the image storage unit 31A based on the identification results (identification results of the face identification unit 32 and the scene identification unit 33) stored in the result storage unit 31B (described later) of the storage unit 31. to correct. For example, when the identification result of the scene identification unit 33 is “landscape”, correction is performed so that blue is emphasized and green is emphasized. The image correction unit 34 may correct the image data by reflecting not only the scene identification result but also the contents of the image data of the image file. For example, when the exposure correction is negative, the image data may be corrected so as not to brighten the dark atmosphere image.

  The printer control unit 35 is realized by a printer control program stored in the CPU 22, the drive signal generation unit 25, the control unit 24, and the memory 23. The printer control unit 35 converts the corrected image data into print data, and causes the printing mechanism 10 to print the image.

=== Scene Identification Processing ===
FIG. 5 is a flowchart of the scene identification process performed by the scene identification unit 33. FIG. 6 is an explanatory diagram of the function of the scene identification unit 33. Each element of the scene identification unit 33 in the figure is realized by software and hardware. The scene identification unit 33 includes a feature amount acquisition unit 40, an overall classifier 50, a partial classifier 60, and an integrated classifier 70 shown in FIG.

First, the feature amount acquisition unit 40 analyzes the image data developed in the image storage unit 31A of the storage unit 31 and acquires partial feature amounts (S101). Specifically, the feature amount acquisition unit 40 divides the image data into 8 × 8 64 blocks, calculates the color average and variance of each block, and acquires the color average and variance as partial feature amounts. Here, each pixel has gradation value data in the YCC color space, and the average value of Y, the average value of Cb, and the average value of Cr are calculated for each block, and the variance of Y and the variance of Cb are calculated. And the variance of Cr are calculated respectively. That is, three color averages and three variances are calculated as partial feature amounts for each block. These color averages and variances indicate the characteristics of the partial images in each block. Note that an average value or variance in the RGB color space may be calculated.
Since the color average and variance are calculated for each block, the feature amount acquisition unit 40 expands the image data for the blocks in the block order without expanding all the image data in the image storage unit 31A. For this reason, the image storage unit 31A does not necessarily have a capacity sufficient to expand all of the image files.

  Next, the feature amount acquisition unit 40 acquires the entire feature amount (S102). Specifically, the feature quantity acquisition unit 40 acquires the overall color average, variance, center of gravity, and shooting information of the image data as the overall feature quantity. Note that these color averages and variances indicate the overall characteristics of the image. The color average, variance, and center of gravity of the entire image data are calculated using the partial feature values calculated previously. For this reason, it is not necessary to re-expand the image data when calculating the entire feature amount, and the calculation speed of the entire feature amount is increased. Although the overall identification process (described later) is performed prior to the partial identification process (described later), the overall feature value is obtained after the partial feature value in order to increase the calculation speed. The shooting information is extracted from the shooting data of the image file. Specifically, information such as the aperture value, shutter speed, and the presence or absence of flash emission is used as the overall feature amount. However, not all shooting data of the image file is used as the entire feature amount.

  Next, the overall classifier 50 performs overall identification processing (S103). The overall identification process is a process for identifying (estimating) an image scene indicated by image data based on the overall feature amount. Details of the overall identification process will be described later.

If the scene can be identified by the overall identification process (YES in S104), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.
If the scene cannot be identified by the overall identification process (NO in S104), the partial classifier 60 performs the partial identification process (S105). The partial identification process is a process for identifying the scene of the entire image indicated by the image data based on the partial feature amount. Details of the partial identification processing will be described later.

When the scene can be identified by the partial identification process (YES in S106), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.
If the scene cannot be identified by the partial identification process (NO in S106), the integrated discriminator 70 performs the integrated identification process (S107). Details of the integrated identification process will be described later.

  When the scene can be identified by the integrated identification process (YES in S108), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. On the other hand, if the scene cannot be identified by the integrated identification process (NO in S108), the image indicated by the image data is “other” (other than “landscape”, “evening scene”, “night scene”, “flower” or “autumn leaves”. Is stored in the result storage unit 31B (S110).

=== Overall identification processing ===
FIG. 7 is a flowchart of the overall identification process. Here, the overall identification process will be described with reference to FIG.

  First, the overall classifier 50 selects one sub-classifier 51 from the plurality of sub-classifiers 51 (S201). The overall classifier 50 is provided with five sub-classifiers 51 for identifying whether an image to be identified (identification target image) belongs to a specific scene. The five sub classifiers 51 identify scenes of scenery, evening scene, night scene, flowers, and autumn leaves, respectively. Here, the overall classifier 50 selects the sub classifier 51 in the order of landscape → evening scene → night scene → flower → autumn leaves (the selection order of the sub classifier 51 will be described later). For this reason, first, the sub classifier 51 (landscape classifier 51L) for identifying whether or not the classification target image belongs to a landscape scene is selected.

Next, the overall classifier 50 refers to the classification target table and determines whether or not a scene should be identified using the selected sub-classifier 51 (S202).
FIG. 8 is an explanatory diagram of the identification target table. This identification target table is stored in the result storage unit 31B of the storage unit 31. In the identification target table, all fields are set to zero in the first stage. In the process of S202, the “No” column is referred to, and if it is zero, it is determined as YES, and if it is 1, it is determined as NO. Here, the overall classifier 50 refers to the “No” column in the “Scenery” column in the identification target table, and determines “YES” because it is zero.

  Next, the sub classifier 51 calculates the probability (confidence) that the classification target image belongs to a specific scene based on the entire feature amount (S203). For the sub classifier 51 of this embodiment, a classification method using a support vector machine (SVM) is used. The support vector machine will be described later. When the classification target image belongs to a specific scene, the discriminant of the sub classifier 51 tends to be a positive value. When the classification target image does not belong to a specific scene, the discriminant of the sub classifier 51 tends to be a negative value. Further, the discriminant becomes a larger value as the certainty that the identification target image belongs to the specific scene is higher. For this reason, if the discriminant value is large, the probability (confidence) that the identification target image belongs to a specific scene is high, and if the discriminant value is small, the probability that the classification target image belongs to a specific scene is low. .

  Next, the sub discriminator 51 determines whether or not the value of the discriminant is larger than the positive threshold (S204). If the value of the discriminant is larger than the positive threshold, the sub discriminator 51 determines that the classification target image belongs to a specific scene.

  FIG. 9 is an explanatory diagram of an affirmative threshold value of the overall identification process. In the figure, the horizontal axis indicates an affirmative threshold, and the vertical axis indicates the probability of recall or precision. FIG. 10 is an explanatory diagram of Recall and Precision. If the discriminant value is greater than or equal to the positive threshold, the identification result is Positive. If the discriminant value is not greater than or equal to the positive threshold, the identification result is Negative.

  Recall indicates the recall rate and detection rate. Recall is the ratio of the number of images identified as belonging to a specific scene to the total number of images of the specific scene. In other words, Recall is the probability that the sub-identifier 51 identifies the image as a positive when the image of the specific scene is identified by the sub-identifier 51 (the probability that the image of the specific scene belongs to the specific scene. ). For example, when a landscape image is identified by the landscape classifier 51L, it indicates the probability that the landscape classifier 51L identifies it as belonging to a landscape scene.

  Precision indicates the correct answer rate and the correct answer rate. Precision is the ratio of the number of images in a particular scene to the total number of images identified as Positive. In other words, Precision indicates the probability that the image to be identified is a specific scene when the sub-classifier 51 that identifies the specific scene identifies it as Positive. For example, when the landscape classifier 51L identifies that it belongs to a landscape scene, it indicates the probability that the identified image is really a landscape image.

As can be seen from FIG. 9, the greater the positive threshold, the greater the Precision. For this reason, the larger the positive threshold value, the higher the probability that an image identified as belonging to a landscape scene, for example, is a landscape image. That is, the greater the positive threshold, the lower the probability of misidentification.
On the other hand, the larger the positive threshold, the smaller the Recall. As a result, for example, even when a landscape image is identified by the landscape classifier 51L, it is difficult to correctly identify it as belonging to a landscape scene. By the way, if the image to be identified can be identified as belonging to a landscape scene (YES in S204), the speed of the overall identification process is increased so as not to identify other remaining scenes (such as sunsets). For this reason, the larger the positive threshold, the lower the overall identification processing speed. Further, if the scene can be identified by the overall identification process, the partial identification process is not performed and the speed of the scene identification process is increased (S104). Therefore, as the positive threshold is increased, the scene identification process speed decreases. Become.
That is, if the positive threshold is too small, the probability of misidentification increases, and if it is too large, the processing speed decreases. In this embodiment, since the correct answer rate (Precision) is set to 97.5%, the landscape affirmation threshold is set to 1.27.

  If the discriminant value is greater than the affirmative threshold value (YES in S204), the sub-classifier 51 determines that the classification target image belongs to a specific scene and sets an affirmative flag (S205). “Set an affirmative flag” means that the “affirmation” column in FIG. In this case, the overall discriminator 50 ends the overall discrimination process without performing discrimination by the next sub discriminator 51. For example, if the image can be identified as a landscape image, the entire identification process is terminated without identifying the sunset scene or the like. In this case, since the identification by the next sub-identifier 51 is omitted, the speed of the overall identification process can be increased.

  If the value of the discriminant is not greater than the positive threshold (NO in S204), the sub discriminator 51 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S206.

  Next, the sub discriminator 51 compares the discriminant value with a negative threshold value (S206). Thereby, the sub classifier 51 determines whether the classification target image does not belong to a predetermined scene. There are two types of such determinations. First, if the value of the discriminant of the sub-identifier 51 of a specific scene is smaller than the first negative threshold, it is determined that the classification target image does not belong to the specific scene. For example, if the discriminant value of the landscape classifier 51L is smaller than the first negative threshold, it is determined that the classification target image does not belong to a landscape scene. Second, if the value of the discriminant of the sub-identifier 51 of a specific scene is larger than the second negative threshold, it is determined that the classification target image does not belong to a scene different from the specific scene. . For example, if the discriminant value of the landscape classifier 51L is larger than the second negative threshold, it is determined that the classification target image does not belong to the night scene.

  FIG. 11 is an explanatory diagram of the first negative threshold. In the figure, the horizontal axis indicates the first negative threshold, and the vertical axis indicates the probability. The bold line in the graph is a True Negative Recall graph, and indicates the probability of correctly identifying an image other than a landscape image as not a landscape image. The thin line in the graph is a False Negative Recall graph, which indicates the probability of erroneously identifying a landscape image that is not a landscape image.

As can be seen from FIG. 11, the smaller the first negative threshold is, the smaller False Negative Recall is. For this reason, the smaller the first negative threshold, the lower the probability that an image identified as not belonging to a landscape scene is a landscape image, for example. That is, the probability of misidentification is reduced.
On the other hand, the True Negative Recall decreases as the first negative threshold decreases. As a result, it is difficult to identify an image other than a landscape image unless it is a landscape image. On the other hand, if it is possible to identify that the identification target image is not a specific scene, the process by the sub partial classifier 61 for the specific scene is omitted during the partial identification process to increase the scene identification processing speed (described later in FIG. 14). S302). For this reason, the scene identification processing speed decreases as the first negative threshold is decreased.
That is, if the first negative threshold is too large, the probability of misidentification increases, and if it is too small, the processing speed decreases. In the present embodiment, in order to set False Negative Recall to 2.5%, the first negative threshold is set to -1.10.

  By the way, if the probability that an image belongs to a landscape scene is high, the probability that the image belongs to a night scene is inevitably low. For this reason, when the discriminant value of the landscape discriminator 51L is large, it may be identified that the scene is not a night scene. In order to perform such identification, a second negative threshold is provided.

  FIG. 12 is an explanatory diagram of the second negative threshold. In the figure, the horizontal axis indicates the value of the landscape discriminant, and the vertical axis indicates the probability. In the same figure, the Recall graph of the night view is drawn with a dotted line together with the Recall and Precision graph of FIG. If attention is paid to this dotted line graph, if the value of the discriminant of the landscape is larger than −0.45, the probability that the image is a night scene image is 2.5%. In other words, if the landscape discriminant value is greater than −0.45, even if the image is identified as not a night scene image, the probability of misidentification is only 2.5%. Therefore, in the present embodiment, the second negative threshold is set to −0.45.

  When the discriminant value is smaller than the first negative threshold value, or when the discriminant value is larger than the second negative threshold value (YES in S206), the sub-classifier 51 determines that the classification target image belongs to a predetermined scene. It is determined not to do so, and a negative flag is set (S207). “Set a negative flag” means to set the “No” column in FIG. For example, when it is determined that the image to be identified does not belong to a landscape scene based on the first negative threshold, the “denial” column in the “landscape” column is 1. Further, when it is determined that the identification target image does not belong to the night scene based on the second negative threshold, the “Negation” field in the “Night scene” field is “1”.

  FIG. 13A is an explanatory diagram of a threshold table. The threshold value table may be stored in the storage unit 31 or may be incorporated in a part of a program for executing the overall identification process. The threshold table stores data related to the affirmative threshold and the negative threshold described above.

  FIG. 13B is an explanatory diagram of threshold values in the landscape classifier 51L described above. An affirmative threshold value and a negative threshold value are preset in the landscape discriminator 51L. 1.27 is set as the positive threshold. The negative threshold includes a first negative threshold and a second negative threshold. -1.10 is set as the first negative threshold. In addition, a value is set for each scene other than the landscape as the second negative threshold.

  FIG. 13C is an explanatory diagram outlining the processing of the landscape classifier 51L described above. Here, for simplification of description, only the night view will be described for the second negative threshold. If the discriminant value is greater than 1.27 (YES in S204), the landscape classifier 51L determines that the classification target image belongs to a landscape scene. If the discriminant value is 1.27 or less (NO in S204) and is greater than −0.45 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to the night scene. To do. If the value of the discriminant is smaller than −1.10 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to a landscape scene. Note that the landscape discriminator 51L also determines whether the image to be identified does not belong to the scene based on the second negative threshold for sunset scenes, flowers, and autumn leaves. However, since these second negative threshold values are larger than the positive threshold values, the landscape discriminator 51L does not determine that the classification target image does not belong to these scenes.

  In the case of NO in S202, in the case of NO in S206, or when the processing in S207 is completed, the overall discriminator 50 determines the presence or absence of the next sub discriminator 51 (S208). Here, since the process by the landscape classifier 51L is finished, the overall classifier 50 determines in S208 that there is a next sub-classifier 51 (evening scene classifier 51S).

  Then, when the process of S205 is finished (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S208 that there is no next sub-classifier 51 (the identification target image is a specific image). When it cannot be determined that the scene belongs to the scene), the overall discriminator 50 ends the overall discrimination process.

  As already described, when the overall identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the overall identification process (S104 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.

  If the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  By the way, although not described above, the overall discriminator 50, when the sub discriminator 51 calculates the discriminant value, the Precision corresponding to the discriminant value is stored in the result storage unit 31B as information on the certainty factor. Remember. Of course, the discriminant value itself may be stored as information on the certainty factor.

=== Partial identification processing ===
FIG. 14 is a flowchart of the partial identification process. The partial identification process is performed when the scene cannot be identified by the overall identification process (NO in S104 of FIG. 5). As will be described below, the partial identification process is a process for identifying the scene of the entire image by identifying each scene of the divided partial images. Here, the partial identification process will be described with reference to FIG.

  First, the partial classifier 60 selects one sub partial classifier 61 from the plurality of sub partial classifiers 61 (S301). The partial discriminator 60 is provided with three sub partial discriminators 61. Each sub partial discriminator 61 discriminates whether or not each partial image divided into 8 × 8 64 blocks belongs to a specific scene. Here, the three sub partial classifiers 61 identify the scenes of sunset, flowers, and autumn leaves, respectively. Here, the partial discriminator 60 selects the sub partial discriminator 61 in the order of sunset scene → flower → autumn leaves (the selection order of the sub partial discriminator 61 will be described later). Therefore, first, the sub partial classifier 61 (evening scene partial classifier 61S) for identifying whether or not the partial image belongs to the sunset scene is selected.

  Next, the partial discriminator 60 refers to the discrimination target table (FIG. 8) and determines whether or not the scene should be discriminated using the selected sub partial discriminator 61 (S302). Here, the partial discriminator 60 refers to the “No” column of the “Evening Scene” column in the classification target table, and determines YES if it is zero, and NO if it is 1. When the evening scene classifier 51S sets a negative flag with the first negative threshold during the overall identification process or when another sub-classifier 51 sets a negative flag with the second negative threshold, in S302 It is judged as NO. If it is determined NO, the sunset partial identification process is omitted, and the partial identification process speed increases. However, for the convenience of explanation, it is assumed that YES is determined here.

Next, the sub partial discriminator 61 selects one partial image from the partial images divided into 8 × 8 64 blocks (S303).
FIG. 15 is an explanatory diagram of the order of partial images selected by the evening scene partial classifier 61S. When a scene of the entire image is identified from the partial image, it is desirable that the partial image used for identification is a portion where the subject exists. Therefore, in this embodiment, thousands of samples of sunset scene images are prepared, each sunset scene image is divided into 64 blocks of 8 × 8, and blocks including sunset scene partial images (sun and sky partial images of the sunset scene) are included. The presence probability of the sunset partial image in each block was calculated based on the extracted block position. And in this embodiment, a partial image is selected in an order from a block with a high existence probability. Note that the selection order information shown in the figure is stored in the memory 23 as part of the program.

  In the case of an evening scene image, since the sky of the evening scene often spreads from the vicinity of the center to the upper half, the existence probability increases in the upper half block from the vicinity of the center. In the case of an evening scene image, the lower 1/3 of the image is shaded by backlight, and the partial image alone often cannot be distinguished from the evening scene or the night scene, so the existence probability is lower in the lower 1/3 block. In the case of a flower image, since the composition is often such that a flower is arranged near the center, the probability of existence of a flower partial image near the center increases.

  Next, the sub partial classifier 61 determines whether or not the partial image belongs to a specific scene based on the partial feature amount of the selected partial image (S304). Similar to the sub classifier 51 of the overall classifier 50, the sub partial classifier 61 uses a discrimination method using a support vector machine (SVM). The support vector machine will be described later. If the discriminant value is a positive value, it is determined that the partial image belongs to a specific scene, and the sub partial classifier 61 increments the positive count value. If the discriminant value is a negative value, it is determined that the partial image does not belong to a specific scene, and the sub partial discriminator 61 increments the negative count value.

  Next, the sub partial discriminator 61 determines whether or not the positive count value is larger than the positive threshold value (S305). The positive count value indicates the number of partial images determined to belong to a specific scene. If the positive count value is larger than the affirmative threshold (YES in S305), the sub partial classifier 61 determines that the classification target image belongs to a specific scene, and sets an affirmative flag (S306). In this case, the partial discriminator 60 ends the partial discriminating process without performing discrimination by the next sub partial discriminator 61. For example, if the image can be identified as an evening scene image, the partial identification process is terminated without identifying flowers and autumn leaves. In this case, since the identification by the next sub partial classifier 61 is omitted, the speed of the partial classification process can be increased.

  If the positive count value is not greater than the positive threshold value (NO in S305), the sub partial classifier 61 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S307.

  If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold (YES in S307), the sub partial discriminator 61 proceeds to the process of S309. If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold value, the positive count value does not become larger than the positive threshold value even if the positive count value is incremented by all the remaining partial images. By proceeding with the process, the remaining partial images are not identified by the support vector machine. Thereby, the speed of the partial identification process can be increased.

If the sub partial discriminator 61 determines NO in S307, the sub partial discriminator 61 determines whether there is a next partial image (S308). In the present embodiment, not all of the partial images divided into 64 are selected in order. In FIG. 15, only the top 10 partial images indicated by thick frames are selected in order. Therefore, when the identification of the tenth partial image is completed, the sub partial classifier 61 determines in S308 that there is no next partial image. (In consideration of this point, the “number of remaining partial images” in S307 is also determined.)
FIG. 16 is a Recall and Precision graph when an evening scene image is identified using only the top 10 partial images. If an affirmative threshold as shown in the figure is set, the accuracy rate (Precision) can be set to about 80%, the recall rate (Recall) can be set to about 90%, and identification with high accuracy is possible.

In this embodiment, the evening scene image is identified using only 10 partial images. For this reason, in the present embodiment, it is possible to increase the speed of the partial identification process compared to the case where the evening scene image is identified using all 64 partial images.
In this embodiment, the sunset scene image is identified using the top tenth partial image having a high existence probability of the sunset scene partial image. For this reason, in the present embodiment, it is possible to set both Recall and Precision higher than the identification of an evening scene image using 10 partial images extracted by ignoring the existence probability.
In this embodiment, the partial images are selected in descending order of the existence probability of the sunset partial image. As a result, the determination in S305 is likely to be YES at an early stage. For this reason, in the present embodiment, the speed of the partial identification process can be increased as compared with the case where the partial images are selected in the order in which the presence probability level is ignored.

  When it is determined YES in S307, or when it is determined that there is no next partial image in S308, the sub partial discriminator 61 determines whether or not the negative count value is larger than the negative threshold (S309). . Since this negative threshold performs substantially the same function as the negative threshold (S206 in FIG. 7) in the above-described overall identification process, detailed description thereof is omitted. When YES is determined in S309, a negative flag is set as in S207 of FIG.

  In the case of NO in S302, in the case of NO in S309, or when the process of S310 is completed, the partial discriminator 60 determines whether or not there is a next sub partial discriminator 61 (S311). In the case after the processing by the evening scene partial classifier 61S is finished, since the flower partial classifier 61F and the autumnal leaves partial classifier 61R are still present as the sub partial classifier 61, the partial classifier 60 determines the next sub partial classifier in S311. It is determined that there is a container 61.

  Then, when the process of S306 is completed (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S311 that there is no next sub partial classifier 61 (the identification target image is specified). If it cannot be determined that the scene belongs to the scene), the partial discriminator 60 ends the partial discrimination processing.

As already described, when the partial identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the partial identification process (S106 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.
When the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

  In the above description, the evening scene partial classifier 61S identifies evening scene images using ten partial images. However, the number of partial images used for identification is not limited to ten. Further, the other sub partial classifier 61 may identify an image using a different number of partial images from the sunset scene partial classifier 61S. In the present embodiment, it is assumed that the flower partial discriminator 61F identifies a flower image using 20 partial images, and the autumnal foliage partial discriminator 61R identifies a autumnal foliage image using 15 partial images.

=== Support vector machine ===
Before describing the integrated identification process, the support vector machine (SVM) used in the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process will be described.

FIG. 17A is an explanatory diagram of determination by the linear support vector machine. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. The learning sample is divided into two classes A and B. In the figure, samples belonging to class A are indicated by circles, and samples belonging to class B are indicated by squares.
A boundary that divides the two-dimensional space into two is defined by learning using the learning sample. The boundary is defined by <w · x> + b = 0 (where x = (x1, x2), w is a weight vector, and <w · x> is an inner product of w and x). However, the boundary is defined by learning using a learning sample so that the margin is maximized. That is, in the case of the figure, the boundary is not a thick dotted line but a thick solid line.
The discrimination is performed using the discriminant f (x) = <w · x> + b. It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined.

  Here, the description is made using a two-dimensional space, but the present invention is not limited to this (that is, the feature amount may be two or more). In this case, the boundary is defined by a hyperplane.

  By the way, there are cases where the two classes cannot be separated by a linear function. In such a case, if the determination is performed by the linear support vector machine, the accuracy of the determination result is lowered. Therefore, if the feature quantity of the input space is nonlinearly transformed, that is, if the input space is nonlinearly mapped to a certain feature space, it can be separated by a linear function in the feature space. This is used in the nonlinear support vector machine.

  FIG. 17B is an explanatory diagram of discrimination using a kernel function. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. If the nonlinear mapping from the input space of FIG. 17B becomes a feature space as shown in FIG. 17A, it can be separated into two classes by a linear function. If the boundary is defined so that the margin is maximized in this feature space, the inverse mapping of the boundary in the feature space becomes the boundary shown in FIG. 17B. As a result, the boundary becomes nonlinear as shown in FIG. 17B.

In this embodiment, by using a Gaussian kernel, the discriminant f (x) becomes as follows (where M is the number of features and N is the number of learning samples (or contributes to the boundary): The number of learning samples), w _i is a weighting factor, y _j is the feature quantity of the learning sample, and x _j is the feature quantity of the input x).

  It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined. Further, the larger the value of the discriminant f (x), the higher the probability that the input x (this input x is different from the learning sample) belongs to the class A. On the contrary, the smaller the value of the discriminant f (x), the lower the probability that the input x (this input x is different from the learning sample) belongs to the class A.

  In the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process, the value of the discriminant f (x) of the support vector machine is used. The calculation of the value of the discriminant f (x) by the support vector machine takes time when the number of learning samples (in this embodiment, tens of thousands) increases. For this reason, the sub partial classifier 61 that needs to calculate the value of the discriminant f (x) a plurality of times requires more processing time than the sub classifier 51 that only needs to calculate the value of the discriminant f (x) once. It takes.

  An evaluation sample is prepared separately from the learning sample. The above Recall and Precision graphs are based on the identification results for the evaluation samples.

=== Integrated identification processing ===
In the above-described overall identification process and partial identification process, the positive threshold value in the sub-classifier 51 and the sub-classifier 61 is set relatively high, and the Precision (correct answer rate) is set high. This is because, for example, if the accuracy rate of the landscape classifier 51L of the overall classifier is set low, the scene classifier 51L erroneously identifies the autumnal image as a landscape image, and before the autumnal classifier 51R performs the classification. This is because a situation occurs in which the entire identification process ends. In the present embodiment, by setting the Precision (accuracy rate) high, an image belonging to a specific scene is identified by the sub-classifier 51 (or sub-partial classifier 61) of the specific scene ( For example, the autumnal leaves image is identified by the autumnal leaves discriminator 51R (or the autumnal leaf partial discriminator 61R).

  However, if the Precision (accuracy rate) of the overall identification process or the partial identification process is set to be high, there is a high possibility that the scene cannot be identified by the overall identification process or the partial identification process. Therefore, in this embodiment, when the scene cannot be identified by the overall identification process and the partial identification process, the integrated identification process described below is performed.

  FIG. 18 is a flowchart of the integrated identification process. As will be described below, the integrated identification process is a process of selecting a scene with the highest certainty factor based on the discriminant value of each sub-classifier 51 in the overall identification process.

  First, the integrated discriminator 70 extracts a positive scene based on the discriminant values of the five sub discriminators 51 (S401). At this time, the value of the discriminant calculated by each sub classifier 51 during the overall identification process is used.

Next, the integrated discriminator 70 determines whether or not a scene having a positive discriminant value exists (S402).
If there is a scene with a positive discriminant value (YES in S402), an affirmative flag is set in the maximum value scene column (S403), and the integrated identification process is terminated. Accordingly, it is determined that the identification target image belongs to the maximum value scene.
On the other hand, if there is no scene having a positive discriminant value (NO in S402), the integrated identification process is terminated without setting an affirmative flag. As a result, there is no scene in the affirmative column of the identification target table in FIG. That is, it cannot be identified to which scene the identification target image belongs.

  As already described, when the integrated identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the integrated identification process (S108 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column. If it is determined NO in S402, the determination in S108 is also NO.

=== Reference Example ===
<Overview>
As already described, the overall discriminator 50 has five sub discriminators 51, and each sub discriminator 51 is sequentially selected during the overall discriminating process. Here, it is desirable that the selection order of the sub-identifiers 51 is such that the expected value of the processing time of the overall identification process is the shortest.

  Therefore, in this reference example, the selection order of the five sub classifiers 51 is determined so that the expected value of the processing time of the overall identification process is shortened.

  FIG. 19 is an explanatory diagram of a selection order determination flow. This decision flow is performed when designing the scene identification program.

  First, all the selection orders of the sub discriminators 51 are listed. Here, since there are five sub discriminators 51, 120 (= 5 factorial) orders are listed (S501).

  Next, an expected value of processing time in each selection order is calculated (S502). Here, the expected value of the processing time is calculated for each of the 120 selection orders. A method for calculating the expected value of the processing time will be described in detail later.

  Then, the shortest of 120 expected processing time values is extracted (S503). Then, the selection order when the expected processing time is the shortest is stored.

<Calculation method of expected value of processing time>
FIG. 20 is a flowchart for calculating an expected value of processing time in a certain selection order.
Here, the expected value of the processing time when the sub discriminator 51 is selected in the order of landscape → evening scene → night scene → flower → autumn leaves will be described. The n-th selected sub classifier 51 is referred to as an “n-th sub classifier”. For example, the landscape classifier 51L is a first sub classifier. Further, the probability (execution probability) that the n-th sub classifier 51 is executed is assumed to be Pn. For example, the execution probability of the evening scene classifier 51S is P2. The processing speed of the n-th sub classifier 51 (processing speed for identifying whether or not the classification target image is a specific scene) is Tn. For example, the processing speed of the night scene classifier 51N is T3.

  First, the execution probability Pn (n is 1 to 5) of each sub classifier 51 is calculated (S601). The execution probability P1 of the first sub classifier 51 (here, the scene classifier 51L) to be executed first is 1. However, when the value of the discriminant of the first sub classifier 51 is larger than the positive threshold or when the value of the discriminant of the first sub classifier is larger than the second negative threshold, the processing after the second sub classifier is omitted. Therefore, the execution probabilities P2 to P5 of the second to fifth sub discriminators 51 are smaller than 1. A method of calculating the execution probability of each sub classifier 51 will be described in detail later.

  Next, an expected value of processing time of each sub classifier 51 is calculated (S602). The expected value of the processing time of each sub classifier 51 is a value obtained by multiplying the execution probability Pn of the sub classifier 51 by the processing speed Tn. For example, the expected value of the processing speed of the flower discriminator 51F is calculated as T4 × P4.

  FIG. 21 is a table of processing times of the sub classifier 51. As shown in the figure, the processing time of each sub classifier is different. This is because the processing time varies depending on the number of samples in the calculation of the value of the discriminant f (x) by the support vector machine. Note that the processing time of each sub classifier is obtained in advance before the processing of S602.

  Next, an expected value Tt of the processing time of the overall identification process is calculated (S603). The expected value Tt of the processing time of the overall identification process is the sum of the expected values of the processing time of each sub classifier. That is, the expected value Tt of the overall identification processing speed is calculated as Tt = (T1 × P1) + (T2 × P2) + (T3 × P3) + (T4 × P4) + (T5 × P5).

<Calculation method of P (i, j)>
Before describing the method of calculating the execution probability Pn of each sub-classifier 51, first, after the i-th sub-classifier is executed, the probability P (i, j executed without the j-th sub-classifier being omitted). ).

  As the case where the j-th sub classifier is omitted, it can be identified by the i-th sub classifier that the classification target image belongs to a specific scene (the discriminant value of the i-th sub classifier is greater than the positive threshold). In some cases, or when the negative flag of the scene of the j-th sub-classifier is set by the i-th sub-classifier (when the discriminant value of the i-th sub-classifier is larger than the second negative threshold). For this reason, P (i, j) can be obtained from the value of the discriminant and the relationship between the positive threshold and the second negative threshold.

  Specifically, the probability that the identification target image can be identified by the i-th sub-classifier belongs to a specific scene is Pposi, and the probability that the negative flag of the scene of the j-th sub-classifier is set by the i-th sub-classifier is Pneg ( i, j), P (i, j) is given by

P (i, j) = 1-max (Pposi, Pneg (i, j))
In order to explain more specifically, the probability P (1,2) that the evening scene classifier 51S is executed after the scenery classifier 51L is executed will be described.

  FIG. 22A shows a probability distribution of discriminant values when a sample image for evaluation is identified by the landscape classifier 51L. In the figure, a positive threshold and a second negative threshold for each scene are shown by dotted lines or the like.

  FIG. 22B is an explanatory diagram of the probability Ppos1 with which the landscape classifier 51L can identify a landscape image. The probability Ppos1 by which the landscape discriminator 51 can identify a landscape image is a probability that the discriminant value is larger than the affirmative threshold, and is thus obtained as an integral value of the shaded area in the figure.

  FIG. 22C is an explanatory diagram of the probability Pneg (1, 2) that the landscape discriminator 51L sets the evening scene negative flag. The probability Pneg (1,2) that the landscape discriminator 51L sets the evening scene negation flag is a probability that the discriminant value of the landscape discriminator 51L is larger than the second negation threshold value of the evening scene. It is obtained as the integral value of the region. In this case, since the second negative threshold value of the evening scene is larger than the positive threshold value, the probability Pneg (1, 2) is smaller than the probability Ppos1.

  Therefore, the probability P (1,2) that the sunset scene classifier 51S is executed after the scenery classifier 51L is executed is obtained as P (1,2) = 1-Ppos1.

For reference, the probability P (1, 3) that the night scene classifier 51N is executed after the scene classifier 51L is executed will be described.
FIG. 22D is an explanatory diagram of the probability Pneg (1, 3) that the landscape classifier 51L sets the night scene negative flag. The probability Pneg (1, 3) that the landscape discriminator 51L sets the night scene negative flag is a probability that the discriminant value of the landscape discriminator 51L is larger than the second negative threshold of the night scene. It is obtained as the integral value of the region. In this case, since the second negative threshold of the night view is a value smaller than the positive threshold, the probability Pneg (1, 3) is a value larger than the probability Ppos1.
Therefore, the probability P (1,3) that the night scene classifier 51N is executed after the scene classifier 51L is executed is obtained as P (1,3) = 1-Pneg (1,3).

  FIG. 23 is a table of probabilities P (i, j) that the j-th sub classifier is executed when the i-th sub classifier is executed. Next, the execution probability Pn of each sub classifier 51 is calculated using the probability P (i, j) calculated in this way.

<Execution probability calculation method>
FIG. 24 is a flowchart for calculating the execution probability Pn of each sub classifier 51 in a certain selection order.

  First, the execution probability Pn of all the sub classifiers 51 is initialized to 1 (S701). Next, i is set to 1 (S702), and the execution probability P1 of the first sub classifier is calculated (S703). Since the first sub classifier is executed first, it is not omitted, so the execution probability P1 is 1.

  Next, it is determined whether or not the execution probabilities of all the sub classifiers 51 have been calculated (S704). Since only the execution probability P1 of the first sub classifier 51 has been calculated, NO is obtained in S704.

  Next, the values of the execution probabilities P2 to P5 of the second to fifth sub classifiers that have not been executed are updated (S705). Here, the execution probability Pn (n is 2 to 5) is updated to P (1, n) × P1.

  Thereafter, i is set to 2 (S706), and the execution probability P2 of the second sub classifier is calculated (S703). The execution probability P2 of the second sub classifier becomes the value P (1,2) × P1 updated in S705.

  Next, it is determined whether or not the execution probabilities of all the sub classifiers 51 have been calculated (S704). Since only the execution probability P2 of the second sub discriminator 51 has been calculated yet, NO is obtained in S704.

  Next, the values of the execution probabilities P3 to P5 of the unexecuted third to fifth sub classifiers are updated (S705). Here, it is necessary to consider separately when the second sub classifier is executed and when it is omitted. First, when the second sub-classifier is executed, the probability Pn that the n-th sub-classifier is executed is P2 × P (2, n) × Pn. On the other hand, when the second sub classifier is omitted, the probability Pn that the nth sub classifier is executed is (1−P2) × Pn. Accordingly, the probability Pn that the nth discriminator is executed is updated to (P2 × P (2, n) + 1−P2) × Pn.

  In this way, the execution probabilities Pn are similarly calculated for the sub-classifiers after the third sub-classifier (S703 to S706).

  When the execution probabilities Pn of all the sub classifiers are calculated (YES in S704), the process ends. Thereby, the execution probability Pn of each sub discriminator 51 in a certain selection order is calculated.

<After determining the selection order>
As described above, the selection order with the shortest expected processing time value is determined from the 120 selection orders. As a result, in this reference example, the selection order of landscape → evening scene → night scene → flower → autumn leaves is determined. Based on this determination, a scene identification program for realizing the overall classifier 50 is configured, and this program is installed in the printer-side controller 20 of the printer 4. As a result, it is possible to realize an overall identification process with a high processing speed.

=== First Embodiment ===
<Overview>
In the reference example described above, the selection order of the sub-identifiers 51 is fixed. For this reason, in the above-described reference example, the selection order with the shortest expected processing time is determined on the assumption that the selection order of the sub-identifiers 51 is fixed.

  On the other hand, in the first embodiment, the order of the sub classifiers to be executed thereafter is changed according to the value of the discriminant of the sub classifier to be executed first. For this reason, in the first embodiment, considering the change of the order of the sub classifiers, the selection order having the shortest expected processing time is determined.

<About the selection order of sub classifiers>
When the landscape discriminator 51L is selected first, there are cases where the night view is not excluded and cases where it is excluded depending on the value of the discriminant. In the former case, it is desirable that the selection order be such that the expected value of the processing time of the remaining four sub classifiers 51 including the night scene classifier 51N is the shortest. On the other hand, in the latter case, it is desirable that the selection order be such that the expected value of the processing time of the remaining three sub discriminators 51 excluding the night scene discriminator 51N is the shortest. As described above, the optimal order of the sub-classifiers executed thereafter may differ depending on the value of the discriminant of the sub-classifier executed first.

  FIG. 25 is a conceptual diagram of tree-structured data (tree data) referred to in order determination. This tree data indicates the selection order of the sub classifier 51, and is data referred to by the overall classifier 50 in S201 of FIG. The tree data is data in which the discriminant value of the sub classifier 51 is associated with the sub classifier 51 to be selected next. In other words, in the tree data, the sub classifier 51 to be selected next branches according to the discriminant value of the sub classifier 51. This tree data may be stored in the storage unit 31 or may be incorporated in a part of a program for executing the overall identification process.

Hereinafter, how the overall discriminator 50 selects the sub discriminator 51 using the tree data in S201 of FIG. 7 will be described.
First, in the first S201, the overall classifier 50 refers to the first tree data and selects the landscape classifier 51L. As already described, if the value of the discriminant of the landscape discriminator 51L is greater than 1.27 (positive threshold) (YES in S204), the landscape discriminator 51L may indicate that the classification target image belongs to a landscape scene. This can be determined, and the processing by the other sub classifier 51 is omitted. If the value of the discriminant of the landscape discriminator 51L is larger than −0.45 and smaller than 1.27 (positive threshold) (YES in S206), a night scene negative flag is set and processing by the night scene discriminator 51N is performed. Omitted (night view is excluded).

  By the way, when the night scene is excluded by the landscape classifier 51L, the remaining sub-classifiers 51 that will perform the classification process are the evening scene classifier 51S, the flower classifier 51F, and the autumnal leaves classifier 51R. Become. Regarding the selection order of the three sub classifiers 51, if the expected value of the processing time is shorter in the order of autumnal leaves → sunset → flowers than in the order of evening scenes → flowers → autumn leaves, the latter selection order is preferred. desirable. However, according to the above-described reference example, the autumnal leaves discriminator 51R is not selected before the evening scene discriminator 51S or the flower discriminator 51F. On the other hand, in the first embodiment, since the type of the sub-classifier to be selected next is determined according to the value of the discriminant, when the night scene is excluded by the landscape classifier 51L, the autumn color classifier 51R Will be selected.

  That is, in the next S201 after the processing by the landscape classifier 51L is completed, the overall classifier 50 refers to the second of the tree data, and the discriminant value of the landscape classifier 51L is −0.45 ( If it is smaller than the second negative threshold corresponding to the night view), the overall discriminator 50 selects the evening scene discriminator 51S. On the other hand, if the discriminant value of the landscape discriminator 51L is larger than −0.45 and smaller than 1.27 (positive threshold), the overall discriminator 50 selects the autumn color discriminator 51R. Thus, in the first embodiment, the type of the sub-classifier to be selected next changes according to the discriminant value of the landscape classifier 51L.

  Thereby, in 1st Embodiment, the expected value of the processing time after a night view is excluded by the landscape identification device 51 becomes short. As a result, the expected value of the processing time of the overall identification process is also shortened.

  Similarly, when only the flowers are excluded by the second evening scene discriminator 51S, the remaining sub discriminators 51 that will execute the discrimination process are the night scene discriminator 51N and the autumnal leaf discriminator 51R. Regarding the order of the two sub classifiers 51, if the expected value of the processing time is shorter in the order of autumn leaves → night view than in the order of night view → autumn leaves, the latter selection order is desirable. However, according to the above-described reference example, the autumnal leaves discriminator 51R is not selected before the night view discriminator 51N. On the other hand, in the first embodiment, since the type of the next sub-classifier to be selected is determined according to the value of the discriminant, if only the flower is excluded by the second evening scene classifier 51R, then the autumn leaves The discriminator 51R is selected.

  In other words, when the sunset scene classifier 51S is selected second, in the next S201 (in the next S201 after the processing by the sunset scene classifier 51S is finished), the overall classifier 50 Refers to the third of the data. At this time, if the value of the discriminant of the evening scene classifier 51S is smaller than −0.66 (second negative threshold corresponding to the flower), the overall classifier 50 selects the night scene classifier 51N. If the discriminant value of the evening scene classifier 51S is larger than −0.66 and smaller than −0.62, the overall classifier 50 selects the autumnal leaves classifier 51R. As described above, in the first embodiment, the type of the sub-classifier 51 to be selected next changes according to the value of the discriminant expression of the evening scene classifier 51L.

  Thereby, in 1st Embodiment, the expected value of the processing time after a flower is excluded by the 2nd evening scene identifier 51S becomes short. As a result, the expected value of the processing time of the overall identification process is also shortened.

  As described above, in the first embodiment, the type of the sub-classifier to be selected next is determined according to the discriminant value. In the first embodiment, the tree data is configured to have an optimal selection order as will be described later, so that the expected value of the processing time of the overall identification process can be shortened.

<How to create tree structure data>
The tree data in FIG. 25 is configured in an optimal selection order that minimizes the expected processing time. The tree data can be created by any procedure as long as it is configured to have an optimal selection order, but in this embodiment, the tree data is constructed by a recursive procedure as described below. To determine the optimal order.

  FIG. 26 is a flowchart of recursive optimal order determination. This flow is performed when designing the scene identification program.

  If the number of sub discriminators 51 of the overall discriminator 50 is 0, the expected value of the processing time of the overall discriminating process is zero. Further, if the number of the sub classifiers 51 of the overall classifier 50 is one, the expected value of the processing time of the overall classification process is the processing time of the sub classifier 51 itself. Therefore, first, it is determined whether the number of unexecuted sub classifiers 51 is one or zero (S801). If zero, the expected value is set to zero, and if it is one, the sub classifier 51 is set. Is set to the expected value (S811), and the process is terminated.

  FIG. 27 is an explanatory diagram of tree candidates (tree candidates) for two sub classifiers. Here, it is assumed that there are two sub-classifiers A and B. Hereinafter, determination of the optimal order of the two discriminators will be described with reference to FIGS. 26 and 27. FIG.

  If the number of sub classifiers 51 is two (NO in S801), first, any one of the sub classifiers is selected (S802). Here, first, it is assumed that the sub classifier A is selected.

  Next, all combinations of sub-classifiers that may occur are created (S803). Here, when the condition Xa1 (for example, the condition that the discriminant value of the sub classifier A is smaller than the second negative threshold), the combination of “unexecuted sub classifier is B” and the condition Ya1, Assume that there are two combinations of “unexecuted sub classifiers are zero”.

  Next, in S804, it is assumed that the former combination (the combination “unexecuted sub-identifier is B”) is selected. In S805, this flow is recursively executed, and since there is one unexecuted sub classifier, the processing time of the sub classifier B is output as an expected value (S811), and the process returns. In S806, since the latter combination remains, the determination is NO.

  In the next S804, the latter combination (the combination “unexecuted sub-identifier is zero”) is selected. This flow is executed recursively, and since the unexecuted sub classifier is zero, an expected value of zero is output (S811), and the process returns. In step S806, since all combinations have been completed, the determination is YES.

  Next, the expected processing time of the tree candidate is calculated (S807). Here, the expected processing time is the sum of the value obtained by multiplying the occurrence probability of a certain condition by the expected value of the combination corresponding to the condition, and the processing time of the sub-identifier selected in S802. That is, the expected value of the processing time when the overall identification process is executed according to the first tree candidate in FIG. 27 (the left tree candidate in FIG. 27) is calculated.

  Next, it is determined whether there is another unexecuted sub classifier (S808). Of the two sub-classifiers, sub-classifier B has not yet been selected, so the determination here is “Yes”. Similarly, the processing of S802 to S807 is performed, and the expected value of the processing time when the overall identification processing is executed according to the second tree candidate in FIG. 27 (the tree candidate on the right side in FIG. 27) is calculated. .

  Then, the tree candidate with the shortest expected processing time is selected (S809). Here, it is assumed that the first tree candidate is selected as a result of comparing the expected processing times of the first tree candidate and the second tree candidate.

  With the above processing, the optimum order when the number of sub-classifiers 51 is two is obtained.

  FIG. 28 is an explanatory diagram of tree candidates (tree candidates) for three sub classifiers. Here, it is assumed that there are three sub-classifiers A to C. Hereinafter, determination of the optimal order of the three classifiers will be described with reference to FIGS.

  If the number of sub classifiers 51 is two (NO in S801), first, any one of the sub classifiers is selected (S802). Here, first, it is assumed that the sub classifier A is selected. In the sub-identifier A, it is assumed that there are three combinations under three conditions (condition Xa1, condition Ya1, and condition Za1).

  First, in S804, it is assumed that the combination in the condition Xa1 (“2 unexecuted sub classifiers B and C”) is selected. In this case, in the next step S805, as already described, the optimum order is determined when the number of sub classifiers is two, the expected value in the optimum order is output (S810), and the process returns. Similarly, the optimum order and expected value of the remaining combinations (the combination in the condition Ya1 and the combination in the condition Za1) are also calculated.

  In S807, an expected value of the processing time when the overall identification process is executed according to the first tree candidate in FIG. 28 is calculated. Similarly, the expected value of the processing time of the remaining tree candidates (not shown) is calculated. Then, the tree candidate with the shortest expected processing time is selected (S809).

  With the above processing, the optimum order when the number of sub-identifiers 51 is three is obtained. Even when the number of sub classifiers is four or more, the optimal order is obtained by the recursive procedure as described above.

  In this way, in the present embodiment, the tree data of FIG. 25 is configured so as to have an optimal selection order. Then, the overall classifier 50 selects the sub-classifier 51 with reference to this tree data at S201 in FIG. 7, thereby speeding up the processing time of the overall identification process.

=== Second Embodiment ===
In the first embodiment described above, the selection order of the five sub classifiers 51 is determined so that the processing time of the total classifying process by the total classifier 50 is increased.
However, the selection order of the three sub partial classifiers may be determined so that the processing time of the partial classification processing by the partial classifier 60 is accelerated.

=== Third Embodiment ===
According to the first embodiment described above, the scene identification program is configured so that the selection order of the sub-identifiers 51 in the overall identification process is the order shown in FIG. 25, and this program is stored in the printer-side controller 20 of the printer 4. It is installed.

  On the other hand, when selecting the sub discriminator 51 in S201 of FIG. 7, for example, the scene discrimination program may be configured such that the printer-side controller 20 executes the processing of FIG.

=== Other Embodiments ===
Although a printer or the like as one embodiment has been described, the above embodiment is for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and it is needless to say that the present invention includes equivalents thereof. In particular, the embodiments described below are also included in the present invention.

<About the printer>
In the above-described embodiment, the printer 4 performs the scene identification process, but the digital still camera 2 may perform the scene identification process. Further, the image identification device that performs the above-described scene identification processing is not limited to the printer 4 or the digital still camera 2. For example, an image identification device such as a photo storage that stores a large amount of image files may perform the scene identification process described above. Of course, a personal computer or a server installed on the Internet may perform the scene identification process.

<About image files>
The image file described above is in the Exif format, but the image file format is not limited to this. Further, the above-described image file is a still image, but may be a moving image. In short, if the image file includes image data and additional data, the scene identification process as described above can be performed.

<About Support Vector Machine>
For the above-described sub classifier 51 and sub partial classifier 61, a classification method using a support vector machine (SVM) is used. However, the method for identifying whether or not the identification target image belongs to a specific scene is not limited to using a support vector machine. For example, pattern recognition such as a neural network may be employed.

<About scene identification>
In the above-described embodiment, the sub classifier 51 and the sub partial classifier 61 identify whether or not the image indicated by the image data belongs to a specific scene. However, the sub discriminator 51 and the sub partial discriminator 61 are not limited to identifying whether or not they belong to a specific scene, as long as they can classify whether or not an image indicated by image data belongs to a specific category. For this reason, the sub classifier 51 and the sub partial classifier 61 may identify whether the image indicated by the image data has a specific pattern shape, for example.

<Selection order of sub classifier / sub partial classifier>
In the above-described embodiment, after the overall identification process by the overall classifier 50, the partial identification process by the partial classifier 60 is performed (see FIGS. 6 and 7). That is, in the above-described embodiment, after the five sub classifiers 51 are selected, three sub partial classifiers 61 are selected. However, the present invention is not limited to this.

  For example, the selection order of the sub classifier 51 and the sub partial classifier 61 may be mixed. In this case, the selection order of the sub classifier 51 and the sub partial classifier 61 is determined so that the expected value of the processing time of the scene classification process is the shortest. However, since the processing time of the sub partial discriminator 61 is slower than the processing time of the sub discriminator 51 (because the sub partial discriminator 61 calculates the discriminant value a plurality of times), the optimum selection order can be calculated. Eventually, after the five sub classifiers 51 are selected, the three sub partial classifiers 61 may be selected.

=== Summary ===
(1) In the above-described embodiment, the overall classifier 50 has a plurality of sub-classifiers 51. The sub-identifier 51 performs identification processing for identifying whether or not the image indicated by the image data belongs to a specific scene. And in the above-mentioned embodiment, the scene (an example of a category) of the image which image data shows is classified by combining several such sub discriminators.
Here, if the identification processing by all the sub-classifiers 51 is performed every time the scene identification processing is performed, the processing speed of the scene identification processing becomes slow. Therefore, in the above-described reference examples and embodiments, when the sub-classifier 51 can identify that the image indicated by the image data belongs to a specific scene, the processing by another sub-classifier 51 is omitted, and the scene The processing speed of the identification process is increased. For example, if the landscape discriminator 51L of the overall discriminator 50 can discriminate a landscape image (YES in S204 in FIG. 7), the processing by the evening scene discriminator 51S of the overall discriminator 50 is omitted, and the processing speed of the scene discrimination processing is increased. .
By the way, when the process of the sub classifier 51 may be omitted in this way, the expected value of the processing time of the overall classification process varies depending on the order of the five sub classifiers 51. Therefore, in the above-described reference example, the five sub-identifiers 51 are sequentially executed in a fixed order in which the expected value of the processing time of the overall identification process is the shortest. As a result, the processing time of the entire identification process is increased.

  However, when the five sub classifiers 51 are sequentially executed in a fixed order of landscape → evening scene → night scene → flower → autumn leaves, the autumnal scene classifier 51R precedes the evening scene classifier 51S and the flower classifier 51F. It does not happen to be selected. On the other hand, when the night scene is excluded by the landscape discriminator 51L, if the expected value of the processing time is shorter in the order of autumnal leaves → evening scene → flower than in the order of evening scene → flower → autumn leaves, the latter order is selected. Is preferable.

  Therefore, in the above-described embodiment, the sub classifier 51 to be executed next is determined according to the value of the discriminant of the sub classifier to be executed first. For example, if the discriminant value of the landscape discriminator 51L is smaller than −0.45 (second negative threshold corresponding to the night scene), the overall discriminator 50 selects the evening scene discriminator 51S. On the other hand, if the discriminant value of the landscape discriminator 51L is larger than −0.45 and smaller than 1.27 (positive threshold), the overall discriminator 50 selects the autumn color discriminator 51R. Thereby, in the first embodiment, the processing speed of the overall identification process can be increased as compared with the reference example in which the selection order is fixed.

(2) In the above-described embodiment, for example, when the landscape classifier 51L can identify that the classification target image is a landscape (YES in S204 in FIG. 10), the processing by the other sub-classifier 51 is no longer necessary. Therefore, for example, the processing by the evening scene classifier 51 is omitted. Thereby, the processing speed of the scene identification process is increased.

(3) In the above-described embodiment, for example, if the value of the discriminant of the landscape discriminator 51L is larger than the second negative threshold, it is determined that the classification target image does not belong to a scene different from the landscape (for example, a night view). The identification by the classifier (for example, night scene classifier 51N) is omitted. As a result, the speed of the identification process can be increased.

(4) The sub-identifier 51 described above calculates a discriminant value (a value corresponding to the probability that the image belongs to a specific category), and identifies whether the image belongs to a specific scene based on the value. is doing. In the above-described embodiment, the discriminator 51 to be executed next is determined using the value of this discriminant. Thereby, there is no need to separately calculate an index for determining the sub classifier 51 to be executed next, so that the determination of the sub classifier 51 to be executed next is simplified.

(5) In the above-described embodiment, the overall classifier 50 determines the next sub-classifier 51 to be executed using the tree data. Here, the tree data is data in which the discriminant value of the sub classifier 51 is associated with the sub classifier 51 to be selected next, as shown in FIG. By using such tree data, the overall classifier 50 can determine the sub classifier 51 to be executed next in accordance with the value of the discriminant of the sub classifier to be executed first.

(6) In the above-described embodiment, the expected processing time of each tree candidate is calculated (see S807 in FIG. 26), and the tree candidate with the shortest expected processing time is selected from all tree candidates (S809). Data is created. Thereby, the processing speed of the whole identification process can be increased.

(7) In the above-described embodiment, a determination flow is prepared in which the n-1 sub discriminators 51 determine the selection order that minimizes the processing time and output the processing time (see FIG. 26). When determining the selection order in which the processing time is the shortest by the n sub discriminators 51, first, one sub discriminator 51 is selected, and the remaining n-1 sub discriminators 51 are the shortest in processing time. The selection order to become is determined by recursive processing, and the expected processing time of the tree candidate that the selected sub-identifier 51 is executed first is calculated (S807). In this way, the expected processing time of n tree candidates is calculated, and the selection order that minimizes the expected processing time is determined (S809).
Thereby, the optimal selection order can be determined by a simple procedure.

(8) The above-described printer (corresponding to an image identification device) includes a printer-side controller 20 (see FIG. 2). The printer-side controller 20 classifies the scene of the image indicated by the image data by combining a plurality of sub-identifiers 51 that identify whether the image indicated by the image data belongs to a specific category. Further, the printer-side controller 20 omits, for example, the processing by the evening scene classifier 51S according to the classification result of the landscape classifier 51L.

  In the above-described embodiment, the printer-side controller 20 determines the sub classifier 51 to be executed next in accordance with the value of the discriminant of the sub classifier to be executed first. Thereby, in the first embodiment, the processing speed of the overall identification process can be increased as compared with the reference example in which the selection order is fixed.

(9) The memory 23 stores a program for causing the printer 4 to execute the processes of FIGS. That is, according to the code for selecting the identification process for identifying whether or not the image indicated by the image data belongs to a specific category in order for each of a plurality of categories and the result of the identification process, A code that omits the identification process that is not performed, and a code that identifies the category of the image based on the result of the identification process. The program also includes a code for causing the image identification apparatus to determine the selection order of the identification process to be performed later according to the result of the identification process to be performed first.

It is explanatory drawing of an image processing system. 2 is an explanatory diagram of a configuration of a printer. FIG. It is explanatory drawing of the automatic correction function of a printer. It is explanatory drawing of the relationship between the scene of an image, and the correction content. It is a flowchart of the scene identification process by a scene identification part. It is explanatory drawing of the function of a scene identification part. It is a flowchart of a whole identification process. It is explanatory drawing of an identification object table. It is explanatory drawing of the affirmation threshold value of the whole identification process. It is explanatory drawing of Recall and Precision. It is explanatory drawing of a 1st negative threshold value. It is explanatory drawing of a 2nd negative threshold value. FIG. 13A is an explanatory diagram of a threshold table. FIG. 13B is an explanatory diagram of threshold values in the landscape classifier. FIG. 13C is an explanatory diagram of an outline of the process of the landscape classifier. It is a flowchart of a partial identification process. It is explanatory drawing of the order of the partial image which an evening scene partial identifier selects. It is a Recall and Precision graph when the evening scene image is identified only by the top 10 partial images. FIG. 17A is an explanatory diagram of determination by the linear support vector machine. FIG. 17B is an explanatory diagram of discrimination using a kernel function. It is a flowchart of an integrated identification process. It is explanatory drawing of the determination flow of the selection order in a reference example. It is a flowchart which calculates the expected value of the processing time in a certain selection order. It is a table | surface of the processing time of the sub discrimination device 51. FIG. This is a probability distribution of discriminant values when an evaluation sample image is identified by the landscape classifier 51L. It is explanatory drawing of the probability Ppos1 in which the landscape identification device 51L can identify a landscape image. It is explanatory drawing of the probability Pneg (1, 2) that the landscape discriminator 51L sets the evening scene negative flag. It is explanatory drawing of probability Pneg (1, 3) that the landscape identification device 51L sets the negative flag of a night view. It is a table | surface of the probability P (i, j) by which a j-th sub discriminator is performed when an i-th sub discriminator is performed. It is a flowchart which calculates the execution probability Pn of each sub discriminator 51 in a certain selection order. It is a conceptual diagram of the data (tree data) of the tree structure referred in the case of order determination in 1st Embodiment. It is a flowchart of recursive optimal order determination in 1st Embodiment. It is explanatory drawing of the tree candidate (tree candidate) of two sub discriminators. It is explanatory drawing of the tree candidate (tree candidate) of three sub classifiers.

Explanation of symbols

2 Digital still camera, 2A mode setting dial,
4 Printer, 6 Memory card,
10 printing mechanism, 11 head, 12 head control unit, 13 motor, 14 sensor,
20 printer-side controller, 21 slots, 22 CPU, 23 memory,
24 control unit, 25 drive signal generator,
31 storage unit, 31A image storage unit, 31B result storage unit,
32 face identification unit, 33 scene identification unit, 34 image correction unit, 35 printer control unit,
40 feature quantity acquisition unit, 50 global classifier, 51 sub classifier, 51L landscape classifier,
51S evening scene classifier, 51N night scene classifier, 51F flower classifier, 51R autumn leaves classifier,
60 partial classifiers, 61 sub partial classifiers, 61S evening scene partial classifiers,
61F Flower partial classifier, 61R Autumn colored partial classifier,
70 Integrated identifier,

Claims (9)

Identification processing for identifying whether the image indicated by the image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
In the image identification method for identifying the category of the image based on the result of the identification process,
An image identification method, wherein a selection order of identification processes to be performed later is determined in accordance with a result of identification processes to be performed first. The image identification method according to claim 1,
An image identification method characterized in that, when it is identified in a certain identification process that the image belongs to a specific category corresponding to the identification process, the identification process that has not yet been performed is omitted. The image identification method according to claim 1 or 2,
An image identification method in which, when it is identified that the image does not belong to a specific category corresponding to an identification process different from the identification process in the identification process, the another identification process is omitted. . The image identification method according to any one of claims 1 to 3,
In the identification process, a value corresponding to the probability that the image belongs to the specific category is calculated, and based on the value, it is identified whether the image belongs to the specific category,
An image identification method, wherein a selection order of identification processing to be performed later is determined according to the value calculated in the identification processing performed first. The image identification method according to any one of claims 1 to 4,
The image identification method, wherein the selection order is determined by using data in which a result of the identification processing performed first and an identification processing performed next are associated with each other. The image identification method according to any one of claims 1 to 5,
The processing time until the category of the image is identified based on the processing time of the identification processing performed later and the occurrence probability of the result of the identification processing performed earlier for a plurality of candidates having different selection orders Expected value of
The image identification method, wherein the selection order is determined by selecting a candidate having a short expected value from the plurality of candidates. The image identification method according to claim 6, comprising:
There is a determination process for determining a candidate with a short expected value when performing n-1 identification processes,
When determining a candidate with a short expected value when performing n identification processes, the expected value is calculated using the determination process for each of n candidates that have different identification processes to be performed first. An image identification method, wherein the selection order is determined by selecting a candidate with a short expected value from candidates. An image identification device comprising a controller,
The controller is
Identification processing for identifying whether the image indicated by the image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
Based on the result of the identification process, the category of the image is identified,
An image identification apparatus, wherein a selection order of identification processing performed later is determined according to the value calculated in the identification processing performed earlier. In the image identification device,
Identification processing for identifying whether an image indicated by image data belongs to a specific category is performed by selecting in order for each of the plurality of categories,
Depending on the result of the identification process, the identification process that has not yet been performed is omitted,
In a program for identifying the category of the image based on the result of the identification process,
A program for causing the image identification apparatus to determine a selection order of identification processing to be performed later according to a result of identification processing to be performed first.

Images