JP4946750B2 - Setting method, identification method and program - Google Patents

Description

  The present invention relates to a setting method, an identification method, and a program.

  Some digital still cameras have a mode setting dial for setting a shooting mode. When the user sets the shooting mode with the dial, the digital still camera determines shooting conditions (such as exposure time) according to the shooting mode and performs shooting. When shooting is performed, the digital still camera generates an image file. In this image file, additional data such as shooting conditions at the time of shooting is added to the image data of the shot image.

It is also possible to identify the category (class) of the image indicated by the image data using the additional data. However, in this case, the identifiable category is limited to the type of data recorded in the additional data. Therefore, image data is analyzed to identify the category of the image indicated by the image data (see Patent Documents 1 and 2).
Japanese Patent Laid-Open No. 10-302067 JP 2006-511000 Gazette

The result of the identification process may not match the user's preference. In such a case, it is desirable to change the setting of the identification process according to the user's preference.
It is an object of the present invention to provide an interface that can be easily changed in accordance with user preferences.

The main invention for achieving the above object, and compares the value with a predetermined threshold of the evaluation function for evaluating the identification target based on the comparison result, the identification identifying whether the identification target to a class belongs A process setting method for extracting a predetermined number of samples from a plurality of samples by extracting samples belonging to the class and samples not belonging to the class according to the threshold And displaying the predetermined number of extracted samples arranged on the display unit in the order of the value of the evaluation function, and displaying a mark between the sample belonging to the class and the sample not belonging to the class, A step of displaying the mark between the sample and another sample by moving the position of the mark in accordance with an instruction of , Depending on the position of the mark determined of the user, a setting method characterized in that it comprises a setting change step of changing the setting of the identification process.

  Other features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

  At least the following matters will become clear from the description of the present specification and the accompanying drawings.

A setting method of identification processing for identifying whether or not an identification target belongs to a certain class,
An extraction step of extracting samples belonging to the class and samples not belonging to the class;
A plurality of the extracted samples are displayed side by side on the display unit, and a mark is displayed between the sample belonging to the class and the sample not belonging to the class, and the position of the mark is moved according to a user instruction A display step of displaying the mark between another sample and the sample;
A setting method comprising: a setting changing step for changing the setting of the identification process in accordance with the position of the mark determined by the user will be apparent.
According to this setting method, it is possible to provide an interface that can be easily changed according to the user's preference.

  In the setting change step, when the position of the mark is determined in a state where a sample belonging to the class is located between the mark and a sample not belonging to the class, the mark and a sample not belonging to the class In the state where the samples belonging to the class between the mark and the attribute information have been changed, the samples that do not belong to the class are located between the mark and the samples belonging to the class. When the position is determined, the sample information that does not belong to the class between the mark and the sample belonging to the class is changed in attribute information to that belonging to the class, and re-learning based on the changed attribute information It is desirable that the setting of the identification process is changed according to the result of relearning. Thereby, the identification process reflecting a user preference can be performed.

  In the extraction step, a representative sample is extracted from each clustered cluster, and in the setting change step, when the attribute information of the representative sample is changed, the sample belonging to the same cluster as the sample It is desirable to change the attribute information. Thereby, the attribute information of a plurality of samples can be collectively changed.

  Preferably, a plurality of identification processes are prepared in advance, and in the setting change step, an identification process corresponding to the position of the mark determined by the user is selected from the plurality of identification processes. This eliminates the need for relearning.

  The identification process is to identify whether the identification target belongs to the class based on a hyperplane separating a space, and in the extraction step, projects the sample onto a normal of the hyperplane, It is desirable to determine a sample to be extracted based on the position of the sample projected on the normal. Thereby, samples can be extracted in descending order of certainty.

  The identification process compares an evaluation function value for evaluating the identification object with a predetermined threshold value, and identifies whether the identification object belongs to the class based on a comparison result, In the changing step, it is desirable to change the evaluation function. Thereby, the identification process reflecting a user preference can be performed.

  The identification process compares an evaluation function value for evaluating the identification object with a predetermined threshold value, and identifies whether the identification object belongs to the class based on a comparison result, In the changing step, it is desirable to change the threshold value. Thereby, the identification process reflecting a user preference can be performed.

An identification method for performing identification processing for identifying whether or not an identification target belongs to a certain class,
An extraction step of extracting samples belonging to the class and samples not belonging to the class;
A plurality of the extracted samples are displayed side by side on the display unit, and a mark is displayed between the sample belonging to the class and the sample not belonging to the class, and the position of the mark is moved according to a user instruction A display step of displaying the mark between another sample and the sample;
A setting change step for changing the setting of the identification process according to the position of the mark determined by the user;
An identification method characterized by comprising an identification step for identifying whether or not the identification object belongs to the class by the identification process whose setting has been changed.
According to this identification method, it is possible to provide an interface that can be easily changed according to the user's preference.

In the identification device that performs identification processing to identify whether the identification target belongs to a certain class,
Samples belonging to the class and samples not belonging to the class are extracted,
A plurality of extracted samples are displayed side by side on the display unit, and a mark is displayed between a sample belonging to the class and a sample not belonging to the class, and the position of the mark is moved according to a user instruction. To display the mark between another sample and the sample,
A program that changes the setting of the identification process according to the position of the mark determined by the user is also made clear.
According to this program, it is possible to cause the identification apparatus to realize an interface that can be easily changed according to the user's preference.

=== Overall Description ===
First, the basic configuration and processing of identification processing will be described. After this description, this embodiment will be described in detail.

<Overall configuration>
FIG. 1 is an explanatory diagram of an image processing system. This image processing system includes a digital still camera 2 and a printer 4.

  The digital still camera 2 is a camera that acquires a digital image by forming an image of a subject on a digital device (CCD or the like). The digital still camera 2 is provided with a mode setting dial 2A. The user can set the shooting mode according to the shooting conditions by using the dial 2A. For example, when the “night scene” mode is set by the dial 2A, the digital still camera 2 performs shooting under shooting conditions suitable for night scene shooting by decreasing the shutter speed or increasing the ISO sensitivity.

  The digital still camera 2 stores an image file generated by photographing in the memory card 6 in accordance with the file format standard. In the image file, not only digital data (image data) of a captured image but also additional data such as a shooting condition (shooting data) at the time of shooting is stored.

  The printer 4 is a printing device that prints an image indicated by image data on paper. The printer 4 is provided with a slot 21 into which the memory card 6 is inserted. The user can take a picture with the digital still camera 2, remove the memory card 6 from the digital still camera 2, and insert the memory card 6 into the slot 21.

  The panel unit 15 includes a display unit 16 and an input unit 17 having various buttons. The panel unit 15 functions as a user interface. The display unit 16 is configured by a liquid crystal display. If the display unit 16 is a touch panel, the display unit 16 also functions as the input unit 17. The display unit 16 displays a setting screen for setting the printer 4, an image of image data read from the memory card, a screen for confirmation and warning to the user, and the like. Various screens displayed by the display unit 16 will be described later.

  FIG. 2 is an explanatory diagram of the configuration of the printer 4. The printer 4 includes a printing mechanism 10 and a printer-side controller 20 that controls the printing mechanism 10. The printing mechanism 10 includes a head 11 that ejects ink, a head control unit 12 that controls the head 11, a motor 13 for conveying paper, and a sensor 14. The printer-side controller 20 includes a memory slot 21 for transmitting and receiving data from the memory card 6, a CPU 22, a memory 23, a control unit 24 for controlling the motor 13, and a drive signal for generating a drive signal (drive waveform). And a generation unit 25. The printer-side controller 20 also includes a panel control unit 26 that controls the panel unit 15.

  When the memory card 6 is inserted into the slot 21, the printer-side controller 20 reads out the image file stored in the memory card 6 and stores it in the memory 23. Then, the printer-side controller 20 converts the image data of the image file into print data for printing by the printing mechanism 10, controls the printing mechanism 10 based on the printing data, and prints an image on paper. This series of operations is called “direct printing”.

  “Direct printing” is not only performed by inserting the memory card 6 into the slot 21, but also by connecting the digital still camera 2 and the printer 4 with a cable (not shown).

  The image file stored in the memory card 6 is composed of image data and additional data. The image data is composed of a plurality of pixel data. The pixel data is data indicating pixel color information (gradation value). An image is formed by arranging the pixels in a matrix. Therefore, the image data is data indicating an image. The additional data includes data indicating the characteristics of image data, shooting data, thumbnail image data, and the like.

<Outline of automatic correction function>
When printing a “person” photo, there is a demand to clean the skin tone. In addition, when printing a “landscape” photograph, there is a demand for emphasizing the blue of the sky and the green of trees and grass. Therefore, the printer 4 has an automatic correction function that analyzes the image file and automatically performs a suitable correction process.

  FIG. 3 is an explanatory diagram of the automatic correction function of the printer 4. Each element of the printer-side controller 20 in the figure is realized by software and hardware.

  The storage unit 31 is realized by a partial area of the memory 23 and the CPU 22. All or part of the image file read from the memory card 6 is developed in the image storage unit 31 </ b> A of the storage unit 31. In addition, the calculation result of each element of the printer-side controller 20 is stored in the result storage unit 31B of the storage unit 31.

  The face identification unit 32 is realized by the CPU 22 and a face identification program stored in the memory 23. The face identification unit 32 analyzes the image data stored in the image storage unit 31A and identifies the presence or absence of a face. When the face identifying unit 32 identifies that there is a face, the image to be identified is identified as belonging to the “person” scene. In this case, the scene identification process by the scene identification unit 33 is not performed. Since the face identification process by the face identification unit 32 is the same as a process that has already been widely performed, detailed description thereof is omitted.

  The scene identification unit 33 is realized by the CPU 22 and a scene identification program stored in the memory 23. The scene identification unit 33 analyzes the image file stored in the image storage unit 31A and identifies the scene of the image indicated by the image data. When the face identifying unit 32 identifies that there is no face, a scene identifying process by the scene identifying unit 33 is performed. As will be described later, the scene identifying unit 33 identifies whether the image to be identified is a “landscape”, “evening scene”, “night scene”, “flower”, “autumn leaves”, or “other” image. .

FIG. 4 is an explanatory diagram of a relationship between an image scene and correction contents.
The image correction unit 34 is realized by the CPU 22 and an image correction program stored in the memory 23. The image correction unit 34 converts the image data of the image storage unit 31A based on the identification results (identification results of the face identification unit 32 and the scene identification unit 33) stored in the result storage unit 31B (described later) of the storage unit 31. to correct. For example, when the identification result of the scene identification unit 33 is “landscape”, correction is performed so that blue is emphasized and green is emphasized. The image correction unit 34 may correct the image data by reflecting not only the scene identification result but also the contents of the image data of the image file. For example, when the exposure correction is negative, the image data may be corrected so as not to brighten the dark atmosphere image.

  The printer control unit 35 is realized by a printer control program stored in the CPU 22, the drive signal generation unit 25, the control unit 24, and the memory 23. The printer control unit 35 converts the corrected image data into print data, and causes the printing mechanism 10 to print the image.

<Scene identification processing>
FIG. 5 is a flowchart of the scene identification process performed by the scene identification unit 33. FIG. 6 is an explanatory diagram of the function of the scene identification unit 33. Each element of the scene identification unit 33 in the figure is realized by software and hardware. The scene identification unit 33 includes a feature amount acquisition unit 40, an overall classifier 50, a partial classifier 60, and an integrated classifier 70 shown in FIG.

First, the feature amount acquisition unit 40 analyzes the image data developed in the image storage unit 31A of the storage unit 31 and acquires partial feature amounts (S101). Specifically, the feature amount acquisition unit 40 divides the image data into 8 × 8 64 blocks, calculates the color average and variance of each block, and acquires the color average and variance as partial feature amounts. Here, each pixel has gradation value data in the YCC color space, and the average value of Y, the average value of Cb, and the average value of Cr are calculated for each block, and the variance of Y and the variance of Cb are calculated. And the variance of Cr are calculated respectively. That is, three color averages and three variances are calculated as partial feature amounts for each block. These color averages and variances indicate the characteristics of the partial images in each block. Note that an average value or variance in the RGB color space may be calculated.
Since the color average and variance are calculated for each block, the feature amount acquisition unit 40 expands the image data for the blocks in the block order without expanding all the image data in the image storage unit 31A. For this reason, the image storage unit 31A does not necessarily have a capacity sufficient to expand all of the image files.

  Next, the feature amount acquisition unit 40 acquires the entire feature amount (S102). Specifically, the feature quantity acquisition unit 40 acquires the overall color average, variance, center of gravity, and shooting information of the image data as the overall feature quantity. Note that these color averages and variances indicate the overall characteristics of the image. The color average, variance, and center of gravity of the entire image data are calculated using the partial feature values calculated previously. For this reason, it is not necessary to re-expand the image data when calculating the entire feature amount, and the calculation speed of the entire feature amount is increased. Although the overall identification process (described later) is performed prior to the partial identification process (described later), the overall feature value is obtained after the partial feature value in order to increase the calculation speed. The shooting information is extracted from the shooting data of the image file. Specifically, information such as the aperture value, shutter speed, and the presence or absence of flash emission is used as the overall feature amount. However, not all shooting data of the image file is used as the entire feature amount.

  Next, the overall classifier 50 performs overall identification processing (S103). The overall identification process is a process for identifying (estimating) an image scene indicated by image data based on the overall feature amount. Details of the overall identification process will be described later.

  If the scene can be identified by the overall identification process (YES in S104), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  If the scene cannot be identified by the overall identification process (NO in S104), the partial classifier 60 performs the partial identification process (S105). The partial identification process is a process for identifying the scene of the entire image indicated by the image data based on the partial feature amount. Details of the partial identification processing will be described later.

  When the scene can be identified by the partial identification process (YES in S106), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

  If the scene cannot be identified by the partial identification process (NO in S106), the integrated discriminator 70 performs the integrated identification process (S107). Details of the integrated identification process will be described later.

  When the scene can be identified by the integrated identification process (YES in S108), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. On the other hand, if the scene cannot be identified by the integrated identification process (NO in S108), the image indicated by the image data is “other” (other than “landscape”, “evening scene”, “night scene”, “flower” or “autumn leaves”. Is stored in the result storage unit 31B (S110).

<Overall identification process>
FIG. 7 is a flowchart of the overall identification process. Here, the overall identification process will be described with reference to FIG.

  First, the overall classifier 50 selects one sub-classifier 51 from the plurality of sub-classifiers 51 (S201). The overall classifier 50 is provided with five sub-classifiers 51 for identifying whether an image to be identified (identification target image) belongs to a specific scene. The five sub classifiers 51 identify scenes of scenery, evening scene, night scene, flowers, and autumn leaves, respectively. Here, the overall classifier 50 selects the sub classifier 51 in the order of landscape → evening scene → night scene → flower → autumn leaves (the selection order of the sub classifier 51 will be described later). For this reason, first, the sub classifier 51 (landscape classifier 51L) for identifying whether or not the classification target image belongs to a landscape scene is selected.

  Next, the overall classifier 50 refers to the classification target table and determines whether or not a scene should be identified using the selected sub-classifier 51 (S202).

  FIG. 8 is an explanatory diagram of the identification target table. This identification target table is stored in the result storage unit 31B of the storage unit 31. In the identification target table, all fields are set to zero in the first stage. In the process of S202, the “No” column is referred to, and if it is zero, it is determined as YES, and if it is 1, it is determined as NO. Here, the overall classifier 50 refers to the “No” column in the “Scenery” column in the identification target table, and determines “YES” because it is zero.

  Next, the sub classifier 51 calculates the probability (confidence) that the classification target image belongs to a specific scene based on the entire feature amount (S203). For the sub classifier 51, a classification method using a support vector machine (SVM) is used. The support vector machine will be described later. When the classification target image belongs to a specific scene, the discriminant of the sub classifier 51 tends to be a positive value. When the classification target image does not belong to a specific scene, the discriminant of the sub classifier 51 tends to be a negative value. Further, the discriminant becomes a larger value as the certainty that the identification target image belongs to the specific scene is higher. For this reason, if the discriminant value is large, the probability (confidence) that the identification target image belongs to a specific scene is high, and if the discriminant value is small, the probability that the classification target image belongs to a specific scene is low. .

  Next, the sub discriminator 51 determines whether or not the value of the discriminant is larger than the positive threshold (S204). If the value of the discriminant is larger than the positive threshold, the sub discriminator 51 determines that the classification target image belongs to a specific scene.

  FIG. 9 is an explanatory diagram of an affirmative threshold value of the overall identification process. In the figure, the horizontal axis indicates an affirmative threshold, and the vertical axis indicates the probability of recall or precision. FIG. 10 is an explanatory diagram of Recall and Precision. If the discriminant value is greater than or equal to the positive threshold, the identification result is Positive. If the discriminant value is not greater than or equal to the positive threshold, the identification result is Negative.

  Recall indicates the recall rate and detection rate. Recall is the ratio of the number of images identified as belonging to a specific scene to the total number of images of the specific scene. In other words, Recall is the probability that the sub-identifier 51 identifies the image as a positive when the image of the specific scene is identified by the sub-identifier 51 (the probability that the image of the specific scene belongs to the specific scene. ). For example, when a landscape image is identified by the landscape classifier 51L, it indicates the probability that the landscape classifier 51L identifies it as belonging to a landscape scene.

  Precision indicates the correct answer rate and the correct answer rate. Precision is the ratio of the number of images in a particular scene to the total number of images identified as Positive. In other words, Precision indicates the probability that the image to be identified is a specific scene when the sub-classifier 51 that identifies the specific scene identifies it as Positive. For example, when the landscape classifier 51L identifies that it belongs to a landscape scene, it indicates the probability that the identified image is really a landscape image.

As can be seen from FIG. 9, the greater the positive threshold, the greater the Precision. For this reason, the larger the positive threshold value, the higher the probability that an image identified as belonging to a landscape scene, for example, is a landscape image. That is, the greater the positive threshold, the lower the probability of misidentification.
On the other hand, the larger the positive threshold, the smaller the Recall. As a result, for example, even when a landscape image is identified by the landscape classifier 51L, it is difficult to correctly identify it as belonging to a landscape scene. By the way, if the image to be identified can be identified as belonging to a landscape scene (YES in S204), the speed of the overall identification process is increased so as not to identify other remaining scenes (such as sunsets). For this reason, the larger the positive threshold, the lower the overall identification processing speed. Further, if the scene can be identified by the overall identification process, the partial identification process is not performed and the speed of the scene identification process is increased (S104). Therefore, as the positive threshold is increased, the scene identification process speed decreases. Become.
That is, if the positive threshold is too small, the probability of misidentification increases, and if it is too large, the processing speed decreases. Here, in order to set the correct answer rate (Precision) to 97.5%, the landscape affirmation threshold is set to 1.27.

  If the discriminant value is greater than the affirmative threshold value (YES in S204), the sub-classifier 51 determines that the classification target image belongs to a specific scene and sets an affirmative flag (S205). “Set an affirmative flag” means that the “affirmation” column in FIG. In this case, the overall discriminator 50 ends the overall discrimination process without performing discrimination by the next sub discriminator 51. For example, if the image can be identified as a landscape image, the entire identification process is terminated without identifying the sunset scene or the like. In this case, since the identification by the next sub-identifier 51 is omitted, the speed of the overall identification process can be increased.

  If the value of the discriminant is not greater than the positive threshold (NO in S204), the sub discriminator 51 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S206.

  Next, the sub discriminator 51 compares the discriminant value with a negative threshold value (S206). Thereby, the sub classifier 51 determines whether the classification target image does not belong to a predetermined scene. There are two types of such determinations. First, if the value of the discriminant of the sub-identifier 51 of a specific scene is smaller than the first negative threshold, it is determined that the classification target image does not belong to the specific scene. For example, if the discriminant value of the landscape classifier 51L is smaller than the first negative threshold, it is determined that the classification target image does not belong to a landscape scene. Second, if the value of the discriminant of the sub-identifier 51 of a specific scene is larger than the second negative threshold, it is determined that the classification target image does not belong to a scene different from the specific scene. . For example, if the discriminant value of the landscape classifier 51L is larger than the second negative threshold, it is determined that the classification target image does not belong to the night scene.

  FIG. 11 is an explanatory diagram of the first negative threshold. In the figure, the horizontal axis indicates the first negative threshold, and the vertical axis indicates the probability. The bold line in the graph is a True Negative Recall graph, and indicates the probability of correctly identifying an image other than a landscape image as not a landscape image. The thin line in the graph is a False Negative Recall graph, which indicates the probability of erroneously identifying a landscape image that is not a landscape image.

As can be seen from FIG. 11, the smaller the first negative threshold is, the smaller False Negative Recall is. For this reason, the smaller the first negative threshold, the lower the probability that an image identified as not belonging to a landscape scene is a landscape image, for example. That is, the probability of misidentification is reduced.
On the other hand, the True Negative Recall decreases as the first negative threshold decreases. As a result, it is difficult to identify an image other than a landscape image unless it is a landscape image. On the other hand, if the identification target image can be identified as not being a specific scene, the process by the sub-partial classifier 61 for the specific scene is omitted during the partial identification process to increase the scene identification processing speed (described later in FIG. 14). S302). For this reason, the scene identification processing speed decreases as the first negative threshold is decreased.
That is, if the first negative threshold is too large, the probability of misidentification increases, and if it is too small, the processing speed decreases. Here, in order to set False Negative Recall to 2.5%, the first negative threshold is set to −1.10.

  By the way, if the probability that an image belongs to a landscape scene is high, the probability that the image belongs to a night scene is inevitably low. For this reason, when the discriminant value of the landscape discriminator 51L is large, it may be identified that the scene is not a night scene. In order to perform such identification, a second negative threshold is provided.

  FIG. 12 is an explanatory diagram of the second negative threshold. In the figure, the horizontal axis indicates the value of the landscape discriminant, and the vertical axis indicates the probability. In the same figure, the Recall graph of the night view is drawn with a dotted line together with the Recall and Precision graph of FIG. If attention is paid to this dotted line graph, if the value of the discriminant of the landscape is larger than −0.45, the probability that the image is a night scene image is 2.5%. In other words, if the landscape discriminant value is greater than −0.45, even if the image is identified as not a night scene image, the probability of misidentification is only 2.5%. Therefore, here, the second negative threshold is set to −0.45.

  When the discriminant value is smaller than the first negative threshold value, or when the discriminant value is larger than the second negative threshold value (YES in S206), the sub-classifier 51 determines that the classification target image belongs to a predetermined scene. It is determined not to do so, and a negative flag is set (S207). “Set a negative flag” means to set the “No” column in FIG. For example, when it is determined that the image to be identified does not belong to a landscape scene based on the first negative threshold, the “denial” column in the “landscape” column is 1. Further, when it is determined that the identification target image does not belong to the night scene based on the second negative threshold, the “Negation” field in the “Night scene” field is “1”.

  FIG. 13A is an explanatory diagram of a threshold table. The threshold value table may be stored in the storage unit 31 or may be incorporated in a part of a program for executing the overall identification process. The threshold table stores data related to the affirmative threshold and the negative threshold described above.

  FIG. 13B is an explanatory diagram of threshold values in the landscape classifier 51L described above. An affirmative threshold value and a negative threshold value are preset in the landscape discriminator 51L. 1.27 is set as the positive threshold. The negative threshold includes a first negative threshold and a second negative threshold. -1.10 is set as the first negative threshold. In addition, a value is set for each scene other than the landscape as the second negative threshold.

  FIG. 13C is an explanatory diagram outlining the processing of the landscape classifier 51L described above. Here, for simplification of description, only the night view will be described for the second negative threshold. If the discriminant value is greater than 1.27 (YES in S204), the landscape classifier 51L determines that the classification target image belongs to a landscape scene. If the discriminant value is 1.27 or less (NO in S204) and is greater than −0.45 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to the night scene. To do. If the value of the discriminant is smaller than −1.10 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to a landscape scene. Note that the landscape discriminator 51L also determines whether the image to be identified does not belong to the scene based on the second negative threshold for sunset scenes, flowers, and autumn leaves. However, since these second negative threshold values are larger than the positive threshold values, the landscape discriminator 51L does not determine that the classification target image does not belong to these scenes.

  In the case of NO in S202, in the case of NO in S206, or when the processing in S207 is completed, the overall discriminator 50 determines the presence or absence of the next sub discriminator 51 (S208). Here, since the process by the landscape classifier 51L is finished, the overall classifier 50 determines in S208 that there is a next sub-classifier 51 (evening scene classifier 51S).

  Then, when the process of S205 is finished (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S208 that there is no next sub-classifier 51 (the identification target image is a specific image). When it cannot be determined that the scene belongs to the scene), the overall discriminator 50 ends the overall discrimination process.

As already described, when the overall identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the overall identification process (S104 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.
If the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  By the way, although not described above, the overall discriminator 50, when the sub discriminator 51 calculates the discriminant value, the Precision corresponding to the discriminant value is stored in the result storage unit 31B as information on the certainty factor. Remember. Of course, the discriminant value itself may be stored as information on the certainty factor.

<Partial identification processing>
FIG. 14 is a flowchart of the partial identification process. The partial identification process is performed when the scene cannot be identified by the overall identification process (NO in S104 of FIG. 5). As will be described below, the partial identification process is a process for identifying the scene of the entire image by identifying each scene of the divided partial images. Here, the partial identification process will be described with reference to FIG.

  First, the partial classifier 60 selects one sub partial classifier 61 from the plurality of sub partial classifiers 61 (S301). The partial discriminator 60 is provided with three sub partial discriminators 61. Each sub partial discriminator 61 discriminates whether or not each partial image divided into 8 × 8 64 blocks belongs to a specific scene. Here, the three sub partial classifiers 61 identify the scenes of sunset, flowers, and autumn leaves, respectively. Here, the partial discriminator 60 selects the sub partial discriminator 61 in the order of sunset scene → flower → autumn leaves (the selection order of the sub partial discriminator 61 will be described later). Therefore, first, the sub partial classifier 61 (evening scene partial classifier 61S) for identifying whether or not the partial image belongs to the sunset scene is selected.

  Next, the partial discriminator 60 refers to the discrimination target table (FIG. 8) and determines whether or not the scene should be discriminated using the selected sub partial discriminator 61 (S302). Here, the partial discriminator 60 refers to the “No” column of the “Evening Scene” column in the classification target table, and determines YES if it is zero, and NO if it is 1. When the evening scene classifier 51S sets a negative flag with the first negative threshold during the overall identification process or when another sub-classifier 51 sets a negative flag with the second negative threshold, in S302 It is judged as NO. If it is determined NO, the sunset partial identification process is omitted, and the partial identification process speed increases. However, for the convenience of explanation, it is assumed that YES is determined here.

Next, the sub partial discriminator 61 selects one partial image from the partial images divided into 8 × 8 64 blocks (S303).
FIG. 15 is an explanatory diagram of the order of partial images selected by the evening scene partial classifier 61S. When a scene of the entire image is identified from the partial image, it is desirable that the partial image used for identification is a portion where the subject exists. Therefore, thousands of samples of sunset scene images are prepared, each sunset scene image is divided into 64 blocks of 8 × 8, and blocks including sunset scene partial images (sun sunset sky and sky partial images) are extracted and extracted. Based on the position of each block, the existence probability of the sunset partial image in each block was calculated. Then, partial images are selected in order from the block having the highest existence probability. Note that the selection order information shown in the figure is stored in the memory 23 as part of the program.

  In the case of an evening scene image, since the sky of the evening scene often spreads from the vicinity of the center to the upper half, the existence probability increases in the upper half block from the vicinity of the center. In the case of an evening scene image, the lower 1/3 of the image is shaded by backlight, and the partial image alone often cannot be distinguished from the evening scene or the night scene, so the existence probability is lower in the lower 1/3 block. In the case of a flower image, since the composition is often such that a flower is arranged near the center, the probability of existence of a flower partial image near the center increases.

  Next, the sub partial classifier 61 determines whether or not the partial image belongs to a specific scene based on the partial feature amount of the selected partial image (S304). Similar to the sub classifier 51 of the overall classifier 50, the sub partial classifier 61 uses a discrimination method using a support vector machine (SVM). The support vector machine will be described later. If the discriminant value is a positive value, it is determined that the partial image belongs to a specific scene, and the sub partial classifier 61 increments the positive count value. If the discriminant value is a negative value, it is determined that the partial image does not belong to a specific scene, and the sub partial discriminator 61 increments the negative count value.

  Next, the sub partial discriminator 61 determines whether or not the positive count value is larger than the positive threshold value (S305). The positive count value indicates the number of partial images determined to belong to a specific scene. If the positive count value is larger than the affirmative threshold (YES in S305), the sub partial classifier 61 determines that the classification target image belongs to a specific scene, and sets an affirmative flag (S306). In this case, the partial discriminator 60 ends the partial discriminating process without performing discrimination by the next sub partial discriminator 61. For example, if the image can be identified as an evening scene image, the partial identification process is terminated without identifying flowers and autumn leaves. In this case, since the identification by the next sub partial classifier 61 is omitted, the speed of the partial classification process can be increased.

  If the positive count value is not greater than the positive threshold value (NO in S305), the sub partial classifier 61 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S307.

  If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold (YES in S307), the sub partial discriminator 61 proceeds to the process of S309. If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold value, the positive count value does not become larger than the positive threshold value even if the positive count value is incremented by all the remaining partial images. By proceeding with the process, the remaining partial images are not identified by the support vector machine. Thereby, the speed of the partial identification process can be increased.

  If the sub partial discriminator 61 determines NO in S307, the sub partial discriminator 61 determines whether there is a next partial image (S308). Here, all of the partial images divided into 64 pieces are not selected in order. In FIG. 15, only the top 10 partial images indicated by thick frames are selected in order. Therefore, when the identification of the tenth partial image is completed, the sub partial discriminator 61 determines that there is no next partial image in S308 (considering this point, “the number of remaining partial images” in S307). Is also determined).

  FIG. 16 is a Recall and Precision graph when an evening scene image is identified using only the top 10 partial images. If an affirmative threshold as shown in the figure is set, the accuracy rate (Precision) can be set to about 80%, the recall rate (Recall) can be set to about 90%, and identification with high accuracy is possible.

In the partial identification process, the evening scene image is identified using only 10 partial images. For this reason, it is possible to increase the speed of the partial identification processing compared to the case where the evening scene image is identified using all 64 partial images.
In the partial identification process, the sunset scene image is identified using the top tenth partial image having a high existence probability of the sunset scene partial image. For this reason, it is possible to set both Recall and Precision higher than the identification of an evening scene image using 10 partial images extracted by ignoring the existence probability.
In the partial identification process, partial images are selected in descending order of the existence probability of the sunset partial image. As a result, the determination in S305 is likely to be YES at an early stage. For this reason, in the present embodiment, the speed of the partial identification process can be increased as compared with the case where the partial images are selected in the order in which the presence probability level is ignored.

  When it is determined YES in S307, or when it is determined that there is no next partial image in S308, the sub partial discriminator 61 determines whether or not the negative count value is larger than the negative threshold (S309). . Since this negative threshold performs substantially the same function as the negative threshold (S206 in FIG. 7) in the above-described overall identification process, detailed description thereof is omitted. When YES is determined in S309, a negative flag is set as in S207 of FIG.

  In the case of NO in S302, in the case of NO in S309, or when the process of S310 is completed, the partial discriminator 60 determines whether or not there is a next sub partial discriminator 61 (S311). In the case after the processing by the evening scene partial classifier 61S is finished, since the flower partial classifier 61F and the autumnal leaves partial classifier 61R are still present as the sub partial classifier 61, the partial classifier 60 determines the next sub partial classifier in S311. It is determined that there is a container 61.

  Then, when the process of S306 is completed (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S311 that there is no next sub partial classifier 61 (the identification target image is specified). If it cannot be determined that the scene belongs to the scene), the partial discriminator 60 ends the partial discrimination processing.

As already described, when the partial identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the partial identification process (S106 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column.
When the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

  In the above description, the evening scene partial classifier 61S identifies evening scene images using ten partial images. However, the number of partial images used for identification is not limited to ten. Further, the other sub partial classifier 61 may identify an image using a different number of partial images from the sunset scene partial classifier 61S. Here, it is assumed that the flower portion identifier 61F identifies a flower image using 20 partial images, and the autumnal leaf portion identifier 61R identifies a autumnal leaf image using 15 partial images.

<Support vector machine>
Before describing the integrated identification process, the support vector machine (SVM) used in the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process will be described.

FIG. 17A is an explanatory diagram of determination by the linear support vector machine. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. The learning sample is divided into two classes A and B. In the figure, samples belonging to class A are indicated by circles, and samples belonging to class B are indicated by squares.
A boundary that divides the two-dimensional space into two is defined by learning using the learning sample. The boundary is defined by <w · x> + b = 0 (where x = (x1, x2), w is a weight vector, and <w · x> is an inner product of w and x). However, the boundary is defined by learning using a learning sample so that the margin is maximized. That is, in the case of the figure, the boundary is not a thick dotted line but a thick solid line.
The discrimination is performed using the discriminant f (x) = <w · x> + b. It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined.

  Here, the description is made using a two-dimensional space, but the present invention is not limited to this (that is, the feature amount may be two or more). In this case, the boundary is defined by a hyperplane.

  By the way, there are cases where the two classes cannot be separated by a linear function. In such a case, if the determination is performed by the linear support vector machine, the accuracy of the determination result is lowered. Therefore, if the feature quantity of the input space is nonlinearly transformed, that is, if the input space is nonlinearly mapped to a certain feature space, it can be separated by a linear function in the feature space. This is used in the nonlinear support vector machine.

  FIG. 17B is an explanatory diagram of discrimination using a kernel function. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. If the nonlinear mapping from the input space of FIG. 17B becomes a feature space as shown in FIG. 17A, it can be separated into two classes by a linear function. If the boundary is defined so that the margin is maximized in this feature space, the inverse mapping of the boundary in the feature space becomes the boundary shown in FIG. 17B. As a result, the boundary becomes nonlinear as shown in FIG. 17B.

Here, by using a Gaussian kernel, the discriminant f (x) becomes as follows (where M is the number of features, and N is the number of learning samples (or learning for contributing to the boundary) The number of samples), w _i is a weighting factor, y _j is the feature quantity of the learning sample, and x _j is the feature quantity of the input x).

  It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined. Further, the larger the value of the discriminant f (x), the higher the probability that the input x (this input x is different from the learning sample) belongs to the class A. On the contrary, the smaller the value of the discriminant f (x), the lower the probability that the input x (this input x is different from the learning sample) belongs to the class A.

  In the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process, the value of the discriminant f (x) of the support vector machine is used. The calculation of the value of the discriminant f (x) by the support vector machine takes time as the number of learning samples increases. For this reason, the sub partial classifier 61 that needs to calculate the value of the discriminant f (x) a plurality of times requires more processing time than the sub classifier 51 that only needs to calculate the value of the discriminant f (x) once. It takes.

  An evaluation sample is prepared separately from the learning sample. The above Recall and Precision graphs are based on the identification results for the evaluation samples.

<Integrated identification processing>
In the above-described overall identification process and partial identification process, the positive threshold value in the sub-classifier 51 and the sub-classifier 61 is set relatively high, and the Precision (correct answer rate) is set high. This is because, for example, if the accuracy rate of the landscape classifier 51L of the overall classifier is set low, the landscape classifier 51L misidentifies the autumnal image as a landscape image, and before the autumnal classifier 51R performs the classification. This is because a situation occurs in which the entire identification process ends. Here, by setting the Precision (accuracy rate) high, an image belonging to a specific scene is identified by the sub-identifier 51 (or sub-partial identifier 61) of the specific scene. (For example, the autumn leaves image is identified by the autumn leaves discriminator 51R (or the autumn leaf partial discriminator 61R))
However, if the Precision (accuracy rate) of the overall identification process or the partial identification process is set to be high, there is a high possibility that the scene cannot be identified by the overall identification process or the partial identification process. Therefore, when the scene cannot be identified by the overall identification process and the partial identification process, the integrated identification process described below is performed.

  FIG. 18 is a flowchart of the integrated identification process. As will be described below, the integrated identification process is a process of selecting a scene with the highest certainty factor based on the discriminant value of each sub-classifier 51 in the overall identification process.

  First, the integrated discriminator 70 extracts a positive scene based on the discriminant values of the five sub discriminators 51 (S401). At this time, the value of the discriminant calculated by each sub classifier 51 during the overall identification process is used.

Next, the integrated discriminator 70 determines whether or not a scene having a positive discriminant value exists (S402).
If there is a scene with a positive discriminant value (YES in S402), an affirmative flag is set in the maximum value scene column (S403), and the integrated identification process is terminated. Accordingly, it is determined that the identification target image belongs to the maximum value scene.
On the other hand, if there is no scene having a positive discriminant value (NO in S402), the integrated identification process is terminated without setting an affirmative flag. As a result, there is no scene in the affirmative column of the identification target table in FIG. That is, it cannot be identified to which scene the identification target image belongs.

  As already described, when the integrated identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the integrated identification process (S108 in FIG. 5). At this time, the scene identification unit 33 refers to the identification target table in FIG. 8 and determines whether or not there is 1 in the “affirmation” column. If it is determined NO in S402, the determination in S108 is also NO.

=== Outline of the present embodiment ===
Because there are individual differences in user preferences, some people prefer that certain images be identified as “landscape”, while others prefer not to be identified as “landscape”. Therefore, in this embodiment, it is possible to reflect the user's preference in the identification process.

  FIG. 19 is an explanatory diagram of a setting screen. The setting screen 161 is a screen displayed on the display unit 16 of the printer 4. On the setting screen 161, five images are displayed corresponding to each scene. These images are all learning sample images of the support vector machine (SVM). Here, the five images L1 to L5 displayed at the top corresponding to “landscape” will be described.

  Five images L1 to L5 are displayed so that the right image of the five images becomes an image irrelevant to the landscape (described later). In the initial setting, the learning samples corresponding to the three images L1 to L3 belong to the landscape, and the learning samples corresponding to the two images L4 and L5 must not belong to the landscape. Has been. Accordingly, at the beginning of the display of the setting screen 161, a boundary setting bar 161A is displayed between the image L3 and the image L4 so as to indicate the boundary between the image belonging to the landscape and the image not belonging to the landscape.

  The position of the boundary setting bar 161A can be changed by the user. For example, when the user determines that the image L3 displayed on the display unit 16 is not a landscape image, the user operates the panel unit 17 to select a boundary setting bar corresponding to the landscape among the five boundary setting bars 161A. 161A is selected, and the boundary setting bar 161A is moved to the left by a distance between the images L2 and L3.

  Then, the processing of the sub classifier 51 is changed according to the set position of the boundary setting bar 161A (described later). As a result, when the landscape discriminator 51L identifies a similar image of the image L3, the landscape discriminator 51L has been identified as belonging to a landscape scene, but can be discriminated as not belonging to a landscape scene. It becomes like this. That is, the user's preference is reflected in the identification process.

  In the following description, first, data stored in the memory 23 of the printer 4 will be described. Next, how to display the setting screen 161 will be described. Next, how the processing of the sub classifier 51 is changed after the boundary is set on the setting screen 161 will be described.

=== First Embodiment ===
<Learning sample data stored in memory>
First, data stored in the memory 23 of the printer 4 will be described. As will be described below, the memory 23 stores a data group shown in FIG. 20A and learning sample image data indicated by white dots in FIG. 20B.

  FIG. 20A shows a data group of learning samples stored in the memory 23. Here, a data group used for the support vector machine of the landscape classifier 51L is shown.

  As shown in the figure, information (image data) of the learning sample image itself is not stored, but the entire feature amount of the learning sample is stored in the memory 23. Further, the weighting coefficient w is also stored in the memory 23 in association with each learning sample. The weighting factor w can be calculated using the data group of the entire feature amount of the learning sample. Here, it is assumed that the weighting factor w is calculated in advance and stored in the memory 23. . The value of the above-described discriminant f (x) is calculated based on the above-described equation 1 using the entire feature amount y and the weight coefficient w of this data group. Note that since the weighting coefficient of the learning sample that does not contribute to the determination of the boundary is zero, it is not necessary to store the entire feature amount of the learning sample in the memory 23. It is assumed that the entire feature amount of the sample for use is stored in the memory 23.

  Furthermore, in the present embodiment, information (attribute information) indicating whether or not the scene belongs to a landscape scene is also stored in association with each learning sample. P is set as attribute information for items belonging to landscape scenes, and N is set as attribute information for items that do not belong. As will be described later, this attribute information is used when displaying the setting screen 161 of FIG. 19 and is changed according to the setting of the boundary setting bar 161A of FIG.

  FIG. 20B is an explanatory diagram of the distribution of each learning sample. Here, for simplification of explanation, learning samples are distributed in a two-dimensional space by two feature amounts. Each dot indicates the position of the learning sample on the two-dimensional space.

  Each learning sample is clustered in advance, and is clustered into 13 clusters (cluster A to cluster M) in the figure. Here, clustering is performed by a known k-means method. A clustering technique based on the k-means method is as follows. (1) First, the computer temporarily determines the position of the center of the cluster. Here, the positions of the 13 centers are provisionally determined at random. (2) Next, the computer classifies each learning sample into the nearest central cluster. Thereby, a new cluster is determined. (3) Next, the computer calculates the average value of the feature values of the learning samples for each cluster, and sets the average value as the position of the center of the new cluster. (4) If the position of the center of the new cluster does not change from the position of the center of the original cluster, the clustering is terminated, and if it has changed, the process returns to (2).

  Note that learning samples having similar properties belong to the same cluster. For example, a cluster A is configured by a learning sample for a blue sky image, or a cluster B is configured by a learning sample for a fresh green image.

  The white dot in FIG. 20B indicates the position of the learning sample closest to the center position of each cluster. The white dot learning sample is a sample representing each cluster (representative sample). The memory 23 stores image data of representative samples indicated by white dots. In other words, the memory 23 stores image data representing an image representing each cluster. As will be described later, this representative image data is used to display the setting screen 161 in FIG.

  As described above, the memory 23 of the printer 4 stores the data group shown in FIG. 20A and the representative sample image data shown by white dots in FIG. 20B. Note that the data indicating the cluster to which each learning sample belongs may or may not be stored in the memory 23. This is because the data indicating the cluster to which each learning sample belongs can be obtained using the data group of FIG. 20A.

<Processing until the setting screen 161 is displayed>
Next, how the printer-side controller 20 displays the setting screen 161 as shown in FIG. 19 will be described.

FIG. 21A is an explanatory diagram showing a state in which the representative sample is projected onto the normal line of the boundary (f (x) = 0). Also here, for simplification of explanation, it is assumed that representative samples are distributed in a two-dimensional space. For the sake of simplicity of explanation, this two-dimensional space is assumed to be a space that can be separated by a linear function as shown in FIG. 17A. Therefore, the boundary (f (x) = 0) that separates the landscape image sample and the non-landscape image sample is defined by a straight line. (In the default setting, the learning samples belonging to the clusters A to G are landscape images, and the learning samples belonging to the clusters H to M are non-landscape images.)
In the figure, the position of the representative sample in the two-dimensional space is indicated by a white dot, and the boundary (f (x) = 0) is indicated by a bold line. This boundary is a default boundary before the setting is changed.

  The printer-side controller 20 defines one normal to the boundary and projects a representative sample on this normal. The projected position is the intersection of a normal line and a straight line that passes through the representative sample and is parallel to the boundary (or a hyperplane if the boundary is a hyperplane). In this way, 13 representative samples are projected onto the normal. In other words, 13 representative samples are arranged in a straight line.

  FIG. 21B is an explanatory diagram of a representative sample projected on the normal line. A representative that is projected on the normal with the normal level horizontal so that the representative sample of the landscape image is located on the left side of the figure, in other words, the representative sample of the non-landscape image is located on the right side of the figure The positional relationship of the sample is shown.

  Next, the printer-side controller 20 defines five sections on the normal line. In the figure, a first section to a fifth section are defined. Each section is defined to have a predetermined length. Further, five sections are defined so that the position of the intersection of the normal line and the boundary (f (x) = 0) in FIG. 21A becomes the boundary between the two sections. Here, the position of the intersection of the normal line and the boundary (f (x) = 0) in FIG. 21A corresponds to the boundary between the third section and the fourth section. Note that there are a plurality of representative samples in each section.

  Next, the printer-side controller 20 extracts image data of a representative sample located at the center of each section. Here, the image data of the representative sample of cluster C is extracted from the first section. Similarly, image data of representative samples of clusters E, F, H, and L are extracted from the second section, the third section, the fourth section, and the fifth section, respectively. At this time, representative samples that belong to a landscape scene in the default settings are extracted from the first to third sections. In addition, representative samples that do not belong to the landscape scene in the default setting are extracted from the fourth section and the fifth section. The extracted image data is considered to represent each section.

  The printer-side controller 20 displays a setting screen 161 on the display unit 16 of the printer 4 using the extracted image data. The image data of the representative sample of the cluster C extracted from the first section is used for displaying the image L1 in FIG. Similarly, the image data of the representative samples of the clusters E, F, H, and L are used for displaying the images L2, L3, L4, and L5 in FIG. 19, respectively.

  Further, since the position of the intersection of the normal line in FIG. 21A and the boundary (f (x) = 0) corresponds to the boundary between the third section and the fourth section, the printer-side controller 20 causes the image L3 in FIG. A boundary setting bar 161A is displayed between (the representative sample image extracted from the third section) and the image L4 (the representative sample image extracted from the fourth section). By the way, since the images L1 to L3 are landscape images and the images L4 and L5 are non-landscape images, the boundary setting bar 161A is displayed between the landscape image and the non-landscape image.

  In the present embodiment, as described above, the position of the representative sample is projected onto the normal line of the boundary, and the representative sample to be extracted is determined based on the position of the representative sample projected onto the normal line. Thereby, in this embodiment, the image of five representative samples is displayed so that the larger the discriminant value is, the left side is. In other words, images of five representative samples can be displayed so that they are arranged from the left in descending order of certainty belonging to a landscape scene.

  Then, since the setting screen 161 in FIG. 19 is displayed as described above, a representative sample of a landscape image is displayed on the left side of the boundary setting bar 161A by default. On the right side of the boundary setting bar 161A, a representative sample of a non-landscape image is displayed by default. Then, the five images L1 to L5 are displayed so that the right image of the five images becomes an image irrelevant to the landscape. In addition, an image displayed near the boundary setting bar 161A is an image in which it is easy to determine whether or not the image is a landscape image depending on the user's preference.

  In the above description, a landscape scene has been described, but the printer controller 20 performs the same processing for other scenes. As a result, the printer-side controller 20 can also display portions other than the scenery on the setting screen 161 in FIG.

<Process after border is set on setting screen 161>
Next, as shown in FIG. 19, a process after the user moves the boundary setting bar 161A to the left and sets it between the images L2 and L3 will be described.

  After the boundary setting bar 161A is moved, on the right side of the boundary setting bar 161A, between the boundary setting bar 161A and the image L4 that is a non-landscape image by default, the image L3 (cluster) that is a landscape image by default is set. F is a representative sample image, which is an image representative of the third section). The fact that the user has moved the boundary setting bar 161A between the images L3 and L4 between the images L2 and L3 means that the learning samples belonging to the clusters F and G belonging to the third section shown in FIG. It can be assumed that the user thinks that the image is not a landscape image but a non-landscape image.

  FIG. 22A is an explanatory diagram of the data group after the change. FIG. 22B is an explanatory diagram of the boundary after the change. Hereinafter, the processing of the printer-side controller 20 after the setting change will be described with reference to these drawings.

  First, the printer-side controller 20 changes the learning sample attribute information belonging to the clusters F and G from P to N. For example, if sample number 3 in FIG. 20A belongs to cluster F or cluster G, the attribute information is changed from P to N as shown in FIG. 22A.

  In the present embodiment, not only the attribute information of the representative sample of the cluster F is changed, but also all the attribute information of the learning samples belonging to the cluster F are changed. Thereby, the attribute information of the learning sample having properties similar to the image that the user desires not to belong to the landscape can be collectively changed by a single operation by the user.

  In the present embodiment, not only the attribute information of the representative sample of the cluster F is changed, but also all the attribute information of the learning samples belonging to the third section are changed. Thereby, it is possible to collectively change the attribute information of the learning sample that is separated from the boundary as much as the image that the user desires not to belong to the landscape by a single operation by the user.

  Next, the printer-side controller 20 performs relearning of the support vector machine based on the entire feature amount and the changed attribute information, and changes the boundary as shown in FIG. 22B. In other words, the printer-side controller 20 performs relearning based on the overall feature amount and the changed attribute information, and changes the weighting coefficient w in FIG. 20A as shown in FIG. 22B. Here, the changed boundary is expressed as f ′ (x) = 0, and the changed weighting coefficient is expressed as w ′. Note that the relearning calculation process is the same as the normal support vector machine learning calculation process, and a description of the relearning will be omitted.

  The weighting factor w (or w ′) becomes zero if it does not contribute to the determination of the boundary. For this reason, the weighting factor w, which was zero in FIG. 20A, may have a value other than zero due to a change. Conversely, the weighting factor w, which had a value other than zero in FIG. 20A, may become zero due to a change. This is why even the learning sample data that does not contribute to the determination of the boundary in the default setting is stored in the data group of FIG. 20A.

  When the landscape discriminator 51L determines whether or not the classification target image belongs to a landscape scene, the landscape discriminator 51L includes the entire feature amount of the learning sample of the data group in FIG. 22B and the changed weight coefficient w ′. Based on the above, the value of the discriminant of the above formula 1 (the value of the discriminant f ′ (x) after the change) is calculated. Note that the learning classifier 51L calculates the value of the discriminant of Equation 1 described above, excluding learning samples whose weighting coefficient w ′ is zero. As a result, the calculation speed is faster than the case of obtaining the discriminant value using all the learning samples.

  By using the discriminant f ′ (x) after the change, it is possible to perform an identification process reflecting the user's preference. For example, if the image L3 (see FIG. 19) is a building image, it is difficult to determine that the identification target image belongs to a landscape scene when the identification target image is a building image. In other words, if the cluster F (see FIG. 22B) is composed of building image learning samples, when the identification target image is a building image, it is determined that the identification target image belongs to a landscape scene. It becomes difficult to be done.

  In the present embodiment, it is possible to easily change settings according to user preferences. If the learning sample images are displayed one by one and the user decides whether the displayed learning sample images are landscapes one by one, the user needs to repeat the determination work many times. It is inconvenient because there is.

  In the above description, the case where the user moves the boundary setting bar 161A to the left has been described. On the other hand, if the user moves the boundary setting bar 161A one place to the right, on the left side of the boundary setting bar 161A, between the boundary setting bar 161A and the image L3 that is a landscape image in the default setting. In the default setting, a non-landscape image L4 (a representative sample image of the cluster H and an image representative of the fourth section) is located. In such a case, the printer-side controller 20 changes the attribute information of the learning sample belonging to the clusters I, H, and J belonging to the fourth section from N to P, and the overall feature amount and the changed attribute information Re-learn the support vector machine based on, and change the boundary. Also in this case, identification processing reflecting user preferences can be performed.

=== Second Embodiment ===
In the first embodiment described above, the support vector machine is relearned when the user changes the position of the boundary setting bar 161A in FIG. In such a form, the printer-side controller 20 needs to execute a program for performing learning processing, and also requires relearning processing time. Thus, in the second embodiment, a plurality of discriminants are prepared in advance, and the discriminant is selected according to the position of the boundary setting bar 161A, so that the printer-side controller 20 does not have to re-learn. I am trying. The discriminant corresponds to an evaluation function for evaluating the identification target image.

  FIG. 23 shows a data group of learning samples according to the second embodiment. Compared to the data group of the first embodiment (see FIG. 20A), in the second embodiment, a plurality of weighting factors w are stored for each learning sample. Here, four types of first weight coefficient to fourth weight coefficient are stored. In other words, four types of discriminants are stored in the memory 23. Note that the attribute information may not be stored in the second embodiment as compared with the data group of the first embodiment (see FIG. 20A). This is because re-learning is unnecessary in the second embodiment.

  Each of the four types of weighting factors is associated with the position of the boundary setting bar 161A. The first weighting coefficient is associated between the image L1 and the image L2 in FIG. Similarly, the second weight coefficient, the third weight coefficient, and the fourth weight coefficient are respectively between the image L2 and the image L3 in FIG. 19, between the image L3 and the image L4, and between the image L4 and the image L5. Is associated with.

  Before the user changes the setting, the printer-side controller 20 (landscape discriminator 51L) calculates the value of the discriminant of the above formula 1 based on the third weighting coefficient and the overall feature amount. In other words, in the default setting, a discriminant using the third weighting coefficient is selected, and the landscape discriminator 51L performs the discrimination process based on this discriminant. Note that the discriminant value is calculated by excluding learning samples with a weighting factor w of zero in order to improve the calculation speed. The third weighting factor is the same value as the default weighting factor w (see FIG. 20A) of the first embodiment.

  When the user moves the boundary setting bar 161A one place to the left and sets it between the image L2 and the image L3, the landscape discriminator 51L is based on the second weighting factor and the overall feature amount, and the above-described equation 1 The value of the discriminant is calculated. In other words, when the user moves the boundary setting bar 161A one place to the left and sets it between the images L2 and L3, a discriminant using the second weighting coefficient is selected, and the discriminant uses this discriminant to determine the landscape classifier. 51L performs the identification process. Note that the discriminant value is calculated by excluding learning samples with a weighting factor w of zero in order to improve the calculation speed. The second weighting factor is the same as the weighting factor w ′ (see FIG. 22B) obtained by relearning in the first embodiment.

  In the above description, the landscape learning sample data group has been described, but the same data group is stored in the memory for other scenes.

  In the second embodiment, as in the first embodiment, identification processing reflecting user preferences can be performed. Furthermore, in the second embodiment, it is not necessary to perform relearning, and therefore it is not necessary to execute a program for performing learning processing by a support vector machine. In addition, since it is not necessary to perform re-learning, the processing load of the printer-side controller 20 after setting is also reduced.

=== Third Embodiment ===
In the first embodiment and the second embodiment described above, the content of the discriminant is changed according to the user's setting. In the third embodiment, by changing the affirmative threshold, the identification process reflecting the user's preference is performed without changing the content of the discriminant.

  Here, the process after the user moves the boundary setting bar 161A (see FIG. 19) one place to the left and sets it between the images L2 and L3 will be described.

  First, the printer-side controller 20 selects a representative sample of the cluster D that is positioned closest to the third section in the second section (see FIG. 21B) and a representative sample of the cluster F that is positioned closest to the second section in the third section. Extract. In other words, the printer-side controller 20 extracts representative samples D and F located near the boundary between the second section and the third section from each of the second section and the third section.

  Next, the printer-side controller 20 calculates the value of the discriminant f (x) using the entire feature amount of the representative sample D. Further, the printer-side controller 20 calculates the value of the discriminant f (x) using the entire feature amount of the representative sample F.

  Next, the printer-side controller 20 calculates the average value of the discriminant value of the representative sample D and the discriminant value of the representative sample F. Then, the printer-side controller 20 changes this average value to the positive threshold value of the landscape discriminator 51L. Thereby, the setting change of the landscape classifier 51L is completed.

  When the landscape discriminator 51L determines whether or not the classification target image belongs to a landscape scene, the landscape discriminator 51L calculates a discriminant value based on the entire feature amount of the classification target image. If the value of the discriminant is greater than the positive threshold after the change (YES in S204 in FIG. 7), the landscape classifier 51L determines that the classification target image belongs to a landscape scene and sets an affirmative flag (S205). .

  In the third embodiment, as in the first embodiment, identification processing reflecting user preferences can be performed. For example, when an image having a property similar to the learning sample belonging to the cluster F is an identification target image, it is determined that the identification target image does not belong to a landscape scene.

  Note that a plurality of positive thresholds for the landscape discriminator 51L may be prepared in advance, and the positive threshold may be selected from a plurality of positive thresholds according to the position of the boundary setting bar 161A. Thereby, the load of the arithmetic processing for calculating | requiring a positive threshold can be reduced.

=== Fourth Embodiment ===
In the fourth embodiment, the discriminant is also changed, and the positive threshold is also changed. Also here, a process after the user moves the boundary setting bar 161A one place to the left and sets it between the image L2 and the image L3 will be described.

  First, the printer-side controller 20 changes the learning sample attribute information belonging to the clusters F and G from P to N. This process is the same as in the first embodiment.

  Next, the printer-side controller 20 re-learns the support vector machine based on the entire feature amount and the changed attribute information, and changes the boundary (changes the discriminant). This process is also the same as in the first embodiment.

  Next, the printer-side controller 20 uses the evaluation sample (however, the attribute information of the evaluation sample is changed according to the setting of the user boundary setting bar 161A), and the discriminant f after the change is made. A Precision graph (see FIG. 9) of the identification result by '(x) is created. Then, an affirmative threshold value is specified such that the recreated Precision is 97.5%. Thereby, the setting change of the landscape classifier 51L is completed.

  When the landscape discriminator 51L determines whether or not the classification target image belongs to a landscape scene, the landscape discriminator 51L calculates a discriminant value based on the entire feature amount of the classification target image. If the value of the discriminant is greater than the positive threshold after the change (YES in S204 in FIG. 7), the landscape classifier 51L determines that the classification target image belongs to a landscape scene and sets an affirmative flag (S205). .

  Also in the fourth embodiment, as in the first embodiment, identification processing reflecting user preferences can be performed.

=== Other Embodiments ===
Although a printer or the like as one embodiment has been described, the above embodiment is for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and it is needless to say that the present invention includes equivalents thereof. In particular, the embodiments described below are also included in the present invention.

<About the printer>
In the above-described embodiment, the printer 4 performs the scene identification process, but the digital still camera 2 may perform the scene identification process. Further, the image identification device that performs the above-described scene identification processing is not limited to the printer 4 or the digital still camera 2. For example, an image identification device such as a photo storage that stores a large amount of image files may perform the scene identification process described above. Of course, a personal computer or a server installed on the Internet may perform the scene identification process.
Note that a program that causes the scene identification device to execute the above-described scene identification processing is also within the scope of the present invention.

<About Support Vector Machine>
For the above-described sub classifier 51 and sub partial classifier 61, a classification method using a support vector machine (SVM) is used. However, the method for identifying whether or not the identification target image belongs to a specific scene is not limited to using a support vector machine. For example, pattern recognition such as a neural network may be employed.

<About scene identification>
In the above-described embodiment, the sub classifier 51 and the sub partial classifier 61 identify whether or not the image indicated by the image data belongs to a specific scene. However, the present invention is not limited to scene identification, and may identify whether it belongs to some class. For example, you may identify whether the image which image data shows is a specific pattern shape.

It is explanatory drawing of an image processing system. 2 is an explanatory diagram of a configuration of a printer. FIG. It is explanatory drawing of the automatic correction function of a printer. It is explanatory drawing of the relationship between the scene of an image, and the correction content. It is a flowchart of the scene identification process by a scene identification part. It is explanatory drawing of the function of a scene identification part. It is a flowchart of a whole identification process. It is explanatory drawing of an identification object table. It is explanatory drawing of the affirmation threshold value of the whole identification process. It is explanatory drawing of Recall and Precision. It is explanatory drawing of a 1st negative threshold value. It is explanatory drawing of a 2nd negative threshold value. FIG. 13A is an explanatory diagram of a threshold table. FIG. 13B is an explanatory diagram of threshold values in the landscape classifier. FIG. 13C is an explanatory diagram of an outline of the process of the landscape classifier. It is a flowchart of a partial identification process. It is explanatory drawing of the order of the partial image which an evening scene partial identifier selects. It is a Recall and Precision graph when the evening scene image is identified only by the top 10 partial images. FIG. 17A is an explanatory diagram of determination by the linear support vector machine. FIG. 17B is an explanatory diagram of discrimination using a kernel function. It is a flowchart of an integrated identification process. It is explanatory drawing of a setting screen. FIG. 20A shows a data group of learning samples stored in the memory 23. FIG. 20B is an explanatory diagram of the distribution of each learning sample. FIG. 21A is an explanatory diagram showing a state in which the representative sample is projected onto the normal line of the boundary (f (x) = 0). FIG. 21B is an explanatory diagram of a representative sample projected on the normal line. FIG. 21B is an explanatory diagram of a representative sample projected on the normal line. FIG. 22A is an explanatory diagram of the data group after the change. FIG. 22B is an explanatory diagram of the boundary after the change. It is a data group of the sample for learning of 2nd Embodiment.

Explanation of symbols

2 Digital still camera, 2A mode setting dial,
4 Printer, 6 Memory card,
10 printing mechanism, 11 head, 12 head control unit, 13 motor,
14 sensors, 15 panel section, 16 display section, 17 input section,
20 printer-side controller, 21 slots, 22 CPU, 23 memory,
24 control unit, 25 drive signal generator,
31 storage unit, 31A image storage unit, 31B result storage unit, 32 face identification unit,
33 scene identification unit, 34 image correction unit, 35 printer control unit,
40 feature quantity acquisition unit, 50 overall classifier, 51 sub classifier,
51L landscape classifier, 51S evening scene classifier, 51N night scene classifier,
51F Flower identifier, 51R Autumn leaves identifier,
60 partial classifiers, 61 sub partial classifiers, 61S evening scene partial classifiers,
61F Flower partial classifier, 61R Autumn colored partial classifier,
70 Integrated identifier, 161 setting screen, 161A Boundary setting bar

Claims (6)

The comparator compares the value with a predetermined threshold of the evaluation function for evaluating the identification target based on the comparison result, a setting method of identifying identification process whether the identification target to a class belongs,
An extraction step of extracting a predetermined number of samples from a plurality of samples by extracting samples belonging to the class according to the threshold and samples not belonging to the class;
The predetermined number of extracted samples are arranged and displayed on the display unit in the order of the value of the evaluation function, and a mark is displayed between the sample belonging to the class and the sample not belonging to the class, and a user instruction Displaying the mark between another sample and the sample by moving the position of the mark according to
A setting change step of changing the setting of the identification process in accordance with the position of the mark determined by the user. The setting method according to claim 1 ,
The identification process is to identify whether the identification object belongs to the class based on a hyperplane separating a space,
In the extracting step, the sample is projected onto the normal of the hyperplane, and the sample to be extracted is determined based on the position of the sample projected on the normal. The setting method according to claim 1 or 2 ,
Prior Symbol setting change step, setting method characterized by changing the evaluation function. It is the setting method in any one of Claims 1-3 ,
Prior Symbol setting change step, setting method characterized by changing the threshold value. The comparator compares the value with a predetermined threshold of the evaluation function for evaluating the identification target based on the comparison result, an identification method that performs an identification process of identifying whether the identification target to a class belongs,
An extraction step of extracting a predetermined number of samples from a plurality of samples by extracting samples belonging to the class according to the threshold and samples not belonging to the class;
The predetermined number of extracted samples are arranged and displayed on the display unit in the order of the value of the evaluation function, and a mark is displayed between the sample belonging to the class and the sample not belonging to the class, and a user instruction Displaying the mark between another sample and the sample by moving the position of the mark according to
A setting change step for changing the setting of the identification process according to the position of the mark determined by the user;
An identification step for identifying whether or not the identification object belongs to the class by the identification process whose setting has been changed. The comparator compares the value with a predetermined threshold of the evaluation function for evaluating the identification target based on the comparison result, the identification apparatus performs identification processing for identifying whether the identification target to a class belongs,
By extracting samples belonging to the class according to the threshold and samples not belonging to the class, a predetermined number of samples are extracted from a plurality of samples ,
The predetermined number of the extracted samples are displayed on the display unit in order of the value of the evaluation function, and at the same time, a mark is displayed between the sample belonging to the class and the sample not belonging to the class. The mark is displayed between another sample by moving the position of the mark according to
A program for changing the setting of the identification process in accordance with the position of the mark determined by the user.

Images