JP2013050837A - Classification processing generation device and classification processing generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate effective classification processing and also enable an operator to easily perform interpretation, modification, or the like of classification processing by displaying contents of the classification processing.SOLUTION: An evaluation unit 22 provides classification processing candidates including predetermined image processing calculation and feature quantity calculation, calculates a plurality of feature quantities of a plurality of original images by performing the image processing calculation and the feature quantity calculation on each original image belonging to any image group, and calculates effectiveness of the plurality of feature quantities in classifying the plurality of original images. A calculation content change unit 23 repeats processing for changing contents of the classification processing candidates while calculating the effectiveness by the evaluation unit 22 until satisfying termination conditions. Then, a display control unit 24 displays on a display 16 contents of classification processing of the classification processing candidate of which effectiveness is equal to or more than a threshold. Thus, it is possible to generate effective classification processing and enable an operator to easily perform interpretation, modification, or the like of classification processing by displaying contents of the classification processing.

Description

  The present invention relates to a technique for generating a classification process for classifying an image into any of a plurality of image groups.

  Traditionally, in the field of research on art history, etc., art works such as paintings have been analyzed and grouped in terms of the age, place where they were drawn, or the contents drawn. It has been broken. Non-Patent Document 1 discloses a technique for quantitatively analyzing a picture by statistical analysis of color information.

  In Patent Document 1, a feature amount for each predetermined class is detected from a person area in an image, an effective value is obtained for each feature amount, and a feature amount to be used in personal recognition processing is selected based on the effective value. A technique is disclosed. Patent Document 2 discloses a technique for realizing efficient search. In this technique, the amount of refinement when a search is performed by specifying the feature item for all combinations of a plurality of elements and feature items. The degree of variation is determined, and the smaller the variation, the higher the narrowing effect, and the effectiveness for narrowing down the feature items is determined based on the degree of variation, and the feature items are ranked. Then, the searcher is notified so that the feature amount is input according to the order.

  Further, Patent Document 3 discloses a person search device capable of grasping in advance the effectiveness of search, and in this device, a feature amount indicating a feature amount of a person in a captured image captured by a plurality of cameras. Information is acquired. Then, the difference between the feature quantity of the first person imaged by the first camera and the feature quantity of the second person imaged by the second camera is calculated for each feature of the person. Based on this, a narrowing-down effectiveness indicating the effectiveness when searching for the first person is calculated for each feature of the person. Patent Documents 4 and 5 disclose techniques for identifying significant features in a support vector machine.

JP 2006-293644 A JP 2009-20578 A JP 2009-42879 A Special table 2008-524675 Special table 2008-129657 gazette

Kurama, “Color Analysis of Late Impressionist Paintings” [online], March 2008, University of Tsukuba, [August 8, 2011 search], Internet <URL: http: // www. cs.tsukuba.ac.jp/H19Syuron/200621005.pdf>

  By the way, in order to automatically classify a large number of works, it is necessary to generate a classification process according to the criteria for grouping. When grouping works such as paintings according to clear criteria such as “age” and “object”, the classification process can be generated relatively easily, but the subjectivity called “style” or “touch” When grouping according to a judgment standard based on a general impression, it is difficult to clearly define or quantify a judgment standard in classification, and it is difficult to generate a classification process. In particular, it is extremely difficult to generate a classification process for an abstract image because its interpretation itself depends on the observer. Due to these problems, it is not possible to conduct a deep study of art works, which hinders understanding of art.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for generating an effective classification process and enabling an operator to easily interpret and correct the classification process. .

  The invention according to claim 1 is a classification processing generation device that generates a classification process for classifying an image into any of a plurality of image groups, and includes a classification processing candidate including a predetermined image processing calculation and a feature amount calculation. Is prepared, the image processing operation is performed on each original image belonging to each of a plurality of image groups using the classification processing candidates to generate a processed image, and the feature amount is calculated for the processed image Evaluation for obtaining a plurality of feature amounts of a plurality of original images by performing an operation, and determining the effectiveness of the plurality of feature amounts in classifying each of the plurality of original images into any of the plurality of image groups. And a process for determining the effectiveness in the evaluation unit while changing the content of at least one of the image processing calculation and the feature amount calculation in the classification processing candidate, and an end condition A calculation content changing unit that repeats until it is satisfied, and a classification process that can use the classification process candidate when the effectiveness is equal to or greater than a threshold when the end condition is satisfied, A display control unit for displaying the contents on the display unit.

  The invention according to claim 2 is the classification processing generation apparatus according to claim 1, wherein the display control unit displays contents of both the image processing calculation and the feature amount calculation in the classification processing in the display unit. To display.

  The invention according to claim 3 is the classification processing generation device according to claim 1 or 2, wherein the display control unit is configured to perform the original image and the image in the classification processing with respect to the original image. The processed image generated by performing the processing operation is simultaneously displayed on the display unit.

  The invention according to claim 4 is the classification processing generation apparatus according to claim 1 or 2, wherein the display control unit corresponds to each of the plurality of image groups and performs the image processing calculation in the classification processing. A plurality of generated processed images are simultaneously displayed on the display unit.

  The invention according to claim 5 is the classification processing generation device according to any one of claims 1 to 4, wherein a plurality of classification processing candidates are prepared in the evaluation unit, and the evaluation unit includes the plurality of classification processing candidates. A plurality of validity levels respectively corresponding to the classification process candidates are obtained, and the calculation content changing unit changes the contents of the at least one calculation in at least one of the plurality of classification processing candidates while the plurality of evaluation levels are changed by the evaluation unit. The process of obtaining the effectiveness of is repeated until the end condition is satisfied.

  The invention according to claim 6 is a classification processing generation method for generating a classification processing for classifying an image into any of a plurality of image groups, and a) a classification including a predetermined image processing calculation and a feature amount calculation. Preparing a processing candidate; b) generating a processed image by performing the image processing operation on each original image belonging to each of a plurality of image groups; and c) performing the processing on the processed image. A step of obtaining a plurality of feature amounts of the plurality of original images by performing a feature amount calculation; and d) the plurality of feature amounts when each of the plurality of original images is classified into any of the plurality of image groups. And e) steps b) to d) while changing the content of at least one of the image processing calculation and the feature amount calculation in the classification processing candidate. And f) a step of displaying the content of at least one operation in the classification process on a display unit as a classification process that can use the classification process candidate when the effectiveness is equal to or greater than a threshold. Prepare.

  The invention according to claim 7 is the classification processing generation method according to claim 6, wherein in the step f), the contents of both the image processing calculation and the feature amount calculation in the classification processing are the display unit. Is displayed.

  The invention according to claim 8 is the classification processing generation method according to claim 6 or 7, wherein, in the step f), the original images and the images in the classification processing for the original images. The processed image generated by performing the processing operation is simultaneously displayed on the display unit.

  The invention according to claim 9 is the classification processing generation method according to claim 6 or 7, wherein in the step f), each of the plurality of image groups corresponds to the image processing operation in the classification processing. The plurality of generated processed images are simultaneously displayed on the display unit.

  A tenth aspect of the present invention is the classification processing generation method according to any one of the sixth to ninth aspects, wherein a plurality of classification processing candidates are prepared in the step a), and the steps b) to d). Thus, a plurality of validity levels respectively corresponding to the plurality of classification processing candidates are obtained, and in the step e), the content of the at least one operation in at least one of the plurality of classification processing candidates is changed. Steps d) to d) are repeated until the end condition is satisfied.

  According to the present invention, an effective classification process is generated and the contents of the classification process are substantially displayed, so that an operator can easily interpret and correct the classification process.

It is a figure which shows the structure of a computer. It is a block diagram which shows the function structure which a computer implement | achieves. It is a figure which shows the flow of the operation | movement which a computer produces | generates a classification process. It is a figure which shows the classification process candidate contained in a group. It is a figure which shows an original image. It is a figure which shows the result of the image processing calculation with respect to an original image, and the feature-value calculation. It is a figure which shows the result of the image processing calculation with respect to an original image, and the feature-value calculation. It is a figure which shows the result of the image processing calculation with respect to an original image, and the feature-value calculation. It is a figure for demonstrating the update of a group. It is a figure for demonstrating the update of a group. It is a figure for demonstrating the update of a group. It is a figure which shows the classification process candidate contained in the group after an update. It is a figure which shows the result of the image processing calculation with respect to an original image, and the feature-value calculation. It is a figure which shows the result of the image processing calculation with respect to an original image, and the feature-value calculation. It is a figure which shows the window displayed on a display. It is a figure which shows the flow of the operation | movement which a computer classifies | categorizes a target image. It is a figure which abstractly shows the module which concerns on an image processing calculation.

  FIG. 1 is a diagram showing a configuration of a computer 1 that realizes a classification processing generation apparatus according to an embodiment of the present invention. The computer 1 generates a classification process for classifying an image such as a painting into any of a plurality of image groups. As will be described later, the classification process includes an image processing calculation and a feature amount calculation, and the classification process enables classification of images to be classified.

  The computer 1 has a general computer system configuration in which a CPU 11 that performs various arithmetic processes, a ROM 12 that stores basic programs, and a RAM 13 that stores various information are connected to a bus line. The bus line further includes a fixed disk 15 that stores information, a display 16 that is a display unit that displays various information, a keyboard 17a and a mouse 17b that accept input from an operator, an optical disk, a magnetic disk, a magneto-optical disk, and a flash. A reading / writing device 18 that reads information from or writes information to a computer-readable recording medium 81 such as a memory, and a communication unit 19 that communicates with the outside appropriately include an interface ( I / F) or the like.

  The computer 1 reads the program 810 from the recording medium 81 in advance via the reading / writing device 18 and stores it in the fixed disk 15. Then, when the program 810 is copied to the RAM 13 and the CPU 11 executes arithmetic processing according to the program in the RAM 13 (that is, when the computer executes the program), the computer 1 performs processing as the classification processing generation device. Do.

  FIG. 2 is a block diagram illustrating a functional configuration realized by the CPU 11, the ROM 12, the RAM 13, the fixed disk 15, and the like when the CPU 11 operates according to the program 810. In FIG. 2, an image input unit 21, an evaluation unit 22, an operation content change unit 23, and a display control unit 24 are functions realized by the CPU 11 and the like. Note that these functions may be realized by a dedicated electric circuit, or an electric circuit may be partially used.

FIG. 3 is a diagram illustrating a flow of operations in which the computer 1 generates a classification process. First, the evaluation unit 22 of FIG. 2 prepares a classification process candidate including a predetermined image processing calculation and feature amount calculation (step S11). Here, in the present embodiment, a classification process that can be used for image classification, which will be described later, is generated by a genetic algorithm. In the process of step S11, three classification process candidates are included in the initial population. Prepared as a gene. A gene is expressed as a sequence G _i of processing in which an arbitrary image processing operation P _i is applied a plurality of times and a feature amount operation C _i is applied last, as shown in Equation 1.

The image processing operations used in this embodiment are “gray value binarization”, “R component value binarization”, “G component value binarization”, “B component value binarization” shown in the left column of Table 1. “Value”, “shrink”, “expansion” and “smoothing”. In “gray value binarization”, a new pixel value 255 is assigned to a pixel having a gray value equal to or greater than “binarization threshold Th” in the right column of Table 1, and a pixel value 0 is assigned to the remaining pixels. Is done. “R component value binarization”, “G component value binarization” and “B component value binarization” are respectively the R (red) component pixel value, the G (green) component pixel value and the B (blue component). ) Same as “gray value binarization” except that the pixel value of the component is compared with the threshold value. The “binarization threshold Th” in “gray value binarization” is 20, and “binary” in “R component value binarization”, “G component value binarization”, and “B component value binarization” The threshold value Th ”is 50. In each of “shrinkage”, “expansion”, and “smoothing”, the processing is performed using the filter of “size Sz” in the right column of Table 1, and “size Sz” is 5 in the present embodiment. . In the image processing operation of the "gray value binarization", the _{I R} pixel value of the R component, a pixel value of the _{I G} G component, the _{I B} as a pixel value of the B component, the gray value (0.2989 × determined by _{I R + 0.5870 × I G +} 0.1140 × I B).

  In addition, the feature amount calculation used in the present embodiment is an operation for obtaining “number of regions” shown in the left column of Table 2, an operation for obtaining “total area of regions”, and an operation for obtaining “number of circular regions”. is there. In each calculation of “number of regions”, “total area of regions”, and “number of circular regions”, a pixel value that is equal to or greater than “pixel value threshold v” (255 in the present embodiment) in the right column of Table 2 is used. A set of pixels that are continuous with each other is treated as one region. Then, the number of regions is obtained in the calculation of “number of regions”, and the total area of all regions is obtained in the calculation of “total area of regions”. Further, in the calculation of “the number of circular regions”, among all the regions, the circularity is equal to or more than “roundness threshold w” (0.9 in the present embodiment) in the right column of Table 2. Is required. In addition, a region area threshold value may be further added, and “region number”, “total area of region”, and “circular region number” may be obtained only for regions having an area equal to or larger than the region area threshold value.

In the following processing, a gene is expressed as a sequence G _i of processing in which the image processing operation P is applied twice and the feature amount operation C is applied last, that is, the gene is (G _i = P _i1 × P _i2 × C _i ). Further, for convenience of explanation, the first image processing calculation (hereinafter referred to as “preceding image processing calculation”) P _i1 is “gray value binarization”, “R component value binarization”, “G component value binary”. And “B component value binarization” (hereinafter collectively referred to as “binarization calculation”), and the second image processing calculation (hereinafter referred to as “subsequent image processing calculation”). P _i2 is limited to any one of “shrinkage”, “expansion”, and “smoothing” (hereinafter collectively referred to as “filter operation”). Of course, each gene may contain one or more image processing operations.

Specifically, the three genes included in the first generation (early generation) population are (G ₁ = P ₁₁ × P ₁₂ × C ₁ ) and (G ₂ = P ₂₁ × P ₂₂ × C ₂ ), respectively. , (G ₃ = P ₃₁ × P ₃₂ × C ₃ ). As shown in FIG. 4, the preceding image processing operation P ₁₁ in the gene G ₁ is “gray value binarization”, the subsequent image processing operation P ₁₂ is “shrinkage”, and the feature amount operation C ₁ is “circular region”. It is the calculation of “number”. The preceding image processing operation P ₂₁ in the gene G ₂ is “R component value binarization”, the subsequent image processing operation P ₂₂ is “expansion”, and the feature amount operation C ₂ is an operation of “total area of region”. is there. The preceding image processing operation P ₃₁ in the gene G ₃ is “G component value binarization”, the subsequent image processing operation P ₃₂ is “smoothing”, and the feature amount operation C ₃ is an operation of “number of regions”. . As will be described later, since the process of updating the population (generation) is performed as necessary, the genes G ₁ , G ₂ , and G ₃ represent the population after the Nth update (N is an integer of 1 or more). It may be configured.

  On the other hand, in the generation of the classification process, a plurality of images (hereinafter referred to as “original images”) belonging to each of a plurality of image groups are subjectively selected and prepared by the operator. The original image in the present embodiment is a color image in which each pixel has respective pixel values of R component, G component, and B component (in this embodiment, any value from 0 to 255).

  FIG. 5 is a diagram showing original images belonging to each image group. As shown in FIG. 5, the original images 31 and 41 belong to the image group A, and the original images 51 and 61 belong to the image group B. The original image 31 includes four objects (graphics) 311 to 314, and the rectangular objects 311 and 313 have R component pixel values of 255, and G component and B component pixel values are both zero ( Hereinafter, it is simply a set of “R pixels”. The elliptical or circular objects 312 and 314 have a B component pixel value of 255, and a set of pixels in which the R component and G component pixel values are both 0 (hereinafter, simply referred to as “B pixel”). It is. In FIG. 5, the same hatching is given to the object of the same color (the same applies to FIGS. 6 to 8, FIG. 13, and FIG. 14 described later).

  The original image 41 includes five objects 411 to 415, rectangular objects 411, 413, and 414 are a set of R pixels, and elliptical or circular objects 412 and 415 are a set of B pixels. The original image 51 includes four objects 511 to 514, and elliptical or circular objects 511 and 512 are a set of R pixels, and rectangular objects 513 and 514 are a set of B pixels. The original image 61 includes five objects 611 to 615. Ellipse or circular objects 611, 612, and 615 are a set of R pixels, and rectangular objects 613 and 614 are a set of B pixels. The background area (that is, the area other than the object) in the original images 31, 41, 51, 61 is a set of pixels in which the pixel values of the R component, the G component, and the B component are all 10.

  In the original images 31, 41, 51, 61 of FIG. 5, the object 311 of the original image 31 has the same shape and size as the object 411 of the original image 41, the object 513 of the original image 51, and the object 614 of the original image 61. (However, only the object 513 has a different direction.) The area is 300. The object 312 of the original image 31 has the same shape and size as the object 412 of the original image 41, the object 511 of the original image 51, and the objects 611 and 612 of the original image 61, and has an area of 600. The object 313 of the original image 31 has the same shape and size as the objects 413 and 414 of the original image 41, the object 514 of the original image 51, and the object 613 of the original image 61 (however, only the object 414 has a different direction). , The area is 600. The object 314 of the original image 31 has the same shape and size as the object 415 of the original image 41, the object 512 of the original image 51, and the object 615 of the original image 61, and has an area of 300. As described above, the original images 31, 41, 51, 61 are composed of four types of objects, and the number, arrangement, angle, and color of the objects are different in each image. Note that the gray value of an object composed of R pixels is 76, the gray value of an object composed of B pixels is 29, and the gray value of the background region is 10.

The original images 31, 41, 51, 61 are input to the evaluation unit 22 by the image input unit 21, and the image processing calculation unit 221 applies each of the current group to each original image 31, 41, 51, 61. The preceding image processing calculation and the subsequent image processing calculation included in the gene are performed in order (step S12). Specifically, for the gene G ₁ represented by (P ₁₁ × P ₁₂ × C ₁ ), the “grey value binarization” that is the preceding image processing operation P ₁₁ is the leftmost column in FIG. The intermediate images 32a, 42a, 52a, and 62a shown in the second column from the left in FIG. 6 are performed for each of the original images 31, 41, 51, and 61 shown in (the column labeled “original image” in the upper row). Is generated. Also, a subsequent image processing operations _{P 12} "contraction" is, the intermediate image 32a, 42a, 52a, made to each 62a, processed image 33a shown in the third column from the left in FIG. 6, 43a, 53a and 63a are generated.

For the gene G ₂ expressed by (P ₂₁ × P ₂₂ × C ₂ ), “R component value binarization” which is the preceding image processing operation P ₂₁ is shown in the leftmost column of FIG. This is performed for each of the original images 31, 41, 51, 61, and intermediate images 32b, 42b, 52b, 62b shown in the second column from the left in FIG. 7 are generated. Also, a subsequent image processing operations _{P 22} "expansion" is, the intermediate image 32 b, 42b, 52 b, made to each 62b, the processed image 33b shown in the third column from the left in FIG. 7, 43 b, 53b and 63b are generated.

Further, for the gene G ₃ expressed by (P ₃₁ × P ₃₂ × C ₃ ), “G component value binarization” which is the preceding image processing operation P ₃₁ is shown in the leftmost column of FIG. This is performed for each of the original images 31, 41, 51, 61, and intermediate images 32c, 42c, 52c, 62c shown in the second column from the left in FIG. 8 are generated. Further, “smoothing” which is the subsequent image processing calculation P ₃₂ is performed on each of the intermediate images 32c, 42c, 52c, and 62c, and processed images 33c and 43c shown in the third column from the left in FIG. , 53c, 63c are generated.

Subsequently, in the feature amount calculation unit 222, the feature amount calculation included in each gene of the group is performed on the processed image corresponding to the gene (step S13). For the gene G ₁ , the calculation of “number of circular areas”, which is the feature amount calculation C ₁ , is performed on each of the processed images 33 a, 43 a, 53 a, 63 a shown in the third column from the left in FIG. As shown in the rightmost column in FIG. 6, the “number of circular areas” in the processed images 33 a, 43 a, 53 a, 63 a is obtained as the feature amount F for the original images 31, 41, 51, 61.

For the gene G ₂ , the calculation of “total area of region”, which is the feature amount calculation C ₂ , is performed on each of the processed images 33 b, 43 b, 53 b, 63 b shown in the third column from the left in FIG. As shown in the rightmost column of FIG. 7, the “total area of the region” in the processed images 33 b, 43 b, 53 b, 63 b is obtained as the feature amount F for the original images 31, 41, 51, 61. For the gene G ₃ , the “number of regions” that is the feature amount calculation C ₃ is performed on each of the processed images 33 c, 43 c, 53 c, and 63 c shown in the third column from the left in FIG. As shown in the rightmost column of FIG. 8, the “number of regions” in the processed images 33 c, 43 c, 53 c, and 63 c is obtained as the feature amount F for the original images 31, 41, 51, and 61.

  As described above, when a plurality of feature amounts for the plurality of original images 31, 41, 51, 61 are obtained for each gene (classification processing candidate), the effectiveness calculation unit 223 depends on the feature amount for each gene. Using variables and image groups as factors, the significance probability (p value) in the one-way analysis of variance is obtained as shown in Table 3 (step S14).

As shown in Table 3, significant probability for gene _{G 1} in the population is 0.423, significant probability for gene _{G 2} is a 1.000, significant probability for gene _{G 3} are calculated impossible (the significance probability formula The denominator at 0 is 0). Here, it is considered that there is a significant difference between the average value of the feature values in the image group A and the average value of the feature values in the image group B as the significance probability is smaller. Therefore, the effectiveness (or evaluation value) of the plurality of feature amounts when each of the plurality of original images 31, 41, 51, 61 is classified into any of the plurality of image groups A and B, that is, the image group. If the effectiveness of each gene in the classification into A and B is defined as an index that becomes high enough to be useful for grouping, for example, the reciprocal of the significance probability can be used as the effectiveness. Thus, the process of step S14 is a process for obtaining the effectiveness of a plurality of feature amounts obtained for each gene.

  In the calculation content changing unit 23, it is confirmed whether or not the group evaluation value is equal to or less than a predetermined evaluation threshold value 0.05 with the minimum significance probability among the significance probabilities for the three genes as the group evaluation value. In the example of Table 3, since the group evaluation value is 0.423, which is larger than the evaluation threshold, the group is updated (that is, the classification process candidate is updated) (step S15). The process in step S15 is regarded as a process for confirming whether or not the end condition is satisfied, with the case where the effectiveness is equal to or greater than a predetermined threshold as the end condition.

In updating the group, first, the gene G ₁ , the gene G _2, and the gene G ₃ included in the current group shown in FIG. 4 are deleted from the group (that is, the gene trap is deleted). (Step S16). Here, the gene significance probability becomes impossible calculation is considered significant probability maximum, gene G ₃ is removed from the population. Subsequently, as shown in FIG. 9, the gene G ₁ having the minimum significance is replicated as the gene G _{1 ′} (that is, the gene is propagated) (step S17). The gene G _{1 ′} is expressed by (P ₁₁ × P ₁₂ × C ₁ ).

When the gene G _{1 ′} is duplicated, as shown in FIG. 10, the subsequent image processing operation P ₂₂ and the feature amount operation C ₂ in the gene G ₂ having the second lowest significance probability, and the subsequent image processing in the gene G _{1 ′} . calculating _{P 12} and the feature and calculating _{C 1} is exchanged (i.e., performs the crossover gene.) (step S18). Thus, a new gene _{G 4} and gene _{G 5} are generated. In addition, one operation is randomly selected from the preceding image processing operation, the subsequent image processing operation, and the feature amount operation of the gene G ₄ and the gene G ₅ generated by the crossover, and is changed to another operation of the same type. (That is, gene mutation is performed) (step S19). In other words, when the preceding image processing operation is selected, it is changed to another binarization operation, when the subsequent image processing operation is selected, it is changed to another filter operation, and the feature amount operation is selected. Is changed to an operation for obtaining another type of feature amount. Here, the changed feature quantity calculating C ₂ in gene G ₄ shown underlined in Figure 11 is from the "total area of the region" to "the number of regions". It may take place mutations in gene G _1.

The update of the population is completed by the above processing, and the gene G ₁ expressed by (P ₁₁ × P ₁₂ × C ₁ ), the gene G ₄ expressed by (P ₄₁ × P ₄₂ × C ₄ ), ( The gene G ₅ expressed by P ₅₁ × P ₅₂ × C ₅ ) is included in the updated group (may be regarded as the (N + 1) th updated group) as a classification processing candidate. As shown in FIG. 12, the preceding image processing calculation P ₁₁ in the gene G ₁ is “gray value binarization”, the subsequent image processing calculation P ₁₂ is “shrinkage”, and the feature amount calculation C ₁ is “circular region”. It is the calculation of “number”. The preceding image processing operation P ₄₁ in the gene G ₄ is “gray value binarization”, the subsequent image processing operation P ₄₂ is “expansion”, and the feature amount operation C ₄ is an operation of “number of regions”. The preceding image processing calculation P ₅₁ in the gene G ₅ is “R component value binarization”, the subsequent image processing calculation P ₅₂ is “shrinkage”, and the feature amount calculation C ₅ is a calculation of “number of circular regions”. .

In the image processing calculation unit 221, a preceding image processing calculation and a subsequent image processing calculation included in each gene of the updated group are sequentially performed on each original image 31, 41, 51, 61 (step S12). Specifically, for the gene G ₄ represented by (P ₄₁ × P ₄₂ × C ₄ ), the “grey value binarization” that is the preceding image processing operation P ₄₁ is the leftmost column in FIG. The intermediate images 32d, 42d, 52d, and 62d shown in the second column from the left in FIG. 13 are generated for the original images 31, 41, 51, and 61 shown in FIG. Also, a subsequent image processing operations _{P 42} "expansion" is, the intermediate image 32d, 42d, 52 d, made to each 62d, processed image 33d shown in the third column from the left in FIG. 13, 43d, 53d and 63d are generated.

For the gene G ₅ represented by (P ₅₁ × P ₅₂ × C ₅ ), the “R component value binarization” that is the preceding image processing operation P ₅₁ is the original image shown in the leftmost column of FIG. The intermediate images 32e, 42e, 52e, 62e shown in the second column from the left in FIG. 14 are generated for each of 31, 41, 51, 61. Further, “shrinkage” that is the subsequent image processing operation P ₅₂ is performed on each of the intermediate images 32e, 42e, 52e, and 62e, and processed images 33e, 43e, and 3e shown in the third column from the left in FIG. 53e and 63e are generated. For the gene G ₁ represented by (P ₁₁ × P ₁₂ × C ₁ ), the processed image 33a, shown in the third column from the left in FIG. 43a, 53a, and 63a are generated.

Subsequently, in the feature amount calculation unit 222, the feature amount calculation included in each gene of the group is performed on the corresponding processed image (step S13). For the gene G ₄ , the “number of regions” that is the feature amount calculation C ₄ is performed on each of the processed images 33 d, 43 d, 53 d, and 63 d shown in the third column from the left in FIG. As shown in the rightmost column of FIG. 13, the “number of regions” in the processed images 33 d, 43 d, 53 d, and 63 d is obtained as the feature amount F for each of the original images 31, 41, 51, and 61.

For the gene G ₅ , the “number of circular areas”, which is the feature amount calculation C ₅ , is performed on each of the processed images 33 e, 43 e, 53 e, 63 e shown in the third column from the left in FIG. As shown in the rightmost column of FIG. 14, the “number of circular areas” in the processed images 33 e, 43 e, 53 e, 63 e is obtained as the feature amount F for each of the original images 31, 41, 51, 61. The gene G _1, similarly to the processing in step S13 in a population before the update, the feature amount F shown in the rightmost column of FIG. 6 is determined. In the effectiveness calculation unit 223, the significance probability (p value) in the one-way analysis of variance is obtained as shown in Table 4 (step S14).

As shown in Table 4, significant probability for gene _{G 1} in the population is 0.423, significant probability for gene _{G 4} are a 0.038, significant probability for gene _{G 5} is 0.038. When it is confirmed that the group evaluation value, which is the minimum value of significance for the three genes, is equal to or lower than the predetermined evaluation threshold value 0.05 and satisfies the end condition (step S15), the calculation content changing unit 23 performs classification. gene _{G 1} is a process candidates, among genes _{G 4} and gene _{G 5,} each gene _{G 4} and gene _{G 5} a significant probability is an evaluation threshold 0.05 or less, the image of the image group a and the image group B Are identified as classification processes that can be used. As described above, the difference between the image group A and the image group B subjectively set by the operator is automatically quantified using the classification process. The contents of the calculation in each classification process are output to the display control unit 24 and displayed on the display 16 (step S20). Further, the contents of the calculation in each classification process are also stored in a storage unit (not shown).

FIG. 15 is a diagram showing the window 7 displayed on the display 16. As shown in FIG. 15 selects one classification process of the top right of the classification process selecting portion 71 Multiple classification processing within the window 7 is selectable gene G ₅ in FIG. 15 as "classification process 05" Has been. In the classification process content display area 72 below the classification process selection unit 71, a sequence of processes in the selected classification process (hereinafter referred to as “selected classification process”), that is, a preceding image processing calculation and a subsequent image processing calculation. The contents of the feature amount calculation (including parameter values such as various threshold values) are displayed in an easily understandable text.

  Two group display areas 73 are arranged in the vertical direction on the left side of the classification processing content display area 72, and a display group selection unit 731 is provided on the upper left of each group display area 73. In each group display area 73, an original image display area 732, an intermediate image display area 733, a processed image display area 734, and a feature amount display area 735 are set. When one image group is selected by the display group selection unit 731, the original image belonging to the image group is displayed in the original image display area 732, and is generated from the original image by the preceding image processing operation of the selection classification process. The intermediate image is displayed in the intermediate image display area 733. Further, a processed image generated from the intermediate image by the subsequent image processing calculation of the selection classification process is displayed in the processed image display area 734, and a feature amount acquired from the processed image by the feature amount calculation of the selection classification process is obtained. It is displayed in the feature amount display area 735. As described above, the window 7 can simultaneously display a plurality of original images, a plurality of intermediate images, a plurality of processed images, and a plurality of feature amounts respectively corresponding to a plurality of image groups. Actually, in the window 7 displayed first, a plurality of group display areas 73 are assigned to different image groups. Note that the name of the immediately preceding image processing operation is also displayed below the intermediate image and the processed image.

  Further, in the window 7 of FIG. 15, it is possible to display other original images belonging to the image group selected by the display group selection unit 731 by scrolling in the original image display area 732. In this case, the intermediate image, processed image, and feature amount corresponding to the original image are displayed in the intermediate image display region 733, the processed image display region 734, and the feature amount display region 735, respectively. In the window 7, a plurality of original images belonging to the same group may be displayed at the same time (the same applies to intermediate images, processed images, and feature values). In addition, the significance probability (or effectiveness) in each classification process may be displayed, and when a plurality of classification processes are acquired as in the above processing example, the rank of the classification process based on the significance probability is displayed. Also good.

By referring to the window 7 in FIG. 15, the worker can quantitatively grasp the effective classification process. For example, a preceding image processing operation P ₄₁ is "gray value binarization", the subsequent image processing operations P ₄₂ is "expansion", the feature calculation C ₄ of the gene G ₄ is an operation of the "region number" The classification process is considered to be for classifying whether or not objects exist densely as a criterion. In addition, the preceding image processing calculation P ₅₁ is “R component value binarization”, the subsequent image processing calculation P ₅₂ is “shrinkage”, and the feature amount calculation C ₅ is a calculation of “number of circular regions”. The classification process No. ₅ is considered to classify whether there are many red circular objects as a criterion. In this processing example, a classification process capable of interpreting the contents is generated as described above, but a classification process that is difficult to interpret may be generated, and such a classification process is also used for image classification described later. It's okay.

  When the classification process for classifying the images into the image group A and the image group B is generated by the above processing, the computer 1 has an image in which the image group to which it belongs is unknown (hereinafter referred to as “target image”). ) Is performed using the classification process (that is, the image classification device is realized by the computer 1).

  FIG. 16 is a diagram illustrating a flow of operations in which the computer 1 classifies target images. In the evaluation unit 22 of FIG. 2, an intermediate image is generated by performing a preceding image processing operation included in one classification process for each target image, and a processed image is obtained by performing a subsequent image processing operation on the intermediate image. It is generated (step S21). Subsequently, the feature amount is obtained by performing the feature amount calculation included in the classification process on the processed image (step S22). Then, by comparing the feature amount with a predetermined threshold value, it is determined whether the target image belongs to image group A or image group B (step S23). As a result, the target images can be grouped with high accuracy and at high speed. Note that after the classification process is generated by the process of FIG. 3, the contents of the classification process may be corrected by the worker, and the target image of FIG. 16 may be classified by the corrected classification process. Note that the determination in step S23 may use a known discriminant analysis using a feature amount or a classification method that is obtained in advance and learns with the feature amount of an image.

  As described above, in the classification processing generation apparatus realized by the computer 1, the evaluation unit 22 performs the image processing calculation and the feature amount calculation on the classification processing candidate for each original image belonging to each of the plurality of image groups. By performing in order, a plurality of feature amounts of a plurality of original images are obtained, and the effectiveness of the plurality of feature amounts in classification into image groups is obtained. Further, in the calculation content changing unit 23, the processing for obtaining the validity level by the evaluation unit 22 while changing the content of the calculation for the image processing calculation and the feature amount calculation in the classification processing candidate is performed until the effectiveness level is equal to or higher than the threshold value. Repeatedly, a classification process candidate when the effectiveness is equal to or greater than the threshold value is acquired as a usable classification process. Thereby, an effective classification process can be easily generated. Then, the contents of both the image processing calculation and the feature amount calculation in the effective classification process are displayed on the display 16 by the display control unit 24. In this way, by displaying the contents of the image processing calculation and the feature amount calculation in the effective classification process, the operator can surely understand the contents of the classification process and easily interpret and correct the classification process. It can be carried out.

  In addition, each original image and a processed image generated by performing image processing calculation in the classification process on the original image are displayed on the display 16 at the same time, so that the operator can select the original image and the processed image. The contents of the classification process can be intuitively understood with reference to, and the classification process can be easily interpreted. Further, a plurality of processed images generated by image processing calculation in the classification process are displayed on the display 16 at the same time, respectively corresponding to the plurality of image groups. As a result, the operator can easily compare a plurality of processed images respectively corresponding to a plurality of image groups, easily understand the difference of processed images between image groups, and interpret classification processing, etc. It can be done efficiently.

  In the classification process generation device, a plurality of classification process candidates are prepared in the evaluation unit 22, and a plurality of effectiveness levels respectively corresponding to the plurality of classification process candidates are obtained approximately in parallel. And the process which calculates | requires several effectiveness in the evaluation part 22, changing the content of the calculation in the classification process candidate contained in a some classification process candidate is repeated until completion | finish conditions are satisfy | filled. Thereby, an effective classification process can be generated efficiently.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made.

  In the above embodiment, the image processing operation is classified into the preceding image processing operation that is a binarization operation and the subsequent image processing operation that is a filter operation. However, the image processing operation is not necessarily the preceding image processing operation and the subsequent image. There is no need to distinguish between processing operations. For example, as shown in FIG. 17, each image processing operation processes a specific input plane 911 among the three input planes 911, and a specific output plane 912 among the three output planes 912. When it is realized as a module for outputting the calculation result, various image processing calculations can be freely combined. Note that it is preferable that an R component image, a G component image, and a B component image in the original image are respectively input to the three input planes 911 in the first image processing calculation. As described above, each classification processing candidate may include one or more image processing operations.

  Other filter processing, edge detection processing, and the like may be included as the image processing calculation. In addition, a combination of a process for converting an image into a frequency domain (FFT conversion) to generate a frequency image, a process for cutting a specific frequency component in the frequency image, and a process for performing inverse FFT conversion on the frequency image is one. An image processing operation may be prepared, and each of these processes may be prepared as one image processing operation.

  The feature amount calculation may include a calculation for obtaining a moment value, a calculation for determining the intensity of a specific frequency in the frequency image (in this case, the feature amount calculation may include FFT conversion), and the like.

  In the classification process generation device, a classification process for classifying an image into any of three or more image groups may be generated. In this case, when determining the effectiveness in step S14 of FIG. 3, for example, multiple comparisons in one-way analysis of variance are performed for each classification processing candidate (gene), and two image groups out of three or more image groups Significance probabilities in all the combinations are acquired, and the largest of these significance probabilities is treated as the attention significance probability corresponding to the effectiveness. Then, using the minimum significant probability among the attention significant probabilities of the plurality of classification processing candidates as a group evaluation value, update of the group (steps S16 to S19) and calculation of the significance probability until the group evaluation value falls below a predetermined evaluation threshold ( Steps S12 to S14) are repeated (step S15). Thereby, a classification process for classifying an image into any of three or more image groups is generated.

In addition, the effectiveness of the classification process candidate can be obtained by various methods. For example, in each classification processing candidate, T types (T is an integer equal to or greater than 1) of feature amount calculation are included, and when applying the technique of JP-T-2008-524675 (Patent Document 4), paragraph 0044 of Patent Document 4 , T types of feature amount computations C ₁ , C ₂ ,..., C _p ,..., C _T (p is an arbitrary integer from 1 to T) in each classification processing candidate each (however, each type of feature amount calculation is. to be performed on the same processed image obtained by performing image processing operations on the original image) feature amount with respect to the original image k is acquired by F _k1, F _{k2, ···, F kp, ···} , with _{F kT,} evaluation value _{E p} (effectiveness) of the feature amount by the feature calculation _{C p} is determined by the number 2.

Here, the largest evaluation value of the T types of feature values is handled as the attention evaluation value. Then, the update of the group (steps S16 to S19) and the calculation of the evaluation value (steps S12 to S14) are repeated (step S15) until the maximum of the attention evaluation values of the plurality of classification processing candidates reaches a predetermined threshold value or more. ), A combination of the feature amount calculation C _p of the feature amount that provides an evaluation value equal to or higher than the threshold and the image processing calculation at that time is used as an available classification process. Further, E (Feature) represented by Formula 1 in paragraph 0054 of JP-T-2008-129657 (Patent Document 5) can also be used as the evaluation value (effectiveness).

  In the above embodiment, the contents of the two classification processing candidates are changed by gene selection, multiplication, crossover, and mutation in the group update (FIG. 3: steps S16 to S19). The contents of a process candidate or three or more classification process candidates may be changed. In the classification processing generation device, by changing the content of the calculation in at least one of the plurality of classification processing candidates, the evaluation unit 22 obtains a plurality of degrees of validity corresponding to the plurality of classification processing candidates approximately in parallel, It is possible to efficiently acquire a classification process (that is, a classification process with high effectiveness) that can be used for image classification.

  Further, the classification process that can be used for image classification may be acquired by a technique other than the genetic algorithm. For example, a plurality of image processing calculations having the same calculation procedure but different parameter values such as threshold values are prepared, and one image processing calculation among the plurality of image processing calculations, one feature value calculation, Are prepared as one classification processing candidate (FIG. 3: step S11). Subsequently, the calculation of the effectiveness using the classification processing candidate is repeated while changing the image processing calculation in the classification processing candidate to another one of the plurality of image processing calculations (step S15, S12 to S14). If the effectiveness is obtained for all combinations of the image processing calculation and the feature value calculation and the end condition is satisfied (step S15), the classification processing that can use the classification processing candidate when the effectiveness is equal to or greater than the threshold value The contents of the classification process are displayed on the display 16 (step S20). Thereby, the operator can easily interpret and correct the classification process. Since the feature amount calculation is not changed, only the contents of the image processing calculation in the classification process may be displayed. In this case, it can be understood that the contents of the effective classification process are substantially displayed.

  Further, as described above, the effectiveness may be obtained while changing only one of the image processing calculation and the feature amount calculation in the classification processing candidate. In the classification processing generation device, the image processing calculation and the feature amount calculation in the classification processing candidate are performed. It is possible to generate an effective classification process by repeating the process of obtaining the effectiveness while changing the content of at least one of the operations until the end condition is satisfied. Then, when the end condition is satisfied, the content of at least one operation in the classification process is displayed on the display 16 as a classification process that can use the classification process candidate when the effectiveness is equal to or greater than the threshold, that is, the effective By displaying (substantially) the contents of the correct classification process, the operator can easily interpret and correct the classification process.

  On the other hand, from the viewpoint of easily generating a more effective classification process, a process for obtaining the effectiveness while changing the content of the calculation for both the image processing calculation and the feature value calculation in the classification process candidate is performed. It is preferable to repeat the process until the threshold value is equal to or greater than the threshold value, and to acquire the classification process candidate when the effectiveness is equal to or greater than the threshold value as an available classification process.

  In step S15 in FIG. 3, various end conditions can be used. For example, when the number of repetitions of steps S16 to S19 and S12 to S14 reaches a predetermined number, it may be determined that the end condition is satisfied. . Further, when the difference between the group evaluation value obtained in the immediately preceding process and the group evaluation value obtained in the current process is equal to or less than a predetermined value, or the state where the difference is equal to or less than the predetermined value. It may be determined that the end condition is satisfied when it is repeated continuously for a certain number of times.

  When a plurality of intermediate images are generated for one original image for each classification processing candidate, it is preferable that all intermediate images can be displayed by scrolling in the intermediate image display area 733.

  In addition to images showing artworks such as paintings, the classification processing generation device can display images showing defects in patterns formed on semiconductor substrates and glass substrates, or various parts of the human body imaged in medical settings. It may be used to generate a classification process for classifying various images, such as images shown, into any of a plurality of image groups. In addition to an image using visible light, an image using X-rays, terahertz waves, or the like may be used. In this case, classification that reflects differences in materials and the like in an object to be imaged is possible.

  The configurations in the above-described embodiments and modifications may be combined as appropriate as long as they do not contradict each other.

DESCRIPTION OF SYMBOLS 1 Computer 16 Display 22 Evaluation part 23 Calculation content change part 24 Display control part 31,41,51,61 Original image 33a-33e, 43a-43e, 53a-53e, 63a-63e Processed image 72 Classification process content display area 73 Group display area 732 Original image display area 734 Processed image display area S11 to S20 Steps

Claims (10)

A classification process generation device that generates a classification process for classifying an image into one of a plurality of image groups,
Classification processing candidates including predetermined image processing calculation and feature amount calculation are prepared, and using the classification processing candidates, the image processing calculation is performed on each original image belonging to each of a plurality of image groups, and processed images And calculating the feature amount on the processed image to obtain a plurality of feature amounts of the plurality of original images, and classifying each of the plurality of original images into one of the plurality of image groups. An evaluation unit for determining the effectiveness of the plurality of feature amounts in performing,
A calculation content changing unit that repeats the process of obtaining the effectiveness in the evaluation unit while changing the content of at least one of the image processing calculation and the feature value calculation in the classification processing candidate until an end condition is satisfied;
Display control for displaying the content of the at least one operation in the classification process on a display unit as a classification process that can use the classification process candidate when the effectiveness is equal to or greater than a threshold when the termination condition is satisfied And
A classification processing generation device comprising: The classification processing generation apparatus according to claim 1,
The classification processing generation apparatus, wherein the display control unit displays contents of both the image processing calculation and the feature amount calculation in the classification processing on the display unit. The classification process generation device according to claim 1 or 2,
The display control unit simultaneously displays each original image and the processed image generated by performing the image processing calculation in the classification process on each original image on the display unit. Classification processing generation device to perform. The classification process generation device according to claim 1 or 2,
The classification processing generation apparatus, wherein the display control unit simultaneously displays a plurality of processed images corresponding to the plurality of image groups and generated by the image processing calculation in the classification processing on the display unit. . A classification processing generation apparatus according to any one of claims 1 to 4,
In the evaluation unit, a plurality of classification processing candidates are prepared,
The evaluation unit obtains a plurality of effectiveness levels corresponding to the plurality of classification processing candidates,
The calculation content changing unit changes the content of the at least one calculation in at least one of the plurality of classification processing candidates while obtaining the plurality of validity levels in the evaluation unit until the end condition is satisfied. A classification processing generation device characterized by repetition. A classification process generation method for generating a classification process for classifying an image into one of a plurality of image groups,
a) preparing a classification processing candidate including predetermined image processing calculation and feature amount calculation;
b) performing the image processing operation on each original image belonging to each of a plurality of image groups to generate a processed image;
c) obtaining a plurality of feature quantities of a plurality of original images by performing the feature quantity computation on the processed image;
d) determining the effectiveness of the plurality of feature amounts when classifying each of the plurality of original images into any of the plurality of image groups;
e) repeating the steps b) to d) until the end condition is satisfied while changing the content of at least one of the image processing calculation and the feature amount calculation in the classification processing candidate;
f) a step of displaying the content of the at least one calculation in the classification process on a display unit as a classification process that can use the classification process candidate when the effectiveness is equal to or higher than a threshold;
A classification processing generation method characterized by comprising: The classification processing generation method according to claim 6,
In the step f), the contents of both the image processing calculation and the feature amount calculation in the classification process are displayed on the display unit. The classification processing generation method according to claim 6 or 7,
In the step f), each original image and the processed image generated by performing the image processing operation in the classification process on each original image are simultaneously displayed on the display unit. Classification processing generation method. The classification processing generation method according to claim 6 or 7,
In the step f), a plurality of processed images corresponding to the plurality of image groups and generated by the image processing calculation in the classification processing are simultaneously displayed on the display unit. Method. A classification processing generation method according to any one of claims 6 to 9,
In the step a), a plurality of classification processing candidates are prepared,
Through the steps b) to d), a plurality of effectiveness levels respectively corresponding to the plurality of classification processing candidates are obtained,
In the step e), the steps b) to d) are repeated until the end condition is satisfied while changing the content of the at least one calculation in at least one of the plurality of classification processing candidates. Classification processing generation method.

Images