JP2008299687A - Image processing/constructing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing/constructing method to perform a process approximate to sensory evaluation by visual observation and to automatically construct image processing only by setting a program through image processing by genetic programming. <P>SOLUTION: In the image processing/constructing method for obtaining an image processing solution approximate to a sensory evaluation value, a sample paper sheet with a variable density pattern is prepared, a subject performs sensory evaluation by visual observation to the sample paper sheet, the evaluation result where opinions of the subjects are in agreement is defined as an evaluation value, the sample paper sheet is received as a transmitted image using an imaging device, the image received for each paper sheet and the evaluation value by the sensory evaluation are paired to be input into the program, the solution is searched using the genetic programming so that the feature quantity obtained by performing a tree structure image processing and the evaluation value by the sensory evaluation may become closer based on a processing program in which image filters are combined in the shape of a tree, and finally obtaining the image processing in the tree structure approximate to a human sensory evaluation. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing construction method, and in particular, a method for obtaining an approximate solution approximating sensory evaluation using genetic programming.

  In recent years, research on genetic algorithms and genetic programming has been actively conducted as one of the optimization methods imitating the mechanism of evolution of living organisms. From research using these evolutionary methods, tree-structured image conversion automatic A generation method (Automatic Construction of Tree-structural Image Transformation) (hereinafter referred to as “ACTIT”) has been proposed.

  This ACTIT gives an original image that is an original image and a target image that is an ideal processing result after image processing for image processing. To what extent the output image obtained by image processing of the original image approximates the target image. It is constructed by combining various image processing filters prepared in advance in a tree structure, and the desired image processing is defined as the output image and the target image are closer (for example, non-patent literature) 1).

  In addition, a neural network that is an artificial neural network that imitates the operation of a human neural network with a computer, arranges one or more intermediate layers between the data input part and the output part, Each of these layers is constituted by a plurality of elements, and the input / output unit and each intermediate layer are connected in a network by a data input / output system. Since this neural network has a nonlinear component, it has a very high degree of approximation capability for various data, so it is possible to change the coupling coefficient by combining multiple units with arbitrary threshold values. To obtain an approximate value.

  However, in ACTIT, it is necessary for a human to create a target image by assuming an output image to some extent with respect to the original image, and there is a problem that a difference portion cannot be assumed for the original image such as sensory evaluation and quality evaluation. However, there is a problem that the target image cannot be made because the target image cannot be made.

  Also, when image processing is constructed using the neural network method, the image processing is expressed by the coupling coefficient between units, so that it is difficult for humans to understand the image processing algorithm and adjustment is difficult. there were.

Shinya Aoki and 1 other, "ACTIT, an automatic construction method for tree-structured image conversion", The Journal of the Institute of Image Information and Television Engineers, The Institute of Image Information and Television Engineers, 1999, Vol. 53, No. 6, p.888-894

  In the above-described automatic construction method for tree structure image conversion, it is necessary for a human to assume a target after image processing to some extent and set a target image.

  In addition, the algorithm that compares the input image with the output image to calculate the fitness used for the evolution of genetic programming addresses the problem that humans cannot assume the image after image processing by sensory evaluation etc. Can not. Further, it is impossible to confirm where the difference between the images that have been ranked from the processed images is. Further, it is not possible to confirm the ranking based on the sensory evaluation order and the difference in manufacturing conditions.

  The present invention has been considered in order to solve the above problems, and automatically constructs image processing that is difficult to produce by humans, and where is the difference in images that have been ranked from the processed images In order to make it possible to check whether the image processing is easier to recognize than a neural network, the ACTIT is improved. Conventionally, the fitness calculation used for the evolution of genetic programming is compared between the input image and the output image. This is an improvement to the algorithm that classifies the difference in the feature value obtained from the output image so that it matches the target specified by the human, and image processing can be automatically constructed simply by setting a program. An object of the present invention is to provide an image processing construction method capable of finding out the core image processing for ranking.

  The image processing construction method of the present invention according to claim 1 is an image processing construction method for searching for an approximate solution that approximates sensory evaluation, wherein sensory evaluation is performed on a substrate provided with a plurality of shade patterns. And classify at least two types of evaluation levels, extract a certain number of samples for each of the divided levels, determine the evaluation value by sensory evaluation for the sample, and transmit the shade pattern of the sample with the imaging device Captured as an image or reflection image, and stored in the storage device a pair of evaluation values obtained by sensory evaluation and the acquired transmission image or reflection image, and combined the image filter with a tree structure for the acquired transmission image or reflection image Image processing is performed by a processing program, and one or more feature quantities are extracted from each processed image, and the extracted feature quantities are approximated to evaluation values by sensory evaluation It is characterized by searching the approximate solution.

  The invention according to claim 2 is the image processing construction method according to claim 1, wherein the search method by the processing program is performed by an evolutionary calculation method of genetic programming.

  The invention according to claim 3 is an image processing construction method for selecting a central image processing by executing an approximate solution search a plurality of times using the evolutionary calculation of genetic programming according to claim 2. is there.

  According to a fourth aspect of the present invention, there is provided use of a printed matter misalignment inspection or print contamination detection based on a printing condition comprising the image processing construction method according to any one of the first to fourth aspects.

  Further, the invention described in claim 5 uses the image processing construction method according to any one of claims 1 to 4 to convert a captured transmission image or reflection image into RGB, HSV, or a sample when the sample is in color. An image processing construction method characterized in that an image for each color is input to each input image of a processing program that is decomposed into YIQ color components and combined in a tree structure.

  According to the image processing construction method of the present invention, it is possible to automatically create image processing that is difficult for a person to create and invent on a base material provided with a shading pattern subjected to sensory evaluation. In addition, ACTIT, which is a conventional method, created a target image by assuming a post-processing image. In the present invention, however, a target image is not necessary, and image processing can be automatically created simply by specifying an evaluation value. Thus, it is possible to cope with a problem that a person cannot expect an image after image processing.

  Furthermore, by performing the image processing of the present invention a plurality of times, it is possible to find a central image processing for ranking.

Hereinafter, an image processing construction method using genetic programming will be described with reference to the drawings for the first embodiment of the image processing construction method of the present invention.
The present invention is not limited to the best mode for carrying out the invention described below, and various other modes can be implemented within the scope of the technical idea described in the claims.

(Embodiment 1)
In the present embodiment, the shading pattern imparted to the base material, which is the object of the image processing construction method, refers to a wetting pattern in which an image appears by the density of the fiber density of the paper itself. In addition, a transmission image is used as an input image captured by the imaging apparatus. As the imaging device, a CCD camera or an image scanner can be considered, but the imaging device is not limited to this.

  FIG. 1 is a diagram showing an overall flow of an image processing construction method for obtaining an image processing solution that approximates a sensory evaluation value by a person according to the first embodiment. As the evaluation sample 101, 100 sheets of paper (hereinafter referred to as “sample sheets”) to which a cut-in pattern is applied are prepared. Three test subjects perform visual sensory evaluation 102 on 100 sample sheets, and classify them into 5 levels in order of quality. The test subject is a specialized panel with specialized knowledge and ability to evaluate.

  For the evaluation results classified by the three subjects into 5 levels, the evaluation result in which the opinions of the three subjects agree is set as the evaluation value 103, and in the subsequent examinations, the samples in which the opinions of the three subjects agree Use paper. In the first embodiment, four sheets are extracted for each level, and 20 sheets are prepared for five levels. In the present embodiment, the sample paper with the same evaluation by the subject is used, but the present invention is not limited to this, and an average value of the evaluation of the subject may be used.

  Twenty five-level sample sheets extracted by sensory evaluation by a subject are captured as a transmission image 104 using an imaging device such as a camera or a scanner. For 20 sample sheets, the image 104 captured for each sample sheet and the sensory evaluation value 103 are paired and input to a program (apparatus). When the input is completed, the computer (apparatus) performs processing, and the image processing generation unit 105 automatically constructs image processing that approximates visual sensory evaluation. Genetic features are obtained so that the feature value obtained by performing tree-structured image processing based on a processing program that combines image filters into a tree structure from the input image captured for each sample sheet approximates the evaluation value based on sensory evaluation. It is possible to search for a solution using genetic programming and finally obtain a tree-structured image processing 106 that approximates human sensory evaluation.

  A structure of a processing program as an example of how to obtain a feature amount obtained by performing tree-structured image processing from an input image captured for each sample sheet in the image processing construction method according to the first embodiment will be described.

A description will be given using a schematic diagram of the image processing 106 having a tree structure as shown in FIG.
The processing program is a program in which various image filters shown in Table 1 are combined in a tree structure. The image input to the tree structure is configured such that the transmission image 104 of the same sample paper is subjected to image processing by each image processing filter to form an output image. As the image processing filter Fn used in the present embodiment, an image processing filter having one input and one output (filters F8, F5, F3, and F2) and two inputs and one output (filters F6, F4, and F1) is used. The input images 1 to 4 are the same image, and an image of one sample sheet is input at a time for image processing of one tree structure. In the first embodiment, 20 sample sheets are prepared. Therefore, image processing is performed 20 times for 20 sheets. When the same image processing (tree structure image processing) is processed with another paper sample (20 sheets), different feature amounts (20 feature amounts) are obtained, so that the feature amounts are searched in order of sensory evaluation.

  An image filter as shown in Table 1 was used as a tree structure node. Table 1 shows the types of image processing filters and detailed image processing contents. An image processing filter is a known general-purpose image processing filter. It can be deleted according to the purpose, internal parameters can be changed, or an image processing filter whose function is specialized according to the purpose can be added. Is possible.

  As the sample paper used in the first embodiment, a transmission image of a paper provided with a 256 × 256 pixel penetration pattern in gray gradation is used. In the first embodiment, the same gray gradation image is used for all the input images. However, when there is a large difference in the color of the sample, the color image is converted into the three primary colors RGB or NTSC (National Television) of light in the color space. For example, the input image 1 has a red component image, the input image 2 has a blue component image, and the input image 3 has a green component. It is also possible to search for an approximate solution by inputting as in the image.

  A program 202 in which various image filters (Table 1) are combined in a tree structure for a plurality of input images (1, 2,..., N) 201 to be image-processed. Then, image processing by each image filter is performed to form an output image 203, and specific image processing is performed on the output image to calculate a feature amount 204. That is, the input image 1 is input to the image processing filter F3, processed and output, and the output image is input to the filter F2. The input image 2 and the input image 3 are input to the 2-input 1-output image processing filter F6, processed and output as one image, and the output image is input to the filter F5. The input image 4 is input to the image processing filter F8 and processed, and the output image and the image output from the image processing filter F5 are input to the filter F4 and processed. The image output from F2 and the image output from F4 are input to the filter F1 and processed, and finally, specific output processing is performed on the output image 203 output by combining the four input images. The feature amount 204 is calculated.

  The output image 203 is an image obtained by applying tree-structured image processing. The feature amount 204 is a process of calculating a feature amount by further performing specific image processing from the image of the output image 203. In the first embodiment, the output image 203 is binarized using the average value of the input images, and the binarized area ratio is calculated as a feature amount. In the first embodiment, only the binarized area ratio is used, but other feature quantities or a plurality of feature quantities may be calculated.

  In the first embodiment, the number of image processing filters Fn is limited by the parameters in the program so as to be a combination of 12 to 15 filters due to the limitation of the processing capability of the computer and the like. is not.

FIG. 3 is a diagram showing a processing flow for performing image processing by the image processing program in the image processing construction method of the first embodiment by genetic programming.
A tree structure image processing program is configured to automatically form using a genetic programming technique.

  In the processing flow, in the initial population generation and fitness evaluation 301, a plurality of tree-structured image processes called individuals in genetic programming are created at random, and the fitness is calculated. The number of individuals in the first embodiment is 200 to 500. The fitness indicates the degree of closeness to the goal in genetic programming, and the individual whose fitness is closer to the goal is more likely to survive in the next generation. As for the fitness in the first embodiment, the difference in the feature amount obtained from the image with respect to the ranking of the sensory evaluation is calculated as shown in Equation 1. As a classification method, other methods such as clustering may be used.

  In parent set selection 302, two individuals are selected from the individual population, and in crossover and mutation 3030, crossover and mutation operations are performed. Crossover is performed by randomly selecting nodes in the tree structure of two selected individuals, cutting them, and exchanging them with each other. In the mutation operation, a node performs a change to another image processing filter, insertion of another image processing filter, deletion of the selected image processing filter, and the like at a certain rate for each individual.

  In the fitness evaluation and next generation solid selection 304, the fitness (Equation 1) is calculated for the individuals that have undergone crossover and mutation, and the elite selection or the highest adaptation that selects the highest fitness is selected. A tournament selection for selecting an individual is performed, and an individual remaining in the next generation is selected. In this Embodiment 1, it processed using the tournament selection.

  In the end condition 305, an individual having the highest fitness is selected from all individuals, and it is determined whether the fitness is the target fitness. If the fitness is not sufficient, the process returns to the process 302, and sufficient The process ends when the fitness level is obtained or when the change in the fitness level decreases. The termination condition for optimization in the first embodiment is the termination condition when the fitness does not change between generations 5,000 to 10,000.

FIG. 4 describes an example of the tree-structured image processing obtained as a result of the search by using an S expression expressed in LISP or the like. Here, LIST Processor is a programming language for list processing. In LISP, it is expressed using a symbol string called S-expression, and both sides of the list are handled in parentheses. () Is a list boundary. Indicates.
(Max (Sbl (Lap (Dke (Max (Bav (Sbl (Mea (Mea (Mea (Rng (Sbl (Grd (input image)))))))))))) shown in 401 of FIG. The obtained image is the first derivative of the input image on the sample paper, Sobel, range, average value, average value, average value, Sobel, binarization by the average value of the input image, maximum value, DarkEdge, Laplacian, Sobel and maximum value The image processing filters are sequentially applied and processed.

  Of the papers other than the sample papers used in the first embodiment, that is, the papers other than the sample papers with which the three subjects agree, 40 sheets are extracted and prepared as sample papers, which are shown by 401 in FIG. When image processing obtained by genetic programming was applied, sensory evaluation and image processing evaluation were consistent for 32 out of 40 images. Further, for example, there are six sample sheets with similar evaluations such as sensory evaluation of 3 and image processing evaluation of 2 or 4, and it can be seen that the automatically generated image processing can approximate sensory evaluation.

  Furthermore, when the search for the approximate solution for sensory evaluation in the first embodiment is executed a plurality of times, different image processing is created each time. Since the genetic algorithm (programming) uses a random variable (random number), depending on the problem, the result may be different every time the search is executed from the beginning. In the present invention, the result is often different every time the search is executed from the beginning. However, by repeatedly executing the search, it is possible to find a place where the same processing is performed. By performing this same process, that is, by finding a central process, it is possible to understand what the image processing characteristics of the object to be inspected are.

401, 402, and 403 in FIG. 4 are the results of executing the approximate solution search in FIG. 3 three times. Although the image processing to be produced is different, there is a common part of (Sbl (Grd (input image))) in the three image processing. This is a process for first-order differentiation of an input image and applying a Sobel filter. By performing the approximate solution search a plurality of times in this way, a central process for approximating the sensory evaluation can be obtained.
In the present embodiment, it was verified that approximation of sensory evaluation can be approximated by using a general image processing filter such as a Sobel filter.

  As described above, since the common image processing may appear by performing the processing of FIG. 3 a plurality of times, this processing can be considered as the processing that is the center of approximation of sensory evaluation. As the usage of the processing obtained in this way, in this embodiment, it can be seen that edge extraction is the center, so another edge processing can be added to the image processing filter of this method and searched. Conceivable.

  In addition, there is a possibility that sensory evaluation can be approximated with one special edge processing filter. As another idea, there is a method of using only the obtained processing and multi-dimensional vectorization with the feature amount by other image processing. . In this way, there are various ways to use it, but this method is characterized in that it is possible to automatically obtain that there is a feature in the edge.

  When determining the quality of a printed material, it is necessary to determine whether the printed material is dirty or blurred, the density or color change of the printed material, and the like. Therefore, as a second embodiment of the image processing construction method of the present invention, image processing of a printed matter having a color difference will be described.

(Embodiment 2)
As sample paper, a printed material with a large difference in color is prepared.
Sample paper is captured as an input image by a scanner, and the captured color image is decomposed into color components such as RGB or YIQ to prepare a plurality of images. In the first embodiment, the input images 201 in FIG. 2 are all the same image. However, in the second embodiment, the input image 1 is an R component and the input image 2 is a Y component in 201 in FIG. By assigning an image to another image with different characteristics, it is possible to create an image process that takes into account differences in color and the like.

(Embodiment 3)
Next, the third embodiment of the image processing construction method according to the present invention can be used for inspection of foreign matter. For example, when it is desired to detect the presence or absence of dirt or foreign matter in printing, a normal printed matter and a printed matter having an abnormality such as a stain are classified, and a transmission image (or a reflection image) of the paper to which the clearance of Embodiment 1 is applied As a result of sensory evaluation, it is possible to automatically construct an image process capable of catching foreign matter by calculating an image and an evaluation value (with or without foreign matter) as a set.

  In the third embodiment, automatic construction of image processing regarding a difference in printed matter depending on printing conditions will be described. For example, there may be a case where the position of the printed material is shifted due to a difference in printing conditions such as printing pressure and humidity. In this case, the printed matter whose position is normal is separated from the printed matter having a positional deviation (102 in FIG. 1), and the positional deviation amount is set as an evaluation value indicated by 103 in FIG. Capture is performed as an image from the imaging apparatus 104 shown in FIG. By performing the search shown in 105 of FIG. 1 with the correct print position and the amount of displacement as a set, image processing that can detect the displacement of the printed matter can be automatically constructed.

(Embodiment 4)
Next, as a fourth embodiment of the image processing construction method according to the present invention, a case where it is used for extraction of oil stains will be described as an example of contamination of printed matter. The determination target is not limited to this, and includes stains caused by printing ink. The basic configuration is the same as in the first embodiment. The evaluation sample 101 is prepared with a printed matter free from oil stains and 10 printed matter (hereinafter referred to as “sample paper”) having different levels and sizes of oil stains. Three test subjects make a judgment of damage based on the degree of oil contamination by the sensory evaluation indicated by 102 in FIG. The judgment of damage was made based on the subject's agreement. The subject is a specialized panel with specialized knowledge and ability for evaluation.

  In the fourth embodiment, the damage is determined based on the density of the oil stain. However, when the oil stain is not allowed, the sample paper without oil stain or the sample paper with oil stain may be classified. Is possible.

  Ten sample sheets with the result of damage classified by three subjects are captured as a 512 × 512 pixel color image 104 using an imaging device such as a camera or a scanner. Oil stains are affected by the hue component in the contents of judgment of damage from light to dark, so in order to make use of color information, color images are converted into hues and hues in the color space. value) and lightness (value) (hereinafter referred to as “HSV”).

  In the present embodiment, the color image is decomposed into HSV. However, the color image is not limited to this, and may be decomposed into RGB, YIQ, or the like as long as it is decomposed into color components. The RGB color image representation has many other color coordinates that are particularly useful for various purposes, one of which is HSV, which facilitates the separation of hue, intensity and saturation from a color image. It is realistic because it is similar to the method of perceiving color.

  For 10 sample sheets, H is assigned to the input image 1 indicated by 201 in FIG. 2, the S component is assigned to the input image 2, and the V component is assigned to the input image 3. To the program (apparatus) 105 shown in FIG. When the input is completed, the computer (apparatus) performs the same processing as in the first embodiment, and the image processing generation unit 105 extracts the degree of oil stains and automatically constructs image processing for determining whether the damage is correct or not.

  The feature value obtained by performing tree-structured image processing based on a processing program in which image filters are combined in a tree structure from the input image captured for each sample sheet is close to the evaluation value of the impairment by sensory evaluation As described above, it is possible to obtain a tree-structured image processing 106 that searches for a solution using genetic programming and finally determines whether or not oil stains are correct.

  In Embodiment 4, the number of image processing filters Fn is limited by parameters in the program so that a maximum of 80 filters can be combined because it is very difficult to search for an approximate solution. The number of individuals in the fourth embodiment is 100 to 200. Parameters such as the fitness calculation method are the same as those in the first embodiment.

  A sheet other than the sample sheet used in the fourth embodiment, that is, ten sheets other than the sample sheet classified as good or bad by the agreement of the three subjects is prepared as a sample sheet. When the image processing obtained by dynamic programming was applied, the judgment of the damage by the sensory evaluation and the judgment of the damage by the image processing coincided in all 10 sheets.

It is the whole flowchart in this embodiment. It is a schematic diagram of image processing of a tree structure. It is a processing flowchart of genetic programming. It is an example of the tree-structure-like image processing described by the S expression of LISP.

Claims (5)

An image processing construction method for searching for an approximate solution approximating sensory evaluation,
Perform a sensory evaluation on a substrate with multiple shade patterns, and classify it into at least two types of evaluation levels.
A sample is extracted by extracting a certain number for each of the divided levels,
Determine the evaluation value by sensory evaluation for the sample,
The shade pattern of the sample is captured as a transmission image or a reflection image by an imaging device,
Store the evaluation value by the sensory evaluation and the captured transmission image or reflection image as a pair in a storage device,
Image processing by a processing program combining an image filter in a tree structure with respect to the captured transmission image or reflection image,
Extracting one or more feature quantities from each image processed image;
An image processing construction method characterized by searching for an approximate solution that approximates the evaluation value by the sensory evaluation for the extracted feature amount by an evolutionary calculation method. The image processing construction method according to claim 1, wherein the approximate solution search method by the processing program is genetic programming. The image processing construction method according to claim 2, wherein the central image processing is selected by executing an approximate solution search a plurality of times using the genetic programming. Use of the image processing construction method according to claim 1, 2 or 3 for inspection of misalignment of printed matter based on printing conditions or detection of printing contamination. When the sample is color, the captured transmission image or reflection image is decomposed into RGB, HSV, or YIQ color components, and each input image of the processing program combined in a tree structure is used as an image for each color. 5. The image processing construction method according to claim 1, 2, 3 or 4, wherein the input is performed.

Images