JP7247578B2 - Image processing system and image processing program - Google Patents

Description

The present invention relates to image processing technology for appearance inspection, and more particularly to an image processing system that optimizes image processing for finding detection targets from images.

Conventionally, in image processing technology for mechanical visual inspection, a method of optimizing image processing is performed by performing image processing using only an image of an inspection object as an input image, and comparing the obtained output image with a teacher image. It is
On the other hand, visual visual inspection by humans not only compares the products to be inspected with normal products, but also compares the good products that people remember and the products before and after being sent to the line with the so-called peripheral vision. A method called visual inspection by humans was not able to be reproduced with the image processing technology of visual inspection by machines.

Japanese Patent No. 5479944 Japanese Patent No. 4862150 Japanese Patent No. 5011533 JP 2008-15817 A

Here, as an image processing technique for finding a detection target from an image, the method described in Patent Document 1 can be mentioned.
In this method, size-dependent crossover is introduced into a parallel image filter automatic generation system by genetic programming (GP) to automatically construct image filters for crack extraction from various types of images. assessment of damage level is carried out.

In addition, the evolutionary computing systems described in Patent Documents 2 to 4 can be cited as techniques using such so-called evolutionary computation. These evolutionary computing systems make it possible to optimize the processing algorithm in each system.
However, even in these techniques, image processing is performed using only an image of an object to be inspected as an input image, and it is not possible to apply peripheral vision such as visual inspection by humans.

Therefore, in the image processing technology for visual inspection by machine, the present inventors performed image processing using not only the image of the inspection object but also the image of the non-defective product as an input image, and compared the obtained output information with the teacher information. By doing so, it is possible to apply peripheral vision to image processing and optimize image processing by evolutionary computation.
In addition, in evolutionary computation, by adopting the technique of genetic network programming (GNP), it is possible to improve the degree of freedom of optimization, thereby completing the present invention.

That is, an object of the present invention is to provide an image processing system and an image processing program capable of optimizing image processing for finding detection target elements from an image in image processing technology for visual inspection.

In order to achieve the above object, the image processing system of the present invention optimizes image processing for detecting detection targets from images by genetic manipulation, which is an optimization technique that mathematically simulates the evolution of living organisms. A population storage unit that stores one or more pieces of individual information that defines the execution order of a plurality of functional units in a network that can include a cycle, and a learning or verification use that displays an inspection object. a non-defective product image input unit for inputting a non-defective product image in which the inspection target is correctly displayed; and processing the inspection target image based on the individual information and the non-defective product image. , an image processing unit that generates output information including at least one of image, number, and position information of detection target elements in the inspection target image; and teacher information including detection target element information for evaluating the output information. an evaluation value calculation unit that compares the output information and the training information to calculate an evaluation value and ranks the individual information based on the evaluation value; and ranking the individual information. and a gene manipulation unit that selects or changes the individual information based on the information and updates the population storage unit.

Further, the image processing program of the present invention is an image processing program for optimizing image processing for detecting a detection target from an image by genetic manipulation, which is an optimization method that mathematically simulates the evolution of living things, wherein the computer a population storage unit that stores one or more pieces of individual information that define the execution order of a plurality of functional units in a network that can include a cycle, and an inspection object for learning or verification that displays the inspection object an inspection target image input unit for inputting an image; a non-defective product image input unit for inputting a non-defective product image in which the inspection target is correctly displayed; processing the inspection target image based on the individual information and the non-defective product image; An image processing unit that generates output information including at least one of the image, number, and position information of detection target elements in the above, and a teacher information input that inputs teacher information including information on the detection target elements for evaluating the output information an evaluation value calculation unit that compares the output information and the teacher information to calculate an evaluation value and ranks the individual information based on the evaluation value; and the individual information based on the ranking information of the individual information. is selected or changed to function as a gene manipulation unit that updates the population storage unit.

According to the present invention, it is possible to provide an image processing system and an image processing program capable of optimizing image processing for finding a detection target from an image in image processing technology for visual inspection.

1 is a block diagram showing the configuration of an image processing system (image processing apparatus) according to a first embodiment of the present invention; FIG. 4 is a flow chart showing a processing procedure by the image processing system according to the first embodiment of the present invention; 4 is a flow chart showing the processing procedure of gene manipulation in the processing procedure by the image processing system of the first embodiment of the present invention; FIG. 11 is a block diagram showing the configuration of an image processing system according to a fourth embodiment of the present invention; FIG. FIG. 12 is a flow chart showing a processing procedure by the image processing system of the fourth embodiment of the invention; FIG. FIG. 10 is a diagram showing the functions (image processing: noise removal) of nodes used in the example and the reference example; FIG. 4 is a diagram showing functions of nodes (image processing: brightness correction, combination) used in the example and the reference example; FIG. 4 is a diagram showing the function (image processing: binarization) of a node used in the example and the reference example; FIG. 4 is a diagram showing the functions (image processing: edge detection) of nodes used in the example and the reference example; FIG. 4 is a diagram showing functions of nodes (image processing: edge detection, frequency filter) used in the example and the reference example; It is a figure which shows the function (image processing: four arithmetic operations, fuzzy operation) of the node used by the Example and the reference example. FIG. 4 is a diagram showing the function of a node (detection target determination) used in the example and the reference example; FIG. 10 is a diagram (No. 1-5) showing learning in Test 1 and its results; FIG. 6 is a diagram (No. 6-10) showing learning and results in Test 1; 1 is a diagram (No. 1-5) showing verification results in Test 1. FIG. FIG. 10 is a diagram showing a graph representing transition of learning evaluation values in test 1; FIG. 10 is a diagram showing the structure of individual information obtained by learning of the example in test 1; FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning in an example in Test 1; FIG. 10 is a diagram showing the structure of individual information obtained by learning a reference example in Test 1; FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning a reference example in Test 1; FIG. 10 is a diagram (No. 1-5) showing learning in Test 2 and its results; FIG. 10 is a diagram (No. 6-10) showing learning in Test 2 and its results; FIG. 10 is a diagram (No. 1-5) showing verification results in Test 2; FIG. 10 is a diagram showing a graph representing transition of learning evaluation values in test 2; FIG. 10 is a diagram showing the structure of individual information obtained by learning of the example in test 2; FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning in an example in Test 2; FIG. 10 is a diagram showing the structure of individual information obtained by learning a reference example in Test 2; FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning of a reference example in Test 2;

Embodiments of an image processing system and an image processing program according to the present invention will be described in detail below. However, the present invention is not limited to the specific contents of the following embodiments and examples described later.

[First embodiment]
First, an image processing system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG. FIG. 1 is a block diagram showing the configuration of an image processing apparatus corresponding to the image processing system of this embodiment, and FIG. 2 is a flow chart showing the processing procedure of the image processing system. FIG. 3 is a flow chart showing the genetic manipulation processing procedure in the processing procedure of the same image processing system.

The image processing system of this embodiment optimizes image processing for detecting a detection target from an image by genetic manipulation, which is an optimization technique that mathematically simulates the evolution of living things.
Specifically, as shown in FIG. 1, the image processing apparatus 1 corresponding to the image processing system of this embodiment includes a functional unit storage unit 10, an individual generation unit 11, an individual group storage unit 12, and an inspection target image input unit. 13, inspection target image storage unit 14, non-defective product image input unit 15, non-defective product image storage unit 16, teacher information input unit 17, teacher information storage unit 18, image processing unit 19, output information storage unit 20, evaluation value calculation unit 21, An evaluation result storage unit 22 and a gene manipulation unit 23 are provided.

All of these components in the image processing apparatus 1 of this embodiment can be provided in one information processing apparatus as shown in FIG. Also, each of these components may be distributed to each device of an image processing system comprising a plurality of information processing devices. This also applies to each embodiment described later.

The functional unit storage unit 10 stores, for each node number, a node (corresponding to a gene), which is a functional unit that executes a certain process. In this embodiment, various nodes for performing image processing can be used as this node, and the nodes shown in FIGS. 6 to 12 are used in the examples described later.
Further, these nodes are stored in the functional unit storage unit 10 together with parameters used for execution of the respective processes, if any.

Furthermore, a node that does not execute any process may be used, or a functional unit that executes a plurality of processes may be used as one node (node set). By using nodes that do not perform any processing as nodes, it is possible to reduce the actual number of nodes when, for example, the set value of the initial number of nodes for generating individual information is too large.

In this specification, "image processing" includes various filter processing for noise removal processing, correction processing such as brightness correction, binarization processing, edge detection processing, frequency filter processing, four arithmetic operations, and fuzzy operations. In addition to image processing in a narrow sense, such as image processing, detection target determination processing, output processing, and the like that can be executed in conjunction with this processing are included.

The individual generation unit 11 uses the nodes stored in the functional unit storage unit 10 to generate individual information consisting of an array of a plurality of nodes. At this time, the individual generation unit 11 can randomly generate a plurality of pieces of individual information based on the plurality of nodes.
At this time, in the individual information, the individual generation unit 11 can define the execution order of the plurality of nodes in a network that can include a cycle. The network is a directed network composed of oriented links and may include feedback. The network can also include linear structures and tree structures.

By using such individual information, it is possible to perform learning including a network structure with more flexible expressive power than when using individual information consisting only of a linear structure or a tree structure. It is superior in deriving practical optimal solutions and practical solutions). In other words, it is difficult for humans to determine in advance which structure will be the optimal solution for the problem, and therefore it is difficult to determine the conditions of the structure prior to machine learning. For this reason, searching for the optimal solution under a wide range of conditions using individual information that defines the execution order of multiple nodes in a network that can include cycles may lead to the possibility of deriving the true optimal solution. It is considered to be effective in increasing

The individual group storage unit 12 stores an individual group composed of a plurality of pieces of individual information generated by the individual generation unit 11 . One generation or a plurality of generations can be stored in the population storage unit 12 as this population, and individual information can be stored in the population storage unit 12 for each generation number.
The evaluation value calculation unit 21 , which will be described later, can rank the individual information for each generation and store the ranking information in the evaluation result storage unit 22 .

The inspection target image input unit 13 inputs the inspection target image for learning or verification in which the inspection target is displayed to the image processing apparatus 1, and stores the inspection target image storage unit 14 for each image identification information.
The non-defective product image input unit 15 inputs the non-defective product image in which the inspection object is correctly displayed to the image processing apparatus 1, and stores the non-defective product image storage unit 16 for each image identification information.

Here, a good product image is an image obtained by photographing a normal inspection object in advance. It simulates the product of
In the image processing system of the present embodiment, peripheral vision can be reproduced by inputting a non-defective product image together with an image to be inspected and performing image processing.

The teacher information input unit 17 inputs the image of the detection target element in the inspection target image corresponding to the image identification information as teacher information for evaluating the output information in association with the image identification information. is stored in the teacher information storage unit 18 every time.
Here, the detection target element can be a defect (crack, scratch, stain, etc.) in the image to be inspected, a specific part, a specific pattern, or the like. Note that the detection target element may correspond to the entire inspection target image.

The image processing unit 19 receives the image to be inspected from the image storage unit 14 to be inspected and also inputs the non-defective product image from the non-defective product image storage unit 16 . Then, individual information is input from the individual group storage unit 12, and image processing is performed on the image to be inspected by sequentially executing a plurality of nodes included in the individual information based on the individual information.
This allows the optimization of image processing for the image under test to be better than simulated peripheral vision by using the good image in the execution of the node.

At this time, each node uses information obtained from the image to execute processing corresponding to each function. Also, each node can selectively execute the next node corresponding to the processing result based on the processing result. Therefore, even nodes included in individual information are not necessarily executed in image processing. Of course, there is a case where all nodes in individual information are executed. Moreover, a node defined only once in one individual information may be executed multiple times due to feedback.

As described above, the node functions include various image filtering processes for noise removal, correction processes such as luminance correction, binarization, edge detection, frequency filtering, four arithmetic operations, fuzzy There is narrowly defined image processing such as arithmetic processing. These are mainly performed by the image processing unit 19 as processing for finding candidates for detection target elements (detection target candidates).

In addition, the detection target determination processing as a function of the node is performed by labeling candidate detection target elements in a certain range in the image by the image processing unit 19, calculating the feature amount, and calculating the feature amount based on a threshold value. is a correct detection target element.
In addition, in the detection target determination process, a process of labeling a candidate of a non-detection target element, calculating its feature amount, and determining that it is not a detection target element by not having the feature amount based on a threshold value or the like. can also be done.

Furthermore, the image processing unit 19 generates and outputs output information.
Specifically, the image of the detection target element in the inspection target image is generated as the output information. Then, this output information can be stored in the output information storage unit 20 for each image identification information of the image to be inspected and for each identification information of individual information.
Thus, the output information obtained by the image processing unit 19 in this embodiment is generated as the image of the detection target element in the inspection target image.

The evaluation value calculation unit 21 receives the output information from the output information storage unit 20 and inputs the teacher information corresponding to the image identification information corresponding to the output information from the teacher information storage unit 18 .
Then, the output information and the teacher information are compared to calculate an evaluation value, and the obtained evaluation value is stored in the evaluation result storage unit 22 for each image identification information of the inspection target image and for each individual identification information. be able to.
At this time, the evaluation value calculation unit 21 can use the output information and the teacher information to compare the brightness of the image of the inspection target element for each pixel, and calculate the mean square error as the evaluation value by the least squares method. can.

Also, when the evaluation value calculation unit 21 has finished calculating evaluation values for all the individual information (or individual information for one generation) stored in the population storage unit 12 and stored in the evaluation result storage unit 22 For example, the individual information can be ranked based on these evaluation values, and the obtained ranking information can be stored in the evaluation result storage unit 22 .

Furthermore, the evaluation value calculation unit 21 can perform end determination for determining whether or not the image processing by the image processing unit 19 is to end. The termination determination can be made based on whether or not the termination condition is satisfied. If the termination condition is satisfied, the image processing by the image processing unit 19 can be terminated, and the processing by the image processing apparatus 1 can be terminated. Then, the individual information with the highest evaluation value stored in the evaluation result storage unit 22 is obtained as the individual information optimized for image processing.

Examples of termination conditions include the case where image processing has been completed for populations of all generations set in advance, and the case where evolution has not occurred in a certain number of generations or more.
A case in which no evolution has occurred is a case in which the evaluation value has not become higher than at the point of comparison. In the embodiment described later, since the mean square error is used as the evaluation value, the smaller the evaluation value, the better the individual information. Therefore, if the image processing is performed for a certain number of generations and the evaluation value does not decrease, it can be determined that the image has not evolved.

Note that the end determination function may be separated from the evaluation value calculation unit 21 and an end determination unit for performing end determination may be provided in the image processing apparatus 1 of the present embodiment.

The gene manipulation unit 23 selects or changes individual information based on the ranking information of the individual information in the evaluation result storage unit 22 and updates the population storage unit 12 .
Specifically, the gene manipulation unit 23 includes an elite individual storage unit that leaves individual information with a high evaluation value to the next generation, an individual selection unit that leaves individual information to the next generation with a certain probability based on the evaluation value, It can have a crossover part that mutually exchanges a part of two pieces of individual information, and a mutation part that randomly rewrites part or all of the selected piece of individual information.

The elite individual storage unit can unconditionally leave the individual information with the best evaluation value in the population of each generation to the next generation.
The individual selection unit can perform a process of selecting individual information with a high ranking of evaluation values in a group of individuals in each generation with a high probability, and leaving the information to the next generation.

As crossover processing, the crossover unit can perform one-way crossover in which the same (node number) sequence in the array of each node of a plurality of pieces of individual information is replaced with a certain probability. In addition, the crossover section performs crossover processing such as one-point crossover in which one array is exchanged with one point of the same arrangement in the array of each node as a boundary, or an array is mutually exchanged with a plurality of points in the same arrangement as a boundary. It is also possible to perform multi-point crossovers with permutations. The crossover section can perform such crossover processing on the individual information selected by the individual selection section.

As mutation processing, the mutation unit rewrites part of the individual information or rewrites all of the individual information with a certain probability for each individual information using the nodes stored in the functional unit storage unit 10. be able to. The mutation section can perform such mutation processing on the individual information selected by the individual selection section. In addition, the mutating section can also generate new individual information using the nodes stored in the functional unit storage section 10 as mutation processing.

If the number of same-line individuals generated based on the same individual information increases, diversity will be lost and evolution will enter a dead end (a state of mathematically falling into a local optimum). On the other hand, if the number of individuals other than the same lineage is increased too much, it becomes closer to random search and efficient evolution does not occur.
Therefore, in the early stage of learning, the mutation unit processes to increase the number of individuals other than the same lineage in order to provide diversity. preferably increased. Moreover, when evolution has stopped, it is possible to preferably increase non-same lineages by means of a mutation site in order to restore diversity.

Then, the gene manipulation unit 23 generates a new generation population and stores it in the population storage unit 12 . At this time, the gene manipulation unit 23 can update the population storage unit 12 by overwriting the population of the previous generation. In addition, the population storage unit 12 can be updated by storing populations of multiple generations for each generation.

In the image processing system of this embodiment, the inspection image input unit 13 inputs an inspection image, and the image processing unit 19 processes the inspection image based on the individual information with the highest evaluation value and the non-defective product image. It is also preferable to have a configuration including a processing result storage unit (not shown) for storing the obtained processing result information.
As the processing result information, for example, an image of the detection target element, a numerical value indicating the number, the position recognized as the detection target, the number recognized as the detection target outside the range of the detection target element, and the like can be used.

In such a configuration, the inspection object image input unit 13 inputs an image obtained by photographing the inspection object as an image for inspection in real time, and the non-defective product image input unit 15 inputs the image of the inspection object. It is also preferable to adopt a configuration in which an image obtained by photographing another inspection object of the same type is input in real time as a non-defective product image.
As other inspection objects of the same type as the inspection object, for example, products before and after the inspection object flowing through the line can be used.

With this configuration of the present embodiment, when inspecting a product, it is possible to use the products before and after the inspection object flowing on the line as non-defective product images, and the image processing system can preferably reproduce peripheral vision. It is possible.

Next, a processing procedure by the image processing system of this embodiment will be described with reference to FIGS. 2 and 3. FIG.
First, the inspection target image input unit 13 in the image processing apparatus 1 inputs an inspection target image and stores it in the inspection target image storage unit 14 (step 10). Also, the non-defective product image input unit 15 inputs a non-defective product image and stores it in the non-defective product image storage unit 16 (step 11). Further, the teacher information input unit 17 inputs an image of the element to be detected as teacher information and stores it in the teacher information storage unit 18 (step 12). Then, an initial population of individuals is generated by the individual generation unit 11 and stored in the population storage unit 12 (step 13). Note that the order of these steps may be changed.

At this time, the individual generation unit 11 can generate individual information by randomly selecting and arranging nodes from the functional unit storage unit 10, and store the individual information in the individual group storage unit 12 for each identification information of the individual information. can. Furthermore, the individual generation unit 11 can also store a group of individuals consisting of a plurality of pieces of individual information as one generation, and store groups of individuals in a plurality of generations for each generation number in the group storage unit 12 .

Next, the image processing unit 19 performs image processing on all individual information in the individual group for each generation of the individual group in the individual group storage unit 12 (steps 14, 15 and 16).
At this time, the image processing unit 19 outputs the image of the detection target element in the inspection target image as output information for each piece of individual information, and stores the image in the output information storage unit 20 .

Next, the evaluation value calculator 21 compares the output information and the teacher information for each piece of individual information to calculate an evaluation value (step 17).
At this time, in the calculation of the evaluation value by the evaluation value calculation unit 21, for example, the least squares method can be used to calculate the mean square error as the evaluation value.
In addition, the evaluation value calculation unit 21 stores the obtained evaluation values in the evaluation result storage unit 22 for each piece of individual information.
Then, the processing from image processing to evaluation value calculation and storage is repeatedly executed for all pieces of individual information of the generation.

Next, the evaluation value calculation unit 21 ranks the individual information based on the evaluation values stored in the evaluation result storage unit 22 (step 18).
Also, the evaluation value calculation unit 21 performs determination processing as to whether or not the termination condition is satisfied (step 19). If the termination condition is satisfied, the processing by the image processing apparatus 1 is terminated.

If the termination condition is not satisfied, the gene manipulation unit 23 executes the gene manipulation process (step 20).
Specifically, as shown in FIG. 3, the elite individual storage unit in the gene manipulation unit 23 selects the individual information with the highest evaluation value as the individual information of the next-generation population (step 30).
Further, the individual selecting unit in the gene manipulation unit 23 selects the individual information having the higher rank of the evaluation value as the individual information of the population of the next generation with a higher probability (step 31).

Further, the crossover unit in the gene manipulation unit 23 performs crossover processing on the individual information selected by the individual selection unit to generate new individual information (step 32).
Further, the mutation section in the gene manipulation section 23 performs mutation processing based on the individual information selected by the individual selection section or without using such individual information to generate new individual information (step 33).
The order of each processing by the elite individual storage unit, individual selection unit, crossover unit, and mutation unit may be changed within the range of the order in which each processing can be executed.

According to the image processing apparatus of the present embodiment, in the image processing technique for appearance inspection, not only the image of the inspection object but also the non-defective product image is processed as an input image, and the obtained output information is processed. Peripheral vision can be applied to image processing by comparison with teacher information, and image processing can be optimized by evolutionary computation.
In addition, in evolutionary computation, it is possible to improve the degree of freedom of optimization by adopting the method of genetic network programming (GNP).

[Second embodiment]
Next, an image processing system according to a second embodiment of the invention will be described. The image processing system of this embodiment differs from that of the first embodiment in that a numerical value indicating the number of detection target elements in an inspection target image is used as teacher information. Other points are the same as in the first embodiment.

That is, although not shown, the image processing apparatus 1a corresponding to the image processing system of this embodiment includes a functional unit storage unit 10a, an individual generation unit 11a, an individual group storage unit 12a, an inspection target image input unit 13a, and an inspection target image storage unit. unit 14a, non-defective product image input unit 15a, non-defective product image storage unit 16a, teacher information input unit 17a, teacher information storage unit 18a, image processing unit 19a, output information storage unit 20a, evaluation value calculation unit 21a, evaluation result storage unit 22a, and a gene manipulation unit 23a, which have the same functions as the corresponding components in the first embodiment except for the following points.

In the image processing apparatus 1a of the present embodiment, the teacher information input unit 17a inputs the detection target element in the inspection target image corresponding to the image identification information as teacher information for evaluating the output information in association with the image identification information. is input, and stored in the teacher information storage unit 18a for each image identification information.

In addition, the image processing unit 19a receives the inspection target image from the inspection target image storage unit 14a and inputs the non-defective product image from the non-defective product image storage unit 16a. Then, individual information is input from the individual group storage unit 12a, and image processing is performed on the image to be inspected by sequentially executing a plurality of nodes included in the individual information based on the individual information.

In this embodiment, the image processing unit 19a generates, as output information, a numerical value indicating the number of detection target elements in the inspection target image. Then, this output information can be stored in the output information storage unit 20a for each image identification information of the image and for each identification information of individual information.
Thus, the output information obtained by the image processing unit 19a in this embodiment is generated as a numerical value indicating the number of detection target elements in the inspection target image.

The evaluation value calculation unit 21a receives the output information from the output information storage unit 20a, and also receives the teacher information corresponding to the image identification information corresponding to the output information from the teacher information storage unit 18a.
Then, the output information and the teacher information are compared to calculate an evaluation value, and the obtained evaluation value is stored in the evaluation result storage unit 22a for each image identification information of the inspection target image and for each individual information identification information. be able to.

Here, when the teacher information is an image, in the evaluation by the least squares method, the evaluation value is strongly affected by the size of the detection target element. is different, the evaluation value tends to deteriorate.
For example, assume that a triangle with a size of 10 pixels is not detected when a triangle is a detection target element in an image.
At this time, if the output of a larger triangle in the same image is slightly smaller than that of the teacher image, if the difference is 10 pixels or more, the evaluation value will deteriorate due to the absence of a 10-pixel triangle. Become. For this reason, in machine learning, the evaluation value tends to improve when triangles that are larger than small triangles are brought closer to the teacher image. . On the other hand, in this embodiment, the undetected error is the same regardless of the size of the detection target element, so that it is difficult to fall into local minima.

According to the image processing system of this embodiment, in addition to being able to apply peripheral vision to image processing, image processing is performed on an inspection target image to create output information indicating the number of detection target elements. , the output information can be compared with numerical values as teacher information. Therefore, the same evaluation value can be obtained regardless of the size of the detection target element, and the problem that the evaluation value deteriorates depending on the ratio of the detection target element region can be solved.

[Third Embodiment]
Next, an image processing system according to a third embodiment of the invention will be described. The image processing system of this embodiment differs from that of the first embodiment in that the position information of the detection target element in the inspection target image is used as the teacher information. Other points are the same as in the first embodiment.

That is, although not shown, the image processing apparatus 1b corresponding to the image processing system of this embodiment includes a functional unit storage unit 10b, an individual generation unit 11b, an individual group storage unit 12b, an inspection target image input unit 13b, and an inspection target image storage unit. unit 14b, non-defective product image input unit 15b, non-defective product image storage unit 16b, teacher information input unit 17b, teacher information storage unit 18b, image processing unit 19b, output information storage unit 20b, evaluation value calculation unit 21b, evaluation result storage unit 22b, and a gene manipulation unit 23b, which have the same functions as the corresponding components in the first embodiment except for the following points.

In the image processing apparatus 1b of the present embodiment, the teacher information input unit 17b inputs the detection target element in the inspection target image corresponding to the image identification information as teacher information for evaluating the output information in association with the image identification information. , and stored in the teacher information storage unit 18b for each image identification information.
As this position information, it is preferable to use the range of the detection target element (coordinates representing the area occupied by the detection target element in the image) or the center of gravity of the detection target element.
As the center of gravity of the detection target element, the center of gravity of the detection target element itself, the center of gravity of the circumscribed rectangle of the detection target element, the center of gravity of the inscribed circle of the detection target element, or the like may be used.

Further, the image processing unit 19b receives the inspection target image from the inspection target image storage unit 14b and also inputs the non-defective product image from the non-defective product image storage unit 16b. Then, individual information is input from the individual group storage unit 12b, and image processing is performed on the image to be inspected by sequentially executing a plurality of nodes included in the individual information based on the individual information.

In this embodiment, the image processing unit 19b generates position information such as the range and the center of gravity of the detection target element in the inspection target image as output information. Then, this output information can be stored in the output information storage unit 20b for each image identification information of the image to be inspected and for each identification information of individual information.
Thus, the output information obtained by the image processing unit 19b in this embodiment is generated as the position information of the detection target element in the inspection target image.

In this embodiment, it is preferable to use the range of the detection target element or the center of gravity of the detection target element for the position information of the detection target element of the output information, as with the teacher information. The types do not need to be the same, and different types of position information may be used, for example, using the range as teacher information and using the center of gravity as output information.

In this embodiment as well, compared to the case where the image of the detection target element is used as the teacher information, the undetected error is the same regardless of the size of the detection target element, so it is less likely to fall into a local solution. .

In this embodiment, the evaluation value can be calculated based on the following criteria, for example, based on the position information of the detection target element in the image.

where n is the number of samples and the number of images. ScoreA, which is the score obtained from the position information of the detection target element, is a detection target element whose center of gravity, which is the position information of the detection target element in the output information, does not exist within the range of the detection target element in the teacher information. , is the mean square error with respect to the number of detection target elements existing outside the range of the detection target element in the teacher information. In addition, ScoreB is the mean square error for the number of relevant teacher information when there is a target element whose center of gravity, which is the position information of the target element in the output information, does not exist within the range of the target element in the teacher information. is. Also, w is weighting information, which makes it possible to change the importance of ScoreB.

For example, in a teacher image corresponding to a certain image, it is assumed that position information of detection target element 1 (teaching range 1) and position information of detection target element 2 (teaching range 2) exist. As a result of binarization and labeling of this image by image processing by the image processing unit 19b, two detection target elements are found. Assume that the output information including the position information (2) of the target element is obtained.

The position information (1) of the detection target element exists within the teacher range 1, the position information (2) of the detection target element exists outside the teacher range 2, and the center of gravity exists within the teacher range 2. Suppose it was not.
In this case, if w=5, the evaluation value for position information (1) of the detection target element is 0, and the evaluation value for position information (2) of the detection target element is (0−1) ² +5× (1−0) ² =6, and an evaluation value of 6 is calculated for this output image and individual information.

Note that the above ScoreA and ScoreB may be calculated by the image processing unit 19b, stored in the output information storage unit 20b, and the evaluation value calculation unit 21b may use them to calculate the evaluation value.
Also, the evaluation value calculation unit 21b can be made to perform the function of acquiring the position information such as the range and the center of gravity of the detection target element in the image. That is, the evaluation value calculation unit 21b may acquire position information based on the image after image processing by the image processing unit 19b, and may calculate the evaluation value based on this.

According to such an image processing system of the present embodiment, in addition to being able to apply peripheral vision to image processing, image processing is performed on an image to be inspected to create output information indicating position information of elements to be detected. Then, the output information can be compared with position information, which is teacher information. Therefore, the same evaluation value can be obtained regardless of the size of the detection target element, and the problem that the evaluation value deteriorates depending on the ratio of the detection target element region can be solved.

[Fourth embodiment]
Next, an image processing system according to a fourth embodiment of the invention will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a block diagram showing the configuration of an image processing apparatus corresponding to the image processing system of this embodiment, and FIG. 5 is a flow chart showing the processing procedure of the image processing system.

As shown in FIG. 4, the image processing apparatus 1c corresponding to the image processing system of this embodiment includes a functional unit storage unit 10c, an individual generation unit 11c, an individual group storage unit 12c, an inspection target image input unit 13c, and an inspection target image. Storage unit 14c, good product image input unit 15c, good product image storage unit 16c, teacher information input unit 17c, teacher information storage unit 18c, image processing unit 19c, output information storage unit 20c, evaluation value calculation unit 21c, evaluation result storage unit 22c , and a gene manipulation unit 23c. The population storage unit 12c also includes a parent population storage unit 121c and a child population storage unit 122c.

Functional unit storage unit 10c, inspection target image input unit 13c, inspection target image storage unit 14c, non-defective product image input unit 15c, non-defective product image storage unit 16c, teacher information input unit 17c, and teacher information storage in image processing apparatus 1c of this embodiment The unit 18c, the output information storage unit 20c, and the evaluation result storage unit 22c can be the same as those in the first embodiment. Also, other configurations of the image processing apparatus 1c of the present embodiment can be the same as those of the first embodiment except for the points described below.

The individual generating unit 11c uses the nodes stored in the functional unit storage unit 10 to generate individual information consisting of an array of a plurality of nodes. At this time, as in the first embodiment, the individual generation unit 11c can define the execution order of a plurality of nodes in the individual information in the form of a network that can include a cycle.
Further, the individual generation unit 11c in the present embodiment can randomly generate a plurality of pieces of parent individual information as individual information, and store an initial parent individual group made up of the parent individual information in the parent individual group storage unit 121c. can.

The gene manipulation unit 23c executes gene manipulation on the parent individual information in the parent individual group storage unit 121c.
Specifically, first, parent individual information for generating child individual information is randomly selected from the parent individual group in the parent individual group storage unit 121c.

The elite individual storage unit in the gene manipulation unit 23c can store a child population consisting of child individual information identical to the selected parent individual information in the child population storage unit 122c.
Then, the crossover unit in the gene operation unit 23c performs crossover processing on the selected parent individual information, generates a plurality of child individual information, and stores a child population consisting of these child individual information. It can be stored in the unit 122c.
In addition, the mutation unit in the gene manipulation unit 23c uses the nodes stored in the functional unit storage unit 10 to execute mutation processing on the selected parent individual information, and the parent individual with a certain probability. A child population consisting of child individual information obtained by rewriting a part of the information or by rewriting all of the information can be stored in the child population storage unit 122c. Further, the mutating section can also generate new individual information using the nodes stored in the functional unit storage section 10c as mutation processing.

Furthermore, based on the ranking information of the child individual information in the evaluation result storage unit 22c, the gene manipulation unit 23c randomly selects the child individual information with the best evaluation value and the child individual information selected with probability from the parent population. It is possible to update the parent individual information in the parent individual group storage unit 121c by replacing it with the above parent individual information selected by the parent individual group storage unit 121c.

The inspection target image input unit 13c inputs the inspection target image for learning or verification in which the inspection target is displayed to the image processing device 1c, and stores the inspection target image storage unit 14c for each image identification information.
The non-defective product image input unit 15c inputs the non-defective product image in which the inspection object is correctly displayed to the image processing device 1c, and stores the non-defective product image in the non-defective product image storage unit 16c for each image identification information.
The teacher information input unit 17c inputs the image of the detection target element in the inspection target image corresponding to the image identification information as teacher information for evaluating the output information in association with the image identification information. is stored in the teacher information storage unit 18c every time.

The image processing unit 19c receives the detection target image from the inspection target image storage unit 14c, the non-defective product image from the non-defective product image storage unit 16c, and the child individual information from the child population storage unit 122c. Image processing is performed on the image to be inspected by sequentially executing a plurality of nodes included in the child individual information based on the individual information. This allows the optimization of the image processing for the image under test to be better than simulated peripheral vision by using the good image in the execution of the node.

Then, the image processing unit 19c creates an image of the detection target element in the inspection target image, generates output information, and outputs this output information for each image identification information of the image and for each identification information of child individual information. It can be stored in the storage unit 20c.

The evaluation value calculation unit 21c receives the output information from the output information storage unit 20c, and also receives the teacher information corresponding to the image identification information corresponding to the output information from the teacher information storage unit 18c.
Then, the evaluation value is calculated by comparing the output information and the teacher information, and the obtained evaluation value is stored in the evaluation result storage unit 22c for each image identification information of the image and for each child individual information identification information. can.

Also, the evaluation value calculation unit 21c finishes calculating the evaluation values for all child individual information (or child individual information for one generation) stored in the child population storage unit 122c in the individual population storage unit 12c, and performs evaluation. When stored in the result storage unit 22c, the child individual information can be ranked based on these evaluation values, and the obtained ranking information can be stored in the evaluation result storage unit 22c.
Furthermore, the evaluation value calculation unit 21c can perform end determination for determining whether or not to end the image processing by the image processing unit 19c.

Next, the processing procedure of the image processing system of this embodiment will be described with reference to FIG.
First, the inspection target image input unit 13c in the image processing device 1c inputs the inspection target image and stores it in the inspection target image storage unit 14c (step 40). Also, the non-defective product image input unit 15c inputs a non-defective product image and stores it in the non-defective product image storage unit 16c (step 41). , is stored in the teacher information storage unit 18c (step 42). Then, an initial parent population is generated by the individual generation unit 11c and stored in the parent population storage unit 121c (step 43).

At this time, the individual generation unit 11c generates parent individual information by randomly selecting and arranging nodes from the functional unit storage unit 10c, and generates parent individual information in the individual group storage unit 12c for each identification information of the parent individual information. It can be stored in the group storage unit 121c. The parent population in the parent population storage unit 121c is updated by a parent population update process, which will be described later, and the following process is repeatedly performed for the set generation (step 44).

The gene manipulation unit 23c randomly selects parent individual information for generating child individual information from the parent individual group in the parent individual group storage unit 121c. (Step 45). Then, genetic manipulation is executed for the selected parent individual information (step 46). The selection of parent individual information in step 45 may be executed as part of the genetic manipulation in step 46 .

That is, the gene manipulation unit 23c executes elite storage, crossover processing, and mutation processing on the randomly selected parent individual information, generates a plurality of child individual information, and generates a plurality of child individual information. The group is stored in the child population storage unit 122c.

Next, the image processing unit 19c executes image processing for all child individual information in the child individual group storage unit 122c (steps 47 and 48).
At this time, the image processing unit 19c outputs the image of the detection target element in the inspection target image as output information for each child individual information, and stores the image in the output information storage unit 20c.

Next, the evaluation value calculation unit 21c compares the output information and the teacher information for each child individual information to calculate an evaluation value (step 49), and stores the obtained evaluation value in the evaluation result storage unit 22. memorize each.
Then, for all child individual information in the child population storage unit 122c, processing from image processing to evaluation value calculation and storage is repeatedly executed.

Next, the evaluation value calculation unit 21c ranks the child individual information based on the evaluation values stored in the evaluation result storage unit 22c (step 50).
Based on the ranking information of the child individual information in the evaluation result storage unit 22c, the gene manipulation unit 23 randomly selects the child individual information with the best evaluation value and the child individual information selected with probability from the parent population. The parent individual information in the parent individual group storage unit 121c in the individual group storage unit 12 is updated by replacing it with the parent individual information selected in (step 51).

The evaluation value calculation unit 21c also determines whether or not the termination condition is satisfied (step 52), and if the termination condition is satisfied, the processing by the image processing device 1c is terminated.

When the processing by the image processing device 1c is terminated, the evaluation value calculation unit 21c calculates evaluation values for all parent individual information in the final generation, ranks the individual information based on the evaluation values, and ranks the obtained rankings. Information can be stored in the evaluation result storage unit 22c.

According to the image processing apparatus of this embodiment, genetic manipulation is executed when a parent is selected from the parent population and child individual information is generated, and the child individual information is used as the parent individual information in the parent population. By exchanging, the parent population can be updated. As a result, compared to the first embodiment, it is possible to search for optimal individual information while maintaining evolutionary diversity.

Further, by combining the configuration of the second embodiment with the image processing system according to the first embodiment of the present invention, it is possible to reproduce the peripheral vision in the image processing system, and the detection target in the inspection target image is used as teacher information. It is also preferable to employ a configuration using an image of the element and a numerical value indicating the number of elements.
In addition, by combining the configuration of the third embodiment with the image processing system according to the first embodiment of the present invention, it is possible to reproduce peripheral vision in the image processing system, and as teacher information, the detection target in the inspection target image It is also preferable to use the image of the element and the positional information.

According to such an image processing system of the present embodiment, for example, when a thick line and a thin line are mixed as detection target elements and fine point noise exists around them, the detection target element When using only the image of , there is a possibility that the fine lines are erased, but when using the number and position information as the teacher information, such an effect is not caused. On the other hand, if the detection target element has a complicated shape and it is important to detect the shape, it may not be possible to detect the shape if the number or position information is used as teacher information. . However, when an image of a detection target element is used as teacher information, such an influence does not occur. Therefore, it is possible to increase the possibility of obtaining better individual information by combining the number and position information of detection target elements and an image as teacher information.

Further, by combining the configuration of the third embodiment with the image processing system according to the second embodiment of the present invention, it is possible to reproduce the peripheral vision in the image processing system, and the detection target in the inspection target image is used as teacher information. It is also preferable to use a numerical value indicating the number of elements and position information.
According to the image processing apparatus of this embodiment, for example, when non-detection target elements such as noise exist around the detection target element, when only position information is used as teacher information, the detection target element and Although there is a possibility that the surrounding non-detection target elements are also extracted at the same time, such an influence does not occur when the number is used as teacher information. On the other hand, when thin portions and thick portions are mixed in the detection target element, and when the number is used as teacher information, if the thin portions are not detected, the thick portions are detected and the evaluation value greatly deteriorates. However, when position information is used as teacher information, there is no such influence. Therefore, it is possible to increase the possibility of obtaining better individual information by combining position information and the number of detection target elements as teacher information.

Further, by combining the configurations of the second embodiment and the third embodiment with the image processing system according to the first embodiment of the present invention, it is possible to reproduce peripheral vision in the image processing system, and as teacher information, an examination It is also preferable to use the image of the detection target element in the target image, the numerical value indicating the number, and the position information.
According to such an image processing apparatus of the present embodiment, for example, when only an image is used as teacher information for a detection target element whose border is not clear, the border of the image of the given teacher information is determined by a person. Since it is an object, there is a possibility that a difference will occur due to the decision made by the person, and that high-quality learning will not be possible. However, if the position information or the number is used as the teacher information, such an influence will not occur. Therefore, it is possible to increase the possibility of obtaining better individual information by combining the position information, the number, and the image of the detection target element as teacher information.

Furthermore, by combining the configuration of the second embodiment or the third embodiment with the image processing system according to the fourth embodiment of the present invention, the teacher information is changed to a numerical value or position information indicating the number of detection target elements, It is also desirable to be able to search for optimal individual information while preserving evolutionary diversity.

(Test 1)
A test in which image processing for detecting a detection target from an image is optimized using the image processing system according to the embodiment of the present invention will be described below with reference to FIGS. 6 to 20. FIG.
As Example 1, an image processing system according to the fourth embodiment of the present invention is used to automatically generate an algorithm (optimization of individual information) for detecting detection target elements simulating defects in an image to be inspected. rice field.
As Reference Example 1, the image processing system of Example 1 with a part changed was used. Specifically, in Reference Example 1, the algorithm is automatically generated using only the image to be inspected without using the non-defective product image in the non-defective product image storage unit 16c for image processing by the image processing unit 19c.

6 to 12 show nodes stored in the functional unit storage unit. The nodes shown in FIG. 6 perform image processing (noise removal), the nodes illustrated in FIG. 7 perform image processing (brightness correction, combination), and the nodes illustrated in FIG. 8 perform image processing ( The nodes shown in FIG. 9 are for image processing (edge detection), and the nodes shown in FIG. 10 are for image processing (edge detection, frequency filtering). , the nodes shown in FIG. 11 perform image processing (four arithmetic operations, fuzzy operations), and the nodes shown in FIG. 12 perform detection target determination processing.

The nodes set in the first embodiment are 2 input nodes, 5 processing nodes, and 1 output node, for a total of 8 nodes. The nodes set in Reference Example 1 are a total of 7 nodes, including 1 input node, 5 processing nodes, and 1 output node. An input node is for inputting an image to be inspected or a non-defective product image, and an output node is for outputting an image of an element to be detected. The five processing nodes indicate the number of nodes randomly selected from the functional unit storage section 10c by the individual generation section 11c.

The learning image displaying the inspection target input by the inspection target image input unit and the verification image are detected as an image (background image) in which the background is randomly given a luminance of 0 to 255 for each pixel. It is an image in which the target element is described and noise is added.
A non-defective product image correctly displaying the inspection object input by the non-defective product image input unit uses the above background image as a non-defective product simulation image.
In the first embodiment and the first reference example, settings other than the image input processing are the same.

In both Example 1 and Reference Example 1, the number of input images used in learning was ten, and the number of input images used in verification was five.

For both Example 1 and Reference Example 1, the mean square error was calculated by the least squares method based on the difference for each pixel using the output information and the teacher information which are images.

The initial number of pieces of parent individual information generated by the individual generation unit is set to 100, two pieces of parent individual information are randomly selected and deleted from the parent individual group, and genetic manipulation is performed on these pieces of parent individual information. As a result, information on 72 offspring individuals was generated (individuals identical to the parent: 2, uniform crossover individuals: 20, mutant individuals: 50).

Further, image processing is performed to select two pieces of child individual information, the child individual information with the best evaluation value based on the evaluation result of the child individual information and the child individual information selected by the probability, and select these child individual information. was returned to the parent population as parent individual information, and the above processing was repeated for the set number of generations.
When the maximum number of generations was set to 10000 generations and it was determined that evolution had stopped (when the evaluation value became 0), the processing was terminated.

The results are shown in FIGS. 13 to 20. FIG. 13 and 14 are diagrams showing learning and its results in Test 1, and FIG. 15 is a diagram showing verification results in Test 1. FIG. Table 1 shows the evaluation values of the verification results in Test 1.

FIG. 16 is a diagram showing a graph representing the transition of the learning evaluation value in Test 1. In FIG.
Furthermore, FIG. 17 is a diagram showing the structure of the individual information obtained by the learning of Example 1, and FIG. 18 is a diagram showing the functions of the nodes used in the individual information obtained by the learning of Example 1. is. 19 is a diagram showing the structure of the individual information obtained by the learning of Reference Example 1, and FIG. 20 is a diagram showing the functions of the nodes used in the individual information obtained by the learning of Reference Example 1. is. Note that squares in the individual information structure indicate node sets, each of which is one processing node.

In FIGS. 13 to 15, the image shown in "output result" is based on the individual information with the best evaluation value. That is, the individual information used to output these output information is the individual information finally obtained (optimized) in each of the first example and the first reference example.
In the image to be inspected in Example 1 and Reference Example 1 of No. 9 in FIG. 14, since the detection target element is difficult to see, the outline thereof is clearly shown in white. This also applies to No. 9 in FIG. 22 in Test 2.

First, referring to FIG. 16 to confirm the transition of evolution, the final evaluation value of Example 1 was better. In addition, since the evolution peaked out around 5000 generations in Example 1 and around 6000 generations in Reference Example 1, it is presumed that Example 1 evolved more efficiently.
The reason for this is presumed to be that, in Example 1, a background image simulating a non-defective product image is added to the input image, so that the amount of information that can be used by machine learning increases.
Also, actually referring to FIG. 17, in the first embodiment, the process of comparing the inspection target image and the background image is performed at an early stage (2d in the figure). In addition, in Example 1, a simpler structure than in Reference Example 1 is obtained. It is presumed that this is due to the effective use of the background image as described above.

Furthermore, in the verification results shown in FIG. 15, Samples Nos. 4 and 5 in Reference Example 1 respond to noise. On the other hand, Example 1 generally obtained better evaluation values than Reference Example 1 (Table 1). Therefore, according to the image processing system of this embodiment, it is suggested that detection target elements can be detected even for unknown data.

Here, in general, even if the performance is good for the images used in learning, it is unclear whether the performance is good for using the results in real situations (= applying unknown data to the learning results). , and there is a phenomenon called "overfitting" in which the performance of the learning result is good, but the performance is poor when actually used (in other words, it can be said that there is no robustness). Therefore, in this test, performance evaluation (cross-validation) for unknown data is performed by preparing verification images that are not used in learning and evaluating the learning results.
According to this test, the difference between the learning result and the verification result is smaller in Example 1 than in Reference Example 1. Therefore, according to the image processing system of this embodiment, even for unknown data It is considered that it is suggested that detection of the detection target element is possible.

Further, in Example 1, it is considered that the obtained individual information (algorithm) was excellent as a result of taking the difference between the image to be inspected and the non-defective product image. In addition, even if there are multiple types of objects to be inspected, it can be said that the image processing that takes such differences cancels out the differences between them, and it is possible to inspect multiple types at once.
As described above, the image processing system of the present embodiment has a wide range of application and high versatility. It can be applied without regenerating.

(Test 2)
Next, using the image processing system according to the embodiment of the present invention, an experiment for optimizing image processing for detecting a detection target from an image will be described with reference to FIGS. 21 to 28. FIG.
As Example 2, an image processing system that combines the configuration of the second embodiment with the fourth embodiment of the present invention is used to automatically generate an algorithm for detecting detection target elements simulating defects in an inspection target image ( Optimization of individual information) was performed. That is, in test 1, only the image of the detection target element was used as the teacher information, but in this test, the image of the detection target element and the numerical value indicating the number were used together as the teacher information.

As Reference Example 2, the image processing system of Example 2 with a part changed was used. Specifically, in Reference Example 2, the algorithm is automatically generated using only the image to be inspected without using the non-defective product image in the non-defective product image storage unit 16c for the image processing by the image processing unit 19c.

The nodes stored in the functional unit storage are the same as in Test 1.
The nodes set in the second embodiment are eight nodes in total, including two input nodes, five processing nodes, and one output node. The nodes set in Reference Example 2 are a total of 7 nodes, including 1 input node, 5 processing nodes, and 1 output node. An input node is for inputting an image to be inspected or a non-defective product image, and an output node is for outputting an image and a numerical value indicating the number of elements to be detected. The five processing nodes indicate the number of nodes randomly selected from the functional unit storage section 10c by the individual generation section 11c.

The learning image displaying the inspection target input by the inspection target image input unit and the verification image are detected as an image (background image) in which the background is randomly given a luminance of 0 to 255 for each pixel. It is an image in which the target element is described and noise is added.
A non-defective product image correctly displaying the inspection object input by the non-defective product image input unit uses the above background image as a non-defective product simulation image.
In the second embodiment and the second reference example, settings other than the image input processing are the same.

In both Example 2 and Reference Example 2, the number of input images used in learning was 10, and the number of input images used in verification was 5.

In both Example 2 and Reference Example 2, the output information and the teacher information are used to weight the image (difference for each pixel) and the number of elements to be detected, and the sum of these values is obtained. used. Specifically, it was calculated by the following formula.
Mean squared error with teacher image x weight 1.0 + mean squared error with number of teachers x weight 0.1

The initial number of pieces of parent individual information generated by the individual generation unit is set to 100, two pieces of parent individual information are randomly selected and deleted from the parent individual group, and genetic manipulation is performed on these pieces of parent individual information. As a result, information on 72 offspring individuals was generated (individuals identical to the parent: 2, uniform crossover individuals: 20, mutant individuals: 50).

Further, image processing is performed to select two pieces of child individual information, the child individual information with the best evaluation value based on the evaluation result of the child individual information and the child individual information selected by the probability, and select these child individual information. was returned to the parent population as parent individual information, and the above processing was repeated for the set number of generations.
When the maximum number of generations was set to 10000 generations and it was determined that evolution had stopped (when the evaluation value became 0), the processing was terminated.

The results are shown in FIGS. 21 to 28. FIG. 21 and 22 are diagrams showing learning and results in test 2, and FIG. 23 is a diagram showing verification results in test 2. FIG. Table 2 shows the evaluation values of the verification results in Test 2.

FIG. 24 is a diagram showing a graph representing the transition of learning evaluation values in test 2. In FIG.
Furthermore, FIG. 25 is a diagram showing the structure of individual information obtained by learning in Example 2, and FIG. 26 is a diagram showing the functions of nodes used in the individual information obtained by learning in Example 2. is. 27 is a diagram showing the structure of the individual information obtained by the learning of Reference Example 2, and FIG. 28 is a diagram showing the functions of the nodes used in the individual information obtained by the learning of Reference Example 2. is. Note that squares in the individual information structure indicate node sets, each of which is one processing node.

In FIGS. 21 to 23, the image shown in "output result" is based on the individual information with the best evaluation value. That is, the individual information used to output these output information is the individual information finally obtained (optimized) in each of Example 2 and Reference Example 2.

First, referring to FIG. 24 to confirm the transition of evolution, the final evaluation value of Example 2 was better. In addition, since the evolution peaked out around 500 generations in Example 2 and around 3800 generations in Reference Example 2, it is presumed that Example 2 evolved more efficiently as in Test 1.

Furthermore, in the verification results shown in FIG. 23, in Reference Example 2, sample No. 1 responded to noise, and sample No. 5 failed to detect some feature points (detection target elements). On the other hand, Example 2 obtained better evaluation values than Reference Example 2 in all samples (Table 2). Therefore, according to the image processing system of this embodiment, it is suggested that detection target elements can be detected even for unknown data.

Here, when Test 1 and Test 2 are compared, in Test 1, images Nos. 7 and 9 for learning in Reference Example 2, Nos. 4 and 5 for verification images, and No. 5 for learning in Example 2 are used. In 7, the output image is output without noise being removed.
This is because if the teacher information is only the image of the detection target element, removing the noise part will also remove the vicinity of the detection target element boundary, and the evaluation value will deteriorate. Alternatively, there is a trade-off relationship of whether to leave the vicinity of the boundary of the detection target element, and as a result, it is considered that machine learning could not generate a node structure that removes noise.

On the other hand, in Test 2, by using a numerical value indicating the number of detection target elements as teacher information, when elements other than detection target elements such as noise are detected, the evaluation value clearly worsens, but Even if the vicinity of the boundary of the element to be detected is scraped to some extent, the evaluation value is not deteriorated. Therefore, machine learning could generate a node structure that removes noise.

Furthermore, in Test 2, the speed at which the evolution converges is overwhelmingly faster than in Test 1, because numerical values indicating the number of elements to be detected are used as teacher information. It is speculated that the early generation has a structure that outputs an image with a lot of noise and a structure that roughly captures the vicinity of the detection target element.
When only the image of the detection target element is used as teacher information and the number indicating the number is not used, it is presumed that machine learning is chasing two rabbits that are evolving while following both structures. By adding the number of elements to be detected, the evaluation value of an image with a lot of noise deteriorates.

The image processing system of the above embodiments can be realized using a computer controlled by the image processing program of the present invention. The CPU of the computer in the image processing system sends instructions to each component of the computer based on the image processing program, and performs predetermined processing required for the operation of the image processing system, such as individual generation processing, image processing, and evaluation value calculation. Processing, genetic manipulation processing, etc. are performed. As described above, each processing and operation in the image processing system of the present invention can be realized by concrete means in which the program and the computer work together.

The program is stored in advance in a recording medium such as ROM, RAM, etc., and is executed by loading the program into the computer from the recording medium installed in the computer.
Also, the recording medium for storing the program can be composed of, for example, a semiconductor memory, a magnetic disk, an optical disk, or any other computer-readable recording means.

As described above, according to the image processing system and the image processing program according to the embodiments of the present invention, it is possible to further optimize the image processing in the image processing technique for finding the detection target from the image.

It goes without saying that the present invention is not limited to the above embodiments and examples, and that various modifications can be made within the scope of the present invention.
For example, each configuration in the image processing apparatus can be distributed to a plurality of information processing apparatuses, or nodes other than those used in the embodiments can be used.
Further, in the embodiment, detection target elements simulating defects in an image are used as detection target elements, but the detection target elements are not particularly limited as long as they are images. An image or the like can also be targeted.

INDUSTRIAL APPLICABILITY The present invention can be suitably used to obtain an information processing apparatus for image inspection in which image processing is optimized when inspection is performed based on image data of equipment, products, etc. .

1, 1c image processing device 10, 10c functional unit storage unit 11, 11c individual generation unit 12, 12c population storage unit 121c parent population storage unit 122c child population storage unit 13, 13c inspection target image input unit 14, 14c inspection Target image storage units 15, 15c Non-defective product image input units 16, 16c Non-defective product image storage units 17, 17c Teacher information input units 18, 18c Teacher information storage units 19, 19c Image processing units 20, 20c Output information storage units 21, 21c Evaluation values Calculation unit 22, 22c Evaluation result storage unit 23, 23c Gene manipulation unit

Claims (9)

An image processing system that optimizes image processing for detecting a detection target from an image by genetic manipulation, which is an optimization method that mathematically simulates the evolution of living things,
a population storage unit that stores one or more pieces of individual information that define the order of execution of a plurality of functional units in a network that can include cycles;
an inspection target image input unit for inputting an inspection target image for learning or verification in which an inspection target is displayed;
a non-defective product image input unit for inputting a non-defective product image in which the inspection object is correctly displayed;
an image processing unit that processes the inspection target image based on the individual information and the non-defective product image, and generates output information including at least one of image, number, and position information of detection target elements in the inspection target image;
a teacher information input unit for inputting teacher information including information on detection target elements for evaluating the output information;
an evaluation value calculation unit that compares the output information and the teacher information to calculate an evaluation value, and ranks the individual information based on the evaluation value;
a gene manipulation unit that selects or changes the individual information based on the ranking information of the individual information and updates the population storage unit ;
The inspection target image input unit inputs an image obtained by photographing the inspection target in real time as an inspection image,
The non-defective product image input unit inputs in real time an image obtained by photographing another inspection object of the same type as the inspection object as the non-defective product image.
An image processing system characterized by: The processing by the image processing unit includes processing for creating an image of a detection target element in the inspection target image,
2. The image processing system according to claim 1, wherein said teacher information and said output information include an image of a detection target element in said inspection target image. The processing by the image processing unit includes processing for counting detection target elements in the inspection target image,
3. The image processing system according to claim 1, wherein the teacher information and the output information include numerical values indicating the number of detection target elements in the inspection target image. The processing by the image processing unit includes processing for acquiring position information of the detection target element in the inspection target image,
4. The image processing system according to claim 1, wherein said teacher information and said output information are position information of detection target elements in said inspection target image. The processing by the image processing unit includes processing for calculating a feature amount of a certain range in the image, and processing for determining whether or not the certain range is an element to be detected based on the feature amount. The image processing system according to any one of claims 1 to 4, wherein: The gene manipulation unit
an elite storage unit that leaves the individual information with the high evaluation value to the next generation;
a selection unit for leaving individual information to the next generation with a certain probability based on the evaluation value;
a crossover processing unit that mutually exchanges a part of two pieces of individual information;
6. The image processing system according to any one of claims 1 to 5, further comprising a mutation unit that randomly rewrites part or all of the selected individual information. 7. The image processing system according to any one of claims 1 to 6, wherein said network is a directed network composed of functional units and oriented links, and includes a linear structure and a tree structure. The inspection target image input unit inputs an inspection image, and outputs processing result information obtained by processing the inspection image by the image processing unit based on the individual information with the highest evaluation value and the non-defective product image. 8. The image processing system according to any one of claims 1 to 7, further comprising a processing result storage unit for storing. An image processing program that optimizes image processing for detecting a detection target from an image by genetic manipulation, which is an optimization method that mathematically simulates biological evolution,
the computer,
an individual group storage unit that stores one or more pieces of individual information that define the order of execution of a plurality of functional units in a network that can include cycles;
an inspection target image input unit for inputting an inspection target image for learning or verification in which an inspection target is displayed;
a non-defective product image input unit for inputting a non-defective product image in which the inspection object is correctly displayed;
an image processing unit that processes the image to be inspected based on the individual information and the non-defective product image, and generates output information including at least one of image, number, and position information of elements to be detected in the image to be inspected;
a teacher information input unit for inputting teacher information including information on detection target elements for evaluating the output information;
an evaluation value calculation unit that compares the output information and the teacher information to calculate an evaluation value and ranks the individual information based on the evaluation value;
Selecting or changing the individual information based on the ranking information of the individual information to function as a genetic manipulation unit for updating the population storage unit ;
causing the inspection target image input unit to input an image obtained by photographing the inspection target in real time as the inspection image;
The non-defective product image input unit is caused to input in real time, as the non-defective product image, an image obtained by photographing another inspection object of the same type as the inspection object.
An image processing program characterized by:

Images