JP7314711B2 - Image processing system and image processing program - Google Patents

Description

The present invention relates to an image processing technique for finding a detection target from an image, and more particularly to an image processing system for optimizing image processing.

Conventionally, in image processing technology for finding detection targets from an image, a method of optimizing image processing is performed by comparing an output image obtained by performing image processing on an input image with a teacher image and calculating an evaluation value.
In this method, there are problems that the evaluation value may deteriorate based on the ratio of the detection target area and the non-detection target area in the image, it takes time to generate the teacher image, and excessive human intervention is required for computer optimization.

Japanese Patent No. 5479944 Japanese Patent No. 4862150 Japanese Patent No. 5011533 JP 2008-15817 A

Here, as an image processing technique for finding a detection target from an image, the method described in Patent Document 1 can be mentioned.
This method introduces size-dependent crossover into a parallel image filter automatic generation system by genetic programming (GP). As a result, by selecting and adopting images with cracks from multiple actual pavement images as training data for filter construction, image filters for extracting cracks are automatically constructed from various types of images, and damage levels are evaluated using these.

In addition, the evolutionary computing systems described in Patent Documents 2 to 4 can be cited as techniques using such so-called evolutionary computation. These evolutionary computing systems make it possible to optimize the processing algorithm in each system.

However, even when these techniques are used, when optimizing image processing by comparing an output image obtained by performing image processing on an input image with a teacher image, as described above, there are problems that the evaluation value deteriorates depending on the ratio of the detection target area, generation of the teacher image is troublesome, and excessive human intervention is required.
For example, when comparing the output image with the teacher image and evaluating the difference in luminance for each pixel using the method of least squares, etc., there was a large difference in the evaluation value based on the size of the detection target even in the same non-detection case.

The inventors of the present invention believed that the effect of the size of the object to be detected on the evaluation value was an obstacle to optimizing the image processing, and performed image processing on the input image to create output information indicating the number of objects to be detected. In addition, in evolutionary computation, by adopting the technique of genetic network programming (GNP), it is possible to improve the degree of freedom of optimization, thereby completing the present invention.

That is, an object of the present invention is to provide an image processing system and an image processing program capable of further optimizing image processing in image processing technology for finding a detection target from an image.

In order to achieve the above object, the image processing system of the present invention is an image processing system that optimizes image processing for detecting a detection target from an image using genetic manipulation, which is an optimization technique that mathematically simulates the evolution of living organisms, comprising: a population storage unit for storing one or more pieces of individual information defining the execution order of a plurality of functional units in a network that can include a cycle; an image input unit for inputting images for learning or verification; a teacher information input unit for inputting teacher information for evaluating output information; an image processing unit that processes the image based on the individual information and generates the output information; an evaluation value calculation unit that compares the output information with the teacher information to calculate an evaluation value and ranks the individual information based on the evaluation value; and a gene operation unit that selects or changes the individual information based on the ranking information of the individual information to update the population storage unit, wherein processing by the image processing unit includes counting detection targets in the image, The output information is a numerical value indicating the number of detection targets in the image.

Further, in the image processing system of the present invention, it is preferable that the processing by the image processing unit includes processing for creating a detection target image representing the detection target, and the teacher information and the output information include the detection target image representing the detection target.

Further, the image processing program of the present invention is an image processing program for optimizing image processing for detecting a detection target from an image using genetic manipulation, which is an optimization technique that mathematically simulates the evolution of living organisms, wherein the computer comprises a population storage unit for storing one or more pieces of individual information that define the execution order of a plurality of functional units in a network that can include a cycle, an image input unit for inputting images for learning or verification, a teacher information input unit for inputting teacher information for evaluating output information, and a teacher information input unit for inputting teacher information for evaluating output information. an evaluation value calculation unit that compares the output information with the teacher information to calculate an evaluation value and ranks the individual information based on the evaluation value; and a gene operation unit that selects or changes the individual information based on the ranking information of the individual information and updates the population storage unit, and the processing by the image processing unit includes counting detection targets in the image, and It is configured to be a numerical value indicating the number of detection targets in the image.

In the image processing program of the present invention, the processing by the image processing unit may include processing for creating a detection target image representing the detection target, and the teacher information and the output information may preferably include the detection target image representing the detection target.

Advantageous Effects of Invention According to the present invention, it is possible to provide an image processing system and an image processing program capable of further optimizing image processing in an image processing technique for finding a detection target from an image.

1 is a block diagram showing the configuration of an image processing apparatus according to a first embodiment of the present invention; FIG. 4 is a flow chart showing processing procedures by the image processing apparatus according to the first embodiment of the present invention; 4 is a flow chart showing a genetic manipulation processing procedure in the processing procedure of the image processing apparatus according to the first embodiment of the present invention; It is a block diagram showing the configuration of the image processing apparatus of the second embodiment of the present invention. 8 is a flow chart showing a processing procedure performed by the image processing apparatus according to the second embodiment of the present invention; FIG. 4 is an explanatory diagram showing functions (image processing: noise removal) of nodes used in the example and the reference example; FIG. 4 is an explanatory diagram showing functions of nodes (image processing: brightness correction, combination) used in the example and the reference example; FIG. 4 is an explanatory diagram showing functions of nodes (image processing: binarization) used in the embodiment and the reference example; FIG. 4 is an explanatory diagram showing functions of nodes (image processing: edge detection) used in the embodiment and the reference example; FIG. 4 is an explanatory diagram showing functions of nodes (image processing: edge detection, four arithmetic operations) used in the example and the reference example; FIG. 4 is an explanatory diagram showing the function of a node (detection target determination) used in the embodiment and the reference example; It is explanatory drawing (No, 1-4) which compared the learning by an Example and a reference example, and the result. It is explanatory drawing (No, 5-8) which compared the learning by an Example and a reference example, and the result. FIG. 10 is an explanatory diagram (No, 1-3) comparing verification results of an example and a reference example; FIG. 10 is a diagram showing a transition graph of normalized evaluation values in learning according to an example and a reference example; It is a figure which shows the structure of the individual information obtained by the learning of an Example. FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning in the example. FIG. 10 is a diagram showing the structure of individual information obtained by learning in a reference example; FIG. 10 is a diagram showing functions of nodes used in individual information obtained by learning in a reference example;

Embodiments of an image processing system and an image processing program according to the present invention will be described in detail below. However, the present invention is not limited to the specific contents of the following embodiments and examples described later.

[First embodiment]
First, an image processing system and an image processing program according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG. FIG. 1 is a block diagram showing the configuration of an image processing apparatus corresponding to the image processing system of this embodiment, and FIG. 2 is a flow chart showing the processing procedure of the image processing apparatus. FIG. 3 is a flow chart showing the procedure of gene manipulation in the procedure of the image processing apparatus.

The image processing system of this embodiment optimizes image processing for detecting a detection target from an image using genetic manipulation, which is an optimization technique that mathematically simulates the evolution of living things.
Specifically, as shown in FIG. 1, the image processing apparatus 1 of the present embodiment includes a functional unit storage unit 10, an individual generation unit 11, an individual group storage unit 12, an image input unit 13, an image storage unit 14, a teacher information input unit 15, a teacher information storage unit 16, an image processing unit 17, an output information storage unit 18, an evaluation value calculation unit 19, an evaluation result storage unit 20, and a gene manipulation unit 21.
All of these components in the image processing apparatus 1 of this embodiment can be provided in one information processing apparatus as shown in FIG. Also, each of these components may be distributed to each device of an image processing system comprising a plurality of information processing devices. This also applies to the second embodiment, which will be described later.

The functional unit storage unit 10 stores, for each node number, a node (corresponding to a gene), which is a functional unit that executes a certain process. In this embodiment, various nodes for performing image processing can be used as this node, and the nodes shown in FIGS. 6 to 11 are used in the examples described later.
Further, these nodes are stored in the functional unit storage unit 10 together with parameters used for execution of the respective processes, if any.

Furthermore, a node that does not execute any process may be used, or a functional unit that executes a plurality of processes may be used as one node (node set). By using nodes that do not perform any processing as nodes, it is possible to reduce the actual number of nodes when, for example, the set value of the initial number of nodes for generating individual information is too large.

In this specification, "image processing" includes various filter processing for noise removal, correction processing such as brightness correction, binarization processing, edge detection processing, and narrowly defined image processing such as four arithmetic operations, as well as detection target determination processing that can be executed along with this, output processing, and the like.

The individual generation unit 11 uses the nodes stored in the functional unit storage unit 10 to generate individual information consisting of an array of a plurality of nodes. At this time, the individual generation unit 11 can randomly generate a plurality of pieces of individual information based on the plurality of nodes.
At this time, in the individual information, the individual generation unit 11 can define the execution order of the plurality of nodes in a network that can include a cycle. The network is a directed network composed of oriented links and may include feedback. The network can also include linear structures and tree structures.

Using such individual information makes it possible to perform learning including a network structure with more flexible expressive power compared to using individual information consisting only of a linear structure or a tree structure. In other words, it is difficult for humans to determine in advance which structure will be the optimal solution for the problem, and therefore it is difficult to determine the conditions of the structure prior to machine learning. For this reason, searching for optimal solutions under a wide range of conditions using individual information that defines the execution order of multiple nodes in a network that can include cycles is considered effective in increasing the possibility of deriving true optimal solutions.

The individual group storage unit 12 stores an individual group composed of a plurality of pieces of individual information generated by the individual generation unit 11 . One generation or a plurality of generations can be stored in the population storage unit 12 as this population, and individual information can be stored in the population storage unit 12 for each generation number.
The evaluation value calculation unit 19 to be described later can rank the individual information for each generation and store the ranking information in the evaluation result storage unit 20 .

The image input unit 13 inputs learning or verification images to the image processing apparatus 1 and stores them in the image storage unit 14 for each image identification information.
The teacher information input unit 15 inputs teacher information for evaluating the output information in association with the image identification information, and stores the teacher information storage unit 16 for each image identification information. In this embodiment, the teacher information is a numerical value indicating the number of detection targets in the image corresponding to the image identification information.
A detection target (detection target element) can be a defect (crack, scratch, stain, etc.) in an image, a specific part, a specific pattern, or the like. Note that the detection target may correspond to the entire image.

Here, when the teacher information is an image, the evaluation value is strongly affected by the size of the detection target in the evaluation by the least squares method. Therefore, the evaluation value is more likely to deteriorate when the shape of the large detection target is different than when there is no small detection target.
For example, assume that a triangle with a size of 10 pixels is not detected when triangles are to be detected in an image.
At this time, if the output of a larger triangle in the same image is slightly smaller than that of the teacher image, if the difference is 10 pixels or more, the evaluation value will deteriorate due to the absence of a 10-pixel triangle. For this reason, in machine learning, the evaluation value tends to improve when triangles that are larger than small triangles are brought closer to the teacher image, and evolution progresses to match triangles that are larger than small triangles, resulting in a state where it is easy to fall into a local optimum. On the other hand, in the present embodiment, undetected errors are the same regardless of the size of the detection target, so that local minima are less likely to occur.

The image processing unit 17 receives the image from the image storage unit 14 and the individual information from the individual group storage unit 12, and sequentially executes a plurality of nodes included in the individual information based on the individual information.
At this time, each node uses information obtained from the image to execute processing corresponding to its function. Also, each node can selectively execute the next node corresponding to the processing result based on the processing result. Therefore, even nodes included in individual information are not necessarily executed in image processing. Of course, there is a case where all nodes in individual information are executed. Moreover, a node defined only once in one individual information may be executed multiple times due to feedback.

As described above, the functions of the node include narrowly defined image processing such as various image filter processing for noise removal, correction processing such as luminance correction, binarization processing, edge detection processing, and four arithmetic operations. These are mainly performed by the image processing unit 17 as processing for finding detection target candidates.

The detection target determination process as a function of the node is performed by the image processing unit 17 by labeling the detection target candidates in a certain range in the image, calculating the feature amount, and determining whether or not the detection target candidate having the feature amount is the correct detection target based on a threshold or the like.
In the detection target determination process, it is also possible to label the non-detection target candidate, calculate its feature amount, and determine that it is not a detection target by not having the feature amount based on a threshold value or the like.

Furthermore, the image processing unit 17 generates and outputs output information.
Specifically, the output information is generated by counting the detection targets in the image. Then, this output information can be stored in the output information storage unit 18 for each image identification information of the image and for each identification information of individual information.
Thus, the output information obtained by the image processing unit 17 in this embodiment is generated as a numerical value indicating the number of detection targets in the image.

Note that the evaluation value calculation unit 19 can also be made to acquire a numerical value indicating the number of detection targets in the image. That is, the evaluation value calculation unit 19 may acquire a numerical value indicating the number of detection targets based on the image after image processing by the image processing unit 17, and calculate the evaluation value based on this.

The evaluation value calculation unit 19 receives the output information from the output information storage unit 18 and inputs the teacher information corresponding to the image identification information corresponding to the output information from the teacher information storage unit 16 .
Then, an evaluation value is calculated by comparing the output information and the teacher information, and the obtained evaluation value can be stored in the evaluation result storage unit 20 for each image identification information of the image and for each individual identification information.
At this time, the evaluation value calculation unit 19 can use the output information and the teacher information to calculate the mean square error as the evaluation value by the method of least squares.

In addition, when evaluation values for all individual information (or individual information for one generation) stored in the population storage unit 12 have been calculated and stored in the evaluation result storage unit 20, the evaluation value calculation unit 19 can rank the individual information based on these evaluation values and store the obtained ranking information in the evaluation result storage unit 20.

Furthermore, the evaluation value calculation unit 19 can perform termination determination for determining whether or not to terminate the image processing by the image processing unit 17 . The termination determination can be made based on whether or not the termination condition is satisfied. If the termination condition is satisfied, the image processing by the image processing unit 17 can be terminated, and the processing by the image processing apparatus 1 can be terminated. Then, the individual information with the highest evaluation value stored in the evaluation result storage unit 20 is obtained as the individual information optimized for image processing.

Examples of termination conditions include the case where image processing has been completed for populations of all generations set in advance, and the case where evolution has not occurred in a certain number of generations or more.
A case in which no evolution has occurred is a case in which the evaluation value has not become higher than at the point of comparison. In the embodiment described later, since the mean square error is used as the evaluation value, the smaller the evaluation value, the better the individual information. Therefore, if the image processing is performed for a certain number of generations and the evaluation value does not decrease, it can be determined that the image has not evolved.

Note that the end determination function may be separated from the evaluation value calculation unit 19, and an end determination unit for performing end determination may be provided in the image processing apparatus 1 of the present embodiment.

The gene manipulation unit 21 selects or changes the individual information based on the ranking information of the individual information in the evaluation result storage unit 20 and updates the population storage unit 12 .
Specifically, the gene manipulation unit 21 can include an elite individual storage unit that leaves individual information with a high evaluation value to the next generation, an individual selection unit that leaves individual information for the next generation with a certain probability based on the evaluation value, a crossover unit that mutually exchanges a part of two pieces of individual information, and a mutation unit that randomly rewrites part or all of the selected individual information.

The elite individual storage unit can unconditionally leave the individual information with the best evaluation value in the population of each generation to the next generation.
The individual selection unit can perform a process of selecting individual information with a high ranking of evaluation values in a group of individuals in each generation with a high probability, and leaving the information to the next generation.

As crossover processing, the crossover unit can perform one-way crossover in which the same (node number) sequence in the array of each node of a plurality of pieces of individual information is replaced with a certain probability. In addition, the crossover unit can perform crossover processing such as one-point crossover in which one array is exchanged with one point of the same arrangement in the array of each node as a boundary, or multiple-point crossover in which the arrays are exchanged with each other with a plurality of points of the same arrangement as a boundary. The crossover section can perform such crossover processing on the individual information selected by the individual selection section.

As mutation processing, the mutation section can rewrite part of the individual information or rewrite all of the individual information with a certain probability for each individual information using the nodes stored in the functional unit storage section 10.例文帳に追加The mutation section can perform such mutation processing on the individual information selected by the individual selection section. In addition, the mutating section can also generate new individual information using the nodes stored in the functional unit storage section 10 as mutation processing.

If the number of same-line individuals generated based on the same individual information increases, diversity will be lost and evolution will enter a dead end (a state of mathematically falling into a local optimum). On the other hand, if the number of individuals other than the same lineage is increased too much, it becomes more like a random search and efficient evolution does not occur.
Therefore, in the early stage of learning, it is preferable to increase the number of individuals other than the same lineage in order to increase diversity, and to increase the same lineage in the middle stage of learning by the individual selection section in order to carry out efficient evolution. In addition, when evolution stops, it is possible to preferably increase non-same lineages by means of mutation sites in order to restore diversity.

Then, the gene manipulation unit 21 generates a new generation population and stores it in the population storage unit 12 . At this time, the gene manipulation unit 21 can update the population storage unit 12 by overwriting the population of the previous generation. In addition, the population storage unit 12 can be updated by storing populations of multiple generations for each generation.

Next, a processing procedure by the image processing apparatus of this embodiment will be described with reference to FIGS. 2 and 3. FIG.
First, the image input section 13 in the image processing apparatus 1 inputs an image (step 10). Also, the teacher information input unit 15 inputs numerical values as teacher information (step 11). Then, an initial population of individuals is generated by the individual generation unit 11 (step 12).
At this time, the individual generation unit 11 can generate individual information by randomly selecting and arranging nodes from the functional unit storage unit 10, and can store the individual information in the individual group storage unit 12 for each identification information of the individual information. Furthermore, the individual generation unit 11 can also store a group of individuals consisting of a plurality of pieces of individual information as one generation, and store groups of individuals in a plurality of generations for each generation number in the group storage unit 12 .

Next, the image processing unit 17 performs image processing on all individual information in the individual group for each generation of the individual group in the individual group storage unit 12 (steps 13, 14, 15).
At this time, the image processing unit 17 outputs a numerical value indicating the number of detection targets in the image as output information for each piece of individual information, and stores the numerical value in the output information storage unit 18 .

Next, the evaluation value calculator 19 compares the output information and the teacher information for each piece of individual information to calculate an evaluation value (step 16).
At this time, in the calculation of the evaluation value by the evaluation value calculation unit 19, for example, the least squares method can be used to calculate the mean square error as the evaluation value.
In addition, the evaluation value calculation unit 19 stores the obtained evaluation values in the evaluation result storage unit 20 for each piece of individual information.
Then, the processing from image processing to evaluation value calculation and storage is repeatedly executed for all pieces of individual information of the generation.

Next, the evaluation value calculation unit 19 ranks the individual information based on the evaluation values stored in the evaluation result storage unit 20 (step 17).
Further, the evaluation value calculation unit 19 performs determination processing as to whether or not the termination condition is satisfied (step 18). If the termination condition is satisfied, the processing by the image processing apparatus 1 is terminated.

If the end condition is not satisfied, the gene manipulation unit 21 executes the gene manipulation process (step 19).
Specifically, as shown in FIG. 3, the elite individual storage unit in the gene manipulation unit 21 selects the individual information with the highest evaluation value as the individual information of the next-generation population (step 20).
Further, the individual selecting unit in the gene manipulation unit 21 selects the individual information having a higher evaluation value as the individual information of the population of the next generation with a higher probability (step 21).

Further, the crossover unit in the gene manipulation unit 21 performs crossover processing on the individual information selected by the individual selection unit to generate new individual information (step 22).
Also, the mutation section in the gene manipulation section 21 performs mutation processing based on the individual information selected by the individual selection section or without using such individual information to generate new individual information (step 23).
The order of each processing by the elite individual storage unit, individual selection unit, crossover unit, and mutation unit may be changed within the range of the order in which each processing can be executed.

According to the image processing apparatus of this embodiment, in the image processing technology for finding detection targets from an image, image processing is performed on an input image to create output information indicating the number of detection targets, and the output information can be compared using numerical values as teacher information. Therefore, the same evaluation value can be obtained regardless of the size of the detection target in the image, and the problem that the evaluation value deteriorates depending on the ratio of the detection target region can be solved. In addition, it is possible to solve the conventional problem that the generation of the teacher image required time and effort and the problem that excessive human intervention was required.

[Second embodiment]
Next, an image processing system and an image processing program according to a second embodiment of the invention will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 is a block diagram showing the configuration of an image processing apparatus corresponding to the image processing system of this embodiment, and FIG. 5 is a flow chart showing the processing procedure of the image processing apparatus.

As shown in FIG. 4, the image processing apparatus 1a of this embodiment includes a functional unit storage unit 10a, an individual generation unit 11a, an individual group storage unit 12a, an image input unit 13a, an image storage unit 14a, a teacher information input unit 15a, a teacher information storage unit 16a, an image processing unit 17a, an output information storage unit 18a, an evaluation value calculation unit 19a, an evaluation result storage unit 20a, and a gene manipulation unit 21a. The population storage unit 12a includes a parent population storage unit 121a and a child population storage unit 122a.

The functional unit storage unit 10a, the image input unit 13a, the image storage unit 14a, the teacher information input unit 15a, the teacher information storage unit 16a, the output information storage unit 18a, and the evaluation result storage unit 20a in the image processing apparatus 1a of the present embodiment can be the same as those in the first embodiment. Also, other configurations of the image processing apparatus 1a of the present embodiment can be the same as those of the first embodiment except for the points described below.

The individual generating unit 11a uses the nodes stored in the functional unit storage unit 10 to generate individual information consisting of an array of a plurality of nodes. At this time, the individual generation unit 11a can define the execution order of a plurality of nodes in the individual information in the form of a network that can include a cycle, as in the first embodiment.
In addition, the individual generation unit 11a in this embodiment can randomly generate a plurality of pieces of parent individual information as individual information, and store an initial parent individual group made up of the parent individual information in the parent individual group storage unit 121a.

The gene manipulation unit 21a executes gene manipulation on the parent individual information in the parent individual group storage unit 121a.
Specifically, first, parent individual information for generating child individual information is randomly selected from the parent individual group in the parent individual group storage unit 121a.

The elite individual storage unit in the gene manipulation unit 21a can store a child population consisting of the same child individual information as the selected parent individual information in the child population storage unit 122a.
Then, the crossover unit in the gene manipulation unit 21a executes crossover processing on the selected parent individual information, generates a plurality of child individual information, and stores a child population consisting of these child individual information in the child population storage unit 122a.
Further, the mutation unit in the gene manipulation unit 21a can execute mutation processing on the selected parent individual information using the nodes stored in the functional unit storage unit 10, and can store in the child population storage unit 122a a child population consisting of child individual information obtained by rewriting part of the parent individual information with a certain probability or by rewriting all of the parent individual information. In addition, the mutation section can also generate new individual information using the nodes stored in the functional unit storage section 10a as the mutation processing.

Furthermore, the gene manipulation unit 21a can update the parent individual information in the parent individual group storage unit 121a by, for example, replacing the child individual information with the best evaluation value and the child individual information selected with probability with the parent individual information randomly selected from the parent individual group based on the ranking information of the child individual information in the evaluation result storage unit 20a.

The image processing unit 17a receives the image from the image storage unit 14a and the child individual information from the child population storage unit 122a, and sequentially executes a plurality of nodes included in the child individual information based on the child individual information. Then, the image processing unit 17a counts the detection targets in the image to generate output information, and stores this output information in the output information storage unit 18a for each image identification information of the image and for each identification information of offspring individual information.

The evaluation value calculation unit 19a receives the output information from the output information storage unit 18a, and also receives the teacher information corresponding to the image identification information corresponding to the output information from the teacher information storage unit 16a.
Then, the output information and the teacher information are compared to calculate an evaluation value, and the obtained evaluation value can be stored in the evaluation result storage unit 20a for each image identification information of the image and for each child individual information identification information.

Further, when the evaluation value calculation unit 19a finishes calculating the evaluation values for all child individual information (or child individual information for one generation) stored in the child individual group storage unit 122a in the individual group storage unit 12a and stores them in the evaluation result storage unit 20a, the evaluation value calculation unit 19a can rank the child individual information based on these evaluation values, and store the obtained ranking information in the evaluation result storage unit 20a.
Furthermore, the evaluation value calculation unit 19a can perform end determination for determining whether or not to end the image processing by the image processing unit 17a.

Next, the processing procedure of the image processing apparatus of this embodiment will be described with reference to FIG.
First, the image input unit 13a in the image processing device 1a inputs an image (step 30). Also, the teacher information input unit 15a inputs numerical values as teacher information (step 31). Then, an initial parent population is generated by the individual generation unit 11a (step 32).

At this time, the individual generation unit 11a can generate parent individual information by randomly selecting and arranging nodes from the functional unit storage unit 10a, and can store the parent individual information in the parent individual group storage unit 121a in the individual group storage unit 12a for each identification information of the parent individual information. The parent population in the parent population storage unit 121a is updated by a parent population update process, which will be described later, and the following process is repeatedly performed for the set generation (step 33).

The gene manipulation unit 21a randomly selects parent individual information for generating child individual information from the parent individual group in the parent individual group storage unit 121a. (Step 34). Then, genetic manipulation is executed for the selected parent individual information (step 35). The selection of parent individual information in step 34 may be executed as part of the genetic manipulation in step 35 .

That is, the gene manipulation unit 21a executes crossover processing and mutation processing on randomly selected parent individual information to generate a plurality of child individual information, and stores a child population consisting of these child individual information in the child population storage unit 122a.

Next, the image processing unit 17a executes image processing for all child individual information in the child individual group storage unit 122a (steps 36 and 37).
At this time, the image processing unit 17a outputs a numerical value indicating the number of detection targets in the image as output information for each child individual information, and stores the numerical value in the output information storage unit 18a.

Next, the evaluation value calculation unit 19a compares the output information and the teacher information for each piece of child individual information to calculate an evaluation value (step 38), and stores the obtained evaluation value in the evaluation result storage unit 20 for each piece of child individual information.
Then, for all child individual information in the child population storage unit 122a, processing from image processing to evaluation value calculation and storage is repeatedly executed.

Next, the evaluation value calculation unit 19a ranks the child individual information based on the evaluation values stored in the evaluation result storage unit 20a (step 39).
Then, based on the ranking information of the child individual information in the evaluation result storage unit 20a, the gene manipulation unit 21 replaces the child individual information with the best evaluation value and the child individual information selected with probability with the parent individual information randomly selected from the parent individual group, thereby updating the parent individual information in the parent individual group storage unit 121a in the population storage unit 12 (step 40).

The evaluation value calculator 19a also determines whether or not the termination condition is satisfied (step 41), and if the termination condition is satisfied, the processing by the image processing apparatus 1a is terminated.

When the processing by the image processing device 1a is terminated, the evaluation value calculation unit 19a can calculate evaluation values for all pieces of individual information in the final generation, rank the pieces of individual information based on the evaluation values, and store the obtained ranking information in the evaluation result storage unit 20a.

According to the image processing apparatus of this embodiment, genetic manipulation is executed when a parent is selected from the parent population and child individual information is generated, and the parent population can be updated by replacing the child individual information with the parent individual information in the parent population. As a result, compared to the first embodiment, it is possible to search for optimal individual information while maintaining evolutionary diversity.

[Third embodiment]
Next, an image processing system and an image processing program according to the third embodiment of the invention will be described. The image processing system of the present embodiment differs from the first embodiment in that a numerical value indicating the number of detection targets in an image and images of detection targets in the image (hereinafter referred to as detection target images) are used as teacher information. Other points are the same as in the first embodiment.

That is, although not shown, the image processing apparatus 1c corresponding to the image processing system of the present embodiment includes a functional unit storage unit 10c, an individual generation unit 11c, an individual group storage unit 12c, an image input unit 13c, an image storage unit 14c, a teacher information input unit 15c, a teacher information storage unit 16c, an image processing unit 17c, an output information storage unit 18c, an evaluation value calculation unit 19c, an evaluation result storage unit 20c, and a gene manipulation unit 21c.

In the image processing apparatus 1c of the present embodiment, the training information input unit 15c inputs a numerical value indicating the number of detection targets in an image corresponding to the image identification information and a detection target image in the same image as training information for evaluating output information in correspondence with the image identification information, and stores them in the training information storage unit 16c for each image identification information.

Further, in the present embodiment, the image processing unit 17c counts the detection targets in the image, and generates the obtained numerical value indicating the number of detection targets and the detection target image in the image as output information. Then, this output information can be stored in the output information storage unit 18c for each image identification information of the image and for each identification information of individual information.
Thus, the output information obtained by the image processing unit 17c in this embodiment is generated as a numerical value indicating the number of detection targets in an image and a detection target image.

The evaluation value calculation unit 19c can use the output information and the teacher information to weight the image (the difference for each pixel) and the number of detection target elements, and calculate the numerical value obtained by adding them together as the evaluation value. Specifically, for example, the evaluation value can be calculated by the following formula.
Mean squared error with teacher image x weight 1.0 + mean squared error with number of teachers x weight 0.1

According to the image processing apparatus of this embodiment, in the image processing technology for finding detection targets from an image, image processing is performed on an input image to create output information including a numerical value indicating the number of detection targets and a detection target image, and the output information can be compared with the numerical value indicating the number of detection targets and the detection target image, which are teacher information. Therefore, it is possible to efficiently generate better individual information than when only numerical values indicating the number of detection targets are used as teacher information.

For example, when only an image is used as teacher information for a detection target with no clear boundary, the boundary of the given teacher information image is decided by a person, so there may be differences depending on the person who decides, and good learning may not be possible. However, when the number is used as the teacher information, such an effect does not occur. Therefore, it is possible to increase the possibility of obtaining better individual information by combining the number of detection targets and an image as teacher information.

The image processing apparatus of the above embodiments can be realized using a computer controlled by the image processing program of the present invention. The CPU of the computer sends instructions to each component of the computer based on the image processing program to perform predetermined processing required for the operation of the image processing device, such as individual generation processing, image processing, evaluation value calculation processing, genetic manipulation processing, and the like. In this manner, each process and operation in the image processing apparatus of the present invention can be realized by concrete means in which the program and the computer work together.

The program is stored in advance in a recording medium such as a ROM or RAM, and is executed by loading the program into the computer from the recording medium installed in the computer. Alternatively, the program can be loaded into the computer via a communication line, for example.
Also, the recording medium for storing the program can be composed of, for example, a semiconductor memory, a magnetic disk, an optical disk, or any other computer-readable recording means.

As described above, according to the image processing system and the image processing program according to the embodiments of the present invention, it is possible to further optimize the image processing in the image processing technique for finding the detection target from the image.

Experiments for optimizing image processing for detecting a detection target from an image using the image processing apparatus according to the embodiment of the present invention will be described below with reference to FIGS. 6 to 19. FIG.
As an example, an image processing apparatus according to the second embodiment of the present invention was used.
Also, as a reference example, the image processing apparatus according to the second embodiment of the present invention with a part changed was used. Specifically, the reference example is changed so that image information is used as teacher information.

In the reference example, the image processing unit did not generate the numerical value indicating the number of detection targets as output information, but generated an image displaying the detection targets as output information and stored it in the output information storage unit. Also, the evaluation value calculation unit compares the image information as the teacher information and the output information for each pixel to calculate the mean square error, which is stored in the evaluation result storage unit as the evaluation value. In addition, gene manipulation was performed in the same manner as in Examples.

Here, for the evaluation value, in the embodiment, the mean square error is calculated based on the difference in the number of detection targets using the numerical output information and the teacher information, and in the reference example, the mean square error is calculated based on the difference for each pixel using the image output information and the teacher information, so they cannot be directly compared. Therefore, the evaluation values were normalized using the following formula so that these evaluation values were between 0 and 1.

6 to 11 show nodes stored in the functional unit storage unit. The nodes shown in FIG. 6 perform image processing (noise removal), the nodes illustrated in FIG. 7 perform image processing (brightness correction, combination), the nodes illustrated in FIG. 8 perform image processing (binarization), the nodes illustrated in FIG. 9 perform image processing (edge detection), the nodes illustrated in FIG.

Circles, triangles, and squares of various sizes and shapes were used as learning and verification images input by the image input unit. In the reference example, a binarized image in which triangles are selected from the learning image and displayed is used as teacher information, and in the embodiment, a numerical value indicating the number of triangles in the learning image is used as teacher information.

In both the example and the reference example, the number of input images used in learning is eight, and the number of input images used in verification is three.
The initial number of pieces of parent individual information generated by the individual generation unit was set to 150, two pieces of parent individual information were randomly selected and deleted from the parent population, genetic manipulation was performed on these pieces of parent individual information, and 52 pieces of child individual information were generated (individuals identical to parent: 2, uniform crossover individuals: 20, mutant individuals: 30).

Further, image processing was performed, and based on the evaluation results of the child individual information, two pieces of child individual information, the child individual information with the best evaluation value and the child individual information selected by the probability, were selected, and these child individual information were returned to the parent population as parent individual information, and the above processing was repeatedly executed for the set number of generations.
When the maximum number of generations was set to 5000 generations and it was determined that evolution had stopped (when the evaluation value after normalization became 0), the processing was terminated.

The results are shown in FIGS. 12 to 19. FIG. 12 and 13 are explanatory diagrams comparing the learning and the results of the examples and the reference examples, and FIG. 14 is an explanatory diagram comparing the verification results of the examples and the reference examples. FIG. 15 is a diagram showing a graph of transition of normalized evaluation values in learning according to the example and the reference example. Furthermore, FIG. 16 is a diagram showing the structure of the individual information obtained by the learning of the example, and FIG. 17 is a diagram showing the functions of the nodes used in the individual information obtained by the learning of the example. 18 is a diagram showing the structure of the individual information obtained by the learning of the reference example, and FIG. 19 is a diagram showing the functions of the nodes used in the individual information obtained by the learning of the reference example.

In FIGS. 12 to 14, the “detection error” in the reference example is the result of visually counting white objects in the output image and calculating the error from the teacher image.
In addition, the "remarks", "places recognized as detection targets in the embodiment", is created as reference information, based on the binarized image created in the image processing unit, it is confirmed where in the image the output information of the example was counted, and the counted portions are marked with white circles.
The image and numerical value shown in "output" are those based on the individual information with the best evaluation value. That is, the individual information used to output these output information is the individual information finally obtained in the examples and reference examples.

In the learning results shown in FIGS. 12 and 13, no detection error occurred in both the example and the reference example. However, when observing the transition of the evaluation value, as shown in FIG. 15, in the embodiment, the evaluation value becomes 0 at the 412th generation, and the learning process is completed. On the other hand, in the reference example, the evaluation value did not become 0, and generations were repeated up to the maximum number of times of learning (set maximum number of generations). This is because in the case of the reference example, the output image and the teacher image are compared for each pixel, so even if there is a difference of even one pixel, it does not become 0.

In the verification results shown in FIG. 14, no detection error occurred in the example. In addition, although the teacher information is used as the number of detection targets, as shown in "Remarks", all the detected locations are triangles, and the detection targets were counted accurately.

On the other hand, in the reference example, sample No. 2 had a detection error. By referring to the output image, it can be seen that it is significantly different from the teacher image. That is, the detection error is 1 as a result of recognizing the background as one detection target in addition to the three triangles. It is considered that this is caused by over-learning for the learning data. That is, it is presumed that the background of sample No. 2 was erroneously recognized as the detection target due to over-learning of the background during learning.

Further, as shown in FIGS. 16 to 19, according to the learning of the example, it is considered that better individual information than the reference example is obtained, and it can be obtained with a smaller number of times of learning than the reference example. Further, according to the individual information obtained by learning in the example, it is possible to perform more accurate image processing than in the reference example.

It goes without saying that the present invention is not limited to the above embodiments and examples, and that various modifications can be made within the scope of the present invention.
For example, each configuration in the image processing apparatus can be distributed to a plurality of information processing apparatuses, or nodes other than those used in the embodiments can be used.
Further, the above image processing apparatus is used for image inspection, and the image input unit inputs an image for inspection, and the image processing unit processes the image for inspection based on the individual information with the highest evaluation value.
Furthermore, it is possible to adopt a configuration in which the second embodiment and the third embodiment are combined.

INDUSTRIAL APPLICABILITY The present invention can be suitably used to obtain an information processing apparatus for image inspection in which image processing is optimized when inspection is performed based on image data of equipment, products, and the like.

Reference Signs List 1, 1a image processing device 10, 10a functional unit storage unit 11, 11a individual generation unit 12, 12a population storage unit 121a parent population storage unit 122a child population storage unit 13, 13a image input unit 14, 14a image storage unit 15, 15a teacher information input unit 16, 16a teacher information storage unit 17, 17a image determination unit 18, 18a Output information storage unit 19, 19a Evaluation value calculation unit 20, 20a Evaluation result storage unit 21, 21a Gene manipulation unit

Claims (10)

An image processing system that optimizes image processing for detecting a detection target from an image using genetic manipulation, which is an optimization method that mathematically simulates the evolution of living things,
a population storage unit that stores one or more pieces of individual information that define the order of execution of a plurality of functional units in a network that can include cycles;
an image input unit for inputting an image for learning or verification;
a teacher information input unit for inputting teacher information for evaluating output information;
an image processing unit that processes the image based on the individual information and generates the output information;
an evaluation value calculation unit that compares the output information and the teacher information to calculate an evaluation value, and ranks the individual information based on the evaluation value;
a gene manipulation unit that selects or changes the individual information based on the ranking information of the individual information and updates the population storage unit;
The processing by the image processing unit includes processing for counting detection targets in the image,
The image processing system, wherein the teacher information and the output information are numerical values indicating the number of detection targets in the image. 2. The image processing system according to claim 1, wherein the processing by said image processing unit includes at least one of noise removal, correction processing, binarization processing, edge detection processing, and arithmetic processing. 3. The image processing system according to claim 1 or 2, wherein the processing by the image processing unit includes processing for calculating a feature amount of a certain range in the image, and processing for determining whether or not the certain range is a detection target based on the feature amount. The gene manipulation unit
an elite storage unit that leaves the individual information with the high evaluation value to the next generation;
a selection unit for leaving individual information to the next generation with a certain probability based on the evaluation value;
a crossover processing unit that mutually exchanges a part of two pieces of individual information;
4. The image processing system according to any one of claims 1 to 3, further comprising a mutation unit that randomly rewrites part or all of the selected individual information. 5. The image processing system according to claim 1, wherein said network is a directed network composed of functional units and oriented links, and includes a linear structure and a tree structure. The image processing system according to any one of claims 1 to 5, wherein the image input unit inputs an inspection image, and includes a processing result storage unit that stores processing result information obtained by processing the inspection image by the image processing unit based on the individual information with the highest evaluation value. The processing by the image processing unit includes processing for creating a detection target image representing the detection target,
7. The image processing system according to claim 1, wherein said teacher information and said output information include a detection target image representing a detection target. An image processing program that optimizes image processing for detecting a detection target from an image using genetic manipulation, which is an optimization method that mathematically simulates the evolution of living things,
the computer,
an individual group storage unit that stores one or more pieces of individual information that define the order of execution of a plurality of functional units in a network that can include cycles;
an image input unit for inputting images for learning or verification;
a teacher information input unit for inputting teacher information for evaluating output information;
an image processing unit that processes the image based on the individual information and generates the output information;
an evaluation value calculation unit that compares the output information and the teacher information to calculate an evaluation value and ranks the individual information based on the evaluation value;
Selecting or changing the individual information based on the ranking information of the individual information to function as a genetic manipulation unit for updating the population storage unit;
The processing by the image processing unit includes processing for counting detection targets in the image,
The image processing program, wherein the teacher information and the output information are numerical values indicating the number of detection targets in the image. The gene manipulation unit
an elite storage unit that leaves the individual information with the high evaluation value to the next generation;
a selection unit for leaving individual information to the next generation with a certain probability based on the evaluation value;
A crossover processing unit that mutually exchanges a part of two pieces of individual information, and
9. The image processing program according to claim 8, comprising a mutation part that randomly rewrites part or all of the selected individual information. The processing by the image processing unit includes processing for creating a detection target image representing the detection target,
10. The image processing program according to claim 8, wherein the teacher information and the output information include a detection target image representing a detection target.

Images