JP2023128941A - Image analysis system, image analysis method, and program - Google Patents

Abstract

To provide technology for effectively performing learning for image analysis that utilizes machine learning.SOLUTION: An image analysis system comprises at least one processor and a memory resource, the processor executing: a learning image acquisition step in which a learning image group {f_i} (i=1, ..., Nf, Nf: number of learning images) is acquired by imaging an evaluation target for learning; a training step in which an evaluation engine is trained using the learning image group {f_i}; an evaluation image acquisition step in which an evaluation image is acquired by imaging the evaluation target; and an evaluation step in which the evaluation image is input to the trained evaluation engine, and an estimated evaluation value is output. In the training step, a sub-learning image group {f'_j(k)} (j=1, ..., Nf'_k, Nf'_k: number of sub-learning images), {f'_j(k)} included in {f_i}, k: number of iterative learning instances) is determined by an image selection engine, the sub-learning image group being a partial collection of the learning image group {f_i} for each iterative learning instance, and the sub-learning image group {f'_j(k)} is used to perform a k-th instance of iterative learning of the evaluation engine.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image analysis system, an image analysis method, and a program.

Many image analysis methods have been proposed that utilize evaluation engines based on machine learning. For example, in the manufacture of industrial products, visual inspection is performed to evaluate the external appearance of the product by determining defects in shape, assembly defects, adhesion of foreign matter, etc. based on inspection images.

Regarding the appearance inspection apparatus disclosed in Patent Document 1, in paragraph [0054], it is stated that "the appearance of the welding location 201 of the workpiece 200 is inspected by the shape measurement unit 21 of the appearance inspection apparatus 20 (step S1)". , in paragraph [0059], ``By executing step S5, the presence or absence of shape defects and the type of shape defects are identified in the acquired image data.Based on this result, the learning dataset may be reviewed or re-created. creation or new creation is performed (step S6), and retraining of the judgment model is performed using the learning data set created in step S6 (step S7)," and in paragraph [0061], " By performing the routine shown in Figure 5 at an appropriate number of times and frequency as necessary, the accuracy of the judgment model for determining the quality of the welded area can be improved, and the accuracy of the determination model for determining the quality of the welded area can be improved. It is possible to improve the accuracy of determining the presence and type of important shape defects.''

International Publication No. 2020/129617

In order to obtain high evaluation performance in image analysis using machine learning, learning using a large number of training images is required. If the number of training images is small, overfitting may occur and generalization performance may deteriorate. On the other hand, an increase in the number of learning images is directly linked to an increase in learning time.

The technique disclosed in Patent Document 1 requires advance preparation such as classifying image data for learning in advance according to the material and shape of the workpiece and performing data expansion processing, which is not efficient.

The present invention has been made in view of the above points, and aims to provide a technique for efficiently performing learning in image analysis using machine learning.

The present application includes a plurality of means for solving at least part of the above problems, examples of which are as follows.

In order to solve the above problems, an image analysis system of the present invention is an image analysis system comprising at least one processor and a memory resource, wherein the processor * images a learning evaluation object and generates a learning image; A training image acquisition step for acquiring a group {f_i} (i=1,...,Nf, Nf: number of training images), *a learning step for training an evaluation engine using the training image group {f_i}, *a learning step for acquiring an evaluation target An evaluation image acquisition step of capturing an evaluation image and an evaluation step of inputting the evaluation image into a trained evaluation engine and outputting an estimated evaluation value, and in the learning step, the learning image is acquired for each iterative learning. Sub training image group {f'_j(k)} which is a subset of group {f_i} (j=1,…,Nf'_k, Nf'_k: number of sub training images, {f'_j(k)}⊂ {f_i}, k: number of iterative learning) is determined by the image selection engine, and the evaluation engine performs the k-th iterative learning using the sub-learning image group {f'_j(k)}.

According to the present invention, it is possible to provide a technique for efficiently performing learning in image analysis using machine learning.

Problems, configurations, and effects other than those described above will be made clear by the description of the embodiments below.

FIG. 2 is a diagram illustrating an example of the entire processing sequence in an image analysis system based on machine learning. It is a diagram showing an example of a learning method of an evaluation engine. FIG. 3 is a diagram schematically showing an example of a learning state of an evaluation engine. It is a figure which shows an example of a selection condition input screen. 1 is a diagram illustrating an example of a hardware configuration of an image analysis system.

In recent years, the performance of machine learning has improved dramatically with the proposal of deep network models such as Convolutional Neural Networks (CNNs). deep convolutional neural networks,”Proc. of NIPS (2012)”). Many image processing methods have been proposed that utilize evaluation engines based on machine learning. For example, as an example of its use in visual inspection, International Publication No. 2020/129617 describes a method for automatically detecting shape defects in welded areas using machine learning. A method of testing is disclosed. Furthermore, image processing based on machine learning is not limited to visual inspection, and covers a wide range of areas, including semantic segmentation, recognition, image classification, image conversion, and image quality improvement.

In learning the evaluation engine, an image of the evaluation object for learning (learning image) is input, and the evaluation engine is trained so that the difference between the estimated evaluation value output from the evaluation engine and the correct evaluation value taught in advance is small. Update internal parameters (network weights, biases, etc.). When performing a visual inspection, the evaluation value is the inspection result such as the presence or absence of defects and the degree of abnormality of the evaluation target, when performing segmentation, the evaluation value is the label of the area, and when performing image conversion, the evaluation value is the inspection result after the conversion. This is an image of

Regarding the timing of updating internal parameters, it is common to divide the training images into several sets called mini-batches and update the internal parameters for each mini-batch, rather than learning all the training images at once. be. This is called mini-batch learning, and when all mini-batches have been learned, all training images have been used for learning. Learning all of these mini-batches once is called one epoch, and by repeating the epoch many times, the internal parameters are optimized. The training images included in the mini-batch may be shuffled every epoch.

In order to obtain high evaluation performance in image processing using machine learning, it is necessary to learn internal parameters using a large number of training images. If the number of training images is small, overfitting may occur and generalization performance may deteriorate. On the other hand, an increase in the number of learning images is directly linked to an increase in learning time. The learning phase requires higher precision in numerical calculations than the evaluation phase (inference processing), so the processing cost is higher.

On the other hand, "data cleansing" is known in which the number of training images is reduced by deleting or merging unnecessary or redundant images included in the training image group in advance. However, the structures and appearances of evaluation targets vary. If there are many pattern variations in the evaluation target object, a large number of learning images are essentially required, and there is a limit to the reduction of the number of learning images by data cleansing. If images to be learned are excluded by sampling or the like, there is a risk that evaluation performance will be impaired. Therefore, a mechanism is desired that can quickly learn internal parameters of an evaluation engine without reducing evaluation performance.

<Overall processing sequence in image analysis system>
Hereinafter, examples of embodiments of the present invention will be described based on the drawings. The embodiments described below do not limit the claimed invention, and all of the elements and combinations thereof described in the embodiments are essential to the solution of the invention. Not necessarily.

FIG. 1 is a diagram showing an example of the entire processing sequence in an image analysis system 1 based on machine learning. Image processing includes visual inspection, semantic segmentation and recognition, image classification, image conversion, and image quality improvement. The processing sequence executed by the image analysis system 1 is broadly divided into a learning phase 110 and an evaluation phase 120.

In the learning phase 110, the evaluation target object P is imaged for learning to obtain a learning image (step S0). The image is obtained by capturing the surface or inside of the evaluation object P as a digital image using an imaging device such as a CCD (Charge Coupled Device) camera, optical microscope, charged particle microscope, ultrasonic inspection device, or X-ray inspection device. do. Note that another example of "acquisition" may be to simply receive an image captured by another system and store it in the storage resource of the image analysis system.

Next, as an option, "data cleansing" may be performed on all the training images Q taken in step S0 to reduce the number of training images by deleting or merging unnecessary or redundant training images in advance (step S1 ). The learning image group finally used for learning is defined as a learning image group {f_i}(R) (i=1,...,Nf, Nf: number of learning images) ("R" is a reference symbol).

A correct evaluation value g_i is assigned to each learning image f_i. In the case of visual inspection, the evaluation value is the inspection result such as the presence or absence of defects in the evaluation target P and the degree of abnormality, in the case of segmentation, the evaluation value is the label of the area, and in the case of image conversion, the evaluation value is the image after conversion. It is. Correct evaluation values are assigned to these evaluation criteria based on the user's visual judgment or numerical values analyzed by other processing devices/means.

Next, the evaluation engine 111 is trained using the learning image {f_i} and the correct evaluation value {g_i} (step S2). The evaluation engine 111 is an estimator that receives a learning image f_i (evaluation image S in the evaluation phase 120) and outputs an estimated evaluation value g^_i.

Various machine learning engines can be used as the evaluation engine 111, and for example, a deep neural network such as a Convolutional Neural Network (CNN) can be used. In the learning phase 110, when the learning image f_i is input, the internal parameters 113 of the evaluation engine 111 are optimized so that an estimated evaluation value g^_i close to the taught correct evaluation value g_i is output. In the case of a neural network, the internal parameters 113 include "hyper parameters" such as the network structure, activation function, learning rate, and learning termination conditions, and "model parameters" such as weights (coupling coefficients) and bias between network nodes. is included. Optimization of this internal parameter 113 is performed by iterative learning, and the learning image selection engine 112 selects the sub-learning image group {f'_j(k)} used in the k-th iterative learning as the learning image group {f_i}(R). Choose from.

In the evaluation phase 120, an image of the actual evaluation target P is captured (step S0), and an evaluation image S is obtained. The evaluation image S is input to the evaluation engine 111 using the internal parameters 113 learned in the learning phase 110, and automatic evaluation is performed (step S3). The user confirms this evaluation result as necessary (step S4).

In order to obtain high evaluation performance in image processing using machine learning, it is necessary to learn internal parameters using a large number of training images. If the number of training images is small, overfitting may occur and generalization performance may deteriorate. On the other hand, an increase in the number of learning images is directly linked to an increase in learning time. The learning phase requires higher precision in numerical calculations than the evaluation phase (inference processing), so the processing cost is higher.

On the other hand, data cleansing is known in which the number of training images is reduced by deleting or merging unnecessary or redundant images included in the training image group in advance, and it may be used in combination with this embodiment (the steps described above). S1). However, the structure and appearance of the evaluation object P are diverse. If there are many pattern variations in the evaluation target P, essentially many learning images are required, and there is a limit to the reduction of learning images by data cleansing. If training images to be learned are excluded by sampling or the like, there is a risk that evaluation performance will be impaired. Therefore, this embodiment provides a mechanism for learning the internal parameters 113 of the evaluation engine 111 at high speed without reducing evaluation performance.

<Fast learning based on sub-learning image group {f'_j(k)}>
FIG. 2 is a diagram illustrating an example of a learning method of the evaluation engine 111. A method for quickly learning the internal parameters 113 of the evaluation engine 111 will be explained using FIG. 2. This embodiment includes a learning image acquisition step of capturing an image of an evaluation object P for learning and acquiring a learning image group {f_i}(R) (i=1,...,Nf, Nf: number of learning images); a learning step (step S2 in FIG. 1) of learning the evaluation engine 111 using the image group {f_i}(R); an evaluation image acquisition step of capturing an evaluation object P to obtain an evaluation image S; is input into the trained evaluation engine 111 to output the evaluation result, and in the learning step, a sub-learning image group {f_i} (R), which is a subset of the learning image group {f_i}(R), is '_j(k)}(T)(j=1,...,Nf'_k, Nf'_k: Number of sub-learning images, {f'_j(k)}⊂{f_i}, k: Number of iterative learning)( "T" is a reference symbol) is determined by the learning image selection engine 112, and the evaluation engine 111 performs the k-th iterative learning (step S22) using the sub-learning image group {f'_j(k)} (T). It is characterized by

I would like to add some supplementary information about this feature. As described above, when there are many pattern variations in the evaluation target P, there is a limit to eliminating learning images in advance by data cleansing. On the other hand, we focused on the fact that in the iterative learning of the evaluation engine 111, the value of the internal parameter 113 changes, so the priority of the learning images to be learned changes every epoch.

Therefore, in this embodiment, instead of dividing each learning image into two choices of use/exclusion in advance, the learning images are dynamically used/excluded according to the learning state of the evaluation engine 111 (temporarily used during iterative learning). (used or excluded). This makes it possible to significantly shorten the learning time while maintaining various image variations in the entire learning step (step S2 in FIG. 1).

Specifically, the selection probability P_k(f_i) (201) ("201" is a reference symbol) is calculated for the learning image f_i for each iterative learning, and k is calculated based on this selection probability P_k(f_i) (201). In the second iterative learning, it is determined whether to use the learning image f_i, and a sub-learning image group {f'_j(k)}(T) is obtained.

For example, if the estimated evaluation value g^_i of the learning image f_i is incorrect in the k-1st iterative learning, the selection probability P_k(f_i) (201) may be set higher in the next k-th iterative learning. desirable. On the other hand, if the estimated evaluation value g^_i becomes the correct value in the k-th iteration learning, the selection probability P_(k+1)(f_i) (201) is set low in the next k+1-th iteration learning. be able to. However, if the estimated evaluation value g^_i is wrong again in the k+1st iterative learning, the selection probability P_(k+2)(f_i) (201) is set larger in the next k+2nd iterative learning. This is desirable.

In this way, the priority of the learning images to be learned changes according to the evaluation result 202 of the evaluation engine 111, so the sub-learning image group {f'_j(k)}(T) is calculated taking this priority into consideration. This makes it possible to reduce the number of learning images in each epoch. A specific example of the evaluation result 202 is whether the estimated evaluation value is correct or incorrect 203.

<How to calculate selection probability P_k(f_i)>
Regarding the learning image selection engine 112, a method of calculating the selection probability P_k(f_i) (201) of the learning image f_i to the sub-learning image group {f'_j(k)}(T) in the k-th iterative learning will be described. In this embodiment, in the learning step (step S2 in FIG. 1), the margin M(f_i ) (204) ("204" is a reference symbol), and the selection probability P_k(f_i)( 201) is characterized by being a function of the margin M(f_i) (204).

I would like to add some supplementary information about this feature. There are several possible methods for calculating the selection probability P_k(f_i) (201) of the learning image f_i in the k-th iterative learning. One way to do this is to calculate a margin M(f_i) (204) that quantifies the degree of correctness or not, instead of judging the estimated evaluation value as a binary value of correct or incorrect. ) (204), the selection probability P_k(f_i) (201) can be determined.

If the margin M(f_i) (204) is high, the selection probability P_k(f_i) (201) can be set low. As a method for calculating the margin M(f_i) (204), for example, the difference between the estimated evaluation value g^_i of the learning image f_i and the correct evaluation value g_i can be used. If the difference is very small, the estimated evaluation value g^_i is correct with a margin. Furthermore, if the difference is large, the evaluation result will be incorrect, but the larger the difference, the lower the margin for the same incorrect answer. By such a method, the margin M(f_i) (204) and the selection probability P_k(f_i) (201) can be calculated as continuous values.

Further, in the present embodiment, in the learning step (step S2 in FIG. 1), the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)}(T) in the k-th iterative learning is (201) is characterized in that it is a function of the degree of similarity between the learning image f_i and the other learning image group {f_a} (a≠i).

I would like to add some supplementary information about this feature. The present embodiment is characterized in that the selection probability P_k(f_i) (201) of the learning image f_i is updated every iterative learning. The degree of similarity between the learning image f_i and other learning images can be used as a criterion when calculating the selection probability P_k(f_i) (201). If very similar learning images exist or there are many similar images, the degree of similarity will be high and the selection probability P_k(f_i) (201) will be low.

As mentioned above, as judgment materials when calculating the selection probability P_k(f_i) (201), the correctness 203 of the estimated evaluation value based on the estimated evaluation value g^_i of the learning image f_i in the k-th iterative learning and the margin M( f_i) (204) can be used, but these values change depending on the learning state. On the other hand, although the similarity between learning images does not change during repeated learning, it is used as a comprehensive judgment material for calculating the selection probability P_k(f_i) (201). can do. That is, the selection probability P_k(f_i) (201) can be given as a function of these multiple judgment materials (step S21).

Regarding FIG. 2, the image analysis system 1 optimizes the internal parameters 113 of the evaluation engine 111 by repeating the processes from step S21 to step S23 using the learning image group {f_i}(R). In step S21, the image analysis system 1 determines the selection probability P_k(f_i) (201). Specifically, the image analysis system 1 uses the k-1st evaluation result 202 to determine the kth selection probability P_k(f_i )(201) is calculated. Thereafter, the image analysis system 1 uses the learning image selection engine 112 to select the sub-learning image group {f'_j( k)} (T).

Next, in step S22, the image analysis system 1 executes learning of the evaluation engine 111. Specifically, the image analysis system 1 applies a pre-assigned correct evaluation value g_i to each of the learning images f_i included in the sub-learning image group {f'_j(k)}(T) selected in step S21. The internal parameters 113 of the evaluation engine 111 are updated so that a close estimated evaluation value g^_i is output.

Next, in step S23, the image analysis system 1 uses the evaluation engine 111 to obtain evaluation results 202 for each learning image. Specifically, the image analysis system 1 estimates each learning image f_i included in the learning image group R{f_j(k)} using the evaluation engine 111 to which the internal parameters 113 learned in step S22 are applied. Calculate the evaluation value g^_i. The image analysis system 1 obtains evaluation results 202 for each learning image f_i included in the learning image group {f_i}(R). As an example, the evaluation result 202 is the correctness 203 of the estimated evaluation value g^_i and the margin M(f_i) (204).

As described above, in this embodiment, since the internal parameters 113 are optimized using the sub-learning image group {f'_j(k)}(T), the evaluation engine 111 can be efficiently trained. In addition, since the evaluation engine 111 is trained using the sub-learning image group {f'_j(k)} (T) selected using the margin M(f_i) (204) and the similarity, it is more efficient. The internal parameters 113 can be optimized.

Therefore, according to this embodiment, in image processing using machine learning, high-speed learning is possible even for evaluation targets having various pattern variations.

<Overview of selection probability>
FIG. 3 is a diagram schematically showing an example of the learning state of the evaluation engine 111. The concept of calculating the selection probability P_k(f_i) (201) will be explained using FIG. 3. As an example of the evaluation engine 111, consider a non-defective product determination in which an evaluation target object P is inspected to determine whether it is a non-defective product or a defective product. Inside the evaluation engine 111, a plurality of features {Ca} (a=1,...,n, n: number of features) that are effective for evaluation are once obtained from the learning image f_i (evaluation image S in the evaluation phase), and It is assumed that an estimated evaluation value is output based on this feature amount.

In FIG. 3, the plots of circles and triangles indicate the distribution of features of the learning images in the k-th iterative learning, and the plots of circles and triangles are the features of the learning images of non-defective and defective products, respectively. Plots existing inside the good product cluster 300 in the kth iteration of learning are determined to be good products, and plots existing outside the good product cluster 300 are determined to be defective products.

Therefore, it is necessary to gradually change the internal parameters 113 (feature amount calculation method and shape of the good product cluster) through iterative learning so that the good product cluster 300 separates good products from defective products as much as possible through iterative learning. Consider the selection probability P_k(f_i) (201) of the learning image f_i to the sub-learning image group {f'_j(k)}(T) in the k-th iterative learning. White, gray, and black plots represent "high", "medium", and "low" selection probabilities, respectively. To simplify the explanation, the selection probability is shown in three stages in FIG. 3, but the actual selection probability can take continuous values.

For learning images 301 of non-defective products that exist near the boundaries of non-defective product clusters 300, the selection probability is set high because the estimated evaluation value 203 may change even if there is a slight change in the feature value or the non-defective product cluster 300. This is desirable. The five learning images 302 of non-defective products located at the center of the non-defective product cluster 300 have correct estimated evaluation values, and because they are located at the center of the non-defective product cluster 300, the training images 302 of the five non-defective products located at the center of the non-defective product cluster 300 are not affected by minute changes in the feature values or the non-defective product cluster 300. There is a low possibility that the estimated evaluation value will turn out to be incorrect. In other words, since this is a learning image with a high degree of margin M(f_i) (204), it is desirable to set the selection probability low.

Since the learning images 303 of two good products existing between the boundary and the center of the good product cluster 300 have a medium margin M(f_i) (204), it is desirable to set the selection probability to a medium value as well.

The same applies to learning images of defective products. In other words, the learning images 304, 305, and 306 of defective products are all determined to be defective products, so they are correct, but the closer they get to the good product cluster 300, the lower the margin M(f_i) (204) is, so the selection probability is increased. It is desirable to set this.

It is desirable to set the selection probability of erroneously determined learning images (307 where a good product is erroneously determined as a defective product, 308 where a defective product is erroneously determined as a non-defective product) to be very high in order to improve the determination results. It is desirable to set a low selection probability for a group of images (309 and 310) that have a high degree of similarity (and are likely to be plotted close to each other in the feature space). Even if the image group has a high degree of similarity, it is desirable to set a high selection probability for the learning images 311 that are erroneously determined.

In this way, the selection probability needs to be determined by comprehensively considering the correctness 203 of the estimated evaluation value, the degree of margin M(f_i) (204), the degree of similarity between images, and the like. By determining the sub-learning image group {f'_j(k)}(T), the number of learning images in each iterative learning is not only reduced, but also the optimization is faster by preferentially learning the learning images that should be learned. There is a possibility that good evaluation performance can be obtained with a small number of iterative learnings.

<Selection condition input screen>
In this embodiment, in the learning step (step S2 in FIG. 1), the number of images Nf' of the sub-learning image group {f'_j(k)}(T) is changed from the number Nf of images of the learning image group {f_i}(R) to the number Nf' of images of the sub-learning image group {f'_j(k)}(T). It has a GUI that accepts the specification of the reduction rate R_k to _k, and is characterized in that the reduction rate R_k is a function of the number k of iterative learning.

I would like to add some supplementary information about this feature. It is possible to specify the reduction rate R_k of the number of learning images in the k-th iterative learning by taking into account constraints on learning time and the like. The reduction rate R_k can be defined as (Nf-Nf'_k)/Nf*100, for example, and in this case, the larger the value, the more training images are reduced and the learning time is shortened. Further, the reduction rate R_k can be changed depending on the number k of iterative learning.

Generally, at the beginning of iterative learning, the internal parameters are not determined and a wide range of parameter searches is required, so it is desirable to set the reduction rate R_k small. On the other hand, in the later stages of iterative learning, the internal parameters begin to converge to the optimal values, so the reduction rate R_k can be set large.

Although the reduction rate R_k has been explained as an example, the value to be specified may be the number of images Nf'_k of the sub-learning image group {f'_j(k)}(T) or the estimated learning time instead of the reduction rate R_k. When the estimated learning time is specified, the number of images Nf'_k of the sub-learning image group {f'_j(k)}(T) is set so that learning is completed within the estimated learning time based on the number of iterative learning. will be decided.

FIG. 4 is a diagram showing an example of a selection condition input screen 400. The selection condition input screen 400 is a GUI (Graphical User Interface) in which the user specifies the learning method of the evaluation engine 111. By checking the check box 401, the specification of the reduction rate R_k becomes valid. It is possible to select how to give the reduction rate R_k using radio buttons 402 to 404.

When the radio button 402 is selected, the reduction rate R_k becomes a specified constant value 406 regardless of the number k of iterative learning, as given by a straight line 405. When radio button 403 is selected, the reduction rate R_k is specified by a line 407, and in the illustrated example, the reduction rate R_k increases until the epoch (number of iterative learning) specified in box 408, and thereafter remains constant as specified in box 409. value. When the radio button 404 is selected, the reduction rate R_k becomes the curve 410 specified in the box 411. These are examples of how to specify the reduction rate R_k, and any shape can be specified. When the image analysis system 1 accepts the designation of the method for setting the reduction rate R_k from the user, it becomes possible for the evaluation engine 111 to learn more efficiently in accordance with needs.

A method for calculating the selection probability P_k(f_i) (201) of the learning image f_i can be specified using the GUI shown in FIG. As mentioned above, the selection probability P_k(f_i) (201) can be calculated using the judgment materials such as the correctness 203 of the estimated evaluation value, the degree of margin M(f_i) (204), the degree of similarity between learning images, and the like. That is, the selection probability P_k(f_i) (201) can be given by a function using these judgment materials as arguments. Arguments to be considered in calculating the selection probability P_k(f_i) (201) can be specified using check boxes 412 to 414. The ratio (weight) of considering each argument in calculating the selection probability P_k(f_i) (201) can be specified in boxes 415 to 417.

In the example of FIG. 4, the values in boxes 415 to 417 are specified as 0.2, 0.6, and 0.2, respectively, and the selection probability P_k(f_i) (201) is determined with emphasis on the margin M(f_i) (204). It turns out. Further, rules can be set regarding the selection of the learning image f_i, and together with the selection probability P_k(f_i) (201), the sub-learning image group {f'_j(k)}(T) is determined. For example, by checking the check box 418, it is possible to enable a rule that learning images that were incorrect in the previous iterative learning are always selected in the next iterative learning.

Furthermore, by checking the check box 419, it is possible to enable a rule that learning samples with margins below the threshold value are always selected. A threshold can be specified in box 421. Furthermore, by checking the check box 420, it is possible to enable a rule that learning images having a degree of similarity equal to or higher than a threshold value are always thinned out. A threshold can be specified in box 422. The arguments and rules described here are examples, and it is also possible to set other arguments and rules.

In this way, the user can specify by various means what kind of learning images should be learned preferentially in each iterative learning. By incorporating the domain knowledge held by the user into the evaluation object P, more appropriate learning becomes possible.

<Hardware configuration>
FIG. 5 is a diagram showing an example of the hardware configuration of the image analysis system 1. The image analysis system 1 includes the above-described imaging device 106 and a computer 100. The imaging device 106 is as described above. The computer 100 is a component that processes the image evaluation method in this embodiment, and includes a processor 101, a storage resource (memory resource) 102, a GUI device 103, an input device 104, and a communication interface 105.

The processor 101 is a processing device such as a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit), but is not limited thereto, and may be any device that can execute the above-described image analysis method. , one processor 101 may be single-core or multi-core.Alternatively, one processor 101 may be a single-core or multi-core circuit.Alternatively, one processor 101 may be a circuit that is a collection of gate arrays (for example, an FPGA (Field -Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit).

The storage resource 102 is a storage device such as RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), non-volatile memory (flash memory, etc.), and is a storage from which programs and data are temporarily read. It functions as an area. The storage resource 102 may store a program (referred to as an image analysis program) that causes the processor 101 to execute the image analysis method described in the above embodiment.

The GUI device 103 is a device that displays a GUI, and is, for example, a display such as an OLCD (Organic Liquid Crystal Display) or a projector, but it may be any device that can display a GUI, and is not limited to this example. The input device 104 is a device that accepts input operations from a user, and is, for example, an input device such as a keyboard, a mouse, or a touch panel. The input device 104 is not particularly limited as long as it is a component that can accept operations from the user, and the input device 104 and the GUI device 103 may be an integrated device.

The communication interface 105 is a USB, Ethernet, Wi-Fi, etc. interface that mediates input and output of information. Note that the communication interface 105 is not limited to the example shown here, as long as it is an interface that can directly receive images from the imaging device 106 or allow the user to send the images to the computer 100. Note that a portable non-volatile storage medium (for example, a flash memory, DVD, CD-ROM, Blu-ray disc, etc.) storing the image can be connected to the communication interface 105 and the image can be stored in the computer 100.

As described above, according to the present embodiment, in image processing using machine learning, high-speed learning is possible even for evaluation targets having various pattern variations.

As mentioned above, the embodiments described so far do not limit the claimed invention, and all of the various elements and combinations thereof described in the embodiments are a means of solving the invention. is not necessarily required.

In addition, the image analysis system 1 may include a plurality of computers 100 and a plurality of imaging devices 106. Further, the image analysis program described above can be distributed to the computer 100 by connecting a portable non-volatile storage medium storing the image analysis program to the communication interface 105. Alternatively, the image analysis program can be distributed to the computer 100 by a program distribution server. In this case, the program distribution server includes a storage resource 102 that stores an image analysis program, a processor that performs distribution processing to distribute the image analysis program, and a communication interface device that can communicate with the communication interface device of the computer 100. . Note that various functions of the image analysis program distributed or distributed to the computer 100 are realized by the processor 101.

Furthermore, as described above, the image analysis system 1 performs a learning phase 110 in which the evaluation engine 111 is trained, and an evaluation phase 120 in which the evaluation image S is evaluated using the evaluation engine 111 learned in the learning phase 110. Execute. The processor 101 that executes the learning phase 110 and the processor 101 that executes the evaluation phase 120 may be the same or different. If the processor 101 that executes the learning phase 110 and the processor 101 that executes the evaluation phase 120 are different, the processor 101 that executes the learning phase 110 provides internal parameters 113 of the evaluation engine 111 to the processor 101 that executes the evaluation phase 120. can be handed over.

1: Image analysis system, 100: Computer, 101: Processor, 102: Storage resource, 103: GUI device, 104: Input device, 105: Communication interface, 106: Imaging device, 110: Learning phase, 111: Evaluation engine, 112 : Learning image selection engine, 113: Internal parameters, 120: Evaluation phase, 201: Selection probability P_k(f_i), 202: Evaluation result, 203: Correctness of estimated evaluation value, 204: Margin M(f_i), 400: Selection Condition input screen, 401, 412, 413, 414, 419, 420: Check box, 402, 403, 404: Radio button, 405: Straight line, 406: Value, 407: Polyline, 408, 409, 411, 415, 416, 417, 421, 422: Box, 410: Curve, P: Evaluation object, Q: All learning image group, R: Learning image group {f_i}, S: Evaluation image, T: Sub-learning image group {f'_j( k)}

Claims (9)

An image analysis system comprising at least one processor and memory resources, the system comprising:
The processor includes:
*Learning image acquisition step of capturing an image of the evaluation object for learning and acquiring a learning image group {f_i} (i=1,...,Nf, Nf: number of learning images) *Evaluation using the learning image group {f_i} A learning step for learning the engine *Evaluation image acquisition step for capturing an evaluation object and acquiring an evaluation image *Evaluation step for inputting an evaluation image into a trained evaluation engine and outputting an estimated evaluation value;
In the learning step, a sub-learning image group {f'_j(k)} (j=1,...,Nf'_k, Nf'_k: sub-learning image number, {f'_j(k)}⊂{f_i}, k: number of iterative learning) is determined by the image selection engine, and the k-th iteration of the evaluation engine is determined using the sub-learning image group {f'_j(k)}. An image analysis system that performs iterative learning. The image analysis system according to claim 1,
In the learning step, calculate a margin M(f_i) that quantifies the degree of correctness of the estimated evaluation value output when the learning image f_i is input to the evaluation engine during iterative learning,
Image analysis characterized in that the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is a function of the margin M(f_i). system. The image analysis system according to claim 1,
In the learning step, the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is determined by the selection probability P_k(f_i) of the learning image f_i and the other learning image group {f_a}( An image analysis system characterized in that the function is a function of similarity between a≠i). The image analysis system according to claim 1,
In the learning step, the method includes a GUI that accepts the specification of a reduction rate R_k from the number of images Nf of the training image group {f_i} to the number of images Nf'_k of the sub-learning image group {f'_j(k)};
An image analysis system characterized in that the reduction rate R_k is a function of the number k of iterative learning. An image analysis method using an image analysis system, the method comprising:
a learning image acquisition step of capturing an evaluation object for learning and acquiring a learning image group {f_i} (i=1,...,Nf, Nf: number of learning images);
a learning step of learning the evaluation engine using the training image group {f_i};
an evaluation image acquisition step of capturing an evaluation image by imaging the evaluation target;
an evaluation step of inputting the evaluation image into a trained evaluation engine and outputting an estimated evaluation value;
In the learning step, a sub-learning image group {f'_j(k)} (j=1,...,Nf'_k, Nf'_k: sub-learning image number, {f'_j(k)}⊂{f_i}, k: number of iterative learning) is determined by the image selection engine, and the k-th iteration of the evaluation engine is determined using the sub-learning image group {f'_j(k)}. An image analysis method characterized by performing iterative learning. The image analysis method according to claim 5,
In the learning step, calculate a margin M(f_i) that quantifies the degree of correctness of the estimated evaluation value output when the learning image f_i is input to the evaluation engine during iterative learning,
Image analysis characterized in that the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is a function of the margin M(f_i). Method. The image analysis method according to claim 5,
In the learning step, the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is determined by the selection probability P_k(f_i) of the learning image f_i and the other learning image group {f_a}( An image analysis method characterized in that it is a function of similarity between a≠i). The image analysis method according to claim 5,
In the learning step, a designation of a reduction rate R_k from the number of images Nf of the training image group {f_i} to the number of images Nf'_k of the sub-learning image group {f'_j(k)} is accepted via the GUI;
An image analysis method, wherein the reduction rate R_k is a function of the number k of iterative learning. A program that causes a processor to execute the image analysis method according to claim 5.

Images