WO2023166773A1

WO2023166773A1 - Image analysis system, image analysis method, and program

Info

Publication number: WO2023166773A1
Application number: PCT/JP2022/036388
Authority: WO
Inventors: 敦宮本
Original assignee: 株式会社日立製作所
Priority date: 2022-03-04
Filing date: 2022-09-29
Publication date: 2023-09-07
Also published as: JP2023128941A

Abstract

The purpose of the present invention is to provide technology for effectively performing learning for image analysis that utilizes machine learning.　This image analysis system comprises at least one processor and a memory resource, and is characterized by the processor executing: a learning image acquisition step in which a learning image group {f_i} (i=1,…,Nf, Nf: number of learning images) is acquired by imaging an evaluation target for learning; a training step in which an evaluation engine is trained using the learning image group {f_i}; an evaluation image acquisition step in which an evaluation image is acquired by imaging an evaluation target; and an evaluation step in which the evaluation image is input to the trained evaluation engine, and an estimated evaluation value is output. In the training step, a sub-learning image group {f'_j(k)} (j=1,…,Nf'_k, Nf'_k: number of sub-learning images), {f'_j(k)}⊂{f_i}, k: number of iterative learning instances) is determined by an image selection engine, the sub-learning image group being a partial collection of the learning image group {f_i} for each iterative learning instance, and the sub-learning image group {f'_j(k)} is used to perform the kth instance of iterative learning of the evaluation engine.

Description

Image analysis system, image analysis method, and program

The present invention relates to an image analysis system, an image analysis method, and a program. The present invention claims priority of Japanese patent application number 2022-033635 filed on March 4, 2022, and for designated countries where incorporation by reference of documents is permitted, the content described in the application is incorporated into this application by reference.

Many image analysis methods have been proposed that utilize evaluation engines based on machine learning. For example, in the manufacture of industrial products, appearance inspections are performed to evaluate the appearance of articles by determining shape defects, assembly defects, adherence of foreign matter, and the like based on inspection images.

Regarding the appearance inspection apparatus disclosed in Patent Document 1, in paragraph [0054], it is described that "the appearance of the welded portion 201 of the workpiece 200 is inspected by the shape measurement unit 21 of the appearance inspection apparatus 20 (step S1)." , in paragraph [0059], "By executing step S5, the presence or absence of shape defects and the type of shape defects are specified in the acquired image data. Based on this result, the learning data set can be reviewed and re- Creation or new creation is performed (step S6), and re-learning of the judgment model is executed using the learning data set created in step S6 (step S7)." By performing the routine shown in FIG. 5 at an appropriate number and frequency as necessary, the accuracy of the judgment model for judging the quality of the shape of the welded part is improved, and the quality of the shape of the welded part is judged. It is possible to improve the accuracy of determining the presence and type of important shape defects."

International publication 2020/129617

In order to obtain high evaluation performance in image analysis using machine learning, learning using a large number of learning image groups is required. If the number of training images is small, over-learning may occur and the generalization performance may deteriorate. On the other hand, an increase in the number of learning images directly leads to an increase in learning time.

The technique disclosed in Patent Document 1 is inefficient because it requires advance preparation such as classifying image data for learning in advance according to the material and shape of the workpiece and performing data expansion processing.

The present invention has been made in view of the above points, and aims to provide a technique for efficiently learning in image analysis using machine learning.

The present application includes multiple means for solving at least part of the above problems, and the following are examples of such means.

In order to solve the above problems, an image analysis system of the present invention is an image analysis system comprising at least one processor and a memory resource, wherein the processor captures an evaluation object for learning to obtain a learning image; A learning image acquisition step for acquiring a group {f_i} (i=1,...,Nf, Nf: the number of learning images), * a learning step for learning an evaluation engine using the learning image group {f_i}, * an evaluation target An evaluation image obtaining step of capturing an evaluation image to obtain an evaluation image, and an evaluation step of inputting the evaluation image to a trained evaluation engine and outputting an estimated evaluation value are executed. sub-learning image group {f'_j(k)} which is a subset of group {f_i} (j=1,...,Nf'_k, Nf'_k: number of sub-learning images, {f'_j(k)} {f_i}, k: the number of iterative learning) is determined by the image selection engine, and the k-th iterative learning of the evaluation engine is performed using the sub-learning image group {f'_j(k)}.

According to the present invention, it is possible to provide a technique for efficient learning in image analysis using machine learning.

Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

FIG. 4 is a diagram showing an example of an overall processing sequence in an image analysis system based on machine learning; FIG. 4 is a diagram showing an example of a learning method of an evaluation engine; FIG. 4 is a diagram schematically showing an example of a learning state of an evaluation engine; It is a figure which shows an example of a selection condition input screen. It is a figure which shows an example of the hardware constitutions of an image analysis system.

In recent years, the proposal of deep network models represented by the Convolutional Neural Network (CNN) has dramatically improved the performance of machine learning (for example, the literature "A. Krizhevsky, I. Sutskever, and G. E. Hinton," Imagenet classification with deep convolutional neural networks, “Proc. of NIPS (2012)”). Many image processing methods that utilize evaluation engines based on machine learning have been proposed. For example, as an example of utilization for visual inspection, in International Publication No. 2020/129617, shape defects in welds can be automatically detected using machine learning. A method of inspecting is disclosed. In addition to visual inspection, machine learning-based image processing covers a wide range of fields, including semantic segmentation, recognition, image classification, image conversion, and image quality improvement.

In the learning of the evaluation engine, an image of an object to be evaluated for learning (learning image) is input, and the evaluation engine is adjusted so that the difference between the estimated evaluation value output from the evaluation engine and the correct evaluation value taught in advance becomes small. update the internal parameters of , such as network weights and biases. In the case of visual inspection, the evaluation value is the inspection result such as the presence or absence of defects and the degree of abnormality of the evaluation target.In the case of segmentation, the evaluation value is the label of the region. is an image of

As for the timing of updating the internal parameters, instead of learning all the training images at once, it is common to divide the training images into sets called mini-batches and update the internal parameters for each mini-batch. be. This is called mini-batch learning, and when all mini-batches have been learned, all training images have been used for learning. Learning all these mini-batches once is called one epoch, and the internal parameters are optimized by repeating the epoch many times. We may also shuffle the training images in the mini-batch for each epoch.

In order to obtain high evaluation performance in image processing using machine learning, it is necessary to learn internal parameters using a large number of training image groups. If the number of training images is small, over-learning may occur and the generalization performance may deteriorate. On the other hand, an increase in the number of learning images directly leads to an increase in learning time. Since the learning phase requires higher numerical calculation accuracy than the evaluation phase (inference processing), the processing cost is high.

On the other hand, "data cleansing" is known, which reduces the number of learning images by deleting or integrating unnecessary or redundant images included in the learning image group in advance. However, the structures and appearances of evaluation objects are diverse. If the object to be evaluated has many pattern variations, a large number of training images are essentially required, and there is a limit to how much training images can be reduced by data cleansing. If images to be learned are excluded by sampling or the like, there is a risk of impairing evaluation performance. Therefore, there is a demand for a mechanism for quickly learning the internal parameters of the evaluation engine without degrading the evaluation performance.

<Overall processing sequence in image analysis system>
Hereinafter, examples of embodiments of the present invention will be described based on the drawings. In addition, the embodiments described below do not limit the invention according to the claims, and all of the elements described in the embodiments and their combinations are essential to the solution of the invention. Not necessarily.

FIG. 1 is a diagram showing an example of an overall processing sequence in an image analysis system 1 based on machine learning. Image processing includes visual inspection, semantic segmentation and recognition, image classification, image transformation and image quality improvement. A processing sequence executed by the image analysis system 1 is roughly divided into a learning phase 110 and an evaluation phase 120 .

In the learning phase 110, the evaluation object P is imaged for learning to acquire a learning image (step S0). An image is acquired by imaging the surface or inside of the evaluation object P as a digital image with an imaging device such as a CCD (Charge Coupled Device) camera, optical microscope, charged particle microscope, ultrasonic inspection device, X-ray inspection device, etc. do. As another example of "acquisition", it is possible to simply receive an image captured by another system and store it in the storage resource of the image analysis system.

Next, as an option, "data cleansing" may be performed on all the learning image groups Q captured in step S0 to reduce the number of learning images by deleting or integrating unnecessary or redundant learning images in advance (step S1 ). A group of learning images finally used for learning is a group of learning images {f_i} (R) (i=1, .

A correct evaluation value g_i is given to each learning image f_i. In the case of appearance inspection, the evaluation value is the inspection result such as the presence or absence of defects and the degree of abnormality of the evaluation object P. In the case of segmentation, the evaluation value is the label of the region.In the case of image conversion, the evaluation value is the image after conversion. is. A correct evaluation value is assigned to these evaluation criteria based on a user's visual judgment or a numerical value analyzed by another processing device/means.

Next, the evaluation engine 111 learns using the learning image {f_i} and the correct evaluation value {g_i} (step S2). The evaluation engine 111 is an estimator that receives a learning image f_i (an evaluation image S in the evaluation phase 120) and outputs an estimated evaluation value g^_i.

A variety of machine learning engines can be used for the evaluation engine 111, and examples include deep neural networks represented by Convolutional Neural Network (CNN). In the learning phase 110, the internal parameters 113 of the evaluation engine 111 are optimized so that an estimated evaluation value g^_i close to the taught correct evaluation value g_i is output when the learning image f_i is input. In the case of a neural network, the internal parameters 113 include "hyperparameters" such as network structure, activation function, learning rate and termination conditions of learning, and "model parameters" such as weights (coupling coefficients) and biases between nodes of the network. is included. Optimization of this internal parameter 113 is performed by iterative learning, and the sub-learning image group {f'_j(k)} used in the k-th iterative learning is selected by the learning image selection engine 112 as the learning image group {f_i} (R) Choose from

In the evaluation phase 120, the actual evaluation object P is imaged (step S0), and an evaluation image S is obtained. The evaluation image S is input to the evaluation engine 111 using the internal parameters 113 learned in the learning phase 110, and automatic evaluation is performed (step S3). The user confirms this evaluation result as necessary (step S4).

On the other hand, data cleansing is known in which unnecessary or redundant images included in a group of learning images are deleted or integrated in advance to reduce the number of learning images. S1). However, the structure and appearance of the evaluation target P are diverse. When many pattern variations exist in the evaluation object P, many learning images are essentially required, and there is a limit to how much learning images can be reduced by data cleansing. If the learning images to be learned are excluded by sampling or the like, there is a risk of impairing the evaluation performance. Therefore, this embodiment provides a mechanism for quickly learning the internal parameters 113 of the evaluation engine 111 without degrading the evaluation performance.

<Fast learning based on sub-learning image group {f'_j(k)}>
FIG. 2 is a diagram showing an example of the learning method of the evaluation engine 111. As shown in FIG. A method for quickly learning the internal parameters 113 of the evaluation engine 111 will be described with reference to FIG. The present embodiment includes a learning image acquisition step of imaging an evaluation object P for learning and acquiring a learning image group {f_i} (R) (i=1, . . . , Nf, Nf: the number of learning images); A learning step (step S2 in FIG. 1) for learning the evaluation engine 111 using the image group {f_i} (R), an evaluation image acquisition step for capturing an evaluation object P and acquiring an evaluation image S, and an evaluation image is input to the trained evaluation engine 111 and the evaluation result is output. In the learning step, sub-learning image group {f '_j(k)} (T) (j=1,...,Nf'_k, Nf'_k: number of sub-learning images, {f'_j(k)}⊂{f_i}, k: number of iterative learning) ( “T” is a reference sign) is determined by the learning image selection engine 112, and the evaluation engine 111 performs k-th iterative learning (step S22) using the sub-learning image group {f′_j(k)} (T). It is characterized by

Supplementary information about this feature. As described above, when there are many pattern variations in the evaluation object P, there is a limit to preliminarily excluding learning images by data cleansing. On the other hand, in the iterative learning of the evaluation engine 111, since the value of the internal parameter 113 changes, the priority of the learning image to be learned changes for each epoch.

Therefore, in this embodiment, each learning image is not divided in advance into two choices of use/exclusion, but the learning image is dynamically used/excluded according to the learning state of the evaluation engine 111 (temporarily during iterative learning). used or excluded). As a result, in the entire learning step (step S2 in FIG. 1), it is possible to greatly shorten the learning time while maintaining various image variations.

Specifically, the selection probability P_k(f_i) (201) ("201" is a reference sign) is calculated for each iterative learning for the learning image f_i, and based on this selection probability P_k(f_i) (201), k A sub-learning image group {f'_j(k)}(T) is obtained by determining whether or not to use the learning image f_i in the iterative learning of the iteration.

For example, if the estimated evaluation value g^_i of the learning image f_i is incorrect in the k-1th iterative learning, the selection probability P_k(f_i) (201) can be set high in the next k-th iterative learning. desirable. On the other hand, if the estimated evaluation value g^_i becomes a correct value in the k-th iterative learning, the selection probability P_(k+1)(f_i) (201) is set low in the next k+1-th iterative learning. be able to. However, if the estimated evaluation value g^_i is incorrect again in the k+1 iterative learning, the selection probability P_(k+2)(f_i) (201) is set large in the next k+2 iterative learning. is desirable.

Since the priority of the learning images to be learned changes according to the evaluation result 202 of the evaluation engine 111 in this way, the sub-learning image group {f'_j(k)}(T) is obtained in consideration of this priority. This makes it possible to reduce the number of learning images in each epoch. A specific example of the evaluation result 202 is whether the estimated evaluation value is right or wrong 203 .

<Method for calculating selection probability P_k(f_i)>
Regarding the learning image selection engine 112, a method of calculating the selection probability P_k(f_i) (201) of the learning image f_i to the sub-learning image group {f'_j(k)}(T) in the k-th iterative learning will be described. In this embodiment, in the learning step (step S2 in FIG. 1), the degree of margin M(f_i ) (204) (“204” is a reference sign), and the selection probability P_k(f_i) ( 201) is characterized by being a function of the margin M(f_i) (204).

Supplementary information about this feature. Several methods of calculating the selection probability P_k(f_i) (201) of the learning image f_i in the k-th iterative learning are conceivable. As one of them, the margin M(f_i) (204) that quantifies the degree of correctness is calculated, and this margin M(f_i ) (204), the selection probability P_k(f_i) (201) can be determined.

If the margin M(f_i) (204) is high, the selection probability P_k(f_i) (201) can be set low. As a method of calculating the margin M(f_i) (204), for example, the difference between the estimated evaluation value g^_i and the correct evaluation value g_i of the learning image f_i can be used. If the difference is very small, it means that the estimated evaluation value g^_i is correct with a margin. Also, if the difference is large, the evaluation result will be an incorrect answer, but the larger the difference, the lower the margin for the same incorrect answer. By such a method, the margin M(f_i) (204) and the selection probability P_k(f_i) (201) can be calculated as continuous values.

In addition, in the learning step (step S2 in FIG. 1), the present embodiment sets the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} (T) in the k-th iterative learning. (201) is characterized by being a function of the degree of similarity between the learning image f_i and other learning image groups {f_a} (a≠i).

Supplementary information about this feature. This embodiment is characterized in that the selection probability P_k(f_i) (201) of the learning image f_i is updated for each iterative learning. As a criterion for calculating the selection probability P_k(f_i) (201), the degree of similarity between the learning image f_i and other learning images can be used. When very similar learning images exist or there are many similar images, the similarity is high and the selection probability P_k(f_i) (201) is low.

As described above, the accuracy 203 of the estimated evaluation value based on the estimated evaluation value g^_i of the learning image f_i in the k-th iterative learning and the margin M( f_i) (204) can be used, but these values will change depending on the learning state. On the other hand, the similarity between training images does not change during iterative learning, but together with those whose values change, it is used as a judgment material for calculating the selection probability P_k(f_i) (201). can do. That is, the selection probability P_k(f_i) (201) can be given as a function of these multiple criteria (step S21).

In addition to FIG. 2, the image analysis system 1 optimizes the internal parameters 113 of the evaluation engine 111 by repeating the processing from step S21 to step S23 using the learning image group {f_i}(R). In step S21, the image analysis system 1 determines selection probabilities P_k(f_i) (201). Specifically, the image analysis system 1 uses the k-1th evaluation result 202 to calculate the kth selection probability P_k(f_i ) (201). After that, the image analysis system 1 uses the learning image selection engine 112 to select a sub-learning image group {f'_j( k)}(T).

Next, in step S22, the image analysis system 1 performs learning of the evaluation engine 111. Specifically, the image analysis system 1 sets the correct evaluation value g_i assigned in advance to Update the internal parameters 113 of the evaluation engine 111 so that the closest estimated evaluation value g^_i is output.

Next, in step S23, the image analysis system 1 uses the evaluation engine 111 to obtain evaluation results 202 for each learning image. Specifically, the image analysis system 1 estimates each of the learning images f_i included in the learning image group R{f_j(k)} using the evaluation engine 111 to which the internal parameters 113 learned in step S22 are applied. Calculate the evaluation value g^_i. The image analysis system 1 obtains an evaluation result 202 for each learning image f_i included in the learning image group {f_i}(R). As an example, the evaluation result 202 is the correctness 203 of the estimated evaluation value g^_i and the degree of margin M(f_i) (204).

As described above, in the present embodiment, since the sub-learning image group {f'_j(k)}(T) is used to optimize the internal parameters 113, the evaluation engine 111 can learn efficiently. In addition, since the evaluation engine 111 learns using the sub-learning image group {f'_j(k)} (T) selected using the margin M(f_i) (204) and the similarity, it is more efficient. Intrinsic parameters 113 can be optimized for

Therefore, according to the present embodiment, in image processing using machine learning, high-speed learning is possible even for evaluation targets having various pattern variations.

<Summary of selection probability>
FIG. 3 is a diagram schematically showing an example of the learning state of the evaluation engine 111. As shown in FIG. The concept of calculating the selection probability P_k(f_i) (201) will be described with reference to FIG. As an example of the evaluation engine 111, let us consider non-defective product determination for inspecting whether the evaluation target P is a non-defective product or a defective product. Inside the evaluation engine 111, a plurality of feature values {Ca} (a=1, . It is considered that an estimated evaluation value is output based on this feature amount.

In Fig. 3, the circle and triangle plots show the distribution of the feature values of the learning images in the k-th iterative learning, and the circle and triangle plots are the feature values of the good and bad product learning images, respectively. Plots existing inside the non-defective product cluster 300 in the k-th iterative learning are determined to be non-defective products, and plots existing outside are determined to be defective products.

Therefore, it is necessary to gradually change the internal parameters 113 (the method of calculating the feature amount and the shape of the non-defective cluster) by iterative learning so that the non-defective product cluster 300 separates non-defective products from non-defective products as much as possible. Consider the selection probability P_k(f_i) (201) of the learning image f_i to the sub-learning image group {f'_j(k)}(T) in the k-th iterative learning. White, gray, and black plots represent 'high', 'medium', and 'low' selection probabilities, respectively. In order to simplify the explanation, the selection probabilities are displayed in three stages in FIG. 3, but the actual selection probabilities can take continuous values.

For the learning image 301 of the non-defective product existing near the boundary of the non-defective product cluster 300, even a slight change in the feature amount or the non-defective product cluster 300 may change the accuracy 203 of the estimated evaluation value, so the selection probability is set high. is desirable. The five non-defective product learning images 302 present in the center of the non-defective product cluster 300 have correct estimated evaluation values and are located in the center of the non-defective product cluster 300. The estimated evaluation value is less likely to turn into an incorrect answer. That is, since this is a learning image with a high degree of margin M(f_i) (204), it is desirable to set the selection probability low.

Since the margin M(f_i) (204) of the two non-defective product learning images 303 existing between the boundary and the center of the non-defective product cluster 300 is medium, it is desirable to set the selection probability to be medium as well.

The learning images for defective products are the same as for non-defective products. That is, the learning

images

304, 305, and 306 of defective products are all correct because they are determined to be defective products. It is desirable to set

It is desirable to set the selection probability to be very high in order to improve the judgment results for erroneously determined learning images (307, which is a non-defective product but is erroneously determined as a defective product, and 308, which is a defective product but is erroneously determined as a non-defective product). It is desirable to set a low selection probability for a group of images (309 and 310) with a high degree of similarity (which are likely to be plotted in the vicinity even in the feature space). It is desirable to set a high selection probability for a learning image 311 that is erroneously determined even if it is an image group with a high degree of similarity.

In this way, the selection probability needs to be determined by comprehensively considering the correctness 203 of the estimated evaluation value, the degree of margin M(f_i) (204), the degree of similarity between images, and the like. Determining the sub-learning image group {f'_j(k)}(T) not only reduces the number of learning images in each iterative learning, but also speeds up optimization by prioritizing learning images to be learned. There is also a possibility that good evaluation performance can be obtained with a small number of iterations.

<Selection condition input screen>
In this embodiment, in the learning step (step S2 in FIG. 1), the number of images Nf of the sub-learning image group {f'_j(k)}(T) is calculated from the number of images Nf of the learning image group {f_i}(R). It is characterized by having a GUI for accepting designation of a reduction rate R_k for _k, and the reduction rate R_k being a function of the number k of iterative learning.

Supplementary information about this feature. It is possible to specify the reduction rate R_k of the number of learning images in the k-th iterative learning, taking into consideration the constraints on the learning time and the like. The reduction rate R_k can be defined by, for example, (Nf-Nf'_k)/Nf*100. In this case, the larger the value, the more the training images are reduced and the learning time is shortened. Also, the reduction rate R_k can be changed by the number k of iterative learning.

Generally, at the beginning of iterative learning, the internal parameters are not fixed and extensive parameter search is required, so it is desirable to set the reduction rate R_k small. On the other hand, since the internal parameters begin to converge to optimal values in the latter stage of iterative learning, the reduction rate R_k can be set large.

Although the reduction rate R_k was used as an example, the specified value may be the number of images Nf'_k of the sub-learning image group {f'_j(k)} (T), or the estimated learning time, instead of the reduction rate R_k. When the estimated learning time is specified, the number of images Nf'_k of the sub-learning image group {f'_j(k)} (T) is set so that learning is completed within the estimated learning time based on the number of iterations of learning. will be determined.

FIG. 4 is a diagram showing an example of a selection condition input screen 400. FIG. The selection condition input screen 400 is a GUI (Graphical User Interface) for the user to specify the learning method of the evaluation engine 111 . By checking the check box 401, the designation of the reduction rate R_k becomes effective. The method of giving the reduction rate R_k can be selected by radio buttons 402-404.

When the radio button 402 is selected, the reduction rate R_k is a specified constant value 406, regardless of the number of iterations k, as given by a straight line 405. When the radio button 403 is selected, the reduction rate R_k is specified by a polygonal line 407. In the illustrated example, the reduction rate R_k increases until the epoch (the number of iterations of learning) specified in the box 408, and after that, it remains constant as specified in the box 409. value. If radio button 404 is selected, reduction rate R_k will be curve 410 specified in box 411 . These are examples of how to specify the reduction rate R_k, and any shape can be specified. The image analysis system 1 accepts the specification of the reduction rate R_k setting method from the user, so that the evaluation engine 111 can be learned more efficiently according to needs.

A method for calculating the selection probability P_k(f_i) (201) of the learning image f_i can be specified using the GUI shown in FIG. As described above, the selection probability P_k(f_i) (201) can be calculated using the correctness 203 of the estimated evaluation value, the degree of margin M(f_i) (204), the similarity between the learning images, and the like. In other words, the selection probability P_k(f_i) (201) can be given by a function having these judgment materials as arguments. Arguments to be considered in calculating the selection probability P_k(f_i) (201) can be designated by check boxes 412-414. Boxes 415-417 can be used to specify the ratio (weight) of each argument to be considered in the calculation of the selection probability P_k(f_i) (201).

In the example of FIG. 4, the values of boxes 415 to 417 are specified as 0.2, 0.6, and 0.2, respectively, and the selection probability P_k(f_i) (201) is determined with emphasis on the margin M(f_i) (204). It will be. Also, a rule can be set for the selection of the learning image f_i, and together with the selection probability P_k(f_i) (201), the sub-learning image group {f'_j(k)} (T) is determined. For example, by checking a check box 418, it is possible to validate the rule that training images that were incorrect in the previous iterative learning must be selected in the next iterative learning.

Also, by checking the check box 419, it is possible to validate the rule that learning samples with margins equal to or less than the threshold are always selected. A threshold can be specified in box 421 . Also, by checking the check box 420, it is possible to validate the rule that training images with a degree of similarity equal to or higher than the threshold are always thinned out. A threshold can be specified in box 422 . The arguments and rules described here are examples, and other arguments and rules can be set.

In this way, the user can use various means to specify what kind of learning image should be preferentially learned in each iterative learning. By incorporating the user's domain knowledge into the evaluation object P, more appropriate learning becomes possible.

<Hardware configuration>
FIG. 5 is a diagram showing an example of the hardware configuration of the image analysis system 1. As shown in FIG. The image analysis system 1 has the aforementioned imaging device 106 and computer 100 . The imaging device 106 is as described above. The computer 100 is a component for processing the image evaluation method in this embodiment, and includes a processor 101 , a memory resource 102 , a GUI device 103 , an input device 104 and a communication interface 105 .

The processor 101 is a processing device such as a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit), but is not limited thereto, and may be any device capable of executing the above-described image analysis method. , one processor 101 may be single core or multicore, or a circuit (for example, FPGA (Field -Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit).

The storage resource 102 is a storage device such as RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), non-volatile memory (flash memory, etc.), etc., and is a memory from which programs and data are temporarily read. act as an area. The storage resource 102 may store a program (referred to as an image analysis program) that causes the processor 101 to execute the image analysis method described in the above embodiments.

The GUI device 103 is a device that displays a GUI, such as a display such as an OLCD (Organic Liquid Crystal Display) or a projector, but is not limited to this example as long as it can display a GUI. The input device 104 is a device that receives an input operation from a user, and is, for example, a keyboard, mouse, touch panel, or the like. The input device 104 is not particularly limited as long as it is a component capable of receiving operations from the user, and the input device 104 and the GUI device 103 may be integrated.

The communication interface 105 is USB, Ethernet, Wi-Fi, etc., and is an interface that mediates input/output of information. Note that the communication interface 105 is not limited to the example shown here as long as it is an interface that can directly receive an image from the imaging device 106 or that allows the user to transmit the image to the computer 100 . A portable non-volatile storage medium (for example, flash memory, DVD, CD-ROM, Blu-ray disc, etc.) storing the image can be connected to the communication interface 105 and the image can be stored in the computer 100 .

As described above, according to the present embodiment, in image processing using machine learning, high-speed learning is possible even for evaluation targets having various pattern variations.

In addition, as described above, the embodiments described so far do not limit the invention according to the scope of claims, and all of the elements described in the embodiments and combinations thereof are means for solving the invention. is not necessarily required for

Additionally, there may be a plurality of computers 100 and a plurality of imaging devices 106 that configure the image analysis system 1 . Further, the image analysis program described above can be distributed to the computer 100 by connecting a portable nonvolatile storage medium storing the image analysis program to the communication interface 105 . Alternatively, the image analysis program can be distributed to computer 100 by a program distribution server. In this case, the program distribution server has a storage resource 102 storing the image analysis program, a processor performing distribution processing for distributing the image analysis program, and a communication interface device capable of communicating with the communication interface device of the computer 100. . Various functions of the image analysis program distributed or distributed to the computer 100 are realized by the processor 101 .

Further, as described above, the image analysis system 1 has a learning phase 110 in which the evaluation engine 111 learns, and an evaluation phase 120 in which the evaluation image S is evaluated using the evaluation engine 111 learned in the learning phase 110. Execute. The processor 101 executing the learning phase 110 and the processor 101 executing the evaluation phase 120 may be the same or different. If the processor 101 executing the learning phase 110 and the processor 101 executing the evaluation phase 120 are different, the processor 101 executing the learning phase 110 will inform the processor 101 executing the evaluation phase 120 of the internal parameters 113 of the evaluation engine 111 can be handed over.

1: Image analysis system, 100: Calculator, 101: Processor, 102: Storage resource, 103: GUI device, 104: Input device, 105: Communication interface, 106: Imaging device, 110: Learning phase, 111: Evaluation engine, 112 : learning image selection engine, 113: internal parameter, 120: evaluation phase, 201: selection probability P_k(f_i), 202: evaluation result, 203: correctness of estimated evaluation value, 204: margin M(f_i), 400: selection Condition input screen, 401, 412, 413, 414, 419, 420: check box, 402, 403, 404: radio button, 405: straight line, 406: value, 407: polygonal line, 408, 409, 411, 415, 416, 417, 421, 422: Box, 410: Curve, P: Evaluation object, Q: All learning image group, R: Learning image group {f_i}, S: Evaluation image, T: Sub-learning image group {f'_j( k)}

Claims

An image analysis system comprising at least one processor and memory resources,
The processor
*Learning image acquisition step of imaging the evaluation object for learning and acquiring learning image group {f_i} (i=1,...,Nf, Nf: number of learning images) *Evaluation using learning image group {f_i} a learning step for learning the engine * an evaluation image acquisition step for capturing an image of an evaluation target to acquire an evaluation image * an evaluation step for inputting the evaluation image into a trained evaluation engine and outputting an estimated evaluation value,
In the learning step, sub-learning image group {f'_j(k)} (j=1,...,Nf'_k, Nf'_k: sub-learning image number, {f'_j(k)}⊂{f_i}, k: number of iterative learning) is determined by the image selection engine, and the k-th An image analysis system characterized by performing iterative learning of
The image analysis system of claim 1, wherein
In the learning step, calculating a degree of margin M(f_i) that quantifies the degree of correctness of the estimated evaluation value output when the learning image f_i is input to the evaluation engine during iterative learning,
The image analysis characterized in that the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is a function of the margin M(f_i). system.
The image analysis system of claim 1, wherein
In the learning step, the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is the learning image f_i and the other learning image group {f_a}( An image analysis system characterized by being a function of similarity between a≠i).
The image analysis system of claim 1, wherein
In the learning step, having a GUI for accepting designation of a reduction rate R_k from the number of images Nf of the learning image group {f_i} to the number of images Nf'_k of the sub-learning image group {f'_j(k)},
The image analysis system, wherein the reduction rate R_k is a function of iterative learning times k.
An image analysis method by an image analysis system,
a learning image acquisition step of imaging an evaluation target for learning and acquiring a learning image group {f_i} (i=1, . . . , Nf, Nf: the number of learning images);
a learning step of learning the evaluation engine using the training image group {f_i};
an evaluation image obtaining step of imaging an evaluation target to obtain an evaluation image;
an evaluation step of inputting the evaluation image into a trained evaluation engine and outputting an estimated evaluation value;
In the learning step, sub-learning image group {f'_j(k)} (j=1,...,Nf'_k, Nf'_k: sub-learning image number, {f'_j(k)}⊂{f_i}, k: number of iterative learning) is determined by the image selection engine, and the k-th An image analysis method characterized by performing iterative learning of
The image analysis method according to claim 5,
In the learning step, calculating a degree of margin M(f_i) that quantifies the degree of correctness of the estimated evaluation value output when the learning image f_i is input to the evaluation engine during iterative learning,
The image analysis characterized in that the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is a function of the margin M(f_i). Method.
The image analysis method according to claim 5,
In the learning step, the selection probability P_k(f_i) of the learning image f_i to the sub-learning image group {f'_j(k)} in the k-th iterative learning is the learning image f_i and the other learning image group {f_a}( A method of image analysis, characterized in that it is a function of similarity between a≠i).
The image analysis method according to claim 5,
In the learning step, receiving through a GUI a designation of a reduction rate R_k from the number of images Nf of the learning image group {f_i} to the number of images Nf'_k of the sub-learning image group {f'_j(k)};
An image analysis method, wherein the reduction rate R_k is a function of the iterative learning number k.
A program that causes a processor to execute the image analysis method according to any one of claims 5 to 8.