JP7308775B2 - Machine learning method and information processing device for machine learning - Google Patents

Description

The present invention relates to information processing technology related to machine learning.

Processing devices using machine learning are required to improve the reliability and accuracy of processing such as recognition obtained by learning.

For example, Patent Literature 1 discloses a technique for selecting learning data in an image recognition technique using machine learning. According to the technology disclosed in Patent Literature 1, it is possible to suppress deterioration in recognition accuracy of image recognition processing due to learning due to unexpected environmental changes.

In addition, Non-Patent Document 1 reports a technique for evaluating epistemic uncertainty and oleatoric uncertainty in recognition in deep learning.

Japanese Unexamined Patent Application Publication No. 2019-21201

“What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?” Alex Kendall et.al, 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.

In Patent Document 1, when image recognition processing is performed, data with low reliability and low frequency of discrimination are selected as data to be used for learning.

However, in cases where the amount of data to be analyzed is large, such as material analysis using an electron microscope, and the data or objects to be recognized from the data have different levels of importance, the reliability of discrimination is low and the frequency of data is low. is used for learning, the analysis accuracy may decrease. For example, even if the shooting environment changes, it is required to be able to detect objects of higher importance.

An object of the present invention is to provide a method of presenting learning conditions that can improve the accuracy of image analysis.

A preferred aspect of the present invention includes a correct/incorrect information generating unit that generates correct/incorrect information of an image analysis result, a reliability determination unit that determines reliability related to analysis in image analysis processing, and a learning condition based on the correct/incorrect information and reliability. and a learning condition output unit that presents the information processing apparatus for machine learning.

Another preferable aspect of the present invention is a machine learning method for an image analysis device in which a model to which machine learning using learning data is applied is implemented, and image analysis is performed on an input image. In this method, the reliability of the image analysis is evaluated including at least one of the reliability of the model and the reliability of the data in the first step of judging whether the analysis result of the image analysis device regarding the input image is correct or incorrect. , the second step, based on the reliability of correctness judgment and image analysis, maintain the status quo, recommend using the input image as learning data, recommend not using the input image as learning data, and a change in the learning set value. a third step, when the analysis result is positive and the model confidence is high, present the status quo; when the analysis result is positive and the model confidence is low , presenting a recommendation to use the input image as learning data, and if the analysis result is incorrect and the reliability of the model is high, presenting a change in the learning set value, and indicating that the analysis result is incorrect. If there is and the reliability of the model is small, a recommendation to use the input image as training data is presented.

Describing another more specific aspect, performing a fifth step of comparing the reliability of the data with a predetermined threshold value to identify large or small; When the reliability of the image is high, presenting the status quo, when the analysis result is positive and the reliability of the data is low, presenting a recommendation not to use the input image as learning data, and analyzing If the result is incorrect and the reliability of the data is high, a change in the learning set value is suggested, and if the analysis result is incorrect and the reliability of the data is low, the input image is used as learning data. present a recommendation not to use

According to the present invention, it is possible to present learning conditions that can improve the accuracy of image analysis. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments.

It is a block diagram which shows the functional structural example of the information processing apparatus for machine learning. FIG. 4 is an explanatory diagram showing an example of an image to be analyzed and an image of an analysis result; FIG. 10 is an explanatory diagram for explaining an example of a method for a user to select whether an analysis is correct or incorrect; FIG. 5 is a table diagram for explaining an example of learning condition determination; FIG. 10 is an explanatory diagram showing an example of correspondence between reliability and correctness/incorrectness of an analysis result; It is a flow chart which shows information processing for machine learning. FIG. 4 is a plan view showing an example screen of an interface used in information processing for machine learning;

Hereinafter, embodiments of the present invention will be described in detail based on the drawings. In addition, the embodiments described below do not limit the invention according to the claims, and all of the elements described in the embodiments and their combinations are essential to the solution of the invention. Not necessarily. In addition, the embodiments disclosed in the present application also include forms in which various elements described in the embodiments are appropriately combined.

In the configuration of the invention described below, the same reference numerals may be used in common for the same parts or parts having similar functions between different drawings, and redundant description may be omitted.

When there are a plurality of elements having the same or similar functions, they may be described with the same reference numerals and different suffixes. However, if there is no need to distinguish between multiple elements, the subscripts may be omitted.

Notations such as “first”, “second”, “third” in this specification etc. are attached to identify the constituent elements, and do not necessarily limit the number, order, or content thereof isn't it. Also, numbers for identifying components are used for each context, and numbers used in one context do not necessarily indicate the same configuration in other contexts. Also, it does not preclude a component identified by a certain number from having the function of a component identified by another number.

The position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, ranges, etc. disclosed in the drawings and the like.

All publications, patents and patent applications cited herein are hereby incorporated by reference into this description.

Elements presented herein in the singular shall include the plural unless the context clearly dictates otherwise.

An example of the information processing device for machine learning described in the embodiments is used for supporting learning of an analysis device to which machine learning is applied. As machine learning, learning data (teacher data) is used to learn a neural network. This device includes a correct/incorrect information generation unit that generates correct/incorrect information of the image analysis result, a reliability determination unit that determines the reliability related to the analysis in the image analysis process, and a learning condition that presents learning conditions based on the correct/incorrect information and reliability. and a condition output unit.

Such an information processing device for machine learning can be configured by a general information processing device such as a server. Like a general server, the information processing device for machine learning includes a control device, a storage device, an input device, an output device, and a bus connecting each part. It is assumed that the program executed by the device is preinstalled in the storage device. In the following description, the control device, storage device, input device, and output device naturally included in the information processing device will not be illustrated, and the description will focus on the functions realized by the device.

Specifically, each function is stored in a storage device and realized by a program executed by a processing device. That is, in the present embodiment, functions such as calculation and control are realized by cooperating with other hardware by executing a program stored in the storage device by the processing device. A program executed by a computer or the like, its function, or a means for realizing that function may be referred to as a "function," "means," "unit," "unit," "module," or the like.

The configuration of the information processing device for machine learning may be composed of a single device, or any part of the input device, output device, processing device, and storage device may be composed of other computers connected by a network. may be The idea of the invention is equivalent, and there is no change.

In the present embodiment, functions equivalent to those configured by software can also be realized by hardware such as FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit). Such aspects are also included in the scope of the present invention.

FIG. 1 is a block diagram showing a configuration example of an information processing apparatus 100 for machine learning according to an embodiment. The machine learning information processing apparatus 100 includes a correct/incorrect information generation unit 101 , a reliability determination unit 102 , a learning condition output unit 103 and a display content generation unit 104 . The correct/wrong information generation unit 101 , reliability determination unit 102 , learning condition output unit 103 and display content generation unit 104 are connected via a bus 106 . The bus 106 holds or mediates transmission of data, control information and analysis information handled by each processing unit connected to the bus 106 .

As described at the beginning, in this embodiment, the correctness/incorrectness information generation unit 101, the reliability determination unit 102, the learning condition output unit 103, and the display content generation unit 104 are implemented by software. In addition, the machine learning information processing apparatus 100 is assumed to include various input/output devices and interfaces that are normally provided as an information processing apparatus.

The machine learning information processing apparatus 100 is connected to an image analysis apparatus 110 and an input/output apparatus 111 via a bus 106 . The information processing apparatus for machine learning 100 may be wired or wirelessly connected to the image analysis apparatus 110 and the input/output apparatus 111 . Although the image analysis device 110 and the input/output device 111 are provided outside the information processing device 100 for machine learning in FIG. 1, they may be built in the information processing device 100 for machine learning.

The image analysis device 110 has a function of analyzing a specific target from an image. The process of analyzing an image may classify the image as to whether or not it has a specific feature and calculate one or more classification results, or detect an area containing a specific target in the image, The position and size of the target in the image may be calculated, or the area including the specific target in the image may be divided into segments and calculated as an image in which the color is different from that of other targets. Hereinafter, processing for analyzing a specific target from an image, represented by image classification, image detection, and image segmentation, will be referred to as image analysis. A known technique is used for image analysis processing.

As a typical example, the image analysis device 110 can be configured with a deep neural network (DNN) or the like in which internal parameters are set by supervised learning. The configuration may be implemented by software, or may be configured by hardware such as FPGA. The image analysis device 110 is generally defined as a function approximator that produces a given output in response to a given input. In the case of a general function approximator, the input may be image data as described above, or voice or other types of data.

FIG. 2 shows an example of an image analysis result by the image analysis device 110. As shown in FIG. A photographed image 210 shows a state in which a crescent-shaped object 201 and an elliptical object 202 are mixed as an example of an image photographed by a microscope. For example, when the captured image 210 is input, the image analysis device 110 identifies and classifies the crescent-shaped object 201 and the elliptical object 202 .

For example, for this photographed image 210, image analysis is performed such that the crescent-shaped object 201 is colored white and segmented, and the elliptical object 202 and the background are colored gray and segmented. Shown in image 220 .

In the analysis result image 220, there are segments 211A, 211B, and 211D that correctly segment the crescent-shaped objects 201A, 201B, and 201D in white, and segments 212A that correctly segment the elliptical object 202A in gray. On the other hand, crescent-shaped object 201C contains incorrectly segmented segment 211C, and elliptical object 202B contains incorrectly segmented segment 212B.

The input/output device 111 has a function of outputting, to the information processing device 100 for machine learning, information regarding the correctness or wrongness of the analysis result processed by the image analysis device 110 . As an example of outputting information about correctness/incorrectness, for example, the analysis result by the image analysis device 110 is displayed as an image on the input/output device 111 in the form of an analysis result image 220, and the user selects whether the analysis is correct/incorrect. The result obtained is output to the information processing apparatus 100 for machine learning.

In this case, the input/output device 111 is composed of, for example, a PC having a display for displaying images and a mouse or keyboard that can be operated by the user. is input by the user. For example, a touch panel display of a smartphone or a tablet terminal may be used, an image result may be printed by a printer, or a user's selection may be input by voice.

The information processed by the input/output device 111 and output to the information processing device 100 for machine learning may be selected by the user as to whether the analysis result is correct or incorrect, or may be the expected analysis result created by the user. As another example of outputting information about correctness/incorrectness, information obtained by analysis means different from the image analysis device 110 is output from the input/output device 111 to the machine learning information processing device 100 . For example, when an image of an object captured by a monocular camera is used to analyze the distance from the camera to the object by the image analysis device 110, the distance measured in advance by a 3D sensor such as a laser scanner or a TOF camera is used as correct information. The input/output device 111 outputs this correct answer information to the information processing device 100 for machine learning. A known technique is used as means for obtaining correct information.

An example of a method for the user to select whether the analysis is correct or incorrect will be described with reference to FIG. The image 310 presented to the user is created based on the analysis result image 220 and prompts the user to select whether the analysis result is correct or not. As an example of selection by the user, in FIG. 3, the cursor 301 is operated to select whether each segment is correct or incorrect. A correct/wrong selection result image 320 shows the result selected by the user.

As a result of selection by the user, when the analysis result is judged to be correct (segmentation of the crescent-shaped object 201 in white or segmentation of the elliptical object 202 in gray), "Correct" is displayed. , those judged to have an incorrect analysis result (segmentation of objects other than the crescent-shaped object 201 into white, or segmentation of the crescent-shaped object 201 into gray) are “wrong”, and the analysis result is not important (hereinafter ) are displayed as "unselected".

The correctness/incorrectness information generation unit 101 generates correctness/incorrectness information of the image analysis result based on the information input from the input/output device 111 . Although the granularity of the correct/incorrect information is arbitrary, in the example of FIG. 3, the correct/incorrect information is given for each segment. That is, in this example, the correct/incorrect information corresponds to a partial area of the image. Alternatively, correct/wrong determination may be performed for each image based on the correct/wrong count or the correct/wrong ratio.

The reliability determination unit 102 determines reliability of image analysis. Here, reliability refers to uncertainty in image analysis. When image analysis processing is statistically modeled by machine learning, etc., if the modeling is not sufficiently precise, the modeled analysis processing will generate uncertainty when given data that is highly difficult to analyze. This may cause variations in the analysis results and reduce the accuracy of the analysis. At this time, it can be said that "the model is uncertain", which is hereinafter referred to as "the model is unreliable". This case is called uncertainty in recognition in Non-Patent Document 1 mentioned above.

In addition, even if the modeling of the image analysis process is sufficiently sophisticated, if the data to be analyzed contains local noise compared to others, and the difficulty of analysis is high, the data may be subject to uncertainty. , the analysis results may vary and the accuracy of the analysis may decrease. At this time, it can be said that "the data are uncertain", and hereinafter referred to as "the data are unreliable". This case is called accidental uncertainty in Non-Patent Document 1 mentioned above.

The reliability determination unit 102 calculates the reliability of the model and the reliability of the data. The reliability determined by the reliability determination unit 102 may be only the reliability of the model, only the reliability of the data, or both the reliability of the model and the reliability of the data. For the method of calculating the reliability of the model and the reliability of the data, for example, the known technique described in Non-Patent Document 1 is used. As described in Non-Patent Document 1, the reliability of a model and the reliability of data can be visually indicated as continuous values corresponding to a partial area of an image, for example, by contrast or color differences. can.

The learning condition output unit 103 determines and presents a learning condition based on the correctness/incorrectness information of the image analysis result generated by the correctness/incorrectness information generation unit 101 and the image analysis reliability determined by the reliability determination unit 102 .

FIG. 4 is a table showing an example of rules for determining learning conditions based on correct/incorrect information of image analysis results and reliability of image analysis. Such rules are determined in advance by the user and stored in a storage device (not shown) of the information processing apparatus 100 for machine learning as data in table format, for example. The learning condition output unit 103 makes determination by referring to the rule.

Table 410 in FIG. 4 is an example of judging the learning condition from the correctness information of the analysis result and the reliability of the model. Judging. The threshold is defined in advance by the user as part of the rule.

In 410(1), it is determined that the analysis result is correct and the reliability of the model is high, so the learning of the image analysis device 110 is considered appropriate. In this case, the determination result is presented so as to maintain the learning conditions (maintain the status quo).

In 410 (2) and (4), since the reliability of the model is low, it is determined that the analysis accuracy is improved by learning by adding the corresponding data to the learning data. By setting the threshold value b smaller than the threshold value a, it is possible to preferentially add (4) with an erroneous analysis result to the learning data.

FIG. 5 shows the relationship between the correct/wrong selection result image 320 and the reliability display image 510 . The reliability display image 510 is also disclosed in Non-Patent Document 1 and indicates the reliability of the model, but the reliability of the data can be similarly displayed. In this example, portions where the reliability is lower than the threshold are shown in gray, and portions above the threshold are shown in white. Here, the portion of segment 511 corresponds to 410(2) where the analysis result is correct but the reliability of the model is low. The portion of segment 512 corresponds to 410(4) where the analysis result is erroneous and the reliability of the model is low.

For addition to the learning data, a set of the original image (input image to the image analysis device 110) corresponding to the correct/wrong selection result image 320 and the correct answer may be added to the learning data. Alternatively, a corresponding portion (for example, a portion corresponding to the segment 511 or 512) in the image may be specified with the cursor 301, cut out, and added to the learning data.

At 410(3), the confidence of the model is high even though the analysis result is incorrect, and corresponds to segment 513 of FIG. Here, it is judged that it is more effective to significantly change the model by changing the learning set value than by adding learning data. The learning setting values are, for example, parameters for setting learning conditions when learning is executed, and hyperparameters for tuning a model.

On the other hand, a table 420 in FIG. 4 is an example of judging the learning condition from the correct/incorrect information of the analysis result and the reliability of the data. It is determined whether

In 420 ( 5 ), since it is determined that the analysis result is correct and the reliability of the data is high, the determination result is presented so as to maintain the learning conditions.

In 420 ( 6 ) ( 8 ), it is thought that the reliability of the data is low and accidental noise is generated in the relevant data, and even if the relevant data is added to the learning data, the analysis accuracy does not improve. Analysis accuracy may decrease. That is, it is better not to use the applicable data for learning data. For this reason, we present that the use of this data as learning data is not recommended. In order to add more training data and improve the analysis accuracy, we propose to reacquire the data. Reacquisition can be both acquiring an image of a different object and reacquiring an image of the same object. This is because, even for images of the same object, there is a possibility that images suitable for learning data can be acquired depending on conditions (for example, imaging conditions). Specifically, it is expected that noise and halation in the image will be improved by re-shooting.

In 420 ( 7 ), the reliability of the data is high despite the fact that the analysis result is incorrect, and it is more effective to change the learning settings to significantly change the model than to reacquire the data. I believe there is.

The display content generation unit 104 generates display content to be presented to the user when the user selects whether the image analysis is correct or incorrect using the input/output device 111, as described with reference to FIG. Any display content that allows the user to determine whether the analysis result is correct or incorrect may be used. In the case of image classification, the classification result may be displayed as characters or as an image. In the case of image detection, the detected area of the image may be displayed with a frame or color added, or only the detected area may be cut out and displayed. In the case of image segmentation, the segmented areas of the image may be colored and displayed, or only the segmented areas may be cut out and displayed. In addition, a known technique may be used as long as it is a technique for indicating the result of image analysis of an image. In addition, display contents may be such that two or more image analysis results are displayed side by side, and correct or incorrect image analysis is selected from among them.

The display content generation unit 104 also generates image data for displaying the output result of the learning condition output unit 103 . Also, an interface image is generated as will be described later with reference to FIG.

One of the determination based on the reliability of the model and the determination based on the reliability of the data shown in FIG. 4 may be performed independently, or both may be performed. When performing both, first make a judgment based on the reliability of the data, exclude images that are not recommended to be adopted as learning data in 420 (6) and (8), and then make a judgment based on the reliability of the model. By doing so, it is preferable to prevent non-recommended data from being used as learning data.

FIG. 6 is a flowchart showing information processing for machine learning by the information processing apparatus 100 for machine learning. FIG. 6 shows a flow chart of the process of determining the learning condition from the correct/incorrect information of the analysis result and the reliability of the model.

Inputs to the information processing apparatus 100 for machine learning are an image analysis result input from the image analysis apparatus 110 and a correct answer input from the input/output device 111 by the user, for example. When it is assumed that learning data will be added, a new image (learning data candidate to be added) other than the learning data is used as the analysis target of the image analysis device 110 .

In S<b>601 , correct/incorrect information is generated by the correct/incorrect information generation unit 101 based on information input from the image analysis device 110 and the input/output device 111 . The correct/incorrect information may be given to each of the analysis result images, or may be given to each of one or more analysis results included in the analysis result image as in the example of FIG. Further, the input/output device 111 may generate correct/wrong information only for images judged correct/wrong by the user. In that case, data targeted in S603 is limited to data judged correct/wrong by the user.

In S602, the reliability determination unit 102 determines the reliability of the model. For the reliability of the analysis result image, for example, it may have a numerical value corresponding to the number of image sizes (horizontal pixel number × vertical pixel number), or a single value for one analysis result image A numerical value may be used. A specific example of reliability determination is disclosed in Non-Patent Document 1, for example.

In S603, the data to be referred to in this embodiment (for example, learning data candidates to be added) are determined from the analysis result image, and the processing from S604 to S610 is executed until all the target data are referred to. When all references are completed, the process ends.

In S604, the correct/wrong information of the analysis result is referred to, and if the analysis result is "correct", the process proceeds to S605, and if the analysis result is "wrong", the process proceeds to S606. In S605, it is determined whether or not the reliability of the model is equal to or greater than the threshold a, and the process proceeds to S607 or S608. In S606, it is determined whether or not the reliability of the model is equal to or greater than the threshold value b, and the process proceeds to S609 or S610. The learning conditions shown in S607 to S610 are as described above.

FIG. 6 illustrates determination using model reliability, but similar processing can be performed using data reliability. In that case, it goes without saying that c and d are used as thresholds.

FIG. 7 shows an example of a GUI (Graphical User Interface) used when executing the information processing apparatus 100 for machine learning.

A GUI screen 700 includes an execution mode 704, a field 705 for specifying a directory storing an initial deep network model, a field 706 for specifying a directory storing input data and learning data, estimation result data, reliability data, and post-learning data. A field 707 for specifying a directory for storing the deep network model, a field 708 for specifying a directory for storing the recommended data after analysis and the deep network model, a field 711 for inputting an execution parameter or an analysis parameter, an execution button 714, a report It has an output button 715 , a screen 702 for displaying an image or reliability data, and fields 701 , 703 , 716 to 721 for designating information to be displayed on the screen 702 .

The content of the screen 700 is generated by the display content generation unit 104 and displayed on, for example, an image monitor of the input/output device 111 . A column 705 selects a model to be installed in the image analysis device 110 . Column 706 selects and recalls various data stored in the storage device. In column 707, the generated data is saved in the storage device. A column 708 stores recommended data and models obtained as a result of the processing shown in FIG. In column 711, parameters to be set during model learning or execution are entered. An execution button 714 executes learning and analysis processing. A report output button 715 outputs, for example, the recommended data obtained as a result of the processing shown in FIG. 6 in a report format. On the screen 702, the images shown in FIGS. 2, 3, and 5 are displayed simultaneously or alternately.

Columns 716 and 719 are input data (which can be displayed, for example, as 210 in FIG. 2), correct data (learning data), training data, inference result data (which can be displayed, for example, as 220 in FIG. 2), and reliability data. (For example, it can be displayed as 310 in FIG. 3) can be selected. Columns 717 and 720 list the data selected in columns 716 and 719 . Each data is stored in the storage device of the information processing apparatus for machine learning 100 or in an external storage device. Columns 718 and 721 are gauges that indicate the position of the list displayed in columns 717 and 720 .

In column 701 , synthesis (shading) or synthesis (color) can be selected, and data obtained by synthesizing the data selected in columns 717 and 720 at the gauge ratio of 703 is displayed on screen 702 . That is, in this example, different images can be displayed in an overlapping manner. Alternatively, a plurality of images may be displayed side by side.

Execution modes 704 may include training, inference, analysis, and the like.

First, set the execution mode 704 to training mode or inference mode, set appropriate directories in columns 705 to 707 and execution parameters in column 711, and press the execution button 714. Then, according to the set directory and execution parameters, Training or inference of the model of the image analysis device 110 is performed. Model training and inference can be performed by known procedures. After execution, the trained deep network model, inference result data, and reliability data are stored in the output directory.

After execution, the desired data can be displayed on the screen 702 by setting fields 701, 703, 716-721. That is, input data, learning data, training data, inference result data, and reliability data can be referenced.

Next, the analysis mode is set, and a directory in which recommended data after analysis is stored is set in a field 708 . When the analysis mode is executed, the processing shown in FIG. 6 is executed. The recommended data after analysis is data that is recommended to be added as learning data, and is data that has been added in steps S608 and S610 of FIG.

By setting columns 701, 703, and 716 to 721, displaying data to be displayed on the screen 702, and selecting correct/incorrect, the correct/incorrect information generation processing S601 of the analysis result can be executed. However, the correctness/incorrectness information of the analysis result may be generated and stored in advance. After that, by pressing an execution button 714 , the processes S 602 to S 610 are executed and the recommended data is stored in the directory of column 708 .

Also, when there is a determination result that the learning conditions should be maintained in the process S607 of FIG. If there is a determination result that the learning set value should be changed in the processing S609 of FIG.

After executing additional learning or parameter change in this way, the training mode is set, the column 706 is set to the directory in which the recommended data is stored, and the directory in the column 707 is displayed by pressing the execution button 714. , the improved deep network model after training, inference result data, and reliability data are stored.

Information processing apparatus for machine learning 100 , correct/incorrect information generation unit 101 , reliability determination unit 102 , learning condition output unit 103 , display content generation unit 104 , bus 106

Claims (15)

a correct/incorrect information generation unit that generates correct/incorrect information of the image analysis result;
A reliability determination unit that determines reliability related to analysis in image analysis processing;
a learning condition output unit that presents a learning condition based on the correctness information and the reliability;
An information processing device for machine learning. In the correct/incorrect information generation unit,
generating the correct/incorrect information from the image analysis result and a user's operation result input with respect to the image analysis result;
2. The information processing apparatus for machine learning according to claim 1, wherein: In the correct/incorrect information generation unit,
The result of analyzing the image to be analyzed by another method is used as correct information, and the correct/incorrect information is generated by comparing the image analysis result with the correct information.
2. The information processing apparatus for machine learning according to claim 1, wherein: In the correct/incorrect information generation unit,
Generate one correct/incorrect information for one image,
2. The information processing apparatus for machine learning according to claim 1, wherein: In the correct/incorrect information generation unit,
generating two or more correct/incorrect information for two or more analysis results contained in one image;
2. The information processing apparatus for machine learning according to claim 1, wherein: The learning conditions are
Adding an image corresponding to the image analysis result for which the correctness information is generated to the learning data,
2. The information processing apparatus for machine learning according to claim 1, wherein: The learning conditions are
Including deprecating the addition of the image corresponding to the image analysis result for which the correctness information was generated to the learning data,
2. The information processing apparatus for machine learning according to claim 1, wherein: The learning conditions are
Including changing settings related to learning,
2. The information processing apparatus for machine learning according to claim 1, wherein: The learning conditions are
Reacquiring an image corresponding to the image analysis result for which the correctness information was generated;
2. The information processing apparatus for machine learning according to claim 1, wherein: In the learning condition output unit,
Reliability is treated as a numerical value, and when the numerical value is greater than a threshold, different learning conditions are presented than when the numerical value is less than the threshold.
2. The information processing apparatus for machine learning according to claim 1, wherein: Regarding the reliability related to the analysis,
using the reliability of the model resulting from the model of image analysis processing,
2. The information processing apparatus for machine learning according to claim 1, wherein: Regarding the reliability related to the analysis,
using the reliability of the data attributed to the image data to be analyzed;
2. The information processing apparatus for machine learning according to claim 1, wherein: A machine learning method for an image analysis device that implements a model to which machine learning using learning data is applied and performs image analysis on an input image,
A first step of judging whether the analysis result of the image analysis device regarding the input image is correct or incorrect;
a second step of assessing reliability of the image analysis including at least one of model reliability and data reliability;
Maintaining the status quo, recommending the use of the input image as learning data, recommending not using the input image as learning data, and learning settings based on the correctness determination and the reliability of the image analysis a third step of presenting at least one of:
A machine learning method that performs performing a fourth step of comparing the reliability of the model to a predetermined threshold to discriminate between large and small;
In the third step,
When the analysis result is positive and the reliability of the model is high, the status quo is presented,
presenting a recommendation to use the input image as learning data when the analysis result is positive and the reliability of the model is small;
When the analysis result is incorrect and the reliability of the model is high, suggesting a change in the learning setting value,
Presenting a recommendation to use the input image as learning data when the analysis result is incorrect and the reliability of the model is low;
14. The machine learning method of claim 13. performing a fifth step of comparing the reliability of the data with a predetermined threshold to identify large or small;
In the third step,
When the analysis result is positive and the reliability of the data is high, the status quo is presented,
presenting a recommendation not to use the input image as learning data when the analysis result is positive and the reliability of the data is low;
When the analysis result is incorrect and the reliability of the data is high, suggesting a change in the learning setting value,
presenting a recommendation not to use the input image as learning data when the analysis result is incorrect and the reliability of the data is low;
14. The machine learning method of claim 13.

Images