WO2023157187A1 - ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法 - Google Patents

ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法 Download PDF

Info

Publication number
WO2023157187A1
WO2023157187A1 PCT/JP2022/006424 JP2022006424W WO2023157187A1 WO 2023157187 A1 WO2023157187 A1 WO 2023157187A1 JP 2022006424 W JP2022006424 W JP 2022006424W WO 2023157187 A1 WO2023157187 A1 WO 2023157187A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
data
output data
training data
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/006424
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
淳 安藤
裕貴 竹内
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Medical Systems Corp
Original Assignee
Olympus Medical Systems Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Medical Systems Corp filed Critical Olympus Medical Systems Corp
Priority to PCT/JP2022/006424 priority Critical patent/WO2023157187A1/ja
Priority to JP2024500822A priority patent/JP7559284B2/ja
Publication of WO2023157187A1 publication Critical patent/WO2023157187A1/ja
Priority to US18/659,852 priority patent/US20240289615A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Definitions

  • the present invention relates to a neural network update device, a neural network update program, and a neural network update method that perform learning using teacher data including images that are inappropriate for AI-based determination.
  • the above AI is realized by constructing a function that, when training data is input, outputs a judgment result corresponding to the training data.
  • Neural networks are often used as functions.
  • AI learning technology using multilayer neural networks is called deep learning.
  • deep learning first, a large amount of teacher data, which is a set of training data and corresponding correct information, is prepared. Correct answer information is created manually by annotation.
  • a neural network is composed of a large number of multiply-accumulate operations, and the multipliers are called weights. "Learning" is performed by adjusting the weights so that the output when the training data included in the teacher data is input to the neural network approaches the corresponding correct information.
  • An inference model which is a trained neural network, can perform "inference” that appropriately derives a solution to unknown inputs.
  • Patent Literature 1 discloses a method of cleansing learning data before learning.
  • An object of the present invention is to provide a neural network update device, a neural network update program, and a neural network update method that can reduce the work of cleansing.
  • An apparatus for updating a neural network includes a processor having hardware, the processor inputs a plurality of training data to a neural network to classify the training data, and the training Output data linked to each data is output, and a plurality of correct information linked to the training data and the output data are compared to calculate a loss value for each of the output data, and the loss value of the output data.
  • a processed loss value is output by comparing it with the correct information, and the neural network is updated using the processed loss value, or processed training data is created by processing the training data linked to the corresponding output data. Then, by inputting the processing training data to the neural network, processing output data that is the result of classifying the processing training data is output, and processing is performed by comparing with the correct information linked to the corresponding output data.
  • a post-loss value is output, and the neural network is updated using the post-processing loss value.
  • a neural network update program is a result of classifying a plurality of training data by inputting a plurality of training data into a neural network in a neural network updating device, and output data associated with each of the training data. is output, and a plurality of correct information linked to the training data and the output data are compared to calculate a loss value for each of the output data, and the loss value among the output data corresponds to a predetermined criterion. selecting data and non-corresponding output data that does not meet the predetermined criteria, creating processed correct information by processing the correct information compared with the corresponding output data, and comparing the corresponding output data with the processed correct information.
  • processed output data which is the result of classifying the processed training data
  • a processed loss value is output by comparing with the correct information linked to the corresponding output data. and update the neural network with the post-processing loss value.
  • a neural network updating method is a neural network updating method using a neural network updating device including a training data acquisition unit, a neural network application unit, and a training data correction unit, wherein the training data acquisition unit includes a plurality of training A result of classifying the training data by acquiring teacher data consisting of data and a plurality of pieces of correct information linked to the training data, and inputting the plurality of training data to the neural network by the neural network application unit, Output data linked to each of the training data is output, and the neural network application unit compares a plurality of correct information linked to the training data and the output data, thereby calculating a loss value for each output data.
  • the training data acquisition unit includes a plurality of training A result of classifying the training data by acquiring teacher data consisting of data and a plurality of pieces of correct information linked to the training data, and inputting the plurality of training data to the neural network by the neural network application unit, Output data linked to each of the training data is output, and the neural network application unit compares a plurality of correct
  • the neural network application unit selects applicable output data whose loss value satisfies a predetermined criterion and non-applicable output data that does not satisfy the predetermined criterion from among the output data, and the teacher data correction unit selects the applicable output data Create processed correct information by processing the correct information compared with, and the neural network application unit outputs a post-processing loss value by comparing the corresponding output data and the processed correct information, and the neural network application unit updates the neural network using the post-processing loss value, or the teacher data correction unit creates processed training data by processing the training data linked to the corresponding output data, and applies the neural network By inputting the processing training data to the neural network, the unit outputs processing output data that is the result of classifying the processing training data, and compares it with the correct information linked to the corresponding output data. A post-processing loss value is output, and the neural network application unit updates the neural network using the post-processing loss value.
  • the present invention it is possible to improve the inference accuracy even if the teacher data containing inappropriate images is used for learning without performing preliminary cleansing.
  • FIG. 1 is a block diagram showing a neural network update device according to a first embodiment of the present invention
  • FIG. FIG. 10 is an explanatory diagram for explaining that the inference accuracy of an inference model obtained by learning is lowered when teacher data including an inappropriate image is used in a comparative example of the neural network update device
  • 4 is a flow chart for explaining the operation of the first embodiment
  • 4 is a flow chart for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment
  • FIG. 10 is an explanatory diagram for
  • FIG. 3 is an explanatory diagram for explaining the effect of the first embodiment in the same example as in FIG. 2;
  • FIG. 4 is a block diagram showing a second embodiment of the present invention;
  • FIG. 9 is a flow chart for explaining the operation of the second embodiment; It is an explanatory view for explaining the operation of the second embodiment. It is an explanatory view for explaining the operation of the second embodiment. It is an explanatory view for explaining the operation of the second embodiment. It is an explanatory view for explaining the operation of the second embodiment. It is an explanatory view for explaining the operation of the second embodiment.
  • FIG. 1 is a block diagram showing a neural network updating device according to the first embodiment of the present invention.
  • a learning loss is calculated, and for teacher data whose learning loss is higher than a predetermined threshold, the correct information is changed to inappropriate for recognition (hereinafter referred to as "unknown"). It improves the inference accuracy of the inference model obtained by learning even when inappropriate images are included in the training data.
  • an endoscopic examination image is used as teacher data and an inference model for performing lesion recognition processing is created will be described. can also be applied.
  • FIG. 2 is an explanatory diagram for explaining a comparative example of the neural network update device.
  • FIG. 2 it will be described that the inference accuracy of an inference model obtained by learning is lowered when teacher data including inappropriate images is used in a comparative example.
  • Teacher data includes training data for learning and correct answer information annotated for each training data.
  • the training data for example, a large number of images obtained by imaging lesions in endoscopy are used.
  • each piece of training data (images P21 to P23) is added with "pancreatic cancer” or "pancreatitis” as correct information depending on the type of lesion in the image.
  • Image portions P21a and P23a in images P21 and P23 are pancreatic cancer.
  • the image portion P22c in the image P22 is pancreatitis, but the image portion P22c is blurred.
  • the image P22 is an image that is removed by cleansing before learning and is not used for learning.
  • Inappropriate images include, for example, blurry images caused by out-of-focus or camera shake, dark images with insufficient amount of light, and images in which the size of a lesion in the image is relatively small.
  • These images P21 to P23 are input to the neural network 2 and learned.
  • the neural network 2 outputs a classification output whose output data is a probability value (hereinafter referred to as score) for each classification.
  • the error between this classification output and the correct answer information is obtained as a learning loss, and the parameters of the neural network 2 are updated so as to reduce the learning loss.
  • a classification output indicating whether the input image is "pancreatic cancer” or "pancreatitis" is obtained.
  • the neural network 2 outputs a classification output indicating "unknown” indicating that the unknown input image does not belong to any of the classifications based on the annotations when creating the teacher data. It can also be done.
  • the training data may contain inappropriate images such as blurred images such as image P22. Even for such inappropriate images, some correct information such as “pancreatic cancer” or “pancreatitis” may be added at the time of annotation as described above. In other words, there are cases in which unknown is not set as correct information for an image of training data even if the image is inappropriate.
  • neural network 2 which is constructed from the results of such repeated learning, was used. The inference accuracy of the inference will decrease.
  • the neural network update device is composed of a data memory 1, a neural network 2, a learning loss calculator 3, a correct information processing unit 4, a learning loss recalculator 5, and a neural network control circuit (hereinafter referred to as NN control circuit) 10. be done. All or each of the learning loss calculation unit 3, the correct information processing unit 4, the learning loss recalculation unit 5, and the NN control circuit 10 use a CPU (Central Processing Unit) or FPGA (Field Programmable Gate Array) or the like. It may be composed of one or more processors. The one or more processors may operate according to a program stored in a memory (not shown) to control each part, or may implement part or all of the functions with hardware electronic circuits. may Further, the neural network 2 may be configured by hardware, and the functions of the neural network 2 may be implemented by programs.
  • a CPU Central Processing Unit
  • FPGA Field Programmable Gate Array
  • the data memory 1 is composed of a predetermined storage medium and stores teacher data including a plurality of training data and correct answer information. As described above, correct information indicating classification other than unknown is assigned to all training data.
  • the data memory 1 is controlled by the NN control circuit 10 to output training data to the neural network 2 and output correct information to the learning loss calculator 3 and the correct information processor 4 .
  • the neural network 2 is composed of an input layer, an intermediate layer (hidden layer), and an output layer, which consist of multiple nodes indicated by circles. Each node is connected to the nodes in the preceding and succeeding layers, and each connection is given a parameter called a weighting factor. Learning is a process of updating parameters so as to minimize learning loss, which will be described later.
  • a convolutional neural network CNN
  • CNN convolutional neural network
  • the NN control circuit 10 is composed of an input control section 11, an initialization section 12, an NN application section 13 and an update section 14.
  • An input control unit 11 as a teacher data acquisition unit acquires training data and teacher data including correct answer information, stores the acquired teacher data in the data memory 1, and controls the output of the training data and the correct answer information in the data memory 1.
  • FIG. Initialization unit 12 initializes the parameters of neural network 2 .
  • the NN application unit 13 applies the training data read from the data memory 1 to the neural network 2 and causes the neural network 2 to output a classification output.
  • the updating unit 14 updates the parameters of the neural network 2 based on the learning loss.
  • the neural network 2 is controlled by the NN control circuit 10 to output a probability value (score) indicating which classification each image has a higher probability of being classified as a classification output for each input image.
  • This classification output is given to the learning loss calculator 3 and the learning loss recalculator 5 .
  • the learning loss calculator 3 receives correct information assigned to each image corresponding to each classification output from the data memory 1, and obtains an error between each classification output and each correct information as a learning loss. In the comparative example of FIG. 2 described above, the parameters of the neural network 2 are updated based on this learning loss.
  • the learning loss from the learning loss calculator 3 is supplied to the correct information processing unit 4 (also referred to as teacher data correction unit).
  • the correct information processing unit 4 calculates a loss value (learning loss) for each output data by comparing a plurality of pieces of correct information linked to the training data with the output data. Then, the correct information processing unit 4 selects applicable output data whose loss value meets a predetermined criterion and non-applicable output data whose loss value does not satisfy the predetermined criterion.
  • a method of determining whether or not the loss value satisfies a predetermined criterion for example, there is a method of comparing a predetermined threshold value with the learning loss.
  • the learning loss exceeds a predetermined threshold, it is regarded as relevant output data, and when it is equal to or less than the threshold, it is regarded as non-corresponding output data.
  • a method of determining whether or not the loss values meet the predetermined criteria for example, there is a method of selecting output data within a predetermined number in descending order of loss values among the output data.
  • a method of determining whether or not the loss values meet the predetermined criteria for example, there is a method of selecting output data within a predetermined number in descending order of loss values as non-corresponding output data.
  • the correct information processing unit 4 processes the correct information in comparison with the corresponding output data whose loss value meets the predetermined criteria.
  • the correct information processing unit 4 is provided with correct information corresponding to each learning loss from the data memory 1, and the learning loss exceeding a predetermined threshold, that is, the classification output and the correct information. For learning losses with relatively large errors, correct information is processed into unknown.
  • the correct information processing unit 4 outputs the processed correct information (processed correct information) to the learning loss recalculation unit 5 .
  • a learning loss recalculator 5 obtains an error between the classification output and the processed correct information for each classification output output from the neural network 2 as a learning loss (hereinafter also referred to as a post-processing loss value).
  • the learning loss is supplied to the NN control circuit 10 .
  • the output data that is the result of classifying the training data is output and compared with the processed correct information. to obtain a post-processing loss value.
  • the update unit 14 of the NN control circuit 10 updates the parameters of the neural network 2 using the learning loss obtained by the learning loss recalculation unit 5.
  • the updating unit 14 may update parameters according to an existing SGD (stochastic gradient descent) algorithm.
  • the update formula for this SGD is known, and each parameter of the neural network 2 is calculated by substituting the value of the learning loss into the SGD update formula.
  • the neural network 2 is controlled by the NN control circuit 10 and classifies the input image according to the updated parameters. Thereafter, similar operations are repeated to perform learning.
  • FIG. 3 and 4 are flow charts for explaining the operation of the first embodiment.
  • 5 to 8 are explanatory diagrams for explaining the operation of the first embodiment.
  • FIG. 9 is an explanatory diagram for explaining the effect of the first embodiment in the same example as in FIG.
  • the initialization unit 12 of the NN control circuit 10 initializes the parameters of the neural network 2.
  • the initialization unit 12 is not an essential component, and parameter initialization is not an essential step.
  • the NN is initialized after starting, but the present invention is not limited to this. For example, it is also possible to apply the present invention to NNs raised by other learning methods without initializing them.
  • the input control unit 11 of the NN control circuit 10 inputs an image, which is training data stored in the data memory 1, to the neural network 2 (S2).
  • the input control unit 11 also inputs the correct answer information stored in the data memory 1 to the learning loss calculating unit 3 and the correct answer information processing unit 4 (S3).
  • mini-batches a predetermined number of images (hereinafter referred to as mini-batches) are extracted from a large number of images, and learning is performed on the extracted mini-batch images.
  • This mini-batch learning is performed for the number of data items, and one unit (hereinafter referred to as an epoch) of learning is performed.
  • the number of epochs performed in training may be predetermined.
  • FIG. 5 shows a mini-batch consisting of four images P1 to P4, which are training data.
  • the image P1 in this mini-batch contains the pancreatic cancer image portion P1a in the image P1.
  • the image P2 and the image P3 respectively include pancreatitis image portions P2b and P3b.
  • the image P4 is an inappropriate image including a blurred image portion P4 ⁇ of pancreatic cancer or pancreatitis. If there is no need to distinguish between the images P1 to P4, the image P may be used as a representative.
  • Correct information indicating that the image portion P1a is an image portion of pancreatic cancer is added to the image P1.
  • the image P2 is provided with correct information indicating that the image portion P2b is the image portion of pancreatitis
  • the image P3 is provided with correct information indicating that the image portion P3b is the image portion of pancreatitis.
  • Correct information indicating that the image portion P4 ⁇ is an image portion of pancreatic cancer or pancreatitis is added to the image P4.
  • FIG. 5 shows correct information AP1 to AP4 set for images P1 to P4, respectively. If there is no need to distinguish between the correct information AP1 to AP4, the correct information may be referred to as the representative correct information AP.
  • the correct information AP divides the image P into 5 ⁇ 4 regions and indicates the probability that each region corresponds to pancreatic cancer, pancreatitis, or unknown.
  • the probability of pancreatic cancer is 1 (bold frame) in the area corresponding to the image portion P1a of the image P1 and 0 in other areas. Further, in the correct information AP1, both the score of pancreatitis and the probability of being unknown are 0 for the entire region. Further, in the correct answer information AP2, the probability of pancreatitis is 1 (thick frame portion) in the area corresponding to the image portion P2b of the image P2, and 0 in other areas. Further, in the correct information AP2, both the probability of pancreatic cancer and the probability of being unknown are 0 for the entire region.
  • the probability of pancreatitis is 1 (thick frame portion) in the area corresponding to the image portion P3b of the image P3, and 0 in other areas. Further, in the correct information AP3, both the probability of pancreatic cancer and the probability of being unknown are 0 for the entire region. Further, in the correct information AP4, the probability of pancreatic cancer is 1 (thick frame portion) in the region corresponding to the image portion P4 ⁇ of the image P4 and 0 in other regions. Further, in the correct information AP4, both the probability of pancreatitis and the probability of being unknown are 0 for the entire region.
  • the NN application unit 13 applies such mini-batch to the neural network 2 (S4).
  • the neural network 2 outputs the classification output shown in the upper center of FIG.
  • the example of FIG. 5 shows a pancreatic cancer score (pancreatic cancer score), a pancreatitis score (pancreatitis score), and an unknown score (unknown score) for each 5 ⁇ 4 region of the image P.
  • FIG. Outputs C1-C4 in FIG. 5 show the classification outputs of neural network 2 for images P1-P4, respectively.
  • the image P1 indicates that the pancreatic cancer score for the region of the image portion P1a is 0.9 (bold frame), which is the highest. Note that the scores for other regions of the image P1 are relatively small, and the value of 0.9 is a relatively outstanding value. Further, as shown in the output C2, the image P2 has the highest pancreatitis score of 0.8 (bold frame) in the area of the image portion P2b. Also, the score for other regions of the image P2 is relatively small, and the value of 0.8 is a relatively outstanding value. Also, as shown in the output C3, the image P3 has the highest pancreatitis score of 0.8 (bold frame) in the area of the image portion P3b. Also, the scores for other regions of the image P3 are relatively small, and the value of 0.8 is a relatively outstanding value.
  • the pancreatic cancer score is 0.1 (bold frame), the pancreatitis score is 0.3 (bold frame), and the unknown The score is 0.3 (bold frame). That is, since the image P4 is an inappropriate image with blurring, it is difficult for the neural network 2 to classify the image as pancreatic cancer indicated in the correct information.
  • the classification output of the neural network 2 is given to the learning loss calculator 3 to calculate the learning loss (S5).
  • the right end of FIG. 5 shows the learning loss values for each of the images P1 to P4. As shown in FIG. 5, for images P1 to P3, the score of pancreatic cancer or pancreatitis is relatively high, and the learning loss is relatively small at 0.1 or 0.2. On the other hand, the image portion P4 ⁇ of the image P4 has a relatively low score for pancreatic cancer and a relatively large learning loss (0.9 ).
  • the learning loss calculator 3 outputs the calculated learning loss to the correct information processing unit 4 .
  • the correct answer information processing unit 4 determines whether or not the learning loss exceeds the threshold in S6. For example, assuming that the threshold is 0.8, the learning loss for image P4 exceeds the threshold in the example of FIG.
  • the correct information processing unit 4 processes the correct information about the image P into unknown (S7).
  • FIG. 6 shows this processing.
  • the correct information AP4 for the image P4 the probability of the image portion P4 ⁇ was 1 (bold frame) for the pancreatic cancer correct answer before the change.
  • the correct answer for pancreatic cancer is 0 (bold frame), and unknown is 1 (bold frame). If the learning loss obtained by the learning loss calculator 3 does not exceed the predetermined threshold value (NO determination in S6), the process proceeds to S9.
  • the correct information processing unit 4 outputs the processed correct information after processing to the learning loss recalculation unit 5 .
  • the learning loss recalculator 5 is also supplied with the classification output from the neural network 2, and the learning loss recalculator 5 recalculates the learning loss of the classification output from the neural network 2 using the processed correct information. (S8).
  • Fig. 7 shows the learning loss obtained by this learning loss recalculation.
  • the learning loss of the image portion P4 ⁇ of the image P4 changes to a relatively small value (0.7) because the correct answer information is changed to unknown.
  • the learning loss recalculator 5 outputs the calculated learning loss to the neural network 2 .
  • the updating unit 14 of the neural network 2 updates the parameters of the neural network 2 based on the input learning loss by, for example, the SGD method (S9).
  • the NN application unit 13 determines whether or not the learning termination condition is satisfied (S10). As described above, the process of extracting mini-batch training data and performing learning is repeated by the number of data items, and learning is performed until the specified number of epochs is reached. The NN application unit 13 determines whether or not the prescribed number of epochs has been reached, and if not (NO judgment in S10), returns the process to S2 and repeats S2 to S10. Moreover, the NN application unit 13 terminates the process when the prescribed number of epochs has been reached (YES determination in S10).
  • FIG. 4 shows the flow during this test.
  • a test image is input in S11 of FIG.
  • a test image is an unknown image.
  • the NN application unit 13 applies the test image stored in the data memory 1 to the neural network 2 (S12).
  • a classification output which is a recognition result, is obtained from the neural network 2 (S13). If the test of FIG. 4 is performed and a valid classification output is obtained, the test is successful. Conversely, if the classification output does not yield a valid output, the test fails. In this case, for example, the teacher data is changed and learning is performed again.
  • FIG. 9 shows an example of the classification output obtained when inference is performed using the training data P21 to P23 similar to FIG. 2 when the test in FIG. 4 is successful.
  • an unknown classification output is obtained for P22.
  • the learning loss is calculated, and for teacher data whose learning loss is higher than a predetermined threshold value, the correct information is changed to unknown, thereby indicating that the information is inappropriate for the teacher data.
  • the inference accuracy of the inference model can be improved. Therefore, there is no need to remove inappropriate images when creating training data, and annotation work can be made more efficient without lowering the inference accuracy of the neural network.
  • FIG. 10 is a block diagram showing a second embodiment of the invention.
  • the same components as those in FIG. 1 are given the same reference numerals, and the description thereof is omitted.
  • the inference accuracy of the neural network is improved by processing the correct information corresponding to the images whose learning loss meets a predetermined criterion and performing learning so that inappropriate images are classified as unknown.
  • the inference accuracy of the neural network is improved by processing the images so that the images whose learning loss meets the predetermined criteria are surely classified as inappropriate images. be.
  • a threshold value is used as a predetermined criterion will be exemplified below, but the present embodiment is not limited to this.
  • the second embodiment differs from the neural network update device of FIG. 1 in that the learning loss recalculation unit 5 is omitted and an image processing unit 9 is used in place of the correct information processing unit 4 .
  • the image processor 9 as a teacher data corrector compares the learning loss (loss value) from the learning loss calculator 3 with a predetermined threshold to determine whether the learning loss exceeds the predetermined threshold. That is, the image processing unit 9 divides output data whose loss value meets a predetermined criterion, i.e., output data whose learning loss exceeds a predetermined threshold, and non-corresponding output data (learning loss below a predetermined threshold). Note that the image processing unit 9 may select, as the corresponding output data, those within a predetermined number in descending order of the loss value among the output data.
  • the correct information processing unit 4 processes the training data in comparison with the corresponding output data whose learning loss exceeds a predetermined threshold, that is, whose loss value meets a predetermined criterion. Specifically, the image processing unit 9 receives an image corresponding to each learning loss from the data memory 1, and the learning loss exceeding a predetermined threshold, that is, the error between the classification output and the correct information is relatively large. For a large learning loss, the image corresponding to the learning loss is processed into an image that is likely to be classified as unknown. For example, the image processing unit 9 may blur an image corresponding to a learning loss exceeding a predetermined threshold.
  • the image processing unit 9 may perform image processing such as reducing the brightness of the image, reducing the resolution of the image, or reducing the size of the lesion in the image.
  • Processed image information (processing training data) obtained by image processing by the image processing section 9 is given to the data memory 1 and stored in place of the original image.
  • FIG. 11 is a flow chart for explaining the operation of the second embodiment.
  • the same steps as in FIG. 3 are denoted by the same reference numerals, and descriptions thereof are omitted.
  • 12 to 15 are explanatory diagrams for explaining the operation of the second embodiment.
  • FIG. 12 shows the learning process in a notation similar to that of FIG. 5.
  • the left end of FIG. 12 shows a mini-batch consisting of four training data images P1, P2, P3, and P4a.
  • the images P1 to P3 are the same images as the images P1 to P3 in FIG.
  • the image P4a is an inappropriate image including a blurred image portion P4 ⁇ of pancreatitis. Note that the images P1 to P4a may be referred to as the image P as a representative when there is no need to distinguish between the images P1 to P4a.
  • correct information AP1 to AP4 of the images P1, P2, P3, and P4a is shown in the lower part of FIG. 12 in the same notation as in FIG.
  • the correct information AP1-AP3 for the images P1-P3 are the same as in FIG.
  • the area corresponding to the blurred image portion P4 ⁇ is set to unknown in advance as shown in the thick frame. ing.
  • the NN application unit 13 applies such mini-batch to the neural network 2 (S4).
  • the neural network 2 outputs the classification output shown in the upper center of FIG.
  • Outputs C1-C4 in FIG. 12 show the classification outputs of neural network 2 for images P1, P2, P3, and P4a, respectively.
  • Outputs C1-C3 in FIG. 12 are scores similar to those in FIG.
  • the output C4 corresponding to the image P4a has a pancreatic cancer score of 0.2 (bold frame), a pancreatitis score of 0.6 (bold frame), and an unknown score for the region of the image portion P4 ⁇ of the image P4a. 0.2 (bold frame).
  • the neural network 2 has a relatively high classification probability of pancreatitis.
  • the classification output of the neural network 2 is given to the learning loss calculator 3 to calculate the learning loss (S5).
  • the right end of FIG. 12 shows the learning loss values for each of the images P1, P2, P3, and P4a. As shown in FIG. 12, for images P1 to P3, the probability of pancreatic cancer or pancreatitis is relatively high, and the learning loss is relatively small at 0.1 or 0.2. In contrast, the learning loss for image P4a is relatively large (0.8).
  • the input image is processed so that the inappropriate image is surely classified as unknown.
  • the learning loss from the learning loss calculator 3 and the image from the data memory 1 are given to the image processing unit 9 .
  • the learning loss calculator 3 determines whether or not the learning loss exceeds the threshold in S6. For example, assuming that the threshold is 0.7, in the example of FIG. 12, the learning loss for image P4a exceeds the threshold.
  • the image processing unit 9 determines that the learning loss exceeds the threshold value (YES in S6 of FIG. 11)
  • the image processing unit 9 processes the image P4a so that it is reliably determined as unknown (S27).
  • the image processing unit 9 can perform image processing that is determined to be unknown by various known image processing.
  • the image processing unit 9 may generate a more reliable blurred image using an averaging filter that averages each area of the image.
  • the image processing unit 9 may perform processing to reduce the resolution and brightness of the image P4a.
  • the image processing unit 9 stores the information of the processed image after the image processing in the data memory 1 instead of the original image (S28).
  • Fig. 13 shows the mini-batch obtained by this image processing.
  • hatching indicates that the image P4a has been changed to a blurred image P4ab, and the pancreatitis region has become a more blurred image portion P4 ⁇ b.
  • FIG. 13 shows an example in which image processing is performed on the entire image P4a, but image processing may be performed only on the image portion P4 ⁇ .
  • the updating unit 14 of the neural network 2 updates the parameters of the neural network 2 based on the input learning loss by, for example, the SGD method (S9). If the termination condition is not satisfied (NO determination in S10), neural network 2 is applied using updated parameters and modified mini-batches.
  • FIG. 14 shows the classification output (processed output data) obtained in this case using the same notation method as in FIG.
  • the output C4 indicated by the thick frame has changed from the previous classification output, and the unknown score of the image P4ab is the largest value (0.8).
  • the learning loss (post-processing loss value) of the classification output for the image P4ab becomes a sufficiently small value (0.2).
  • the updating unit 14 updates the parameters of the neural network 2 based on the sufficiently small learning loss obtained in this way, so that the neural network 2 finally obtained as a result of learning is High inference accuracy is obtained.
  • the present invention may allow the neural network 2 to identify the type of organ to be observed, may identify the degree of progression of a lesion, or may identify the degree of invasion of a lesion. It may be used to identify the presence or absence of past treatment, to estimate the blood vessel area, or to estimate the size of the lesion. Examples of the above-mentioned past treatments include, for example, removal of Helicobacter pylori.
  • image information identified as pancreatic cancer and image information identified as pancreatitis are lost for each.
  • a determination may be made as to whether the value meets predetermined criteria.
  • image information judged to be pancreatic cancer and image information judged to be pancreatitis for each A determination may be made as to whether the loss value meets predetermined criteria.
  • Modification 2 may be combined with Modification 1 for appraisal targets.
  • five pieces of image information in descending order of the loss value for each of the image information identified as the pharynx, the image information identified as the esophagus, and the image information identified as the stomach as non-corresponding output data. It is also possible to select and use the rest as the relevant output data. By doing so, there is an advantage that the amount of image information in each category of the pharynx, esophagus, and stomach can be made uniform, and deterioration in classification performance can be suppressed.
  • the learning loss is calculated, and for the teacher data whose learning loss is higher than a predetermined threshold, the image corresponding to the learning loss is processed so that the teacher data It is possible to improve the inference accuracy of the inference model even when an inappropriate image is included in the inference model. Therefore, there is no need to remove inappropriate images when creating training data, and annotation work can be made more efficient without lowering the inference accuracy of the neural network.
  • the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the gist of the present invention at the implementation stage.
  • various inventions can be formed by appropriate combinations of the plurality of constituent elements disclosed in the above embodiments. For example, some components of all components shown in the embodiments may be omitted. Furthermore, components across different embodiments may be combined as appropriate.
  • the program can be recorded or stored in whole or in part as a computer program product on portable media such as flexible disks, CD-ROMs, non-volatile memories, etc., or storage media such as hard disks and volatile memories. It can be distributed or provided at the time of product shipment or via a portable medium or communication line.
  • a user can easily realize the neural network update device of the present embodiment by downloading the program via a communication network and installing it on the computer, or by installing it on the computer from a recording medium. can.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
PCT/JP2022/006424 2022-02-17 2022-02-17 ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法 Ceased WO2023157187A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2022/006424 WO2023157187A1 (ja) 2022-02-17 2022-02-17 ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法
JP2024500822A JP7559284B2 (ja) 2022-02-17 2022-02-17 ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法
US18/659,852 US20240289615A1 (en) 2022-02-17 2024-05-09 Neural network update device, non-transitory recording medium recording neural network update program, and neural network update method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/006424 WO2023157187A1 (ja) 2022-02-17 2022-02-17 ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/659,852 Continuation US20240289615A1 (en) 2022-02-17 2024-05-09 Neural network update device, non-transitory recording medium recording neural network update program, and neural network update method

Publications (1)

Publication Number Publication Date
WO2023157187A1 true WO2023157187A1 (ja) 2023-08-24

Family

ID=87577937

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/006424 Ceased WO2023157187A1 (ja) 2022-02-17 2022-02-17 ニューラルネットワーク更新装置、ニューラルネットワーク更新プログラム及びニューラルネットワーク更新方法

Country Status (3)

Country Link
US (1) US20240289615A1 (https=)
JP (1) JP7559284B2 (https=)
WO (1) WO2023157187A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20250039063A1 (en) * 2023-07-27 2025-01-30 Mediatek Inc. Task-Aware Information Hiding

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020052475A (ja) * 2018-09-21 2020-04-02 株式会社Ｓｃｒｅｅｎホールディングス 分類器構築方法、画像分類方法、分類器構築装置および画像分類装置

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020052475A (ja) * 2018-09-21 2020-04-02 株式会社Ｓｃｒｅｅｎホールディングス 分類器構築方法、画像分類方法、分類器構築装置および画像分類装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Binary Classification with a third 'uncertain' class label", STACK EXCHANGE INC, 11 November 2021 (2021-11-11), XP093085298, Retrieved from the Internet <URL:https://stats.stackexchange.com/questions/550633/binary-classification-with-a-third-uncertain-class-label> [retrieved on 20230925] *

Also Published As

Publication number Publication date
US20240289615A1 (en) 2024-08-29
JP7559284B2 (ja) 2024-10-01
JPWO2023157187A1 (https=) 2023-08-24

Similar Documents

Publication Publication Date Title
JP7603995B2 (ja) ビデオ内視鏡検査における品質評価
Allan et al. 2017 robotic instrument segmentation challenge
JP7104810B2 (ja) 画像処理システム、学習済みモデル及び画像処理方法
CN113989301B (zh) 一种融合多种注意力机制神经网络的结直肠息肉分割方法
EP4187484B1 (en) Cbd-net-based medical endoscopic image denoising method
WO2021106174A1 (ja) 画像処理方法、学習装置及び画像処理装置
US9721191B2 (en) Method and system for image recognition of an instrument
Marhamati et al. LAIU-Net: A learning-to-augment incorporated robust U-Net for depressed humans’ tongue segmentation
US11790537B2 (en) Tracking device, endoscope system, and tracking method
CN116894783B (zh) 基于时变约束的对抗生成网络模型的金属伪影去除方法
CN112862835A (zh) 冠脉血管分割方法、装置、设备及计算机可读存储介质
WO2021176605A1 (ja) 学習データ作成システム及び学習データ作成方法
CN117522734A (zh) 用于对医学图像去模糊和去噪的系统和方法
US20240289615A1 (en) Neural network update device, non-transitory recording medium recording neural network update program, and neural network update method
CN116228781A (zh) 冠脉血管分割方法、装置、电子设备及存储介质
US12530765B2 (en) Systems and methods for calcium-free computed tomography angiography
CN115223193B (zh) 一种基于病灶特征重要性的胶囊内窥镜图像病灶识别方法
Bajčeta et al. Retinal blood vessels segmentation using ant colony optimization
CN114943650B (zh) 一种图像去模糊方法、装置、计算机设备和存储介质
CN116245847B (zh) 用于增强医学图像的方法和计算机程序产品
Hiraman et al. Efficient region of interest detection for liver segmentation using 3D CT scans
CN116681715A (zh) 基于像素值变化的血管分割方法、装置、设备和存储介质
Yang et al. Laparoscopic Image Desmoking Using the U-Net with New Loss Function and Integrated Differentiable Wiener Filter
EP3654251B1 (en) DETERMINING A PROCESSING SEQUENCE TO PROCESS AN IMAGE
US20250391165A1 (en) Image processing apparatus, image processing method, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22927089

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024500822

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22927089

Country of ref document: EP

Kind code of ref document: A1