WO2022209500A1 - 機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム - Google Patents

機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム Download PDF

Info

Publication number
WO2022209500A1
WO2022209500A1 PCT/JP2022/008070 JP2022008070W WO2022209500A1 WO 2022209500 A1 WO2022209500 A1 WO 2022209500A1 JP 2022008070 W JP2022008070 W JP 2022008070W WO 2022209500 A1 WO2022209500 A1 WO 2022209500A1
Authority
WO
WIPO (PCT)
Prior art keywords
annotation information
learning model
annotators
machine learning
medical image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/008070
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
篤志 橘
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2023510680A priority Critical patent/JP7776495B2/ja
Publication of WO2022209500A1 publication Critical patent/WO2022209500A1/ja
Priority to US18/472,268 priority patent/US20240013894A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G06V2201/031—Recognition of patterns in medical or anatomical images of internal organs

Definitions

  • the technology of the present disclosure relates to a machine learning model creation support device, a method of operating the machine learning model creation support device, and an operation program for the machine learning model creation support device.
  • machine learning models have been developed for recognizing subjects in medical images, such as recognizing tumors in abdominal tomographic images taken by a CT (Computed Tomography) device on a pixel-by-pixel basis.
  • Such machine learning models require annotation information as correct data in the learning phase or accuracy evaluation phase.
  • Annotation information is information generated by assigning a label according to the class of the object to be recognized to the original image paired in the correct data.
  • the annotation information is information generated by assigning a label “tumor” to the pixels of the tumor in the abdominal tomographic image, which is the original image.
  • One embodiment of the technology of the present disclosure is a machine learning model creation support device that can easily obtain appropriate annotation information to be used as correct data for a machine learning model, an operation method of the machine learning model creation support device, a machine An operating program for a learning model creation support device is provided.
  • a machine learning model creation support device of the present disclosure includes a processor, and the processor includes a plurality of annotation information generated by a plurality of annotators assigning a plurality of labels corresponding to a plurality of classes to a region of the same medical image. , derives for each class commonality data indicating commonality in how labels are assigned by multiple annotators for a plurality of pieces of annotation information, and based on the commonality data and preset definite conditions, Generate fixed annotation information to be used as correct data for the learning model.
  • the operation method of the machine learning model creation support device of the present disclosure acquires a plurality of annotation information generated by a plurality of annotators assigning a plurality of labels corresponding to a plurality of classes to a region of the same medical image. , for a plurality of pieces of annotation information, deriving for each class commonality data indicating commonality in how labels are assigned by a plurality of annotators; generating definitive annotation information to be used as correct data for the learning model.
  • An operating program of a machine learning model creation support device of the present disclosure obtains a plurality of annotation information generated by a plurality of annotators assigning a plurality of labels corresponding to a plurality of classes to a region of the same medical image. , for a plurality of pieces of annotation information, deriving for each class commonality data indicating commonality in how labels are assigned by a plurality of annotators; Generating fixed annotation information to be used as correct data for the learning model is executed by a computer.
  • FIG. 3 is a diagram showing medical images and annotation information transmitted and received between a machine learning model creation support server and an annotator terminal; It is a figure which shows annotation information.
  • FIG. 3 is a diagram which shows the computer which comprises a machine-learning model preparation assistance server.
  • FIG. 10 is a diagram showing annotation information in which labels of different classes are assigned to the same region;
  • FIG. 10 is a diagram showing annotation information in which labels of different classes are assigned to the same region;
  • FIG. 10 is a diagram illustrating another example of determination conditions and processing of the derivation unit and the generation unit;
  • FIG. 10 is a diagram showing another example of commonality data, determination conditions, and processing of the derivation unit and the generation unit;
  • FIG. 11 is a diagram showing a fourth embodiment of attaching display condition data to a medical image;
  • FIG. 10 is a diagram showing an annotation information generation screen displayed on the display of the annotator terminal;
  • FIG. 11 is a diagram showing an annotation information generation screen when a generation end button is operated;
  • FIG. 12 is a diagram showing a fifth embodiment in which a human body area in which a human body is captured in a medical image is detected and commonality data is derived only for the detected human body area.
  • FIG. 10 is a diagram illustrating another example of determination conditions and processing of the derivation unit and the generation unit
  • FIG. 10 is a diagram showing another example of commonality data, determination conditions, and processing of the derivation unit and the generation unit
  • FIG. 11 is
  • FIG. 10 is a diagram showing an annotation information generation screen in which a hatched human body region is displayed; It is a figure which shows annotator information.
  • FIG. 22 is a diagram showing a seventh embodiment in which different decision conditions are set for the central portion and peripheral portion of the class area;
  • FIG. 20 is a diagram showing an eighth embodiment in which, in the finalized annotation information, the reliability indication value of the label of the peripheral portion of the class area is set lower than that of the central portion of the class area;
  • FIG. 10 is a diagram showing how to assign a label when the class is blood vessels;
  • FIG. 22 illustrates a ninth embodiment of transmitting annotation information and final annotation information to an annotator terminal;
  • FIG. 10 is a diagram showing an information comparison screen displayed on the display of the annotator terminal;
  • the machine learning model creation support system 2 includes a machine learning model creation support server (hereinafter abbreviated as support server) 10 and annotator terminals 11A, 11B, and 11C.
  • the support server 10 and the annotator terminals 11A to 11C are connected via a network 12 so as to be mutually communicable.
  • the network 12 is, for example, the Internet or a WAN (Wide Area Network).
  • the support server 10 is, for example, a server computer, a workstation, or the like, and is an example of a "machine learning model creation support device" according to the technology of the present disclosure.
  • Annotator terminal 11A has display 13A and input device 14A
  • annotator terminal 11B has display 13B and input device 14B
  • annotator terminal 11C has display 13C and input device 14C.
  • the annotator terminal 11A is operated by the annotator 15A
  • the annotator terminal 11B is operated by the annotator 15B
  • the annotator terminal 11C is operated by the annotator 15C.
  • the annotator terminals 11A to 11C are, for example, personal computers, tablet terminals, and the like.
  • the annotators 15A to 15C are doctors, for example, and are requested by the support server 10 to generate annotation information 21 (see FIG. 2).
  • the annotator terminals 11A to 11C are collectively referred to as the annotator terminal 11 when there is no particular need to distinguish them.
  • displays 13A-13C, input devices 14A-14C, and annotators 15A-15C may also be collectively referred to as display 13, input device 14, and annotator 15.
  • FIG. Note that the input device 14 is, for example, at least one of a keyboard, mouse, touch panel, microphone, gesture recognition device, and the like.
  • the support server 10 transmits the same medical image 20 to the annotator terminals 11A-11C.
  • the medical image 20 an abdominal tomographic image of an axial section taken by a CT apparatus is exemplified.
  • the medical image 20 is an original image for assigning a label according to a class that is a subject to be recognized based on a preset task.
  • the same medical image 20 refers to the medical image 20 of the same medical imaging device (also called modality) such as a CT device, the same patient, and the same imaging date and time.
  • the annotator terminal 11 displays the medical image 20 on the display 13.
  • the annotator terminal 11 receives an input for labeling the medical image 20 in units of pixels from the annotator 15 via the input device 14 .
  • annotation information 21A is generated by the annotator 15A at the annotator terminal 11A
  • annotation information 21B is generated by the annotator 15B at the annotator terminal 11B
  • annotation information 21C is generated by the annotator 15C at the annotator terminal 11C.
  • the annotation information 21A to 21C may be collectively referred to as the annotation information 21 in some cases.
  • the annotation information 21 is generated for each tomographic plane of the abdominal tomographic image.
  • the human body structure is drawn in the annotation information 21 to aid understanding, but the actual annotation information 21 does not include data of the human body structure, but only data of the attached labels (hereinafter referred to as 3, etc.).
  • the annotation information 21 is information in which pairs of label types and position coordinates of pixels of the labeled medical image 20 are registered.
  • the annotator terminal 11 transmits the annotation information 21 to the support server 10.
  • the support server 10 receives annotation information 21 from the annotator terminal 11 .
  • the annotation information 21 includes a first region 25 labeled as liver, a second region 26 labeled as tumor, and a second region 26 labeled as bleeding. and a third region 27 .
  • the second region 26 is naturally not specified when the annotator 15 determines that no tumor exists.
  • the third region 27 is not designated when the annotator 15 determines that there is no bleeding site.
  • the computer that configures the support server 10 includes a storage 30, a memory 31, a processor 32, a communication section 33, a display 34, and an input device 35. These are interconnected via bus lines 36 .
  • the storage 30 is a hard disk drive built into the computer constituting the support server 10 or connected via a cable or network. Alternatively, the storage 30 is a disk array in which a plurality of hard disk drives are connected.
  • the storage 30 stores a control program such as an operating system, various application programs (hereinafter abbreviated as AP (Application Program)), various data associated with these programs, and the like.
  • AP Application Program
  • a solid state drive may be used instead of the hard disk drive.
  • the memory 31 is a work memory for the processor 32 to execute processing.
  • the memory 31 is, for example, a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).
  • the processor 32 loads the program stored in the storage 30 into the memory 31 and executes processing according to the program. Thereby, the processor 32 comprehensively controls each part of the computer.
  • the processor 32 is, for example, a CPU (Central Processing Unit).
  • the memory 31 is an example of a “memory” according to the technology of the present disclosure. Note that the storage 30, or the storage 30 and the memory 31 may be defined as an example of the "memory” according to the technology of the present disclosure.
  • the communication unit 33 is a network interface that controls transmission of various information via the network 12 and the like.
  • the display 34 displays various screens. Various screens are provided with operation functions by GUI (Graphical User Interface).
  • a computer that configures the support server 10 receives input of operation instructions from the input device 35 through various screens.
  • the input device 35 is at least one of a keyboard, mouse, touch panel, microphone, gesture recognition device, and the like.
  • the storage 30 stores an operating program 40 .
  • the operation program 40 is an AP for causing a computer that configures the support server 10 to function as a "machine learning model creation support device" according to the technology of the present disclosure. That is, the operating program 40 is an example of the "operating program of the machine learning model creation support device" according to the technology of the present disclosure.
  • the storage 30 also stores the medical image 20 , annotation information 21 , finalized conditions 41 , and finalized annotation information 42 . Although only one medical image 20 is drawn, a plurality of medical images 20 are actually stored in the storage 30 .
  • the annotation information 21 and the fixed annotation information 42 are also the same.
  • the processor 32 When the operating program 40 is started, the processor 32 cooperates with the memory 31 and the like to cooperate with the read/write (hereinafter abbreviated as RW (Read Write)) control section 50, image transmission section 51, information reception section 52, derivation It functions as a unit 53 and a generation unit 54 .
  • RW Read Write
  • the RW control unit 50 controls storage of various information in the storage 30 and reading of various information in the storage 30 .
  • the RW control unit 50 reads the medical image 20 from the storage 30 and outputs the read medical image 20 to the image transmission unit 51 .
  • the RW control unit 50 also reads out the definite condition 41 from the storage 30 and outputs the read definite condition 41 to the generation unit 54 .
  • the image transmission unit 51 transmits the medical image 20 from the RW control unit 50 to the pre-registered annotator terminal 11 .
  • the information receiving unit 52 receives the annotation information 21 from the annotator terminal 11. Thereby, the support server 10 acquires the annotation information 21 .
  • the information receiving section 52 outputs the received annotation information 21 to the RW control section 50 .
  • the RW control unit 50 stores the annotation information 21 in the storage 30.
  • FIG. 5 shows that the annotation information 21A to 21C are received simultaneously by the information receiving unit 52, the reception timings of the annotation information 21A to 21C are actually different.
  • the information receiving unit 52 outputs the annotation information 21 to the RW control unit 50 each time it receives the annotation information 21, and the RW control unit 50 outputs the annotation information 21 each time the annotation information 21 is input from the information receiving unit 52. Store in the storage 30 .
  • the RW control unit 50 reads the annotation information 21A-21C from the storage 30 and outputs the read annotation information 21A-21C to the derivation unit 53 and the generation unit 54.
  • the derivation unit 53 derives commonality data 60 .
  • the commonality data 60 is data indicating the commonality in how the three annotators 15A to 15C assign labels to the three pieces of annotation information 21A to 21C.
  • the derivation unit 53 outputs the derived commonality data 60 to the generation unit 54 .
  • the generating unit 54 generates fixed annotation information 42 based on the commonality data 60 from the deriving unit 53, the annotation information 21A to 21C from the RW control unit 50, and the fixed condition 41.
  • the finalized annotation information 42 is annotation information that is finally used as correct data for the machine learning model.
  • the generator 54 outputs the finalized annotation information 42 to the RW controller 50 .
  • the RW control unit 50 stores the confirmed annotation information 42 from the generation unit 54 in the storage 30 .
  • the definitive annotation information 42 is used as correct data together with the medical image 20, which is the original image, in the learning phase of the machine learning model or the accuracy evaluation phase.
  • a medical image 20 is input to the machine learning model.
  • the output annotation information output from the machine learning model is then compared with the definitive annotation information 42 to calculate the loss of the machine learning model.
  • the machine learning model is updated according to the loss.
  • the input of the medical image 20, the output of the output annotation information, the calculation of the loss, and the update are repeated while the set of the medical image 20 and the confirmed annotation information 42, that is, the correct data, is exchanged. This trains the machine learning model.
  • the medical image 20 is input to the machine learning model that has undergone a certain amount of learning. Then, the output annotation information output from the machine learning model and the definitive annotation information 42 are compared to calculate the loss, and the accuracy of the machine learning model is evaluated based on the loss. In the accuracy evaluation phase, only accuracy is evaluated and no update is performed. A machine learning model determined to have a preset accuracy or higher in the accuracy evaluation phase is used in the practical use phase.
  • the correct data used in the learning phase is also called learning data, and the correct data used in the accuracy evaluation phase is also called evaluation data.
  • the first region 25A is the region labeled liver in the annotation information 21A
  • the first region 25B is the region labeled liver in the annotation information 21B
  • the first region 25B is labeled liver.
  • a region 25C is a region labeled as liver in the annotation information 21C.
  • the second region 26A is a region labeled as tumor in the annotation information 21A
  • the second region 26B is a region labeled as tumor in the annotation information 21B
  • the second region 26C is the annotation information 21C. is the labeled region of the tumor in .
  • the third region 27A is a region labeled bleeding in the annotation information 21A
  • the third region 27B is a region labeled bleeding in the annotation information 21B
  • the third region 27C is the annotation information 21C. is the area labeled as hemorrhagic in .
  • the derivation unit 53 counts the number of annotators 15 to whom the liver, tumor, and bleeding labels have been assigned, and uses the counted number as the numerical value of the commonality data 60 .
  • the value of the commonality data 60 of pixels labeled by one annotator 15 is one, and the value of the commonality data 60 of pixels labeled by two annotators 15 is two.
  • the numerical value of the commonality data 60 of the pixels labeled by the three annotators 15 is three.
  • the derivation unit 53 derives commonality data 60 for each class of liver, tumor, and bleeding. Note that the number of annotators 15 to which labels have been assigned is an example of the “numerical value relating to the number of annotators to which labels have been assigned” according to the technology of the present disclosure.
  • the determination condition 41 of the present embodiment is to adopt the label of the region with the numerical value of 3 in the commonality data 60, that is, the region labeled by all the annotators 15.
  • the generation unit 54 generates a first region 25X labeled as liver by three annotators 15, a second region 26X labeled as tumor by three annotators 15, and a bleeding region 26X labeled as tumor by three annotators 15. generated the finalized annotation information 42 including the third region 27X labeled with .
  • the processor 32 of the support server 10 functions as the RW control unit 50, the image transmission unit 51, the information reception unit 52, the derivation unit 53, and the generation unit 54 as shown in FIG. be done.
  • the medical image 20 is read from the storage 30 by the RW control unit 50 .
  • the read medical image 20 is output from the RW control unit 50 to the image transmission unit 51 .
  • the medical image 20 is transmitted to the annotator terminal 11 by the image transmission unit 51 .
  • the annotation information 21 is generated based on the medical image 20 by the annotator 15 . As shown in FIG. 3, the annotation information 21 is generated by assigning three labels corresponding to three classes of liver, tumor, and bleeding to the same medical image 20 region. The annotation information 21 is transmitted from the annotator terminal 11 to the support server 10 .
  • the annotation information 21 from the annotator terminal 11 is received by the information receiving unit 52 .
  • the annotation information 21 is obtained (step ST100).
  • the annotation information 21 is output from the information receiving section 52 to the RW control section 50 and stored in the storage 30 by the RW control section 50 .
  • the annotation information 21 is read from the storage 30 by the RW control unit 50 .
  • the read annotation information 21 is output from the RW control unit 50 to the derivation unit 53 and the generation unit 54 .
  • the derivation unit 53 derives, for each class, commonality data 60 indicating the commonality of labeling methods by the plurality of annotators 15A to 15C for the plurality of annotation information 21A to 21C. (step ST110).
  • the commonality data 60 is a count value of the number of annotators 15 who have assigned the labels of liver, tumor, and bleeding.
  • the commonality data 60 is output from the derivation unit 53 to the generation unit 54 .
  • the generation unit 54 generates the defined annotation information 42 based on the commonality data 60 and the defined conditions 41 (step ST120).
  • final annotation information 42 is generated that includes a first region 25X, a second region 26X, and a third region 27X labeled liver, tumor, and bleeding by three annotators 15, respectively.
  • the finalized annotation information 42 is output from the generation unit 54 to the RW control unit 50 and stored in the storage 30 by the RW control unit 50 .
  • the processor 32 of the support server 10 includes the information receiving section 52, the deriving section 53, and the generating section .
  • the information receiving unit 52 acquires the annotation information 21A to 21C from the annotator terminals 11A to 11C by receiving them.
  • the annotation information 21A-21C is generated by the annotators 15A-15C assigning three labels corresponding to three classes to the same region of the medical image 20.
  • the derivation unit 53 derives, for each class, commonality data 60 indicating the commonality of labeling methods by the annotators 15A to 15C for the annotation information 21A to 21C.
  • the generation unit 54 generates definite annotation information 42 to be used as correct data for the machine learning model based on the commonality data 60 and the preset definite conditions 41 .
  • the method of labeling differs depending on the annotator 15, resulting in a difference between the plurality of annotation information 21.
  • the annotation information 21 is generated by assigning multiple labels corresponding to multiple classes, the more labels are assigned, the greater the difference between the multiple annotation information 21. Become.
  • the commonality data 60 is derived for each class, and the definitive annotation information 42 is generated based on the derived commonality data 60. Therefore, it is possible to easily obtain appropriate definitive annotation information 42 to be used as correct data for the machine learning model.
  • the annotation information 63 of this embodiment includes a first region 65 labeled liver, a second region 66 labeled liver and tumor, liver, tumor, and a third region 67 labeled bleeding.
  • a second region 66 and a third region 67 are regions labeled with different classes.
  • the annotation information 63 is information in which different class labels are assigned to the same region. Therefore, as the number of types of labels applied to the same area increases, the difference between the plurality of pieces of annotation information 21 becomes even greater than in the first embodiment. Therefore, the effect of easily obtaining the appropriate definitive annotation information 42 to be used as the correct data of the machine learning model can be further exhibited.
  • the determination condition 70 of the present embodiment adopts labels for areas where the numerical value of the commonality data 60 is 2 or more, that is, areas where the number of labeled annotators is 2 or more. This is the content. Therefore, the generating unit 54 generates the first region 25Y labeled as liver by two or more annotators 15, the second region labeled as tumor by two or more annotators 15 (not shown), and The above annotator 15 generates final annotation information 42 including a third region (not shown) labeled as bleeding. It should be noted that "2" to "two people” in the determination condition 70 are examples of the "threshold" according to the technology of the present disclosure.
  • the derivation unit 53 (not shown) of the present embodiment uses commonality data 75 in which the ratio of labeled annotators 15 is registered for each position coordinate of a labeled pixel. to derive
  • the derivation unit 53 derives commonality data 75 in which the ratio of annotators labeled with the liver shown in the drawing is registered.
  • the deriving unit 53 also includes commonality data 75 in which the ratio of the annotators 15 labeled with the tumor is registered and commonality data 75 in which the ratio of the annotators 15 labeled with the bleeding is registered (not shown).
  • the ratio of the annotators 15 to which the label is assigned is an example of the “numerical value related to the number of annotators to which the label is assigned” according to the technology of the present disclosure.
  • the confirmation condition 76 of the present embodiment is to adopt the label of the area where the percentage of the annotators 15 who have given the label is 90% or more.
  • the generation unit 54 (not shown) of the present embodiment determines the adoption or non-adoption of labels for each position coordinate as shown in Table 77 based on the commonality data 75 and the determination conditions 76 . Specifically, the generation unit 54 determines to adopt the label of the position coordinates for which the ratio of the annotators 15 assigned the label of the commonality data 75 is 90% or more of the determination condition 76 . On the other hand, the generation unit 54 determines not to adopt (not adopt) the label of the position coordinates for which the ratio of the annotators 15 assigned the label of the commonality data 75 is less than 90% of the determination condition 76 .
  • the generation unit 54 generates the finalized annotation information 42 based on the acceptance/rejection result. Note that "90%" of the determination condition 76 is an example of a "threshold" according to the technology of the present disclosure.
  • the final annotation information 42 is inevitably influenced by the annotation information 21 with the relatively small labeled area.
  • the 3_1 embodiment and the 3_2 embodiment using the determination conditions 70 and 76 that the given label is adopted only when the numerical value of the commonality data 60 or 75 is equal to or greater than the threshold, It is possible to generate definitive annotation information 42 labeled over a wider area, which is less affected by annotation information 21 with a relatively small labeled area.
  • the number of annotators 15 should be two or more, and is not limited to three. Therefore, the number of pieces of annotation information 21 may be two or more, and is not limited to three. Also, the case where the number of annotators 15 and the confirmation conditions 41, 70, and 76 are fixed has been exemplified, but the present invention is not limited to this.
  • the number of annotators 15 and the defined conditions 41, 70, and 76 may be variable. For example, the configuration may be such that the user who operates the support server 10 can change the settings of the number of annotators 15 and the confirmation conditions 41 , 70 , and 76 .
  • the image transmission unit 80 of this embodiment attaches the display condition data 81 to the medical image 20 and transmits it to the annotator terminals 11A to 11C.
  • the display condition data 81 includes a window level (WL), a window width (WW), and a slice position.
  • the window level and window width are parameters related to the display gradation of the medical image 20 .
  • the window level is the central value of the display area of the medical image 20 that is set with respect to the pixel values of the original image of the medical image 20 .
  • the window width is a numerical value indicating the width of the display area of the medical image 20 .
  • the slice position indicates the position of the tomographic plane when the medical image 20 is a tomographic image as in this example.
  • An annotation information generation screen 85 shown in FIG. 12 is displayed as an example on the display 13 of the annotator terminal 11 .
  • the annotation information generation screen 85 has a task display area 86, a tool button group 87, an image display area 88, and the like.
  • the content of the set task is displayed in the task display area 86 .
  • the tool button group 87 consists of tool buttons of various tools for the annotator 15 to designate a label corresponding to the class designated by the task.
  • Various tools are, for example, a designated class switching tool, a line drawing tool, an area filling tool, an area erasing tool, and the like.
  • a medical image 20 is displayed in the image display area 88 .
  • Annotation information 21 is generated by labeling the medical image 20 displayed in the image display area 88 using various tools.
  • the medical image 20 is initially displayed under display conditions according to the attached display condition data 81 .
  • FIG. 12 shows an example in which the medical image 20 is displayed under display conditions according to the display condition data 81 illustrated in FIG. 11 .
  • a send-back button 89 and a display gradation change button 90 are provided at the bottom of the image display area 88 .
  • the send-back button 89 it is possible to change the slice position.
  • the display gradation change button 90 it is possible to change the window level and the window width.
  • the annotator 15 can freely change the slice position of the medical image 20 . Therefore, the annotator 15 can review the medical image 20 at a specific slice position many times. Also, the annotator 15 can freely change the display conditions of the medical image 20 .
  • illustration is omitted, the medical image 20 in the image display area 88 can be translated and enlarged and reduced.
  • a temporary save button 91 and a generation end button 92 are further provided.
  • the temporary save button 91 When the temporary save button 91 is operated, the annotation information 21 generated so far is temporarily saved in the storage of the annotator terminal 11 .
  • the generation end button 92 When the generation end button 92 is operated, a dialog box 95 pops up as shown in FIG. 13 as an example.
  • the display condition of the medical image 20 in the image display area 88 is set to the display condition according to the attached display condition data 81 .
  • the dialog box 95 is a GUI for asking the annotator 15 whether or not to really finish generating the annotation information 21.
  • a dialog box 95 is provided with a Yes button 96 and a No button 97 .
  • the yes button 96 is operated, the generated annotation information 21 is transmitted to the support server 10 .
  • the No button 97 is selected, the dialog box 95 disappears and the annotation information 21 can be generated.
  • the display condition data 81 is attached to the medical image 20.
  • the display condition data 81 is data for displaying the medical image 20 under the same display condition when multiple annotators 15 view the medical image 20 . Therefore, as shown in FIGS. 12 and 13, the same display conditions can be set for a plurality of annotator terminals 11. FIG. Therefore, it is possible to suppress the occurrence of differences in labeling by the annotators 15 due to differences in display conditions.
  • each annotator 15 by first displaying the medical image 20 under the display conditions according to the display condition data 81 , the recognition of each annotator 15 may be deviated from the start of generation of the annotation information 21 . can be suppressed. Further, as shown in FIG. 13, by displaying the medical image 20 under the display conditions according to the display condition data 81 when completing the generation of the annotation information 21, each annotator 15 can It is possible to suppress the occurrence of deviation in the recognition of
  • the medical image 20 is displayed under display conditions according to the display condition data 81 both when the generation of the annotation information 21 is started and when the generation of the annotation information 21 is finished.
  • the display conditions of the medical image 20 are the same, but the present invention is not limited to this.
  • the medical image 20 may be displayed under display conditions according to the display condition data 81 only when the generation of the annotation information 21 is started or when the generation of the annotation information 21 is finished.
  • the processor of the support server of this embodiment functions as a detection unit 105 in addition to the processing units 50 to 54 (not shown except the derivation unit 53) of the first embodiment.
  • the detection unit 105 detects a human body region 106 in which a human body is captured in the medical image 20 using body surface recognition technology.
  • Detecting unit 105 outputs human body region information 107 that is the detection result of human body region 106 to derivation unit 53 .
  • the human body region information 107 is specifically the position coordinates of the pixels of the medical image 20 corresponding to the human body region 106 .
  • the derivation unit 53 of this embodiment derives the commonality data 60 only for the human body region 106 .
  • the class of the medical image 20 is set only for the human body region 106, so it is not necessary to derive the commonality data 60 for regions other than the human body region 106. Therefore, in the fifth embodiment, the detection unit 105 detects the human body region 106 in which the human body appears in the medical image 20, and the deriving unit 53 derives the commonality data 60 only for the detected human body region 106. . Therefore, the processing load on the derivation unit 53 can be reduced, and as a result, the generation of the finalized annotation information 42 can be accelerated.
  • the commonality data 75 of the 3_2 embodiment can be used instead of the commonality data 60 .
  • the human body region 106 detected by the detection unit 105 is displayed with a colored hatching 115 indicated by hatching. may be displayed so as to be identifiable from the area of In this way, the annotator 15 can be alerted not to erroneously label regions other than the human body region 106 .
  • annotation information generation screen 85 may be configured so that labels cannot be added to areas other than the human body area 106 . Such a configuration can also prevent erroneous labeling of regions other than the human body region 106 .
  • the deriving unit 53 of this embodiment counts the number of annotators 15 when deriving the commonality data 60 according to the annotator information 120 .
  • the annotator information 120 registers the attribute of the annotator 15 and the number of people counted when deriving the commonality data 60 for each annotator ID (Identification Data) for identifying each annotator 15 .
  • Attributes include years of service and qualifications. Qualifications include Radiology Training Instructor and Radiological Diagnostic Specialist.
  • the sixth embodiment when deriving the commonality data 60, weighting according to the attributes of the annotator 15 is performed. Therefore, annotators 15 with relatively long years of service and/or qualified annotators 15, or other annotators 15 whose labeling accuracy is considered to be relatively high, increase the count of the number of people in the area labeled. be able to. In this way, if the determination condition is to adopt the label of the region where the numerical value of the commonality data 60 is equal to or greater than the threshold, as in the determination condition 70 of the 3_1 embodiment, the accuracy of labeling can be improved. The probability that the label assigned by the annotator 15 that is considered to have a relatively high probability of being adopted as the label of the final annotation information 42 increases. As a result, the reliability of the finalized annotation information 42 can be enhanced.
  • the commonality data 75 of the above 3_2 embodiment can be used instead of the commonality data 60.
  • the percentage of regions labeled by the annotator 15 whose labeling accuracy is considered to be relatively high is relatively high.
  • the probability that the label will be adopted as the label of the finalized annotation information 42 is increased, and as a result, the reliability of the finalized annotation information 42 can be enhanced.
  • the side to which the annotators 15 whose labeling accuracy is considered to be relatively high belongs. may be adopted. Specifically, there are four annotators 15, and two annotators 15 have given labels and two have not given labels. If it is on the side that has not been attached, it is not adopted as the label of the fixed annotation information 42 .
  • the attribute may include the specialty field of the annotator 15. In this case, for example, if the task is related to the specialized field, the count number is added. Also, the number of published papers of the annotator 15 may be included in the attribute. In this case, the count number is increased when the number of published papers is greater than or equal to the first threshold, and the count number is decreased when the number of published papers is less than the second threshold.
  • the generation unit 54 (not shown) of the present embodiment generates finalized annotation information 42 based on two finalized conditions 125A and 125B.
  • a determination condition 125A is applied to the center 126A of the region 126 of the class.
  • determination condition 125B applies to edge 126B of region 126 .
  • the determination condition 125A is to adopt the label of the area where the percentage of the annotators 15 who have given the label is 70% or more.
  • the confirmation condition 125B is to adopt the label of the area where the percentage of the annotators 15 who have given the label is 90% or more.
  • the threshold of the confirmation condition 125B applied to the peripheral portion 126B is set higher than the threshold of the confirmation condition 125A applied to the central portion 126A.
  • the central portion 126A and the peripheral portion 126B are sorted, for example, as follows. First, the regions labeled by all the annotators 15 are determined, and the center of the determined region is set as the center of the central portion 126A. Note that the center is, for example, at least one of the center of gravity, the inner center, the outer center, and the orthocenter. The regions labeled by all annotators 15 are then enlarged, for example, by 20% without moving the center. The region labeled by all the annotators 15 is defined as a central portion 126A, and the region bordered by the region enlarged by 20% and the region labeled by all the annotators 15 is defined as a peripheral portion 126B.
  • each annotator 15 may obtain the center of each labeled region, further obtain the center of each of the obtained centers, and use this as the center of the central portion 126A.
  • region with which the label was given was made into the marginal part, it is not restricted to this. For example, when the distance from the center of the labeled region to the outer edge is 100, the distance from the center to 80 may be the center portion, and the remaining distance from 80 to 100 may be the edge portion.
  • the seventh embodiment there are defined conditions 125A applied to the central portion 126A of the class region 126, and defined conditions 125B applied to the marginal portion 126B. It is different from the edge portion 126B. Further, it is more difficult to satisfy the definite condition 125B than the definite condition 125A. The edge 126B is particularly prone to labeling errors because it is a boundary with other areas. Therefore, by increasing the degree of difficulty in satisfying the conditions for the peripheral portion 126B rather than for the central portion 126A, the reliability of the finalized annotation information 42 for the peripheral portion 126B can be ensured.
  • the generation unit 54 expresses the reliability of the label of the peripheral part 131B of the class area 131 in the final annotation information 130 rather than the central part 131A of the class area 131.
  • Set the numerical value (hereinafter referred to as the reliability display value) to a low value.
  • the generation unit 54 sets the reliability display value of the central portion 131A to 1, which is the maximum value, and sets the reliability display value of the peripheral portion 131B to 0.8.
  • the central portion 131A and the peripheral portion 131B are also selected in the same manner as the central portion 126A and the peripheral portion 126B in the seventh embodiment.
  • the generation unit 54 sets the reliability display value of the label of the peripheral portion 131B lower than that of the central portion 131A of the class region 131 in the final annotation information 130 . Therefore, when a machine learning model is learned using the definite annotation information 130, it is possible to reduce the occurrence frequency of so-called false positives, in which a region that is not a class is recognized as a class, especially in the early stage of the learning phase.
  • the label reliability display value in the finalized annotation information may be set according to the ratio of the annotators 15 to which labels have been assigned, as described in the 3_2 embodiment above.
  • the ratio of annotators 15 to which labels are assigned is used as the reliability indication value. Specifically, when the percentage is 80%, the reliability indication value is 0.8, and when the percentage is 20%, the reliability indication value is 0.2.
  • the generation unit 54 preferably assigns a label to an expanded region 136 larger than the region 135 that satisfies the definite condition in the definite annotation information.
  • the expanded area 136 is, for example, an area that is larger than the area 135 that satisfies the confirmation condition by the set number of pixels.
  • the set number of pixels is 1, for example.
  • the boundary is often unclear, so there is a demand for recognizing the periphery of the blood vessel as a blood vessel with a margin, and this demand can be met.
  • the extended area 136 may be an area that includes a distance of, for example, 120 from the outer edge when the distance from the center of the area 135 that satisfies the determination condition to the outer edge is 100.
  • the center of the region 135 that satisfies the confirmation condition can be obtained in the same manner as the center of the regions labeled by all the annotators 15 in the seventh embodiment.
  • the processor of the support server of this embodiment functions as an information transmission section 140 in addition to the processing sections 50 to 54 of the first embodiment.
  • the information transmission unit 140 transmits the annotation information 21 and the fixed annotation information 42 stored in the storage 30 to the annotator terminal 11 .
  • the annotation information 21 may include not only information generated by the annotator 15 of the annotator terminal 11 that transmits, but also information generated by other than the annotator 15 of the annotator terminal 11 that transmits.
  • an information comparison screen 145 shown in FIG. 21 is displayed on the display 13 as an example.
  • the information comparison screen 145 includes an annotation information display area 146 in which the annotation information 21 is superimposed on the medical image 20 and displayed, and a confirmed annotation information display area 147 in which the confirmed annotation information 42 is superimposed on the medical image 20 and displayed. have.
  • the annotation information display area 146 and the finalized annotation information display area 147 are arranged side by side.
  • annotation information display area 146 the annotation information 21 generated by the annotator 15 is initially displayed.
  • the medical image 20 superimposed on the annotation information 21 and the medical image 20 superimposed on the finalized annotation information 42 are initially displayed under the same display conditions according to the display condition data 81 .
  • FIG. 21 shows an example in which regions labeled with liver, tumor, and hemorrhage are all displayed. It is also possible to display by class such as only.
  • send-back buttons 148 and 149 having the same functions as the send-back button 89 and display gradation change button 90 of the annotation information generation screen 85, and display gradation.
  • Change buttons 150 and 151 are provided. Therefore, as in the case of the annotation information generation screen 85, it is possible to change the slice position, window level and window width.
  • a display mode switching button 152 When the display mode switching button 152 is operated, the annotation information 21 in the annotation information display area 146 is moved to the confirmed annotation information display area 147, and the annotation information 21 and the confirmed annotation information 42 are superimposed and displayed. In this case, the portions where the labels of the annotation information 21 and the final annotation information 42 overlap are displayed in a darker color than the portions where the labels do not overlap.
  • the correction button 153 When the correction button 153 is operated, the screen transitions to the annotation information generation screen 85, and the annotation information 21 can be corrected.
  • the display of the annotation information display area 146 switches from the annotation information 21 generated by the relevant annotator 15 to annotation information 21 generated by another annotator 15 .
  • the end button 155 is operated, the display of the information comparison screen 145 is erased.
  • the information transmitting unit 140 transmits the annotation information 21 and the finalized annotation information 42 to the annotator terminal 11 operated by the annotator 15 . Therefore, as shown in FIG. 21, the annotation information 21 and the finalized annotation information 42 can be displayed on the annotator 15 so as to be comparable.
  • the annotator 15 can compare the annotation information 21 generated by him/herself and the finalized annotation information 42, and can make use of it in generating the annotation information 21 in the future. In some cases, the annotation information 21 can be corrected while referring to the finalized annotation information 42 .
  • the display conditions for the medical image 20 superimposed on the annotation information 21 and the medical image 20 superimposed on the finalized annotation information 42 are set most among the display conditions set by each annotator 15 on the annotation information generation screen 85.
  • the display condition may be set such that the number of times the display is performed is large. Alternatively, the following may be done. That is, the support server 10 receives the window level and the window width at the end of generating the annotation information 21 together with the annotation information 21 from each annotator terminal 11 . Among the received window level and window width, the support server 10 applies the window level and window width attached to the annotation information 21 similar to the confirmed annotation information 42 to the medical image 20 superimposed on the annotation information 21 and the confirmed annotation information 42 . A display condition for the medical image 20 superimposed on the annotation information 42 is assumed.
  • an image management server that accumulates and manages the medical images 20 may be provided separately from the support server 10 , and the medical images 20 may be transmitted from the image management server to the annotator terminal 11 .
  • the medical image 20 is not limited to an abdominal tomographic image captured by the illustrated CT device.
  • a tomographic image of the head taken by an MRI (Magnetic Resonance Imaging) device may be used.
  • medical images are not limited to three-dimensional images such as tomographic images.
  • it may be a two-dimensional image such as a simple radiographic image.
  • a PET (Positron Emission Tomography) image, a SPECT (Single Photon Emission Computed Tomography) image, an endoscopic image, an ultrasound image, a fundus examination image, or the like may be used.
  • Classes to be labeled are not limited to liver, tumor, hemorrhage, and blood vessels as examples.
  • Other organs such as the brain, eyes, spleen, and kidneys or anatomical parts of organs such as vertebrae and bones such as ribs, S1-S10 of the lung, pancreatic head, pancreatic body, and pancreatic tail, and more.
  • may be other abnormal findings such as cysts, atrophy, ductal stenosis, or ductal dilatation. It may also be a pacemaker, an artificial joint, a bolt for fracture treatment, or the like.
  • the display condition data 81 of the fourth embodiment is set to values adapted to the type of the medical image 20, the organ to be classified, or the body type of the patient.
  • the display condition data 81 is further set to a value adapted to the irradiation dose.
  • a label may be assigned to a rectangular frame (when the medical image 20 is a two-dimensional image) or a box-shaped frame (when the medical image 20 is a three-dimensional image) surrounding an entire class such as a tumor. In this case, for example, the label given to the area where all the frames overlap is adopted as the label of the finalized annotation information 42 .
  • annotator 15 is described as a person such as a doctor, but is not limited to this.
  • Annotators 15 may be machine learning models.
  • Various screens such as the annotation information generation screen 85 may be sent from the support server 10 to the annotator terminal 11 in the form of screen data for web distribution created in a markup language such as XML (Extensible Markup Language).
  • XML Extensible Markup Language
  • the annotator terminal 11 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 13 .
  • JSON Javascript (registered trademark) Object Notation
  • the hardware configuration of the computer that constitutes the support server 10 can be modified in various ways.
  • the support server 10 can be composed of a plurality of server computers separated as hardware for the purpose of improving processing capability and reliability.
  • the functions of the RW control unit 50, the image transmission unit 51, and the information reception unit 52, and the functions of the derivation unit 53 and the generation unit 54 are distributed to two server computers.
  • the support server 10 is composed of two server computers. A part or all of the function of each processing unit of the support server 10 may be performed by the annotator terminal 11 .
  • the hardware configuration of the computer of the support server 10 can be changed as appropriate according to required performance such as processing power, safety, and reliability.
  • APs such as the operating program 40 can of course be duplicated or distributed and stored in multiple storages for the purpose of ensuring safety and reliability.
  • the RW control unit 50, the image transmission units 51 and 80, the information reception unit 52, the derivation unit 53, the generation unit 54, the detection unit 105, and the information transmission unit 140 perform various processes.
  • the various processors include FPGA (Field Programmable Gate Array), etc., whose circuit configuration can be changed after manufacturing.
  • Programmable Logic Device (PLD) which is a processor, and/or ASIC (Application Specific Integrated Circuit), etc. etc. are included.
  • One processing unit may be configured with one of these various processors, or a combination of two or more processors of the same or different type (for example, a combination of a plurality of FPGAs and/or a CPU and combination with FPGA). Also, a plurality of processing units may be configured by one processor.
  • a single processor is configured by combining one or more CPUs and software.
  • a processor functions as multiple processing units.
  • SoC System On Chip
  • a processor that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. be.
  • the various processing units are configured using one or more of the above various processors as a hardware structure.
  • an electric circuit combining circuit elements such as semiconductor elements can be used.
  • the annotation information is preferably information in which labels of different classes are assigned to the same region.
  • the commonality data is a numerical value related to the number of annotators assigned labels for multiple classes, and the confirmation condition is that the assigned label is adopted only when the numerical value is greater than or equal to the threshold. preferable.
  • the medical image is preferably attached with display condition data for displaying the medical image under the same display condition when a plurality of annotators view the medical image.
  • the processor detects human body regions in which human bodies appear in the medical image and derives commonality data only for the detected human body regions.
  • the processor weights according to the attributes of the annotators when deriving the commonality data.
  • the processor sets a lower numerical value representing the reliability of the label at the edge of the class area than at the center of the class area in the final annotation information.
  • the processor preferably transmits the annotation information and the confirmed annotation information to an annotator terminal used by the annotator.
  • the technology of the present disclosure can also appropriately combine various embodiments and/or various modifications described above. Moreover, it is needless to say that various configurations can be employed without departing from the scope of the present invention without being limited to the above embodiments. Furthermore, the technology of the present disclosure extends to storage media that non-temporarily store programs in addition to programs.
  • a and/or B is synonymous with “at least one of A and B.” That is, “A and/or B” means that only A, only B, or a combination of A and B may be used.
  • a and/or B means that only A, only B, or a combination of A and B may be used.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Data Mining & Analysis (AREA)
  • Primary Health Care (AREA)
  • Artificial Intelligence (AREA)
  • Epidemiology (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Pathology (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
PCT/JP2022/008070 2021-03-30 2022-02-25 機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム Ceased WO2022209500A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2023510680A JP7776495B2 (ja) 2021-03-30 2022-02-25 機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム
US18/472,268 US20240013894A1 (en) 2021-03-30 2023-09-22 Machine learning model creation support apparatus, method of operating machine learning model creation support apparatus, and program for operating machine learning model creation support apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-057322 2021-03-30
JP2021057322 2021-03-30

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/472,268 Continuation US20240013894A1 (en) 2021-03-30 2023-09-22 Machine learning model creation support apparatus, method of operating machine learning model creation support apparatus, and program for operating machine learning model creation support apparatus

Publications (1)

Publication Number Publication Date
WO2022209500A1 true WO2022209500A1 (ja) 2022-10-06

Family

ID=83458482

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/008070 Ceased WO2022209500A1 (ja) 2021-03-30 2022-02-25 機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム

Country Status (3)

Country Link
US (1) US20240013894A1 (https=)
JP (1) JP7776495B2 (https=)
WO (1) WO2022209500A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025182042A1 (ja) * 2024-02-29 2025-09-04 富士通株式会社 アノテーション支援プログラム、方法、及び装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019003485A1 (ja) * 2017-06-30 2019-01-03 株式会社Ａｂｅｊａ 機械学習又は推論のための計算機システム及び方法
JP2019096006A (ja) * 2017-11-21 2019-06-20 キヤノン株式会社 情報処理装置、情報処理方法
JP2020091543A (ja) * 2018-12-03 2020-06-11 キヤノン株式会社 学習装置、処理装置、ニューラルネットワーク、学習方法、及びプログラム
WO2020194662A1 (ja) * 2019-03-28 2020-10-01 オリンパス株式会社 情報処理システム、内視鏡システム、学習済みモデル、情報記憶媒体、情報処理方法及び学習済みモデルの製造方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3706128B1 (en) * 2018-11-09 2026-01-28 Lunit Inc. Method for managing annotation work, and apparatus and system supporting same
JP7596092B2 (ja) * 2019-08-30 2024-12-09 キヤノン株式会社 情報処理装置、情報処理方法、情報処理システム及びプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019003485A1 (ja) * 2017-06-30 2019-01-03 株式会社Ａｂｅｊａ 機械学習又は推論のための計算機システム及び方法
JP2019096006A (ja) * 2017-11-21 2019-06-20 キヤノン株式会社 情報処理装置、情報処理方法
JP2020091543A (ja) * 2018-12-03 2020-06-11 キヤノン株式会社 学習装置、処理装置、ニューラルネットワーク、学習方法、及びプログラム
WO2020194662A1 (ja) * 2019-03-28 2020-10-01 オリンパス株式会社 情報処理システム、内視鏡システム、学習済みモデル、情報記憶媒体、情報処理方法及び学習済みモデルの製造方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025182042A1 (ja) * 2024-02-29 2025-09-04 富士通株式会社 アノテーション支援プログラム、方法、及び装置

Also Published As

Publication number Publication date
JP7776495B2 (ja) 2025-11-26
US20240013894A1 (en) 2024-01-11
JPWO2022209500A1 (https=) 2022-10-06

Similar Documents

Publication Publication Date Title
Waite et al. Interpretive error in radiology
US11139067B2 (en) Medical image display device, method, and program
US12315626B2 (en) Automated protocoling in medical imaging systems
JP6768862B2 (ja) 医用画像処理方法、医用画像処理装置、医用画像処理システム及び医用画像処理プログラム
US11295510B2 (en) Method and apparatus for use of function-function surfaces and higher-order structures as a tool
WO2020179234A1 (ja) 画像診断支援装置および画像処理方法
WO2022264755A1 (ja) 医用画像診断システム、医用画像診断方法及びプログラム
JP2020171480A (ja) 医用画像処理装置及び医用画像処理システム
US20240013524A1 (en) Information processing apparatus, method of operating information processing apparatus, and program for operating information processing apparatus
Meltzer et al. Detection and characterization of solid pulmonary nodules at digital chest tomosynthesis: data from a cohort of the pilot Swedish cardiopulmonary bioimage study
Chen et al. Machine learning to detect cervical spine fractures missed by radiologists on CT: analysis using seven award-winning models from the 2022 RSNA Cervical Spine Fracture AI Challenge
US12205710B2 (en) Image generation device, image generation method, image generation program, learning device, learning method, and learning program
WO2022209500A1 (ja) 機械学習モデル作成支援装置、機械学習モデル作成支援装置の作動方法、機械学習モデル作成支援装置の作動プログラム
JP2026020375A (ja) 医用画像診断システム、医用画像診断システムの作動方法及びプログラム
GB2611602A (en) A System and Method for quality assurance of medical image segmentation
US20240037739A1 (en) Image processing apparatus, image processing method, and image processing program
Hoebel et al. Not without context—a multiple methods study on evaluation and correction of automated brain tumor segmentations by experts
CN119150993A (zh) 多模态生成模型训练方法及装置、文本生成方法及装置
JP6838971B2 (ja) 医用画像保管装置および医用画像保管システム
US11923072B2 (en) Image diagnosis supporting device and image processing method
CN113538395A (zh) 图像处理方法、装置、设备、可读存储介质
US20240274287A1 (en) Image diagnosis assisting device, operation method of image diagnosis assisting device, and a program
US20230223138A1 (en) Medical information processing system, medical information processing method, and storage medium
US20240386564A1 (en) Image processing apparatus, image processing method, and medium
US20240331146A1 (en) Image processing apparatus, image processing method, image processing program, learning apparatus, learning method, and learning program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22779725

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023510680

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22779725

Country of ref document: EP

Kind code of ref document: A1