WO2021122670A1 - Environnement de co-entraînement pour améliorer mutuellement l'extraction de concepts à partir de notes cliniques et la classification d'images médicales - Google Patents

Environnement de co-entraînement pour améliorer mutuellement l'extraction de concepts à partir de notes cliniques et la classification d'images médicales Download PDF

Info

Publication number
WO2021122670A1
WO2021122670A1 PCT/EP2020/086320 EP2020086320W WO2021122670A1 WO 2021122670 A1 WO2021122670 A1 WO 2021122670A1 EP 2020086320 W EP2020086320 W EP 2020086320W WO 2021122670 A1 WO2021122670 A1 WO 2021122670A1
Authority
WO
WIPO (PCT)
Prior art keywords
text
machine learning
learning model
reports
labeled
Prior art date
Application number
PCT/EP2020/086320
Other languages
English (en)
Inventor
Ashequl Qadir
Kathy Mi Young LEE
Claire Yunzhu ZHAO
Minnan XU
Original Assignee
Koninklijke Philips N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips N.V. filed Critical Koninklijke Philips N.V.
Priority to CN202080088182.6A priority Critical patent/CN114868193A/zh
Priority to US17/781,081 priority patent/US20230005252A1/en
Priority to EP20839243.1A priority patent/EP4078603A1/fr
Publication of WO2021122670A1 publication Critical patent/WO2021122670A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06V10/7747—Organisation of the process, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Definitions

  • Various exemplary embodiments disclosed herein relate generally to a co-training framework to mutually improve concept extraction from clinical notes and medical image classification.
  • Various embodiments relate to a system for training a text report identification machine learning model and an image identification machine learning model, including: a memory; a processor connected to the memory, the processor configured to: initially train a text report machine learning model, using a labeled set of text reports including text pre-processing the text report and extracting features from the pre-processed text report, wherein the extracted features are input into the text report machine learning model; initially train an image machine learning model, using a labeled set of images; apply the initially trained text report machine learning model to a first set of unlabeled text reports with associated images to label the associated images; select a first portion of labeled associated images; re -train the image machine learning model using the selected first portion of labeled associated images; apply the initially trained image machine learning model to a first set of unlabeled images with associated text reports to label the associated text reports; select a first portion of labeled associated text reports; and re -train the text report machine learning model using the selected first portion of labeled associated
  • selecting a portion of labeled associated images includes selecting associated images with text report machine learning model outputs having a confidence level above a first confidence threshold
  • selecting a portion of labeled associated text reports includes selecting associated text reports with image machine learning model outputs having a confidence level above a second confidence threshold.
  • selecting a portion of labeled associated images further includes selecting the N associated images with text report machine learning model outputs having the highest confidence levels, wherein N is a predetermined value
  • selecting a portion of labeled associated text reports further includes selecting the M associated text reports with image machine learning model outputs having the highest confidence levels, wherein M is a predetermined value.
  • selecting a portion of labeled associated images includes selecting associated images with text report machine learning model outputs having a confidence level above a first confidence threshold, when there are more than N selected associated images, further selecting N associated images with text reports that have the highest confidence levels, selecting a portion of labeled associated text reports includes selecting associated text reports with image machine learning model outputs having a confidence level above a second confidence threshold, and when there are more than M selected associated text reports, further selecting M associated text reports with image that have the highest confidence levels.
  • the processor is further configured to: apply the retrained text report machine learning model to a second set of unlabeled text reports with associated images to label the associated images; select a second portion of labeled associated images; re -train the retrained image machine learning model using the selected second portion of labeled associated images; apply the retrained image machine learning model to a second set of unlabeled images with associated text reports to label the associated text reports; select a second portion of labeled associated text reports; and re -train the retrained text report machine learning model using the selected second portion of labeled associated text reports.
  • FIG. 1 For various embodiments, relate to a method for training a text report identification machine learning model and an image identification machine learning model, including: initially training a text report machine learning model, using a labeled set of text reports including text pre processing the text report and extracting features from the pre-processed text report, wherein the extracted features are input into the text report machine learning model; initially training an image machine learning model, using a labeled set of images; applying the initially trained text report machine learning model to a first set of unlabeled text reports with associated images to label the associated images; selecting a first portion of labeled associated images; re -training the image machine learning model using the selected first portion of labeled associated images; applying the initially trained image machine learning model to a first set of unlabeled images with associated text reports to label the associated text reports; selecting a first portion of labeled associated text reports; and re -training the text report machine learning model using the selected first portion of labeled associated text reports.
  • selecting a portion of labeled associated images includes selecting associated images with text report machine learning model outputs having a confidence level above a first confidence threshold
  • selecting a portion of labeled associated text reports includes selecting associated text reports with image machine learning model outputs having a confidence level above a second confidence threshold.
  • selecting a portion of labeled associated images further includes selecting the N associated images with text report machine learning model outputs having the highest confidence levels
  • selecting a portion of labeled associated text reports further includes selecting the M associated text reports with image machine learning model outputs having the highest confidence levels.
  • selecting a portion of labeled associated images includes selecting associated images with text report machine learning model outputs having a confidence level above a first confidence threshold, when there are more than N selected associated images, further selecting N associated images with text reports that have the highest confidence levels, selecting a portion of labeled associated text reports includes selecting associated text reports with image machine learning model outputs having a confidence level above a second confidence threshold, and when there are more than M selected associated text reports, further selecting M associated text reports with image that have the highest confidence levels.
  • Various embodiments are described, further including: applying the retrained text report machine learning model to a second set of unlabeled text reports with associated images to label the associated images; selecting a second portion of labeled associated images; re -training the retrained image machine learning model using the selected second portion of labeled associated images; applying the retrained image machine learning model to a second set of unlabeled images with associated text reports to label the associated text reports; selecting a second portion of labeled associated text reports; and re -training the retrained text report machine learning model using the selected second portion of labeled associated text reports.
  • FIG. 1 illustrates a co-training system that leverages two views of the data - a text view and an image view;
  • FIG. 2 shows a flow diagram illustrating the training of the text identification model
  • FIG. 3 illustrates the use of the trained text identification model on a set of input unlabeled text reports to produce a set of labels based upon a set of extracted concepts
  • FIG. 4 shows a flow diagram illustrating the image identification model
  • FIG. 5 illustrates an exemplary hardware diagram for implementing co-training system.
  • Embodiments of a co-training system will be described herein that implement a co-training framework where an image -based classifier and a text-based classifier mutually generate supplemental training instances for each other in an iterative semi-supervised learning paradigm to gradually improve their individual performances.
  • the embodiment of a co-training framework will address this problem by individually training text-based and image -based classifiers with text-based and image -based data that each identify new instances from unlabeled data to generate supplemental training instances for the other, thus allowing each classifier to improve over time as the labeled training data expands in each iteration.
  • FIG. 1 illustrates a co-training system that leverages two views of the data - (1) a text view, and (2) an image view.
  • a text-based concept identification model 104 is trained using labeled reports 102 associated with images.
  • the text identification model 104 may be a text classifier that uses various textual data features and a machine learning algorithm suitable for sequence labeling.
  • Example machine learning algorithms may include conditional random field (CRF) classifier, bidirectional long short-term memory (BiTSTM) networks, BiTSTM-CRF, etc.
  • CRF conditional random field
  • BiTSTM bidirectional long short-term memory
  • BiTSTM-CRF bidirectional long short-term memory
  • Unlabeled text reports 106 are input into the text identification model 104 to generate labeled reports 108.
  • the unlabeled images 110 associated with the unlabeled reports 108 are then labeled using the labels from the labeled reports 108 produced by the text identification model 104.
  • the text identification model 104 may produce a confidence value associated with its labeled outputs.
  • These labeled images 112 may then be used as further training samples for an image identification model 124. Not all of the labeled images 112 may be used as further training samples. In one embodiment, only labeled images 112 associated with labeled reports 108 that have a confidence level above a specified threshold value may be used to further train the image identification model 124.
  • the number of further training samples may be limited to a threshold number N of training samples, by selecting the N samples with the highest confidence levels. In yet another embodiment if there are more than N samples that exceed the threshold value, only the N values with the highest confidence values may be selected as the training samples. Other methods of limiting the further training samples may be used to ensure that the further training samples are of high enough quality to improve the training of the image identification model 124. These selected training samples may then be sent 114 to further train the image identification model 124.
  • FIG. 2 shows a flow diagram illustrating the training 200 of the text identification model 104.
  • the labeled reports 202 first undergo text preprocessing 204.
  • the text pre-processing 204 may include tokenization, lemmatization, case normalization, stopwords removal processing, etc.
  • This text pre-processing 204 takes in the raw text of the labeled reports and processes them into a consistent format in order to facilitate feature extraction.
  • the preprocessed text undergoes feature extraction 206.
  • Feature extraction may include looking at each current word in the context of prior words and next words. Also, context words may be identified.
  • Various types of features maybe extracted including morphological, orthographic, lexical, and syntactic features.
  • the feature extraction is used to provide inputs into the text identification model 212. Such features need to be defined in a consistent matter so that model may be trained to generate a consistent known set of extracted concepts out of the text identification model 212.
  • the table below gives examples of feature names, followed by example text, and the resulting feature value.
  • the first four entries use the example text of Cardiomegaly with the following feature names: Word/Phrase, Towercase, 1 Character suffix, and 2 Character suffix.
  • the associated values are: Cardiomegaly, cardiomegaly, y, and ly. Many other text feature examples are further demonstrated.
  • FIG. 3 illustrates the use of the trained text identification model 212 on a set of input unlabeled text reports to produce a set of labels based upon a set of extracted concepts 304. This may correspond to steps 106 and 108 from FIG. 1.
  • a large number of unlabeled text reports 302 may be input into the text identification model 212 to produce outputs of concepts identified in the unlabeled reports 302. These are the concepts that may be used to label the associated unlabeled images 110 to produce labeled images 112.
  • an image -based concept identification model 124 will be trained using labeled images 122 associated with text reports.
  • the image identification model 124 may be an image classifier that uses various image features and a machine learning algorithm suitable for image processing.
  • An example machine learning algorithm may include convolutional neural networks-based (CNN) class activation mapping model, but other image classifying models may be used.
  • CNN convolutional neural networks-based
  • Unlabeled images 126 are input into the image identification model 124 to generate labeled images 128.
  • the unlabeled text reports 130 associated with the unlabeled images 126 are then labeled using the labels from the labeled images 128 produced by the image identification model 124.
  • the image identification model 124 may produce a confidence value associated with its labeled outputs.
  • the documents associated with these labeled images 132 may then be used as further training samples for the text identification model 104. Not all of the labeled reports 132 may be used as further training samples. In one embodiment, only labeled reports 132 associated with labeled images 128 that have a confidence level above a specified threshold value may be used to further train the text identification model 104.
  • the number of further training samples may be limited to a threshold number M of training samples, by selecting the M samples with the highest confidence levels. In yet another embodiment if there are more than M samples that exceed the threshold value, only the M values with the highest confidence values may be selected as the training samples. Other methods of limiting the further training samples may be used to ensure that the further training samples are of high enough quality to improve the training of the report identification model 104. These selected training samples may then be sent 134 to further train the text identification model 104.
  • FIG. 4 shows a flow diagram 400 illustrating the image identification model 424.
  • the unlabeled images 426 are input into the image identification model 424.
  • the image identification model 424 corresponds to the image identification model 124 in FIG. 1.
  • the image identification model 424 may include a CNN applied to the image to extract features from the unlabeled input images 426.
  • the CNN may employ various layers including convolution layers and pooling layers in various configurations along with the application of rectified linear units (ReLU). Other types of layers may also be used in various configurations.
  • ReLU rectified linear units
  • Other types of layers may also be used in various configurations.
  • the extracted features are input into classification layers to identify various concepts associated with the image.
  • the classification layers may be fully connected layers that flatten the data and then use a softmax layer to produce outputs 428 indicating the presence of the concepts found in the image.
  • the concepts output from the image identification model 424 are consistent with the concepts output from the text identification model.
  • the image identification model 424 may be trained with a small set of initial labeled images as previously described and this initial training is supplemented with selected outputs 114 from applying the text identification model 104 on unlabeled text reports 106.
  • the features classified by the two models need to be defined in a consistent matter so that models may be trained to generate a consistent known set of extracted concepts out of the two models.
  • both trained models 104, 124 will be individually applied to unlabeled data that has clinical reports and their accompanying images paired with each other.
  • the text identification model 104 will be applied to the unstructured text reports to identify a predefined set of medical concepts in the notes in the associated text reports, which will then be used to label the associated unlabeled images.
  • the image identification based model 124 will be applied to unlabeled images to identify a set of medical concepts in the images which will then be used to label the associated unlabeled text reports.
  • the text reports with identified medical concepts and their corresponding images 114 will be used to supplement the training data 122 of the image identification model 124 to increase its training data for re -training the image identification model 124.
  • the images with identified medical concepts and their corresponding text reports 134 will be used to supplement the training data 102 of the text identification model to increase its training data for re-training the text identification model 104.
  • first portion of the unlabeled text reports and images may be used to cross train the machine learning models. Then a second portion of the unlabeled text reports and images may be used to cross train the machine learning models. This process is repeated until no new reports or images can be identified from the unlabeled data to supplement the labeled training data or concept extraction results starts to degrade as tested on a validation data set.
  • the embodiments of the co-training system described herein have various benefits.
  • the co-training system leverages text modality of data in text reports associated with images to improve an image -based classifier to classify test images for concepts and identify a target set of concepts in the images.
  • the co-training system further leverages image modality of data in images associated with text reports to improve a text-based classifier for information extraction from clinical reports.
  • the co-training system also utilizes information from unlabeled clinical test images and reports which is a limitation of supervised learning systems.
  • the co-training system further expands the initial labeled training data in iterations for both image -based and text-based models by incorporating image and text data views in co-training where the data modalities are associated but not overlapping.
  • the co-training system leverages multimodal data associated with clinical tests to improve computational models in each modality.
  • co-training system is described herein using medical images, such as X-ray, MRI, ultrasound, etc., and their associated medical reports, other images with associated text descriptions may be included. Such examples could be images and associated text found in catalogs, instruction and installations manuals, books, product web sites, social media web sites, news web sites, etc.
  • the co-training system described herein may be used to co-train a text identification and image identification models for use in classifying images and text reports in such situations. As described above, a small set of labeled information may be used to initially train the models, which training can then be supplemented using the co-training system to expand the training data from unlabeled data that includes both an image and associated text. This leads to be better and more robust text identification and image identification models when only a small labeled training set is available.
  • FIG. 5 illustrates an exemplary hardware diagram 500 for implementing co-training system.
  • the device 500 includes a processor 520, memory 530, user interface 540, network interface 550, and storage 560 interconnected via one or more system buses 510. It will be understood that FIG. 5 constitutes, in some respects, an abstraction and that the actual organization of the components of the device 500 may be more complex than illustrated.
  • the processor 520 may be any hardware device capable of executing instructions stored in memory 530 or storage 560 or otherwise processing data.
  • the processor may include a microprocessor, a graphics processing unit (GPU), field programmable gate array (FPGA), application-specific integrated circuit (ASIC), any processor capable of parallel computing, or other similar devices.
  • GPU graphics processing unit
  • FPGA field programmable gate array
  • ASIC application-specific integrated circuit
  • the memory 530 may include various memories such as, for example LI, L2, or L3 cache or system memory. As such, the memory 530 may include static random-access memory (SRAM), dynamic RAM (DRAM), flash memory, read only memory (ROM), or other similar memory devices.
  • SRAM static random-access memory
  • DRAM dynamic RAM
  • ROM read only memory
  • the user interface 540 may include one or more devices for enabling communication with a user and may present information such.
  • the user interface 540 may include a display, a touch interface, a mouse, and/or a keyboard for receiving user commands.
  • the user interface 540 may include a command line interface or graphical user interface that may be presented to a remote terminal via the network interface 550.
  • the network interface 550 may include one or more devices for enabling communication with other hardware devices.
  • the network interface 550 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol or other communications protocols, including wireless protocols.
  • the network interface 550 may implement a TCP/IP stack for communication according to the TCP/IP protocols.
  • NIC network interface card
  • the storage 560 may include one or more machine-readable storage media such as read only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, or similar storage media.
  • the storage 560 may store instructions for execution by the processor 520 or data upon with the processor 520 may operate.
  • the storage 560 may store abase operating system 561 for controlling various basic operations of the hardware 500.
  • the storage 562 may store instructions for implementing the co-training system described above including training the machine learning models and running the machine learning models on data to classify.
  • the memory 530 may also be considered to constitute a “storage device” and the storage 560 may be considered a “memory.” Various other arrangements will be apparent. Further, the memory 530 and storage 560 may both be considered to be “non-transitory machine-readable media.” As used herein, the term “non- transitory” will be understood to exclude transitory signals but to include all forms of storage, including both volatile and non-volatile memories.
  • the host device 500 is shown as including one of each described component, the various components may be duplicated in various embodiments.
  • the processor 520 may include multiple microprocessors that are configured to independently execute the methods described herein or are configured to perform steps or subroutines of the methods described herein such that the multiple processors cooperate to achieve the functionality described herein. Such plurality of processors may be of the same or different types.
  • the various hardware components may belong to separate physical systems.
  • the processor 520 may include a first processor in a first server and a second processor in a second server.
  • the co-training system described herein provides many benefits as described above.
  • the co-training system improves the classification of images and associated text, by using a co-training framework that allows for the training of the machine learning models using a small set of labeled training data supplemented by cross training using unlabeled data processed by the machine learning models.
  • This system provides a technical improvement in image and text identification systems.
  • non-transitory machine-readable storage medium will be understood to exclude a transitory propagation signal but to include all forms of volatile and non volatile memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un système et un procédé pour entraîner un modèle d'apprentissage automatique d'identification de rapport textuel et un modèle d'apprentissage automatique d'identification d'image, comprenant : entraînement initial d'un modèle d'apprentissage automatique de rapport textuel, en utilisant un ensemble étiqueté de rapports textuels comprenant le prétraitement du texte du rapport textuel et l'extraction de caractéristiques à partir du rapport textuel pré-traité, les caractéristiques extraites étant entrées dans le modèle d'apprentissage automatique de rapport textuel ; entraînement initial d'un modèle d'apprentissage automatique d'image, en utilisant un ensemble étiqueté d'images ; application du modèle d'apprentissage automatique de rapport textuel entraîné initialement à un premier ensemble de rapports textuels non étiquetés avec des images associées pour étiqueter les images associées ; sélection d'une première portion d'images associées étiquetées ; nouvel entraînement du modèle d'apprentissage automatique d'image en utilisant la première portion sélectionnée d'images associées étiquetées ; application du modèle d'apprentissage automatique d'image entraîné initialement à un premier ensemble d'images non étiquetées avec des rapports textuels associés pour étiqueter les rapports textuels associés ; sélection d'une première portion de rapports textuels associés étiquetés ; et nouvel entraînement du modèle d'apprentissage automatique de rapport textuel en utilisant la première portion sélectionnée de rapports textuels associés étiquetés.
PCT/EP2020/086320 2019-12-18 2020-12-16 Environnement de co-entraînement pour améliorer mutuellement l'extraction de concepts à partir de notes cliniques et la classification d'images médicales WO2021122670A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202080088182.6A CN114868193A (zh) 2019-12-18 2020-12-16 相互改善来自临床笔记和医学图像分类的概念提取的协同训练框架
US17/781,081 US20230005252A1 (en) 2019-12-18 2020-12-16 A co-training framework to mutually improve concept extraction from clinical notes and medical image classification
EP20839243.1A EP4078603A1 (fr) 2019-12-18 2020-12-16 Environnement de co-entraînement pour améliorer mutuellement l'extraction de concepts à partir de notes cliniques et la classification d'images médicales

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962949836P 2019-12-18 2019-12-18
US62/949,836 2019-12-18

Publications (1)

Publication Number Publication Date
WO2021122670A1 true WO2021122670A1 (fr) 2021-06-24

Family

ID=74181087

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2020/086320 WO2021122670A1 (fr) 2019-12-18 2020-12-16 Environnement de co-entraînement pour améliorer mutuellement l'extraction de concepts à partir de notes cliniques et la classification d'images médicales

Country Status (4)

Country Link
US (1) US20230005252A1 (fr)
EP (1) EP4078603A1 (fr)
CN (1) CN114868193A (fr)
WO (1) WO2021122670A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE202022100604U1 (de) 2022-02-02 2022-02-16 Pankaj Agarwal Intelligentes System zur automatischen Klassifizierung medizinischer Bilder mittels Bildverarbeitung und künstlicher Intelligenz
WO2023142532A1 (fr) * 2022-01-26 2023-08-03 华为云计算技术有限公司 Procédé et appareil de formation de modèle d'inférence

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE112020005870T5 (de) * 2019-11-29 2022-11-03 Fujifilm Corporation Unterstützungsvorrichtung für dokumentenerstellung, unterstützungsverfahren für dokumentenerstellung und unterstützungsprogramm für dokumentenerstellung
WO2021123083A1 (fr) * 2019-12-18 2021-06-24 Koninklijke Philips N.V. Procédé pour améliorer la sélection d'instance dans une structure d'amorçage pour une extraction de concept à partir de documents textuels

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019160557A1 (fr) * 2018-02-16 2019-08-22 Google Llc Extraction automatisée d'étiquettes structurées à partir d'un texte médical à l'aide de réseaux à convolution profonds et utilisation de celles-ci pour entrainer un modèle de vision artificielle

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11265168B2 (en) * 2018-03-07 2022-03-01 Private Identity Llc Systems and methods for privacy-enabled biometric processing
US11720621B2 (en) * 2019-03-18 2023-08-08 Apple Inc. Systems and methods for naming objects based on object content
US20210295551A1 (en) * 2020-03-19 2021-09-23 Unitedhealth Group Incorporated Systems and methods for automated digital image selection and pre-processing for automated content analysis

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019160557A1 (fr) * 2018-02-16 2019-08-22 Google Llc Extraction automatisée d'étiquettes structurées à partir d'un texte médical à l'aide de réseaux à convolution profonds et utilisation de celles-ci pour entrainer un modèle de vision artificielle

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023142532A1 (fr) * 2022-01-26 2023-08-03 华为云计算技术有限公司 Procédé et appareil de formation de modèle d'inférence
DE202022100604U1 (de) 2022-02-02 2022-02-16 Pankaj Agarwal Intelligentes System zur automatischen Klassifizierung medizinischer Bilder mittels Bildverarbeitung und künstlicher Intelligenz

Also Published As

Publication number Publication date
CN114868193A (zh) 2022-08-05
EP4078603A1 (fr) 2022-10-26
US20230005252A1 (en) 2023-01-05

Similar Documents

Publication Publication Date Title
US20230005252A1 (en) A co-training framework to mutually improve concept extraction from clinical notes and medical image classification
US11544529B2 (en) Semi-supervised classification with stacked autoencoder
EP3869385B1 (fr) Procédé d'extraction de données structurelles d'une image, appareil et dispositif
CN106649853A (zh) 一种基于深度学习的短文本聚类方法
CN112686345B (zh) 一种基于注意力机制的脱机英文手写识别方法
CN108509427B (zh) 文本数据的数据处理方法及应用
US11055560B2 (en) Unsupervised domain adaptation from generic forms for new OCR forms
CN113836938B (zh) 文本相似度的计算方法及装置、存储介质、电子装置
US20230169332A1 (en) Method and system for machine learning from imbalanced data with noisy labels
WO2019115236A1 (fr) Lecture indépendante et dépendante à l'aide de réseaux récurrents pour inférence de langage naturel
CN113434683B (zh) 文本分类方法、装置、介质及电子设备
US20190197433A1 (en) Methods for adaptive information extraction through adaptive learning of human annotators and devices thereof
WO2019115200A1 (fr) Système et procédé de création efficace d'un ensemble d'une inférence de langage naturel
JPWO2014073206A1 (ja) 情報処理装置、及び、情報処理方法
US20230015207A1 (en) Method to improve instance selection in bootstrapping framework for concept extraction from text documents
WO2022174499A1 (fr) Procédé et appareil de prédiction de limites prosodiques textuelles, dispositif informatique et support de stockage
JP2019028984A (ja) 非常に大きな画像集合における近似重複画像をクラスタ化するためのシステム及び方法、複数の画像をクラスタ化するための方法及びシステム、プログラム、複数の内容項目をクラスタ化するための方法
CN114881169A (zh) 使用随机特征损坏的自监督对比学习
Zhang et al. A novel multimodal retrieval model based on ELM
CN111767710B (zh) 印尼语的情感分类方法、装置、设备及介质
Buoy et al. Khmer text classification using word embedding and neural networks
Kumar et al. Point feature based recognition of handwritten Meetei Mayek script
Lakshmi An efficient telugu word image retrieval system using deep cluster
Guo et al. Combining LVQ with SVM technique for image semantic annotation
Sharma et al. Implementation of Technology to Recognize Segmented and Non-Segmented Text: A Survey

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20839243

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020839243

Country of ref document: EP

Effective date: 20220718