WO2023133223A1 - Estimation de difficulté d'étude d'image médicale - Google Patents

Estimation de difficulté d'étude d'image médicale Download PDF

Info

Publication number
WO2023133223A1
WO2023133223A1 PCT/US2023/010240 US2023010240W WO2023133223A1 WO 2023133223 A1 WO2023133223 A1 WO 2023133223A1 US 2023010240 W US2023010240 W US 2023010240W WO 2023133223 A1 WO2023133223 A1 WO 2023133223A1
Authority
WO
WIPO (PCT)
Prior art keywords
medical image
study
studies
image study
prior
Prior art date
Application number
PCT/US2023/010240
Other languages
English (en)
Inventor
Sun Young Park
Dustin Michael SARGENT
Benedikt Graf
Larissa Christina SCHUDLO
Marwan Sati
Original Assignee
Merative Us L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Merative Us L.P. filed Critical Merative Us L.P.
Publication of WO2023133223A1 publication Critical patent/WO2023133223A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • Embodiments described herein relate to systems and methods for estimating a difficulty metric of a medical image study.
  • Some systems and methods use various machine learning models of an artificial intelligence (Al) system to estimate a difficulty metric of a medical image study, wherein the medical image study is assigned for review based on the difficulty metric.
  • Al artificial intelligence
  • a medical image study may include one or more medical images captured of a patient.
  • An image study may also include information regarding the patient, image study information, order information, or a combination thereof.
  • a healthcare provider such as a radiologist, may receive the medical image study for review and generation of an associated report (e.g., with annotations, notes, finding, diagnoses, etc.).
  • RVUs Relative Value Units
  • RVUs are a current measure for standardizing a difficulty level of various types of medical imaging studies. RVUs may be used to determine reimbursement for healthcare providers for different study types.
  • embodiments described herein recognize that RVUs do not account for many factors that significantly affect the complexity and difficulty of a medical image study, such as studies with current and multiple priors, clinical findings, demographic information of a patient (e.g., age, gender, body mass index, etc.), study details (contrast or no contrast), etc.
  • embodiments described herein recognize that greater amounts of relevant priors increase the amount of work needed to review the medical image study, particularly when the priors include multiple findings or impressions that may be correlated with artificial intelligence and computer aided diagnosis findings in a current exam.
  • Embodiments described herein also recognize that accurately measuring study difficulty or complexity is important for efficient workload balancing and medical image study distribution among healthcare providers. [0017] Accordingly, embodiments described herein provide methods and systems for estimating a difficulty metric of a medical image study. The methods and systems can use models of an artificial intelligence (Al) system to learn patterns of study difficulty using factors of information of the medical image study, for example, such as medical image study information, information regarding a patient, information regarding a prior image study, etc.
  • Al artificial intelligence
  • embodiments described herein can use ensemble methods to account for factors of information of the medical image study that RVUs do not consider.
  • Ensemble methods are a machine learning technique that combines various machine learning models to produce a predictive performance that any of the various machine learning models alone cannot produce.
  • using an Al system as described herein to assign medical image studies to healthcare providers provides a difficulty metric, which is more effective than RVUs, to estimate and balance workload of healthcare providers.
  • the machine learning model of the Al system as described herein can automatically adjust over time to changing parameters of medical image studies.
  • embodiments described herein use models of an artificial intelligence (Al) system to automatically assign medical image studies to a healthcare provider.
  • Al artificial intelligence
  • one embodiment provides a computer-implemented method for assigning a medical image study for review.
  • the method includes receiving a plurality of labeled medical image studies, wherein each of the plurality of labeled medical image studies including a medical image study and a label representing a difficulty of the respective medical image study.
  • the method also includes receiving, for each of the plurality of labeled medical image studies, one or more prior image studies of a patient associated with the respective labeled medical image study.
  • the method further includes creating a set of training data including the plurality of labeled medical image studies and the one or more prior image studies received for each of the plurality of labeled medical image studies and training an artificial intelligence (Al) system using the set of training data.
  • the method includes estimating, using the Al system as trained, a difficulty metric for an unlabeled medical image study based on the unlabeled medical image study and one or more prior image studies of a patient associated with the unlabeled image study and assigning the unlabeled medical image study for review based on the difficulty metric.
  • Another embodiment provides a system for assigning a medical image study for review.
  • the system includes an electronic processor.
  • the electronic processor is configured to receive a plurality of labeled medical image studies, wherein each of the plurality of labeled medical image studies including a medical image study and a label representing a difficulty of the respective medical image study.
  • the electronic processor is also configured to receive, for each of the plurality of labeled medical image studies, one or more prior image studies of a patient associated with the respective labeled medical image study, create a set of training data including the plurality of labeled medical image studies and the one or more prior image studies received for each of the plurality of labeled medical image studies, and train an artificial intelligence (Al) system using the set of training data.
  • Al artificial intelligence
  • the electronic processor is further configured to estimate, using the Al system as trained, a difficulty metric for an unlabeled medical image study based on the unlabeled medical image study and one or more prior image studies of a patient associated with the unlabeled image study and assign the unlabeled medical image study for review based on the difficulty metric.
  • a further embodiment provides a non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions.
  • the set of functions include receiving a plurality of labeled medical image studies, wherein each of the plurality of labeled medical image studies including a medical image study and a label representing a difficult of the respective medical image study, and receiving, for each of the plurality of labeled medical image studies, one or more prior image studies of a patient associated with the respective labeled medical image study.
  • the set of functions further include creating a set of training data including the plurality of labeled medical image studies and the one or more prior image studies received for each of the plurality of labeled medical image studies, training an artificial intelligence (Al) system using the set of training data, estimating, using the Al system as trained, a difficulty metric for an unlabeled medical image study based on the unlabeled medical image study and one or more prior image studies of a patient associated with the unlabeled image study, and assigning the unlabeled medical image study for review based on the difficulty metric.
  • Al artificial intelligence
  • FIG. 1 schematically illustrates a medical study assignment system according to some embodiments.
  • FIG. 2 schematically illustrates assignment of medical image studies to individual care provider worklists according to some embodiments.
  • FIG. 3 A illustrates a training workflow of a difficulty model according to some embodiments.
  • FIG. 3B illustrates a medical image study scoring workflow of a difficulty model according to some embodiments.
  • FIG. 3C illustrates a scoring workflow of a model of the difficulty model of FIG. 3B.
  • FIG. 4 is a flowchart illustrating a method performed by the medical study assignment system of FIG. 1.
  • a plurality of hardware and software-based devices, as well as a plurality of different structural components may be utilized to implement the embodiments.
  • embodiments may include hardware, software, and electronic components or modules that, for purposes of discussion, may be illustrated and described as if the majority of the components were implemented solely in hardware.
  • the electronic-based aspects of the embodiments may be implemented in software (e.g., stored on non-transitory computer-readable medium) executable by one or more processors.
  • a plurality of hardware and software-based devices, as well as a plurality of different structural components may be utilized to implement the embodiments.
  • mobile device may include one or more electronic processors, one or more memory modules including non-transitory computer-readable medium, one or more input/output interfaces, and various connections (e.g., a system bus) connecting the components.
  • FIG. 1 illustrates a medical image study assignment system 100 according to some embodiments.
  • the system 100 includes a server 105, an information repository 110, and a workstation 120.
  • the server 105, the information repository 110, and the workstation 120 communicate over one or more wired or wireless communication networks 115. Portions of the wireless communication networks 115 may be implemented using a wide area network, such as the Internet, a local area network, such as a BluetoothTM network or Wi-Fi, and combinations or derivatives thereof.
  • the system 100 may include more or fewer servers and the single server 105 illustrated in FIG.
  • the functionality described herein is performed via a plurality of servers in a distributed or cloudcomputing environment.
  • the server 105 may communicate with multiple information repositories.
  • the system 100 may include more workstations and the single workstation 120 illustrated in FIG. 1 is purely for illustrative purposes.
  • the system 100 includes a plurality of workstations 120, each workstation associated with a care provider.
  • the components illustrated in system 100 may communicate through one or more intermediary devices (not shown).
  • the information repository 110 stores medical data, including, for example, medical image studies.
  • a medical image study may comprise a plurality of images captured of a patient using an imaging modality.
  • the information repository 110 may include a picture archiving and communication system (PACS) that stores various types of medical images.
  • the information repository 110 may also store other medical data such as patient information, reports for prior exams, pathology reports or results, or the like.
  • the information repository 110 may include an electronic medical record (EMR) system, hospital information system (HIS), a radiology information system (RIS).
  • EMR electronic medical record
  • HIS hospital information system
  • RIS radiology information system
  • the information repository 110 may also be included as part of the server 105.
  • the information repository 110 may represent multiple servers or systems, such as for example, a PACS, an EMR system, a RIS, and the like. Accordingly, the server 105 may be configured to communicate with multiple systems or servers to perform the functionality described herein. Alternatively or in addition, the information repository 110 may represent an intermediary device configured to communicate with the server 105 and one or more additional systems or servers (e.g., a PACS, an EMR system, a RIS, etc.). Accordingly, the medical data stored in or accessible through the information repository 110 can include patient information, images, reports of findings, pathology reports or results, EMR information, historical reading times of the medical image studies, relative value units (RVUs), etc.
  • RVUs relative value units
  • the patient information stored in or accessible through the information repository 110 can include information such as demographic information, procedure history, disease history, etc. related to a specific patient.
  • the images stored in the information repository 110 are generated by an imaging modality (not shown), such as an X-ray, a computed tomography (CT) scanner, a magnetic resonance imaging (MRI) scanner, or the like.
  • the information repository 110 may also be included as part of an imaging modality.
  • the images stored in the information repository 110 may be grouped into image studies. In some embodiments, images within an image study are generated by the same image modality (not shown) for a patient.
  • an image study can include metadata.
  • the metadata may include study description, number of series/slices, an imaging modality type or identifier, and patient information.
  • the metadata may be defined according to one or more standards for communicating medical data, such as, for example, the digital imaging and communications in medicine (DICOM) standard, the health level seven (HL7) standard, or the like.
  • DICOM digital imaging and communications in medicine
  • HL7 health level seven
  • Reports or findings stored in or accessible through the information repository 110 can include reports or findings automatically generated by one or more systems, such as, for example, one or more computer-aided diagnosis (CAD) systems, artificial intelligence systems, or the like.
  • the reports and findings can include electronic reports or findings generated by a radiologist or other healthcare professional, such as for, example, an image study report, a pathology report, or the like.
  • a radiologist may use a RIS to create an electronic report for an image study, wherein the report includes findings or impressions, one or more diagnoses, annotations, measurements, or the like. Metadata regarding such reports or findings can also be stored in or accessible through the information repository 110.
  • timing information relating to completion of an image study report can be stored, which may represent how long it took a radiologist to read an image study and create the associated report.
  • other information relating to how a report was generated can be stored, such as, for example, what images (or what number of images) were reviewed as part of creating a report, what or what number of prior reports were reviewed as part of creating a report, or the like.
  • the server 105 includes an electronic processor 130, a memory 135, and a communication interface 140.
  • the electronic processor 130, the memory 135, and the communication interface 140 communicate wirelessly, over wired communication channels or buses, or a combination thereof.
  • the server 105 may include additional components than those illustrated in FIG. 1 in various configurations.
  • the server 105 includes multiple electronic processors, multiple memory modules, multiple communication interfaces, or a combination thereof.
  • the functionality described herein as being performed by the server 105 may be performed in a distributed nature by a plurality of computers located in various geographic locations.
  • the functionality described herein as being performed by the server 105 may be performed by a plurality of computers included in a cloud computing environment.
  • the electronic processor 130 may be, for example, a microprocessor, an applicationspecific integrated circuit (ASIC), and the like.
  • the electronic processor 130 is generally configured to execute software instructions to perform a set of functions, including the functions described herein.
  • the memory 135 includes a non-transitory computer-readable medium and stores data, including instructions executable by the electronic processor 130.
  • the communication interface 140 may be, for example, a wired or wireless transceiver or port, for communication over the communication network 115 and, optionally, one or more additional communication networks or connections.
  • the memory 135 of the server 105 includes a difficulty model 145, which may be part of a medical image study assignment engine executed via the server 105.
  • the difficulty model 145 may be, for example, an artificial intelligence system.
  • the memory 135 may store a worklist table that identifies a workload of each of a plurality of healthcare providers working within the system 100, such as a plurality of radiologists.
  • the server 105 uses the difficulty model 145 to determine a difficulty metric for each image study, wherein the difficult metric can be used to assign each the medical image study to a care provider (i.e., assign to a particular worklist table) within the system 100.
  • a care provider i.e., assign to a particular worklist table
  • FIG. 2 illustrates a workflow 200 for assigning medical image studies to care providers.
  • server 105 stores (or has access to) a worklist assignment table 210 that includes, among other things, an identifier for each medical image study needing review.
  • the server 105 also stores (or has access to) a plurality of care provider worklists 215.
  • the server 105 uses the difficulty model 145 to assign each medical image study (e.g., medical image study 205A) of the plurality of medical image studies 205 to one of the care provider worklists 215 (such as, for example, the care provider A worklist 215 A, the care provider B worklist 215B, or the care provider C worklist 215C).
  • Server 105 receives completed medical image studies and associated medical information from the information repository 110 to train the difficulty model 145.
  • the difficulty model 145 may be trained, for example, by a supervised learning method using labeled medical image studies to estimate a difficulty metric of a received unlabeled medical image study.
  • a supervised learning method is a machine learning task that learns a function that maps an input to an output based on a set of input-output pairs.
  • the set of input-output pairs (e.g., a set of training data) may include a medical image study (i.e., input) tagged with one or more labels and a difficulty metric (i.e., output).
  • an expert can provide a label (e.g., a numerical scale, such as from 1-10) for a medical image study that represents a difficulty of an image study.
  • a label for a medical image study can be automatically assigned based on a predicted reading time associated with historical reading times of comparable medical image studies (e.g., excluding outliers).
  • the information used to train the difficulty model 145 can include a plurality completed or labeled (i.e., reviewed and assigned a difficulty label or score) image studies (images included in each study) and associated additional information.
  • the training data used to train the difficulty model 145 can include a plurality of factors that impact study difficulty and, thus, result in a more accurate model 145 that considers multiple factors that can influence the difficulty of a medical image study (i.e., in addition to the images themselves included in the medical image study needing to be assigned and analyzed).
  • This additional information received and included in the training data can include, for example, study information (e.g., a study description, a number of series or slices, a modality type, a number of prior studies, a total number of images in prior studies, an imaging protocol used, or the like), patient information (e.g., demographic information, disease history, etc.), one or more prior image studies, one or more exam reports (e.g., a report for the image study, reports for prior image studies, findings, impressions, annotations, pathology reports, etc.), CAD or other Al findings in the image study or prior image studies, reading time information for the image study, an RVU assigned to the image study, or a combination thereof.
  • study information e.g., a study description, a number of series or slices, a modality type, a number of prior studies, a total number of images in prior studies, an imaging protocol used, or the like
  • patient information e.g., demographic information, disease history, etc.
  • exam reports e.g.,
  • embodiments described herein train the model using additional relevant information, such as, for example, patient demographic information and medical history information.
  • additional relevant information such as, for example, patient demographic information and medical history information.
  • a number of prior image studies associated with a patient can impact the difficulty in analyzing a new image study for the patient. For example, when a patient has only a single prior image study available, analyzing a new image study for this patient may be less complex than when the patient has multiple prior image studies available.
  • the models described herein may use the number of prior image studies, the types of prior image studies, a timing of prior image studies, findings in prior image studies (e.g., how a lesion or area of interest has changed over time between one or more prior studies), or the like to output an improved difficulty metric for an unlabeled image study that results in improved user efficiency as well as computing resource efficiency (e.g., given a more accurate initially-assigned metric and associated radiologist assignment).
  • some embodiments described herein receive prior image study information of the current patient, prior exam information of the current patient, and current exam information of the unlabeled medical image study.
  • the prior image study information may include a number of prior image studies associated with the current patient, a number of image series in a prior image study associated with the current patient, and a total number of images in the prior image studies associated with the current patient.
  • the prior exam information may include findings and impressions in prior exam reports associated with the current patient, and the current exam information may include computer-aided diagnosis (CAD) results of the unlabeled medical image study.
  • CAD computer-aided diagnosis
  • the difficulty model 145 uses this information in combination with the unlabeled medical images study itself (and optionally additional information as described above) to estimate a difficulty metric for the unlabeled medical image study.
  • the difficult model 145 can be trained using training data that includes similar data (prior image study information, prior exam information, and CAD results) for labeled medical image studies to correlate prior image study, exam reports, and CAD results to associated difficulty metrics and, thus, recognize the fact that higher number of prior image studies the more work generally needed to review an image study, especially if there are multiple findings or impressions that could be correlated with CAD results (including Al-driven results) in the image study being assigned a difficulty metric.
  • similar data prior image study information, prior exam information, and CAD results
  • CAD results including Al-driven results
  • the difficulty model 145 is trained to estimate or output a difficulty metric of a medical image study based on the training data created from a plurality of completed or labeled medical image studies.
  • FIG. 3 A illustrates a training workflow 300 of the difficulty model 145.
  • the difficulty model 145 includes a model A 145-1, a model B 145-2, and a combiner 145-3.
  • each of the models A 145-1 and B 145-2 can be configured to output a difficulty metric (also referred to as a “difficulty sub-metric” herein) and the combiner 145-3 can be configured to generate the overall difficulty metric for a medical image study based on the outputs of the two models A 145-1 and 5 145-2.
  • a difficulty metric also referred to as a “difficulty sub-metric” herein
  • the combiner 145-3 can be configured to generate the overall difficulty metric for a medical image study based on the outputs of the two models A 145-1 and 5 145-2.
  • the difficulty model 145 receives training data A 310 and training data B 320 to train the model A 145-1 and the model B 145-2, respectively, wherein each model, once trained, is configured to output a respective difficulty metric (e.g., a difficult sub-metric) for a medical image study.
  • the combiner 145-3 is configured to generate a respective difficulty metric for the medical image study based on the output of the model A 145- 1 and the model B 145-2 (e.g., combining the sub-metrics, averaging the sub-metrics, or the like).
  • the models 145-1 and 145-2 may be trained using different sets of data.
  • the training data A 310 may include the patient information 330 and the medical records 332 that correspond to a plurality of labeled medical image studies.
  • This patient and procedure information can be pulled from data stored in or available through the information repository 110 using natural language processing (NLP) techniques.
  • NLP techniques can be used to pull and standardize (e.g., categorize) relevant information (e.g., a normal, benign, or malignant finding) from image study reports stored in a RIS.
  • the patient information 330 may include, for example, demographic information such as a gender, age, weight, medical condition, ethnicity, geographic location, or the like, or a combination thereof. Additionally, the patient information 330 may include, for example, disease history, such as abnormal condition of a part, organ, or system of a patient resulting from various causes, such as infection, inflammation, environmental factors, or genetic defects.
  • the medical records 332 may include, for example, medical records such as prior reports, findings/impressions, annotations, pathology reports, pathology results, computer aided diagnosis (CAD) or other artificial intelligence (Al) findings in current and prior exams. Additionally, the medical records 332 may include, for example, medical image study descriptions that identify the purpose of the study, type of data collected, and/or how the collected data will be used.
  • medical records such as prior reports, findings/impressions, annotations, pathology reports, pathology results, computer aided diagnosis (CAD) or other artificial intelligence (Al) findings in current and prior exams.
  • CAD computer aided diagnosis
  • Al artificial intelligence
  • the training data B 320 includes the patient information 330, the medical records 332, and images 334 that correspond to the plurality of labeled medical image studies of the information repository 110.
  • the images 334 may include, for example, a study description, number of series images/slices, modality, number of priors, imaging protocol, image volume, relative pathological findings from prior reports images, annotations, biopsies, etc. Additionally, the images 334 may include metadata, such as lesion findings and findings of lesion complexity (e.g., number, size, shape, mass, calcification, etc.) of current and prior images of the images 334.
  • the model A 145-1 may be, for example, a machine learning model for estimating relationships between a dependent variable (e.g., difficulty metric, procedure information, etc.) and one or more independent variables, such as the patient information 330 and the medical records 332.
  • the model A 145-1 is configured to use regression analysis using the patient and procedure information for the labeled image studies.
  • the difficulty model 145 utilizes the model A 145-1 to identify causal relationships between a dependent variable and a collection of independent variables in a fixed dataset, such as medical study information (e.g., the patient information 330 and the medical records 332) of a medical image study.
  • the model B 145-2 may be, for example, a sequence machine learning model for estimating an output (e.g., difficulty metric, medical image study complexity, etc.) based on a sequence of data inputs, such as the patient information 330, the medical records 332, and the images 334.
  • the model B 145-2 may be, for example, a recurrent neural network (RNN), temporal convolutional network (TCN), long-short term memory (LSTM), or any other machine learning model capable of analyzing time-series data.
  • RNN recurrent neural network
  • TCN temporal convolutional network
  • LSTM long-short term memory
  • the difficulty model 145 utilizes the model B 145-2 to identify causal relationships between a complexity of a medical image study (e.g., output, difficulty metric, etc.) and time series data (e.g., the patient information 330, the medical records 332, and the images 334) corresponding to a patient of the medical image study.
  • a complexity of a medical image study e.g., output, difficulty metric, etc.
  • time series data e.g., the patient information 330, the medical records 332, and the images 334
  • the combiner 145-3 is configured to generate a final or overall difficulty metric for a medical image study based on the output of the model A 145-1 and the model B 145-2.
  • the combiner 145-3 may, for example, determine a sum (e.g., difficulty metric) of respective outputs of the model A 145-1 and the model B 145-2.
  • the combiner 145-3 may, for example, determine an average (e.g., difficulty metric) of respective outputs of the model A 145-1 and the model B 145-2.
  • Other pooling, stacking, and boosting algorithms can be used by the combiner 145-3 in various embodiments.
  • FIG. 3A illustrates the patient information 330, the medical records 332, and images 334 as separate inputs of the information repository 110
  • the server 105 may receive the inputs in various combinations and from various sources. Accordingly, the patient information 330, the medical records 332, and images 334 are shown as separate inputs in FIG. 3 A for illustrative purposes.
  • FIG. 3B illustrates a medical image study workflow 350 for assigning a difficulty metric to an unlabeled medical image study using the difficulty model 145.
  • the difficulty model 145 receives an unlabeled medical image study and associated information, which may include, for example, a patient information 430, medical records 432, and images 434 included in the unlabeled image study.
  • an unlabeled medical image study and associated information may include, for example, a patient information 430, medical records 432, and images 434 included in the unlabeled image study.
  • the patient information 430, the medical records 432 are input into the model A 145-1, and the patient information 330, the medical records 332, and the images 334 are input into the model B 145-2.
  • the outputs of the model A 145-1 and the model B 145-2 are combined via the combiner 145-3 to generate a difficulty metric for the unlabeled medical image study.
  • FIG. 3B illustrates the patient information 430, the medical records 432, and images 434 as separate inputs of the information repository 110
  • the server 105 may receive the inputs in various combinations and from various sources. Accordingly, the patient information 430, the medical records 432, and images 434 are shown as separate inputs in FIG. 3B for illustrative purposes.
  • FIG. 3C illustrates a scoring workflow 355 of a model of the difficulty model of FIG. 3B for assigning a difficulty value to an unlabeled medical image study using the model B 145-2 of the difficulty model 145.
  • the difficulty model B 145-2 receives an unlabeled medical image study and associated information, which may include, for example, a medical records 432A, prior images 434A, and current images 434B included in the unlabeled image study.
  • relevant information of prior reports of the medical records 432A such as, for example, a normal, benign, or malignant finding are input into the model B 145-2.
  • prior findings related to lesion complexity of prior images, annotations, biopsies, or computer aided diagnosis (CAD) results of the prior images 434A such as, for example, number of lesions, size of lesions, lesion mass, calcification, etc.
  • relevant information of the current images 434B such as, for example, current lesion findings, CAD results, etc., are input into the model B 145-2.
  • the model B 145-2 outputs a difficulty value that corresponds to identified causal relationships complexity of findings of the unlabeled medical image study. For example, a higher number of lesions findings in current images of an unlabeled medical image study with respect to prior reports can result in a higher difficulty value.
  • the difficulty metric assigned to the medical image study needing review can be used to assign the medical image study to a healthcare provider (e.g., a radiologist) and, in particular, can be used to provide a more balanced distribution of image studies needing review.
  • FIG. 4 is a flowchart illustrating a method 400 for estimating a difficulty metric of a medical image study and assigning the medical image study for review based on the difficulty metric.
  • the method 400 may be performed by the server 105 (i.e., the electronic processor 130 implementing the difficulty model 145). However, in other embodiments, the method 400 may be performed by multiple servers or systems in various configurations and distributions.
  • the method 400 includes receiving labeled medical image study information including labeled medical image studies (image studies with an associated difficult metric) and associated information as described above (at block 405).
  • labeled medical image studies may be uploaded to the information repository 110, and the server 105 may receive the medical image studies and use the medical image studies to access or receive the associated information regarding the medical image studies from (e.g., through a push or pull configuration) the information repository 110, other data sources, or a combination thereof as described above.
  • the labeled medical image study information can include not only labeled image studies but also associated patient and procedure information, reports, and prior image studies and associated reports and findings.
  • the method 400 includes creating a set of training data including the labeled medical image study information (at block 410).
  • the server 105 may utilize a plurality of received medical image studies uploaded to the information repository 110 to create a labeled set of data that may include information, such as input-output pairs, in memory 135.
  • the inputoutput pairs may include a set of features of a medical image study (e.g., input) and difficulty metric corresponding to the set features (e.g., output).
  • the labels i.e., the difficulty metrics
  • the difficulty metrics may be defined manually by an expert or determined based on reading information.
  • the method 400 includes training an artificial intelligence system using the set of training data (at block 415).
  • the server 105 inputs a created labeled set of data into the difficulty model 145.
  • the server 105 reserves a segment of the plurality of received medical image studies uploaded to the information repository 110 to create a test set of data, which qualifies performance of the difficulty model 145.
  • the server 105 inputs the test set of data into the difficulty model 145 to determine an accuracy of the difficulty model 145.
  • the server 105 may iteratively input labeled set of data and the test set of data into the difficulty model 145 until performance of the difficulty model 145 reaches a target accuracy.
  • the difficulty model 145 includes multiple (e.g., two) models, wherein each model can be trained using a particular subset of the training data.
  • the method 400 also includes, after training the difficulty model 145, receiving an unlabeled medical image study (at block 420).
  • an unlabeled medical image study may be uploaded to the information repository 110.
  • the server 105 can use information included in the uploaded image study to access or receive associated medical information regarding the unlabeled medical image study, such as from the information repository 110, other data sources, or a combination thereof.
  • the associated information can include patient information, procedure information, prior image studies, reports associated with prior image studies, pathology reports, CAD or Al results for the prior image studies, the unlabeled image study, or combinations thereof.
  • the type of information used to estimate a difficulty metric for an unlabeled image study via the difficulty model 145 is similar to the data used to train the difficulty model 145 (e.g., the same type of data with the exception of a label).
  • the server 105 provides the medical study information of the unlabeled medical image study to the difficulty model 145, which estimates a difficulty metric for the unlabeled medical image study (at block 425).
  • the method 400 further includes assigning the unlabeled medical image study for review based on the estimated difficulty metric (at block 430). For example, the server 105 assigns an unlabeled medical image study to an identifier of a care provider in a worklist table stored in the memory 135. In this example, the server 105 may assign the unlabeled medical image study to a care provider with an available status based on the worklist table.
  • the server 105 receives a total workload (e.g., a cumulative difficulty metric for a care provider) from a worklist table stored in the memory 135 for each care provider working within the system 100.
  • the server 105 may assign the unlabeled medical image study to a care provider using the total workload for each care provider and a determined difficulty metric for the unlabeled medical image study.
  • the server 105 may assign the unlabeled medical image study to adhere to a set of parameters, such as a cumulative difficulty metric threshold, an average total workload of care providers of the worklist table, etc. to balance workloads.
  • the method 400 includes transmitting a received unlabeled medical image study to a workstation of a care provider.
  • the processor 130 may route the unlabeled medical image study to workstation 120 of a care provider using updated information (e.g., assignment information) of a worklist table stored in the memory 135.
  • embodiments described herein account for the many factors that can contribute to the difficulty of reviewing a medical image study, including whether a current study has multiple prior studies and prior findings or reports and patient information.
  • Using artificial intelligence allows embodiments described herein to learn patterns of study difficulty taking into account these factors, which allows for more accurate difficulty metrics and, consequently, more balanced workload distribution among radiologists.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Pathology (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

Procédés et systèmes permettant d' attribuer une étude d'image médicale en vue d'une révision. Un procédé consiste à recevoir une pluralité d'études d'image médicale étiquetées et une ou plusieurs études d'image antérieures d'un patient associées à chaque étude d'une pluralité d'études d'image médicale étiquetées. Le procédé consiste également à créer un ensemble de données de formation comprenant la pluralité d'études d'image médicale étiquetées et l'étude ou les études d'image antérieures reçues pour chaque étude de la pluralité d'études d'image médicale étiquetées et la formation d'un système d'intelligence artificielle (IA) à l'aide de l'ensemble de données de formation. De plus, le procédé consiste à estimer, à l'aide du système d'IA formé, une mesure de difficulté pour une étude d'image médicale non étiquetée sur la base de l'étude d'image médicale non étiquetée et d'une ou de plusieurs études d'image antérieures d'un patient associées à l'étude d'image non étiquetée et l'affectation de l'étude d'image médicale non étiquetée à une révision sur la base de la mesure de difficulté.
PCT/US2023/010240 2022-01-05 2023-01-05 Estimation de difficulté d'étude d'image médicale WO2023133223A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US17/569,465 2022-01-05
US17/569,465 US20230214994A1 (en) 2022-01-05 2022-01-05 Medical image study difficulty estimation

Publications (1)

Publication Number Publication Date
WO2023133223A1 true WO2023133223A1 (fr) 2023-07-13

Family

ID=86991971

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/010240 WO2023133223A1 (fr) 2022-01-05 2023-01-05 Estimation de difficulté d'étude d'image médicale

Country Status (2)

Country Link
US (1) US20230214994A1 (fr)
WO (1) WO2023133223A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190355269A1 (en) * 2018-05-17 2019-11-21 Pearson Education, Inc. Diagnostic analyzer for visual-spatial content
US20200381122A1 (en) * 2019-05-31 2020-12-03 PAIGE.AI, Inc. Systems and methods for processing images of slides to automatically prioritize the processed images of slides for digital pathology
WO2020238626A1 (fr) * 2019-05-29 2020-12-03 腾讯科技（深圳）有限公司 Procédé et dispositif de détermination d'état d'image, appareil, système et support d'enregistrement informatique
US20210201190A1 (en) * 2019-12-27 2021-07-01 GE Precision Healthcare LLC Machine learning model development and optimization process that ensures performance validation and data sufficiency for regulatory approval
US11210785B1 (en) * 2019-10-17 2021-12-28 Robert Edwin Douglas Labeling system for cross-sectional medical imaging examinations

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190355269A1 (en) * 2018-05-17 2019-11-21 Pearson Education, Inc. Diagnostic analyzer for visual-spatial content
WO2020238626A1 (fr) * 2019-05-29 2020-12-03 腾讯科技（深圳）有限公司 Procédé et dispositif de détermination d'état d'image, appareil, système et support d'enregistrement informatique
US20200381122A1 (en) * 2019-05-31 2020-12-03 PAIGE.AI, Inc. Systems and methods for processing images of slides to automatically prioritize the processed images of slides for digital pathology
US11210785B1 (en) * 2019-10-17 2021-12-28 Robert Edwin Douglas Labeling system for cross-sectional medical imaging examinations
US20210201190A1 (en) * 2019-12-27 2021-07-01 GE Precision Healthcare LLC Machine learning model development and optimization process that ensures performance validation and data sufficiency for regulatory approval

Also Published As

Publication number Publication date
US20230214994A1 (en) 2023-07-06

Similar Documents

Publication Publication Date Title
US10937164B2 (en) Medical evaluation machine learning workflows and processes
CN108784655B (zh) 针对医疗患者的快速评估和后果分析
Varma et al. Automated abnormality detection in lower extremity radiographs using deep learning
EP3215968B1 (fr) Étiquetage de structure d'intérêt anatomique optimisé
US20190156947A1 (en) Automated information collection and evaluation of clinical data
JP6014059B2 (ja) 医療データの知的リンキング方法及びシステム
JP7374202B2 (ja) 機械学習システムおよび方法、統合サーバ、プログラムならびに推論モデルの作成方法
US20210035680A1 (en) Systems and methods for automating clinical workflow decisions and generating a priority read indicator
Kapoor et al. Workflow applications of artificial intelligence in radiology and an overview of available tools
US20220237898A1 (en) Machine learning system and method, integration server, information processing apparatus, program, and inference model creation method
Martin-Carreras et al. From data to value: how artificial intelligence augments the radiology business to create value
Singhal et al. Opportunities and challenges for biomarker discovery using electronic health record data
JP7278256B2 (ja) 画像獲得ワークフローを最適化するためのデバイス、システム、及び方法
US20230214994A1 (en) Medical image study difficulty estimation
CN112447287A (zh) 自动化的临床工作流程
EP3667674A1 (fr) Procédé et système d'évaluation d'images de différents patients, programme informatique et support d'informations lisible par voie électronique
US20200075163A1 (en) Diagnostic decision support for patient management
US20230309967A1 (en) Surfacing insights into left and right ventricular dysfunction through deep learning
US11749398B2 (en) Medical image recognition system and medical image recognition method
US20230238096A1 (en) Method and system for providing a medical report
Ma et al. Design and development of an ethnically-diverse imaging informatics-based eFolder system for multiple sclerosis patients
US20220319650A1 (en) Method and System for Providing Information About a State of Health of a Patient
EP4216229A1 (fr) Abonnement et récupération de données d'imagerie médicale
US20200279137A1 (en) Systems and Methods for Evaluating Artificial Intelligence Applications in Clinical Practice
US20240136068A1 (en) Method and device for providing medical prediction by using artificial intelligence model

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23737598

Country of ref document: EP

Kind code of ref document: A1