WO2020024058A1

WO2020024058A1 - Predicting outcomes using universal models

Info

Publication number: WO2020024058A1
Application number: PCT/CA2019/051055
Authority: WO
Inventors: Martin CARRIER-VALLIERES; Avishek Chatterjee; Ives R. LEVESQUE; Caroline REINHOLD; Jan Seuntjens
Original assignee: The Royal Institution For The Advancement Of Learning/Mcgill University
Priority date: 2018-08-02
Filing date: 2019-08-01
Publication date: 2020-02-06

Abstract

There is described herein methods and systems for generating universal models, normalizing patient features, combining normalized clinical datasets, and predicting patient outcomes from universal models. Clinical datasets may be combined from different institutions, and patient outcomes may be predicted from data collected from institutions not included in the original clinical datasets. One or more medical imaging datasets representing a patient anatomy are acquired. A set of features is extracted from each of medical imaging dataset and normalized to generate one or more normalized datasets. A universal model is then generated from the one or more normalized datasets.

Description

PREDICTING OUTCOMES USING UNIVERSAL MODELS

CROSS REFERENCE TO RELATED APPLICATIONS

[001] This patent application claims priority of US provisional Application Serial No. 62/713,606, filed on August 2, 2018, the entire contents of which are hereby incorporated by reference.

TECHNICAL FIELD

[002] The present disclosure relates generally to generation of universal models from datasets and predicting outcomes from features using universal models.

BACKGROUND OF THE ART

[003] Radiomics is used in the diagnosis or prognosis of various medical conditions. Clinical imaging datasets may be used to generate predictive features, which may subsequently be used to predict the outcome of subjects, such as disease progression or response to treatment, using a model.

[004] In many cases, clinical imaging datasets from two or more institutions are combined to increase sample size. However, directly combining clinical imaging datasets, which may have been collected with different patient populations, protocols or scanners for example, may deteriorate the generation of models. Furthermore, models generated from clinical datasets may not be universally applicable to predict patient outcomes at other institutions. Similar issues arise in other fields where datasets are used to generate predictive features, such as genomics.

[005] Therefore, improvements are needed.

SUMMARY

[006] In accordance with one broad aspect, there is provided a method for generating a universal radiomics-based predictive model in a medical system, the universal model used for predicting a patient outcome. The method comprises acquiring, at a processor, one or more medical imaging datasets representing an anatomy of a patient, extracting, by the processor, a set of features from each of the one or more medical imaging datasets, normalizing, by the processor, the set of features to generate one or more normalized datasets, and generating, by the processor, the universal model from the one or more normalized datasets.

[007] In some embodiments, acquiring the one or more medical imaging datasets comprises acquiring one of computed tomography data, positron emission tomography data, magnetic resonance data, and ultrasound data.

[008] In some embodiments, the method further comprises identifying, from the one or more medical imaging datasets, at least one region of interest. The set of features is extracted by performing one or more texture analyses in the at least one region of interest.

[009] In some embodiments, the method further comprises assessing whether the one or more medical imaging datasets are balanced and equalizing the one or more medical imaging datasets if the one or more medical imaging datasets are unbalanced.

[010] In some embodiments, equalizing the one or more medical imaging datasets comprises randomly removing data from a majority class of the one or more medical imaging datasets until the one or more medical imaging datasets are balanced.

[011] In some embodiments, normalizing the set of features comprises re scaling a distribution of each feature to have a value between 0 and 1 .

[012] In some embodiments, normalizing the set of features comprises standardizing a distribution of each feature to have zero mean and unit standard deviation.

[013] In some embodiments, the method further comprises combining at least two of the one or more normalized datasets into a combined dataset. The universal model is generated from the combined dataset.

[014] In some embodiments, the at least two normalized datasets are combined through concatenation.

[015] In some embodiments, the method further comprises equalizing the combined dataset. [016] In some embodiments, the method further comprises normalizing the combined dataset.

[017] In some embodiments, the universal model is generated using a machine learning tool configured to correlate the normalized set of features in the combined dataset with one or more patient outcomes and to predict each patient outcome accordingly.

[018] In accordance with another broad aspect, there is provided a method for providing a patient outcome from a universal radiomics-based predictive model. The method comprises retrieving, by a processor, the universal model from memory, validating the universal model on a balanced and normalized testing dataset, extracting, by the processor, a set of patient features from one or more medical imaging datasets, normalizing, by the processor, the set of patient features to generate a normalized patient dataset, applying, by the processor, the universal model to the normalized patient dataset to predict the patient outcome, and outputting, by the processor, the patient outcome.

[019] In some embodiments, the testing dataset is balanced through equalization.

[020] In some embodiments, normalizing the set of patient features comprises re-scaling a distribution of each feature to have a value between 0 and 1 .

[021] In some embodiments, normalizing the set of patient features comprises standardizing a distribution of each feature to have zero mean and unit standard deviation.

[022] In some embodiments, the set of patient features is normalized to data acquired at a same institution.

[023] In accordance with yet another broad aspect, there is provided a system for generating a universal radiomics-based predictive model in a medical system. The system comprises at least one processing unit and at least one non-transitory computer-readable memory having stored thereon program instructions executable by the at least one processing unit for acquiring one or more medical imaging datasets representing an anatomy of a patient, extracting a set of features from each of the one or more medical imaging datasets, normalizing the set of features to generate one or more normalized datasets, and generating the universal model from the one or more normalized datasets.

[024] In some embodiments, the program instructions are executable by the at least one processing unit for acquiring the one or more medical imaging datasets comprising acquiring one of computed tomography data, positron emission tomography data, magnetic resonance data, and ultrasound data.

[025] In some embodiments, the program instructions are further executable by the at least one processing unit for identifying, from the one or more medical imaging datasets, at least one region of interest, and the program instructions are executable by the at least one processing unit for extracting the set of features by performing one or more texture analyses in the at least one region of interest.

[026] In some embodiments, the program instructions are further executable by the at least one processing unit for assessing whether the one or more medical imaging datasets are balanced and equalizing the one or more medical imaging datasets if the one or more medical imaging datasets are unbalanced.

[027] In some embodiments, the program instructions are executable by the at least one processing unit for equalizing the one or more medical imaging datasets comprising randomly removing data from a majority class of the one or more medical imaging datasets until the one or more medical imaging datasets are balanced.

[028] In some embodiments, the program instructions are executable by the at least one processing unit for normalizing the set of features comprising re scaling a distribution of each feature to have a value between 0 and 1 .

[029] In some embodiments, the program instructions are executable by the at least one processing unit for normalizing the set of features comprising standardizing a distribution of each feature to have zero mean and unit standard deviation.

[030] In some embodiments, the program instructions are further executable by the at least one processing unit for combining at least two of the one or more normalized datasets into a combined dataset, and the program instructions are executable by the at least one processing unit for generating the universal model from the combined dataset [031] In some embodiments, the program instructions are executable by the at least one processing unit for combining the at least two normalized datasets through concatenation.

[032] In some embodiments, the program instructions are further executable by the at least one processing unit for equalizing the combined dataset.

[033] In some embodiments, the program instructions are further executable by the at least one processing unit for normalizing the combined dataset.

[034] In some embodiments, the program instructions are executable by the at least one processing unit for generating the universal model using a machine learning tool configured to correlate the normalized set of features in the combined dataset with one or more patient outcomes and to predict each patient outcome accordingly.

[035] In accordance with yet another broad aspect, there is provided a system for providing a patient outcome from a universal radiomics-based predictive model. The system comprises at least one processing unit and at least one non-transitory computer-readable memory having stored thereon program instructions executable by the at least one processing unit for retrieving the universal model from memory, validating the universal model on a balanced and normalized testing dataset, extracting a set of patient features from one or more medical imaging datasets, normalizing the set of patient features to generate a normalized patient dataset, applying the universal model to the normalized patient dataset to predict the patient outcome, and outputting the patient outcome.

[036] In some embodiments, the program instructions are executable by the at least one processing unit for balancing the testing dataset through equalization.

[037] In some embodiments, the program instructions are executable by the at least one processing unit for normalizing the set of patient features comprising re-scaling a distribution of each feature to have a value between 0 and 1 .

[038] In some embodiments, the program instructions are executable by the at least one processing unit for normalizing the set of patient features comprising standardizing a distribution of each feature to have zero mean and unit standard deviation. [039] In some embodiments, the program instructions are executable by the at least one processing unit for normalizing the set of patient features to data acquired at a same institution.

[040] In accordance with yet another broad aspect, there is provided a computer readable medium having stored thereon program code executable by a processor for generating a universal radiomics-based predictive model in a medical system. The program code comprise instructions for acquiring one or more medical imaging datasets representing an anatomy of a patient, extracting a set of features from each of the one or more medical imaging datasets, normalizing the set of features to generate one or more normalized datasets, and generating the universal model from the one or more normalized datasets.

[041] In accordance with yet another broad aspect, there is provided a computer readable medium having stored thereon program code executable by a processor for providing a patient outcome from a universal radiomics-based predictive model. The program code comprises instructions for retrieving the universal model from memory, validating the universal model on a balanced and normalized testing dataset, extracting a set of patient features from one or more medical imaging datasets, normalizing the set of patient features to generate a normalized patient dataset, applying the universal model to the normalized patient dataset to predict the patient outcome, and outputting the patient outcome.

[042] Any of the above features may be used together, in any combination. BRIEF DESCRIPTION OF THE DRAWINGS

[043] Further features and advantages of the present invention will become apparent from the following detailed description, taken in combination with the appended drawings, in which:

[044] Figure 1 is a schematic illustration of a medical system, in accordance with an illustrative embodiment;

[045] Figure 2 is a flowchart of an example method for generating universal models, in accordance with an illustrative embodiment; [046] Figure 3 is a flowchart of an example method for predicting patient outcomes, in accordance with an illustrative embodiment;

[047] Figure 4 is a block diagram illustrating an example computing device for generating universal models and predicting patient outcomes from universal models, in accordance with an illustrative embodiment; and

[048] Figure 5 is a block diagram illustrating an example computer program product configured to store instructions for generating universal models and predicting patient outcomes from universal models, in accordance with an illustrative embodiment;

[049] It will be noted that throughout the appended drawings, like features are identified by like reference numerals.

DETAILED DESCRIPTION

[050] This disclosure is drawn, inter alia, to methods, systems, products, devices, and/or apparatus generally related to the generation of a universal model from one or more datasets. Universal models may be used for the purposes of predicting the outcome of a subject from image features. However, it is to be understood that universal models may be used for any other application requiring prediction from data.

[051] Figure 1 is a schematic illustration of a medical system 100 in accordance with at least some embodiments described herein. The various components described in Figure 1 are merely examples, and other variations, including eliminating components, combining components, and substituting components are all contemplated.

[052] Figure 1 shows multiple medical images 105,1 10, and 1 15 which may, for example, have been acquired by modalities such as computed tomography (CT), positron emission tomography (PET), magnetic resonance imaging (MRI), or ultrasound imaging. Each image 105, 1 10, 1 15 shows a patient representation 120, 125, 130 with spatially varying intensities that represent the anatomy and/or function of a patient. In some cases, image modalities may be configured to acquire multiple types of images with the same device. For a liver cancer patient, for example, a magnetic resonance (MR) scanner may be used to acquire T1 -weighted non-enhanced gradient echo images, T2-weighted fast spin echo images, and Dynamic Contrast Enhanced (DCE)-MRI at three (3) time-points for arterial, venous and equilibrium phases, respectively, as well as a delayed post-contrast scan, all with the same device. In another example, for a uterine cancer patient, an MR scanner may be used to acquire a T2-weighted fast spin echo, T1 -weighted fast gradient echo with dynamic gadolinium contrast enhancement, a post-contrast image and a diffusion-weighted MRI image with two b-values and their related maps of the apparent diffusion coefficient.

[053] Regions of interest (ROI) 135, 140, 145, 150, 155 and 160 may be identified on each image 105, 1 10, 1 15 manually by a clinician, or automatically using a suitable algorithm. ROIs 135, 140, 145, 150, 155, 160 may in some cases correspond to anatomical features such as the uterus or the liver, a tumor, or a portion of a tumor such as a periphery, rim, or inner region.

[054] Candidate predictive features may be extracted from medical images 105, 1 10 and 1 15. Example feature extraction algorithms that have been found to be useful in some cases are global texture (e.g., Variance, Skewness, and Kurtosis), Gray-level Co-occurrence Matrix (GLCM) (e.g., Energy, Contrast, Entropy, Homogeneity, Correlation, SumAverage, Variance, Dissimilarity), Gray-level Run Length Matrix (GLRLM) (e.g., Short Run Emphasis (SRE), Long Run Emphasis (LRE), Gray-Level Nonuniformity (GLN), Run-Length Nonuniformity (RLN), Run Percentage (RP), Low Gray-level Run Emphasis (LGRE), High Gray-level Run Emphasis (HGRE), Short Run Low Gray-level Emphasis (SRLGE), Short Run High Gray-level Emphasis (SRHGE), Long Run Low Gray-level Emphasis (LRLGE), Long Run High Gray-level Emphasis (LRHGE), Gray-Level Variance (GLV), Run-Length Variance (RLV)), Gray-level Size Zone Matrix (GLSZM) (e.g., Small Zone Emphasis (SZE), Large Zone Emphasis (LZE), Gray-Level Nonuniformity (GLN), Zone-Size Nonuniformity (ZSN), Zone Percentage (ZP), Low Gray-Level Zone Emphasis (LGZE), High Gray-Level Zone Emphasis (HGZE), Small Zone Low Gray-Level Emphasis (SZLGE), Small Zone High Gray-Level Emphasis (SZHGE), Large Zone Low Gray-Level Emphasis (LZLGE), Large Zone High Gray-Level Emphasis (LZHGE), Gray-Level Variance (GLV), Zone-Size Variance (ZSV)), and Neighborhood Gray-Tone Difference Matrix (NGTDM) (e.g., Coarseness, Contrast, Busyness, Complexity, Strength). Virtually any extraction of features from medical image elements may be contemplated. In some cases the medical image elements may be pixels, voxels, the ROIs 135, 140, 145, 150, 155 and 160, or combinations thereof.

[055] Each medical image 105, 1 10, 1 15, and hence each potential predictive feature, is labelled with one or more patient outcomes. Example outcomes are: lymphovascular space invasion, which is an independent prognostic factor for recurrence and survival among women with intermediate-to-high risk early- stage endometroid endometrial cancer; International Federation of Gynecology and Obstetrics (FIGO) stage; tumor grade; the absence of desmoplasia, i.e., the growth of fibrous or connective tissue around a malignant neoplasm. Outcomes may in some cases be pathologically proven.

[056] Predictive features may depend on the imaging device used, the ROIs 135, 140, 145, 150, 155, 160 selected by the physician, the indication, and the patient population seen at a particular institution, among many other confounding factors. Imaging data collected at a single institution may not be universally applicable to patients at another institution, with a particular combination of imaging device, physician, institution, patient demographic, and geographical location.

[057] In some embodiments, features are pre-extracted from images 105, 1 10 and 1 15 using a third-party workstation (not shown) and transferred manually or automatically to a computing system 160. In other embodiments, the images are transferred directly to the computing system 160. Data may be processed by a processor 165 and stored in a memory 170. Data may be manipulated, edited, or appended by a user via interactions with user input devices 175 such as a keyboard and mouse, and display devices 180 such as a computer monitor. Patient outcomes for each feature and/or image 105, 1 10, 1 15 may be transferred from a third-party system (not shown) to the computing system 160, and/or entered directly into the computing system 160 by the user, and stored in the memory 170.

[058] The processor 165 may be implemented, for example, using one or more central processing units (CPUs), with each CPU having one or more processing cores. The processor 165 may perform tasks using software (e.g., executable instructions) stored in the memory 170, for example. Additionally, the processor 165 may perform random samplings within the dataset, partition data into one or more subgroups, compute correlations between features and outcomes, compute scatter plots, calculate metrics, identify strong, stable features, equalize datasets, normalize datasets, combine datasets, generate universal models, cause universal models to be stored, and cause datasets to be stored. Processing tasks may also be implemented, in some embodiments using one or more graphical processing units (GPUs).

[059] The memory 170 may be generally any electronic storage, including volatile or nonvolatile memory, which may encode instructions for performing functions described herein.

[060] In some embodiments, a user may select a patient dataset and labeled outcome that have been stored in the memory 170 and identify this dataset using user input devices 175. The user may then select a feature set and store it in memory 170. In some implementations, the features may be calculated using the processor 165. The feature set may contain multiple variations for different algorithms used to extract features, imaging sequences and modalities, and ROIs.

[061] In some embodiments, some features may be removed from memory 170 if they are deemed to be not useful based, for example, on one or more statistical techniques known in the art. Features may also be combined to form new features using, for example, algebraic procedures.

[062] Once the feature set has been chosen, the processor 165 may be used to verify whether the dataset is balanced, i.e., whether it contains equal amounts of data corresponding to each potential outcome, or approximately equal amounts within a user-specified threshold. For example, in some clinical datasets, there may be more data for patients that test negative for cancer than those that test positive, resulting in an unbalanced dataset. In such cases, the dataset may be equalized.

[063] In some embodiments, equalizing is performed by randomly removing data until there is an approximately equal amount of data corresponding to each outcome. For example, if the majority of data has a‘negative’ cancer diagnosis, then patients with‘negative’ diagnosis may be randomly removed until the number of positive and negative diagnoses remaining in the dataset is equal.

[064] Once the feature set has been chosen, and equalized if necessary, the processor 165 may be used to normalize the features. Normalized features may subsequently be stored in memory 170. In some embodiments, the data is normalized by re-scaling, i.e., the features are made to have values between 0 and 1. This may be accomplished by finding the maximum value of each feature, the minimum value of each feature, and for each feature subtracting its minimum followed by dividing by the difference between its maximum and minimum.

[065] In some preferred embodiments, the features are normalized by standardization, i.e., the feature distribution is made to have zero mean and unit standard deviation. This may be accomplished by finding the mean and standard deviation of each feature, and for each feature subtracting its mean followed by dividing by the standard deviation. Standardization is particularly robust to outliers in the data.

[066] In some embodiments, normalized data may be directly combined with a machine learning tool to generate a universal model. In other embodiments, normalized data from a single institution may not be sufficiently robust. In the latter case, multiple datasets from different institutions may be combined into a combined dataset, for example by concatenating the normalized features of each clinical dataset. In some preferred embodiments, the combined dataset may itself be equalized and normalized. A universal model may be generated from the combined dataset. The universal model may be stored in memory 170 and/or displayed on display devices 180.

[067] A universal model may be used to predict outcomes from normalized features, whether the features have been collected from a patient at an institution used to collect the training data, or at a different institution. In some preferred embodiments, patient features are normalized using a clinical dataset that has been obtained under the same or similar situation as the patient - for example, at the same institution, using the same scanner, and using the same protocol. In this manner, the normalization parameters such as mean and standard deviation are relevant to the patient and provide robust normalization. [068] In some embodiments, prior to using a universal model at a new institution, the universal model may first be validated using a testing data set. If the universal model is able to correctly predict outcomes from features in the testing dataset, within acceptability criteria agreed upon by the clinicians, then the universal model may be safely used at the institution.

[069] Figure 2 is an example method 200 for generating a universal model from a clinical dataset in accordance with at least some embodiments of the present disclosure. The operations described in the blocks 205 through 230 may be performed in response to execution (such as by one or more processors described herein) of computer-executable instructions stored in a computer-readable medium, such as a computer-readable medium of a computing device or some other controller similarly configured.

[070] An example process may begin with block 205, where one or more preliminary clinical datasets are defined. These may, for example, have been generated at different institutions, or at the same institution with different equipment or protocols, and may not be directly combined with any degree of predictive accuracy. Clinical datasets may contain features and corresponding labeled outcomes. In some embodiments, the features may be defined from medical images, using for example various types of texture analyses in the image or, in some cases, in regions of interest defined in the images. In some embodiments, features may be further processed, including excluding unstable features or combining multiple features into single features.

[071] Block 205 may be followed by a verification step performed at block 207 to establish whether an equalization operation is necessary. The verification step may comprise determining whether there are approximately an equal number of classes of outcomes in each dataset. If it is determined at block 207 that an equalization operation is necessary, the method 200 flows to block 210, where the clinical datasets are equalized. Datasets that are not balanced may in some embodiments be equalized by randomly removing features from the majority class until there is an approximately equal amount of data from each class. If it is determined at block 207 that an equalization operation is not necessary, the method 200 flows directly to block 215. [072] At block 215, the clinical datasets may be normalized. In some embodiments, this may include rescaling, where the distribution of each feature is scaled to be between 0 and 1 . In some preferred embodiments normalization may include standardization, where the distribution of each feature is standardized to have zero mean and unit standard deviation.

[073] Block 215 may be followed by block 220, where normalized clinical datasets may be combined through concatenation to generate a combined dataset. In some embodiments, features from the combined dataset may be further excluded or combined, normalized, or equalized.

[074] Block 220 may be followed by block 225, where a universal model may be generated from the combined clinical dataset. This may include selecting a machine learning algorithm which is able to successfully relate normalized features in the combined clinical dataset with outcomes. A universal model may include the combined clinical dataset paired with a selected machine learning algorithm.

[075] Block 225 may be followed by block 230, where the universal model is stored in computer memory.

[076] Figure 3 is an example method 300 for predicting a patient outcome from a universal model in accordance with at least some embodiments of the present disclosure. The operations described in the blocks 305 through 325 may be performed in response to execution (such as by one or more processors described herein) of computer-executable instructions stored in a computer-readable medium, such as a computer-readable medium of a computing device or some other controller similarly configured.

[077] An example process may begin with block 305, where a universal model may be loaded from memory. The universal model may have been previously generated using the methods described herein.

[078] Block 305 may be followed by block 310, where the universal model is tested on a testing dataset to ensure that it is applicable to the institution and/or other conditions under study. If the test fails, as determined at block 312 , blocks 315-325 may not be executed and the method 300 ends.

[079] Otherwise, if it is determined at block 312 that the test was successful, block 310 may be followed by block 315, where patient features for a particular patient under study are defined. These may include patient images, or features extracted from patient images. Features are then normalized, for example using re-scaling or standardization. In one embodiment, the normalization parameters (e.g. minimum and maximum for rescaling, mean, and standard deviation) are obtained from the testing dataset.

[080] Block 315 may be followed by block 320, where a patient outcome is predicted by applying the universal model to the normalized patient features.

[081] Block 320 may be followed by block 325, where the patient outcome may be displayed to a clinician to assist in determining a diagnosis or prognosis for the patient.

[082] The blocks included in Figures 2 and 3 are for illustration purposes. In some embodiments, the blocks may be performed in a different order. In some other embodiments, various blocks may be eliminated. In still other embodiments, various blocks may be divided into additional blocks, supplemented with other blocks, or combined together into fewer blocks. Other variations of these specific blocks are contemplated, including changes in the order of the blocks, changes in the content of the blocks being split or combined into other blocks, and the like.

[083] Figure 4 is a block diagram illustrating an example embodiment of a computing device 400 that is arranged for generating predictive features in accordance with the present disclosure. In some embodiments, computing device 400 includes one or more processors 410 and system memory 420 comprised in a base module 401. A memory bus 430 may be used for communicating between the processor 410 and the system memory 420.

[084] Depending on the desired configuration, processor 410 may be of any type including but not limited to a microprocessor (mR), a microcontroller (pC), a digital signal processor (DSP), or any combination thereof. Processor 410 may include one more levels of caching, such as a level one cache 41 1 and a level two cache 412, a processor core 413, and registers 414. An example processor core 413 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller 415 may also be used with the processor 410, or in some implementations the memory controller 415 may be an internal part of the processor 410.

[085] Depending on the desired configuration, the system memory 420 may be of any type including, but not limited, to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. System memory 420 may include an operating system 421 , one or more applications 422, and program data 424. Application(s) 422 may include a universal model procedure 423 that is arranged to provide a universal model as described herein. Program data 424 may include a universal model 425, which may comprise one or more medical images, features, outcomes, ROIs, machine learning algorithms and/or other information useful for predicting outcomes. In some embodiments, application(s) 422 may be arranged to operate with program data 424 on an operating system 421 such that any of the procedures described herein may be performed. This described configuration is illustrated in Figure 4 by those components within the base module 401.

[086] Computing device 400 may have additional features or functionality, and additional interfaces to facilitate communications between the base module 401 and any other devices and interfaces. For example, a bus/interface controller 440 may be used to facilitate communications between the base module 401 and one or more storage devices 450 via a storage interface bus 441. The storage devices 450 may be removable storage devices 451 , non-removable storage devices 452, or a combination thereof. Examples of removable storage and non-removable storage devices comprise magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.

[087] System memory 420, removable storage 451 and non-removable storage 452 are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, Electrically-Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc (CD)-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by computing device 400. Any such computer storage media may be part of computing device 400.

[088] Computing device 400 may also include an interface bus 442 for facilitating communication from various interface devices (e.g., output interfaces, peripheral interfaces, and communication interfaces) to the base module 401 via the bus/interface controller 440. Example output devices 460 include a graphics processing unit 461 and an audio processing unit 462, which may be configured to communicate to various external devices such as a display or speakers via one or more audio/visual (A/V) ports 463. Example peripheral interfaces 470 comprise a serial interface controller 471 or a parallel interface controller 472, which may be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 473. An example communication device 480 comprises a network controller 481 , which may be arranged to facilitate communications with one or more other computing devices 490 over a network communication link via one or more communication ports 482.

[089] The network communication link may be one example of a communication media. Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A modulated data signal may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media.

[090] Computing device 400 may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that includes any of the above functions. Computing device 400 may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.

[091] Figure 5 is a block diagram illustrating an example computer program product 500 that is arranged to store instructions for generating universal models and predicting patient outcomes from universal models, in accordance with the present disclosure. The signal bearing medium 502 which may be implemented as or include a computer-readable medium 506, a computer recordable medium 508, a computer communications medium 510, or combinations thereof, stores programming instructions 504 that may configure the processing unit to perform all or some of the processes previously described. These instructions may include, for example, equalize clinical datasets, normalize clinical datasets, combine normalized clinical datasets, generate universal models, store universal models, test universal models, and predict patient outcomes.

[092] The present disclosure is not to be limited in terms of the particular examples described herein, which are intended as illustrations of various aspects. Many modifications and examples can be made, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and examples are intended to fall within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled.

[093] It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as“having at least,” the term“includes” should be interpreted as “includes but is not limited to,” etc.).

[094] It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases“at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to examples containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or“at least one” and indefinite articles such as“a” or“an” (e.g., “a” and/or“an” should be interpreted to mean“at least one” or“one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of“two recitations,” without other modifiers, means at least two recitations, or two or more recitations).

[095] Furthermore, in those instances where a convention analogous to“at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g.,“a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to“at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g.,“a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase“A or B” will be understood to include the possibilities of“A” or“B” or“A and B.”

[096] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[097] As will be understood by one skilled in the art, for any and all purposes, such as in terms of providing a written description, all ranges disclosed herein also encompass any and all possible subranges and combinations of subranges thereof. Any listed range can be easily recognized as sufficiently describing and enabling the same range being broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third and upper third, etc. As will also be understood by one skilled in the art all language such as“up to,”“at least,”“greater than,”“less than,” and the like include the number recited and refer to ranges which can be subsequently broken down into subranges as discussed above. Finally, as will be understood by one skilled in the art, a range includes each individual member. Thus, for example, a group having 1 -3 items refers to groups having 1 , 2, or 3 items. Similarly, a group having 1-5 items refers to groups having 1 , 2, 3, 4, or 5 items, and so forth.

[098] While the foregoing detailed description has set forth various examples of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples, such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, or examples can be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In one example, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, those skilled in the art will recognize that some aspects of the examples disclosed herein, in whole or in part, can be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. For example, if a user determines that speed and accuracy are paramount, the user may opt for a mainly hardware and/or firmware vehicle; if flexibility is paramount, the user may opt for a mainly software implementation; or, yet again alternatively, the user may opt for some combination of hardware, software, and/or firmware.

[099] In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein are capable of being distributed as a program product in a variety of forms, and that an illustrative example of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, a computer memory, etc.; and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).

[0100]Those skilled in the art will recognize that it is common within the art to describe devices and/or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and/or processes into data processing systems. That is, at least a portion of the devices and/or processes described herein can be integrated into a data processing system via a reasonable amount of experimentation. Those having skill in the art will recognize that a typical data processing system generally includes one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing/communication and/or network computing/communication systems.

[0101]The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures can be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being "operably connected", or "operably coupled", to each other to achieve the desired functionality, and any two components capable of being so associated can also be viewed as being "operably couplable", to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.

[0102]While various aspects and examples have been disclosed herein, other aspects and examples will be apparent to those skilled in the art. The various aspects and examples disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

CLAIMS:

1. A method for generating a universal radiomics-based predictive model in a medical system, the universal model used for predicting a patient outcome, the method comprising:

acquiring, at a processor, one or more medical imaging datasets representing an anatomy of a patient;

extracting, by the processor, a set of features from each of the one or more medical imaging datasets;

normalizing, by the processor, the set of features to generate one or more normalized datasets; and

generating, by the processor, the universal model from the one or more normalized datasets.

2. The method of claim 1 , wherein acquiring the one or more medical imaging datasets comprises acquiring one of computed tomography data, positron emission tomography data, magnetic resonance data, and ultrasound data.

3. The method of claim 1 or 2, further comprising identifying, from the one or more medical imaging datasets, at least one region of interest, wherein the set of features is extracted by performing one or more texture analyses in the at least one region of interest.

4. The method of any one of claims 1 to 3, further comprising assessing whether the one or more medical imaging datasets are balanced and equalizing the one or more medical imaging datasets if the one or more medical imaging datasets are unbalanced.

5. The method of claim 4, wherein equalizing the one or more medical imaging datasets comprises randomly removing data from a majority class of the one or more medical imaging datasets until the one or more medical imaging datasets are balanced.

6. The method of any one of claims 1 to 5, wherein normalizing the set of features comprises re-scaling a distribution of each feature to have a value between 0 and 1.

7. The method of any one of claims 1 to 6, wherein normalizing the set of features comprises standardizing a distribution of each feature to have zero mean and unit standard deviation.

8. The method of any one of claims 1 to 7, further comprising combining at least two of the one or more normalized datasets into a combined dataset, wherein the universal model is generated from the combined dataset.

9. The method of claim 8, wherein the at least two normalized datasets are combined through concatenation.

10. The method of claim 8, further comprising equalizing the combined dataset.

1 1. The method of claim 8, further comprising normalizing the combined dataset.

12. The method of any one of claims 1 to 1 1 , wherein the universal model is generated using a machine learning tool configured to correlate the normalized set of features in the combined dataset with one or more patient outcomes and to predict each patient outcome accordingly.

13. A method for providing a patient outcome from a universal radiomics- based predictive model, the method comprising:

retrieving, by a processor, the universal model from memory;

validating the universal model on a balanced and normalized testing dataset;

extracting, by the processor, a set of patient features from one or more medical imaging datasets;

normalizing, by the processor, the set of patient features to generate a normalized patient dataset;

applying, by the processor, the universal model to the normalized patient dataset to predict the patient outcome; and

outputting, by the processor, the patient outcome.

14. The method of claim 13, wherein the testing dataset is balanced through equalization.

15. The method of claim 13 or 14, wherein normalizing the set of patient features comprises re-scaling a distribution of each feature to have a value between 0 and 1.

16. The method of claim 13 or 14, wherein normalizing the set of patient features comprises standardizing a distribution of each feature to have zero mean and unit standard deviation.

17. The method of any one of claims 13 to 16, wherein the set of patient features is normalized to data acquired at a same institution.

18. A system for generating a universal radiomics-based predictive model in a medical system, the system comprising:

at least one processing unit; and

at least one non-transitory computer-readable memory having stored thereon program instructions executable by the at least one processing unit for:

acquiring one or more medical imaging datasets representing an anatomy of a patient,

extracting a set of features from each of the one or more medical imaging datasets,

normalizing the set of features to generate one or more normalized datasets, and

generating the universal model from the one or more normalized datasets.

19. The system of claim 18, wherein the program instructions are executable by the at least one processing unit for acquiring the one or more medical imaging datasets comprising acquiring one of computed tomography data, positron emission tomography data, magnetic resonance data, and ultrasound data.

20. The system of claim 18 or 19, wherein the program instructions are further executable by the at least one processing unit for identifying, from the one or more medical imaging datasets, at least one region of interest, and further wherein the program instructions are executable by the at least one processing unit for extracting the set of features by performing one or more texture analyses in the at least one region of interest.

21. The system of any one of claims 18 to 20, wherein the program instructions are further executable by the at least one processing unit for assessing whether the one or more medical imaging datasets are balanced and equalizing the one or more medical imaging datasets if the one or more medical imaging datasets are unbalanced.

22. The system of claim 21 , wherein the program instructions are executable by the at least one processing unit for equalizing the one or more medical imaging datasets comprising randomly removing data from a majority class of the one or more medical imaging datasets until the one or more medical imaging datasets are balanced.

23. The system of any one of claims 18 to 22, wherein the program instructions are executable by the at least one processing unit for normalizing the set of features comprising re-scaling a distribution of each feature to have a value between 0 and 1.

24. The system of any one of claims 18 to 23, wherein the program instructions are executable by the at least one processing unit for normalizing the set of features comprising standardizing a distribution of each feature to have zero mean and unit standard deviation.

25. The system of any one of claims 18 to 24, wherein the program instructions are further executable by the at least one processing unit for combining at least two of the one or more normalized datasets into a combined dataset, and further wherein the program instructions are executable by the at least one processing unit for generating the universal model from the combined dataset.

26. The system of claim 25, wherein the program instructions are executable by the at least one processing unit for combining the at least two normalized datasets through concatenation.

27. The system of claim 25, wherein the program instructions are further executable by the at least one processing unit for equalizing the combined dataset.

28. The system of claim 25, wherein the program instructions are further executable by the at least one processing unit for normalizing the combined dataset.

29. The system of any one of claims 18 to 28, wherein the program instructions are executable by the at least one processing unit for generating the universal model using a machine learning tool configured to correlate the normalized set of features in the combined dataset with one or more patient outcomes and to predict each patient outcome accordingly.

30. A system for providing a patient outcome from a universal radiomics- based predictive model, the system comprising:

at least one processing unit; and

retrieving the universal model from memory,

validating the universal model on a balanced and normalized testing dataset,

extracting a set of patient features from one or more medical imaging datasets,

normalizing the set of patient features to generate a normalized patient dataset,

applying the universal model to the normalized patient dataset to predict the patient outcome, and

outputting the patient outcome.

31. The system of claim 30, wherein the program instructions are executable by the at least one processing unit for balancing the testing dataset through equalization.

32. The system of claim 30 or 31 , wherein the program instructions are executable by the at least one processing unit for normalizing the set of patient features comprising re-scaling a distribution of each feature to have a value between 0 and 1.

33. The system of any one of claims 30 to 32, wherein the program instructions are executable by the at least one processing unit for normalizing the set of patient features comprising standardizing a distribution of each feature to have zero mean and unit standard deviation.

34. The system of any one of claims 30 to 33, wherein the program instructions are executable by the at least one processing unit for normalizing the set of patient features to data acquired at a same institution.

35. A computer readable medium having stored thereon program code executable by a processor for generating a universal radiomics-based predictive model in a medical system, the program code comprising instructions for:

acquiring one or more medical imaging datasets representing an anatomy of a patient;

extracting a set of features from each of the one or more medical imaging datasets;

normalizing the set of features to generate one or more normalized datasets; and

generating the universal model from the one or more normalized datasets.

36. A computer readable medium having stored thereon program code executable by a processor for providing a patient outcome from a universal radiomics-based predictive model, the program code comprising instructions for:

retrieving the universal model from memory;

validating the universal model on a balanced and normalized testing dataset;

extracting a set of patient features from one or more medical imaging datasets;

normalizing the set of patient features to generate a normalized patient dataset;

applying the universal model to the normalized patient dataset to predict the patient outcome; and

outputting the patient outcome.