WO2015048103A1 - Early therapy response assessment of lesions - Google Patents
Early therapy response assessment of lesions Download PDFInfo
- Publication number
- WO2015048103A1 WO2015048103A1 PCT/US2014/057175 US2014057175W WO2015048103A1 WO 2015048103 A1 WO2015048103 A1 WO 2015048103A1 US 2014057175 W US2014057175 W US 2014057175W WO 2015048103 A1 WO2015048103 A1 WO 2015048103A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- therapy
- convolving
- features
- images
- texture
- Prior art date
Links
- 238000002560 therapeutic procedure Methods 0.000 title claims abstract description 125
- 230000004044 response Effects 0.000 title claims abstract description 60
- 230000003902 lesion Effects 0.000 title claims description 78
- 238000012549 training Methods 0.000 claims abstract description 65
- 238000004458 analytical method Methods 0.000 claims abstract description 26
- 238000000034 method Methods 0.000 claims description 41
- 238000009258 post-therapy Methods 0.000 claims description 31
- 238000003384 imaging method Methods 0.000 claims description 13
- 238000002591 computed tomography Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 10
- 238000012706 support-vector machine Methods 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 5
- 238000007637 random forest analysis Methods 0.000 claims description 4
- 230000002087 whitening effect Effects 0.000 claims description 2
- 238000012935 Averaging Methods 0.000 claims 1
- 238000010801 machine learning Methods 0.000 abstract description 6
- 230000015654 memory Effects 0.000 description 16
- 238000002059 diagnostic imaging Methods 0.000 description 12
- 210000001519 tissue Anatomy 0.000 description 10
- 206010028980 Neoplasm Diseases 0.000 description 9
- 210000000056 organ Anatomy 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000013459 approach Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000012880 independent component analysis Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 229940079593 drug Drugs 0.000 description 3
- 239000003814 drug Substances 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- ZCYVEMRRCGMTRW-UHFFFAOYSA-N 7553-56-2 Chemical compound [I] ZCYVEMRRCGMTRW-UHFFFAOYSA-N 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 229910052740 iodine Inorganic materials 0.000 description 2
- 239000011630 iodine Substances 0.000 description 2
- 210000004185 liver Anatomy 0.000 description 2
- 238000002483 medication Methods 0.000 description 2
- 230000017074 necrotic cell death Effects 0.000 description 2
- 230000002956 necrotizing effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000001959 radiotherapy Methods 0.000 description 2
- 238000011282 treatment Methods 0.000 description 2
- 238000002604 ultrasonography Methods 0.000 description 2
- 206010011732 Cyst Diseases 0.000 description 1
- 208000032843 Hemorrhage Diseases 0.000 description 1
- 206010019695 Hepatic neoplasm Diseases 0.000 description 1
- 238000002583 angiography Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 238000002512 chemotherapy Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 239000002872 contrast media Substances 0.000 description 1
- 208000031513 cyst Diseases 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000002594 fluoroscopy Methods 0.000 description 1
- 208000014018 liver neoplasm Diseases 0.000 description 1
- 210000004072 lung Anatomy 0.000 description 1
- 210000001165 lymph node Anatomy 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001338 necrotic effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011275 oncology therapy Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000002600 positron emission tomography Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000002250 progressing effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000002603 single-photon emission computed tomography Methods 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000002626 targeted therapy Methods 0.000 description 1
- 210000005166 vasculature Anatomy 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
- A61B6/03—Computed tomography [CT]
- A61B6/032—Transmission computed tomography [CT]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
- A61B6/5217—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data extracting a diagnostic or physiological parameter from medical diagnostic data
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
- A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
- A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
- A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2134—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
- G06T7/42—Analysis of texture based on statistical description of texture using transform domain methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/76—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7753—Incorporation of unlabelled data, e.g. multiple instance learning [MIL]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2576/00—Medical imaging apparatus involving image processing or analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/42—Detecting, measuring or recording for evaluating the gastrointestinal, the endocrine or the exocrine systems
- A61B5/4222—Evaluating particular parts, e.g. particular organs
- A61B5/4244—Evaluating particular parts, e.g. particular organs liver
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4848—Monitoring or testing the effects of treatment, e.g. of medication
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7271—Specific aspects of physiological measurement analysis
- A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
- G06T2207/10136—3D ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
- G06T2207/20012—Locally adaptive
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20064—Wavelet transform [DWT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30056—Liver; Hepatic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
Definitions
- the present embodiments relate to early therapy response assessment.
- computer-aided response assessment is provided.
- the response of a tumor under therapy is measured based on geometric measures, such as the diameter (e.g., RECIST or WHO criteria) or volume.
- geometric measures such as the diameter (e.g., RECIST or WHO criteria) or volume.
- a clinician monitors the geometric measures in radiological scans (e.g. CT or MR!) for a long period of time until a decision can be made about outcome.
- Radiological scans e.g. CT or MR
- Many rounds of therapy are performed before the clinician makes a decision based on the geometric measures about whether the therapy is likely to succeed or not. Long periods of therapy may result in higher dosage for the patient (e.g., radiation therapy and/or x-ray imaging for the geometric measures) and greater cost of the therapy drugs.
- the lesion appearance (e.g. enhancement pattern) changes earlier than geometric changes of the lesion occur.
- the lesion may maintain its size for a longer time while its tissue is already turning necrotic.
- the texture of a lesion gives insight into the therapy response at a much earlier stage, when the size or shape of the lesion is still largely unaffected.
- texture it may be possible to identify cases with a similar medical condition in a database for which the applied therapy and its outcome are known. This may help the doctor in estimating the effectiveness of different therapies and chose the best strategy.
- texture as a parameter
- Texture features are input for machine learning a classifier and for using a machine learnt classifier.
- data driven texture features are derived from training images.
- Such data driven texture features are independent analysis features, such as features from independent subspace analysis. The texture features may be used to predict the outcome of therapy based on a few number of or even one scan of the patient.
- a method for therapy response assessment.
- Pre-therapy and post therapy medical images of a patient are obtained.
- the medical images represent at least one lesion of the patient.
- a processor convolves the pre-therapy and post therapy medical images with a texture feature learned from training images.
- the processor classifies a therapy response of the lesion with a machine-learnt classifier with a result of the convolving as an input feature to the machine-learnt classifier.
- a non-transitory computer readable storage medium has stored therein data representing instructions executable by a programmed processor for therapy response assessment.
- the storage medium includes instructions for: with only one or two scans, extracting texture features for a lesion with a filter kernel, the filter kernel being independently based on image data; and predicting an outcome of therapy on the lesion, the predicting being a function of the texture features.
- a processor subjects patches of lesions represented in a plurality of training frames of data to independent subspace analysis.
- the processor creates a number of image filters from the independent subspace analysis of the patches and calculates texture features by application of the image filters to the lesions as represented in the training frames of data.
- the processor learns a predictor of therapy response as a function of the texture features.
- Figure 1 shows an example process for therapy response assessment using geometric measures
- Figure 2 shows an example process for therapy response assessment using texture features
- Figure 3 is a flow chart diagram of one embodiment of a method for training in therapy response assessment
- Figure 4 illustrates example texture features learned from training images
- Figures 5A-C show results of convolving the features of Figure 4 with three different lesions
- Figure 6 is a flow chart diagram of one embodiment of a method for application in therapy response assessment.
- Figure 7 is a block diagram of one embodiment of a system for therapy response assessment.
- a computer assists a doctor in estimating the effectiveness of different cancer therapies and choosing the best strategy. Given the variability of lesions, imaging settings, situations, and number of cases, it may take a doctor or other medical professional an unreasonable amount of time to identify similar cases and patients for making an informed prediction. As a result, there may be a great amount of variability in predictions by different medical professionals. A computer may more quickly access the information and more consistently predict therapy response. Given the subtlety of prediction from, at least in part, medical images, a processor may better perform or at least provide a more efficient second perspective on therapy response assessment.
- Figure 1 shows an approach using geometric measures where a pre-therapy scan is performed and then a sequence of N (e.g., 5 or more) scans are performed after therapy or during a sequence of therapy
- Figure 2 shows acquiring the pre-therapy scan and a single post therapy scan. Texture features reflecting cell density, vasculature, necrosis, hemorrhage, other
- the prediction may be made earlier in the process, possibly avoiding further therapy where the outcome is likely negative.
- the prediction may be made based on the pre-therapy scan before therapy. Early therapy outcome prediction is provided, possibly reducing exposure of a patient to therapy that is likely not successful.
- the texture of organs and/or lesions is used to predict the likelihood of future cancer development.
- the features are extracted using Independent Subspace Analysis (ISA) or other data-driven approach.
- ISA Independent Subspace Analysis
- training scans are used to determine texture that is appropriate for and/or determinative of outcome for a given type of lesion, organ, therapy, and/or other situation. This is different from traditional features where a fixed formula describes how to calculate the feature value.
- ISA or other data-driven development of the texture features different sets of training images and ground truth labels may generate different image filters, and hence different feature values.
- the advantage of such a data- driven approach is that, as the training dataset grows larger, more meaningful patterns of texture may be discovered without relying on expert knowledge to manually define heuristic and sub-optimal mathematical formulas for patterns.
- the learning-based approach also allows training the system for different therapy types (e.g. chemo-therapy. targeted therapy, radiation therapy) resulting in therapy-specific response features that are automatically learnt.
- Data-driven texture features allow for different features to be learned for different types of lesions, tissues, therapy, and/or imaging modalities.
- Figures 3 and 6 show methods for therapy response assessment.
- the method for therapy response assessment may be a method to learn how to access outcome of therapy or may be a method for assessing the outcome with a learnt predictor.
- Figure 3 is directed to machine training of the therapy outcome predictor.
- Figure 8 is directed to application of a machine-learnt therapy outcome predictor.
- a machine such as a processor, computer, or server, implements some or all of the acts.
- the system of Figure 7 implements the methods in one embodiment.
- a user may select the image files for application of the therapy response predictor by the processor, select the image from which to learn features by a processor, and/or identify a region of interest.
- act 44 of Figure 3 is not provided.
- acts 60 and/or 64 of Figure 6 are not provided.
- acts for capturing images and/or acts using detected information are provided. Acts for configuring, input, or output may be provided.
- Figure 3 shows a method for learning in therapy response assessment.
- a processor performs the learning, such as by performing acts 44-52.
- the processor both creates texture features from training data in acts 46-48 and learns a predictor with the texture features as inputs in act 52.
- the learnt feature may be used by the processor to calculate a feature value and/or by the processor to train a classifier.
- act 40 one or more images of an object are obtained.
- the images are obtained by data transfer, capture, and/or loading from memory.
- the images are frames of data representing a patient at given times.
- the images are scalar values or display values (e.g., red green blue (RGB)).
- the images may have been previously displayed or have not yet been displayed.
- the images are acquired from a scan.
- the images are captured using any one or more sensors.
- images of organs are captured using x-ray, computed tomography (CT), fluoroscopy, angiography, magnetic resonance, ultrasound, positron emission tomography, or single photon emission computed tomography.
- CT computed tomography
- a given pre or post therapy scan provides one or more images.
- images for different phases of a multiple phase contrast enhanced scan e.g., native, arterial, and venous phases of contrast agent wash in and/or wash out
- Dual Energy CT images including an iodine map, are acquired as part of a scan.
- the pre and post therapy scans are the same or different types of scans. Each scan provides one or multiple images. Multiple images of the same or different patients use the same or different imaging modality with the same or different settings (e.g., field of view).
- the object of interest in a medical image may be an organ, a cyst, a tumor, caicification or other anomaly. Any type of lesion may be imaged.
- the organ surrounding the lesion may be scanned.
- the image may represent healthy tissue.
- the image may represent the lesion in two or three- dimensions. For example, the image may be of pixels or voxels.
- images are acquired for the same type of lesion with or without other common arrangements (e.g., patient characteristics (male vs. female or age), imaging modality, organ, type of therapy, and/or stage).
- Pre and post therapy images for many, such as tens, hundreds, thousands, or more, patients are obtained.
- the images are associated with or linked to known outcomes. For example, all of the images are labeled with ground truth.
- just the images associated with positive outcome are used. Alternatively, just images associated with negative outcome or both positive and negative outcomes are used.
- images labeled for both positive and negative outcome are used.
- one or more regions of interest are identified in each of the images.
- Manual e.g., user input
- automated tumor detection or segmentation is used to find the region of interest. Any now known or later developed region of interest identification may be used.
- the region of interest is a bounding box or other shape enclosing the lesion or is a segmentation specifically of the lesson. More than one region of interest may be provided on a given image. For example, multiple lesions are represented in a given image. As another example, one or more regions of interest for healthy tissue are identified. In one embodiment, nested regions of interest are provided, such as one for an organ as a whole or in part and a sub-set of that region being a region for a lesion.
- the region of interest is to be used for feature extraction.
- Tumor associated features may be extracted from within the region of interest.
- the approach is to combine multiple sets of different features for training a classifier.
- Such features may also be extracted from other reference regions in the image data, such as the whole liver parenchyma, to complement the tumor features extracted from within the lesion specific ROI and to provide the classifier with additional information.
- the delineated lesion region of interest may be expanded to assure that the border or gradient from lesion to healthy tissue is included.
- the region of interest is dilated (enlarged) by the processor to include surrounding information.
- the dilation is by a percentage (e.g., 5-10% expansion), by a number of samples (e.g., adding 2-10 samples from a center), or by another process.
- the region of interest is originally defined by the user or processor to include the
- the region of interest may be sub-divided.
- a moving window is defined to cover a part of the region of interest.
- any step size e.g., 1 pixel, 2 pixels, or other distance
- overlapping sub-sets of the region of interest are defined.
- These patches or sub-sets are uniformly defined within the region of interest with a fixed step size.
- Any size of patch may be used, such as 8x8 pixels.
- the patch size and/or step size vary or are not uniform.
- the patches are whitened by the processor. Within each kernel or patch, the values are altered to set the mean to a zero value and make the variance equal to one. Other normalization may be used. In one embodiment, the patches are whitened with a principal component analysis. Alternatively, whitening is not performed.
- ISA independent subspace analysis
- ICA independent component analysis
- the independent analysis examines the data of the patches to find text patterns.
- the processor analyzes many or all of the patches of many or all of the regions of interests from the different patients to learn one or more texture features. Data-driven machine learning is used by the processor to recognize one or more patterns common across various patches and patients. Both ISA and ICA find features that form independent subspaces, but ISA is found to be more robust to local variations than ICA.
- the patches are subjected to independent subspace analysis (ISA) by the processor.
- the patches are subjected to independent component analysis by the processor. The patches are decomposed into different components to find patterns.
- the independent analysis or other pattern recognition by the processor from the many examples results in a number, N, of image filters, in act 48, the image fiiters are created by the application of the independent analysis in act 46. Any number of common patterns in the various training examples may be created. For example, five image filters are learned from the training images.
- ground truth is used.
- the training data associated with desired outcome, as indicated by the ground truth, is used for creating the image filters.
- training data is used.
- other training data is used.
- the independent analysis learns texture features that distinguish between texture with desired outcome from texture with an undesired outcome, so the training data is divided into sets based on the ground truth.
- the image filters are created without consideration of ground truth. Texture common to both necrotizing and non-necrotizing lesions are found by computer implemented analysis.
- the image filters have a same size and shape as the patches. Alternatively, the image filters have a different size and/or shape as the patches.
- the image filters are binary masks or include any number of gradations, such as including an average or full dynamic range of the patches from the training data.
- Figure 4 shows five example image filters learned by a processor from pattern recognition using independent subspace analysis.
- the image filters were created in act 48 from CT images of liver tumors of patients.
- image filters represent more common patterns across the set of lesions for the training data. The same, different, or no image filters are created for healthy tissue. Given different training data, different image filters may result.
- texture features are calculated by the processor.
- the image filters (see Figure 4 for example) represent texture features for a given situation, but are not themselves the inputs for learning a predictor.
- the image filters are applied to the training data to calculate values for the texture features.
- the image filters are applied to the same training data from which the image filters were created. Alternatively, a different set of training data is used.
- the image filters are applied to the appropriate regions of interest.
- the image filters are applied to lesions as represented in the training data.
- the image filters are applied to the regions of interest for that combination.
- the image filters are applied by spatial filtering.
- the image filters are one, two, or three-dimensional filters.
- the N filters are applied to each tumor by a convolution operation (e.g., dot product or other similarity measure).
- the image filter is positioned relative to the legion and the intensities of the image filter are multiplied with the data of the image in the patch of the region of interest. A sum or average of the resulting convolution operation (e.g., dot product or other similarity measure).
- multiplications provides a value for the image filter at that position. By shifting the image filter, such as was done for the patches, values for different locations are determined. Any convolution of the image filters with the lesions as represented in the training frames of data may be used.
- Figures 5A-C show convolution results of the image filters of Figure 4 with three different lesions.
- the left to right sequence of texture features represented by the image filters of Figure 4 is maintained in Figures 5A-C.
- the left most CT slice image is of the lesion being filtered. Where the lesion includes texture more closely resembling the image filter, the resulting pixel has a higher intensity (e.g., more white).
- the differences in scale between Figures 5A-C are due to differences in scale and/or size of the lesion.
- the resulting values (i.e., filter responses) from the convolution are combined to indicate a feature value for that lesion based on the given image filter.
- the intensity values are summed or averaged, in the example of Figure 5A, the average intensity for each of the texture features is calculated, providing a single value for each image filter.
- the average response per filter per tumor is recorded as a value of the ISA feature.
- a feature is calculated from the results of the
- the resulting values are the texture features as used for the training in act 52.
- Other information may be used as part of the input feature vector for training.
- the other information may represent the patient (e.g., age, sex, or family history), stage of the lesion, type of therapy, other situational information, and/or other image derived features.
- texture features are determined from the training data.
- Haralick texture features such as contrast, entropy, and/or others, are used, in another example or in addition, homogeneity, energy, and dissimilarity texture features are used.
- the homogeneity for example, has the following equation:
- g is the number of gray levels in the image.
- the texture features rely on predetermined formulas rather than features learned from training data by a processor. Any now known or later developed texture features may be used.
- texture features based on local binary patterns are used.
- the local binary patterns compare the intensity of each pixel with its neighboring pixels and return a code that compactly summarizes the differences. These codes are then summarized over the ROi through a histogram.
- a data pyramid may be used, where the same image at different resolutions (e.g., by decimation) is provided.
- the local binary patterns are then calculated for each resolution or level of the pyramid, as each pyramid level contains textura! information in different scale and details.
- the texture features are calculated from the training data for one pre-therapy scan and one post therapy scan. In alternative embodiments, only pre-therapy scan, only the first post therapy scan, or additional post therapy scans are used.
- texture features are the same for pre and post therapy scans.
- different features are provide for different scans relative to the therapy (i.e., !earn texture features for pre-therapy images different than texture features learned from post therapy images).
- the ground truth for therapy response is used to train, by a processor, the predictor of the therapy response.
- the ground truth is binary, such as successful or not successful outcome from therapy.
- the ground truth has a greater resolution, such as the number of months without re-occurrence, time of remission, change in size, time to narcotization, or other measure of success of therapy.
- the processor uses the ground truth and data-driven texture features, with or without other features to train a classifier.
- Any machine learning may be used, such as a probabilistic boosting tree, support vector machine, or other machine learning classifier.
- Other classifiers may include a single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, or combinations thereof may be used.
- Multi-class classifiers include CART,
- neural network e.g., multi-layer perception ⁇ , mixture models, or others.
- Error-correcting output code may be used.
- the features are fed into a machine learning classifier or regressor, such as a Support Vector Machine (SVM) or
- SVM Support Vector Machine
- the training provides a matrix associating input values of features to outputs or predictions.
- the matrix or trained classifier e.g., learnt classifier
- the predictor is a predictor of the therapy response.
- the predictor may be used predict the therapy outcome early in the therapy process, such as from one, two, or less than five scans.
- Different predictors may be learnt using different training data.
- the system may be trained for different lesion entities (liver, lung, lymph nodes, or other), different therapies, different cohorts of patients, different imaging modality, or other differences, resulting in specific therapy response features that are automatically learnt.
- Either separate systems can be trained or the system may incorporate additional non-image features, such as the type of therapy, lesion entity, or time after onset of therapy to augment image-based features.
- a predictor generic to different situations e.g., different types of lesions, different organs, or different imaging modality
- distinguishing features are learned. Less than all of the input features may be determinative in a given situation.
- the training may select some features and not others to be used for predicting therapy response.
- the learning may indicate only a sub-set of features to be used for classification. In alternative embodiments, all of the features of the input feature vector are used by the learnt predictor.
- the learnt predictor is applied to assess therapy response for a given patient. For patients with a same type of lesion or other situation, the same learnt predictor may be used.
- Figure 6 shows one embodiment of application for therapy response assessment.
- the same or a different processor than used for training performs the acts of Figure 8.
- the matrix is used by a server for online assessment by customers.
- a medical institution purchases the matrix and applies the matrix for use with their patients.
- one or more images are obtained.
- the images are obtained by transfer, loading from memory, or as output by scanning.
- the images are pre and/or post therapy images for the patient.
- the images are from a pre-therapy scan and only a few (e.g., only one or only two) post therapy scans.
- a few (e.g., only one or only two) post therapy scans are used.
- an image or images from only one scan are used.
- the images represent a lesion in the patient. uitiple lesions may be represented.
- the data representing the lesion is obtained by medical imaging, such as from CT scanning with a CT system.
- An x-ray source and detector mounted on a gantry scan the patient.
- a computer reconstructs a volume of the patient from the detected x-ray attenuations.
- a slice or projection from the volume may be used as a frame of data of an image representing the lesion.
- Other computed tomography images may be used, such as contrast enhanced or iodine images.
- act 56 one or more regions of interest are identified.
- the lesion as represented in each of the images is located.
- the same or different segmentation or region designation used for training is used in application.
- a processor automatically segments the legion from healthy tissue.
- a user manually positions a box designating the region of interest.
- the region of interest includes the lesion and a border of the lesion. Surrounding information may or may not be included. The region may or may not be dilated to include the surrounding information.
- act 58 texture features are extracted with independent analysis.
- the independent analysis was performed for learning the texture features from the training data.
- the resulting texture feature is used in the application to extract one or more values.
- the image filters or filter kernels learned from the training data are convolved with the lesions as represented by the obtained images.
- the filter kernels are convolved with the regions of interest.
- the convolution is performed for each of the images representing the lesion.
- the pre and/or post therapy medical images at least for the regions of interest, are convolved with the filter kernels. Any number of filter kernels is used, such as three or more (e.g., five in the example of Figure 4). Each filter kernel is convolved with each of the regions of interest in each of the images.
- the texture features are extracted from this limited number of images.
- the filter kernels used to extract the texture features are learned from the images of many patients, but the resulting filter kernels are applied to the images of a given patient for application.
- the texture feature (e.g., filter kernel) to be convolved with the image is learnt automatically from training images with or without ground truth information.
- the ground truth may be used to identify texture for desired outcome from texture with undesired outcome or therapy response.
- the texture feature for convolution with the images is based on independent analysis, such as independent subspace analysis. By using data to develop the texture feature, different texture features result from the use of different training data. Training data appropriate for a given application is used to provide the texture feature.
- the convolution is limited to the appropriate regions of interest.
- filter kernels developed for lesions are used to filter regions of interest corresponding to or representing lesions.
- the filtering is not performed outside of the region of interest.
- the filter kernels are used to filter the images of the patient.
- the same or different calculation used to calculate feature value for training the classifier are used to calculate the value for the texture feature. For example, the results of the convolution are summed or averaged.
- the intensities output from the filtering are summed. The sum may be divided by the number of samples (e.g., pixels) in the region of interest.
- act 60 other features are obtained.
- Other texture features may be extracted. For example, filters defined by mathematical formulas are convolved with the images. Any formula-based texture features may be used, such as discussed above for training. Pre-designed texture features, such as
- Haar wavelets homogeneity, or local binary patterns, may be calculated.
- the features may be geometric, such as change in area or volume.
- the features may be patient related, such as smoking history, medications, or family history.
- the features may be a stage or other score related to the lesion.
- a therapy response of the lesion of the patient is classified.
- a processor implementing a machine-learnt classifier, predicts the therapy response.
- the features such as one or more features resulting from convolving the data-driven or training data-based texture feature, are input to the predictor.
- the predicted response to therapy based on the input feature vector is output.
- the outcome of therapy is predicted as a function of the texture features.
- the prediction is performed without any geometric measures as input.
- an area or volume may be included in the feature vector, but shrinkage or change over time of geometry is not included. Instead, texture information is used.
- shrinkage or other change in geometry over time is used as an input feature with texture features.
- the therapy response is classified with just imaging features from data-driven or training data learnt texture features.
- other imaging features are used, such as texture features from pre-designed or mathematical formula-based texture features in the classifier input feature vector.
- the limited number of post therapy medical images corresponds to only one scan, only two scans, or only three scans. This provides for early assessment of therapy response. Additional images from other scans may be used in alternative embodiments. In an alternative embodiment, only one image, or images from only one scan are used, such as predicting therapy response from pre-therapy images or from a single post therapy scan.
- the classification is performed with any machine-learnt classifier.
- the predictor is a support vector machine or a regression random forest. Other classifiers may be used.
- the classifier predicts an outcome of a given therapy.
- the prediction may occur after one round of therapy, so the classifier predicts an outcome of continuing therapy.
- the success or not of the therapy sequence is predicted before progressing further in the therapy sequence.
- the outcome is predicted after therapy is complete, but before the lesion fully responds to the therapy.
- the classifier outputs a likelihood of therapy success or failure for the lesion of the patient.
- the likelihood is a binary indication, such as success or failure (e.g., lesion eradicated or not, or lesion growth stopped or not).
- the likelihood is a success rating with a scale of three or more.
- the likelihood is a percentage chance of success or failure of treatment.
- a SVM, Regression Random Forest, Bayesian, or other machine- learnt classifier may provide a probability as an output. The probability is a ranking or percentage likelihood of success or failure in treatment of the lesion.
- the classifier may be trained based on any definition of success or failure. For example, stopping growth, causing a percentage shrinkage, eradicating, or causing necrosis may be used as the definition of success.
- the likelihood of therapy response is output.
- the output is to a display, to a computer, or to storage.
- the likelihood is added to a computerized medical record for the patient.
- an image or multiple images of the lesion are displayed to the user.
- the likelihood is indicated on or adjacent to the image. Color coding, text, or a numerical value may be used to represent the likelihood.
- Figure 7 shows a system for therapy response assessment.
- the system is a host computer, control station, work station, server, or other arrangement.
- the system includes the display 14, memory 16, and processor 18. Additional, different, or fewer components may be provided.
- the system is for training, such as using images from the medical imaging system 1 1 as ground truth.
- the system is for application of the learned features and classifier, such as using images from the medical imaging system 1 1 for predicting response to therapy for a patient.
- the system is for training, such as using images from the medical imaging system 1 1 as ground truth.
- the system is for application of the learned features and classifier, such as using images from the medical imaging system 1 1 for predicting response to therapy for a patient.
- the learned features and classifier such as using images from the medical imaging system 1 1 for predicting response to therapy for a patient.
- the medical imaging system 11 is part of the system.
- a picture archiving and communications system (PACS) or other memory is provided instead of or in addition to the medical imaging system 1 1 for supplying images.
- the display 14, processor 18, and memory 18 may be part of a computer, server, or other system for image processing images from the medical imaging system 1 1 .
- a workstation or control station for the medical imaging system 1 1 may be used.
- a separate or remote device not part of the medical imaging system 11 is used. Instead, the therapy assessment is performed remotely from the medical imaging system 1 1.
- the processor 18 and memory 16 are part of a server hosting the therapy response assessment function for use by a medical professional computer as the client.
- the client and server are interconnected by a network, such as an intranet or the internet.
- the client may be a computer of the medical imaging system 1 1 or a computer of a medical professional, and the server may be provided by a manufacturer, provider, host, or creator of the therapy response assessment system.
- the medical imaging system 1 1 is any now known or later developed imaging system.
- the medical imaging system 1 1 is a computed tomography, ultrasound, x-ray, magnetic resonance, or functional imaging system.
- a computed tomography system an x-ray source and detector are mounted on or in a gantry on opposite sides of a patient space and corresponding patient bed. As the gantry moves the source and detector around the patient, a sequence of x-ray projections of the patient are acquired.
- a processor such as the processor 18 or a different processor, reconstructs the x-ray attenuation in three-dimensions or for one or more slices.
- the display 14 is a CRT, LCD, projector, plasma, printer, smart phone or other now known or later developed display device for displaying the images, learned texture features, and/or therapy response assessment information.
- the display 14 displays two images, information about the images, therapy information, and an indication of whether the therapy is predicted to be successful.
- the display 14 may be of data-driven features, statistics, feature information, optimization information, or other training information.
- the texture features (e.g., data learnt texture features), other features, classifiers, matrices, outputs, images, regions of interest, patches, and/or other information are stored in a non-transitory computer readable memory, such as the memory 18.
- the memory 16 is an external storage device, RAM, ROM, database, and/or a local memory (e.g., solid state drive or hard drive).
- the same or different non-transitory computer readable media may be used for instructions and other data.
- the memory 18 may be implemented using a database management system (DBMS) managed by the processor 18 and residing on a memory, such as a hard disk, RAM, or removable media.
- DBMS database management system
- the memory 16 is internal to the processor 18 (e.g. cache).
- non-transitory computer-readable storage media or memories such as a cache, buffer, RAM, removable media, hard drive or other computer readable storage media (e.g., the memory 16).
- Computer readable storage media include various types of volatile and nonvolatile storage media.
- the functions, acts or tasks illustrated in the figures or described herein are executed in response to one or more sets of instructions stored in or on computer readable storage media.
- the functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination.
- the instructions are stored on a removable media device for reading by local or remote systems.
- the instructions are stored in a remote location for transfer through a computer network.
- the instructions are stored within a given computer, CPU, GPU or system. Because some of the constituent system components and method steps depicted in the accompanying figures may be implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the present embodiments are programmed.
- a program may be uploaded to, and executed by, the processor 18 comprising any suitable architecture.
- processing strategies may include multiprocessing, multitasking, parallel processing and the like.
- the processor 18 is implemented on a computer platform having hardware, such as one or more central processing units (CPU), a random access memory (RAM), and input/output (I/O) interface(s).
- the computer platform also includes an operating system and microinstruction code.
- microinstruction code or part of the program (or combination thereof) which is executed via the operating system is executed via the operating system.
- the processor 18 is one or more processors in a network.
- the processor 18 is configured to obtain images. A region of interest may be determined by the processor 18 or received from a user interface.
- the processor 18 is configured to learn features or extract learned features. For example, an independent analysis is performed on a collection of regions of interest representing lesions. As another example, a learnt texture feature, such as a filter kernel from independent analysis, is applied by the processor 18. The processor 18 may train a classifier using the learned or data-driven features, training data, and ground truth information.
- the processor 18 is configured to classify based on the learned features.
- the features are extracted from one or more images for a given patient.
- the values of the features are input to a learnt classifier.
- the processor 18 determines an output of the machine-leamt classifier based on the input features.
- the output is provided to the memory 16, the display 14, or a network interface.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Veterinary Medicine (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Optics & Photonics (AREA)
- High Energy & Nuclear Physics (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
Abstract
For therapy response assessment, texture features are input for machine learning a classifier and for using a machine learnt classifier. Rather than or in addition to using formula-based texture features, data driven texture features are derived from training images. Such data driven texture features are independent analysis features, such as features from independent subspace analysis. The texture features may be used to predict the outcome of therapy based on a few number of or even one scan of the patient
Description
EARLY THERAPY RESPONSE ASSESSMENT OF LESIONS
Related Application
[0001] The present patent document claims the benefit of the filing date under 35 U.S.C. §1 19(e) of Provisional U.S. Patent Application Serial No. 61/882,143, filed Sept. 25, 2013, the disclosure of which is hereby
incorporated by reference.
Background
[0002] The present embodiments relate to early therapy response assessment. In particular, computer-aided response assessment is provided.
[0003] The response of a tumor under therapy is measured based on geometric measures, such as the diameter (e.g., RECIST or WHO criteria) or volume. A clinician monitors the geometric measures in radiological scans (e.g. CT or MR!) for a long period of time until a decision can be made about outcome. Many rounds of therapy are performed before the clinician makes a decision based on the geometric measures about whether the therapy is likely to succeed or not. Long periods of therapy may result in higher dosage for the patient (e.g., radiation therapy and/or x-ray imaging for the geometric measures) and greater cost of the therapy drugs.
[0004] Typically, the lesion appearance (e.g. enhancement pattern) changes earlier than geometric changes of the lesion occur. The lesion may maintain its size for a longer time while its tissue is already turning necrotic. The texture of a lesion gives insight into the therapy response at a much earlier stage, when the size or shape of the lesion is still largely unaffected. Using texture as a parameter, it may be possible to identify cases with a similar medical condition in a database for which the applied therapy and its outcome are known. This may help the doctor in estimating the effectiveness of different therapies and chose the best strategy. However, such
approaches are time consuming and/or difficult for a doctor or other person to perform.
Summary
[0005] Systems, methods, and computer readable media are provided for therapy response assessment. Texture features are input for machine learning a classifier and for using a machine learnt classifier. Rather than or in addition to using formula-based texture features, data driven texture features are derived from training images. Such data driven texture features are independent analysis features, such as features from independent subspace analysis. The texture features may be used to predict the outcome of therapy based on a few number of or even one scan of the patient.
[0006] In a first aspect, a method is provided for therapy response assessment. Pre-therapy and post therapy medical images of a patient are obtained. The medical images represent at least one lesion of the patient. A processor convolves the pre-therapy and post therapy medical images with a texture feature learned from training images. The processor classifies a therapy response of the lesion with a machine-learnt classifier with a result of the convolving as an input feature to the machine-learnt classifier.
[0007] In a second aspect, a non-transitory computer readable storage medium has stored therein data representing instructions executable by a programmed processor for therapy response assessment. The storage medium includes instructions for: with only one or two scans, extracting texture features for a lesion with a filter kernel, the filter kernel being independently based on image data; and predicting an outcome of therapy on the lesion, the predicting being a function of the texture features.
[0008] In a third aspect, a method is provided for therapy response assessment. A processor subjects patches of lesions represented in a plurality of training frames of data to independent subspace analysis. The processor creates a number of image filters from the independent subspace analysis of the patches and calculates texture features by application of the image filters to the lesions as represented in the training frames of data. The processor learns a predictor of therapy response as a function of the texture features.
[0009] Any one or more of the aspects described above may be used alone or in combination. These and other aspects, features and advantages
will become apparent from the following detailed description of preferred embodiments, which is to be read in connection with the accompanying drawings. The present invention is defined by the following claims, and nothing in this section should be taken as a limitation on those claims.
Further aspects and advantages of the invention are discussed below in conjunction with the preferred embodiments and may be later claimed independently or in combination.
Brief Description of the Drawings
[0010] The components and the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the embodiments. Moreover, in the figures, like reference numerals designate corresponding parts throughout the different views.
[0011] Figure 1 shows an example process for therapy response assessment using geometric measures;
[0012] Figure 2 shows an example process for therapy response assessment using texture features;
[0013] Figure 3 is a flow chart diagram of one embodiment of a method for training in therapy response assessment;
[0014] Figure 4 illustrates example texture features learned from training images;
[0015] Figures 5A-C show results of convolving the features of Figure 4 with three different lesions;
[0016] Figure 6 is a flow chart diagram of one embodiment of a method for application in therapy response assessment; and
[0017] Figure 7 is a block diagram of one embodiment of a system for therapy response assessment.
Detailed Description of Embodiments
[0018] In therapy response assessment, a computer assists a doctor in estimating the effectiveness of different cancer therapies and choosing the best strategy. Given the variability of lesions, imaging settings, situations, and number of cases, it may take a doctor or other medical professional an
unreasonable amount of time to identify similar cases and patients for making an informed prediction. As a result, there may be a great amount of variability in predictions by different medical professionals. A computer may more quickly access the information and more consistently predict therapy response. Given the subtlety of prediction from, at least in part, medical images, a processor may better perform or at least provide a more efficient second perspective on therapy response assessment.
[0019] In order to judge the state of a tumor or lesion in an objective manner, measures of therapy response suitable at early therapy stages are used. The effect of the therapy is assessed as early as possibie. Using two scans, or possibly with only one scan, the predictor may decide whether the therapy will fail or succeed. Queues that may be missed by a doctor with such a few images may be identified by the processor applying the predictor. The decision making happens at an earlier stage.
[0020] Figure 1 shows an approach using geometric measures where a pre-therapy scan is performed and then a sequence of N (e.g., 5 or more) scans are performed after therapy or during a sequence of therapy
applications. The clinician then uses the scans to determine geometric changes to predict the success or failure of the therapy. Meanwhile, the patient has been exposed to the medications and/or radiation associated with therapy and/or monitoring many times. Conversely, Figure 2 shows acquiring the pre-therapy scan and a single post therapy scan. Texture features reflecting cell density, vasculature, necrosis, hemorrhage, other
characteristics, or combinations thereof are extracted and used for automated (e.g., machine learnt) prediction of success or failure of the therapy. The prediction may be made earlier in the process, possibly avoiding further therapy where the outcome is likely negative. The prediction may be made based on the pre-therapy scan before therapy. Early therapy outcome prediction is provided, possibly reducing exposure of a patient to therapy that is likely not successful.
[0021] In the prediction, the texture of organs and/or lesions is used to predict the likelihood of future cancer development. The features are extracted using Independent Subspace Analysis (ISA) or other data-driven
approach. Rather than or in addition to using programmer designed formulas for measuring texture, training scans are used to determine texture that is appropriate for and/or determinative of outcome for a given type of lesion, organ, therapy, and/or other situation. This is different from traditional features where a fixed formula describes how to calculate the feature value. With ISA or other data-driven development of the texture features, different sets of training images and ground truth labels may generate different image filters, and hence different feature values. The advantage of such a data- driven approach is that, as the training dataset grows larger, more meaningful patterns of texture may be discovered without relying on expert knowledge to manually define heuristic and sub-optimal mathematical formulas for patterns. The learning-based approach also allows training the system for different therapy types (e.g. chemo-therapy. targeted therapy, radiation therapy) resulting in therapy-specific response features that are automatically learnt. Data-driven texture features allow for different features to be learned for different types of lesions, tissues, therapy, and/or imaging modalities.
[0022] Figures 3 and 6 show methods for therapy response assessment. The method for therapy response assessment may be a method to learn how to access outcome of therapy or may be a method for assessing the outcome with a learnt predictor. Figure 3 is directed to machine training of the therapy outcome predictor. Figure 8 is directed to application of a machine-learnt therapy outcome predictor. In both cases, a machine, such as a processor, computer, or server, implements some or all of the acts. The system of Figure 7 implements the methods in one embodiment. A user may select the image files for application of the therapy response predictor by the processor, select the image from which to learn features by a processor, and/or identify a region of interest. Use of the machine allows processing large volumes (e.g., images of many pixels and/or many images) of information that may not be efficiently handled by humans (at least in comparative time frames for a given volume of information), may be unrealistically handled by humans in the needed time frame, or may not even be possible by humans due to subtleties and/or timing.
[0023] The methods are provided in the orders shown, but other orders may be provided. For example in Figure 6, acts 58 and 60 may be performed in parallel or opposite order.
[0024] Additional, different or fewer acts may be provided. For example, act 44 of Figure 3 is not provided. As another example, acts 60 and/or 64 of Figure 6 are not provided. In yet other examples, acts for capturing images and/or acts using detected information are provided. Acts for configuring, input, or output may be provided.
[0025] Figure 3 shows a method for learning in therapy response assessment. A processor performs the learning, such as by performing acts 44-52. The processor both creates texture features from training data in acts 46-48 and learns a predictor with the texture features as inputs in act 52. The learnt feature may be used by the processor to calculate a feature value and/or by the processor to train a classifier.
[0026] In act 40, one or more images of an object are obtained. The images are obtained by data transfer, capture, and/or loading from memory.
[0027] The images are frames of data representing a patient at given times. The images are scalar values or display values (e.g., red green blue (RGB)). The images may have been previously displayed or have not yet been displayed.
[0028] The images are acquired from a scan. The images are captured using any one or more sensors. For example, images of organs are captured using x-ray, computed tomography (CT), fluoroscopy, angiography, magnetic resonance, ultrasound, positron emission tomography, or single photon emission computed tomography. A given pre or post therapy scan provides one or more images. For example, images for different phases of a multiple phase contrast enhanced scan (e.g., native, arterial, and venous phases of contrast agent wash in and/or wash out) are acquired. As another example, Dual Energy CT images, including an iodine map, are acquired as part of a scan.
[0029] The pre and post therapy scans are the same or different types of scans. Each scan provides one or multiple images. Multiple images of the
same or different patients use the same or different imaging modality with the same or different settings (e.g., field of view).
[0030] The object of interest in a medical image may be an organ, a cyst, a tumor, caicification or other anomaly. Any type of lesion may be imaged. The organ surrounding the lesion may be scanned. The image may represent healthy tissue. The image may represent the lesion in two or three- dimensions. For example, the image may be of pixels or voxels.
[0031] For training, images are acquired for the same type of lesion with or without other common arrangements (e.g., patient characteristics (male vs. female or age), imaging modality, organ, type of therapy, and/or stage). Pre and post therapy images for many, such as tens, hundreds, thousands, or more, patients are obtained. The images are associated with or linked to known outcomes. For example, all of the images are labeled with ground truth. For creating texture features, just the images associated with positive outcome are used. Alternatively, just images associated with negative outcome or both positive and negative outcomes are used. For training the predictor, images labeled for both positive and negative outcome are used.
[0032] In act 42, one or more regions of interest (ROi) are identified in each of the images. Manual (e.g., user input) or automated tumor detection or segmentation is used to find the region of interest. Any now known or later developed region of interest identification may be used.
[0033] The region of interest is a bounding box or other shape enclosing the lesion or is a segmentation specifically of the lesson. More than one region of interest may be provided on a given image. For example, multiple lesions are represented in a given image. As another example, one or more regions of interest for healthy tissue are identified. In one embodiment, nested regions of interest are provided, such as one for an organ as a whole or in part and a sub-set of that region being a region for a lesion.
[0034] The region of interest is to be used for feature extraction. Tumor associated features may be extracted from within the region of interest. The approach is to combine multiple sets of different features for training a classifier. Such features may also be extracted from other reference regions in the image data, such as the whole liver parenchyma, to complement the
tumor features extracted from within the lesion specific ROI and to provide the classifier with additional information.
[0035] The delineated lesion region of interest may be expanded to assure that the border or gradient from lesion to healthy tissue is included. The region of interest is dilated (enlarged) by the processor to include surrounding information. The dilation is by a percentage (e.g., 5-10% expansion), by a number of samples (e.g., adding 2-10 samples from a center), or by another process. Alternatively or additionally, the region of interest is originally defined by the user or processor to include the
surrounding information. In alternative embodiments, no expansion is provided.
[0038] The region of interest may be sub-divided. For example, a moving window is defined to cover a part of the region of interest. By moving the window with any step size (e.g., 1 pixel, 2 pixels, or other distance), overlapping sub-sets of the region of interest are defined. These patches or sub-sets are uniformly defined within the region of interest with a fixed step size. Any size of patch may be used, such as 8x8 pixels. In alternative embodiments, the patch size and/or step size vary or are not uniform.
[0037] In act 44, the patches are whitened by the processor. Within each kernel or patch, the values are altered to set the mean to a zero value and make the variance equal to one. Other normalization may be used. In one embodiment, the patches are whitened with a principal component analysis. Alternatively, whitening is not performed.
[0038] In act 46, independent subspace analysis (ISA) or independent component analysis (ICA) is applied to the patches, such as the whitened patches. The independent analysis examines the data of the patches to find text patterns. The processor analyzes many or all of the patches of many or all of the regions of interests from the different patients to learn one or more texture features. Data-driven machine learning is used by the processor to recognize one or more patterns common across various patches and patients. Both ISA and ICA find features that form independent subspaces, but ISA is found to be more robust to local variations than ICA.
[0039] In one embodiment, the patches are subjected to independent subspace analysis (ISA) by the processor. In another embodiment, the patches are subjected to independent component analysis by the processor. The patches are decomposed into different components to find patterns.
Other pattern recognition, with or without independence, may be used.
[0040] The independent analysis or other pattern recognition by the processor from the many examples results in a number, N, of image filters, in act 48, the image fiiters are created by the application of the independent analysis in act 46. Any number of common patterns in the various training examples may be created. For example, five image filters are learned from the training images.
[0041] To distinguish texture of lesions with desired response from others, the ground truth is used. The training data associated with desired outcome, as indicated by the ground truth, is used for creating the image filters.
Alternatively or additionally, other training data is used. In other
embodiments, the independent analysis learns texture features that distinguish between texture with desired outcome from texture with an undesired outcome, so the training data is divided into sets based on the ground truth. In alternative embodiments, the image filters are created without consideration of ground truth. Texture common to both necrotizing and non-necrotizing lesions are found by computer implemented analysis.
[0042] The image filters have a same size and shape as the patches. Alternatively, the image filters have a different size and/or shape as the patches. The image filters are binary masks or include any number of gradations, such as including an average or full dynamic range of the patches from the training data.
[0043] Figure 4 shows five example image filters learned by a processor from pattern recognition using independent subspace analysis. The image filters were created in act 48 from CT images of liver tumors of patients.
These image filters represent more common patterns across the set of lesions for the training data. The same, different, or no image filters are created for healthy tissue. Given different training data, different image filters may result.
[0044] In act 50, texture features are calculated by the processor. The image filters (see Figure 4 for example) represent texture features for a given situation, but are not themselves the inputs for learning a predictor. The image filters are applied to the training data to calculate values for the texture features. The image filters are applied to the same training data from which the image filters were created. Alternatively, a different set of training data is used.
[0045] The image filters are applied to the appropriate regions of interest. For image filters representing lesion texture, the image filters are applied to lesions as represented in the training data. For image filters representing healthy tissue or a combination of healthy tissue and lesion tissue, the image filters are applied to the regions of interest for that combination.
[0046] The image filters are applied by spatial filtering. The image filters are one, two, or three-dimensional filters. The N filters are applied to each tumor by a convolution operation (e.g., dot product or other similarity measure). The image filter is positioned relative to the legion and the intensities of the image filter are multiplied with the data of the image in the patch of the region of interest. A sum or average of the resulting
multiplications provides a value for the image filter at that position. By shifting the image filter, such as was done for the patches, values for different locations are determined. Any convolution of the image filters with the lesions as represented in the training frames of data may be used.
[0047] Figures 5A-C show convolution results of the image filters of Figure 4 with three different lesions. The left to right sequence of texture features represented by the image filters of Figure 4 is maintained in Figures 5A-C. The left most CT slice image is of the lesion being filtered. Where the lesion includes texture more closely resembling the image filter, the resulting pixel has a higher intensity (e.g., more white). The differences in scale between Figures 5A-C are due to differences in scale and/or size of the lesion.
[0048] The resulting values (i.e., filter responses) from the convolution are combined to indicate a feature value for that lesion based on the given image filter. For example, the intensity values are summed or averaged, in the example of Figure 5A, the average intensity for each of the texture features is
calculated, providing a single value for each image filter. For example, the average response per filter per tumor is recorded as a value of the ISA feature. Alternatively, a feature is calculated from the results of the
convolution, such as using a Haar wavelet feature or other pre-designed feature from the data driven feature provided by the image filter convolution with the image data.
[0049] The resulting values are the texture features as used for the training in act 52. Other information may be used as part of the input feature vector for training. The other information may represent the patient (e.g., age, sex, or family history), stage of the lesion, type of therapy, other situational information, and/or other image derived features.
[0050] In one embodiment, other texture features are determined from the training data. For example, Haralick texture features, such as contrast, entropy, and/or others, are used, in another example or in addition, homogeneity, energy, and dissimilarity texture features are used. The homogeneity, for example, has the following equation:
where counts the number of times a pixel with value ' is adjacent to a pixel with value J and then dividing by the total number of such comparisons V
made. g is the number of gray levels in the image. The texture features rely on predetermined formulas rather than features learned from training data by a processor. Any now known or later developed texture features may be used.
[0051] In another additional or alternative embodiment, texture features based on local binary patterns are used. The local binary patterns compare the intensity of each pixel with its neighboring pixels and return a code that compactly summarizes the differences. These codes are then summarized over the ROi through a histogram. A data pyramid may be used, where the same image at different resolutions (e.g., by decimation) is provided. The local binary patterns are then calculated for each resolution or level of the
pyramid, as each pyramid level contains textura! information in different scale and details.
[0052] In one embodiment, the texture features are calculated from the training data for one pre-therapy scan and one post therapy scan. In alternative embodiments, only pre-therapy scan, only the first post therapy scan, or additional post therapy scans are used.
[0053] The texture features, whether data driven or other texture features, are the same for pre and post therapy scans. In alternative embodiments, different features are provide for different scans relative to the therapy (i.e., !earn texture features for pre-therapy images different than texture features learned from post therapy images).
[0054] Once the values for the input feature vector are calculated and/or obtained, the ground truth for therapy response is used to train, by a processor, the predictor of the therapy response. The ground truth is binary, such as successful or not successful outcome from therapy. Alternatively, the ground truth has a greater resolution, such as the number of months without re-occurrence, time of remission, change in size, time to narcotization, or other measure of success of therapy.
[0055] In act 52, a predictor of therapy response is learned by a
processor. The processor uses the ground truth and data-driven texture features, with or without other features to train a classifier. Any machine learning may be used, such as a probabilistic boosting tree, support vector machine, or other machine learning classifier. Other classifiers may include a single class or binary classifier, collection of different classifiers, cascaded classifiers, hierarchal classifier, multi-class classifier, model-based classifier, or combinations thereof may be used. Multi-class classifiers include CART,
K-nearest neighbors, neural network (e.g., multi-layer perception}, mixture models, or others. Error-correcting output code (ECOC) may be used.
[0056] In one embodiment, the features are fed into a machine learning classifier or regressor, such as a Support Vector Machine (SVM) or
Regression Random Forest, for training. The training provides a matrix associating input values of features to outputs or predictions. The matrix or trained classifier (e.g., learnt classifier) is a predictor of the therapy response.
During training, the set of ground truth cases with known therapy outcome are provided for learning the system parameters. The predictor may be used predict the therapy outcome early in the therapy process, such as from one, two, or less than five scans.
[0057] Different predictors may be learnt using different training data. The system may be trained for different lesion entities (liver, lung, lymph nodes, or other), different therapies, different cohorts of patients, different imaging modality, or other differences, resulting in specific therapy response features that are automatically learnt. Either separate systems can be trained or the system may incorporate additional non-image features, such as the type of therapy, lesion entity, or time after onset of therapy to augment image-based features. Alternatively, a predictor generic to different situations (e.g., different types of lesions, different organs, or different imaging modality) is trained from training data with the same generic situation.
[0058] During the optimization to train the predictor, different
distinguishing features are learned. Less than all of the input features may be determinative in a given situation. The training may select some features and not others to be used for predicting therapy response. The learning may indicate only a sub-set of features to be used for classification. In alternative embodiments, all of the features of the input feature vector are used by the learnt predictor.
[0059] The learnt predictor is applied to assess therapy response for a given patient. For patients with a same type of lesion or other situation, the same learnt predictor may be used. Figure 6 shows one embodiment of application for therapy response assessment. The same or a different processor than used for training performs the acts of Figure 8. For example, the matrix is used by a server for online assessment by customers. As another example, a medical institution purchases the matrix and applies the matrix for use with their patients.
[0060] In act 54, one or more images are obtained. The images are obtained by transfer, loading from memory, or as output by scanning. The images are pre and/or post therapy images for the patient. In one
embodiment, the images are from a pre-therapy scan and only a few (e.g.,
only one or only two) post therapy scans. In yet other embodiments, an image or images from only one scan (e.g., pre or post therapy) are used.
[0061] The images represent a lesion in the patient. uitiple lesions may be represented. The data representing the lesion is obtained by medical imaging, such as from CT scanning with a CT system. An x-ray source and detector mounted on a gantry scan the patient. A computer reconstructs a volume of the patient from the detected x-ray attenuations. A slice or projection from the volume may be used as a frame of data of an image representing the lesion. Other computed tomography images may be used, such as contrast enhanced or iodine images.
[0062] In act 56, one or more regions of interest are identified. The lesion as represented in each of the images is located. The same or different segmentation or region designation used for training is used in application. For example, a processor automatically segments the legion from healthy tissue. As another example, a user manually positions a box designating the region of interest.
[0063] The region of interest includes the lesion and a border of the lesion. Surrounding information may or may not be included. The region may or may not be dilated to include the surrounding information.
[0064] In act 58, texture features are extracted with independent analysis. The independent analysis was performed for learning the texture features from the training data. The resulting texture feature is used in the application to extract one or more values. The image filters or filter kernels learned from the training data are convolved with the lesions as represented by the obtained images. The filter kernels are convolved with the regions of interest.
[0065] The convolution is performed for each of the images representing the lesion. The pre and/or post therapy medical images, at least for the regions of interest, are convolved with the filter kernels. Any number of filter kernels is used, such as three or more (e.g., five in the example of Figure 4). Each filter kernel is convolved with each of the regions of interest in each of the images.
[6086] Since only one, only two, or a few number of scans are performed for early prediction, the texture features are extracted from this limited number
of images. The filter kernels used to extract the texture features are learned from the images of many patients, but the resulting filter kernels are applied to the images of a given patient for application.
[0067] The texture feature (e.g., filter kernel) to be convolved with the image is learnt automatically from training images with or without ground truth information. The ground truth may be used to identify texture for desired outcome from texture with undesired outcome or therapy response. The texture feature for convolution with the images is based on independent analysis, such as independent subspace analysis. By using data to develop the texture feature, different texture features result from the use of different training data. Training data appropriate for a given application is used to provide the texture feature.
[0068] The convolution is limited to the appropriate regions of interest. For example, filter kernels developed for lesions are used to filter regions of interest corresponding to or representing lesions. The filtering is not performed outside of the region of interest.
[0069] The filter kernels are used to filter the images of the patient. The same or different calculation used to calculate feature value for training the classifier are used to calculate the value for the texture feature. For example, the results of the convolution are summed or averaged. The intensities output from the filtering are summed. The sum may be divided by the number of samples (e.g., pixels) in the region of interest.
[0070] In act 60, other features are obtained. Other texture features may be extracted. For example, filters defined by mathematical formulas are convolved with the images. Any formula-based texture features may be used, such as discussed above for training. Pre-designed texture features, such as
Haar wavelets, homogeneity, or local binary patterns, may be calculated.
[0071] Other non-texture features may be calculated. The features may be geometric, such as change in area or volume. The features may be patient related, such as smoking history, medications, or family history. The features may be a stage or other score related to the lesion.
[0072] In act 62, a therapy response of the lesion of the patient is classified. A processor, implementing a machine-learnt classifier, predicts the
therapy response. The features, such as one or more features resulting from convolving the data-driven or training data-based texture feature, are input to the predictor. The predicted response to therapy based on the input feature vector is output. The outcome of therapy is predicted as a function of the texture features.
[0073] In one embodiment, the prediction is performed without any geometric measures as input. In other embodiments, an area or volume may be included in the feature vector, but shrinkage or change over time of geometry is not included. Instead, texture information is used. In other embodiments, shrinkage or other change in geometry over time is used as an input feature with texture features.
[0074] In one embodiment, the therapy response is classified with just imaging features from data-driven or training data learnt texture features. In other embodiments, other imaging features are used, such as texture features from pre-designed or mathematical formula-based texture features in the classifier input feature vector.
[0075] For features from imaging, only features from the pre~therapy medical images and a limited number of post therapy medical images are used. The limited number of post therapy medical images corresponds to only one scan, only two scans, or only three scans. This provides for early assessment of therapy response. Additional images from other scans may be used in alternative embodiments. In an alternative embodiment, only one image, or images from only one scan are used, such as predicting therapy response from pre-therapy images or from a single post therapy scan.
[0078] The classification is performed with any machine-learnt classifier. For example, the predictor is a support vector machine or a regression random forest. Other classifiers may be used.
[0077] The classifier predicts an outcome of a given therapy. The prediction may occur after one round of therapy, so the classifier predicts an outcome of continuing therapy. After obtaining a post therapy scan and corresponding medical image or images, the success or not of the therapy sequence is predicted before progressing further in the therapy sequence.
Alternatively, the outcome is predicted after therapy is complete, but before the lesion fully responds to the therapy.
[0078] The classifier outputs a likelihood of therapy success or failure for the lesion of the patient. The likelihood is a binary indication, such as success or failure (e.g., lesion eradicated or not, or lesion growth stopped or not). Alternatively, the likelihood is a success rating with a scale of three or more. For example, the likelihood is a percentage chance of success or failure of treatment. A SVM, Regression Random Forest, Bayesian, or other machine- learnt classifier may provide a probability as an output. The probability is a ranking or percentage likelihood of success or failure in treatment of the lesion.
[0079] The classifier may be trained based on any definition of success or failure. For example, stopping growth, causing a percentage shrinkage, eradicating, or causing necrosis may be used as the definition of success.
[0080] In act 84, the likelihood of therapy response is output. The output is to a display, to a computer, or to storage. For example, the likelihood is added to a computerized medical record for the patient. As another example, an image or multiple images of the lesion are displayed to the user. The likelihood is indicated on or adjacent to the image. Color coding, text, or a numerical value may be used to represent the likelihood.
[0081] Figure 7 shows a system for therapy response assessment. The system is a host computer, control station, work station, server, or other arrangement. The system includes the display 14, memory 16, and processor 18. Additional, different, or fewer components may be provided. The system is for training, such as using images from the medical imaging system 1 1 as ground truth. Alternatively, the system is for application of the learned features and classifier, such as using images from the medical imaging system 1 1 for predicting response to therapy for a patient. In other
embodiments, the medical imaging system 11 is part of the system. In yet other embodiments, a picture archiving and communications system (PACS) or other memory is provided instead of or in addition to the medical imaging system 1 1 for supplying images.
[0082] The display 14, processor 18, and memory 18 may be part of a computer, server, or other system for image processing images from the medical imaging system 1 1 . A workstation or control station for the medical imaging system 1 1 may be used. Alternatively, a separate or remote device not part of the medical imaging system 11 is used. Instead, the therapy assessment is performed remotely from the medical imaging system 1 1.
[0083] in one embodiment, the processor 18 and memory 16 are part of a server hosting the therapy response assessment function for use by a medical professional computer as the client. The client and server are interconnected by a network, such as an intranet or the internet. The client may be a computer of the medical imaging system 1 1 or a computer of a medical professional, and the server may be provided by a manufacturer, provider, host, or creator of the therapy response assessment system.
[0084] The medical imaging system 1 1 is any now known or later developed imaging system. For example, the medical imaging system 1 1 is a computed tomography, ultrasound, x-ray, magnetic resonance, or functional imaging system. As a computed tomography system, an x-ray source and detector are mounted on or in a gantry on opposite sides of a patient space and corresponding patient bed. As the gantry moves the source and detector around the patient, a sequence of x-ray projections of the patient are acquired. A processor, such as the processor 18 or a different processor, reconstructs the x-ray attenuation in three-dimensions or for one or more slices.
[0085] The display 14 is a CRT, LCD, projector, plasma, printer, smart phone or other now known or later developed display device for displaying the images, learned texture features, and/or therapy response assessment information. For example, the display 14 displays two images, information about the images, therapy information, and an indication of whether the therapy is predicted to be successful. In a training environment, the display 14 may be of data-driven features, statistics, feature information, optimization information, or other training information.
[0086] The texture features (e.g., data learnt texture features), other features, classifiers, matrices, outputs, images, regions of interest, patches,
and/or other information are stored in a non-transitory computer readable memory, such as the memory 18. The memory 16 is an external storage device, RAM, ROM, database, and/or a local memory (e.g., solid state drive or hard drive). The same or different non-transitory computer readable media may be used for instructions and other data. The memory 18 may be implemented using a database management system (DBMS) managed by the processor 18 and residing on a memory, such as a hard disk, RAM, or removable media. Alternatively, the memory 16 is internal to the processor 18 (e.g. cache).
[0087] The instructions for implementing the therapy response
assessment in training or application processes, methods and/or techniques discussed herein are provided on non-transitory computer-readable storage media or memories, such as a cache, buffer, RAM, removable media, hard drive or other computer readable storage media (e.g., the memory 16).
Computer readable storage media include various types of volatile and nonvolatile storage media. The functions, acts or tasks illustrated in the figures or described herein are executed in response to one or more sets of instructions stored in or on computer readable storage media. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro code and the like, operating alone or in combination.
[0088] In one embodiment, the instructions are stored on a removable media device for reading by local or remote systems. In other embodiments, the instructions are stored in a remote location for transfer through a computer network. In yet other embodiments, the instructions are stored within a given computer, CPU, GPU or system. Because some of the constituent system components and method steps depicted in the accompanying figures may be implemented in software, the actual connections between the system components (or the process steps) may differ depending upon the manner in which the present embodiments are programmed.
[0089] A program may be uploaded to, and executed by, the processor 18 comprising any suitable architecture. Likewise, processing strategies may
include multiprocessing, multitasking, parallel processing and the like. The processor 18 is implemented on a computer platform having hardware, such as one or more central processing units (CPU), a random access memory (RAM), and input/output (I/O) interface(s). The computer platform also includes an operating system and microinstruction code. The various processes and functions described herein may be either part of the
microinstruction code or part of the program (or combination thereof) which is executed via the operating system. Alternatively, the processor 18 is one or more processors in a network.
[0090] The processor 18 is configured to obtain images. A region of interest may be determined by the processor 18 or received from a user interface.
[0091] The processor 18 is configured to learn features or extract learned features. For example, an independent analysis is performed on a collection of regions of interest representing lesions. As another example, a learnt texture feature, such as a filter kernel from independent analysis, is applied by the processor 18. The processor 18 may train a classifier using the learned or data-driven features, training data, and ground truth information.
[0092] The processor 18 is configured to classify based on the learned features. The features are extracted from one or more images for a given patient. The values of the features are input to a learnt classifier. The processor 18 determines an output of the machine-leamt classifier based on the input features. The output is provided to the memory 16, the display 14, or a network interface.
[0093] Various improvements described herein may be used together or separately. Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the scope or spirit of the invention.
Claims
1 . A method for therapy response assessment, the method comprising: obtaining a pre-therapy medical image of a patient, the pre-therapy medical image representing at least one lesion of the patient;
obtaining a post-therapy medical image of the patient, the post-therapy medical image of the patient representing the at least one lesion of the patient;
convolving, by a processor, the pre-therapy and post therapy medical images with a texture feature learned from training images; and
classifying, by the processor, a therapy response of the lesion with a machine-learnt classifier with a result of the convolving as an input feature to the machine-learnt classifier.
2. The method of claim 1 wherein obtaining the pre-therapy and post therapy medical images comprises obtaining computed tomography images.
3. The method of claim 1 wherein obtaining the post-therapy medical image comprises obtaining only the post-therapy medical image or only the post-therapy medical image and one more post therapy medical image, and wherein classifying comprises classifying with the input feature including, for just features from imaging, only features from the pre-therapy and post- therapy medical images.
4. The method of claim 1 wherein convolving comprises convolving with the texture feature comprising an independent sub-space analysis feature.
5. The method of claim 1 wherein convolving comprises convolving with the texture feature and at least two other texture features learned from the training images.
6. The method of claim 1 wherein convolving comprises convolving with the texture feature learned automatically from the training images and labeled ground truths.
7. The method of claim 1 wherein convolving comprises filtering with a kernel defined by the texture feature and summing intensities output from the filtering, the result being a function of the sum of the intensities.
8. The method of claim 1 wherein convolving comprises convolving with the texture feature comprising a training image-based feature such that different training images result in different texture features.
9. The method of claim 1 wherein classifying comprises classifying with a support vector machine or a regression random forest.
10. The method of claim 1 wherein classifying comprises predicting an outcome of continuing therapy after obtaining the post-therapy medical image.
1 1. The method of claim 1 wherein classifying comprises indicating a likelihood of therapy success or failure for the lesion.
12. The method of claim 1 further comprising identifying regions of interest in the pre-therapy and post-therapy medical images including the lesion and a border of the lesion, and wherein convolving comprises convolving the texture feature with the regions of interest and not with regions outside the regions of interest.
13. The method of claim 1 further comprising:
convolving, by the processor, the pre-therapy and post therapy medical images with mathematical formula-based texture features;
wherein classifying comprises classifying the therapy response with the result and results from the convolving with the mathematical formula-based textures features as an input vector including the input feature.
14. In a non-transitory computer readable storage medium having stored therein data representing instructions executable by a programmed processor for therapy response assessment, the storage medium comprising
instructions for:
with only one or two scans, extracting texture features for a lesion with a filter kernel, the filter kernel being independently based on image data; and predicting an outcome of therapy on the lesion, the predicting being a function of the texture features.
15. The non-transitory computer readable storage medium of claim 14 wherein extracting the texture features comprises convolving scan data of the one or two scans with the filter kernel and additional filter kernels, the filter kernel and additional filter kernels being developed from independent subspace analysis of the Image data, the image data comprising training images.
16. The non-transitory computer readable storage medium of claim 14 wherein predicting comprises predicting with a machine trained classifier without consideration of shrinkage of the lesion.
17. The non-transitory computer readable storage medium of claim 14 further comprising outputting a likelihood of the outcome to a display.
18. A method for therapy response assessment, the method comprising: subjecting, by a processor, patches of lesions represented in a plurality of training frames of data to independent subspace analysis;
creating, by the processor, a number of image filters from the independent subspace analysis of the patches;
calculating, by the processor, texture features by application of the image filters to the lesions as represented in the training frames of data; and learning, by the processor, a predictor of therapy response as a function of the texture features.
19. The method of claim 18 further comprising whitening the patches with principle component analysis prior to the creating.
20. The method of claim 18 wherein calculating the texture features comprises convolving the image filters with the lesions as represented in the
training frames of data and averaging responses of the convolving, the averages comprising the texture features.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201480064452.4A CN105765629B (en) | 2013-09-25 | 2014-09-24 | The early response to treatment of lesion is assessed |
EP14793653.8A EP3050026B1 (en) | 2013-09-25 | 2014-09-24 | Early therapy response assessment of lesions |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201361882143P | 2013-09-25 | 2013-09-25 | |
US61/882,143 | 2013-09-25 | ||
US14/471,501 | 2014-08-28 | ||
US14/471,501 US9655563B2 (en) | 2013-09-25 | 2014-08-28 | Early therapy response assessment of lesions |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015048103A1 true WO2015048103A1 (en) | 2015-04-02 |
Family
ID=52691536
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2014/057175 WO2015048103A1 (en) | 2013-09-25 | 2014-09-24 | Early therapy response assessment of lesions |
Country Status (4)
Country | Link |
---|---|
US (1) | US9655563B2 (en) |
EP (1) | EP3050026B1 (en) |
CN (1) | CN105765629B (en) |
WO (1) | WO2015048103A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3293736A1 (en) * | 2016-09-09 | 2018-03-14 | Siemens Healthcare GmbH | Tissue characterization based on machine learning in medical imaging |
US10779787B2 (en) * | 2017-08-11 | 2020-09-22 | Siemens Healthcare Gmbh | Method for analyzing image data from a patient after a minimally invasive intervention, analysis apparatus, computer program and electronically readable data storage medium |
Families Citing this family (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015189811A1 (en) * | 2014-06-13 | 2015-12-17 | Siemens Medical Solutions Usa, Inc. | Improved image reconstruction for a volume based on projection data sets |
EP3286695A1 (en) * | 2015-04-24 | 2018-02-28 | Canfield Scientific, Incorporated | Dermatological feature tracking over multiple images |
WO2017023569A1 (en) * | 2015-08-04 | 2017-02-09 | Siemens Aktiengesellschaft | Visual representation learning for brain tumor classification |
US10127660B2 (en) * | 2015-12-30 | 2018-11-13 | Case Western Reserve University | Radiomic features on diagnostic magnetic resonance enterography |
US10169871B2 (en) | 2016-01-21 | 2019-01-01 | Elekta, Inc. | Systems and methods for segmentation of intra-patient medical images |
US10282588B2 (en) * | 2016-06-09 | 2019-05-07 | Siemens Healthcare Gmbh | Image-based tumor phenotyping with machine learning from synthetic data |
BR112019010225A8 (en) * | 2016-11-22 | 2023-04-04 | Hyperfine Res Inc | SYSTEMS AND METHODS FOR AUTOMATED DETECTION IN MAGNETIC RESONANCE IMAGES |
US10627464B2 (en) | 2016-11-22 | 2020-04-21 | Hyperfine Research, Inc. | Low-field magnetic resonance imaging methods and apparatus |
US10111632B2 (en) * | 2017-01-31 | 2018-10-30 | Siemens Healthcare Gmbh | System and method for breast cancer detection in X-ray images |
WO2018157381A1 (en) * | 2017-03-03 | 2018-09-07 | 深圳大学 | Method and apparatus for intelligently classifying pathological slice image |
EP3646240A4 (en) | 2017-06-26 | 2021-03-17 | The Research Foundation for The State University of New York | System, method, and computer-accessible medium for virtual pancreatography |
CN108169184B (en) * | 2017-09-05 | 2021-01-05 | 哈尔滨工业大学 | Establishment method and application of tumor classification identification model |
JP2019076531A (en) | 2017-10-25 | 2019-05-23 | テルモ株式会社 | Treatment method |
JP2019076526A (en) | 2017-10-25 | 2019-05-23 | テルモ株式会社 | Treatment method |
JP2019076527A (en) | 2017-10-25 | 2019-05-23 | テルモ株式会社 | Treatment method |
JP2019080749A (en) | 2017-10-30 | 2019-05-30 | テルモ株式会社 | Treatment method |
JP2019080876A (en) | 2017-10-31 | 2019-05-30 | テルモ株式会社 | Method for medical treatment |
JP2019080875A (en) | 2017-10-31 | 2019-05-30 | テルモ株式会社 | Treatment method |
EP3493159B1 (en) | 2017-11-30 | 2023-11-29 | Siemens Healthcare GmbH | Method and system for evaluation of tumor tissue by unfolding morphological and texture properties |
US10559080B2 (en) * | 2017-12-27 | 2020-02-11 | International Business Machines Corporation | Adaptive segmentation of lesions in medical images |
US10878576B2 (en) | 2018-02-14 | 2020-12-29 | Elekta, Inc. | Atlas-based segmentation using deep-learning |
TWI668666B (en) * | 2018-02-14 | 2019-08-11 | China Medical University Hospital | Prediction model for grouping hepatocellular carcinoma, prediction system thereof, and method for determining hepatocellular carcinoma group |
US11756667B2 (en) * | 2018-05-30 | 2023-09-12 | Siemens Healthcare Gmbh | Decision support system for medical therapy planning |
CN109308495B (en) * | 2018-07-05 | 2021-07-02 | 科亚医疗科技股份有限公司 | Apparatus and system for automatically predicting physiological condition from medical image of patient |
US11049294B2 (en) * | 2018-10-02 | 2021-06-29 | Canon Medical Systems Corporation | Activity-dependent, spatially-varying regularization parameter design for regularized image reconstruction |
US11241208B2 (en) * | 2018-11-19 | 2022-02-08 | Terumo Kabushiki Kaisha | Diagnostic method, method for validation of diagnostic method, and treatment method |
US10813612B2 (en) | 2019-01-25 | 2020-10-27 | Cleerly, Inc. | Systems and method of characterizing high risk plaques |
CN110009599A (en) * | 2019-02-01 | 2019-07-12 | 腾讯科技(深圳)有限公司 | Liver masses detection method, device, equipment and storage medium |
EP4003174A4 (en) * | 2019-07-29 | 2022-12-28 | Shenzhen Xpectvision Technology Co., Ltd. | Systems and methods for three-dimensional imaging |
US20220392065A1 (en) | 2020-01-07 | 2022-12-08 | Cleerly, Inc. | Systems, methods, and devices for medical image analysis, diagnosis, risk stratification, decision making and/or disease tracking |
AU2021205821A1 (en) | 2020-01-07 | 2022-07-21 | Cleerly, Inc. | Systems, methods, and devices for medical image analysis, diagnosis, risk stratification, decision making and/or disease tracking |
US11969280B2 (en) | 2020-01-07 | 2024-04-30 | Cleerly, Inc. | Systems, methods, and devices for medical image analysis, diagnosis, risk stratification, decision making and/or disease tracking |
TWI743693B (en) * | 2020-02-27 | 2021-10-21 | 國立陽明交通大學 | Benign tumor development trend assessment system, server computing device thereof and computer readable storage medium |
US11610303B2 (en) * | 2020-03-03 | 2023-03-21 | The University Court Of The University Of Edinburgh | Data processing apparatus and method |
EP4129196A4 (en) * | 2020-03-27 | 2023-08-23 | Terumo Kabushiki Kaisha | Program, information processing method, information processing device, and model generation method |
US11744535B2 (en) * | 2021-03-23 | 2023-09-05 | International Business Machines Corporation | Automated population based assessment of contrast absorption phases |
US20230289963A1 (en) | 2022-03-10 | 2023-09-14 | Cleerly, Inc. | Systems, devices, and methods for non-invasive image-based plaque analysis and risk determination |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8078262B2 (en) * | 2001-04-16 | 2011-12-13 | The Johns Hopkins University | Method for imaging and spectroscopy of tumors and determination of the efficacy of anti-tumor drug therapies |
US7343030B2 (en) * | 2003-08-05 | 2008-03-11 | Imquant, Inc. | Dynamic tumor treatment system |
EP3242270B1 (en) * | 2010-12-20 | 2021-03-03 | Koninklijke Philips N.V. | System and method for automatic generation of initial radiation treatment plans |
CN102509113B (en) * | 2011-11-08 | 2013-04-24 | 浙江大学 | Brain tumor MIB-1 index range detection method |
CN102622749B (en) * | 2012-02-22 | 2014-07-30 | 中国科学院自动化研究所 | Automatic segmentation method of three-dimensional magnetic resonance image of brain structure |
US9408540B2 (en) * | 2012-02-27 | 2016-08-09 | Ovio Technologies, Inc. | Rotatable imaging system |
CN102663436B (en) * | 2012-05-03 | 2014-04-16 | 武汉大学 | Self-adapting characteristic extracting method for optical texture images and synthetic aperture radar (SAR) images |
US9370304B2 (en) * | 2012-06-06 | 2016-06-21 | The Regents Of The University Of Michigan | Subvolume identification for prediction of treatment outcome |
-
2014
- 2014-08-28 US US14/471,501 patent/US9655563B2/en active Active
- 2014-09-24 CN CN201480064452.4A patent/CN105765629B/en active Active
- 2014-09-24 EP EP14793653.8A patent/EP3050026B1/en active Active
- 2014-09-24 WO PCT/US2014/057175 patent/WO2015048103A1/en active Application Filing
Non-Patent Citations (1)
Title |
---|
PETER MADAY ET AL: "Imaging as a Surrogate for the Early Prediction and Assessment of Treatment Response through the Analysis of 4-D Texture Ensembles (ISEPARATE)", 20 September 2010, MEDICAL COMPUTER VISION. RECOGNITION TECHNIQUES AND APPLICATIONS IN MEDICAL IMAGING, SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 164 - 173, ISBN: 978-3-642-18420-8, XP019160105 * |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3293736A1 (en) * | 2016-09-09 | 2018-03-14 | Siemens Healthcare GmbH | Tissue characterization based on machine learning in medical imaging |
US10748277B2 (en) | 2016-09-09 | 2020-08-18 | Siemens Healthcare Gmbh | Tissue characterization based on machine learning in medical imaging |
US10779787B2 (en) * | 2017-08-11 | 2020-09-22 | Siemens Healthcare Gmbh | Method for analyzing image data from a patient after a minimally invasive intervention, analysis apparatus, computer program and electronically readable data storage medium |
Also Published As
Publication number | Publication date |
---|---|
US9655563B2 (en) | 2017-05-23 |
EP3050026B1 (en) | 2021-06-23 |
US20150087957A1 (en) | 2015-03-26 |
CN105765629A (en) | 2016-07-13 |
EP3050026A1 (en) | 2016-08-03 |
CN105765629B (en) | 2019-08-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9655563B2 (en) | Early therapy response assessment of lesions | |
US10255997B2 (en) | Medical analytics system | |
Golan et al. | Lung nodule detection in CT images using deep convolutional neural networks | |
Gegundez-Arias et al. | A new deep learning method for blood vessel segmentation in retinal images based on convolutional kernels and modified U-Net model | |
Messay et al. | Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the lung image database consortium and image database resource initiative dataset | |
Tomita et al. | Automatic post-stroke lesion segmentation on MR images using 3D residual convolutional neural network | |
Raja'S et al. | Labeling of lumbar discs using both pixel-and object-level features with a two-level probabilistic model | |
Kalyan et al. | Artificial neural network application in the diagnosis of disease conditions with liver ultrasound images | |
Sánchez et al. | Contextual computer-aided detection: Improving bright lesion detection in retinal images and coronary calcification identification in CT scans | |
JP2020518915A (en) | System and method for automated fundus image analysis | |
WO2018057714A1 (en) | Systems, methods and media for automatically generating a bone age assessment from a radiograph | |
EP3479349B1 (en) | Change detection in medical images | |
CN112529834A (en) | Spatial distribution of pathological image patterns in 3D image data | |
US20180365876A1 (en) | Method, apparatus and system for spine labeling | |
US20200126221A1 (en) | Computer aided diagnosis system for mild cognitive impairment | |
Ricciardi et al. | A deep learning classifier for digital breast tomosynthesis | |
Hasan | A hybrid approach of using particle swarm optimization and volumetric active contour without edge for segmenting brain tumors in MRI scan | |
Kumar et al. | A lightweight asymmetric U-Net framework for acute ischemic stroke lesion segmentation in CT and CTP images | |
US20230316510A1 (en) | Systems and methods for generating biomarker activation maps | |
Sridhar et al. | Lung Segment Anything Model (LuSAM): A Prompt-integrated Framework for Automated Lung Segmentation on ICU Chest X-Ray Images | |
Kadry et al. | Computer assisted detection of low/high grade nodule from lung CT scan slices using handcrafted features | |
Nanvaee et al. | Hepatocellular Carcinoma Diagnosis Based on Ultrasound Images Using Feature Selection Techniques and K-nearest Neighbor Classifier | |
US20240170151A1 (en) | Interface and deep learning model for lesion annotation, measurement, and phenotype-driven early diagnosis (ampd) | |
Pitale et al. | Machine Learning Algorithms for Classifying Disorder in Brain and Lung Biomedical Images | |
Sumathi et al. | Harnessing Deep Learning (DL) for Image Inpainting in Healthcare System-Methods and Challenges |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14793653 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2014793653 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2014793653 Country of ref document: EP |