US20230419485A1 - Autonomous diagnosis of a disorder in a patient from image analysis - Google Patents
Autonomous diagnosis of a disorder in a patient from image analysis Download PDFInfo
- Publication number
- US20230419485A1 US20230419485A1 US18/367,384 US202318367384A US2023419485A1 US 20230419485 A1 US20230419485 A1 US 20230419485A1 US 202318367384 A US202318367384 A US 202318367384A US 2023419485 A1 US2023419485 A1 US 2023419485A1
- Authority
- US
- United States
- Prior art keywords
- image
- retinal
- training
- sample
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003745 diagnosis Methods 0.000 title claims description 19
- 238000010191 image analysis Methods 0.000 title 1
- 238000000034 method Methods 0.000 claims abstract description 85
- 238000012549 training Methods 0.000 claims abstract description 76
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 63
- 201000010099 disease Diseases 0.000 claims abstract description 58
- 238000013528 artificial neural network Methods 0.000 claims abstract description 51
- 230000004256 retinal image Effects 0.000 claims abstract description 49
- 238000001514 detection method Methods 0.000 claims description 39
- 238000003860 storage Methods 0.000 claims description 25
- 208000037111 Retinal Hemorrhage Diseases 0.000 claims description 12
- 238000003384 imaging method Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 10
- 238000002583 angiography Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 210000001525 retina Anatomy 0.000 claims description 7
- 238000002591 computed tomography Methods 0.000 claims description 6
- 238000010801 machine learning Methods 0.000 claims description 6
- 230000002207 retinal effect Effects 0.000 claims description 6
- 210000000416 exudates and transudate Anatomy 0.000 claims description 5
- 238000002595 magnetic resonance imaging Methods 0.000 claims description 4
- 208000032843 Hemorrhage Diseases 0.000 claims description 3
- 208000009857 Microaneurysm Diseases 0.000 claims description 3
- 238000002573 colposcopy Methods 0.000 claims description 3
- 238000002574 cystoscopy Methods 0.000 claims description 3
- 238000002592 echocardiography Methods 0.000 claims description 3
- 238000001839 endoscopy Methods 0.000 claims description 3
- 238000002357 laparoscopic surgery Methods 0.000 claims description 3
- 230000007576 microinfarct Effects 0.000 claims description 3
- 238000000386 microscopy Methods 0.000 claims description 3
- 238000009206 nuclear medicine Methods 0.000 claims description 3
- 238000012014 optical coherence tomography Methods 0.000 claims description 3
- 238000002600 positron emission tomography Methods 0.000 claims description 3
- 238000002601 radiography Methods 0.000 claims description 3
- 238000002603 single-photon emission computed tomography Methods 0.000 claims description 3
- 238000004611 spectroscopical analysis Methods 0.000 claims description 3
- 238000001931 thermography Methods 0.000 claims description 3
- 239000012502 diagnostic product Substances 0.000 claims 9
- 208000035475 disorder Diseases 0.000 claims 5
- 238000013178 mathematical model Methods 0.000 claims 3
- 229920000742 Cotton Polymers 0.000 claims 2
- 238000013534 fluorescein angiography Methods 0.000 claims 2
- 238000002594 fluoroscopy Methods 0.000 claims 2
- 238000012360 testing method Methods 0.000 abstract description 33
- 238000009826 distribution Methods 0.000 description 24
- 238000013527 convolutional neural network Methods 0.000 description 22
- 230000015654 memory Effects 0.000 description 18
- 238000005070 sampling Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 7
- 208000017442 Retinal disease Diseases 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000003902 lesion Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000003416 augmentation Effects 0.000 description 3
- 238000013434 data augmentation Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000007480 spreading Effects 0.000 description 2
- 238000003892 spreading Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 206010002329 Aneurysm Diseases 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 206010038862 Retinal exudates Diseases 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- GNBHRKFJIUUOQI-UHFFFAOYSA-N fluorescein Chemical compound O1C(=O)C2=CC=CC=C2C21C1=CC=C(O)C=C1OC1=CC(O)=CC=C21 GNBHRKFJIUUOQI-UHFFFAOYSA-N 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 238000009738 saturating Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
Definitions
- NNs Deep neural networks
- Convolutional Neural Networks as well as other types of multilevel neural networks
- NNs are an existing method for improved feature learning, classification, and detection. They have been applied to a wide range of different image types, including medical images).
- image subsets patches
- a system for detecting lesions in a retinal image including: a) a feature extractor, configured to receive retinal image input from a user, and further configured to extract image features from the retinal image, which can be a multilevel neural network or a classic feature extractor; b) a multilevel neural network, trained according to the training methods disclosed herein and configured to receive image features from the feature extractor, and further configured to output scalar image features; and c) a machine learning program, configured to receive scalar image features from the neural network and further configured to output disease diagnosis.
- the image features are spatial distributions of image features.
- spatial distributions are heat maps.
- the spatial distributions are point-wise outputs.
- One general aspect includes a computing device for diagnosing disease in a retinal image including: a) a processor; b) a memory that includes: i) a feature extractor system ii) a neural network; iii) a machine learning program; and iv) instructions that, when executed by the processor, cause the processor to perform acts including: receiving a retinal image and providing the retinal image to the feature extractor; extracting image features and providing image features to the neural network; generating scalar features from the image features; providing scalar features to the machine learning program; and determining whether disease is present in the retinal image.
- Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
- FIG. 1 is a flowchart of disclosed methods, according to certain embodiments.
- FIG. 2 is a flowchart of disclosed methods, according to certain embodiment.
- FIG. 3 is a flowchart of disclosed methods, according to certain, embodiment.
- FIG. 4 is exemplary retinal image and Feature Image, according, to certain embodiments.
- FIG. 5 is exemplary retinal image and Feature Image, according to certain embodiments.
- FIG. 6 is shows sample distribution with and without confounder training, according to certain embodiments.
- FIG. 8 shows exemplary data of disease detection in retinal images, according to certain embodiments.
- An “image” is defined as a reproduction or imitation of the form of a person or thing, or specific characteristics thereof, in digital form.
- An image can be, but is not limited to, a JPEG image, a PNG image, a GIF image, a TIFF image, or any other digital image format known in the art. “Image” is used interchangeably with “photograph.”
- a “feature(s)” is defined as a group of one or more descriptive characteristics of objects that can discriminate for disease.
- a feature can be a numeric attribute.
- a “set of features” is defined as a customized group of one or more descriptive characteristics of objects which most accurately classify objects indicative of disease.
- a “threshold” is defined as a level, point, or value above which something is true or will take place and below which it is not or will not, such levels, points, or values include probabilities, sizes in pixels, and values representing pixel brightness.
- Thesholding is defined as modifying pixels that contain a characteristic either above or below a selected threshold value.
- Train_0 means a set of images that have been reviewed by one or more experts as not having the Feature of Interest anywhere within the image. These images are used exclusively in the training dataset. This data should be selected from a general dataset containing many different sources.
- Test_0 means a set of images that have been reviewed by one or more experts as not having the Feature of Interest anywhere within the image. These images are used exclusively in the testing dataset. This data should be selected from a general dataset containing many different sources.
- Train_1 means a set of images that have the Feature of Interest labeled by an expert. These images are separated for use exclusively in the training dataset. This data should be selected from a general dataset containing many different sources.
- Test_1 means a set of images that have the Feature of Interest labeled by an expert. These images are separated for use exclusively in the testing dataset. This data should be selected from a general dataset containing many different sources.
- Feature of Interest means the object(s) in a retinal image which the multilevel neural network is being trained to recognize.
- the feature of interest is generally a retinal image object that is indicative of disease and may include but is not limited to microaneurysms, dot hemorrhages, flame-shaped hemorrhages, sub-intimal hemorrhages, sub-retinal hemorrhages, pre-retinal hemorrhages, micro-infarctions, cotton-wool spots, and yellow exudates.
- Feature Image means a vector image (or image stack). At the most basic level this may be the color planes of an image (e.g. RGB image would be a Feature Image with vector length of 3). Other Feature Images may include a variety of pre-processed versions of the input image.
- the method further includes the steps of: a) creating a spatial distribution of features in an image; and b) inputting the spatial distribution of features into a neural network trained according to steps (a)-(f), where the output of the neural network is disease diagnosis.
- the spatial distribution of features is a heat map.
- the spatial distribution of features is a point-wise output from feature detection.
- the step of creating a spatial distribution of features in an image further includes creating scalar features.
- the scalar features are created as histograms. The method further including training a machine learning algorithm, where the input is scalar features and the output is disease diagnosis.
- computing device is a remotely accessible server.
- the neural network of the disclosed computing device has been trained with steps including: a) extracting one or more features images from a Train_0 set, a Test_0 set, a Train_1 set and a Test_1 set; b) combining and randomizing the feature images from Train_0 and Train_1 into a training data set; c) combining and randomizing the feature images from test_0 and test_1 into a testing dataset; d) training a plurality of neural networks having different architectures using a subset of the training dataset while testing on a subset of the testing dataset; e) identifying the best neural network based on each of the plurality of neural networks performance on the testing data set; f) inputting images from Test_0, Train_1, Train_0 and Test_1 to the best neural network and identifying a limited number of false positives and false negative and adding the false positives and false negatives to the training dataset and testing dataset; and g) repeating steps d)-g)
- the disclosed systems and methods are applied in a comprehensive automated retinal disease screening system that takes one or more retinal images of a single patient, applies the method to each of the images to detect a variety features related to disease, and then combines the results with the outputs of more classical disease detection methods using a combination method into a single number indicating the likelihood of the presence of disease.
- This disclosure demonstrates a process for creating, training and applying CNNs to retinal images in order to successfully detect features related to disease detection in a way that is useful for retinal disease detection in patients. Further, the same general method is able to combine features for the purpose of diagnosing disease from the detected features.
- the conventional approach is limited by the user's ability to identify features that they deem important to the classical system's method for classification through intuition.
- the instantly disclosed systems and methods allows the system to generate features, some of which may not be intuitive even to the expert, from more general representations of the usefulness for the detection, localization and classification of features, particularly disease features, in the retina.
- Other approaches have used statistical methods to generate such features, for example, by using a maximum variance objective function.
- a maximum variance objective function does not guarantee an optimal solution on a more suitable metric, namely a detection or classification performance metric as is used by the multilevel neural network including convolutional neural networks.
- CT computed tomography
- MRI magnetic resonance imaging
- radiography magnetic resonance
- angioscopy optical coherence tomography
- color flow Doppler cystoscopy
- cystoscopy diaphanography
- echocardiography fluorescein
- angiography laparoscopy
- magnetic resonance angiography positron emission tomography
- single-photon emission computed tomography x-ray angiography
- nuclear medicine biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scan, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluroscopy.
- the following disclosed methods are applied to create and refine a deep NN for feature detection in retinal images.
- Feature Images 110 a - d are extracted from each of Train_0 102 , Test_0 106 , Train_1 104 and Test_1 108 .
- Feature image extraction involves sampling a region of interest (ROI) from the image.
- ROI region of interest
- FIG. 4 shows a representative sample of ROI sampling from Train_0, with the ROI identified as “patch.”
- FIG. 4 shows a representative sample of ROI sampling from Train_1, with the ROI identified as “patch.”
- the limited number of Feature Images 110 are randomly sampled from the image.
- the limited number of Feature Images are structurally sampled, such as by way of grid-sampling, from the image. In the case of images with disease marked as ground truth, target sampling may be used to sample only certain desired points, such as regions marked by experts.
- Feature Image extraction may be constrained such that the entirety of the image is within the field-of-view.
- techniques may be applied to allow for reliable extraction of Feature Images that lie partially outside the field-of-view, such as mirroring pixel intensities across the field-of-view boundary.
- disease features may be extracted and placed on a normalized image to create an enhanced source image.
- a large problem specific to retinal images is variation in the background of the image due to variation in the image acquisition process and anatomical differences between individuals.
- lesions and disease features can be isolated from images containing disease and placed in normal images that exhibit certain kinds of background variation.
- These enhanced source images are then treated as positive image during the Feature Image extraction steps. This disease feature transfer can be accomplished by either image processing methods or by building a (deep) model.
- an additional data augmentation step is performed.
- a data augmentation step may be applied to a set or subset of the sampled patches.
- Exemplary data augmentation steps include but are not limited to modifying the sampled patch in a controlled manner such as flipping, scaling, rotating, translating or saturating a feature(s) within the Feature Image. Saturation of a feature within the Feature Image means substantially reducing the contrast of that channel of the Feature Image. This is especially helpful for retinal images as the red plane in retinal images is occasionally fully saturated.
- Feature Images 110 from Train_0 110 a and Train_1 110 b are combined and randomized, including the Feature Image augmentations (if any) into a training dataset 112 .
- the trained NNs can be utilized for disease detection in retinal images 202 according to at least the following methods, best shown in FIG. 2 .
- spatial distribution of features in the image is created in a variety of ways.
- spatial distribution is created as point-wise outputs from feature detection.
- spatial distribution of features in the image is provided by creation of heat maps. Heat maps are created by spreading the influence of the feature detection at a point over an area (either over a distance or the entirety of the initial Feature Image space).
- the heat maps may be post-processed using techniques such as blurring, sharpening, or other techniques known in the art.
- a NN can be leveraged by inputting the spatial feature map into the NN 206 and outputting a disease diagnosis.
- the confounder training mechanism still applies.
- the trained NNs can be utilized for disease detection in retinal images according to at least the following methods, best shown in FIG. 3 .
- spatial distributions may be created in a variety of ways.
- spatial distribution is created as point-wise outputs from feature detection.
- spatial distribution of features in the image is provided by creation of heat maps.
- Heat maps are created by spreading the influence of the feature detection at a point over an area (either over a distance or the entirety of the initial Feature Image space). In both cases, the heat maps may be post-processed using techniques such as blurring, sharpening, or other techniques known in the art.
- the feature maps can be used to create scalar features 304 using, for instance, histograms.
- the feature map also provides information about the location of the detected disease features, this location information could be included in the extracted scalar features 304 .
- the disclosed method further comprises chaining together a plurality of trained CNN systems.
- each of the trained CNN performs a subtask required for the analysis.
- the outputs of each of the CNNs is then and classifying the outputs together into a single image or patient level output that combines these inputs using a classifier or CNN.
- the conventional approach in the art is to leverage CNNs to tackle the entire problem, which especially with sparse data in medical imaging leads to intractable processing times.
- the task is subdivided into different detection/segmentation/classification tasks that are each trained optimally as provided in the disclosed methods, and are then tied together into a second stage. This can be repeated in multiple stages. Further, this allows, for the use of the same training images for all different modules.
- the processing of the disclosed methods and systems can be performed by software components.
- the disclosed system and method can be described in the general context of computer-executable instructions, such as program modules, being executed by one or more computers or other devices.
- program modules comprise computer code, routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
- the disclosed method can also be practiced in grid-based and distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules can be located in both local and remote computer storage media including memory storage devices.
- the components of the computer 1201 can comprise, but are not limited to, one or more processors or processing units 1203 , a system memory 1212 , and a system bus 1213 that couples various system components including the processor 1203 to the system memory 1212 .
- the system can utilize parallel computing.
- the system bus 1213 represents one or more of several possible types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.
- bus architectures can comprise an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, an Accelerated Graphics Port (AGP) bus, and a Peripheral Component Interconnects (PCI), a PCI-Express bus, a Personal Computer Memory Card Industry Association (PCMCIA), Universal Serial Bus (USB) and the like.
- ISA Industry Standard Architecture
- MCA Micro Channel Architecture
- EISA Enhanced ISA
- VESA Video Electronics Standards Association
- AGP Accelerated Graphics Port
- PCI Peripheral Component Interconnects
- PCI-Express PCI-Express
- PCMCIA Personal Computer Memory Card Industry Association
- USB Universal Serial Bus
- the computer 1201 typically comprises a variety of computer readable media. Exemplary readable media can be any available media that is accessible by the computer 1201 and comprises, for example and not meant to be limiting, both volatile and non-volatile media, removable and non-removable media.
- the system memory 1212 comprises computer readable media in the form of volatile memory, such as random access memory (RAM), and/or non-volatile memory, such as read only memory (ROM).
- RAM random access memory
- ROM read only memory
- the system memory 1212 typically contains data such as imaging data 1207 and/or program modules such as operating system 1205 and imaging software 1206 that are immediately accessible to and/or are presently operated on by the processing unit 1203 .
- any number of program modules can be stored on the mass storage device 12012 , including by way of example, an operating system 1205 and imaging software 1206 .
- Each of the operating system 1205 and imaging software 1206 (or some combination thereof) can comprise elements of the programming and the imaging software 1206 .
- Imaging data 1207 can also be stored on the mass storage device 12012 .
- Imaging data 1207 can be stored in any of one or more databases known in the art. Examples of such databases comprise, DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®, mySQL®, PostgreSQL®, and the like. The databases can be centralized or distributed across multiple systems.
- the user can enter commands and information into the computer 1201 via an input device (not shown).
- input devices comprise, but are not limited to, a keyboard, pointing device (e.g., a “mouse”), a microphone, a joystick, a scanner, tactile input devices such as gloves, and other body coverings, and the like
- a human machine interface 1202 that is coupled to the system bus 1213 , but can be connected by other interface and bus structures, such as a parallel port, game port, an IEEE 13912 Port (also known as a Firewire port), a serial port, or a universal serial bus (USB).
- a display device 1211 can also be connected to the system bus 1213 via an interface, such as a display adapter 1209 .
- the computer 1201 can have more than one display adapter 1209 and the computer 1201 can have more than one display device 1211 .
- a display device can be a monitor, an LCD (Liquid Crystal Display), or a projector.
- other output peripheral devices can comprise components such as speakers (not shown) and a printer (not shown) which can be connected to the computer 1201 via Input/Output Interface 1210 . Any step and/or result of the methods can be output in any form to an output device.
- Such output can be any form of visual representation, including, but not limited to, textual, graphical, animation, audio, tactile, and the like.
- a retinal image acquisition device 101 can be coupled to computer 1201 via Input/Output Interface 1210 .
- fundus camera 100 can transfer images captured to the computer 1201 for analysis and storage.
Abstract
Description
- This application is a divisional application of U.S. application Ser. No. 16/175,318, filed on Oct. 30, 2018, which is a divisional application of U.S. application Ser. No. 15/092,421, filed on Apr. 6, 2016, now U.S. Pat. No. 10,115,194, which claims priority from U.S. Provisional Application No. 62/143,301, filed Apr. 6, 2015, each of which is hereby incorporated herein by reference in its entirety for all purposes.
- Disclosed herein are systems and methods for retinal feature detection, learning, and classification for the purpose of retinal disease detection.
- Detecting, segmenting and classifying objects and other structures of interest in a background retinal, and even any other medical, image is crucial for detection and diagnosis of disease, its absence, as well as for biometrics. Deep neural networks (NNs), including Convolutional Neural Networks, as well as other types of multilevel neural networks, are an existing method for improved feature learning, classification, and detection. They have been applied to a wide range of different image types, including medical images). However, the straightforward use of NNs to classify entire retinal images or image subsets (patches) for these tasks, as is the art in other problems, may not lead to maximized performance in retinal images. This is because the problem is underconstrained: there is a large variance in color, structure, and texture of the normal retina as well as a large diversity of objects, and a large variability in shapes, colors, textures, and other features of these objects, versus a relatively sparse number of retinal images with annotations at a sufficient level, that these machine learning algorithms require for optimal performance. Medical images with annotations are sparse and expensive, because it both ultimately derives from patients so there are ethical concerns that prevent an unlimited number of images to be obtained, as well annotating these images which requires experts in that field, in contrast with other domains especially in computer vision where NNs are typically can be trained on unlimited number of images can be obtained and annotation can typically be done by any adult.
- To tackle this, other approaches have introduced additional variance into the object samples by translating, rotating and otherwise deforming patches or samples artificially. However, this still does not introduce sufficient ‘real’ variance, in both objects, as well as backgrounds. Accordingly, there is a need in the art for a process for creating, training and applying NNs to retinal images in order to maximally successfully detect objects in backgrounds related to disease detection in a way that is useful for retinal disease detection in patients.
- Disclosed herein are systems methods and devices for diagnosing disease in medical images. In certain aspects, disclosed is a method for training a neural network to detect features in a retinal image including the steps of: a) extracting one or more features images from a Train_0 set, a Test_0 set, a Train_1 set and a Test_1 set; b) combining and randomizing the feature images from Train_0 and Train_1 into a Training data set; c) combining and randomizing the feature images from Test_0 and Test_1 into a testing dataset; d) training a plurality of neural networks having different architectures using a subset of the training dataset while testing on a subset of the testing dataset; e) identifying the best neural network based on each of the plurality of neural networks performance on the testing data set; f) inputting images from Test_0, Train_1, Train_0 and Test_1 to the best neural network and identifying a set of false positives (positive confounders) and a set false negatives (negatives confounders) and adding the false positives and false negatives to the training dataset and testing dataset; and g) repeating steps d)-g) until an objective performance threshold is reached. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
- Further disclosed herein is a system for detecting lesions in a retinal image including: a) a feature extractor, configured to receive retinal image input from a user, and further configured to extract image features from the retinal image, which can be a multilevel neural network or a classic feature extractor; b) a multilevel neural network, trained according to the training methods disclosed herein and configured to receive image features from the feature extractor, and further configured to output scalar image features; and c) a machine learning program, configured to receive scalar image features from the neural network and further configured to output disease diagnosis. In certain aspects, the image features are spatial distributions of image features. In further aspects, spatial distributions are heat maps. According to still further aspects, the spatial distributions are point-wise outputs.
- One general aspect includes a computing device for diagnosing disease in a retinal image including: a) a processor; b) a memory that includes: i) a feature extractor system ii) a neural network; iii) a machine learning program; and iv) instructions that, when executed by the processor, cause the processor to perform acts including: receiving a retinal image and providing the retinal image to the feature extractor; extracting image features and providing image features to the neural network; generating scalar features from the image features; providing scalar features to the machine learning program; and determining whether disease is present in the retinal image. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
- While multiple embodiments are disclosed, still other embodiments of the present invention will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative embodiments of the invention. As will be realized, the invention is capable of modifications in various obvious aspects, all without departing from the spirit and scope of the present invention. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.
- The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
-
FIG. 1 is a flowchart of disclosed methods, according to certain embodiments. -
FIG. 2 is a flowchart of disclosed methods, according to certain embodiment. -
FIG. 3 is a flowchart of disclosed methods, according to certain, embodiment. -
FIG. 4 is exemplary retinal image and Feature Image, according, to certain embodiments. -
FIG. 5 is exemplary retinal image and Feature Image, according to certain embodiments. -
FIG. 6 is shows sample distribution with and without confounder training, according to certain embodiments. -
FIG. 7 shows exemplary data of disease detection in retinal images, according to certain embodiments. -
FIG. 8 shows exemplary data of disease detection in retinal images, according to certain embodiments. - Ranges can be expressed herein as from “about” one particular value, and/or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint. It is also understood that there are a number of values disclosed herein, and that each value is also herein disclosed as “about” that particular value in addition to the value itself. For example, if the value “10” is disclosed, then “about 10” is also disclosed. It is also understood that when a value is disclosed that “less than or equal to” the value, “greater than or equal to the value” and possible ranges between values are also disclosed, as appropriately understood by the skilled artisan. For example, if the value “10” is disclosed the “less than or equal to 10” as well as “greater than or equal to 10” is also disclosed. It is also understood that the throughout the application, data are provided in a number of different formats, and that these data, represent endpoints and starting points, and ranges for any combination of the data points. For example, if a particular data point “10” and a particular data point 15 are disclosed, it is understood that greater than, greater than or equal to, less than, less than or equal to, and equal to 10 and 15 are considered disclosed as well as between 10 and 15.
- Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.
- An “image” is defined as a reproduction or imitation of the form of a person or thing, or specific characteristics thereof, in digital form. An image can be, but is not limited to, a JPEG image, a PNG image, a GIF image, a TIFF image, or any other digital image format known in the art. “Image” is used interchangeably with “photograph.”
- An “object” is defined as a tangible entity with defined borders contained within a digital photograph. An object can be a pixel or a group of pixels.
- A “feature(s)” is defined as a group of one or more descriptive characteristics of objects that can discriminate for disease. A feature can be a numeric attribute.
- A “set of features” is defined as a customized group of one or more descriptive characteristics of objects which most accurately classify objects indicative of disease.
- A “threshold” is defined as a level, point, or value above which something is true or will take place and below which it is not or will not, such levels, points, or values include probabilities, sizes in pixels, and values representing pixel brightness.
- “Thresholding” is defined as modifying pixels that contain a characteristic either above or below a selected threshold value.
- As used herein, “Train_0” means a set of images that have been reviewed by one or more experts as not having the Feature of Interest anywhere within the image. These images are used exclusively in the training dataset. This data should be selected from a general dataset containing many different sources.
- As used herein, “Test_0” means a set of images that have been reviewed by one or more experts as not having the Feature of Interest anywhere within the image. These images are used exclusively in the testing dataset. This data should be selected from a general dataset containing many different sources.
- As used herein, “Train_1” means a set of images that have the Feature of Interest labeled by an expert. These images are separated for use exclusively in the training dataset. This data should be selected from a general dataset containing many different sources.
- As used herein, “Test_1” means a set of images that have the Feature of Interest labeled by an expert. These images are separated for use exclusively in the testing dataset. This data should be selected from a general dataset containing many different sources.
- As used herein, “Feature of Interest” means the object(s) in a retinal image which the multilevel neural network is being trained to recognize. The feature of interest is generally a retinal image object that is indicative of disease and may include but is not limited to microaneurysms, dot hemorrhages, flame-shaped hemorrhages, sub-intimal hemorrhages, sub-retinal hemorrhages, pre-retinal hemorrhages, micro-infarctions, cotton-wool spots, and yellow exudates.
- As used herein, “Feature Image” means a vector image (or image stack). At the most basic level this may be the color planes of an image (e.g. RGB image would be a Feature Image with vector length of 3). Other Feature Images may include a variety of pre-processed versions of the input image.
- As used herein, “
Epoch 1” means a complete cycle through a dataset wherein every sample and label is presented to the network for training/testing. - As used herein, “Positive sample” means a sample that contains an instance of what is trying to be detected.
- As used herein, “Negative sample” means a sample that does not contain an instance of what is trying to be detected.
- As used herein, “Confounder Sample” a sample that causes a false-positive or false-negative outcome from a multilevel neural network including convolutional neural networks.
- Disclosed herein are systems methods and devices for diagnosing disease in medical images. In certain aspects, disclosed is a method for training a neural network to detect features in a retinal image including the steps of: a) extracting one or more features images from a Train_0 set, a Test_0 set, a Train_1 set and a Test_1 set; b) combining and randomizing the feature images from Train_0 and Train_1 into a Training data set; c) combining and randomizing the feature images from Test_0 and Test_1 into a testing dataset; d) training a plurality of neural networks having different architectures using a subset of the training dataset while testing on a subset of the testing dataset; e) identifying the best neural network based on each of the plurality of neural networks performance on the testing data set; f) inputting images from Test_0, Train_1, Train_0 and Test_1 to the best neural network and identifying a limited number of false positives and false negative and adding the false positives and false negatives to the training dataset and testing dataset; and g) repeating steps d)-g) until an objective performance threshold is reached. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
- According to certain embodiments, the feature extracting step further includes sampling a region of interest (ROI). In certain aspects, sampling is performed by random sampling. In further aspects, sampling is performed by grid sampling. According to still further aspects, the disclosed method further comprises the step of resampling the images of Train_0, Test_0, Train_1 and Test_1 after the identification of the best neural network with a sampling method different from the sampling method performed in the initial sampling. In yet further aspects, images undergo preprocessing prior to the step of extracting one or more Feature Images. According to still further aspects, the objective performance threshold is softmax loss, mean squared error, area under the ROC curve or accuracy. According to certain aspects, the method further includes the steps of: a) creating a spatial distribution of features in an image; and b) inputting the spatial distribution of features into a neural network trained according to steps (a)-(f), where the output of the neural network is disease diagnosis. According to further aspects, the spatial distribution of features is a heat map. According to yet further aspects, the spatial distribution of features is a point-wise output from feature detection. In certain aspects, the step of creating a spatial distribution of features in an image further includes creating scalar features. In certain aspects, the scalar features are created as histograms. The method further including training a machine learning algorithm, where the input is scalar features and the output is disease diagnosis.
- Further disclosed herein is a system for detecting lesions in a medical image including: a) a feature extractor, configured receive retinal image input from a user, and further configured to extract image features from the retinal image; b) a neural network, trained according to the training methods disclosed herein and configured to receive image features from the feature extractor, and further configured to output scalar image features; and c) a machine learning program, configured to receive scalar image features from the neural network and further configured to output disease diagnosis. In certain aspects, the image features are spatial distributions of image features. In further aspects, spatial distributions are heat maps. According to still further aspects, the spatial distributions are point-wise outputs.
- One general aspect includes a computing device for diagnosing disease in a medical image including: a) a processor; b) a memory that includes: i) a feature extractor system ii) a neural network; iii) a machine learning program; and iv) instructions that, when executed by the processor, cause the processor to perform acts including: receiving a medical image and providing the medical image to the feature extractor; extracting image features and providing image features to the neural network; generating scalar features from the image features; providing scalar features to the machine learning program; and determining whether disease is present in the medical image. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
- In certain aspects computing device is a remotely accessible server. In further aspects, the neural network of the disclosed computing device has been trained with steps including: a) extracting one or more features images from a Train_0 set, a Test_0 set, a Train_1 set and a Test_1 set; b) combining and randomizing the feature images from Train_0 and Train_1 into a training data set; c) combining and randomizing the feature images from test_0 and test_1 into a testing dataset; d) training a plurality of neural networks having different architectures using a subset of the training dataset while testing on a subset of the testing dataset; e) identifying the best neural network based on each of the plurality of neural networks performance on the testing data set; f) inputting images from Test_0, Train_1, Train_0 and Test_1 to the best neural network and identifying a limited number of false positives and false negative and adding the false positives and false negatives to the training dataset and testing dataset; and g) repeating steps d)-g) until an objective performance threshold is reached. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
- Disclosed herein are systems and methods which are applied to medical images. In certain embodiments, the disclosed systems and methods are applied in a comprehensive automated retinal disease screening system that takes one or more retinal images of a single patient, applies the method to each of the images to detect a variety features related to disease, and then combines the results with the outputs of more classical disease detection methods using a combination method into a single number indicating the likelihood of the presence of disease. This disclosure demonstrates a process for creating, training and applying CNNs to retinal images in order to successfully detect features related to disease detection in a way that is useful for retinal disease detection in patients. Further, the same general method is able to combine features for the purpose of diagnosing disease from the detected features.
- Retinal lesion detection algorithms using a classic detection pipeline, wherein image information is processed by a candidate detector; then a candidate segmentor; then a feature extractor; then a classifier, are well known. The systems and methods disclosed herein are an improvement on classic detection methods. The disclosed system provides for the application of a feature learning and detection/classification method, deep neural networks to the detection and localization of features relating to retinal disease in retinal images. The amount of data normally required to obtain good results from deep networks is challenging in medical images, the disclosed systems and methods implement several innovations to make deep learning on medical images with limited ground-truth maximally effective.
- The conventional approach is limited by the user's ability to identify features that they deem important to the classical system's method for classification through intuition. The instantly disclosed systems and methods allows the system to generate features, some of which may not be intuitive even to the expert, from more general representations of the usefulness for the detection, localization and classification of features, particularly disease features, in the retina. Other approaches have used statistical methods to generate such features, for example, by using a maximum variance objective function. However such an objective function does not guarantee an optimal solution on a more suitable metric, namely a detection or classification performance metric as is used by the multilevel neural network including convolutional neural networks.
- The iterative nature used for dataset generation with positive and negative confounders allows for the system's influences to be updated with less common and more difficult to classify samples. This is a critical step for the application to retinal images that, by their nature, vary widely in appearance due to anatomical differences between subjects. Further variation is added by differences between cameras and operator skill levels. Since only a limited set of manually, expert annotated data is available, this step is critical in order to achieve generalizable results on large datasets.
- The aspects of the disclosed method that set it apart from current systems and processes known in the art include, but are not limited to: 1) the steps needed to extract positive and negative samples; 2) the way in which samples are augmented within the dataset to achieve satisfactory detection results; 3) the way the datasets are iteratively updated to increase the influence of less common confounders in the training process and 4) the way in which detection and classification results are combined for patient level disease diagnosis; 5) the dataset produced in the final iteration of this invention is more suitable for discovery and training of other CNN architectures by relative over-representation of confounder samples.
- The embodiments described herein relate generally to diagnostic medical images. Although any type of medical image can be used, these embodiments will be illustrated in conjunction with retinal images. However, the disclosed methods systems and devices can also be used with medical images of other ocular structures, or any other biological tissues the image of which can support the diagnosis of a disease condition. Furthermore, the methods disclose herein can be used with a variety of imaging modalities including but not limited to: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein, angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scan, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluroscopy.
- The following disclosed methods are applied to create and refine a deep NN for feature detection in retinal images.
-
- 1. Feature Image Extraction
- As shown in the flow chart of
FIG. 1 , a limited number of Feature Images 110 a-d are extracted from each ofTrain_0 102,Test_0 106,Train_1 104 andTest_1 108. Feature image extraction involves sampling a region of interest (ROI) from the image.FIG. 4 shows a representative sample of ROI sampling from Train_0, with the ROI identified as “patch.”FIG. 4 shows a representative sample of ROI sampling from Train_1, with the ROI identified as “patch.” According to certain embodiments, the limited number of Feature Images 110 are randomly sampled from the image. According to further embodiments, the limited number of Feature Images are structurally sampled, such as by way of grid-sampling, from the image. In the case of images with disease marked as ground truth, target sampling may be used to sample only certain desired points, such as regions marked by experts. - In certain implementations, Feature Image extraction may be constrained such that the entirety of the image is within the field-of-view. Alternatively, techniques may be applied to allow for reliable extraction of Feature Images that lie partially outside the field-of-view, such as mirroring pixel intensities across the field-of-view boundary.
- According to certain embodiments, the Feature Images may undergo preprocessing, prior to feature extraction. The preprocessing step may comprise steps such as image normalization, background normalization, or color plane manipulation. Other preprocessing steps known in the art are also possible.
- According to certain implementations, disease features may be extracted and placed on a normalized image to create an enhanced source image. A large problem specific to retinal images is variation in the background of the image due to variation in the image acquisition process and anatomical differences between individuals. To increase the robustness of the trained NN against such influences, lesions and disease features can be isolated from images containing disease and placed in normal images that exhibit certain kinds of background variation. These enhanced source images are then treated as positive image during the Feature Image extraction steps. This disease feature transfer can be accomplished by either image processing methods or by building a (deep) model.
- Importantly, if the source image contains disease features, only positive samples are extracted. This is important as it removes the necessity to identify every disease feature in the image.
- Several methods may be used to determine if a Feature Image has disease in the image using a marked ground truth. For example, if at least N points in the central x % of the sample patch is marked as disease, then a positive determination is made. Values for N can be 1 or higher and values of x can be any value greater than 0.
- According to certain implementations, an additional data augmentation step is performed. In these implementations, a data augmentation step may be applied to a set or subset of the sampled patches. Exemplary data augmentation steps include but are not limited to modifying the sampled patch in a controlled manner such as flipping, scaling, rotating, translating or saturating a feature(s) within the Feature Image. Saturation of a feature within the Feature Image means substantially reducing the contrast of that channel of the Feature Image. This is especially helpful for retinal images as the red plane in retinal images is occasionally fully saturated.
-
- 2. Combine and randomize the feature images from
Train_0 102 andTrain_1 104
- 2. Combine and randomize the feature images from
- Following the Feature Image extraction step, Feature Images 110 from
Train_0 110 a andTrain_1 110 b are combined and randomized, including the Feature Image augmentations (if any) into atraining dataset 112. -
- 3. Combine and randomize the Feature Images from
Test_0 110 c andTest_1 110 d, including Feature Image augmentations, into atesting dataset 114. - 4. Train a plurality of
NNs 116 with different architectures using (part of) thetraining dataset 112 while testing on (part of) thetesting dataset 114.
- 3. Combine and randomize the Feature Images from
- Once the
training dataset 112 and thetesting dataset 114 are defined, they are used to train and test a plurality ofNNs 116. The architectures of the plurality of NNs may be predefined, or iteratively defined using an optimization algorithm such as provided by a hyperparameter optimization method. In certain embodiments, the initial training runs are performed with only a subset of the training dataset and testing dataset. -
- 5. Select the best NN(s).
- The selection of the best NN is based on a pre-defined objective function, e.g. on the ability to separate the positive from negative samples within the
test dataset 114. Values that may be an indicator of class separation include softmax loss, mean squared error, area under the ROC curve or accuracy. -
- 6. Run the Best NN(s) on the
Original Train_0 102,Test_0 106,Train_1 104 andTest_1 108.
- 6. Run the Best NN(s) on the
- In certain implementations, following the selection of the best NN(s) 118, those selected can then be run on the
original Train_0 102,Test_0 106,Train_1 104 andTest_1 108. A limited number of false positives and/or false negatives, known as positive and negative confounders are selected to add to the training set and testing set, as no single example should exist in either sets. While adding data, it is important to randomize the samples for more effective NN optimization. - According to certain embodiments, the sampling scheme can be changed from the initial sampling method. This provides an effective method to find positive and negative confounders. For instance, if grid sampling was originally used, off-setting the grid will make it so samples are not duplicated in the train or test set. Augmentation of the confounder samples using methods defined above is effective to increase the number of confounder samples in the training set which in turn increases their influence on the NN training. This can be effective to increase the influence of confounder samples even if they are uncommon.
-
- 7. (for confounder training) Re-run steps 5-7 as many times as necessary to obtain satisfactory results on some performance measure, e.g. good results for the final patient level disease classification.
- Leveraging NN Feature Detection for Disease Detection in Retinal Images
- Once one or more of the plurality of NNs are trained according to the methods disclosed above, the trained NNs can be utilized for disease detection in
retinal images 202 according to at least the following methods, best shown inFIG. 2 . -
- 1. Spatial Distribution of Features in an Image
- For all features, feature detection of
images 204 is performed to get a spatial distribution of features in the image. These spatial distributions may be created in a variety of ways. In certain embodiments, spatial distribution is created as point-wise outputs from feature detection. In certain alternative embodiments, spatial distribution of features in the image is provided by creation of heat maps. Heat maps are created by spreading the influence of the feature detection at a point over an area (either over a distance or the entirety of the initial Feature Image space). - In both cases, the heat maps may be post-processed using techniques such as blurring, sharpening, or other techniques known in the art.
-
- 2. Train a
NN 206 with input using the same procedure as above where the input is, in part, the spatial feature maps and the output is adisease diagnosis 210.
- 2. Train a
- Once the spatial distribution of features in an image is established, a NN can be leveraged by inputting the spatial feature map into the
NN 206 and outputting a disease diagnosis. The confounder training mechanism still applies. - According to certain alternative embodiments, once one or more of the plurality of NNs are trained according to the methods disclosed above, the trained NNs can be utilized for disease detection in retinal images according to at least the following methods, best shown in
FIG. 3 . -
- 1. For all features, perform feature detection of
images 302 to get a spatial distribution of features in the image.
- 1. For all features, perform feature detection of
- These spatial distributions may be created in a variety of ways. In certain embodiments, spatial distribution is created as point-wise outputs from feature detection. In certain alternative embodiments, spatial distribution of features in the image is provided by creation of heat maps. Heat maps are created by spreading the influence of the feature detection at a point over an area (either over a distance or the entirety of the initial Feature Image space). In both cases, the heat maps may be post-processed using techniques such as blurring, sharpening, or other techniques known in the art.
-
- 2. Create scalar features from the spatial feature maps
- Once the spatial feature maps are created, the feature maps can be used to create
scalar features 304 using, for instance, histograms. As the feature map also provides information about the location of the detected disease features, this location information could be included in the extracted scalar features 304. In certain embodiments, it may be desirable to skipstep 1 and simply convert the output of feature detection directly into scalar features, bypassing the creation of a spatial feature map. Regardless of whether or not a feature map is created, the location information stays available. -
- 3. Train a conventional machine learning algorithm where the output is a
disease diagnosis 306 or staging.
- 3. Train a conventional machine learning algorithm where the output is a
- According to further embodiments, the disclosed method further comprises chaining together a plurality of trained CNN systems. In these embodiments, each of the trained CNN performs a subtask required for the analysis. The outputs of each of the CNNs is then and classifying the outputs together into a single image or patient level output that combines these inputs using a classifier or CNN. The conventional approach in the art is to leverage CNNs to tackle the entire problem, which especially with sparse data in medical imaging leads to intractable processing times. Here however, the task is subdivided into different detection/segmentation/classification tasks that are each trained optimally as provided in the disclosed methods, and are then tied together into a second stage. This can be repeated in multiple stages. Further, this allows, for the use of the same training images for all different modules.
- The present methods and systems can be operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that can be suitable for use with the system and method comprise, but are not limited to, personal computers, server computers, laptop devices, and multiprocessor systems. Additional examples comprise set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that comprise any of the above systems or devices, and the like.
- The processing of the disclosed methods and systems can be performed by software components. The disclosed system and method can be described in the general context of computer-executable instructions, such as program modules, being executed by one or more computers or other devices. Generally, program modules comprise computer code, routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The disclosed method can also be practiced in grid-based and distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.
- Further, one skilled in the art will appreciate that the systems and methods disclosed herein can be implemented via a computing device, best shown in
FIG. 6 , in the form of acomputer 1201. The components of thecomputer 1201 can comprise, but are not limited to, one or more processors orprocessing units 1203, asystem memory 1212, and a system bus 1213 that couples various system components including theprocessor 1203 to thesystem memory 1212. In the case ofmultiple processing units 1203, the system can utilize parallel computing. - The system bus 1213 represents one or more of several possible types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures can comprise an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, an Accelerated Graphics Port (AGP) bus, and a Peripheral Component Interconnects (PCI), a PCI-Express bus, a Personal Computer Memory Card Industry Association (PCMCIA), Universal Serial Bus (USB) and the like. The bus 1213, and all buses specified in this description can also be implemented over a wired or wireless network connection and each of the subsystems, including the
processor 1203, a mass storage device 12012, anoperating system 1205,imaging software 1206,imaging data 1207, anetwork adapter 1208,system memory 1212, an Input/Output Interface 1210, adisplay adapter 1209, adisplay device 1211, and ahuman machine interface 1202, can be contained within one or more remote computing devices 1214 a,b,c at physically separate locations, connected through buses of this form, in effect implementing a fully distributed system. - The
computer 1201 typically comprises a variety of computer readable media. Exemplary readable media can be any available media that is accessible by thecomputer 1201 and comprises, for example and not meant to be limiting, both volatile and non-volatile media, removable and non-removable media. Thesystem memory 1212 comprises computer readable media in the form of volatile memory, such as random access memory (RAM), and/or non-volatile memory, such as read only memory (ROM). Thesystem memory 1212 typically contains data such asimaging data 1207 and/or program modules such asoperating system 1205 andimaging software 1206 that are immediately accessible to and/or are presently operated on by theprocessing unit 1203. - In another aspect, the
computer 1201 can also comprise other removable/non-removable, volatile/non-volatile computer storage media. By way of example,FIG. 6 illustrates a mass storage device 12012 which can provide non-volatile storage of computer code, computer readable instructions, data structures, program modules, and other data for thecomputer 1201. For example and not meant to be limiting, a mass storage device 12012 can be a hard disk, a removable magnetic disk, a removable optical disk, magnetic cassettes or other magnetic storage devices, flash memory cards, CD-ROM, digital versatile disks (DVD) or other optical storage, random access memories (RAM), read only memories (ROM), electrically erasable programmable read-only memory (EEPROM), and the like. - Optionally, any number of program modules can be stored on the mass storage device 12012, including by way of example, an
operating system 1205 andimaging software 1206. Each of theoperating system 1205 and imaging software 1206 (or some combination thereof) can comprise elements of the programming and theimaging software 1206.Imaging data 1207 can also be stored on the mass storage device 12012.Imaging data 1207 can be stored in any of one or more databases known in the art. Examples of such databases comprise, DB2®, Microsoft® Access, Microsoft® SQL Server, Oracle®, mySQL®, PostgreSQL®, and the like. The databases can be centralized or distributed across multiple systems. - In another aspect, the user can enter commands and information into the
computer 1201 via an input device (not shown). Examples of such input devices comprise, but are not limited to, a keyboard, pointing device (e.g., a “mouse”), a microphone, a joystick, a scanner, tactile input devices such as gloves, and other body coverings, and the like These and other input devices can be connected to theprocessing unit 1203 via ahuman machine interface 1202 that is coupled to the system bus 1213, but can be connected by other interface and bus structures, such as a parallel port, game port, an IEEE 13912 Port (also known as a Firewire port), a serial port, or a universal serial bus (USB). - In yet another aspect, a
display device 1211 can also be connected to the system bus 1213 via an interface, such as adisplay adapter 1209. It is contemplated that thecomputer 1201 can have more than onedisplay adapter 1209 and thecomputer 1201 can have more than onedisplay device 1211. For example, a display device can be a monitor, an LCD (Liquid Crystal Display), or a projector. In addition to thedisplay device 1211, other output peripheral devices can comprise components such as speakers (not shown) and a printer (not shown) which can be connected to thecomputer 1201 via Input/Output Interface 1210. Any step and/or result of the methods can be output in any form to an output device. Such output can be any form of visual representation, including, but not limited to, textual, graphical, animation, audio, tactile, and the like. In an aspect, a retinal image acquisition device 101 can be coupled tocomputer 1201 via Input/Output Interface 1210. For example,fundus camera 100 can transfer images captured to thecomputer 1201 for analysis and storage. In further embodiments, input images earlier acquired a stored on a hard drive or other storage media and are uploaded by way of the Input/Output Interface. - The
computer 1201 can operate in a networked environment using logical connections to one or more remote computing devices 1214 a,b,c. By way of example, a remote computing device can be a personal computer, portable computer, a server, a router, a network computer, a peer device or other common network node, and so on. Logical connections between thecomputer 1201 and a remote computing device 1214 a,b,c can be made via a local area network (LAN) and a general wide area network (WAN). Such network connections can be through anetwork adapter 1208. Anetwork adapter 1208 can be implemented in both wired and wireless environments. Such networking environments are conventional and commonplace in offices, enterprise-wide computer networks, intranets, and theInternet 1215. - For purposes of illustration, application programs and other executable program components such as the
operating system 1205 are illustrated herein as discrete blocks, although it is recognized that such programs and components reside at various times in different storage components of thecomputing device 1201, and are executed by the data processor(s) of the computer. An implementation ofimaging software 1206 can be stored on or transmitted across some form of computer readable media. Any of the disclosed methods can be performed by computer readable instructions embodied on computer readable media. Computer readable media can be any available media that can be accessed by a computer. By way of example and not meant to be limiting, computer readable media can comprise “computer storage media” and “communications media.” “Computer storage media” comprise volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, or other data. Exemplary computer storage media comprises, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. - The following examples are put forth so as to provide those of ordinary skill in the art with a complete disclosure and description of certain examples of how the compounds, compositions, articles, devices and/or methods claimed herein are made and evaluated, and are intended to be purely exemplary of the invention and are not intended to limit the scope of what the inventors regard as their invention. However, those of skill in the art should, in light of the present disclosure, appreciate that many changes can be made in the specific embodiments which are disclosed and still obtain a like or similar result without departing from the spirit and scope of the invention.
-
FIG. 7 shows sample distribution with and without confounder training. As can be seen, the distribution of both the positive and negative samples improve (more negative samples pulled left and positive samples pushed right). At a threshold of 0.8, ˜33.5 samples marked as false positive would be expected. After confounder training, only 19.3 samples are marked as false positive at same threshold. This is done by artificially making the hardest 0.5% of the population 7% of the training population which causes the CNN to focus the training on the hard to train samples. - A confounder sample is a sample that causes a false-positive or false-negative outcome from a CNN. They are difficult to classify correctly because confounder samples often look more like the object that the CNN is detecting versus the background, for example a flash artifact having similar appearance to exudates in a retinal image).
- Conventional wisdom in the field suggests that to improve CNN training performance, use more data. This is, in general, a good rule. However, the difficulty with resolving confounder samples is that the features causing the sample to be a confounder are rare within the dataset (again, for example flash artifacts in retinal images). Their rarity means that they have little influence on the CNN optimization. This patent claims that to improve performance on confounders, one should increase their presence in the training data to a level much higher than their representation in the whole of the training data. By inflating their relative presence in the training data, one is effectively increasing their influence on the CNN optimization.
- To illustrate this, an experiment was run using a CNN that detects the presence of exudates in a retinal image. First, a CNN was trained using a random sampling of samples from the training data. The CNN was then applied on the training data to find false positive samples, positive confounders; these samples were added to the training data from the original training to create the “confounder-enhanced training dataset.” To compare, additional samples were randomly added to the training dataset to create the “random-enhanced training dataset.” The CNN was then retrained against the confounder-enhanced training dataset and the random-enhanced training dataset to compare the effects of confounder training versus random addition of data to the training dataset.
- The results (shown in
FIG. 8 ) illustrate that after random addition of samples, the specificity (given a sensitivity of 99%) was estimated at 98.36%. Using the confounder-enhanced training dataset, the specificity (given a sensitivity of 99%) was estimated at 98.64%. This is a reduction in the false-positive noise by 17.1% for classifications at the pixel-level. - To illustrate that this can enable better image-level classifications, images in a test set were stratified into disease and no-disease based on the maximum response of the CNN in the image (results were spatially blurred to remove isolated maxima). An ROC was created from the maximum response values. The AUC for the confounder-enhanced training data was 0.886 versus 0.869 for the random-enhanced training data.
- Although the present invention has been described with reference to preferred embodiments, persons skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the invention.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/367,384 US20230419485A1 (en) | 2015-04-06 | 2023-09-12 | Autonomous diagnosis of a disorder in a patient from image analysis |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201562143301P | 2015-04-06 | 2015-04-06 | |
US15/092,421 US10115194B2 (en) | 2015-04-06 | 2016-04-06 | Systems and methods for feature detection in retinal images |
US16/175,318 US11790523B2 (en) | 2015-04-06 | 2018-10-30 | Autonomous diagnosis of a disorder in a patient from image analysis |
US18/367,384 US20230419485A1 (en) | 2015-04-06 | 2023-09-12 | Autonomous diagnosis of a disorder in a patient from image analysis |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/175,318 Division US11790523B2 (en) | 2015-04-06 | 2018-10-30 | Autonomous diagnosis of a disorder in a patient from image analysis |
Publications (1)
Publication Number | Publication Date |
---|---|
US20230419485A1 true US20230419485A1 (en) | 2023-12-28 |
Family
ID=57017339
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/092,421 Active 2036-09-16 US10115194B2 (en) | 2015-04-06 | 2016-04-06 | Systems and methods for feature detection in retinal images |
US16/175,318 Active 2037-02-18 US11790523B2 (en) | 2015-04-06 | 2018-10-30 | Autonomous diagnosis of a disorder in a patient from image analysis |
US18/367,384 Pending US20230419485A1 (en) | 2015-04-06 | 2023-09-12 | Autonomous diagnosis of a disorder in a patient from image analysis |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/092,421 Active 2036-09-16 US10115194B2 (en) | 2015-04-06 | 2016-04-06 | Systems and methods for feature detection in retinal images |
US16/175,318 Active 2037-02-18 US11790523B2 (en) | 2015-04-06 | 2018-10-30 | Autonomous diagnosis of a disorder in a patient from image analysis |
Country Status (1)
Country | Link |
---|---|
US (3) | US10115194B2 (en) |
Families Citing this family (78)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11481411B2 (en) * | 2010-09-01 | 2022-10-25 | Apixio, Inc. | Systems and methods for automated generation classifiers |
US10140699B2 (en) | 2010-12-07 | 2018-11-27 | University Of Iowa Research Foundation | Optimal, user-friendly, object background separation |
AU2014210579B2 (en) | 2014-07-09 | 2019-10-10 | Baylor College Of Medicine | Providing information to a user through somatosensory feedback |
US10115194B2 (en) * | 2015-04-06 | 2018-10-30 | IDx, LLC | Systems and methods for feature detection in retinal images |
CN108603922A (en) * | 2015-11-29 | 2018-09-28 | 阿特瑞斯公司 | Automatic cardiac volume is divided |
US10860879B2 (en) * | 2016-05-16 | 2020-12-08 | Raytheon Technologies Corporation | Deep convolutional neural networks for crack detection from image data |
US10765314B2 (en) | 2016-05-29 | 2020-09-08 | Novasight Ltd. | Display system and method |
US20180012359A1 (en) * | 2016-07-06 | 2018-01-11 | Marinko Venci Sarunic | Systems and Methods for Automated Image Classification and Segmentation |
US10699538B2 (en) | 2016-07-27 | 2020-06-30 | Neosensory, Inc. | Method and system for determining and providing sensory experiences |
WO2018048907A1 (en) | 2016-09-06 | 2018-03-15 | Neosensory, Inc. C/O Tmc+260 | Method and system for providing adjunct sensory information to a user |
WO2018107371A1 (en) * | 2016-12-13 | 2018-06-21 | 上海联影医疗科技有限公司 | Image searching system and method |
CN106803247B (en) * | 2016-12-13 | 2021-01-22 | 上海交通大学 | Microangioma image identification method based on multistage screening convolutional neural network |
US10902598B2 (en) | 2017-01-27 | 2021-01-26 | Arterys Inc. | Automated segmentation utilizing fully convolutional networks |
US10660576B2 (en) | 2017-01-30 | 2020-05-26 | Cognizant Technology Solutions India Pvt. Ltd. | System and method for detecting retinopathy |
WO2018151770A1 (en) | 2017-02-16 | 2018-08-23 | Neosensory, Inc. | Method and system for transforming language inputs into haptic outputs |
US10744058B2 (en) | 2017-04-20 | 2020-08-18 | Neosensory, Inc. | Method and system for providing information to a user |
CN107203778A (en) * | 2017-05-05 | 2017-09-26 | 平安科技(深圳)有限公司 | PVR intensity grade detecting system and method |
CN107230221B (en) * | 2017-05-25 | 2019-07-09 | 武汉理工大学 | One kind being based on convolutional neural networks adaptive background modeling object detecting method |
CN109003252A (en) * | 2017-05-30 | 2018-12-14 | 正凯人工智能私人有限公司 | Image processing method and system |
CN107529645B (en) * | 2017-06-29 | 2019-09-10 | 重庆邮电大学 | A kind of heart sound intelligent diagnosis system and method based on deep learning |
CN107346430A (en) * | 2017-06-29 | 2017-11-14 | 济南浪潮高新科技投资发展有限公司 | A kind of nuclear magnetic resonance image auto-screening method |
CN109390053B (en) * | 2017-08-02 | 2021-01-08 | 上海市第六人民医院 | Fundus image processing method, fundus image processing apparatus, computer device, and storage medium |
US11200665B2 (en) | 2017-08-02 | 2021-12-14 | Shanghai Sixth People's Hospital | Fundus image processing method, computer apparatus, and storage medium |
KR101977645B1 (en) | 2017-08-25 | 2019-06-12 | 주식회사 메디웨일 | Eye image analysis method |
US10606982B2 (en) * | 2017-09-06 | 2020-03-31 | International Business Machines Corporation | Iterative semi-automatic annotation for workload reduction in medical image labeling |
US10565708B2 (en) | 2017-09-06 | 2020-02-18 | International Business Machines Corporation | Disease detection algorithms trainable with small number of positive samples |
CN107507184A (en) * | 2017-09-26 | 2017-12-22 | 上海辉明软件有限公司 | Method for building up, device and the electronic equipment of focus model |
AU2018347610A1 (en) * | 2017-10-13 | 2020-05-07 | Ai Technologies Inc. | Deep learning-based diagnosis and referral of ophthalmic diseases and disorders |
CN108122265A (en) * | 2017-11-13 | 2018-06-05 | 深圳先进技术研究院 | A kind of CT reconstruction images optimization method and system |
CN108053456A (en) * | 2017-11-13 | 2018-05-18 | 深圳先进技术研究院 | A kind of PET reconstruction images optimization method and system |
WO2019103912A2 (en) | 2017-11-22 | 2019-05-31 | Arterys Inc. | Content based image retrieval for lesion analysis |
CN108230296B (en) * | 2017-11-30 | 2023-04-07 | 腾讯科技(深圳)有限公司 | Image feature recognition method and device, storage medium and electronic device |
CN107993229B (en) * | 2017-12-15 | 2021-11-19 | 中科微光医疗研究中心(西安)有限公司 | Tissue classification method and device based on cardiovascular IVOCT image |
US20190034716A1 (en) | 2017-12-28 | 2019-01-31 | Intel Corporation | Privacy-preserving demographics identification |
US11132797B2 (en) * | 2017-12-28 | 2021-09-28 | Topcon Corporation | Automatically identifying regions of interest of an object from horizontal images using a machine learning guided imaging system |
CN112384947A (en) * | 2017-12-28 | 2021-02-19 | 英特尔公司 | Visual fog |
WO2019200535A1 (en) * | 2018-04-17 | 2019-10-24 | 深圳华大生命科学研究院 | Artificial intelligence-based ophthalmic disease diagnostic modeling method, apparatus, and system |
CN108876776B (en) * | 2018-06-13 | 2021-08-24 | 东软集团股份有限公司 | Classification model generation method, fundus image classification method and device |
US11389131B2 (en) | 2018-06-27 | 2022-07-19 | Denti.Ai Technology Inc. | Systems and methods for processing of dental images |
CN108921092B (en) * | 2018-07-02 | 2021-12-17 | 浙江工业大学 | Melanoma classification method based on convolution neural network model secondary integration |
CN109003264B (en) * | 2018-07-05 | 2022-05-06 | 腾讯医疗健康(深圳)有限公司 | Retinopathy image type identification method and device and storage medium |
KR102301058B1 (en) * | 2018-08-24 | 2021-09-10 | 주식회사 메디웨일 | Diagnosis assistance system and control method thereof |
US11023785B2 (en) | 2018-07-23 | 2021-06-01 | International Business Machines Corporation | Sparse MRI data collection and classification using machine learning |
WO2020028726A1 (en) | 2018-08-01 | 2020-02-06 | Idx Technologies, Inc. | Autonomous diagnosis of ear diseases from biomarker data |
CN110163236B (en) * | 2018-10-15 | 2023-08-29 | 腾讯科技(深圳)有限公司 | Model training method and device, storage medium and electronic device |
CN109447149B (en) * | 2018-10-25 | 2021-11-02 | 腾讯医疗健康(深圳)有限公司 | Training method and device for detection model and terminal equipment |
CN109542745B (en) * | 2018-11-20 | 2021-11-19 | 郑州云海信息技术有限公司 | IO test method, device, equipment and medium |
US11282198B2 (en) * | 2018-11-21 | 2022-03-22 | Enlitic, Inc. | Heat map generating system and methods for use therewith |
US10354205B1 (en) | 2018-11-29 | 2019-07-16 | Capital One Services, Llc | Machine learning system and apparatus for sampling labelled data |
KR102333670B1 (en) * | 2018-12-06 | 2021-12-02 | 주식회사 메디웨일 | Diagnostic auxiliary image providing device based on eye image |
CN109658466B (en) * | 2018-12-10 | 2023-02-24 | 苏州大学 | Disease retina optical coherence tomography image simulation generation method |
US10963757B2 (en) | 2018-12-14 | 2021-03-30 | Industrial Technology Research Institute | Neural network model fusion method and electronic device using the same |
CN109816002B (en) * | 2019-01-11 | 2022-09-06 | 广东工业大学 | Single sparse self-encoder weak and small target detection method based on feature self-migration |
CN109902734A (en) * | 2019-02-22 | 2019-06-18 | 中国石油大学(华东) | A kind of cholelithiasis CT medical image method for quickly identifying based on YOLO convolutional neural networks |
EP3719807A1 (en) * | 2019-04-04 | 2020-10-07 | Optos PLC | Predicting a pathological condition from a medical image |
CN110197493B (en) * | 2019-05-24 | 2021-04-23 | 清华大学深圳研究生院 | Fundus image blood vessel segmentation method |
CN110263755B (en) * | 2019-06-28 | 2021-04-27 | 上海鹰瞳医疗科技有限公司 | Eye ground image recognition model training method, eye ground image recognition method and eye ground image recognition device |
US11676701B2 (en) | 2019-09-05 | 2023-06-13 | Pearl Inc. | Systems and methods for automated medical image analysis |
US10984529B2 (en) | 2019-09-05 | 2021-04-20 | Pearl Inc. | Systems and methods for automated medical image annotation |
US11506771B2 (en) * | 2019-09-24 | 2022-11-22 | GE Precision Healthcare LLC | System and methods for flash suppression in ultrasound imaging |
US11467667B2 (en) | 2019-09-25 | 2022-10-11 | Neosensory, Inc. | System and method for haptic stimulation |
US11467668B2 (en) | 2019-10-21 | 2022-10-11 | Neosensory, Inc. | System and method for representing virtual object information with haptic stimulation |
CN110969191B (en) * | 2019-11-07 | 2022-10-25 | 吉林大学 | Glaucoma prevalence probability prediction method based on similarity maintenance metric learning method |
CN110870770B (en) * | 2019-11-21 | 2021-05-11 | 大连理工大学 | ICA-CNN classified fMRI space activation map smoothing and broadening method |
CN110916661B (en) * | 2019-11-21 | 2021-06-08 | 大连理工大学 | ICA-CNN classified fMRI intracerebral data time pre-filtering and amplifying method |
US11079854B2 (en) | 2020-01-07 | 2021-08-03 | Neosensory, Inc. | Method and system for haptic stimulation |
US11055789B1 (en) | 2020-01-17 | 2021-07-06 | Pearl Inc. | Systems and methods for insurance fraud detection |
KR102306279B1 (en) * | 2020-01-29 | 2021-09-29 | 자이메드 주식회사 | Method for facilitating diagnosis of fundus image, apparatus therefor and system including the same |
CN111461165A (en) * | 2020-02-26 | 2020-07-28 | 上海商汤智能科技有限公司 | Image recognition method, recognition model training method, related device and equipment |
CN111489387B (en) * | 2020-04-09 | 2023-10-20 | 湖南盛鼎科技发展有限责任公司 | Remote sensing image building area calculation method |
CN111797923A (en) * | 2020-07-03 | 2020-10-20 | 北京阅视智能技术有限责任公司 | Training method of image classification model, and image classification method and device |
CN112115863B (en) * | 2020-09-18 | 2022-10-18 | 成都理工大学 | Human body action reconstruction method and system based on Doppler radar time-frequency image sequence and cross convolution neural network |
US11497675B2 (en) | 2020-10-23 | 2022-11-15 | Neosensory, Inc. | Method and system for multimodal stimulation |
US11776677B2 (en) | 2021-01-06 | 2023-10-03 | Pearl Inc. | Computer vision-based analysis of provider data |
US11862147B2 (en) | 2021-08-13 | 2024-01-02 | Neosensory, Inc. | Method and system for enhancing the intelligibility of information for a user |
KR102488789B1 (en) * | 2022-01-18 | 2023-01-17 | (주)플레이오니 | Prediction and classification method, apparatus and program using one class anomaly detection model based on artificial intelligence |
ES2948062A1 (en) * | 2022-02-10 | 2023-08-30 | Neurogenesis Ia Tech Sl | Method for creating a model based on deep learning and device for implementing the model created by such method (Machine-translation by Google Translate, not legally binding) |
CO2022011610A1 (en) * | 2022-08-17 | 2024-02-26 | Univ Simon Bolivar | Method implemented by machine learning for the detection of blood vessels through pixel clustering |
Family Cites Families (149)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS605663B2 (en) | 1982-12-24 | 1985-02-13 | 三菱マテリアル株式会社 | Tungsten based cermet |
US4998533A (en) | 1986-07-15 | 1991-03-12 | Winkelman James W | Apparatus and method for in vivo analysis of red and white blood cell indices |
US5040225A (en) | 1987-12-07 | 1991-08-13 | Gdp, Inc. | Image analysis method |
US4883061A (en) | 1988-02-29 | 1989-11-28 | The Board Of Trustees Of The University Of Illinois | Method and apparatus for measuring the thickness of eye components |
EP0366804B1 (en) * | 1988-03-25 | 1997-12-10 | Hitachi, Ltd. | Method of recognizing image structures |
US5233517A (en) | 1990-04-30 | 1993-08-03 | Jindra Lawrence F | Early glaucoma detection by Fourier transform analysis of digitized eye fundus images |
JP3058680B2 (en) | 1990-11-30 | 2000-07-04 | 株式会社トプコン | Fundus image processing device |
US6198532B1 (en) | 1991-02-22 | 2001-03-06 | Applied Spectral Imaging Ltd. | Spectral bio-imaging of the eye |
US5303709A (en) | 1991-12-16 | 1994-04-19 | Dreher Andreas W | Retinal eye disease diagnostic system |
US7477758B2 (en) * | 1992-05-05 | 2009-01-13 | Automotive Technologies International, Inc. | System and method for detecting objects in vehicular compartments |
US5557686A (en) * | 1993-01-13 | 1996-09-17 | University Of Alabama | Method and apparatus for verification of a computer user's identification, based on keystroke characteristics |
US5878746A (en) * | 1993-08-25 | 1999-03-09 | Lemelson; Jerome H. | Computerized medical diagnostic system |
US5868134A (en) | 1993-09-21 | 1999-02-09 | Kabushiki Kaisha Topcon | Retinal disease analyzer |
US6104828A (en) | 1994-03-24 | 2000-08-15 | Kabushiki Kaisha Topcon | Ophthalmologic image processor |
US5857030A (en) | 1995-08-18 | 1999-01-05 | Eastman Kodak Company | Automated method and system for digital image processing of radiologic images utilizing artificial neural networks |
JP2001521422A (en) | 1997-04-17 | 2001-11-06 | アビモ グループ リミテッド | Eye microcirculation inspection and processing equipment |
US5867251A (en) | 1997-05-02 | 1999-02-02 | The General Hospital Corp. | Scanning ophthalmoscope with spatial light modulators |
US6041132A (en) | 1997-07-29 | 2000-03-21 | General Electric Company | Computed tomography inspection of composite ply structure |
US6044181A (en) | 1997-08-01 | 2000-03-28 | Microsoft Corporation | Focal length estimation method and apparatus for construction of panoramic mosaic images |
JP4698831B2 (en) | 1997-12-05 | 2011-06-08 | ダイナミック ディジタル デプス リサーチ プロプライエタリー リミテッド | Image conversion and coding technology |
US6000799A (en) | 1998-05-01 | 1999-12-14 | Jozef F. Van De Velde | Maxwellian view and modulation control options for the scanning laser ophthalmoscope |
US6996549B2 (en) | 1998-05-01 | 2006-02-07 | Health Discovery Corporation | Computer-aided image analysis |
US6276798B1 (en) | 1998-09-29 | 2001-08-21 | Applied Spectral Imaging, Ltd. | Spectral bio-imaging of the eye |
US6674894B1 (en) | 1999-04-20 | 2004-01-06 | University Of Utah Research Foundation | Method and apparatus for enhancing an image using data optimization and segmentation |
GB9909966D0 (en) | 1999-04-29 | 1999-06-30 | Torsana Diabetes Diagnostics A | Analysis of fundus images |
US6714672B1 (en) | 1999-10-27 | 2004-03-30 | Canon Kabushiki Kaisha | Automated stereo fundus evaluation |
US6567682B1 (en) | 1999-11-16 | 2003-05-20 | Carecord Technologies, Inc. | Apparatus and method for lesion feature identification and characterization |
US6549646B1 (en) * | 2000-02-15 | 2003-04-15 | Deus Technologies, Llc | Divide-and-conquer method and system for the detection of lung nodule in radiological images |
US7224357B2 (en) | 2000-05-03 | 2007-05-29 | University Of Southern California | Three-dimensional modeling based on photographic images |
US7104958B2 (en) | 2001-10-01 | 2006-09-12 | New Health Sciences, Inc. | Systems and methods for investigating intracranial pressure |
US6678548B1 (en) * | 2000-10-20 | 2004-01-13 | The Trustees Of The University Of Pennsylvania | Unified probabilistic framework for predicting and detecting seizure onsets in the brain and multitherapeutic device |
US6453057B1 (en) | 2000-11-02 | 2002-09-17 | Retinal Technologies, L.L.C. | Method for generating a unique consistent signal pattern for identification of an individual |
US6744933B2 (en) | 2001-01-18 | 2004-06-01 | Siemens Corporate Research, Inc. | Method for image alignment under non-uniform illumination variations |
US6845260B2 (en) | 2001-07-18 | 2005-01-18 | Koninklijke Philips Electronics N.V. | Automatic vessel indentification for angiographic screening |
US6728561B2 (en) | 2001-08-14 | 2004-04-27 | University Of Alabama In Huntsville | Multispectral image processing method, apparatus and computer program product for determining the blood oxygen saturation in a vessel |
DE10151314A1 (en) | 2001-10-17 | 2003-04-30 | Zeiss Carl Jena Gmbh | Ophthalmic examination device |
JP3998464B2 (en) | 2001-11-30 | 2007-10-24 | 三洋電機株式会社 | Video giving device |
ATE453903T1 (en) * | 2001-12-21 | 2010-01-15 | Ibm | METHOD AND CIRCUIT FOR IMAGE SCALING USING NEURONAL NETWORKS |
US7206435B2 (en) | 2002-03-26 | 2007-04-17 | Honda Giken Kogyo Kabushiki Kaisha | Real-time eye detection and tracking under various light conditions |
US7263214B2 (en) | 2002-05-15 | 2007-08-28 | Ge Medical Systems Global Technology Company Llc | Computer aided diagnosis from multiple energy images |
US20040064057A1 (en) | 2002-07-31 | 2004-04-01 | Massachusetts Institute Of Technology | Measuring circulating blood volume through retinal vasculometry |
US20040105074A1 (en) | 2002-08-02 | 2004-06-03 | Peter Soliz | Digital stereo image analyzer for automated analyses of human retinopathy |
AU2002950873A0 (en) | 2002-08-20 | 2002-09-12 | Robert Sellars | A device for providing multi-directional movement |
US6992775B2 (en) | 2002-08-29 | 2006-01-31 | Kestrel Corporation | Hyperspectral retinal imager |
US6731782B2 (en) | 2002-10-02 | 2004-05-04 | Virtualscopics, Llc | Method and system for automatic identification and quantification of abnormal anatomical structures in medical images |
US6830336B2 (en) | 2002-11-01 | 2004-12-14 | Inoveon Corporation | Automated generation of fundus images based on processing of acquired images |
US8045770B2 (en) | 2003-03-24 | 2011-10-25 | Cornell Research Foundation, Inc. | System and method for three-dimensional image rendering and analysis |
SG140445A1 (en) | 2003-07-28 | 2008-03-28 | Sony Corp | Method and apparatus for automatically recognizing audio data |
US7409587B2 (en) | 2004-08-24 | 2008-08-05 | Symantec Operating Corporation | Recovering from storage transaction failures using checkpoints |
JP4860472B2 (en) | 2003-10-09 | 2012-01-25 | ユニヴァーシティ オブ ヨーク | Image recognition |
US7545965B2 (en) * | 2003-11-10 | 2009-06-09 | The University Of Chicago | Image modification and detection using massive training artificial neural networks (MTANN) |
US20050129306A1 (en) * | 2003-12-12 | 2005-06-16 | Xianglin Wang | Method and apparatus for image deinterlacing using neural networks |
WO2005069377A1 (en) | 2004-01-19 | 2005-07-28 | Matsushita Electric Industrial Co., Ltd. | Solid-state imaging device and its manufacturing method |
US20070165243A1 (en) | 2004-02-09 | 2007-07-19 | Cheol-Gwon Kang | Device for measuring 3d shape using irregular pattern and method for the same |
US7620501B2 (en) | 2004-05-06 | 2009-11-17 | Siemens Medical Solutions Usa, Inc. | Artery-vein separation and vessel modeling |
US7907769B2 (en) | 2004-05-13 | 2011-03-15 | The Charles Stark Draper Laboratory, Inc. | Image-based methods for measuring global nuclear patterns as epigenetic markers of cell differentiation |
WO2005114557A2 (en) | 2004-05-13 | 2005-12-01 | Proximex | Multimodal high-dimensional data fusion for classification and identification |
US7316512B2 (en) | 2004-07-30 | 2008-01-08 | General Electric Company | Interconnect device |
GB2418314A (en) | 2004-09-16 | 2006-03-22 | Sharp Kk | A system for combining multiple disparity maps |
US7574028B2 (en) | 2004-11-23 | 2009-08-11 | Carestream Health, Inc. | Method for recognizing projection views of radiographs |
US7301644B2 (en) | 2004-12-02 | 2007-11-27 | University Of Miami | Enhanced optical coherence tomography for anatomical mapping |
US7372996B2 (en) | 2004-12-27 | 2008-05-13 | Trw Automotive U.S. Llc | Method and apparatus for determining the position of a vehicle seat |
US7715597B2 (en) | 2004-12-29 | 2010-05-11 | Fotonation Ireland Limited | Method and component for image recognition |
US7474775B2 (en) * | 2005-03-31 | 2009-01-06 | University Of Iowa Research Foundation | Automatic detection of red lesions in digital color fundus photographs |
US7232240B2 (en) | 2005-05-06 | 2007-06-19 | Northrop Grumann Corporation | Extended source laser illuminator |
CA2610345C (en) | 2005-06-02 | 2013-12-24 | The Medipattern Corporation | System and method of computer-aided detection |
US7995810B2 (en) | 2005-06-24 | 2011-08-09 | The University Of Iowa Research Foundation | System and methods for image segmentation in n-dimensional space |
US8098907B2 (en) | 2005-07-01 | 2012-01-17 | Siemens Corporation | Method and system for local adaptive detection of microaneurysms in digital fundus images |
US7343032B2 (en) | 2005-09-01 | 2008-03-11 | Fujifilm Corporation | Method and apparatus for automatic and dynamic vessel detection |
US7914451B2 (en) | 2005-09-15 | 2011-03-29 | Innervision Medical Technologies Inc. | Determining attributes using ultrasound |
US8023739B2 (en) | 2005-09-27 | 2011-09-20 | Battelle Memorial Institute | Processes, data structures, and apparatuses for representing knowledge |
US20070092864A1 (en) | 2005-09-30 | 2007-04-26 | The University Of Iowa Research Foundation | Treatment planning methods, devices and systems |
JP4958254B2 (en) | 2005-09-30 | 2012-06-20 | 興和株式会社 | Image analysis system and image analysis program |
US7524061B2 (en) | 2005-10-12 | 2009-04-28 | Siemens Corporate Research, Inc. | System and method for robust optic disk detection in retinal images using vessel structure and radon transform |
GB0522968D0 (en) | 2005-11-11 | 2005-12-21 | Popovich Milan M | Holographic illumination device |
US8094928B2 (en) | 2005-11-14 | 2012-01-10 | Microsoft Corporation | Stereo video for gaming |
KR100727033B1 (en) | 2005-12-07 | 2007-06-12 | 한국전자통신연구원 | Apparatus and method for vision processing on network based intelligent service robot system and the system using the same |
US8463065B2 (en) | 2005-12-07 | 2013-06-11 | Commonwealth Scientific And Industrial Research Organisation | Linear feature detection method and apparatus |
US8150155B2 (en) | 2006-02-07 | 2012-04-03 | Qualcomm Incorporated | Multi-mode region-of-interest video object segmentation |
DE102006010105A1 (en) | 2006-02-28 | 2007-08-30 | Carl Zeiss Meditec Ag | Ophthalmologic device for observation and or documentation of eye, has homogeneous lighting, and lighting equipment consists of light source, homogenization unit and projection device |
US7712898B2 (en) | 2006-04-03 | 2010-05-11 | University Of Iowa Research Foundation | Methods and systems for optic nerve head segmentation |
DE102006018445B4 (en) | 2006-04-18 | 2008-04-24 | Imedos Gmbh | Apparatus and method for determining arterio-venous ratio values by quantitative analysis of retinal vessels |
TW200741417A (en) | 2006-04-26 | 2007-11-01 | Aopen Inc | Computer chassis structure that allows to be unfolded into components in sheet shape for easy assembly |
US8249334B2 (en) | 2006-05-11 | 2012-08-21 | Primesense Ltd. | Modeling of humanoid forms from depth maps |
US8311310B2 (en) | 2006-08-11 | 2012-11-13 | Koninklijke Philips Electronics N.V. | Methods and apparatus to integrate systematic data scaling into genetic algorithm-based feature subset selection |
US8180134B2 (en) | 2006-09-26 | 2012-05-15 | Oregon Health & Science University | In vivo structural and flow imaging |
CA2670214A1 (en) | 2006-11-21 | 2008-05-29 | Mantisvision Ltd. | 3d geometric modeling and 3d video content creation |
US8121362B2 (en) | 2007-04-30 | 2012-02-21 | Siemens Medical Solutions Usa, Inc. | Registration of medical images using learned-based matching functions |
GB0708676D0 (en) | 2007-05-04 | 2007-06-13 | Imec Inter Uni Micro Electr | A Method for real-time/on-line performing of multi view multimedia applications |
WO2008150840A1 (en) | 2007-05-29 | 2008-12-11 | University Of Iowa Research Foundation | Methods and systems for determining optimal features for classifying patterns or objects in images |
US8474978B2 (en) | 2007-06-15 | 2013-07-02 | University Of Southern California | Pattern analysis of retinal maps for the diagnosis of optic nerve diseases by optical coherence tomography |
US20100182406A1 (en) | 2007-07-12 | 2010-07-22 | Benitez Ana B | System and method for three-dimensional object reconstruction from two-dimensional images |
US8090173B2 (en) | 2007-11-27 | 2012-01-03 | Siemens Medical Solutions Usa, Inc. | System and method for blood vessel bifurcation detection in thoracic CT scans |
JP2009151896A (en) | 2007-12-21 | 2009-07-09 | Sony Corp | Image processing system, motion picture reproducing system, and processing method and program for them |
US8194936B2 (en) | 2008-04-25 | 2012-06-05 | University Of Iowa Research Foundation | Optimal registration of multiple deformed images using a physical model of the imaging distortion |
JP2011520503A (en) * | 2008-05-14 | 2011-07-21 | エージェンシー フォー サイエンス,テクノロジー アンド リサーチ | Automatic concave nipple ratio measurement system |
US7941462B2 (en) | 2008-09-24 | 2011-05-10 | Toshiba Medical Visualization Systems Europe, Limited | Method and apparatus for classification of coronary artery image data |
FR2937530B1 (en) | 2008-10-24 | 2012-02-24 | Biospace Med | MEASURING INTRINSIC GEOMETRIC SIZES WITH AN ANATOMIC SYSTEM |
WO2010048632A1 (en) | 2008-10-24 | 2010-04-29 | Real D | Stereoscopic image format with depth information |
US20100142766A1 (en) | 2008-12-04 | 2010-06-10 | Alan Duncan Fleming | Image Analysis |
US8616702B2 (en) | 2009-02-25 | 2013-12-31 | University Of Iowa Research Foundation | Hybrid laser ophthalmoscope |
CN102112056A (en) | 2009-03-03 | 2011-06-29 | 富士胶片株式会社 | Image processing device and method, and program |
US8548231B2 (en) * | 2009-04-02 | 2013-10-01 | Siemens Corporation | Predicate logic based image grammars for complex visual pattern recognition |
AU2009201637B2 (en) | 2009-04-24 | 2011-08-11 | Canon Kabushiki Kaisha | Processing multi-view digital images |
US9679389B2 (en) | 2009-05-19 | 2017-06-13 | Algotec Systems Ltd. | Method and system for blood vessel segmentation and classification |
US8494243B2 (en) | 2009-07-29 | 2013-07-23 | Siemens Aktiengesellschaft | Deformable 2D-3D registration of structure |
WO2011025451A1 (en) * | 2009-08-24 | 2011-03-03 | Singapore Health Services Pte Ltd | A method and system of determining a grade of nuclear cataract |
KR20110064197A (en) | 2009-12-07 | 2011-06-15 | 삼성전자주식회사 | Object recognition system and method the same |
WO2012048070A1 (en) | 2010-10-07 | 2012-04-12 | Siemens Corporation | Non-rigid composition of multiple overlapping medical imaging volumes |
US10140699B2 (en) * | 2010-12-07 | 2018-11-27 | University Of Iowa Research Foundation | Optimal, user-friendly, object background separation |
US20150379708A1 (en) * | 2010-12-07 | 2015-12-31 | University Of Iowa Research Foundation | Methods and systems for vessel bifurcation detection |
WO2014158345A1 (en) | 2013-01-31 | 2014-10-02 | University Of Iowa Research Foundation | Methods and systems for vessel bifurcation detection |
SG190730A1 (en) * | 2010-12-09 | 2013-07-31 | Univ Nanyang Tech | Method and an apparatus for determining vein patterns from a colour image |
WO2012100225A1 (en) | 2011-01-20 | 2012-07-26 | University Of Iowa Research Foundation | Systems and methods for generating a three-dimensional shape from stereo color images |
CA2825169A1 (en) | 2011-01-20 | 2012-07-26 | University Of Iowa Research Foundation | Automated determination of arteriovenous ratio in images of blood vessels |
US8989514B2 (en) | 2011-02-03 | 2015-03-24 | Voxeleron Llc | Method and system for image analysis and interpretation |
JP5276225B2 (en) | 2011-02-22 | 2013-08-28 | オリンパスメディカルシステムズ株式会社 | Medical image processing apparatus and method of operating medical image processing apparatus |
JP5263995B2 (en) | 2011-03-18 | 2013-08-14 | 富士フイルム株式会社 | Network construction apparatus and method, and program |
JP5701138B2 (en) | 2011-04-19 | 2015-04-15 | 富士フイルム株式会社 | Medical image processing apparatus and method, and program |
JP2012223338A (en) | 2011-04-19 | 2012-11-15 | Fujifilm Corp | Tree structure creation apparatus, method, and program |
JP5391229B2 (en) | 2011-04-27 | 2014-01-15 | 富士フイルム株式会社 | Tree structure extraction apparatus and method, and program |
US9547804B2 (en) * | 2011-08-25 | 2017-01-17 | Cornell University | Retinal encoder for machine vision |
GB201201230D0 (en) | 2012-01-25 | 2012-03-07 | Univ Delft Tech | Adaptive multi-dimensional data decomposition |
US9545196B2 (en) | 2012-05-04 | 2017-01-17 | University Of Iowa Research Foundation | Automated assessment of glaucoma loss from optical coherence tomography |
US9536338B2 (en) | 2012-07-31 | 2017-01-03 | Microsoft Technology Licensing, Llc | Animating objects using the human body |
JP5833994B2 (en) | 2012-09-20 | 2015-12-16 | 富士フイルム株式会社 | Image processing apparatus and method, and program |
US9307926B2 (en) | 2012-10-05 | 2016-04-12 | Volcano Corporation | Automatic stent detection |
WO2014143891A1 (en) | 2013-03-15 | 2014-09-18 | University Of Iowa Research Foundation | Automated separation of binary overlapping trees |
US9700219B2 (en) * | 2013-10-17 | 2017-07-11 | Siemens Healthcare Gmbh | Method and system for machine learning based assessment of fractional flow reserve |
US9002085B1 (en) * | 2013-10-22 | 2015-04-07 | Eyenuk, Inc. | Systems and methods for automatically generating descriptions of retinal images |
US8965112B1 (en) * | 2013-12-09 | 2015-02-24 | Google Inc. | Sequence transcription with deep neural networks |
US10410355B2 (en) | 2014-03-21 | 2019-09-10 | U.S. Department Of Veterans Affairs | Methods and systems for image analysis using non-euclidean deformed graphs |
WO2015148018A1 (en) * | 2014-03-27 | 2015-10-01 | Bellybit, Inc. | Systems, devices, and methods for tracking abdominal orientation and activity |
CA2944829C (en) * | 2014-05-23 | 2022-10-25 | Ting Chen | Systems and methods for detection of biological structures and/or patterns in images |
US10628940B2 (en) * | 2014-09-08 | 2020-04-21 | The Cleveland Clinic Foundation | Automated analysis of angiographic images |
CN106462940A (en) * | 2014-10-09 | 2017-02-22 | 微软技术许可有限责任公司 | Generic object detection in images |
US10736517B2 (en) * | 2014-10-09 | 2020-08-11 | Panasonic Intellectual Property Management Co., Ltd. | Non-contact blood-pressure measuring device and non-contact blood-pressure measuring method |
US10346726B2 (en) * | 2014-12-15 | 2019-07-09 | Samsung Electronics Co., Ltd. | Image recognition method and apparatus, image verification method and apparatus, learning method and apparatus to recognize image, and learning method and apparatus to verify image |
US9836849B2 (en) * | 2015-01-28 | 2017-12-05 | University Of Florida Research Foundation, Inc. | Method for the autonomous image segmentation of flow systems |
US10115194B2 (en) * | 2015-04-06 | 2018-10-30 | IDx, LLC | Systems and methods for feature detection in retinal images |
JP6726856B2 (en) * | 2015-07-17 | 2020-07-22 | パナソニックIpマネジメント株式会社 | Attention information presentation device and attention information presentation method |
US9881234B2 (en) * | 2015-11-25 | 2018-01-30 | Baidu Usa Llc. | Systems and methods for end-to-end object detection |
JP2017128837A (en) * | 2016-01-22 | 2017-07-27 | 富士通株式会社 | Tool, biosensor, and sensing device |
JP2017136165A (en) * | 2016-02-02 | 2017-08-10 | 富士通株式会社 | Sensor information processing device, sensor unit, and sensor information processing program |
US9754219B1 (en) * | 2016-07-01 | 2017-09-05 | Intraspexion Inc. | Using classified text and deep learning algorithms to identify entertainment risk and provide early warning |
US10636141B2 (en) * | 2017-02-09 | 2020-04-28 | Siemens Healthcare Gmbh | Adversarial and dual inverse deep learning networks for medical image analysis |
US10489908B2 (en) * | 2017-02-22 | 2019-11-26 | Siemens Healthcare Gmbh | Deep convolutional encoder-decoder for prostate cancer detection and classification |
US10223610B1 (en) * | 2017-10-15 | 2019-03-05 | International Business Machines Corporation | System and method for detection and classification of findings in images |
US10430946B1 (en) * | 2019-03-14 | 2019-10-01 | Inception Institute of Artificial Intelligence, Ltd. | Medical image segmentation and severity grading using neural network architectures with semi-supervised learning techniques |
-
2016
- 2016-04-06 US US15/092,421 patent/US10115194B2/en active Active
-
2018
- 2018-10-30 US US16/175,318 patent/US11790523B2/en active Active
-
2023
- 2023-09-12 US US18/367,384 patent/US20230419485A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US20160292856A1 (en) | 2016-10-06 |
US20190130566A1 (en) | 2019-05-02 |
US10115194B2 (en) | 2018-10-30 |
US11790523B2 (en) | 2023-10-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20230419485A1 (en) | Autonomous diagnosis of a disorder in a patient from image analysis | |
Khan et al. | Lungs nodule detection framework from computed tomography images using support vector machine | |
US10111632B2 (en) | System and method for breast cancer detection in X-ray images | |
Choi et al. | Genetic programming-based feature transform and classification for the automatic detection of pulmonary nodules on computed tomography images | |
Tan et al. | A novel computer‐aided lung nodule detection system for CT images | |
Campadelli et al. | A fully automated method for lung nodule detection from postero-anterior chest radiographs | |
Blanc et al. | Artificial intelligence solution to classify pulmonary nodules on CT | |
US11308611B2 (en) | Reducing false positive detections of malignant lesions using multi-parametric magnetic resonance imaging | |
JP2018117883A (en) | Learning device, identification device, identification method and program | |
US20180082104A1 (en) | Classification of cellular images and videos | |
Pino Peña et al. | Automatic emphysema detection using weakly labeled HRCT lung images | |
US20200074631A1 (en) | Systems And Methods For Identifying Implanted Medical Devices | |
Iakovidis et al. | Blood detection in wireless capsule endoscope images based on salient superpixels | |
Rezaie et al. | Detection of lung nodules on medical images by the use of fractal segmentation | |
Rao et al. | Early detection of lung cancer using machine learning technique | |
Xiong et al. | Lung field segmentation using weighted sparse shape composition with robust initialization | |
Zhao et al. | False positive reduction for lung nodule CAD using support vector machines and genetic algorithms | |
Sridhar et al. | Lung Segment Anything Model (LuSAM): A Prompt-integrated Framework for Automated Lung Segmentation on ICU Chest X-Ray Images | |
Dalju et al. | Superpixel-based segmentation and classification of gastrointestinal landmarks and diseases | |
Cazzolato et al. | Breath: Heat maps assisting the detection of abnormal lung regions in ct scans | |
Naeem et al. | DVFNet: A deep feature fusion-based model for the multiclassification of skin cancer utilizing dermoscopy images | |
Lynn et al. | Segmentation of skin lesion towards melanoma skin cancer classification | |
Chowdhury et al. | Segmentation of retina images to detect abnormalities arising from diabetic retinopathy | |
KR102650919B1 (en) | Medical image classifying method using contrastive learning based on patch and analysis apparatus | |
Kukar et al. | Multi-resolution image parametrization in stepwise diagnostics of coronary artery disease |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: IDX TECHNOLOGIES INC., IOWA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:IDX, LLC;REEL/FRAME:064888/0315 Effective date: 20180912 Owner name: IDX, LLC, IOWA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NIEMEIJER, MEINDERT;AMELON, RYAN;CLARIDA, WARREN;AND OTHERS;REEL/FRAME:064888/0303 Effective date: 20160504 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: DIGITAL DIAGNOSTICS INC., IOWA Free format text: CHANGE OF NAME;ASSIGNOR:IDX TECHNOLOGIES INC.;REEL/FRAME:065967/0516 Effective date: 20200529 |