EP3483895A1 - Detecting and classifying medical images based on continuously-learning whole body landmarks detections - Google Patents
Detecting and classifying medical images based on continuously-learning whole body landmarks detections Download PDFInfo
- Publication number
- EP3483895A1 EP3483895A1 EP18205581.4A EP18205581A EP3483895A1 EP 3483895 A1 EP3483895 A1 EP 3483895A1 EP 18205581 A EP18205581 A EP 18205581A EP 3483895 A1 EP3483895 A1 EP 3483895A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- medical image
- image
- metadata tags
- scanner
- medical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000001514 detection method Methods 0.000 title description 7
- 238000000034 method Methods 0.000 claims abstract description 77
- 238000010801 machine learning Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims description 33
- 238000012986 modification Methods 0.000 claims description 19
- 230000004048 modification Effects 0.000 claims description 19
- 238000002595 magnetic resonance imaging Methods 0.000 claims description 6
- 238000007637 random forest analysis Methods 0.000 claims description 6
- 238000012217 deletion Methods 0.000 claims description 5
- 230000037430 deletion Effects 0.000 claims description 5
- 230000002787 reinforcement Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000015654 memory Effects 0.000 description 17
- 210000000056 organ Anatomy 0.000 description 16
- 238000013459 approach Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000004044 response Effects 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 210000004185 liver Anatomy 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000003066 decision tree Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 210000004072 lung Anatomy 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000007792 addition Methods 0.000 description 3
- 210000003127 knee Anatomy 0.000 description 3
- 210000003484 anatomy Anatomy 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 210000001981 hip bone Anatomy 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012706 support-vector machine Methods 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 210000001015 abdomen Anatomy 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000002591 computed tomography Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001493 electron microscopy Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000002610 neuroimaging Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10104—Positron emission tomography [PET]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10108—Single photon emission computed tomography [SPECT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
Definitions
- the invention relates generally to the detection and classification of medical images based on continuously-learning whole body landmarks detection.
- the techniques described herein may be applied, for example, as a post-processing step during image acquisition to provide human-readable information (referred to herein as "tags") that specify body parts and organs present in an image.
- the present disclosure is directed to overcoming these and other problems of the prior art.
- Embodiments of the present invention address and overcome one or more of the above shortcomings and drawbacks, by providing methods, systems, and apparatuses related to the detection and classification of medical images based on continuously-learning whole body landmarks detection. More generally, the techniques described herein address how to accurately obtain body part and organ information of the medical image without relying or deducing information from the metadata. More specifically, rather than parsing metadata, the techniques described herein derive information from the image itself.
- a computer-implemented method for automatically generating metadata tags for a medical image includes receiving a medical image and automatically identifying a set of body landmarks in the medical image using one or more machine learning models.
- a set of rules are applied to the set of body landmarks to identify anatomical objects present in the image.
- one or more machine learning models are applied to the set of body landmarks to identify anatomical objects present in the image.
- the one or more machine learning models comprise a deep reinforcement learning model.
- each rule in the set of rules defines an upper bound landmark and a lower bound landmark for a corresponding anatomical object or a left bound landmark and a right bound landmark for the corresponding anatomical object.
- the method advantageously can further comprise, detecting a modification of the medical image by one or more users; and based on the modification of the medical image, automatically modifying one or more rules in the set of rules.
- the modification of the medical image comprising a modification or deletion of one or more of the metadata tags or an addition of one or more new metadata tags.
- a computer-implemented method for automatically generating metadata tags for a medical image comprising: receiving a medical image; identifying a set of body landmarks in the medical image; applying one or more machine learning models to the set of body landmarks to identify anatomical objects present in the image; generating one or more metadata tags corresponding to the anatomical objects; storing the metadata tags in the medical image; and transferring the medical image with the metadata tags to a data repository.
- the one or more machine learning models comprise a random forest.
- the random forest is trained using a plurality of medical images labeled with an upper bound landmark, a lower bound landmark, and one or more labeled anatomical objects located between the upper bound landmark and the lower bound landmark.
- the plurality of medical images are further labeled with a left bound landmark and a right bound landmark, and the one or more labeled anatomical objects are located between the left bound landmark and the right bound landmark.
- the method further comprises: detecting a modification of the medical image by one or more users; and based on the modification of the medical image, retraining the one or more machine learning models.
- the modification of the medical image comprising a modification or deletion of one or more of the metadata tags and an addition of one or more new metadata tags.
- the medical image is acquired using a medical image scanner and the method is performed immediately following image reconstruction on a computer connected to the medical image scanner.
- the medical image scanner can be a MRI scanner.
- a system for automatically generating metadata tags for a medical image includes a medical image scanner and an image processing system comprising one or more processors.
- the medical image scanner is configured to acquire a medical image.
- the processors in the image processing system are configured to automatically identify a set of body landmarks in the medical image. Based on those landmarks, the processors identify anatomical objects present in the image and generate one or more metadata tags corresponding to the anatomical objects.
- the processors then store the metadata tags in the medical image and transfer the medical image with the metadata tags to a data repository.
- the medical image scanner is preferably a MRI scanner.
- the system according to the third aspect of the invention is preferably adapted to perform a computer-implemented method for automatically generating metadata tags for a medical image according to the first and the second aspect of the invention.
- Systems, methods, and apparatuses are described herein which relate generally to the detection and classification of medical images based on landmarks detected in the images.
- the techniques described herein may be understood as entailing two distinct steps.
- a landmarking engine is applied to a medical image to identify a set of landmarks present in the image.
- metadata tags are identified that describe body parts and/or organs present in the image.
- the identification of metadata tags can be performed using a rules-based or machine learning-based approach.
- the techniques described herein aid in the automation of metadata tag generation.
- the disclosed techniques may be used to correct any incorrect tags and ensure consistency across an image archive or other repository.
- FIG. 1A illustrates a method 100 for detecting and classifying medical images based on landmarks detected in the images.
- an image processing system receives one or more medical images.
- image processing system refers to a computer system with resource capable of processing images in an optimal manner.
- One example architecture is shown below with reference to FIG. 4 . It should be noted that this architecture is merely exemplary and, in other embodiments, different architectures can be used. For example, in some embodiments, multiple compute nodes are a used in a cluster or a cloud infrastructure such as Amazon Web Services (AWS).
- AWS Amazon Web Services
- the term “receives” should be broadly understood to include any technique for the image processing system acquiring the medical images.
- the medical images may be sent to the image processing system as an input.
- the image processing system collects the medical images from a database or other storage medium when a software application corresponding to the method 100 is executed.
- the image processing system is included in the image scanner that acquires the medical images.
- the image processing system is executed by a computer within a magnetic resonance image (MRI) system. After the MRI data is acquired and reconstructed, the method 100 is automatically executed using the reconstructed images as input.
- MRI magnetic resonance image
- the image processing system automatically identifies landmarks in the medical image by executing one or more machine learning models with the images as input.
- These landmarks may correspond to body parts (e.g., abdomen), organs (e.g., liver), or other anatomical objects (e.g., veins or nerves).
- landmarks for abnormalities e.g., lesions
- Each machine learning model is trained based on labeled training images to identify landmarks at various anatomical views.
- the output of these machine learning models is a set of body landmarks that it recognizes in the image volume, such as 'Liver Top', 'Carina Bifurcation' or 'Right Knee'.
- One example machine learning algorithm for navigation of image parsing with deep reinforcement learning is described in U.S. Patent No. 9,569,736 entitled "Intelligent medical image landmark detection," the entirety of which is incorporated herein by reference.
- a rules engine is executed to detect which body part(s) and organ(s) are present in the medical image.
- the term "rules engine” is used herein to refer to any executable application or other software that executes rules. Each rule checks for the occurrence of conditions and, if the condition is met, one or more actions are performed.
- the rules used herein may be understood as listing one or more body parts or organs for a group of landmarks linked by one or more conditions.
- a rule may indicate that the inclusion of two particular landmarks indicates that one or more organs are present the input image.
- More complex rules may be possible as well.
- one rule may indicate that the presence of certain landmarks and the absence of other landmarks correspond to body parts or organs.
- rules can be executed in sequence to provide more complex tags. For example, after executing a first rule to determine a first body part, a second rule can be executed that uses both the landmarks and the first body part to determine a second body part. In this way, sub-sections of anatomy can be identified.
- the first body part may specify the presence of the heart in the image and the second body part may indicate specify chambers, arteries, etc.
- the list of various body parts and organs are then stored as metadata tags in the medical images.
- the metadata tag Right Knee' may be generated.
- the correspondence between anatomical objects and the metadata tags may be established, for example, using a dictionary lookup system. That is, given an anatomical object, the algorithm returns the particular tag.
- every anatomical object has a defined upper bound and a lower bound.
- the uppermost (or "upper bound") landmark and the lowermost (or "lower bound”) landmark may be used. For example, FIG.
- each anatomical object also has a defined left bound and a right bound between which the object must be located.
- left and right in this context are with reference to the coordinate system that defines the upper and lower bounds.
- the descriptors output by the landmarking algorithm can be used directly as the tags in the metadata.
- the dictionary lookup algorithm may be robust enough to provide different tags for a particular anatomical object based on one or more input parameters. For example, one medical institution may use the tag "Upper Liver” to refer to the upper portion of the liver, while another medical institution may use the tag "Liver Top.” The medical institution or another identifier could be provided as an input to the dictionary lookup system to allow it to select the appropriate tag for a particular institution.
- the metadata tags are stored in correspondence with the medical image at step 125.
- the tags may be stored directly in the image itself. If embedded metadata is not supported, the tags may be saved in a separate file with a reference to the image.
- a data model e.g., json
- This meta information can then be indexed for an easy, unstructured search.
- the image with its metadata is stored in an archive or other data repository for later access and analysis. Once stored, the images are searchable by one or more users via their metadata tags.
- the image used as input may already comprise DICOM tags or other metadata with anatomical objects.
- the method 100 may be configured to ignore this metadata and only consider as input the landmarks and their position. In this way, the method 100 solves the problem where medical images do not contain metadata information (i.e., NIfTI) or the DICOM header (meta data) is empty or inaccurate.
- FIG. 2 illustrates an alternative method 200 for detecting and classifying medical images based on landmarks detected in the images.
- the rule-based approach is replaced by a machine-learning approach, allowing the rules for body part and organ definition to be fine-tuned via a machine learning based method.
- steps 205 and 210 may be implemented in the same manner as discussed above with reference to steps 105 and 110 of the method 100 of FIG. 1A .
- a machine learning model is used to identify metadata tags based on landmarks.
- the rules for body part and organ definition can be fine-tuned via a machine learning.
- Examples of machine learning models that may be applied at step 215 include support vector machines (SVMs), decision trees or random forests, standard neural networks, and convolutional neural networks.
- Various techniques may be used for training the machine learning model used at step 215.
- training is performed by providing a set of images with labeled landmarks and corresponding body parts.
- This training set is preferably large and diverse enough such that the trained model can support a wide range of anatomies and views.
- Various techniques may be used for labeling the data.
- the machine learning model is trained using medical images labeled with an upper bound landmark, a lower bound landmark, and one or more labeled anatomical objects located between the upper bound landmark and the lower bound landmark.
- the medical images may be further labeled with a left bound landmark and a right bound landmark, and the one or more labeled anatomical objects are located between the left bound landmark and the right bound landmark.
- each decision tree comprising a randomly selected subset of the landmarks.
- a subset is split into daughter nodes by considering the landmarks in the training data.
- the rules of each randomly generated decision tree are used to predict an anatomical object.
- the votes of each predicted anatomical object are calculated and the highest voted object is considered the most like object corresponding to the input landmarks.
- the list of various body parts and organs output by the machine learning model is applied as metadata information to the medical images.
- the metadata tags Once all of the metadata tags have been established, they are stored in a data repository in correspondence with the medical image at step 225. Once in the data repository, the images are searchable via their metadata tags. The details of implementing steps 220 and 225 are similar to those discussed above with regards to steps 120 and 125 of FIG. 1A .
- FIG. 3 illustrates a system that could be used to implement the method 200 illustrated in FIG. 2 .
- a magnetic resonance imaging (MRI) scanner 305 sends image data to an image processing system 335.
- an extract, transform, and load (ETL) module 310 transforms the image, as necessary to be used as input to the landmark engine 315. Additionally, the ETL module may extract relevant information (e.g., image dimensions, modality, etc.) to generate parameters for the landmark engine 315.
- the landmark engine 315 executes as described above with reference to FIG. 1A to identify a set of landmarks.
- the machine learning model(s) 320 are executed on the set of landmark marks to produce a tagged image (i.e., an image with the appropriate body parts, etc. stored in metadata tags of the image.
- a data output module 325 is configured to communicate with a data repository 345 to store the tagged image.
- a logging module 330 in the image processing system 335 records the tagged image and possibly other information used in processing the image (e.g., parameters to the landmark engine 315 and machine learning models 320). In this way, operation of the image processing system 335 can be verified and validated. Additionally, the logging module 330 may be used to debug the image processing system 335 if any erroneous tags or other data are detected.
- a user 340 can access, retrieve, and use the tagged images.
- the user 340 may provide feedback to the image processing system 335. In the example of FIG. 3 , this feedback is provided to the image processing system 335; however, in other embodiments, the user 340 may provide the feedback to the data repository 345 which, in turn, relays the feedback to the image processing system 335.
- the user may provide explicit feedback such as a rating, an accuracy measurement, etc. The feedback could also be to provide a ticket to correct a tag that was mis-tagged (because machine learning based systems are not 100% accurate in the beginning).
- the feedback can be in the form of modification of the tags by the user 340.
- the image processing system 335 may automatically detect any additions, deletions, or modifications to the metadata tags by the user 340. For example, the image processing system 335 may periodically review the contents of the data repository 345 to identify files with metadata tags modified by the user 340. The image processing system 335 may then use the image to retrain the machine learning models 320 to further increase their overall accuracy.
- FIG. 4 illustrates an exemplary computing environment 400 within which embodiments of the invention may be implemented.
- the computing environment 400 includes computer system 410, which is one example of a computing system upon which embodiments of the invention may be implemented.
- Computers and computing environments, such as computer system 410 and computing environment 400, are known to those of skill in the art and thus are described briefly herein.
- the computer system 410 may include a communication mechanism such as a bus 421 or other communication mechanism for communicating information within the computer system 410.
- the computer system 410 further includes one or more processors 420 coupled with the bus 421 for processing the information.
- the processors 420 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art.
- the computer system 410 also includes a system memory 430 coupled to the bus 421 for storing information and instructions to be executed by processors 420.
- the system memory 430 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 431 and/or random access memory (RAM) 432.
- the system memory RAM 432 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM).
- the system memory ROM 431 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM).
- the system memory 430 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 420.
- a basic input/output system (BIOS) 433 contains the basic routines that help to transfer information between elements within computer system 410, such as during start-up, may be stored in ROM 431.
- RAM 432 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by the processors 420.
- System memory 430 may additionally include, for example, operating system 434, application programs 435, other program modules 436 and program data 437.
- the application programs 435 may include, for example, the ETL module, the landmarking engine, the machine learning models, and the other components of the image processing system described above with reference to FIG. 3 .
- the computer system 410 also includes a disk controller 440 coupled to the bus 421 to control one or more storage devices for storing information and instructions, such as a hard disk 441 and a removable media drive 442 (e.g., floppy disk drive, compact disc drive, tape drive, and/or solid state drive).
- the storage devices may be added to the computer system 410 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).
- SCSI small computer system interface
- IDE integrated device electronics
- USB Universal Serial Bus
- FireWire FireWire
- the computer system 410 may also include a display controller 465 coupled to the bus 421 to control a display 466, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user.
- the computer system includes an input interface 460 and one or more input devices, such as a keyboard 462 and a pointing device 461, for interacting with a computer user and providing information to the processors 420.
- the pointing device 461 for example, may be a mouse, a trackball, or a pointing stick for communicating direction information and command selections to the processors 420 and for controlling cursor movement on the display 466.
- the display 466 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 461.
- the computer system 410 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 420 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 430.
- a memory such as the system memory 430.
- Such instructions may be read into the system memory 430 from another computer readable medium, such as a hard disk 441 or a removable media drive 442.
- the hard disk 441 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security.
- the processors 420 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 430.
- hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
- the computer system 410 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein.
- the term "computer readable medium” as used herein refers to any medium that participates in providing instructions to the processor 420 for execution.
- a computer readable medium may take many forms including, but not limited to, non-volatile media, volatile media, and transmission media.
- Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as hard disk 441 or removable media drive 442.
- Non-limiting examples of volatile media include dynamic memory, such as system memory 430.
- Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the bus 421.
- Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
- the computing environment 400 may further include the computer system 410 operating in a networked environment using logical connections to one or more image scanners such as imaging device 480.
- the imaging device 480 may be a radiology scanner such as a magnetic resonance (MR) scanner, PET/MR, X-ray or a CT scanner.
- computer system 410 may include modem 472 for establishing communications with the imaging device 480 or a remote computing system over a network 471, such as the Internet. Modem 472 may be connected to bus 421 via user network interface 470, or via another appropriate mechanism. It should be noted that, although the imaging device 480 is illustrated as being connected to the computer system 410 over the network 471 in the example presented in FIG.
- the computer system 410 may be directly connected to the image scanner 480.
- the computer system 410 and the image scanner 480 are co-located in the same room or in adjacent rooms, and the devices are connected using any transmission media generally known in the art.
- Network 471 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 410 and other computers (e.g., remote computer 480).
- the network 471 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-11 or any other wired connection generally known in the art.
- Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 471.
- the embodiments of the present disclosure may be implemented with any combination of hardware and software.
- the embodiments of the present disclosure may be included in an article of manufacture (e.g., one or more computer program products) having, for example, computer-readable, non-transitory media.
- the media has embodied therein, for instance, computer readable program code for providing and facilitating the mechanisms of the embodiments of the present disclosure.
- the article of manufacture can be included as part of a computer system or sold separately.
- image refers to multi-dimensional data composed of discrete image elements (e.g., pixels for 2-D images and voxels for 3-D images).
- the image may be, for example, a medical image of a subject collected by computer tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to one of skill in the art.
- the image may also be provided from non-medical contexts, such as, for example, remote sensing systems, electron microscopy, etc.
- the techniques described herein may generally be applied to images of any dimension, e.g., a 2-D picture or a 3-D volume.
- the domain of the image is typically a 2- or 3-dimensional rectangular array, wherein each pixel or voxel can be addressed with reference to a set of 2 or 3 mutually orthogonal axes.
- digital and digitized as used herein will refer to images or volumes, as appropriate, in a digital or digitized format acquired via a digital acquisition system or via conversion from an analog image.
- An executable application comprises code or machine readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input.
- An executable procedure is a segment of code or machine readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.
- GUI graphical user interface
- the GUI comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions.
- the GUI also includes an executable procedure or executable application.
- the executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user.
- the processor under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Radiology & Medical Imaging (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Health & Medical Sciences (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
A computer-implemented method for automatically generating metadata tags for a medical image includes receiving a medical image and automatically identifying a set of body landmarks in the medical image using one or more machine learning models. A set of rules are applied to the set of body landmarks to identify anatomical objects present in the image. As an alternative to using the set of rules, in some embodiments, one or more machine learning models to the set of body landmarks to identify anatomical objects present in the image. Once the anatomical objects are identified, metadata tags corresponding to the anatomical objects are generated and stored in the medical image. Then, the medical image with the metadata tags is transferred to a data repository.
Description
- The invention relates generally to the detection and classification of medical images based on continuously-learning whole body landmarks detection. The techniques described herein may be applied, for example, as a post-processing step during image acquisition to provide human-readable information (referred to herein as "tags") that specify body parts and organs present in an image.
- In the healthcare domain, researchers, scientist and other professionals work with medical images acquired through various image modalities. Finding specific cases or images from a large archive is often tedious, manual work, because not enough information is stored with the medical images. In addition, these medical images often follow an industry-standard format, such as DICOM images, while others may follow other standards (e.g., Neuroimaging Informatics Technology Initiative or "NIfTI") or be a more generic export (e.g., jpg, png, etc.). Depending on the format, these medical images may or may not have metadata information associated with the image, including tags describing the body part examined, organ of interest, acquisition protocol, or study description. Many image formats do not allow the specification of metadata tags or other metadata information (e.g., NIfTI, jpg). Even in formats where metadata can be used, the values are often missing or the information is incorrect. Correct and descriptive metadata often depends on the department's acquisition process at scan time, and often does not have a standardized approach across departments or institutions.
- Conventional solutions only approach this problem through the meta data information via a rules-based approach. This current approach is limited to DICOM formatted images; it also assumes that the metadata information is readily available, and that it is error-free. The solution approach looks at the metadata tag 'StudyDescription'. This tag is filled with free-text describing how the study has been performed. From this free-text the solution deduces what anatomical object was studied and can be extracted. This solution has been proven to be somewhat effective at 99.94%, but does not consider the case when the metadata is not available (as is the case in other medical imaging formats), is left blank, or is incorrectly filled out. Additionally, in some instances, there is also the issue that the description filled out in the Study Description does not follow a rigorous guideline and is not harmonized (e.g., using the keyword "Lung" or "Chest").
- The present disclosure is directed to overcoming these and other problems of the prior art.
- Embodiments of the present invention address and overcome one or more of the above shortcomings and drawbacks, by providing methods, systems, and apparatuses related to the detection and classification of medical images based on continuously-learning whole body landmarks detection. More generally, the techniques described herein address how to accurately obtain body part and organ information of the medical image without relying or deducing information from the metadata. More specifically, rather than parsing metadata, the techniques described herein derive information from the image itself.
- According to a first aspect of the invention, a computer-implemented method for automatically generating metadata tags for a medical image includes receiving a medical image and automatically identifying a set of body landmarks in the medical image using one or more machine learning models. A set of rules are applied to the set of body landmarks to identify anatomical objects present in the image. As an alternative to using the set of rules, in some embodiments, one or more machine learning models are applied to the set of body landmarks to identify anatomical objects present in the image. Once the anatomical objects are identified, metadata tags corresponding to the anatomical objects are generated and stored in the medical image. Then, the medical image with the metadata tags is transferred to a data repository. Preferred is a method, wherein the one or more machine learning models comprise a deep reinforcement learning model.
Further, preferred is a method, wherein each rule in the set of rules defines an upper bound landmark and a lower bound landmark for a corresponding anatomical object or a left bound landmark and a right bound landmark for the corresponding anatomical object.
The method advantageously can further comprise, detecting a modification of the medical image by one or more users; and based on the modification of the medical image, automatically modifying one or more rules in the set of rules. Additionally a method is preferred, wherein the modification of the medical image comprising a modification or deletion of one or more of the metadata tags or an addition of one or more new metadata tags.
Further, preferred is a method, wherein the medical image is acquired using a medical image scanner and the method is performed immediately following image reconstruction on a computer connected to the medical image scanner. The medical image scanner can be a magnetic resonance imaging scanner.
According to a second aspect of the invention, a computer-implemented method for automatically generating metadata tags for a medical image is provided, the method comprising: receiving a medical image; identifying a set of body landmarks in the medical image; applying one or more machine learning models to the set of body landmarks to identify anatomical objects present in the image; generating one or more metadata tags corresponding to the anatomical objects; storing the metadata tags in the medical image; and transferring the medical image with the metadata tags to a data repository.
Preferred is a method according to the second aspect, wherein the one or more machine learning models comprise a random forest. Additionally a method is preferred, wherein the random forest is trained using a plurality of medical images labeled with an upper bound landmark, a lower bound landmark, and one or more labeled anatomical objects located between the upper bound landmark and the lower bound landmark. Preferred is such a method, wherein the plurality of medical images are further labeled with a left bound landmark and a right bound landmark, and the one or more labeled anatomical objects are located between the left bound landmark and the right bound landmark.
Preferred is a method according to the second aspect, the method further comprises: detecting a modification of the medical image by one or more users; and based on the modification of the medical image, retraining the one or more machine learning models. Preferred is a method, wherein the modification of the medical image comprising a modification or deletion of one or more of the metadata tags and an addition of one or more new metadata tags.
Further, preferred is a method according to the second aspect, wherein the medical image is acquired using a medical image scanner and the method is performed immediately following image reconstruction on a computer connected to the medical image scanner. The medical image scanner can be a MRI scanner. - According to other embodiments according to a third aspect of the invention, a system for automatically generating metadata tags for a medical image includes a medical image scanner and an image processing system comprising one or more processors. The medical image scanner is configured to acquire a medical image. The processors in the image processing system are configured to automatically identify a set of body landmarks in the medical image. Based on those landmarks, the processors identify anatomical objects present in the image and generate one or more metadata tags corresponding to the anatomical objects. The processors then store the metadata tags in the medical image and transfer the medical image with the metadata tags to a data repository. The medical image scanner is preferably a MRI scanner. The system according to the third aspect of the invention is preferably adapted to perform a computer-implemented method for automatically generating metadata tags for a medical image according to the first and the second aspect of the invention.
- Additional features and advantages of the invention will be made apparent from the following detailed description of illustrative embodiments that proceeds with reference to the accompanying drawings.
- The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there are shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:
-
FIG. 1A illustrates a method for detecting and classifying medical images based on landmarks detected in the images; - For example,
FIG. 1B shows example medical image where the upper bound landmark is labeled as "Right Lung Top" and the lower bound landmark is labeled as "Right Hip Bone;" -
FIG. 2 illustrates an alternative method for detecting and classifying medical images based on landmarks detected in the images; -
FIG. 3 illustrates a system that could be used to implement the method illustrated inFIG. 2 ; and -
FIG. 4 illustrates an exemplary computing environment within which embodiments of the invention may be implemented. - Systems, methods, and apparatuses are described herein which relate generally to the detection and classification of medical images based on landmarks detected in the images. Briefly, the techniques described herein may be understood as entailing two distinct steps. First, a landmarking engine is applied to a medical image to identify a set of landmarks present in the image. Then, using these landmarks, metadata tags are identified that describe body parts and/or organs present in the image. As described in further detail below, the identification of metadata tags can be performed using a rules-based or machine learning-based approach. The techniques described herein aid in the automation of metadata tag generation. Also, where metadata tags are already present in an image, the disclosed techniques may be used to correct any incorrect tags and ensure consistency across an image archive or other repository.
-
FIG. 1A illustrates amethod 100 for detecting and classifying medical images based on landmarks detected in the images. Starting atstep 105, an image processing system receives one or more medical images. The term "image processing system," as used herein refers to a computer system with resource capable of processing images in an optimal manner. One example architecture is shown below with reference toFIG. 4 . It should be noted that this architecture is merely exemplary and, in other embodiments, different architectures can be used. For example, in some embodiments, multiple compute nodes are a used in a cluster or a cloud infrastructure such as Amazon Web Services (AWS). The term "receives" should be broadly understood to include any technique for the image processing system acquiring the medical images. Thus, in some embodiments, the medical images may be sent to the image processing system as an input. In other embodiments, the image processing system collects the medical images from a database or other storage medium when a software application corresponding to themethod 100 is executed. In some embodiments, the image processing system is included in the image scanner that acquires the medical images. For example, in one embodiment, the image processing system is executed by a computer within a magnetic resonance image (MRI) system. After the MRI data is acquired and reconstructed, themethod 100 is automatically executed using the reconstructed images as input. - At
step 110 the image processing system automatically identifies landmarks in the medical image by executing one or more machine learning models with the images as input. These landmarks may correspond to body parts (e.g., abdomen), organs (e.g., liver), or other anatomical objects (e.g., veins or nerves). In some embodiments, landmarks for abnormalities (e.g., lesions) may also be identified. Each machine learning model is trained based on labeled training images to identify landmarks at various anatomical views. The output of these machine learning models is a set of body landmarks that it recognizes in the image volume, such as 'Liver Top', 'Carina Bifurcation' or 'Right Knee'. One example machine learning algorithm for navigation of image parsing with deep reinforcement learning is described inU.S. Patent No. 9,569,736 - Next, at
step 115, using the set of landmarks (i.e., name of the landmark and position), a rules engine is executed to detect which body part(s) and organ(s) are present in the medical image. The term "rules engine" is used herein to refer to any executable application or other software that executes rules. Each rule checks for the occurrence of conditions and, if the condition is met, one or more actions are performed. - At their most basic level, the rules used herein may be understood as listing one or more body parts or organs for a group of landmarks linked by one or more conditions. For example, a rule may indicate that the inclusion of two particular landmarks indicates that one or more organs are present the input image. More complex rules may be possible as well. For example, one rule may indicate that the presence of certain landmarks and the absence of other landmarks correspond to body parts or organs. Additionally, rules can be executed in sequence to provide more complex tags. For example, after executing a first rule to determine a first body part, a second rule can be executed that uses both the landmarks and the first body part to determine a second body part. In this way, sub-sections of anatomy can be identified. For example, the first body part may specify the presence of the heart in the image and the second body part may indicate specify chambers, arteries, etc.
- Continuing with reference to
FIG. 1A , atstep 120, the list of various body parts and organs are then stored as metadata tags in the medical images. For example, if the processing performed atstep 115, indicates the presence of the right knee, the metadata tag Right Knee' may be generated. The correspondence between anatomical objects and the metadata tags may be established, for example, using a dictionary lookup system. That is, given an anatomical object, the algorithm returns the particular tag. In some embodiments, to detect the body part(s) and organ(s) in the medical image, every anatomical object has a defined upper bound and a lower bound. As an input to the dictionary lookup system, the uppermost (or "upper bound") landmark and the lowermost (or "lower bound") landmark may be used. For example,FIG. 1B show an example of medical images where the upper bound landmark is labeled as "Right Lung Top" and the lower bound landmark is labeled as "Right Hip Bone." To output the result, the dictionary lookup system looks which organs and body parts have both their bounds between the uppermost and lowermost landmark of the image. In some embodiments, each anatomical object also has a defined left bound and a right bound between which the object must be located. The terms "left" and "right" in this context are with reference to the coordinate system that defines the upper and lower bounds. - The descriptors output by the landmarking algorithm (e.g., "Right Lung Top") can be used directly as the tags in the metadata. Alternatively, in some embodiments, the dictionary lookup algorithm may be robust enough to provide different tags for a particular anatomical object based on one or more input parameters. For example, one medical institution may use the tag "Upper Liver" to refer to the upper portion of the liver, while another medical institution may use the tag "Liver Top." The medical institution or another identifier could be provided as an input to the dictionary lookup system to allow it to select the appropriate tag for a particular institution.
- Once all of the metadata tags have been established, they are stored in correspondence with the medical image at
step 125. For example, if the format of the image supports embedded metadata, the tags may be stored directly in the image itself. If embedded metadata is not supported, the tags may be saved in a separate file with a reference to the image. Alternatively, in some embodiments, a data model (e.g., json) is developed to define a custom meta model for body part / organ / type and reference to the image itself. This meta information can then be indexed for an easy, unstructured search. Finally, atstep 125, the image with its metadata is stored in an archive or other data repository for later access and analysis. Once stored, the images are searchable by one or more users via their metadata tags. - In some instances, the image used as input may already comprise DICOM tags or other metadata with anatomical objects. In some embodiments, the
method 100 may be configured to ignore this metadata and only consider as input the landmarks and their position. In this way, themethod 100 solves the problem where medical images do not contain metadata information (i.e., NIfTI) or the DICOM header (meta data) is empty or inaccurate. -
FIG. 2 illustrates analternative method 200 for detecting and classifying medical images based on landmarks detected in the images. In this example, the rule-based approach is replaced by a machine-learning approach, allowing the rules for body part and organ definition to be fine-tuned via a machine learning based method. Here steps 205 and 210 may be implemented in the same manner as discussed above with reference tosteps method 100 ofFIG. 1A . However, atstep 215, instead of the rules-based approach, a machine learning model is used to identify metadata tags based on landmarks. Using such a machine-learning approach, the rules for body part and organ definition can be fine-tuned via a machine learning. Examples of machine learning models that may be applied atstep 215 include support vector machines (SVMs), decision trees or random forests, standard neural networks, and convolutional neural networks. - Various techniques may be used for training the machine learning model used at
step 215. In general, training is performed by providing a set of images with labeled landmarks and corresponding body parts. This training set is preferably large and diverse enough such that the trained model can support a wide range of anatomies and views. Various techniques may be used for labeling the data. For example, in some embodiments, the machine learning model is trained using medical images labeled with an upper bound landmark, a lower bound landmark, and one or more labeled anatomical objects located between the upper bound landmark and the lower bound landmark. The medical images may be further labeled with a left bound landmark and a right bound landmark, and the one or more labeled anatomical objects are located between the left bound landmark and the right bound landmark. The details of the training will depend on the type and characteristics of the model being used. For example, in a random forest implementation, a plurality of decision trees is generated with each decision tree comprising a randomly selected subset of the landmarks. To generate a tree, a subset is split into daughter nodes by considering the landmarks in the training data. During deployment, as new landmarks are received, the rules of each randomly generated decision tree are used to predict an anatomical object. The votes of each predicted anatomical object are calculated and the highest voted object is considered the most like object corresponding to the input landmarks. - Continuing with reference to
FIG. 2 , atstep 220, the list of various body parts and organs output by the machine learning model is applied as metadata information to the medical images. Once all of the metadata tags have been established, they are stored in a data repository in correspondence with the medical image atstep 225. Once in the data repository, the images are searchable via their metadata tags. The details of implementingsteps steps FIG. 1A . -
FIG. 3 illustrates a system that could be used to implement themethod 200 illustrated inFIG. 2 . In this example, a magnetic resonance imaging (MRI)scanner 305 sends image data to animage processing system 335. Within theimage processing system 335, an extract, transform, and load (ETL)module 310 transforms the image, as necessary to be used as input to thelandmark engine 315. Additionally, the ETL module may extract relevant information (e.g., image dimensions, modality, etc.) to generate parameters for thelandmark engine 315. Thelandmark engine 315 executes as described above with reference toFIG. 1A to identify a set of landmarks. Then, the machine learning model(s) 320 are executed on the set of landmark marks to produce a tagged image (i.e., an image with the appropriate body parts, etc. stored in metadata tags of the image. Adata output module 325 is configured to communicate with adata repository 345 to store the tagged image. Additionally, alogging module 330 in theimage processing system 335 records the tagged image and possibly other information used in processing the image (e.g., parameters to thelandmark engine 315 and machine learning models 320). In this way, operation of theimage processing system 335 can be verified and validated. Additionally, thelogging module 330 may be used to debug theimage processing system 335 if any erroneous tags or other data are detected. - Once the tagged image is stored in the
data repository 345, auser 340 can access, retrieve, and use the tagged images. In some instances, theuser 340 may provide feedback to theimage processing system 335. In the example ofFIG. 3 , this feedback is provided to theimage processing system 335; however, in other embodiments, theuser 340 may provide the feedback to thedata repository 345 which, in turn, relays the feedback to theimage processing system 335. In some embodiments, the user may provide explicit feedback such as a rating, an accuracy measurement, etc. The feedback could also be to provide a ticket to correct a tag that was mis-tagged (because machine learning based systems are not 100% accurate in the beginning). In other embodiments, the feedback can be in the form of modification of the tags by theuser 340. That is, theimage processing system 335 may automatically detect any additions, deletions, or modifications to the metadata tags by theuser 340. For example, theimage processing system 335 may periodically review the contents of thedata repository 345 to identify files with metadata tags modified by theuser 340. Theimage processing system 335 may then use the image to retrain themachine learning models 320 to further increase their overall accuracy. -
FIG. 4 illustrates anexemplary computing environment 400 within which embodiments of the invention may be implemented. Thecomputing environment 400 includescomputer system 410, which is one example of a computing system upon which embodiments of the invention may be implemented. Computers and computing environments, such ascomputer system 410 andcomputing environment 400, are known to those of skill in the art and thus are described briefly herein. - As shown in
FIG. 4 , thecomputer system 410 may include a communication mechanism such as a bus 421 or other communication mechanism for communicating information within thecomputer system 410. Thecomputer system 410 further includes one ormore processors 420 coupled with the bus 421 for processing the information. Theprocessors 420 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. - The
computer system 410 also includes asystem memory 430 coupled to the bus 421 for storing information and instructions to be executed byprocessors 420. Thesystem memory 430 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 431 and/or random access memory (RAM) 432. Thesystem memory RAM 432 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). Thesystem memory ROM 431 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, thesystem memory 430 may be used for storing temporary variables or other intermediate information during the execution of instructions by theprocessors 420. A basic input/output system (BIOS) 433 contains the basic routines that help to transfer information between elements withincomputer system 410, such as during start-up, may be stored inROM 431.RAM 432 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by theprocessors 420.System memory 430 may additionally include, for example,operating system 434,application programs 435,other program modules 436 andprogram data 437. Theapplication programs 435 may include, for example, the ETL module, the landmarking engine, the machine learning models, and the other components of the image processing system described above with reference toFIG. 3 . - The
computer system 410 also includes adisk controller 440 coupled to the bus 421 to control one or more storage devices for storing information and instructions, such as ahard disk 441 and a removable media drive 442 (e.g., floppy disk drive, compact disc drive, tape drive, and/or solid state drive). The storage devices may be added to thecomputer system 410 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). - The
computer system 410 may also include adisplay controller 465 coupled to the bus 421 to control adisplay 466, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes aninput interface 460 and one or more input devices, such as akeyboard 462 and apointing device 461, for interacting with a computer user and providing information to theprocessors 420. Thepointing device 461, for example, may be a mouse, a trackball, or a pointing stick for communicating direction information and command selections to theprocessors 420 and for controlling cursor movement on thedisplay 466. Thedisplay 466 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by thepointing device 461. - The
computer system 410 may perform a portion or all of the processing steps of embodiments of the invention in response to theprocessors 420 executing one or more sequences of one or more instructions contained in a memory, such as thesystem memory 430. Such instructions may be read into thesystem memory 430 from another computer readable medium, such as ahard disk 441 or aremovable media drive 442. Thehard disk 441 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. Theprocessors 420 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained insystem memory 430. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software. - As stated above, the
computer system 410 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term "computer readable medium" as used herein refers to any medium that participates in providing instructions to theprocessor 420 for execution. A computer readable medium may take many forms including, but not limited to, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such ashard disk 441 or removable media drive 442. Non-limiting examples of volatile media include dynamic memory, such assystem memory 430. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the bus 421. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications. - The
computing environment 400 may further include thecomputer system 410 operating in a networked environment using logical connections to one or more image scanners such asimaging device 480. Theimaging device 480 may be a radiology scanner such as a magnetic resonance (MR) scanner, PET/MR, X-ray or a CT scanner. When used in a networking environment,computer system 410 may includemodem 472 for establishing communications with theimaging device 480 or a remote computing system over anetwork 471, such as the Internet.Modem 472 may be connected to bus 421 viauser network interface 470, or via another appropriate mechanism. It should be noted that, although theimaging device 480 is illustrated as being connected to thecomputer system 410 over thenetwork 471 in the example presented inFIG. 4 , in other embodiments of the present invention, thecomputer system 410 may be directly connected to theimage scanner 480. For example, in one embodiment thecomputer system 410 and theimage scanner 480 are co-located in the same room or in adjacent rooms, and the devices are connected using any transmission media generally known in the art. -
Network 471 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication betweencomputer system 410 and other computers (e.g., remote computer 480). Thenetwork 471 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-11 or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in thenetwork 471. - The embodiments of the present disclosure may be implemented with any combination of hardware and software. In addition, the embodiments of the present disclosure may be included in an article of manufacture (e.g., one or more computer program products) having, for example, computer-readable, non-transitory media. The media has embodied therein, for instance, computer readable program code for providing and facilitating the mechanisms of the embodiments of the present disclosure. The article of manufacture can be included as part of a computer system or sold separately.
- While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
- Unless stated otherwise as apparent from the following discussion, it will be appreciated that terms such as "segmenting," "generating," "registering," "determining," "aligning," "positioning," "processing," "computing," "selecting," "estimating," "detecting," "tracking" or the like may refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (e.g., electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. Embodiments of the methods described herein may be implemented using computer software. If written in a programming language conforming to a recognized standard, sequences of instructions designed to implement the methods can be compiled for execution on a variety of hardware platforms and for interface to a variety of operating systems. In addition, embodiments of the present invention are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement embodiments of the present invention.
- As used herein, the term "image" refers to multi-dimensional data composed of discrete image elements (e.g., pixels for 2-D images and voxels for 3-D images). The image may be, for example, a medical image of a subject collected by computer tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to one of skill in the art. The image may also be provided from non-medical contexts, such as, for example, remote sensing systems, electron microscopy, etc. The techniques described herein may generally be applied to images of any dimension, e.g., a 2-D picture or a 3-D volume. For a 2- or 3-dimensional image, the domain of the image is typically a 2- or 3-dimensional rectangular array, wherein each pixel or voxel can be addressed with reference to a set of 2 or 3 mutually orthogonal axes. The terms "digital" and "digitized" as used herein will refer to images or volumes, as appropriate, in a digital or digitized format acquired via a digital acquisition system or via conversion from an analog image.
- An executable application, as used herein, comprises code or machine readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.
- A "graphical user interface" (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.
- The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.
The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and/or combinations thereof. No claim element herein is to be construed under the provisions of 35 U.S.C. 112(f) the element is expressly recited using the phrase "means for."
Claims (17)
- A computer-implemented method for automatically generating metadata tags for a medical image, the method comprising:receiving (105, 335) a medical image;automatically identifying (110, 315) a set of body landmarks in the medical image using one or more machine learning models (315);applying (115) a set of rules to the set of body landmarks to identify anatomical objects present in the image;generating (120, 325) one or more metadata tags corresponding to the anatomical objects;storing (120, 325) the metadata tags in the medical image; andtransferring (125, 335) the medical image with the metadata tags to a data repository (345).
- The method according to claim 1, wherein the one or more machine learning models (320) comprise a deep reinforcement learning model.
- The method according to claims 1 or 2, wherein each rule in the set of rules defines (i) an upper bound landmark and a lower bound landmark for a corresponding anatomical object or (ii) a left bound landmark and a right bound landmark for the corresponding anatomical object.
- The method according to any of the preceding claims, further comprising:detecting (320, 335) a modification of the medical image by one or more users (340); andbased on the modification of the medical image, automatically modifying (320, 335) one or more rules in the set of rules.
- The method according to claim 4, wherein the modification of the medical image comprising a (i) modification or deletion of one or more of the metadata tags or (ii) an addition of one or more new metadata tags.
- The method according to any of the preceding claims, wherein the medical image is acquired using a medical image scanner (480) and the method is performed immediately following image reconstruction on a computer (335, 400) connected to the medical image scanner (305).
- The method according to claim 6, wherein the medical image scanner (480) is a magnetic resonance imaging (MRI) scanner.
- A computer-implemented method for automatically generating metadata tags for a medical image, the method comprising:receiving (205, 335) a medical image;identifying (210, 335) a set of body landmarks in the medical image;applying (215, 335) one or more machine learning models (320) to the set of body landmarks to identify anatomical objects present in the image;generating (220, 335) one or more metadata tags corresponding to the anatomical objects;storing (220, 335) the metadata tags in the medical image; andtransferring (225) the medical image with the metadata tags to a data repository (345).
- The method according to claim 8, wherein the one or more machine learning models (320) comprise a random forest.
- The method according to claim 9, wherein the random forest is trained using a plurality of medical images labeled with an upper bound landmark, a lower bound landmark, and one or more labeled anatomical objects located between the upper bound landmark and the lower bound landmark.
- The method according to claim 10, wherein the plurality of medical images are further labeled with a left bound landmark and a right bound landmark, and the one or more labeled anatomical objects are located between the left bound landmark and the right bound landmark.
- The method according to any of the preceding claims 8 to 11, further comprising:detecting (320, 335) a modification of the medical image by one or more users (340); andbased on the modification of the medical image, retraining (320, 335) the one or more machine learning models (320).
- The method according to claim 12, wherein the modification of the medical image comprising (i) a modification or deletion of one or more of the metadata tags and (ii) an addition of one or more new metadata tags.
- The method according to any of the preceding claims 8 to 13, wherein the medical image is acquired using a medical image scanner (305) and the method is performed immediately following image reconstruction on a computer connected to the medical image scanner (305).
- The method according to claim 14, wherein the medical image scanner (305) is a MRI scanner.
- A system for automatically generating metadata tags for a medical image, the system comprising:a medical image scanner (305) configured to acquire a medical image;an image processing system (335, 400) comprising one or more processors (420) configured toautomatically identify (210, 315) a set of body landmarks in the medical image; identify (320) anatomical objects present in the image based on the set of body landmarks;generate (325) one or more metadata tags corresponding to the anatomical objects;store (325) the metadata tags in the medical image; andtransfer (325) the medical image with the metadata tags to a data repository (345).
- The system according to claim 16, wherein the medical image scanner (305) is a MRI scanner.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/810,694 US10489907B2 (en) | 2017-11-13 | 2017-11-13 | Artifact identification and/or correction for medical imaging |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3483895A1 true EP3483895A1 (en) | 2019-05-15 |
Family
ID=64277529
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18205581.4A Withdrawn EP3483895A1 (en) | 2017-11-13 | 2018-11-12 | Detecting and classifying medical images based on continuously-learning whole body landmarks detections |
Country Status (2)
Country | Link |
---|---|
US (1) | US10489907B2 (en) |
EP (1) | EP3483895A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111754436A (en) * | 2020-06-24 | 2020-10-09 | 上海联影医疗科技有限公司 | Acceleration method for medical image artifact correction, computer device and storage medium |
EP3772720A1 (en) * | 2019-08-08 | 2021-02-10 | Siemens Healthcare GmbH | Method and system for image analysis |
CN113539439A (en) * | 2021-07-16 | 2021-10-22 | 数坤(北京)网络科技股份有限公司 | Medical image processing method and device, computer equipment and storage medium |
US20210358595A1 (en) * | 2020-05-12 | 2021-11-18 | Siemens Healthcare Gmbh | Body representations |
DE202022100604U1 (en) | 2022-02-02 | 2022-02-16 | Pankaj Agarwal | Intelligent system for automatic classification of medical images using image processing and artificial intelligence |
EP3975195A1 (en) * | 2020-09-29 | 2022-03-30 | RaySearch Laboratories AB | Method, computer program and computer system for use in the processing of medical images |
Families Citing this family (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10949966B2 (en) * | 2017-08-18 | 2021-03-16 | Siemens Healthcare Gmbh | Detecting and classifying medical images based on continuously-learning whole body landmarks detections |
JP6782675B2 (en) * | 2017-09-29 | 2020-11-11 | 富士フイルム株式会社 | Discrimination result providing device, operation method of discrimination result providing device, discrimination result providing program, and discrimination result providing system |
US10748036B2 (en) * | 2017-11-21 | 2020-08-18 | Nvidia Corporation | Training a neural network to predict superpixels using segmentation-aware affinity loss |
JP7134805B2 (en) * | 2018-09-20 | 2022-09-12 | キヤノンメディカルシステムズ株式会社 | Medical information processing device and medical information processing system |
US10915990B2 (en) * | 2018-10-18 | 2021-02-09 | General Electric Company | Systems and methods for denoising medical images with deep learning network |
US11200975B2 (en) * | 2018-11-06 | 2021-12-14 | International Business Machines Corporation | Framework for modeling collections and their management |
CN112089419A (en) * | 2019-05-29 | 2020-12-18 | 通用电气精准医疗有限责任公司 | Medical imaging method and system, non-transitory computer readable storage medium |
EP3748384A1 (en) | 2019-06-04 | 2020-12-09 | Koninklijke Philips N.V. | Spiral mr imaging with off-resonance artefact correction |
US11100684B2 (en) * | 2019-07-11 | 2021-08-24 | Canon Medical Systems Corporation | Apparatus and method for artifact detection and correction using deep learning |
US12045943B2 (en) | 2019-08-16 | 2024-07-23 | Howmedica Osteonics Corp. | Pre-operative planning of surgical revision procedures for orthopedic joints |
CN110796613B (en) * | 2019-10-10 | 2023-09-26 | 东软医疗系统股份有限公司 | Automatic identification method and device for image artifacts |
CN110910465B (en) | 2019-11-21 | 2023-12-26 | 上海联影医疗科技股份有限公司 | Motion artifact correction method and system |
CA3159947A1 (en) * | 2019-12-02 | 2021-06-10 | Brendan Thomas CRABB | Medical image synthesis for motion correction using generative adversarial networks |
CN111080584B (en) * | 2019-12-03 | 2023-10-31 | 上海联影智能医疗科技有限公司 | Quality control method for medical image, computer device and readable storage medium |
CN114830172A (en) * | 2019-12-18 | 2022-07-29 | 化学影像公司 | System and method for a combined imaging modality for improved tissue detection |
CN111145875B (en) * | 2019-12-27 | 2023-05-12 | 上海联影医疗科技股份有限公司 | Data analysis system |
WO2021150973A1 (en) * | 2020-01-24 | 2021-07-29 | Duke University | Intelligent automated imaging system |
US20230110904A1 (en) * | 2020-01-31 | 2023-04-13 | The General Hospital Corporation | Systems and methods for artifact reduction in tomosynthesis with deep learning image processing |
CN111445447B (en) * | 2020-03-16 | 2024-03-01 | 东软医疗系统股份有限公司 | CT image anomaly detection method and device |
CN111798439A (en) * | 2020-07-11 | 2020-10-20 | 大连东软教育科技集团有限公司 | Medical image quality interpretation method and system for online and offline fusion and storage medium |
CN111863205A (en) * | 2020-07-23 | 2020-10-30 | 山东协和学院 | Accurate image recognition system and image recognition method |
CN113706643B (en) * | 2020-09-09 | 2023-06-30 | 南京邮电大学 | Head CT metal artifact correction method based on homomorphic adaptation learning |
CN112150574B (en) * | 2020-09-28 | 2022-11-08 | 上海联影医疗科技股份有限公司 | Method, system and device for automatically correcting image artifacts and storage medium |
US20220192748A1 (en) | 2020-12-22 | 2022-06-23 | Biosense Webster (Israel) Ltd. | Displaying annotations on design line formed on anatomical map |
US11633168B2 (en) * | 2021-04-02 | 2023-04-25 | AIX Scan, Inc. | Fast 3D radiography with multiple pulsed X-ray sources by deflecting tube electron beam using electro-magnetic field |
US20220319158A1 (en) * | 2021-04-05 | 2022-10-06 | Nec Laboratories America, Inc. | Cell nuclei classification with artifact area avoidance |
CN113256529B (en) * | 2021-06-09 | 2021-10-15 | 腾讯科技(深圳)有限公司 | Image processing method, image processing device, computer equipment and storage medium |
US11330145B1 (en) * | 2021-06-10 | 2022-05-10 | Bank Of America Corporation | Image processing edge device for document noise removal |
CN113539437A (en) * | 2021-06-25 | 2021-10-22 | 李懋 | Method and system for dynamically prompting MR artifacts in diagnostic reporting system |
CN113538613A (en) * | 2021-06-25 | 2021-10-22 | 李懋 | Method and system for recommending scanning scheme and simultaneously dynamically prompting MR scanning artifact |
CN115131452A (en) * | 2022-04-19 | 2022-09-30 | 腾讯医疗健康(深圳)有限公司 | Image processing method and device for artifact removal |
WO2024040280A1 (en) * | 2022-08-26 | 2024-02-29 | Curvebeam Ai Limited | Method and system for removing foreign material from images |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009017715A1 (en) * | 2007-08-02 | 2009-02-05 | Siemens Medical Solutions Usa, Inc. | Joint detection and localization of multiple anatomical landmarks through learning |
WO2016036516A1 (en) * | 2014-09-02 | 2016-03-10 | Impac Medical Systems, Inc. | Systems and methods for segmenting medical images based on anatomical landmark-based features |
US20160350919A1 (en) * | 2015-06-01 | 2016-12-01 | Virtual Radiologic Corporation | Medical evaluation machine learning workflows and processes |
US9569736B1 (en) | 2015-09-16 | 2017-02-14 | Siemens Healthcare Gmbh | Intelligent medical image landmark detection |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5812628A (en) | 1996-12-12 | 1998-09-22 | General Electric Company | Methods and apparatus for detecting partial volume image artifacts |
US9289153B2 (en) * | 1998-09-14 | 2016-03-22 | The Board Of Trustees Of The Leland Stanford Junior University | Joint and cartilage diagnosis, assessment and modeling |
US6517488B1 (en) * | 2000-06-29 | 2003-02-11 | Acuson Corporation | Medical diagnostic ultrasound system and method for identifying constrictions |
WO2010063015A1 (en) * | 2008-11-27 | 2010-06-03 | Sonocine, Inc. | System and method for location of anomalies in a body scan |
US20100160768A1 (en) * | 2008-12-24 | 2010-06-24 | Marrouche Nassir F | Therapeutic outcome assessment for atrial fibrillation |
WO2012056379A1 (en) | 2010-10-27 | 2012-05-03 | Koninklijke Philips Electronics N.V. | Image artifact identification and mitigation |
US9554772B2 (en) * | 2014-03-05 | 2017-01-31 | Mammen Thomas | Non-invasive imager for medical applications |
US9427205B1 (en) * | 2015-03-20 | 2016-08-30 | General Electic Company | Systems and methods for artifact removal for computed tomography imaging |
US10430688B2 (en) * | 2015-05-27 | 2019-10-01 | Siemens Medical Solutions Usa, Inc. | Knowledge-based ultrasound image enhancement |
US10210613B2 (en) * | 2016-05-12 | 2019-02-19 | Siemens Healthcare Gmbh | Multiple landmark detection in medical images based on hierarchical feature learning and end-to-end training |
-
2017
- 2017-11-13 US US15/810,694 patent/US10489907B2/en active Active
-
2018
- 2018-11-12 EP EP18205581.4A patent/EP3483895A1/en not_active Withdrawn
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009017715A1 (en) * | 2007-08-02 | 2009-02-05 | Siemens Medical Solutions Usa, Inc. | Joint detection and localization of multiple anatomical landmarks through learning |
WO2016036516A1 (en) * | 2014-09-02 | 2016-03-10 | Impac Medical Systems, Inc. | Systems and methods for segmenting medical images based on anatomical landmark-based features |
US20160350919A1 (en) * | 2015-06-01 | 2016-12-01 | Virtual Radiologic Corporation | Medical evaluation machine learning workflows and processes |
US9569736B1 (en) | 2015-09-16 | 2017-02-14 | Siemens Healthcare Gmbh | Intelligent medical image landmark detection |
Non-Patent Citations (2)
Title |
---|
MICROSOFT RESEARCH: "Research in Focus: Project InnerEye - Assistive AI for Cancer Treatment", YOUTBE, 18 July 2017 (2017-07-18), pages 1 pp., XP054979253, Retrieved from the Internet <URL:https://www.youtube.com/watch?v=jaFTXi56bFI> [retrieved on 20190329] * |
MUEEN A ET AL: "Automatic Multilevel Medical Image Annotation and Retrieval", JOURNAL OF DIGITAL IMAGING ; THE JOURNAL OF THE SOCIETY FOR COMPUTER APPLICATIONS IN RADIOLOGY, SPRINGER-VERLAG, NE, vol. 21, no. 3, 11 September 2007 (2007-09-11), pages 290 - 295, XP019596204, ISSN: 1618-727X * |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3772720A1 (en) * | 2019-08-08 | 2021-02-10 | Siemens Healthcare GmbH | Method and system for image analysis |
US11282203B2 (en) | 2019-08-08 | 2022-03-22 | Siemens Healthcare Gmbh | Method and system for image analysis |
US20210358595A1 (en) * | 2020-05-12 | 2021-11-18 | Siemens Healthcare Gmbh | Body representations |
US11837352B2 (en) * | 2020-05-12 | 2023-12-05 | Siemens Healthcare Gmbh | Body representations |
CN111754436A (en) * | 2020-06-24 | 2020-10-09 | 上海联影医疗科技有限公司 | Acceleration method for medical image artifact correction, computer device and storage medium |
CN111754436B (en) * | 2020-06-24 | 2024-05-03 | 上海联影医疗科技股份有限公司 | Acceleration method for medical image artifact correction, computer device and storage medium |
EP3975195A1 (en) * | 2020-09-29 | 2022-03-30 | RaySearch Laboratories AB | Method, computer program and computer system for use in the processing of medical images |
CN113539439A (en) * | 2021-07-16 | 2021-10-22 | 数坤(北京)网络科技股份有限公司 | Medical image processing method and device, computer equipment and storage medium |
DE202022100604U1 (en) | 2022-02-02 | 2022-02-16 | Pankaj Agarwal | Intelligent system for automatic classification of medical images using image processing and artificial intelligence |
Also Published As
Publication number | Publication date |
---|---|
US20190147588A1 (en) | 2019-05-16 |
US10489907B2 (en) | 2019-11-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3483895A1 (en) | Detecting and classifying medical images based on continuously-learning whole body landmarks detections | |
EP3444824B1 (en) | Detecting and classifying medical images based on continuously-learning whole body landmarks detections | |
US11176188B2 (en) | Visualization framework based on document representation learning | |
US10902588B2 (en) | Anatomical segmentation identifying modes and viewpoints with deep learning across modalities | |
US20240225447A1 (en) | Dynamic self-learning medical image method and system | |
Dikici et al. | Integrating AI into radiology workflow: levels of research, production, and feedback maturity | |
US9892361B2 (en) | Method and system for cross-domain synthesis of medical images using contextual deep network | |
EP3246836A1 (en) | Automatic generation of radiology reports from images and automatic rule out of images without findings | |
Sander et al. | Towards increased trustworthiness of deep learning segmentation methods on cardiac MRI | |
US11621075B2 (en) | Systems, methods, and apparatus for diagnostic inferencing with a multimodal deep memory network | |
JP2020530177A (en) | Computer-aided diagnosis using deep neural network | |
US7889898B2 (en) | System and method for semantic indexing and navigation of volumetric images | |
CN109460756B (en) | Medical image processing method and device, electronic equipment and computer readable medium | |
US20130136322A1 (en) | Image-Based Detection Using Hierarchical Learning | |
KR20190117969A (en) | Method for semi supervised reinforcement learning using data with label and data without label together and apparatus using the same | |
JP2017533522A (en) | Picture archiving system with text image linking based on text recognition | |
CN113656706A (en) | Information pushing method and device based on multi-mode deep learning model | |
Khakzar et al. | Learning interpretable features via adversarially robust optimization | |
Filice | Radiology-pathology correlation to facilitate peer learning: an overview including recent artificial intelligence methods | |
US20200321098A1 (en) | System and method for viewing medical image | |
WO2023219836A1 (en) | Method for automating radiology workflow | |
WO2023274599A1 (en) | Methods and systems for automated follow-up reading of medical image data | |
de Araujo et al. | Data preparation for artificial intelligence | |
JP7478518B2 (en) | Image interpretation support device and image interpretation support method | |
US20240037920A1 (en) | Continual-learning and transfer-learning based on-site adaptation of image classification and object localization modules |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20181112 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20191116 |