CN111161231A - System and method for enhancing a patient positioning system - Google Patents

System and method for enhancing a patient positioning system Download PDF

Info

Publication number
CN111161231A
CN111161231A CN201911346034.5A CN201911346034A CN111161231A CN 111161231 A CN111161231 A CN 111161231A CN 201911346034 A CN201911346034 A CN 201911346034A CN 111161231 A CN111161231 A CN 111161231A
Authority
CN
China
Prior art keywords
image
neural network
quality assessment
feedback
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911346034.5A
Other languages
Chinese (zh)
Inventor
阿比舍克·沙玛
阿伦·因南耶
吴子彦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai United Imaging Intelligent Healthcare Co Ltd
Original Assignee
Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai United Imaging Intelligent Healthcare Co Ltd filed Critical Shanghai United Imaging Intelligent Healthcare Co Ltd
Publication of CN111161231A publication Critical patent/CN111161231A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/542Event management; Broadcasting; Multicasting; Notifications
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • G06N20/20Ensemble learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00General purpose image data processing
    • G06T1/0007Image acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/40ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30168Image quality inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Epidemiology (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

A method and system for acquiring medical images using a medical imaging device. For example, a computer-implemented method for acquiring medical images of a patient with a medical imaging device includes: determining a first positioning instruction through a first neural network, and acquiring a first image based on the first positioning instruction; receiving a first image; identifying one or more first features associated with the acquired first image; determining a first quality assessment based on the one or more first features; generating a first feedback based on the first quality assessment; receiving first feedback through a first neural network; and changing one or more first parameters of the first neural network based on the first feedback.

Description

System and method for enhancing a patient positioning system
Technical Field
Certain embodiments of the present invention are directed to locating an object. More specifically, some embodiments of the present invention provide methods and systems for positioning a patient. By way of example only, some embodiments of the invention are applied to enhance patient positioning systems. It will be appreciated that the invention has broader applicability.
Background
Conventional patient positioning systems, such as patient positioning systems for medical CT, MR, X-ray or ultrasound scanners, are prone to errors in estimating the degree of deviation of the patient's posture from a reference posture. Due to such errors, multiple scans are often required to obtain satisfactory scanning results. Once a conventional patient positioning system is deployed to a hospital, manual examination and labeling is relied upon to identify errors in the medical scan that are produced by the scanner. Some patient positioning systems may receive infrequent updates, such as monthly or seasonal, to include manual feedback to help reduce the estimation of errors. Such a procedure is inefficient, and therefore, it is desirable to have a method and system that enhances the patient positioning system with greater efficiency to improve patient positioning.
Disclosure of Invention
Certain embodiments of the present invention are directed to locating an object. More specifically, some embodiments of the present invention provide methods and systems for positioning a patient. By way of example only, some embodiments of the invention are applied to enhance patient positioning systems. It will be appreciated that the invention has broader applicability.
In various embodiments, a computer-implemented method for acquiring medical images of a patient with a medical imaging device, comprising: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network previously trained for positioning; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving the first image; identifying one or more first features associated with the acquired first image; determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating a first feedback based at least in part on the first quality assessment; receiving first feedback through a first neural network; and changing one or more first parameters of a previously trained first neural network based at least in part on the first feedback. In certain examples, the computer-implemented method is performed by one or more processors.
In various embodiments, a system for a computer-implemented method for acquiring medical images of a patient with a medical imaging device, comprising: a protocol receiving module configured to receive a scan protocol; an instruction determination module configured to determine a first positioning instruction based at least in part on a scanning protocol by a first neural network, the first neural network previously trained for positioning; an image acquisition module configured to acquire a first image based at least in part on a first positioning instruction and a scanning protocol; an image receiving module configured to receive a first image; a feature identification module configured to identify one or more first features associated with the acquired first image; a quality assessment module configured to determine a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; a feedback generation module configured to generate a first feedback based at least in part on the first quality assessment; a feedback receiving module configured to receive first feedback through a first neural network; and a parameter change module configured to change one or more first parameters of a previously trained first neural network based at least in part on the first feedback.
In various embodiments, a non-transitory computer readable medium having stored thereon instructions that, when executed by a processor, perform processes comprising: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network previously trained for positioning; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving the first image; identifying one or more first features associated with the acquired first image; determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating a first feedback based at least in part on the first quality assessment; receiving, by a first neural network, first feedback from a second neural network; and changing one or more first parameters of a previously trained first neural network based at least in part on the first feedback.
In various embodiments, a method for acquiring medical images of a patient with a medical imaging device includes: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network previously trained for positioning; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving a first image; identifying one or more first features associated with the acquired first image; determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating a first feedback based at least in part on the first quality assessment; receiving first feedback through a first neural network; and changing, by the first neural network, one or more first parameters of the first neural network that was previously trained based at least in part on the first feedback.
According to embodiments, one or more benefits may be realized. These benefits and various additional objects, features and advantages of the present invention can be fully understood with reference to the detailed description and accompanying drawings that follow.
Drawings
Fig. 1 is a simplified diagram illustrating a system for acquiring medical images of a patient with a medical imaging device, in accordance with some embodiments.
Fig. 2 is a simplified diagram illustrating a method for acquiring medical images of a patient with a medical imaging device, in accordance with some embodiments.
Fig. 3 is a simplified diagram illustrating a method for acquiring medical images of a patient with a medical imaging device, in accordance with some embodiments.
FIG. 4 is a simplified diagram illustrating a computing system according to some embodiments.
Figure 5 is a simplified diagram illustrating a neural network according to some embodiments.
Detailed Description
Certain embodiments of the present invention are directed to locating an object. More specifically, some embodiments of the present invention provide methods and systems for positioning a patient. By way of example only, some embodiments of the invention are applied to enhance patient positioning systems. It will be appreciated that the invention has broader applicability.
Fig. 1 is a simplified diagram illustrating a system for acquiring medical images of a patient with a medical imaging device according to some embodiments of the present invention. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In some examples, system 10 includes a protocol receiving module 12, an instruction determining module 14, an image acquisition module 16, an image receiving module 18, a feature identification module 20, a quality assessment module 22, a feedback generation module 24, a feedback receiving module 26, and a parameter changing module 28. In some examples, system 10 also includes a redundant objects module 30, a training module 32, and/or an image selection module 34. In different examples, system 10 is configured to augment a patient positioning system and/or system 10 is a patient positioning system configured to be self-augmented. While the above is shown using selected components, there are many alternatives, modifications, and variations. For example, some of the components may be expanded and/or combined. Other components may also be inserted like those indicated above. The arrangement of these components may be interchanged with other components substituted in accordance with the embodiment.
In various embodiments, the protocol receiving module 12 is configured to receive a scanning protocol, such as a scanning protocol selected by a user. In some examples, the scan protocol includes a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate. In various examples, the scan protocol is selected from a menu, for example, through a user interface.
In various embodiments, the instruction determination module 14 is configured to determine the positioning instructions based at least in part on the scanning protocol (e.g., the scanning protocol received by the protocol reception module 12). In some examples, instruction determination module 14 includes or is configured to use a localization neural network. In some examples, the localization neural network is a neural network that is trained (e.g., previously trained) for localizing the subject. In some examples, the object is part of a patient. In certain examples, the instruction determination module 14 is configured to determine the positioning instruction based at least in part on a relative position between the patient position (e.g., the target position) and the reference position. For example, the positioning instructions include directions for adjusting the position of an imaging system (e.g., a scanning probe) and/or directions for adjusting the position of a patient or a portion of a patient. In some examples, the patient position is acquired based at least in part on a patient image acquired by the imaging system. In some examples, the reference location is selected based at least in part on a scanning protocol and/or patient information. In some examples, instruction determination module 14 is further configured to determine the target region based at least in part on the scanning protocol. For example, the target region includes a body part and/or a body organ. In some examples, instruction determination module 14 is further configured to determine a scanning technique (e.g., based at least in part on a scanning protocol) and determine a scanning path based at least in part on the scanning technique.
In various embodiments, the image acquisition module 16 is configured to acquire images based at least in part on positioning instructions (e.g., positioning instructions determined by the instruction determination module 14) and/or scanning protocols (e.g., scanning protocols received by the protocol reception module 12). In some examples, the image acquisition module 16 is configured to send positioning instructions to a medical imaging device (e.g., a scanner), such as to a positioning system (e.g., a robotic scanning platform and/or robotic arm) of the medical imaging device, in order to position the target (e.g., a patient). In some examples, image acquisition module 16 is configured to send imaging instructions to an imaging system (e.g., a scanning probe) to acquire images according to a scanning protocol. In various examples, image acquisition module 16 is configured to acquire an image by selecting an image from a pre-generated image database that includes one or more images previously acquired.
In various embodiments, image receiving module 18 is configured to receive an image. For example, the image receiving module 18 is configured to receive images through a quality assessment neural network, such as a neural network trained (e.g., previously trained) for quality assessment. In some examples, image receiving module 18 is configured to input the image into a quality assessment neural network for quality assessment. In some examples, the quality assessment is referred to as quality assurance.
In various embodiments, feature identification module 20 is configured to identify one or more features associated with an image (e.g., an image acquired by image acquisition module 16). In some examples, feature recognition module 20 is configured to identify one or more features associated with the image, for example, using a neural network trained to identify and/or extract the one or more features. In some examples, the neural network trained to identify the one or more features is the same as the neural network trained for quality assessment. In different examples, the feature identification module 20 is configured to identify landmarks (landmark), visual features, geometric shapes and/or superfluous objects, for example, as features.
In various embodiments, quality assessment module 22 is configured to determine a quality assessment based at least in part on one or more features (e.g., one or more features identified by feature identification module 20) by a neural network trained for identifying one or more features and/or by a neural network trained for quality assessment. In some examples, the quality assessment is associated with an image (e.g., an image from which one or more features are identified). In some examples, the quality assessment is a quality score, such as a number from zero to one. In some examples, quality assessment module 22 is configured to compare the identified one or more features (e.g., features identified by feature identification module 20) to a list of target features and identify one or more missing features. In various examples, the list of target features includes one or more target features that, if identified as being in an image, facilitate a satisfactory quality assessment. For example, the more target features identified from an image, the higher the quality score of the image.
In various embodiments, feedback generation module 24 is configured to generate feedback based at least in part on a quality assessment, such as a quality assessment determined by a quality assessment neural network and/or a quality assessment determined by a neural network trained (e.g., previously trained) for the quality assessment. In some examples, the feedback is a quality assessment. In some examples, the feedback is a quality score. In different examples, the feedback includes an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and/or a translational rotational deviation matrix.
In various embodiments, feedback receiving module 26 is configured to receive feedback (e.g., feedback generated by feedback generating module 24), for example, by a positioning neural network (e.g., a neural network trained for positioning). In some examples, feedback receiving module 26 is configured to receive feedback from a quality assessment neural network (e.g., a neural network trained for quality assessment). In different examples, feedback receiving module 26 is configured to transfer or direct feedback (e.g., feedback generated by feedback generating module 24) from the quality assessment neural network to the positioning neural network.
In various embodiments, parameter changing module 28 is configured to change one or more parameters of a positioning neural network (e.g., a neural network trained for positioning) based at least in part on feedback (e.g., feedback generated by feedback generating module 24). In some examples, parameter change module 28 is part of the localization neural network and/or is configured to iteratively change one or more parameters of the localization neural network. For example, if the quality assessment generated by the quality assessment neural network is not satisfactory (e.g., when compared to a quality threshold), after changing one or more parameters of the positioning neural network, the instruction determination module 14 is further configured to determine updated positioning instructions by the positioning neural network, image acquisition module 16 is further configured to acquire a substitute image, image receiving module 18 is configured to receive the substitute image, feature identification module 20 is configured to identify one or more updated features from the substitute image, quality assessment module 22 is configured to determine an updated quality assessment corresponding to the substitute image, and if the updated quality assessment is not satisfactory (e.g., when compared to a quality threshold), the parameter change module 28 is further configured to change one or more parameters of the positioning neural network.
In various embodiments, the redundant object module 30 is configured to prompt a removal instruction (e.g., prompt a user for a removal instruction) to remove the redundant object. For example, redundant object module 30 is configured to prompt a removal instruction for removing the redundant object if the one or more features identified by feature identification module 20 include a redundant object. In some examples, the redundant object module 30 is configured to acquire a substitute image, for example, by controlling the image acquisition module 16. In some examples, the redundant object module 30 is configured to acquire a substitute image according to positioning instructions and a scanning protocol, for example, after the redundant object is removed. In a different example, the redundant object module 30 is configured to determine a substitute positioning instruction as the redundant object is avoided and to acquire a substitute image according to the substitute positioning instruction. In some examples, the substitute image is referred to as a substitute image.
In various embodiments, training module 32 is configured to train a quality assessment neural network (e.g., a neural network trained or to be trained for quality assessment). In some examples, training module 32 is configured to receive training medical images through a quality assessment neural network. For example, the training module 32 is configured to input the training medical image into a quality assessment neural network. In some examples, training module 32 is configured to receive a target output (e.g., label data (ground route)) associated with the training medical image. In various examples, training module 32 is configured to generate a training assessment (e.g., a training score) associated with the training medical image, for example, by utilizing a quality assessment neural network. In some examples, training module 32 is configured to generate training feedback based at least in part on the training assessment and/or the target output. In some examples, training module 32 is configured to change one or more parameters of the quality assessment neural network based at least in part on the training feedback. In some examples, the training feedback includes an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and/or a translational-rotational deviation matrix (e.g., a translational-rotational deviation matrix including translational and/or rotational elements). In some examples, training module 32 is configured to determine the loss based at least in part on the training assessment and/or the target output. In various examples, training module 32 is configured to change one or more parameters based at least in part on the loss using a gradient descent-based machine learning framework.
In various embodiments, the image selection module 34 is configured to select an image to be a medical image, such as a medical image to be output to a display. In some examples, image selection module 34 is configured to select an image corresponding to the quality assessment as the medical image if the quality assessment (e.g., the quality assessment generated by quality assessment module 22) satisfies a predetermined quality threshold. In some examples, image selection module 34 is configured to determine an image corresponding to the quality assessment as an image not eligible to be selected as a medical image if the quality assessment does not meet a predetermined quality threshold.
Fig. 2 is a simplified diagram illustrating a method for acquiring medical images of a patient with a medical imaging device, according to some embodiments of the invention. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In some examples, the method S100 includes a process of receiving a scan protocol S102, a process of determining a positioning instruction by a positioning neural network S104, a process of acquiring an image S106, a process of receiving an image by a quality assessment neural network S108, a process of identifying one or more features by a quality assessment neural network S110, a process of determining a quality assessment by a quality assessment neural network S112, a process of generating feedback by a quality assessment neural network S114, a process of receiving feedback by a positioning neural network S116, and a process of changing one or more parameters of a positioning neural network S118. Although a selected set of processes has been shown above for use in the method, many alternatives, modifications and variations are possible. For example, some of the processes may be extended and/or combined. Other procedures may also be inserted into those indicated above. The order of these processes may be interchanged with other processes as alternatives according to the embodiment.
In various embodiments, the process S102 of receiving a scan protocol includes receiving a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate. In some examples, receiving the scan protocol includes selecting the scan protocol from a menu (e.g., via a user interface).
In various embodiments, the process of determining the positioning instructions by a positioning neural network (e.g., a neural network trained to position the object) S104 includes determining the positioning instructions based at least in part on the scanning protocol. In certain examples, determining the positioning instructions includes determining the positioning instructions based at least in part on a relative position between the patient position (e.g., the position of the target object) and the reference position. For example, determining positioning instructions includes determining directions for adjusting a position of an imaging system (e.g., a scanning probe) and/or determining directions for adjusting a position of a patient or a portion of a patient. In some examples, determining the positioning instructions includes acquiring a patient position, e.g., based at least in part on the acquired patient images. In certain examples, determining the positioning instructions includes selecting a reference location based at least in part on the scanning protocol and/or the patient information. In some examples, determining the positioning instructions includes determining the target region based at least in part on a scanning protocol. For example, determining the positioning instructions includes determining a body part and/or a body organ. In some examples, determining the positioning instructions includes determining a scanning technique (e.g., determining the scanning technique based at least in part on a scanning protocol) and determining a scanning path based at least in part on the scanning technique.
In various embodiments, the process of acquiring an image S106 includes acquiring an image based at least in part on the positioning instructions and/or the scanning protocol. In some examples, acquiring the image includes sending positioning instructions to a medical imaging device (e.g., a scanner), such as to a positioning system (e.g., a robotic scanning platform and/or robotic arm) of the medical imaging device, to position the target (e.g., a patient). In some examples, acquiring the image includes sending imaging instructions to an imaging system (e.g., a scanning probe) to acquire the image according to a scanning protocol. In various examples, acquiring the image includes acquiring the image by selecting the image from a pre-generated image database that includes one or more images previously acquired.
In various embodiments, the process of receiving an image through a quality assessment neural network S108 includes inputting the image into a quality assessment neural network for quality assessment.
In various embodiments, the process S110 of identifying one or more features by the quality assessment neural network includes identifying one or more features associated with an image (e.g., an image received by the quality assessment neural network). In some examples, identifying the one or more features includes extracting the one or more features associated with the image by a quality assessment neural network. In some examples, identifying the one or more features includes identifying landmarks, visual features, geometric shapes, and/or redundant objects.
In various embodiments, the process S112 of determining a quality assessment by a quality assessment neural network includes determining a quality assessment based at least in part on one or more features (e.g., one or more features identified by a feature extraction neural network and/or a quality assessment neural network). In some examples, determining the quality assessment includes comparing the identified one or more features to a list of target features and identifying one or more missing features. In some examples, determining the quality assessment includes determining the quality assessment based at least in part on the identified one or more missing features.
In various embodiments, the process of generating feedback by the quality assessment neural network S114 includes generating feedback based at least in part on the quality assessment. In some examples, generating the feedback includes using the quality assessment as the feedback. In some examples, generating the feedback includes generating a quality score. In different examples, generating the feedback includes generating an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and/or a translational rotational deviation matrix.
In various embodiments, the process of receiving feedback through the positioning neural network S116 includes receiving feedback from a quality assessment neural network (e.g., a neural network trained for quality assessment). In various examples, receiving the feedback includes transferring or directing the feedback from the quality assessment neural network to a positioning neural network.
In various embodiments, the process of changing one or more parameters of the positioning neural network S118 includes changing one or more parameters of the positioning neural network based at least in part on the feedback and/or the quality assessment. In some examples, one or more of processes S102, S104, S106, S108, S110, S112, S114, S116, and S118 are repeated. For example, if the quality assessment generated by the quality assessment neural network is unsatisfactory (e.g., when compared to a quality threshold), after one or more parameters of the localization neural network change, the method S100 further comprises determining, by the localization neural network, updated localization instructions, obtaining a substitute image, receiving the substitute image, identifying one or more updated features from the substitute image, determining an updated quality assessment corresponding to the substitute image, and changing one or more parameters of the localization neural network if the updated quality assessment is unsatisfactory (e.g., when compared to the quality threshold).
In some embodiments, method S100 further includes prompting a removal instruction (e.g., prompting a user for a removal instruction) to remove the redundant object. For example, if the identified one or more features (e.g., one or more features identified by the quality assessment module) include redundant objects, a cue removal instruction is executed. In some examples, method S100 further includes, for example, after removing the unwanted object, acquiring a substitute image. In some examples, method S100 further includes determining an alternative positioning instruction and acquiring an alternative image according to the alternative positioning instruction as the redundant object is avoided.
In certain embodiments, the method S100 includes training a quality assessment neural network (e.g., prior to executing the receive scan protocol). In some examples, training the quality assessment neural network includes receiving, by the quality assessment neural network, a training medical image, receiving a target output associated with the training medical image, generating a training assessment associated with the training medical image quality assessment neural network, generating training feedback based at least in part on the training assessment and the target output, and changing one or more parameters of the quality assessment neural network based at least in part on the training feedback. In some examples, generating the training feedback includes generating an authenticity classification, a translational rotational bias matrix, a translational bias matrix, and/or a rotational bias matrix. In various examples, generating the training feedback includes determining a loss based at least in part on the training assessment and the target output. In some examples, changing one or more parameters of the quality assessment neural network includes changing the one or more parameters based at least in part on the loss using a gradient descent-based machine learning framework.
In certain embodiments, method S100 includes selecting an image to be a medical image, e.g., a medical image to be output to a display. In some examples, selecting the image includes selecting the image corresponding to the quality assessment as the medical image if the quality assessment satisfies a predetermined quality threshold.
Fig. 3 is a simplified diagram illustrating a method for acquiring medical images of a patient with a medical imaging device according to some embodiments of the invention. This diagram is merely an example, which should not unduly limit the scope of the claims. Many changes, substitutions, and alterations may be recognized by those of ordinary skill in the art. In some examples, the method S200 includes a process S202 of receiving a scanning protocol, a process S204 of determining a positioning instruction by positioning a neural network, a process S206 of acquiring an image, a process S208 of receiving an image, a process S210 of identifying one or more features, a process S212 of determining a quality assessment, a process S214 of generating feedback, a process S216 of receiving feedback by positioning a neural network, and a process S218 of changing one or more parameters of positioning a neural network. Although a selected set of processes has been shown above for use in the method, many alternatives, modifications and variations are possible. For example, some of the processes may be extended and/or combined. Other procedures may also be inserted into those indicated above. The order of these processes may be interchanged with other processes as alternatives according to the embodiment.
In various embodiments, the process S202 of receiving a scan protocol includes receiving a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate. In some examples, receiving the scan protocol includes selecting the scan protocol from a menu (e.g., via a user interface).
In various embodiments, the process S204 of determining the positioning instructions by a positioning neural network (e.g., a neural network trained to position the object) includes determining the positioning instructions based at least in part on the scanning protocol. In certain examples, determining the positioning instructions includes determining the positioning instructions based at least in part on a relative position between the patient position (e.g., the position of the target object) and the reference position. For example, determining positioning instructions includes determining directions for adjusting a position of an imaging system (e.g., a scanning probe) and/or determining directions for adjusting a position of a patient or a portion of a patient. In some examples, determining the positioning instructions includes acquiring a patient position, e.g., based at least in part on the acquired patient images. In certain examples, determining the positioning instructions includes selecting a reference location based at least in part on the scanning protocol and/or the patient information. In some examples, determining the positioning instructions includes determining the target region based at least in part on a scanning protocol. For example, determining the positioning instructions includes determining a body part and/or a body organ. In some examples, determining the positioning instructions includes determining a scanning technique (e.g., based at least in part on a scanning protocol) and determining a scanning path based at least in part on the scanning technique.
In various embodiments, the process of acquiring an image S206 includes acquiring an image based at least in part on the positioning instructions and/or the scanning protocol. In some examples, acquiring the image includes sending positioning instructions to a medical imaging device (e.g., a scanner), for example, to a positioning system (e.g., a robotic scanning platform and/or robotic arm) of the medical imaging device, in order to position the target (e.g., a patient). In some examples, acquiring the image includes sending imaging instructions to an imaging system (e.g., a scanning probe) to acquire the image according to a scanning protocol. In various examples, acquiring the image includes acquiring the image by selecting the image from a pre-generated image database that includes one or more images previously acquired.
In various embodiments, the process of receiving an image S208 includes receiving an image by a user, such as an expert, doctor, and/or medical personnel.
In various embodiments, the process of identifying one or more features S210 includes identifying the one or more features at least in part by a user. In some examples, identifying, at least in part, by the user, the one or more features includes identifying one or more features associated with an image (e.g., an image received by the user). In some examples, identifying the one or more features includes annotating, at least in part by the user, the one or more features associated with the image. In some examples, identifying the one or more features includes identifying landmarks, visual features, geometric shapes, and/or redundant objects.
In various embodiments, the process of determining a quality assessment S212 includes determining a quality assessment at least in part by a user. In some examples, determining, at least in part, by the user, the quality assessment includes determining the quality assessment based, at least in part, on one or more features (e.g., one or more features identified, at least in part, by the user). In some examples, determining the quality assessment includes comparing, at least in part by the user, the identified one or more features to a list of target features and identifying one or more missing features. In some examples, determining the quality assessment includes determining the quality assessment at least in part by the user based at least in part on the identified one or more missing features.
In various embodiments, the process of generating feedback S214 includes generating feedback at least in part by the user. In some examples, generating, at least in part, feedback by the user includes generating, at least in part, feedback by the user based, at least in part, on the quality assessment. In some examples, generating the feedback includes using the quality assessment as the feedback. In some examples, generating the feedback includes generating the quality score at least in part by the user. In various examples, generating the feedback includes generating, at least in part, by the user, an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and/or a translational and rotational deviation matrix.
In various embodiments, the process of receiving feedback through the positioning neural network S216 includes receiving feedback from a quality assessment neural network (e.g., a neural network trained for quality assessment). In various examples, receiving the feedback includes transferring or directing the feedback from the quality assessment neural network to a positioning neural network.
In various embodiments, the process of changing one or more parameters of the positioning neural network S218 includes changing one or more parameters of the positioning neural network based at least in part on the feedback and/or the quality assessment. In some examples, one or more of processes S202, S204, S206, S208, S210, S212, S214, S216, and S218 are repeated. For example, if the quality assessment generated by the quality assessment neural network is unsatisfactory (e.g., when compared to a quality threshold), after one or more parameters of the localization neural network change, the method S200 further includes determining, by the localization neural network, updated localization instructions, obtaining a substitute image, receiving the substitute image, identifying one or more updated features from the substitute image, determining an updated quality assessment corresponding to the substitute image, and changing one or more parameters of the localization neural network if the updated quality assessment is unsatisfactory (e.g., when compared to the quality threshold).
FIG. 4 is a simplified diagram illustrating a computing system according to some embodiments. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. In some examples, computing system 6000 is a general purpose computing device. In some examples, computing system 6000 includes one or more processing units 6002 (e.g., one or more processors), one or more system memories 6004, one or more buses 6006, one or more input/output (I/O) interfaces 6008, and/or one or more network adapters 6012. In certain examples, one or more buses 6006 connect various system components, including, for example, one or more system memories 6004, one or more processing units 6002, one or more input/output (I/O) interfaces 6008, and/or one or more network adapters 6012. Although a selected set of components has been shown above for use in the computing system, many alternatives, modifications, and variations are possible. For example, some of the components may be expanded and/or combined. Other components may be inserted into those noted above. Depending on the embodiment, the arrangement of these components may be interchanged with other components substituted.
In some examples, computing system 6000 is a computer (e.g., server computer, client computer), smartphone, tablet, or wearable device. In some examples, computing system 6000 performs some or all of the processes (e.g., steps) of method S100 and/or method S200. In some examples, some or all of the processes (e.g., steps) of method S100 and/or method S200 are performed by one or more processing units 6002 as directed by one or more code. For example, the one or more codes are stored in one or more system memories 6004 (e.g., one or more non-transitory computer-readable media) and readable by computing system 6000 (e.g., readable by one or more processing units 6002). In different examples, one or more of system memory 6004 includes one or more computer-readable media in the form of volatile memory, such as Random Access Memory (RAM)6014, cache memory 6016, and/or storage system 6018 (e.g., floppy disks, CD-ROMs, and/or DVD-ROMs).
In some examples, one or more input/output (I/O) interfaces 6008 of computing system 6000 are configured to communicate with one or more external devices 6010, such as a keyboard, a pointing device, and/or a display. In some examples, one or more network adapters 6012 of computing system 6000 are configured to communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network (e.g., the internet)). In different examples, additional hardware and/or software modules may be used with computing system 6000, such as one or more microcode and/or one or more device drivers.
FIG. 5 is a simplified diagram illustrating a neural network, according to some embodiments. This diagram is merely an example, which should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. The neural network 8000 is an artificial neural network. In some examples, the neural network 8000 includes an input layer 8002, one or more hidden layers 8004, and an output layer 8006. For example, the one or more hidden layers 8004 includes L neural network layers, including a first neural network layer, …, an ith neural network layer, …, and an lth neural network layer, where L is a positive integer and i is an integer greater than or equal to 1 and less than or equal to L. Although selected sets of components have been shown above for use in the neural network, many alternatives, modifications and variations are possible. For example, some of the components may be expanded and/or combined. Other components may be inserted into the components noted above. Depending on the embodiment, the arrangement of these components may be interchanged with other components substituted.
In some examples, some or all of the processes (e.g., steps) of method S100 and/or method S200 are performed by neural network 8000 (e.g., using computing system 6000). In certain examples, some or all of the processes (e.g., steps) of method S100 and/or method S200 are performed by one or more processing units 6002, which processing unit 6002 is directed by one or more codes that implement neural network 8000. For example, one or more codes for the neural network 8000 are stored in one or more system memories 6004 (e.g., one or more non-transitory computer-readable media) and readable by the computing system 6000 (e.g., by the one or more processing units 6002).
In some examples, the neural network 8000 is a deep neural network (e.g., a convolutional neural network). In some examples, each neural network layer of the one or more hidden layers 8004 includes multiple sub-layers. For example, the ith neural network layer includes a convolutional layer, an activation layer, and a pooling layer. For example, the convolutional layer is configured to perform feature extraction on an input (e.g., an input received by the input layer, or an input from a previous neural network layer), the activation layer is configured to apply a non-linear activation function (e.g., a ReLU function) to an output of the convolutional layer, and the pooling layer is configured to compress (e.g., configured to downsample, such as by performing maximum pooling or average pooling) the output of the activation layer. As an example, output layer 8006 includes one or more fully connected layers.
As discussed above and further emphasized here, fig. 5 is merely an example, and should not unduly limit the scope of the claims. One of ordinary skill in the art would recognize many variations, alternatives, and modifications. For example, the neural network 8000 is replaced with an algorithm that is not an artificial neural network. As an example, the neural network 8000 is replaced with a machine learning model of a non-artificial neural network.
In various embodiments, a computer-implemented method for acquiring medical images of a patient with a medical imaging device includes: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network previously trained for positioning; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving (e.g., by a second neural network previously trained for quality assessment) a first image; identifying (e.g., by a second neural network) one or more first features associated with the acquired first image; determining (e.g., by a second neural network) a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating (e.g., by a second neural network) first feedback based at least in part on the first quality assessment; receiving first feedback (e.g., first feedback from a second neural network) through a first neural network; and changing one or more first parameters of a previously trained first neural network based at least in part on the first feedback. In some examples, the computer-implemented methods are performed by one or more processors. In some examples, the computer-implemented method is implemented at least in accordance with method S100 of fig. 2 and/or method S200 of fig. 3. In certain examples, the method is implemented by at least the system 10 of fig. 1.
In some embodiments, the computer-implemented method further comprises: selecting the first image as the medical image if the first quality assessment satisfies a predetermined quality threshold; and if the first quality assessment fails to meet a predetermined quality threshold: determining a second positioning instruction using the first neural network with the changed one or more first parameters; acquiring a second image according to a second positioning instruction and a scanning protocol; receiving (e.g., receiving through a second neural network) a second image; identifying (e.g., by a second neural network) one or more second features associated with the acquired second image; determining a second quality assessment associated with the second image based at least in part on the identified one or more second features (e.g., by a second neural network); and selecting the second image as the medical image if the second quality assessment satisfies a predetermined quality threshold.
In some embodiments, the computer-implemented method further comprises: training a second neural network prior to performing the receive scan protocol; wherein training the second neural network for quality assessment comprises: receiving a training medical image through a second neural network; receiving a target output associated with the training medical image; generating, by a second neural network, a training assessment associated with the training medical image; generating training feedback based at least in part on the training assessment and the target output; and changing one or more second parameters of the second neural network based at least in part on the training feedback.
In some embodiments, the training feedback includes an authenticity classification, a translational rotational bias matrix, a translational bias matrix, and/or a rotational bias matrix.
In some embodiments, generating the training feedback includes determining a loss based at least in part on the training assessment and the target output; and changing one or more second parameters of the second neural network comprises changing the one or more second parameters based at least in part on the loss using a gradient descent-based machine learning framework.
In some embodiments, receiving a scan protocol includes receiving a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate.
In some embodiments, determining, by the first neural network, the first positioning instruction based at least in part on the scanning protocol comprises: determining a target region based at least in part on a scanning protocol; determining a scanning technique for scanning the target area; and determining a first scan path based at least in part on the scanning technique.
In some embodiments, acquiring the first image based at least in part on the first positioning instructions and the scanning protocol comprises: sending the first positioning instructions to a positioning system of the medical imaging device for positioning the patient to a first relative position with respect to an imaging system of the medical imaging device; and sending imaging instructions to the imaging system to acquire the first image according to the scan protocol.
In some embodiments, identifying (e.g., by the second neural network) the one or more first features associated with the acquired first image includes identifying landmarks, visual features, geometric shapes, and/or unwanted objects.
In some embodiments, if the one or more first features include redundant objects, the method further comprises: prompting a removal instruction for removing redundant objects and acquiring a first substitute image according to a first positioning instruction and a scanning protocol; and/or determining a second positioning instruction and acquiring a second image according to the second positioning instruction as redundant objects are avoided.
In some embodiments, determining (e.g., by a second neural network) a first quality assessment associated with the first image based at least in part on the identified one or more first features includes comparing the identified one or more first features to a list of target features and identifying one or more missing features.
In various embodiments, a system for acquiring medical images of a patient with a medical imaging device comprises: a protocol receiving module configured to receive a scan protocol; an instruction determination module configured to determine a first positioning instruction based at least in part on a scanning protocol by a first neural network, the first neural network previously trained for positioning; an image acquisition module configured to acquire a first image based at least in part on a first positioning instruction and a scanning protocol; an image receiving module configured to receive a first image (e.g., through a second neural network previously trained for quality assessment); a feature identification module configured to identify one or more first features associated with the acquired first image (e.g., by a second neural network); a quality assessment module configured to determine a first quality assessment (e.g., by a second neural network) based at least in part on the identified one or more first features, the first quality assessment associated with the first image; a feedback generation module configured to generate a first feedback based at least in part on the first quality assessment (e.g., by a second neural network); a feedback receiving module configured to receive first feedback through a first neural network (e.g., from a second neural network); a parameter changing module configured to change one or more first parameters of a previously trained first neural network based at least in part on the first feedback. In some examples, the system is implemented and/or configured at least in accordance with system 10 of fig. 1 to perform at least method S100 of fig. 2 and/or method S200 of fig. 3.
In some embodiments, the system further comprises: an image selection module configured to select the first image as the medical image if the first quality assessment satisfies a predetermined quality threshold. In some examples, if the first quality assessment fails to meet a predetermined quality threshold, then: the instruction determination module is further configured to determine a second positioning instruction using the first neural network having the changed one or more first parameters; the image acquisition module is further configured to acquire a second image according to a second positioning instruction and the scanning protocol; the image receiving module is further configured to receive (e.g., receive via a second neural network) a second image; the feature identification module is further configured to identify one or more second features associated with the acquired second image (e.g., by a second neural network); the quality assessment module is further configured to determine a second quality assessment associated with the second image based at least in part on the identified one or more second features (e.g., by a second neural network); and the image selection module is further configured to select the second image as the medical image if the second quality assessment satisfies a predetermined quality threshold.
In some embodiments, the system further comprises a training module configured to train the second neural network. Wherein the training module is configured to: receiving a training medical image through a second neural network; receiving a target output associated with the training medical image; generating, by a second neural network, a training assessment associated with the training medical image; generating training feedback based at least in part on the training assessment and a target output; one or more second parameters of the second neural network are changed based at least in part on the training feedback.
In some embodiments, the training feedback includes an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and/or a translational rotational deviation matrix.
In some embodiments, the training module is further configured to: determining a loss based at least in part on the training assessment and the target output; and changing one or more second parameters based at least in part on the loss using a machine learning framework based on gradient descent.
In some embodiments, the scan protocol includes a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate.
In some embodiments, the instruction determination module is further configured to determine a target region based at least in part on the scanning protocol; determining a scanning technique for scanning the target area; and determining a first scan path based at least in part on the scanning technique.
In some embodiments, the image acquisition module is further configured to: sending the first positioning instructions to a positioning system of the medical imaging device for positioning the patient to a first relative position with respect to an imaging system of the medical imaging device; and sending imaging instructions to the imaging system to acquire the first image according to the scan protocol.
In some embodiments, the feature identification module is further configured to identify landmarks, visual features, geometric shapes, and/or unwanted objects.
In some embodiments, the system further includes a redundant object module configured to prompt a removal instruction to remove a redundant object if the one or more first features include the redundant object and obtain a first substitute image according to the first positioning instruction and the scanning protocol; and/or determining a second positioning instruction and acquiring a second image according to the second positioning instruction as redundant objects are avoided.
In some embodiments, the quality assessment module is further configured to compare the identified one or more first features to a list of target features and identify one or more missing features.
In various embodiments, a non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor, perform processes comprising: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network having been previously trained to over-position; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving a first image (e.g., through a second neural network previously trained for quality assessment); identifying (e.g., by a second neural network) one or more first features associated with the acquired first image; determining (e.g., by a second neural network) a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating a first feedback based at least in part on the first quality assessment (e.g., by a second neural network); receiving, by a first neural network, first feedback from a second neural network; and changing one or more first parameters of a previously trained first neural network based at least in part on the first feedback. In some examples, the non-transitory computer-readable medium having instructions stored thereon is implemented at least in accordance with method S100 of fig. 2 and/or system 10 (e.g., terminal) of fig. 1.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, further performs processes comprising: selecting the first image as the medical image if the first quality assessment satisfies a predetermined quality threshold; and if the first quality assessment fails to meet a predetermined quality threshold, then: determining a second positioning instruction using the first neural network having the one or more first parameters changed; acquiring a second image according to a second positioning instruction and a scanning protocol; receiving (e.g., through a second neural network) a second image; identifying (e.g., by a second neural network) one or more second features associated with the acquired second image; determining (e.g., by a second neural network) a second quality assessment associated with the second image based at least in part on the identified one or more second features; and selecting the second image as the medical image if the second quality assessment satisfies a predetermined quality threshold.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, further performs processes comprising: training a second neural network prior to performing the receive scan protocol; wherein training the second neural network for quality assessment comprises: receiving a training medical image through a second neural network; receiving a target output associated with the training medical image; generating, by a second neural network, a training assessment associated with the training medical image; generating training feedback based at least in part on the training assessment and a target output; and changing one or more second parameters of the second neural network based at least in part on the training feedback.
In some embodiments, the training feedback includes an authenticity classification, a translational rotational bias matrix, a translational bias matrix, and/or a rotational bias matrix.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, performs processes comprising: determining a loss based at least in part on the training assessment and the target output; and changing one or more second parameters of the second neural network comprises changing one or more second parameters based at least in part on the loss using a gradient descent-based machine learning framework.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, performs processes comprising: receive a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and/or a scan rate.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, performs processes comprising: determining a target region based at least in part on a scanning protocol; determining a scanning technique for scanning the target area; and determining a first scan path based at least in part on the scanning technique.
In some embodiments, the non-transitory computer-readable medium, when executed by a processor, performs processes comprising: transmitting the first positioning instructions to a positioning system of the medical imaging device for positioning the patient to a first relative position with respect to an imaging system of the medical imaging device; and sending imaging instructions to the imaging system to acquire the first image according to the scan protocol.
In some embodiments, a non-transitory computer readable medium, which when executed by a processor, performs processes comprising: landmarks, visual features, geometries, and/or unwanted objects are identified.
In some embodiments, if the one or more first features include redundant objects, the non-transitory computer readable medium when executed by the processor further performs processes comprising: prompting a removal instruction for removing redundant objects and acquiring a first substitute image according to a first positioning instruction and a scanning protocol; and/or determining a second positioning instruction as the redundant object is avoided and acquiring a second image according to the second positioning instruction.
In some embodiments, the non-transitory computer readable medium, when executed by a processor, performs processes comprising: the identified one or more first features are compared to a list of target features, and one or more missing features are identified.
In various embodiments, a method for acquiring medical images of a patient with a medical imaging device includes: receiving a scanning protocol; determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network being pre-trained for positioning; acquiring a first image based at least in part on the first positioning instructions and the scanning protocol; receiving a first image; identifying one or more first features associated with the acquired first image; determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image; generating a first feedback based at least in part on the first quality assessment; receiving first feedback through a first neural network; and changing, by the first neural network, one or more first parameters of the previously trained first neural network based at least in part on the first feedback.
In some embodiments, the method further comprises: identifying one or more second features associated with the acquired first image by a second neural network, the second neural network being pre-trained for quality assessment; determining, by the second neural network, a second quality assessment based at least in part on the identified one or more second features, the second quality assessment associated with the first image; generating, by the second neural network, second feedback based at least in part on the second quality assessment; receiving first feedback through a second neural network; and changing, by the second neural network, one or more second parameters of a previously trained second neural network based at least in part on the received first feedback and the determined second feedback.
For example, some or all of the elements of the various embodiments of the invention may be implemented using one or more software elements, one or more hardware elements, and/or a combination of one or more software and hardware elements, alone and/or in combination with at least one other element. In another example, some or all of the components of the different embodiments of the present invention are each implemented in one or more circuits, such as one or more analog circuits and/or one or more digital circuits, alone and/or in combination with at least one other component. In yet another example, while the embodiments described above refer to particular features, the scope of the present invention also includes embodiments having different combinations of features and embodiments that do not include all of the described features. In yet another example, different embodiments and/or examples of the invention may be combined.
In addition, the methods and systems described herein may be implemented on many different types of processing devices by program code (including program instructions executable by a device processing subsystem). The software program instructions may include source code, object code, machine code, or any other stored data that is operable to cause a processing system to perform methods and operations described herein. However, other implementations may also be used, such as firmware or even appropriately designed hardware configured to perform the methods and systems described herein.
The data (e.g., associations, mappings, data inputs, data outputs, intermediate data results, final data results, etc.) of the systems and methods can be stored and implemented in one or more different types of computer-implemented data stores, such as different types of storage devices and programming constructs (e.g., RAM, ROM, EEPROM, flash, flat files, databases, programmed data structures, programmed variables, IF-THEN (or similar types) statement constructs, application programming interfaces, etc.). Note that the data structures describe formats for organizing and storing data in databases, programs, memory, or other computer-readable media for use by a computer program.
The systems and methods may be provided on many different types of computer-readable media including computer storage mechanisms (e.g., CD-ROM, diskette, RAM, flash memory, computer hard drive, DVD, etc.), where the computer-readable media contains instructions (e.g., software) for execution by a processor to perform the operations of the methods described herein and to implement the systems described herein. The computer components, software modules, functions, data stores, and data structures described herein may be interconnected directly or indirectly to enable a desired data flow for their operations. It should also be noted that a module or processor comprises code units which perform software operations and may be implemented, for example, as subroutine code units, or software function code units, or objects (as in object-oriented paradigm), or applets, or computer script language, or other types of computer code. The software components and/or functions may reside on one computer or be distributed across multiple computers depending on the circumstances at hand.
The computing system may include a client device and a server. A client device and server are typically remote from each other and typically interact through a communication network. The relationship of client device and server arises by virtue of computer programs running on the respective computers and having a client device-server relationship to each other.
This description contains many details specific to particular embodiments. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations, one or more features from a combination can in some cases be removed from the combination and the combination can be oriented, for example, in a sub-combination or a variation of a sub-combination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequence shown, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated within one software product or packaged into multiple software products.
While specific embodiments of the invention have been described, those skilled in the art will appreciate that there are other embodiments that are equivalent to the described embodiments. It should be understood, therefore, that the intention is not to limit the invention to the particular embodiments described.

Claims (20)

1. A computer-implemented method for acquiring medical images of a patient with a medical imaging device, the method comprising:
receiving a scanning protocol;
determining, by a first neural network based at least in part on the scanning protocol, a first positioning instruction, the first neural network previously trained for positioning;
acquiring a first image based at least in part on the first positioning instructions and the scanning protocol;
receiving the first image;
identifying one or more first features associated with the acquired first image;
determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image;
generating a first feedback based at least in part on the first quality assessment;
receiving the first feedback through the first neural network; and
changing one or more first parameters of the first neural network previously trained based at least in part on the first feedback;
wherein the computer-implemented method is performed by one or more processors.
2. The computer-implemented method of claim 1, further comprising:
selecting the first image as the medical image if the first quality assessment satisfies a predetermined quality threshold; and
if the first quality assessment fails to meet the predetermined quality threshold:
determining a second positioning instruction using the first neural network with the one or more first parameters changed;
acquiring a second image according to the second positioning instruction and the scanning protocol;
receiving the second image;
identifying one or more second features associated with the acquired second image;
determining a second quality assessment associated with the second image based at least in part on the identified one or more second features; and
selecting the second image as the medical image if the second quality assessment satisfies the predetermined quality threshold.
3. The computer-implemented method of claim 1, further comprising:
training a second neural network for quality assessment prior to performing the receive scan protocol;
wherein training the second neural network for quality assessment comprises:
receiving training medical imagery through the second neural network;
receiving a target output associated with the training medical image;
generating, by the second neural network, a training assessment associated with the training medical image;
generating training feedback based at least in part on the training assessment and the target output; and
changing one or more second parameters of the second neural network based at least in part on the training feedback.
4. The computer-implemented method of claim 3, wherein the training feedback comprises one selected from a group consisting of an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and a translational-rotational deviation matrix.
5. The computer-implemented method of claim 3,
the generating training feedback comprises determining a loss based at least in part on the training assessment and the target output; and
the changing one or more second parameters of the second neural network includes changing one or more of the second parameters based at least in part on the loss using a gradient descent-based machine learning framework.
6. The computer-implemented method of claim 1, wherein receiving a scan protocol comprises receiving at least one selected from a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and a scan rate.
7. The computer-implemented method of claim 1, wherein determining, by the first neural network, the first positioning instruction based at least in part on the scanning protocol comprises:
determining a target region based at least in part on the scanning protocol;
determining a scanning technique for scanning the target region; and
a first scan path is determined based at least in part on the scanning technique.
8. The computer-implemented method of claim 1, wherein the acquiring a first image based at least in part on a first positioning instruction and the scanning protocol comprises:
sending the first positioning instructions to a positioning system of the medical imaging device in order to position a patient to a first relative position with respect to an imaging system of the medical imaging device; and
sending imaging instructions to the imaging system to acquire the first image according to the scanning protocol.
9. The computer-implemented method of claim 1, wherein the identifying one or more first features associated with the acquired first image comprises identifying at least one selected from a landmark, a visual feature, a geometric shape, and a redundant object.
10. The computer-implemented method of claim 9, wherein if the one or more first features include the redundant object, the method further comprises one of:
prompting a removal instruction for removing the redundant objects, and acquiring a first substitute image according to the first positioning instruction and the scanning protocol; and
and determining a second positioning instruction along with the avoidance of the redundant object, and acquiring a second image according to the second positioning instruction.
11. The computer-implemented method of claim 1, wherein the determining a first quality assessment associated with the first image based at least in part on the identified one or more first features comprises comparing the identified one or more first features to a list of target features and identifying one or more missing features.
12. A system for a computer-implemented method of acquiring medical images of a patient with a medical imaging device, the system comprising:
a protocol receiving module configured to receive a scan protocol;
an instruction determination module configured to determine, by a first neural network that was previously trained for positioning, a first positioning instruction based at least in part on the scanning protocol;
an image acquisition module configured to acquire a first image based at least in part on the first positioning instructions and the scanning protocol;
an image receiving module configured to receive the first image;
a feature identification module configured to identify one or more first features associated with the acquired first image;
a quality assessment module configured to determine a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image;
a feedback generation module configured to generate a first feedback based at least in part on the first quality assessment;
a feedback receiving module configured to receive the first feedback through a first neural network; and
a parameter change module configured to change one or more first parameters of the first neural network that were previously trained based at least in part on the first feedback.
13. The system of claim 12, further comprising:
an image selection module configured to select the first image as the medical image if the first quality assessment satisfies a predetermined quality threshold;
wherein if the first quality assessment fails to meet the predetermined quality threshold:
the instruction determination module is further configured to determine a second positioning instruction using the first neural network with the one or more first parameters changed;
the image acquisition module is further configured to acquire a second image according to the second positioning instruction and the scanning protocol;
the image receiving module is further configured to receive the second image;
the feature identification module is further configured to identify one or more second features associated with the acquired second image;
the quality assessment module is further configured to determine a second quality assessment associated with the second image based at least in part on the identified one or more second features; and
the image selection module is further configured to select the second image as the medical image if the second quality assessment satisfies the predetermined quality threshold.
14. The system of claim 12, further comprising:
a training module configured to train a second neural network, the training module configured to:
receiving training medical imagery through the second neural network;
receiving a target output associated with the training medical image;
generating, by the second neural network, a training assessment associated with the training medical image;
generating training feedback based at least in part on the training assessment and the target output; and
changing one or more second parameters of the second neural network based at least in part on the training feedback.
15. The system of claim 14, wherein the training feedback comprises one selected from a group consisting of an authenticity classification, a translational deviation matrix, a rotational deviation matrix, and a translational-rotational deviation matrix.
16. The system of claim 14, wherein the training module is further configured to:
determining a loss based at least in part on the training assessment and the target output; and
changing one or more of the second parameters based at least in part on the loss using a machine learning framework based on gradient descent.
17. The system of claim 13, wherein the scan protocol comprises at least one selected from a scan type, a patient body type, a target body part, a sampling mask, a magnification, a working distance, a resolution, and a scan rate.
18. The system of claim 13, wherein the instruction determination module is further configured to:
determining a target region based at least in part on the scanning protocol;
determining a scanning technique for scanning the target region; and
a first scan path is determined based at least in part on the scanning technique.
19. A method of acquiring medical images of a patient with a medical imaging device, the method comprising:
receiving a scanning protocol;
determining, by a first neural network based at least in part on a scanning protocol, a first positioning instruction, the first neural network previously trained for positioning;
acquiring a first image based at least in part on the first positioning instructions and the scanning protocol;
receiving the first image;
identifying one or more first features associated with the acquired first image;
determining a first quality assessment based at least in part on the identified one or more first features, the first quality assessment associated with the first image;
generating a first feedback based at least in part on the first quality assessment;
receiving the first feedback through the first neural network; and
changing, by the first neural network, one or more first parameters of the first neural network that were previously trained based at least in part on the first feedback.
20. The method of claim 19, further comprising:
identifying one or more second features associated with the acquired first image by a second neural network, the second neural network previously trained for quality assessment;
determining, by the second neural network, a second quality assessment based at least in part on the identified one or more second features, the second quality assessment associated with the first image;
generating, by the second neural network, second feedback based at least in part on the second quality assessment;
receiving the first feedback through the second neural network; and
changing, by the second neural network, one or more of the second parameters of the second neural network that was previously trained based at least in part on the received first feedback and the determined second feedback.
CN201911346034.5A 2019-10-01 2019-12-24 System and method for enhancing a patient positioning system Pending CN111161231A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US16/589,656 2019-10-01
US16/589,656 US20210096934A1 (en) 2019-10-01 2019-10-01 Systems and methods for enhancing a patient positioning system

Publications (1)

Publication Number Publication Date
CN111161231A true CN111161231A (en) 2020-05-15

Family

ID=70557906

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911346034.5A Pending CN111161231A (en) 2019-10-01 2019-12-24 System and method for enhancing a patient positioning system

Country Status (2)

Country Link
US (1) US20210096934A1 (en)
CN (1) CN111161231A (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113379862A (en) * 2021-06-08 2021-09-10 苏州晟诺医疗科技有限公司 Method, apparatus, medium, and electronic device for obtaining scan parameter value

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202104929U (en) * 2011-05-11 2012-01-11 上海生物医学工程研究中心 Device for body CT geometric correction
US20140152957A1 (en) * 2012-11-30 2014-06-05 Kabushiki Kaisha Topcon Optic neuropathy detection with three-dimensional optical coherence tomography
CN105894508A (en) * 2016-03-31 2016-08-24 上海联影医疗科技有限公司 Method for evaluating automatic positioning quality of medical image
US20170011535A1 (en) * 2015-07-09 2017-01-12 Sirona Dental Systems Gmbh Method, apparatus, and computer readable medium for removing unwanted objects from a tomogram
CN107403457A (en) * 2017-07-28 2017-11-28 上海联影医疗科技有限公司 Medical imaging procedure, equipment and multi-modal medical imaging procedure
US20170372232A1 (en) * 2016-06-27 2017-12-28 Purepredictive, Inc. Data quality detection and compensation for machine learning
CN107767928A (en) * 2017-09-15 2018-03-06 深圳市前海安测信息技术有限公司 Medical image report preparing system and method based on artificial intelligence
CN108670286A (en) * 2018-06-13 2018-10-19 上海联影医疗科技有限公司 A kind of CT system and CT scan method
US20190228547A1 (en) * 2018-01-24 2019-07-25 New York University Systems and methods for diagnostic oriented image quality assessment
US20190223821A1 (en) * 2018-01-24 2019-07-25 Palodex Group Oy X-ray imaging unit for x-ray imaging
CN110114834A (en) * 2016-11-23 2019-08-09 通用电气公司 Deep learning medical system and method for medical procedure
US20190282214A1 (en) * 2018-03-16 2019-09-19 Samsung Medison Co., Ltd. Medical imaging apparatus, medical imaging apparatus control method, and computer program product

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8190234B2 (en) * 2000-07-28 2012-05-29 Fonar Corporation Movable patient support with spatial locating feature
CN108304755B (en) * 2017-03-08 2021-05-18 腾讯科技(深圳)有限公司 Training method and device of neural network model for image processing
US10878311B2 (en) * 2018-09-28 2020-12-29 General Electric Company Image quality-guided magnetic resonance imaging configuration
US11195116B2 (en) * 2018-10-31 2021-12-07 International Business Machines Corporation Dynamic boltzmann machine for predicting general distributions of time series datasets
US11751848B2 (en) * 2019-01-07 2023-09-12 Bfly Operations, Inc. Methods and apparatuses for ultrasound data collection
US10891537B2 (en) * 2019-03-20 2021-01-12 Huawei Technologies Co., Ltd. Convolutional neural network-based image processing method and image processing apparatus

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202104929U (en) * 2011-05-11 2012-01-11 上海生物医学工程研究中心 Device for body CT geometric correction
US20140152957A1 (en) * 2012-11-30 2014-06-05 Kabushiki Kaisha Topcon Optic neuropathy detection with three-dimensional optical coherence tomography
US20170011535A1 (en) * 2015-07-09 2017-01-12 Sirona Dental Systems Gmbh Method, apparatus, and computer readable medium for removing unwanted objects from a tomogram
CN105894508A (en) * 2016-03-31 2016-08-24 上海联影医疗科技有限公司 Method for evaluating automatic positioning quality of medical image
US20170372232A1 (en) * 2016-06-27 2017-12-28 Purepredictive, Inc. Data quality detection and compensation for machine learning
CN110114834A (en) * 2016-11-23 2019-08-09 通用电气公司 Deep learning medical system and method for medical procedure
CN107403457A (en) * 2017-07-28 2017-11-28 上海联影医疗科技有限公司 Medical imaging procedure, equipment and multi-modal medical imaging procedure
CN107767928A (en) * 2017-09-15 2018-03-06 深圳市前海安测信息技术有限公司 Medical image report preparing system and method based on artificial intelligence
US20190228547A1 (en) * 2018-01-24 2019-07-25 New York University Systems and methods for diagnostic oriented image quality assessment
US20190223821A1 (en) * 2018-01-24 2019-07-25 Palodex Group Oy X-ray imaging unit for x-ray imaging
CN110063741A (en) * 2018-01-24 2019-07-30 帕洛代克斯集团有限公司 X-ray imaging unit for x-ray imaging
US20190282214A1 (en) * 2018-03-16 2019-09-19 Samsung Medison Co., Ltd. Medical imaging apparatus, medical imaging apparatus control method, and computer program product
CN108670286A (en) * 2018-06-13 2018-10-19 上海联影医疗科技有限公司 A kind of CT system and CT scan method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
HENG LI 等: "Use of treatment log files in spot scanning proton therapy as part of patient-specific quality assurance", 《MED PHYS》, vol. 40, no. 2, 28 February 2013 (2013-02-28), XP012170945, DOI: 10.1118/1.4773312 *
唐璠: "肿瘤患者放射治疗的精确定位和质量控制", 《医疗装备》, vol. 31, no. 19, 31 October 2018 (2018-10-31) *
李乐山;黄丹;孙汇苑;杨文琳;: "基于卷积神经网络算法对于乳腺癌医学影像的精确分析", 电脑迷, no. 02, 8 February 2018 (2018-02-08) *

Also Published As

Publication number Publication date
US20210096934A1 (en) 2021-04-01

Similar Documents

Publication Publication Date Title
US20240119719A1 (en) Identification and tracking of a predefined object in a set of images from a medical image scanner during a surgical procedure
US20210209766A1 (en) Method and system for automatically segmenting blood vessel in medical image by using machine learning and image processing algorithm
US9959486B2 (en) Voxel-level machine learning with or without cloud-based support in medical imaging
EP3525171A1 (en) Method and system for 3d reconstruction of x-ray ct volume and segmentation mask from a few x-ray radiographs
US20230360313A1 (en) Autonomous level identification of anatomical bony structures on 3d medical imagery
EP3355273B1 (en) Coarse orientation detection in image data
US11080889B2 (en) Methods and systems for providing guidance for adjusting an object based on similarity
Zheng Statistical shape model‐based reconstruction of a scaled, patient‐specific surface model of the pelvis from a single standard AP x‐ray radiograph
CN111353524B (en) System and method for locating patient features
US10303965B2 (en) Defective pixel identification using machine learning
EP3295373A1 (en) A system and method for surgical guidance and intra-operative pathology through endo-microscopic tissue differentiation
US10610124B2 (en) Localization of fibrous neural structures
CN113688912A (en) Confrontation sample generation method, device, equipment and medium based on artificial intelligence
US11545255B2 (en) Systems and methods for classifying an anomaly medical image using variational autoencoder
EP3107031A1 (en) Method, apparatus and system for spine labeling
CN111563496A (en) Continuous learning for automatic view planning for image acquisition
US11282218B2 (en) Systems and methods for providing medical guidance using a patient depth image
CN111161231A (en) System and method for enhancing a patient positioning system
US11565129B2 (en) Binary tracking of an anatomical tracking structure on medical images
JP2012128638A (en) Image processing device, alignment method and program
CN112308764A (en) Image registration method and device
US11836946B2 (en) Methods and devices for guiding a patient
Daly et al. Multimodal image registration using multiresolution genetic optimization
CN113962957A (en) Medical image processing method, bone image processing method, device and equipment
Zhang et al. Localizing landmark sets in head CTs using random forests and a heuristic search algorithm for registration initialization

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination