Classifications

• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H30/00—ICT specially adapted for the handling or processing of medical images
  • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/04—Positioning of patients; Tiltable beds or the like
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/54—Control of apparatus or devices for radiation diagnosis
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
  • A61N5/00—Radiation therapy
  • A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
  • A61N5/1048—Monitoring, verifying, controlling systems and methods
  • A61N5/1049—Monitoring, verifying, controlling systems and methods for verifying the position of the patient with respect to the radiation beam
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
  • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H30/00—ICT specially adapted for the handling or processing of medical images
  • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
  • A61B6/04—Positioning of patients; Tiltable beds or the like
  • A61B6/0487—Motor-assisted positioning
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
  • A61N5/00—Radiation therapy
  • A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
  • A61N5/1048—Monitoring, verifying, controlling systems and methods
  • A61N5/1049—Monitoring, verifying, controlling systems and methods for verifying the position of the patient with respect to the radiation beam
  • A61N2005/1055—Monitoring, verifying, controlling systems and methods for verifying the position of the patient with respect to the radiation beam using magnetic resonance imaging [MRI]
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
  • A61N5/00—Radiation therapy
  • A61N5/10—X-ray therapy; Gamma-ray therapy; Particle-irradiation therapy
  • A61N5/1048—Monitoring, verifying, controlling systems and methods
  • A61N5/1049—Monitoring, verifying, controlling systems and methods for verifying the position of the patient with respect to the radiation beam
  • A61N2005/1059—Monitoring, verifying, controlling systems and methods for verifying the position of the patient with respect to the radiation beam using cameras imaging the patient
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10028—Range image; Depth image; 3D point clouds
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
  • G06T2207/10081—Computed x-ray tomography [CT]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
  • G06T2207/10088—Magnetic resonance imaging [MRI]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10072—Tomographic images
  • G06T2207/10104—Positron emission tomography [PET]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10116—X-ray image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10132—Ultrasound image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20081—Training; Learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20084—Artificial neural networks [ANN]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30168—Image quality inspection

Definitions

• the present invention relates to medical imaging, and particularly to a computer- implemented method, apparatus, system, and computer program for prospective quality assessment for imaging examination prior to acquisition.
• US 2019/183439 Al discloses a method for positioning a body region of a patient for a radiography acquisition by a radiography system.
• a computer- implemented method for prospective quality assessment for imaging examination prior to acquisition comprising the following steps: a) receiving sensor data of a body part of a patient to be imaged by a medical imaging apparatus using a medical imaging modality; b) generating a quality metric from the received sensor data using a data-driven model, wherein the data-driven model has been trained based on a training dataset that comprises a plurality of training examples, each training example comprises sensor data of the body part acquired in an imaging session and an associated quality metric derived from image data acquired using the medical imaging modality in the imaging session; and c) providing the generated quality metric for prospective quality assessment for imaging examination prior to acquisition.
• a computer-implemented method is proposed to enable automated prediction of quality metrics prior to image formation, by exploiting data from sensors.
• the proposed method involves obtaining sensor data of a body part of a patient to be imaged during positioning/preparation of an imaging examination.
• the sensor data may be acquired by any suitable sensor, such as an optical sensor, a thermal sensor, a depth sensor, an array of radio frequency sensors, a fibre optical range sensor, or any combination thereof. If the patient is lying on a patient support, the sensor data may also be acquired by a sensor embedded in the patient support, such as a pressure sensor embedded in the patient support, an ultrasound sensor embedded in the patient support, etc.
• a quality metric may be derived from the received sensor data using a trained data-driven model.
• the data-driven model has been trained to predict the quality metric of to-be-acquired medical image data directly from the sensor data input prior to the actually imaging examination.
• the quality metric may be derived directly from the received sensor data.
• an anatomy model of the target anatomy may be fitted to the received sensor data, and the quality metric may be derived from the fitted anatomy model.
• the quality metric may also be referred to as predicted quality metric.
• the data-driven model has been trained on a training data set that comprises a plurality training examples. Each example comprises sensor data of the body part acquired in an imaging session and an associated quality metric derived from image data acquired using the medical imaging modality in the same imaging session.
• the quality metric may be manually annotated by skilled clinicians and/or derived from the image data in an automated way.
• the data-driven model has been trained to learn an association between sensor data acquired during positioning/preparation of an imaging examination and a quality metric derived from image data acquired by a medical scanner after the actual imaging examination.
• An exemplary training process will be explained with respect to the embodiment shown in Fig. 4.
• the quality metric may be a single value. In some examples, the quality metric may be a vector of numerical values. Each numerical value represents a deviation of a position and/or a rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and/or rotation of the anatomical feature.
• the data-driven model may also be referred to as quality prediction model. Exemplary data-driven models may include, but are not limited to, artificial neural networks, trained random forests, and a model based segmentation approach.
• the generated quality metric is then provided for prospective quality assessment for imaging examination prior to acquisition.
• the trained data-driven model may be applied by prospectively displaying predicted quality measures during preparation/positioning of the exam.
• the generated quality metric may be compared with a reference quality metric and an acoustic signal may be generated indicating whether the patient is correctly positioned.
• the generated quality metric may be compared with a reference quality metric and the medical scanner may be triggered to perform image acquisition if the difference between the generated quality metric and the reference quality metric is less than a threshold.
• imaging modalities may include, but not limited to, X-ray imaging, MR imaging, CT imaging, positron-emission tomography (PET) imaging, and automatically-steered ultrasound imaging.
• imaging modalities may include a combined therapy/diagnostic apparatus, such as an MR-Linac apparatus, an MR proton therapy apparatus, and/or a cone beam CT apparatus.
• the proposed method may allow an operator to optimize the predicted quality metric prior to taking the actual exam. For example, an instant prediction of the current positioning quality may be indicated, while moving the patient being observed from the sensor. This may greatly improve the quality of medical image data acquired in the actual imaging examination, thereby leading to fewer retakes, less delayed treatment to patients, shortened workflow, and higher patient rate. In X-ray and CT exams, fewer retakes may also reduce radiation doses for patients.
• the generation of a quality metric from the received sensor data comprises: generating the quality metric directly from the received the sensor data; or fitting an anatomy model of a target anatomy to the sensor data and generating the quality metric from the fitted anatomy model of the target anatomy.
• the anatomy model of the target anatomy may be a two-dimensional model, a three- dimensional model, or a higher-dimensional model, which models joint dependent articulation (e.g. flexion) or anatomical variations.
• the computer-implemented method further comprises: determining whether the generated quality metric meets a predetermined criterion; and generating a signal indicative of whether the generated quality metric meets the predetermined criterion.
• the signal comprises a signal for controlling a device to inform an operator whether the patient is ready for image acquisition.
• the device is a speaker configured to generate an acoustic signal indicating whether the patient is correctly positioned.
• the device is a lighting device configured to generate a light signal indicating whether the patient is correctly positioned.
• the device is a haptic device configured to apply forces, vibrations, or motions to the patient and/or the operator indicating whether the patient is correctly positioned.
• the signal comprises a signal for triggering the medical imaging apparatus to start image acquisition.
• the medical imaging apparatus may automatically start image acquisition in response to a signal indicating that the patient is correctly positioned.
• the computer-implemented method further comprises: receiving image data of the body part of the patient after image acquisition; determining, based on the received imaged data, a further quality metric; and determining a difference between the quality metric generated from the sensor data before image acquisition and the further quality metric derived from the image data after image acquisition; and further training the data-driven model using the difference.
• the computer-implemented method further comprises: receiving a user input indicative of a user-defined quality metric of the received image data; and determining a difference between the quality metric generated from the sensor data before image acquisition and the user-defined quality metric; and further training the data-driven model using the difference.
• a user e.g. radiographer
• Putting this expert feedback into the continuous learning loop implements an even more reliable strongly -supervised learning strategy. This will be explained hereinafter and in particular with respect to the embodiments shown in Fig. 7 and Fig. 8.
• the quality metric is a vector of numerical values, each numerical value representing a deviation of a position and/or a rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and/or rotation of the anatomical feature.
• the X-ray image quality may be determined in terms of a quality vector with different quality aspects, such as Field-of-View (FOV) compliance, rotation, and flexion. This will be explained hereinafter and in particular with respect to the embodiment shown in Fig. 8.
• FOV Field-of-View
• the sensor data is acquired by one or more of the following: an optical sensor, a depth sensor, a thermal sensor, a pressure sensor, an ultrasound sensor, and an array of radio frequency sensors.
• the medical imaging modality comprises one or more of: magnetic resonance imaging, ultrasound imaging, X-ray imaging, computer tomography imaging, and positron-emission tomography imaging.
• the medical imaging modality comprises a hybrid modality including one or more of: MR-Linac, MR proton therapy, and cone beam computer tomography.
• an apparatus for prospective quality assessment for imaging examination prior to acquisition comprising one or more processing unit(s) to generate a quality metric, wherein the processing unit(s) include instructions, which when executed on the one or more processing unit(s) perform the method steps according to the first aspect and any associated example.
• a system comprising: a medical imaging apparatus configured to acquire image data of a body part of a patient; a sensor configmed to acquire sensor data of the body part of the patient; and an apparatus according to the second aspect and any associated example configured to receive the sensor data and provide a quality metric for prospective quality assessment for imaging examination prior to acquisition.
• the medical imaging apparatus is configured to start image acquisition according to the provided quality metric.
• the system further comprises a device configured to inform whether the patient is ready for image acquisition based on the quality metric.
• the device is a speaker configured to generate an acoustic signal indicating whether the patient is correctly positioned.
• the device is a lighting device configured to generate a light signal indicating whether the patient is correctly positioned.
• the device is a haptic device configured to apply forces, vibrations, or motions to the patient and/or the operator informing whether the patient is correctly positioned.
• a computer program product comprising instructions which, when the program is executed by at least one processing unit, cause the at least one processing unit to carry out the steps of the method according to the first aspect and any associated example.
• Fig. 1 illustrates a block diagram of an exemplary apparatus.
• Fig. 2 is a flow chart of an exemplary computer-implemented method.
• Fig. 3 schematically illustrates an example of a system.
• Fig. 4 is a flowchart of a method for generating training data for training the data-driven model.
• Fig. 5 shows a further example of a system.
• Fig. 6 shows an exemplary lateral image of an ankle.
• Fig. 7 shows an exemplary method for implementing a weakly -supervised continuous learning.
• Fig. 8 shows an example of a continuously learning framework for quality metric prediction with example data.
• Fig. 1 illustrates a block diagram of an exemplary apparatus 10 for prospective quality assessment for imaging examination prior to acquisition.
• the apparatus 10 may include an input unit 12, one or more processing units 14, and an output unit 16.
• the apparatus 10 may comprise various physical and/or logical components for communicating and manipulating information, which may be implemented as hardware components (e.g., computing devices, processors, logic devices), executable computer program instructions (e.g., firmware, software) to be executed by various hardware components, or any combination thereof, as desired for a given set of design parameters or performance constraints.
• hardware components e.g., computing devices, processors, logic devices
• executable computer program instructions e.g., firmware, software
• the apparatus 10 may be embodied as, or in, a device or apparatus, such as a server, workstation, imaging device, or mobile device.
• the apparatus 10 may comprise one or more microprocessors or computer processors, which execute appropriate software.
• the processing unit 14 of the apparatus 10 may be embodied by one or more of these processors.
• the software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a nonvolatile memory such as flash.
• the software may comprise instructions configuring the one or more processors to perform the functions described herein.
• the apparatus 10 may be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.
• the functional units of the apparatus 10, e.g., input unit 12, processing unit 14, and output unit 16 may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA).
• the input unit 12 and the output unit 16 may be implemented by respective interfaces of the apparatus.
• each functional unit of the apparatus may be implemented in the form of a circuit.
• the apparatus 10 may also be implemented in a distributed manner.
• some or all units of the apparatus 10 may be arranged as separate modules in a distributed architecture and connected in a suitable communication network, such as a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, Internet, LAN (Local Area Network), Wireless LAN (Local Area Network), WAN (Wide Area Network), and the like.
• 3GPP 3rd Generation Partnership Project
• LTE Long Term Evolution
• Internet such as a 3rd Generation Partnership Project (LTE) network
• LAN Local Area Network
• Wireless LAN Local Area Network
• WAN Wide Area Network
• the input unit 12 may include hardware and/or software to enable the apparatus 10 to receive sensor data from a sensor via a wired connection or via a wireless connection.
• the processing unit(s) 14 may execute instructions to perform the method described herein, which will be explained in detail with respect to the embodiment shown in Fig. 2.
• the output unit 16 may include hardware and/or software to enable the apparatus to communicate with other devices (e.g. display, storage devices, etc.) and/or a network (LTE, LAN, Wireless LAN, etc.) to provide the generated quality metric.
• devices e.g. display, storage devices, etc.
• a network LTE, LAN, Wireless LAN, etc.
• Fig. 2 is a flowchart of a computer-implemented method 200 for prospective quality assessment for imaging examination prior to acquisition.
• the method 200 is described below with reference to Fig. 3.
• the method 200 is not, however, limited to the example of Fig. 3.
• the computer-implemented method 200 may be implemented as a device, module, or related component in a set of logic instructions stored in a non-transitory machine- or computer-readable storage medium such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc., in configurable logic such as, for example, programmable logic arrays (PLAs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), in fixed-functionality hardware logic using circuit technology such as, for example, application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof.
• a non-transitory machine- or computer-readable storage medium such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc.
• configurable logic such as, for example, programmable logic arrays (PLAs), field programmable gate arrays (FPGAs), complex programm
• computer program code to carry out operations shown in the method 200 may be written in any combination of one or more programming languages, including an object oriented programming language such as JAVA, SMALLTALK, C++, Python, or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages.
• object oriented programming language such as JAVA, SMALLTALK, C++, Python, or the like
• conventional procedural programming languages such as the "C" programming language or similar programming languages.
• the computer-implemented method 200 may be implemented as the apparatus 10 shown in Fig. 1
• Fig. 3 schematically shows a system 100 according to an embodiment of the present disclosure.
• the system 100 comprises an apparatus 10, a medical imaging apparatus 20, a patient support 22 movable along a central system axis of the medical imaging apparatus, a sensor 30, and a console 40.
• the medical imaging apparatus 20 is configured to acquire image data of a body part of a patient.
• the sensor 30 is configured to acquire sensor data of the body part of the patient.
• the apparatus 10 is configured to receive the sensor data and provide a quality metric for prospective quality assessment for imaging examination prior to acquisition.
• the console 40 may be coupled to a screen or monitor 42 on which the acquired images or imager settings may be viewed or reviewed.
• An operator such as a medical lab technical, can control via the console 40 an image acquisition.
• the apparatus 10 is a separate device configured to communicate with the sensor 30 through a wireless and/or a wire-based interface.
• the apparatus 10 may resident in the console 40 running as software routines.
• an apparatus such as apparatus 10 shown in Fig. 1, receives sensor data of a body part of a patient from a sensor that monitors the body part during positioning/preparation of an imaging examination.
• the body part of the patient is to be imaged by a medical imaging apparatus using a medical imaging modality.
• the apparatus may receive the sensor data from the sensor via a wired connection or via a wireless connection.
• the apparatus may have a sensor data interface to access the sensor data, which may take various forms, such as a network interface to a local area network, but also a video interface, e.g. HDMI, etc.
• the sensor 30 is a depth camera.
• the sensor 30 may also be e.g. a thermal sensor, an ultrasound sensor e.g. embedded in the patient support 22, an array of radio frequency sensors, a pressure sensor embedded in the patient support 22, an fibre optical range sensor, or any combination thereof.
• Fig. 3 only shows a single sensor, it will be appreciated that a sensor arrangement may be provided that comprises two or more sensors.
• the sensor 30 may be arranged remote from the medical imaging apparatus 20, e.g. on the ceiling of the room, such that the sensor has a field of view including at least part of an area, where the patient is positioned for imaging.
• the sensor such as a pressure sensor, may be embedded in the patient support 22.
• the body part of the patient will be imaged by an MR imaging apparatus.
• the medical imaging apparatus 20 may include, but are not limited to, an X-ray imaging apparatus, a CT imaging apparatus, a PET imaging apparatus, an automatically-steered ultrasound imaging apparatus, etc.
• the medical imaging apparatus 20 may also be a combined therapy /diagnostic apparatus, such as an MR-Linac apparatus, an MR proton therapy apparatus, and a cone beam CT apparatus.
• the apparatus generates a quality metric from the received sensor data using a data-driven model.
• the data-driven model has been trained to predict a quality metric of image data to be acquired by the medical imaging apparatus from (nonionizing) sensor data acquired by a sensor (e.g. depth camera) prior to the actual imaging examination.
• the data-driven model may be artificial neural networks, trained random forests, a model based segmentation approach, or the like.
• the apparatus may generate the quality metric directly from the received sensor data.
• the apparatus may fit an anatomy model of a target anatomy to the received sensor data, and then generate the quality metric from the fitted anatomy model of the target anatomy.
• the anatomy model of the target anatomy may be e.g. a two-dimensional model, a three-dimensional model, or a fourdimensional model.
• the quality metric is a vector of numerical values. Each numerical value represents a deviation of a position and/or a rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and/or rotation of the anatomical feature.
• the numerical values may comprise ordinal values, such as 0 representing a very small deviation, 0.5 representing a small deviation, and 1 representing a large deviation.
• the ordinal values may be determined by comparing the deviation with one or more predetermined thresholds.
• the numerical values may comprise continuous values, such as deviations in some target coordinate system, e.g. shift in [mm], rotation in degrees, FOV opening in percent, etc.
• Different numerical values may reflect different quality aspects.
• the X-ray image quality may be determined in terms of a quality vector with different quality aspects, such as Field-of-View (FOV) compliance, rotation, and flexion.
• the quality metric may also be a single value, such as a weighted sum of the above-described numerical values.
• the data-driven model has been trained based on a training dataset that comprises a plurality of training examples.
• Each training example comprises sensor data of the body part acquired in an imaging session and an associated quality metric derived from image data acquired using the medical imaging modality in the imaging session.
• An exemplary training process will be discussed hereinafter and in particular with respect to the embodiment shown in Fig. 4.
• the apparatus provides the generated quality metric for prospective quality assessment for imaging examination prior to acquisition.
• the trained data-driven model can be inferred during positioning/preparation of an imaging examination in order to provide a predicted quality metric. This may allow an operator (e.g. radiographer) to optimize the predicted quality metric prior to taking the actual exam. For instance, an instant prediction of the current positioning quality may be indicated, while moving the patient being observed from the sensor.
• an operator e.g. radiographer
• the apparatus 10 provides the generated quality metric to the console 40.
• the generated quality metric may be viewed or reviewed.
• An operator can control via the console 40 an image acquisition by actuating a joystick or pedal or other suitable input means coupled to the console 40, if the generated quality metric meets a criterion.
• the apparatus 10 may determine whether the generated quality metric meets a predetermined criterion and then generate a signal indicative of whether the generated quality metric meets the predetermined criterion.
• the generated signal may comprise a signal for controlling a device to inform whether the patient is ready for image acquisition.
• the device may be a speaker configured to generate an acoustic signal informing the patient that the patient is correctly positioned and the body part to be imaged should be kept motionless throughout the examination.
• the device may be a lighting device to generate a light signal (e.g. green light) informing the patient and/or the operator that the patient is correctly positioned.
• the generated signal may comprise a signal for triggering the medical imaging apparatus to start image acquisition.
• the operator does not need to manually control an image acquisition. Rather, the medical imaging apparatus 20 may be triggered to perform automated image acquisition by a signal indicating that the patient is correctly position for image acquisition.
• a warning signal may also be generated informing the patient to keep the body part to be imaged motionless throughout the imaging examination.
• the computer-implemented method, apparatus, and system described herein may thus improve the quality of medical image data acquired in the actual imaging examination, thereby leading to fewer retakes, less delayed treatment to patients, shortened workflow, and higher patient rate. In X-ray and CT exams, fewer retakes may also reduce radiation doses for patients.
• Fig. 4 is a flowchart of a method 300 for generating training data for training the data- driven model. The method 300 is described below with reference to Fig. 5 and Fig. 6. The method 300 is not, however, limited to the examples of Fig. 5 and Fig. 6.
• Fig. 5 shows a further example of a system 100.
• the system 100 comprises a medical imaging apparatus 20, which is an X-ray imaging apparatus comprising an X-ray source 20a and an X-ray detector 20b.
• the X-ray detector 20b is spaced from the X-ray source 20a to accommodate a body part of a patient to be imaged, such as the ankle shown in Fig. 5.
• a collimated X-ray beam emanates from the X-ray source 20a, passes through the patient at a region of interest (ROI), experiences attenuation by interaction with matter therein, and the attenuated beam then strikes the surface of the X-ray detector 20b.
• the density of the organic material making up the ROI determines the level of attenuation. High-density material (such as bone) causes higher attenuation than less dense materials (such as tissue).
• the registered digital values for the X-ray are then consolidated into an array of digital values forming an X-ray projection image for a given acquisition time and projection direction.
• the console 40 may be coupled to a screen or monitor 42 on which the acquired X-ray images or imager settings may be viewed or reviewed.
• An operator such as a medical lab technical can control via the console 40 an image acquisition run by releasing individual X-ray exposures for example by actuating a joystick or pedal or other suitable input means coupled to the console 40.
• a standard series may include an anteroposterior (AP) image, a Mortise image and a lateral image.
• AP an anteroposterior
• a Mortise image When calcaneal pathology is suspected, an additional image can be made in axial direction.
• an AP image of the ankle is acquired by the X-ray imaging apparatus 20.
• the apparatus 10 may be an apparatus as described with respect to Fig. 1.
• the apparatus 10 is a separate device configured to communicate with the sensor 30 through a wireless and/or a wire-based interface.
• the apparatus 10 may resident in the console 40 running as software routines.
• the sensor 30 may be a depth camera located close to or mounted to the tube head 20a.
• the depth camera can provide optical and depth information about the observed scene.
• the geometrical arrangement is known.
• Such sensor may be used before the acquisition to support the radiographer in assessing the patient position.
• image data of a body part of a patient is received.
• the image data of the body part may be retrieved from a database.
• the image data in the database may be obtained from routine examinations and/or research examinations in one or more departments or sites.
• the database may comprise image data of one or more body parts from one or more patients.
• the image data may be previously acquired by e.g. an X-ray imaging apparatus, an MR imaging apparatus, a CT imaging apparatus, or a PET imaging apparatus.
• the image data may also be acquired by a combined therapy/diagnostic apparatus, such as an MR-Linac apparatus, an MR proton therapy apparatus, a cone beam CT apparatus.
• the image data may comprise various images of the ankle, which can made by the X-ray imaging apparatus 20 shown in Fig. 5.
• An exemplary lateral image of the ankle is shown in Fig. 6.
• sensor data of the body part of the patient is received.
• the received sensor data and the received image data may be previously acquired in the same imaging examination.
• Sensor data may be acquired by any suitable sensor, such as an optical sensor, a depth sensor, a thermal sensor, a pressure sensor, an ultrasound sensor, an array of radio frequency sensors, or any combination thereof.
• the sensor 30 may be a depth camera for capture a depth image of the ankle.
• one or more features are extracted from the image data.
• the one or more features extracted from the image data may include, but are not limited to, landmarks, organ boundaries, region-of-interest, image labels in the image data acquired by the medical imaging apparatus, or any combination thereof.
• a fully automated procedure may be used to compute the one or more features, such as bone contours C1-C4 for tibia, fibula, talus, and calcaneus shown in Fig. 6.
• One option is to use an image-processing algorithm, such as an image detection algorithm that finds an approximate centre and bounding box of the one or more features in the medical image.
• the position and/or orientation of each extracted feature is compared with a desired position and/or orientation of the respective feature to determine a deviation of the feature.
• the skeletal X-ray image quality may be evaluated based on field-of-view (FOV) compliance, medio-lateral rotation, cranio-caudal rotation, joint flexion, etc.
• FOV field-of-view
• the quality metric may be a vector of numerical values. Each numerical value represents a deviation of a position and/or a rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and/or rotation of the anatomical feature.
• the X-ray image quality may be determined in terms of a quality vector with different quality aspects, such as FOV, rotation, and flexion.
• the quality metric derived from the image data acquired by the medical imaging apparatus and the sensor data acquired by the sensor are provided as training data.
• an anatomy model of the target anatomy may be fitted to the sensor data.
• the quality metric derived from the image data acquired by the medical imaging apparatus and the fitted anatomy model are provided as training data
• the data-driven model is trained on the training data.
• the annotated training data may be generated from routine examinations at one or more clinical sites.
• This continuous and unsupervised process of acquiring annotated training data provides a basis for the generation of a large ground truth database, thereby overcoming the disadvantage of the limited number of training sets obtained by a manual or semi-automated process. If data with new and not foreseen characteristics arise, or if the desired outcome of the trained system needs to be adapted, the continuously extending training set database may allow retaining of the machine-learning model for changes happening over time.
• the data-driven model may be continuously trained in the inference phase, that is, in the process of using a trained machine-learning algorithm to make a prediction.
• Fig. 7 shows an exemplary method 400 for implementing a weakly -supervised continuous learning.
• a quality metric is generated from sensor data acquired prior to an image acquisition following the procedures indicated with dashed arrows.
• the quality metric may also be referred to as predicted quality metric.
• the acquired sensor data is received in a way similar to the embodiment described with respect to block 210 in Fig. 2.
• a quality metric is generated from the received sensor data using the trained data-driven model in a way similar to the embodiment described with respect to block 220 in Fig. 2.
• the generated quality metric i.e. the predicted quality metric
• a further quality metric is generated from image data acquired by a medical imaging apparatus after the actual imaging examination following the procedures indicated with solid arrows.
• the further quality metric may also be referred to as actual quality metric.
• the acquired image data is received in a way similar to the embodiment described with respect to block 310 in Fig. 4.
• one or more features are extracted from the image data. The position and/or orientation of each extracted feature is compared with a desired position and/or orientation of the respective feature to determine a deviation of the feature, in a way similar to the embodiment described with respect to block 330 in Fig. 4.
• a quality metric is generated, which may be a vector of numerical values.
• Each numerical value represents a deviation of a position and/or a rotation of an anatomical feature in the image data acquired using the medical imaging modality from a desired position and/or rotation of the anatomical feature. This may be done in a way similar to the embodiment described with respect to block 340 in Fig. 4.
• an operator may review and check the quality metric vector for the exam, provide feedback, e.g. confirm/alter specific metrics, and put this expert feedback into the continuous learning loop, which implements an even more reliable strongly- supervised learning strategy.
• the apparatus 10 may receive a user input indicative of a user- defined quality metric of the received image data, determine a difference between the quality metric generated from the sensor data before image acquisition and the user-defined quality metric, and further train the data-driven model using the difference.
• the difference between the predicted quality metric derived from the sensor data prior to the image acquisition and the actual quality metric derived from the image data acquired after the actual image acquisition may be used as loss function to implement a weakly-supervised continuous learning.
• the continuous learning loop can be adapted to department-specific quality requirements (e.g. the width of the field-of-view) without the need of manual annotations.
• Fig. 8 shows an example of a continuously learning framework for quality metric prediction with example data.
• an actual quality metric is derived from image data acquired by a medical imaging apparatus (e.g. X-ray imaging apparatus shown in Fig. 5) following the procedures described with respect to blocks 410, 430, 440, and 460 in Fig. 7.
• a predicted quality metric is derived from sensor data acquired prior to image acquisition following the procedures described with respect to blocks 420, 470 and 480 in Fig. 7.
• the X-ray image quality is predicted in terms of a quality vector with different quality aspects (such as, field-of-view, rotation, and flexion).
• An RGB/depth camera may provide sensor data to predict quality metrics using a continuously updated data- driven model.
• a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.
• the computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention.
• This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above-described apparatus.
• the computing unit can be adapted to operate automatically and/or to execute the orders of a user.
• a computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method of the invention.
• This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.
• the computer program element might be able to provide all necessary steps to fulfil the procedure of an exemplary embodiment of the method as described above.
• a computer readable medium such as a CD-ROM
• the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
• a computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
• a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
• the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network.
• a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.

Landscapes

• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Medical Informatics (AREA)
• Radiology & Medical Imaging (AREA)
• General Health & Medical Sciences (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Public Health (AREA)
• Physics & Mathematics (AREA)
• Biomedical Technology (AREA)
• Animal Behavior & Ethology (AREA)
• Pathology (AREA)
• Veterinary Medicine (AREA)
• Optics & Photonics (AREA)
• General Physics & Mathematics (AREA)
• Biophysics (AREA)
• Heart & Thoracic Surgery (AREA)
• Molecular Biology (AREA)
• Surgery (AREA)
• Theoretical Computer Science (AREA)
• High Energy & Nuclear Physics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Epidemiology (AREA)
• Primary Health Care (AREA)
• Quality & Reliability (AREA)
• Apparatus For Radiation Diagnosis (AREA)
• Magnetic Resonance Imaging Apparatus (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=78598661

Family Applications (1)

Country Status (5)

Families Citing this family (1)

Citations (5)

Family Cites Families (2)

• 2021
  • 2021-09-30 EP EP21200178.8A patent/EP4159138A1/en not_active Withdrawn
• 2022
  • 2022-09-22 CN CN202280066386.9A patent/CN118042988A/zh active Pending
  • 2022-09-22 WO PCT/EP2022/076353 patent/WO2023052229A1/en not_active Ceased
  • 2022-09-22 EP EP22789912.7A patent/EP4408292B1/en active Active
  • 2022-09-22 US US18/695,386 patent/US20250006343A1/en active Pending
  • 2022-09-22 JP JP2024516608A patent/JP2024538516A/ja active Pending

Patent Citations (5)

Also Published As

Similar Documents

Legal Events