WO2022184736A1

WO2022184736A1 - Processing medical imaging data generated by a medical imaging device

Info

Publication number: WO2022184736A1
Application number: PCT/EP2022/055197
Authority: WO
Inventors: Claas Bontus; Nikolas David SCHNELLBÄCHER; Michael Grass
Original assignee: Koninklijke Philips N.V.
Priority date: 2021-03-05
Filing date: 2022-03-02
Publication date: 2022-09-09

Abstract

Imaging data is produced at a desired point during a motion of a region of interest. Imaging data obtained at the desired point is processed, together with imaging data obtained at other points with a predetermined distance to the desired points, to produce an output dataset including imaging data at the desired point.

Description

PROCESSING MEDICAL IMAGING DATA GENERATED BY A MEDICAL IMAGING

DEVICE

FIELD OF THE INVENTION

The present invention relates to the field of medical imaging and, in particular, to the processing of medical imaging data.

BACKGROUND OF THE INVENTION

Medical imaging is becoming of increasing importance in healthcare for aiding in the assessment and/or diagnosis of a patient of subject.

One area in which medical imaging faces a number of problems is when imaging a region of interest that changes during a motion, e.g. a motion caused by an anatomical cycle, such as a cardiac cycle or a respiration cycle. This is because the movement of the region of interest means that long imaging acquisition times cannot be used to generate a high- resolution image due to motion blur.

As a working example, generation of a high-resolution computed tomography (CT) image of a heart at a particular point or phase of the cardiac cycle would require extremely high doses of X-ray radiation in order to obtain sufficient data for generating the CT image. This results in increased patient, subject risk, exposure during the medical imaging process.

There is an ongoing desire to improve the generation of medical images and, in particular, improve the accuracy and precision of medical images of a region of interest at a particular point during a motion of the region of interest.

SUMMARY OF THE INVENTION

The invention is defined by the claims.

According to examples in accordance with one aspect of the invention, there is provided a computer-implemented method of processing medical imaging data generated by a medical imaging device configured to generate medical imaging data of a region of interest at different points during a motion of the region of interest during an imaging operation,.

The computer-implemented method comprises obtaining a first input dataset comprising medical imaging data generated by the medical imaging device at a desired point during the motion of the region of interest; obtaining at least one further input dataset comprising the medical imaging data generated by the medical imaging device at a respective at least one other point of the motion of the region of interest, wherein a difference between the desired point of the motion of the region of interest and each respective other point of the motion of the region of interest is predetermined; inputting the first input dataset and the at least one further input dataset to a machine-learning algorithm, wherein the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate an output dataset, wherein the output dataset is different to the first input dataset and comprises output medical imaging data of the region of interest at the desired point of the motion of the region of interest; and processing, using the machine-learning algorithm, the first input data set and the at least one further input dataset to generate the output dataset.

The present invention proposes to process medical imaging data for different points of a motion of region of interest. In particular, medical imaging data obtained at a plurality of different points in the motion of the region of interest are processed with a machine-learning algorithm to generate medical imaging data at a single point in the motion, the so-called desired point.

The input medical imaging data includes a first input dataset that comprises medical imaging data obtained at the desired point in the motion and also includes one or more further input datasets that comprise medical imaging data obtained at other points in the motion. The difference between the desired point and the other points is predetermined. This facilitates a consistent input to the machine-learning algorithm.

Medical imaging data obtained at different points in a motion of a region of interest can provide useful spatial information or additional structural information for processing, e.g. reducing noise, in medical imaging data obtained at a specific point during the motion. In particular, appropriately trained machine-learning algorithms are able to process the combined information to improve the medical imaging data obtained for a desired point.

This allows for naturally existing information to be used to increase the amount of data for machine-learning algorithms without additional cost, i.e. additional image acquisition. This facilitates improved speed of generating high-quality medical imaging data and/or facilitates the use of less accurate medical imaging devices, e.g. those that produce images having more noise, such as low-dose CT scanners.

Thus, there is a proposed a concept of a multi-channel input for a machine-learning algorithm, where each channel provides medical imaging obtained at a different point in a motion of a region of interest.

The “region of interest” refers to a region or feature of the subject’s anatomy, e.g. a heart, the lungs, a cardiac region, a respiratory region, a stomach, a muscle and so on. Motion of the region of interest may be any voluntary or involuntary movement of the region of interest, e.g. due to an anatomical cycle, such as motion of the heart, or movement of the muscle, e.g. a subject contracting or relaxing a muscle.

The output dataset may, for instance, be output to a user interface configured to display a visual representation of the medical imaging data contained in the output dataset. In other examples, the output dataset may be output to a memory arrangement or unit, e.g. for further processing, analysis or reference. In yet other examples, the output dataset may be output to further processing modules/elements.

The input datasets may be obtained, for instance, from the medical imaging device and/or a memory.

The output medical imaging data is reconstructed medical imaging data of the region of interest at the desired point of the motion of the region of interest. Thus, it temporally corresponds to the medical imaging data of the first input dataset. In this way, the output medical imaging data is effectively a processed version of the medical imaging data of the first input dataset.

For the sake of clarity, the medical imaging data of the first input dataset may be referred to as “first input medical imaging data”. The medical imaging data of each further input dataset may be referred to as “first further input medical imaging data”, where the term “first” is replaced by other ordinal numbers where appropriate, e.g. second, third, fourth etc.

The method may be carried out by a device, processing circuitry, e.g. of the device, and/or a processing arrangement. The machine-learning algorithm may be hosted by the processing circuitry and/or the processing arrangement.

For the purposes of the present application, the term “at least one” may be replaced by the term “one or more” where appropriate. Thus, “at least one” element is considered to refer to either a single element or a plurality of elements.

The method may comprise using the machine learning algorithm to reduce, based on the first input dataset and the at least one further input dataset, noise and/or artefacts in the medical imaging data of the first input dataset to generate the output dataset. Thus, the machine-learning algorithm may be configured to reduce noise and/or artefacts in the medical imaging data of the first input dataset, using the first input dataset and the at least one further input dataset, to generate the output dataset.

The proposed concept is advantageous when used to reduce noise/artefacts in the medical imaging data. This is because missing or erroneous data in the first input dataset can be supplemented or corrected using data found in the at least one further input dataset, e.g. which may provide information on a same region of interest. However, the machine-learning algorithm may be configured to perform other operations on the input datasets and, in particular, the first input dataset. In particular, the machine-learning algorithm may be configured to improve image characteristics of the medical imaging data contained in the first input dataset. For instance, the machine-learning algorithm may be configured to increase a resolution, increase a homogeneity, increase a signal -to-noise ratio, and/or increase a level of contrast in the medical imaging data of the first input dataset.

In some examples, the machine-learning algorithm is configured to transform the medical imaging data of the first input data to a predetermined coordinate system, using the first input dataset and the at least one further input dataset, to generate the output dataset.

Thus, the input datasets can be used to effectively register the medical imaging data of the first input dataset to a predetermined coordinate system. This embodiment recognizes that a machine-learning algorithm is capable of registering imaging data into one, i.e. a single, coordinate system.

Of course, a combination of any previously described operations performed by the machine-learning algorithm may be adopted. In other words, the machine-learning algorithm may be configured to carry out a combination of any herein described operations, e.g. to reduce a noise level of the medical imaging data and transform the medical imaging data to a predetermined coordinate system.

In some examples, the motion of the region of interest cycles according to an anatomical cycle; the first input dataset comprises the medical imaging data generated by the medical imaging device at a desired point of the anatomical cycle; each further input dataset comprises the medical imaging data generated by the medical imaging device at a respective other point of the anatomical cycle; and the difference between the desired point of the anatomical cycle and each respective other point of the anatomical cycle is predetermined.

An anatomical cycle refers to process of a subject’s anatomy that can modelled cyclically, such as a cardiac cycle, a respiratory cycle, a brain cycle, e.g. during sleep or due to blood flow, a spasm cycle, e.g. a cycle between spasms of a muscle or diaphragm and so on. A point in an anatomical cycle refers to a particular progress of phase through an anatomical cycle.

In some examples, a point of the anatomical cycle is represented by a percentage indicating a progress of the region of interest through the anatomical cycle; and for each of the at least one further input dataset(s), the difference between the percentage representing the desired point of the anatomical cycle and the percentage representing the other point of the anatomical cycle is a multiple of a first predetermined percentage value.

The first predetermined percentage value may be no greater than 10%, for example, no greater than 5%. Thus, the first predetermined percentage value may be no greater than 5% or 10%.

The at least one further input dataset may comprise no more than ten further input datasets, for instance, no more than four further input datasets. Thus, the at least one further input dataset may comprise no more than four further input datasets or ten further input datasets. In other words, the combination of the at least one further input dataset may contain medical imaging data from no more than ten or four different points of the motion of the region of interest, e.g. in the anatomical cycle.

The at least one further input dataset may contain medical imaging data from: a first number of points of the motion of the region of interest that precede the desired point of the motion of the region of interest; and a second number of points of the motion of the region of interest that follow the desired point of the motion of the region of interest, wherein the first and second number of points are equal.

Thus, the points of the motion that are represented by the medical imaging data of the further input datasets may be symmetric about the point of the motion represented by the medical imaging of the first input dataset. When the points are symmetrical about the desired point, the difference in motion state relative to the desired/target point is more likely to be at it smallest, thereby providing more contextually, i.e. in time and/or with respect to a particular motion state, relevant data for generating the output dataset.

In some examples, the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate a plurality of output datasets, the plurality of output datasets comprising: a first output dataset, different to the first input dataset, comprising medical imaging data of the region of interest at the desired point of the motion of the region of interest; and at least one further output dataset including, for each of a respective at least one further input dataset, a respective further output dataset different to the respective further input dataset, the further output dataset comprising medical imaging data of the region of interest at the other point of the motion of the region of interest of the further input dataset. In these examples, the method may comprise using the machine- learning algorithm to process the first input data set and at least one further input dataset to generate the first output dataset and the at least one further output dataset. Thus, processing, using the machine-learning algorithm, the first and further input datasets may comprise processing the first and further input datasets to generate the first output dataset and the one or more further output datasets.

In other words, the machine-learning algorithm may be configured to provide multiple output datasets containing medical image data at different points of the motion of the region of interest. Thus, a single machine-learning algorithm may be configured to perform the process of multiple machine-learning algorithms.

In some examples, the at least one further output dataset may comprise a further output dataset for each further input dataset. Thus, there may be a same number of output datasets as there are input datasets. However, this element is not essential, and in some examples only a subset, i.e. not all, of the further input datasets have a corresponding further output dataset.

In some examples, the method comprises using the machine learning algorithm to reduce, based on the first input dataset and the at least one further input dataset, noise and/or artefacts in the medical imaging data of the first input dataset and each of the at least one further input dataset to generate a plurality of output datasets.

Thus, the machine-learning algorithm may be configured to reduce noise and/or artefacts in the medical imaging data of the first input dataset and each further input dataset, using the first input dataset and the at least one further input dataset, to generate the plurality of output datasets.

In some examples, the machine-learning algorithm is configured to transform the medical imaging data of the first input data, and each further input dataset having a respective further output dataset, to a predetermined coordinate system, using the first input dataset and the at least one further input dataset, to generate the output dataset and the at least one further output datasets.

Thus, the input datasets can be used to effectively register the medical imaging data of a plurality of input datasets to each other and/or a predetermined coordinate system. This embodiment recognizes that a machine-learning algorithm is capable of registering imaging data into one, i.e. a single, coordinate system.

Of course, a combination of any previously described operations performed by the machine-learning algorithm may be adopted. In other words, the machine-learning algorithm may be configured to carry out a combination of any herein described operations, e.g. to both reduce a noise level of the medical imaging data and transform the medical imaging data to a predetermined coordinate system. Preferably, the machine-learning algorithm is a neural network. An example of a suitable neural network is a convolutional neural network, e.g. that makes use of a U-Net, ResNet or deep feed-forward neural network architecture, or a recurrent neural network. For improving the registration of elements of the region of interest during the processing, and thereby improving the accuracy of the output of the machine-learning algorithm, a multi resolution convolutional neural network approach could be applied. Other suitable machine- learning algorithms will be apparent to the skilled person, e.g. support-vector machines. As another example, a machine-learning algorithm may be a conventional, i.e. traditional, image processing algorithm, e.g. a traditional motion correction and/or denoising algorithms, where unknown model hyper-parameters or physical parameters are tuned or refined, e.g. by multi layer preceptrons or the like, to improve the performance of such algorithms.

In some examples, the region of interest is a heart and the motion of the region of interest is a cardiac cycle of the heart.

The medical imaging data of the first input dataset and the at least one further input dataset may be generated during a same, e.g. single, anatomical cycle of the region of interest. This ensures that the most contextually relevant data is used to construct the output dataset(s).

The medical imaging data may comprise CT data generated by a CT scanner, optionally wherein the computerized tomography data is generated using a low-dose CT technique. The skilled person will appreciate that a low-dose CT technique may differ depending upon the characteristics of a particular scanner, and that different CT scanners may have different definitions of a low-dose technique compared to a high-dose technique.

There is also proposed a non-transitory computer-readable medium for storing executable instructions that, when executed by processing circuitry, cause the processing circuitry to perform any herein described method. There is also proposed a computer program product comprising computer program code means which, when executed on a computing device having a processing system, cause the processing system to perform the method herein described.

There is also proposed a device configured to process medical imaging data generated by a medical imaging system configured to generate medical imaging data of a region of interest at different points during a motion of the region of interest in an imaging operation, the device comprising: processing circuitry; and a memory containing instructions that, when executed by the processing circuitry, configure the processing circuitry to: obtain a first input dataset comprising the medical imaging data generated by the medical imaging system at a desired point of the motion of the region of interest; obtain at least one further input dataset comprising the medical imaging data generated by the medical imaging system at a respective at least one other point of the motion of the region of interest, wherein a difference between the desired point of the motion of the region of interest and each respective other point of the motion of the region of interest is predetermined; input the first input dataset and the at least one further input dataset to a machine-learning algorithm, wherein the machine- learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate an output dataset, wherein the output dataset is different to the first input dataset and comprises output imaging data of the region of interest at the desired point of the motion of the region of interest; and process, using the machine-learning algorithm, the first input data set and the at least one further input dataset to generate the output dataset.

There is also provided a processing arrangement configured to process medical imaging data generated by a medical imaging device configured to, during an imaging operation, generate medical imaging data of a region of interest at different points during a motion of the region of interest.

The processing arrangement is configured to: obtain a first input dataset comprising the medical imaging data generated by the medical imaging system at a desired point of the motion of the region of interest; obtain at least one further input dataset comprising the medical imaging data generated by the medical imaging system at a respective at least one other point of the motion of the region of interest, wherein a difference between the desired point of the motion of the region of interest and each respective other point of the motion of the region of interest is predetermined; input the first input dataset and the at least one further input dataset to a machine-learning algorithm, wherein the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate an output dataset, wherein the output dataset is different to the first input dataset and comprises output imaging data of the region of interest at the desired point of the motion of the region of interest; and process, using the machine-learning algorithm, the first input data set and the at least one further input dataset to generate the output dataset.

The device and/or processing arrangement may be configured to perform any herein described method and vice versa. The skilled person would be able to appropriately modify the device, processing arrangement, method and/or computer program product accordingly.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show more clearly how it may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:

Figure 1 illustrates a conceptual approach adopted by embodiments;

Figure 2 is a flowchart illustrating a method;

Figure 3 illustrates a conceptual approach adopted by other embodiments;

Figure 4 is a flowchart illustrating another method; and

Figure 5 illustrates a device according to an embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The invention will be described with reference to the Figures.

It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, systems and methods, are intended for purposes of illustration only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems and methods of the present invention will become better understood from the following description, appended claims, and accompanying drawings. It should be understood that the Figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the Figures to indicate the same or similar parts.

The invention provides a mechanism for producing imaging data at a desired point during a motion of a region of interest. Imaging data obtained at the desired point is processed, together with imaging data obtained at other points with a predetermined distance to the desired points, to produce an output dataset including imaging data at the desired point.

The processing is performed to improve imaging characteristics of the imaging data at the desired point, e.g. to reduce noise and/or artefacts in the imaging data, to increase a resolution of the imaging data, to improve a contrast of the imaging data and/or to homogenize the imaging data.

Embodiments of the invention are based on the realization that imaging data captured at known different points of the motion of the region of interest can provide valuable information for improving the imaging characteristics of the imaging data at the desired point. It has also been recognized that a machine-learning algorithm provides the capability for processing imaging data captured at different points to provide imaging data captured at a desired point, e.g. at the additional data can act as additional features for the machine- learning algorithm.

Embodiments of the invention can be employed to improve the appearance of any medical imaging data generated using any suitable imaging modality, e.g. CT imaging data, ultrasound imaging data, MR imaging data, PET imaging data and so on.

In the context of the present application, medical imaging data is data obtained by a medical imaging device during an imaging process. Medical imaging data may comprise, for example, an image obtained from a medical imaging process or raw, i.e. unprocessed, data obtained by a medical imaging device, such as a sinogram for a CT scanner or raw ultrasound data, e.g. data that has not undergone beamforming.

For ease of explanation, embodiments will be hereafter described assuming that the medical imaging data comprises an image. However, the skilled person would be readily capable of adapting below-described approaches for other forms of medical imaging data.

Figure 1 conceptually illustrates an overview of an approach 100 adopted by embodiments of the present disclosure. The approach may, for instance, be adopted by a processing arrangement communicatively coupled to a medical imaging device and/or a storage arrangement receiving data from a medical imaging device.

A plurality of input datasets 110, 1(pi) - I(p_n) is input to a machine learning algorithm 120. The machine-learning algorithm 120 processes the input datasets 110 to produce an output dataset 130, 1(p_k). The purpose of the machine-learning algorithm 120 is to produce an output dataset 130 containing output medical imaging data of a region of interest at a desired point during a motion of the region of interest. The machine-learning method processes input datasets containing imaging data of the region of interest at the desired point and imaging data of the region of interest at other predetermined points during the motion of region of interest.

Thus, the machine-learning algorithm receives, as input, original medical imaging data at a desired point, and one or more other points, during a motion of a region of interest and produces, as output, reconstructed medical imaging data at the desired point.

In other words, the machine-learning algorithm 120 may receive multi-phase imaging data as input, and provides imaging data at a desired phase as output. The input imaging data includes imaging data at the desired phase, as well as imaging data at other predetermined phases.

As described in more detail below, the machine-learning algorithm 120 may be configured to improve image characteristics of the imaging data at the desired point in time using the imaging data obtained at the other points in time. The imaging characteristics may include, for instance, an amount of noise, a number of artefacts, a resolution, a homogeneity, signal-to-noise ratio, and/or a level of contrast.

In some examples or embodiments of the invention, the machine-learning algorithm 120 may be configured to transform the medical imaging data to medical imaging data in some predetermined coordinate system, i.e. to register the imaging data with respect to some predetermined coordinate system.

Of course, a combination of both of these approaches may be employed, so that the machine-learning algorithm 120 both converts and/or transforms the medical imaging data to medical imaging data in some predetermined coordinate system and improves image characteristics of the imaging data.

As one example, the machine-learning algorithm may be configured to receive low- dose CT imaging data and output simulated high-dose CT imaging data, e.g. transformed to a particular coordinate system.

The plurality of input datasets 110 therefore includes at least a first input dataset I(p_k) that contains medical imaging data obtained by a medical imaging device at a desired point during a motion of a region of interest. For instance, the first input dataset may include a medical image of a region of interest, e.g. the heart, at a particular point of an anatomical cycle of the region of interest. In another example, the first input dataset may include a medical image of the region of interest at a particular point during another, e.g. predetermined, motion of the region of interest, such as a contraction process of a muscle.

The plurality of input datasets 110 also includes at least one further input dataset I(pi), I(p2), I(p_n). Each at least one further input dataset contains medical imaging data generated by a medical imaging device at another point, i.e. at a different point in time compared to the first point, during the motion of the region of interest.

The difference between the first point of the first input dataset and each other point of a respective further input dataset is predetermined. In particular, there may be a predetermined difference in time and/or difference in phase/stage of a known/predetermined motion and/or a difference in a progress through a known/predetermined motion.

Thus, a point may represent a point in time, e.g. since a motion started, a progress point, e.g. a measure, such as a fraction or percentage, of progress through a predetermined motion, or a phase/stage point, e.g. a measure of a current phase/stage of a predetermined motion. In one embodiment, each point represents a progress of the region of interest through a predetermined motion, e.g. a progress through a known anatomical cycle or known movement. The difference between the first point and each other point may represent a predetermined difference in the progress through the predetermined motion.

As one exemplary embodiment, the region of interest may be a heart and the motion may result from a cardiac cycle of the heart, so that the motion is cyclical. The first point may represent a point during the cardiac cycle at which the ventricles start to retract. One of the other points may represent a point during the cardiac cycle at which the ventricles begin to relax.

As another exemplary embodiment, start and end points of an anatomical cycle may be determined using known means, e.g. a start and end of a cardiac cycle may be identified by processing heartrate information from a separate heartrate monitor or through known image analysis mechanisms. The first point may correspond to a predetermined progress through the anatomical cycle, e.g. at 50% of the way through the anatomical cycle. Each other point may correspond to other predetermined progress through the anatomical cycle, e.g. 40% though the anatomical cycle of 60% through the anatomical cycle.

In some exemplary embodiments, a point of the anatomical cycle is represented by a percentage indicating a progress of the region of interest through the anatomical cycle, e.g., for a heart, the progress through a cardiac cycle or, for the lungs, a progress through the respiratory cycle. This effectively means that a point represents a particular phase or stage of the motion of the region of interest.

In these exemplary embodiments, for each further input dataset, the difference between the percentage representing the desired point of the anatomical cycle and the percentage representing the other point of the anatomical cycle is a multiple of a first predetermined percentage value. The first predetermined percentage value may be, for example, no greater than 10% or, for example, no greater than 5%.

This approach means that the most contextually relevant imaging data is included with the first input dataset, improving the performance and accuracy of the machine-learning algorithm.

If the motion is an anatomical cycle, the input datasets may contain medical imaging data within only a single anatomical cycle and/or across a plurality of cycles, e.g. within adjacent anatomical cycles only.

Each point may represent a start/end to a stage/phase of the motion performed by the region of interest. For instance, the target/desired point may represent a first phase of the motion, e.g. when a motion has reached some predetermined criteria - such as blood entering a left and/or right ventricle of a heart, and each other point may represent another phase of the motion, e.g. when the motion has reached some other predetermined criteria, such as blood leaving a left and/or right ventricle of the heart or blood entering the left or right atrium.

Each point may represent a time since the motion of the region of interest begun.

Thus, the target/desired point may represent a point at which a first time period has elapsed since the motion of the region of interest begun, with the other points representing points at which other times periods have elapsed.

The output dataset 130, 1(p_k) comprises output imaging data at the desired point of the motion of the region of interest. Thus, the output dataset contains imaging data at the same point as the first input dataset provided to the machine-learning algorithm. The output dataset differs from the first input dataset and, in particular, may contain imaging data at the desired point during the motion of the region of interest that has improved image characteristics compared to the imaging data contained in the first input dataset.

The machine-learning algorithm is configured to process the input datasets 110 to provide, as output, an output dataset that includes reconstructed imaging data at the desired point of the motion of the region of interest. The reconstructed imaging data may have improved imaging characteristics, such as those previously described, compared to the imaging data contained in the first input dataset and/or be registered/transformed/converted to some predetermined coordinate system

The process f(.) performed by the machine-learning algorithm can be modelled as follows:

The machine-learning algorithm is hosted by a processing arrangement.

Embodiments of the invention make use of a machine-learning algorithm to process input datasets to produce an output dataset. A machine-learning algorithm is capable of performing the feature registration tasks required, e.g. to map features of different input datasets to one another, in order to generate the output dataset.

A machine-learning algorithm is any self-training algorithm that processes input data in order to produce or predict output data. Here, the input data input datasets, including the first input dataset and the at least one further input dataset, and the output data comprises the output dataset previously described. Suitable machine-learning algorithms for being employed in the present invention will be apparent to the skilled person. Examples of suitable machine-learning algorithms include decision tree algorithms and artificial neural networks. Other machine-learning algorithms, such as logistic regression, support vector machines or Naive Bayesian models are suitable alternatives.

The structure of a neural network is inspired by the human brain. Neural networks are comprised of layers, each layer comprising a plurality of neurons. Each neuron comprises a mathematical operation. In particular, each neuron may comprise a different weighted combination of a single type of transformation, e.g. the same type of transformation, sigmoid etc. but with different weightings. In the process of processing input data, the mathematical operation of each neuron is performed on the input data to produce a numerical output, and the outputs of each layer in the neural network are fed into the next layer sequentially. The final layer provides the output.

Methods of training a machine-learning algorithm are well known. Typically, such methods comprise obtaining a training dataset, comprising training input data entries and corresponding training output data entries. An initialized machine-learning algorithm is applied to each input data entry to generate predicted output data entries. An error between the predicted output data entries and corresponding training output data entries is used to modify the machine-learning algorithm. This process can be repeated until the error converges, and the predicted output data entries are sufficiently similar to the training output data entries. This is commonly known as a supervised learning technique.

For example, where the machine-learning algorithm is formed from a neural network, weightings of the mathematical operation of each neuron may be modified until the error converges. Known methods of modifying a neural network include gradient descent, backpropagation algorithms and so on.

The training input data entries correspond to example input datasets. The training output data entries correspond to example output datasets.

As a working example, consider a scenario in which the machine-learning algorithm is to be trained to denoise a low-dose CT image, e.g. to produce a denoised CT image modelling or simulating a high-dose CT image. For this scenario, the output dataset would be a denoised CT image, e.g. modelling a CT image obtained using a high-dose of radiation, and the input datasets are CT images obtained using a low-dose of radiation, including a CT image captured at a desired point of a motion of a region of interest and at least one other CT image each captured at a respective other point of the motion of the region of interest. In this scenario, a training input data entry and a training output data entry of the training dataset may be generated by first acquiring high-dose CT images at a plurality of points of the motion of the region of interest. The plurality of points includes the desired point, and the other points for the further input datasets. The high-dose CT image at the desired/target point is selected to act as the training output data entry. Projection noise may be added to the high-dose CT images to produce simulated low-dose CT images at a plurality of points of the motion of the region of interest. The training input data entry is generated by selecting the simulated low-dose CT image at the desired/target point and one or more other CT images at the points for the further input datasets from the simulated low-dose CT images.

The machine-learning network may, for example, be a convolutional neural network (CNN), using a U-Net, ResNet or deep feed-forward neural network architecture. In particular, for improved joint registration during the processing by the machine-learning algorithm, i.e. the task of mapping characteristics of imaging data from one point to imaging data at another point, a multi-resolution CNN approach is preferred.

From the foregoing, it will be understood that the embodiments of the invention are to take, as input, image datasets at different points in time/phase/progress as input and output an image dataset at a target/desired point having improved image characteristics and/or being transformed/converted/mapped to some predetermined coordinate system. The input image datasets include an image dataset at the target/desired point, so that the image characteristics of this input image dataset and/or at the predetermined coordinate system are effectively improved.

For instance, the machine-learning algorithm may de-noise the input image dataset at the target/desired point to produce a denoised image dataset.

The other points of the medical imaging data in the at least one further input dataset could be symmetrically centered around the desired/target point.

Thus, for instance, if the first input dataset comprises medical imaging data captured at 40% progress through an anatomical cycle, the at least one further input dataset may comprise two further input datasets: a first further input dataset captured at 35% progress through the anatomical cycle and a second further input dataset captured at 45% progress through the anatomical cycle. In this example, the at least one further input dataset may further comprise two more further input datasets for a total of at least four further input dataset: a third further input dataset captured at 30% progress through the anatomical cycle and a fourth further input dataset captured at 50% progress through the anatomical cycle. Symmetrically centering the points of the further input datasets around the desired/target point improves the performance capability of the machine-learning algorithm, as such data provides more contextually relevant data for improving the image characteristics of the imaging data at the desired/target point.

This is merely an exemplary embodiment, and the described method is not limited to this mechanism of point selection but generally can be used with various combinations of imaging data obtained at different points of a motion. In particular, the desired/target point does not have to be the central point of the points of the imaging data contained in the input datasets, but could equally be any other non-central point.

Figure 2 illustrates a method 200 according to an embodiment of the invention.

The method 200 may be carried out by a processing arrangement, such as a processing arrangement communicatively coupled to a medical imaging device, formed as an aspect of the medical imaging device, or communicatively coupled to a memory arrangement, e.g. that stores data generated by a medical imaging device.

The method 200 comprises a step 210 of obtaining a first input dataset 205. The first input dataset 205 contains medical imaging data obtained, generated and/or captured by a medical imaging device at a desired point during a motion of a region of interest.

The method 200 also comprises a step 220 of obtaining at least one further input dataset 207. Each further input dataset comprises medical imaging data obtained, generated and/or captured by the medical imaging device at a respective different other point of the motion of the region of interest. For each further input dataset, the difference between the desired point of the motion and the other point of the motion is predetermined and non-zero.

Here, an “other point” means that the point is different to the desired point. Each “other point” is also different to every other “other point”.

Steps 210 and 220 can be performed at the same time, e.g. during a single obtaining process. Steps 210 and 220 may be performed by the processing arrangement obtaining, e.g. receiving as input, the datasets from the medical imaging device and/or a memory arrangement.

The method further comprises a process 230 of generating an output dataset 209 by processing the first input dataset 205 and the one or more other datasets 207, using a machine-learning algorithm. The machine-learning algorithm is hosted by the processing arrangement.

The process 230 may comprise inputting the first input dataset and the further input dataset(s) into the machine-learning algorithm in a step 231. The process 230 may then process the input datasets using the machine-learning algorithm, in a step 232, to produce the output dataset. The process 233 may then output, e.g. to a user interface or memory arrangement, the produced output dataset 209.

The output dataset 209 comprises reconstructed imaging data at the desired/target point of the motion of the region of interest, i.e. at the same point of the motion as represented by the imaging data of the first input dataset. In particular, the output dataset comprises imaging data having improved imaging characteristics compared to the imaging data contained in the first input dataset, e.g. denoised imaging data.

Approaches for processing the first input dataset and the further input dataset(s) to produce the output dataset have been previously described.

In some exemplary embodiments, the method may be further adapted to comprise a step of receiving a user input, e.g. provided via a user interface. The user input may indicate which of an obtained plurality of input datasets is the first input dataset, e.g. by indicating which point of the motion of the region of interest is the desired point.

The skilled person will appreciate that each input dataset may be accompanied by metadata indicating which point of the motion of the region of interest is represented by the medical imaging data of the input dataset. This allows the different forms of imaging data to be distinguished from one another, e.g. to allow identification of the first input dataset and/or the desired point of the motion of the region of interest.

Figure 3 conceptually illustrates an overview of another approach 300 adopted by embodiments of the present disclosure.

Like the approach 100, the approach 300 processes input datasets 310 using a machine-learning algorithm 320.

However, rather than generating only a single output dataset containing medical imaging data at the desired point the motion of the region of interest, as illustrated in Figures 1 and 2, the approach generates a plurality 330 of output datasets I(pi) - I(p_n), each containing medical imaging data at a different desired point of the motion of the region of interest.

The plurality of output datasets includes a first output dataset I(p_k), different to the first input dataset I(p_k), comprising medical imaging data of the region of interest at the desired point of the motion of the region of interest. The first output dataset thereby corresponds to the output dataset produced by the approach described with reference to Figures 1 and 2. The plurality of output datasets further includes one or more further output datasets. Each further output dataset corresponds to one of the further input datasets received as input, and contains medical imaging data of the region of interest at the other point of the corresponding further input dataset. In some examples, there may be the same number of further output datasets as further input datasets, each further output dataset corresponding to a respective further input dataset, i.e. corresponding to a same point in the motion of the region of interest.

Thus, each further output dataset comprises medical imaging data of the region of interest at one of the other points of the motion of the region of interest. In this way, at least one of the other points acts as a further desired point of the motion of the region of interest.

Put another way, Figure 3 illustrates an approach in which there are multiple desired points of the region of interest, and multiple output datasets are generated, each containing reconstructed imaging data at a respective desired point of the region of interest. This is performed by processing input datasets, which include input datasets that contain medical imaging data captured at a respective desired point of the region of interest, as well as optionally further input datasets at other points of motion of the region of interest.

The machine-learning algorithm may be adapted accordingly, i.e. trained to provide multiple output datasets.

Figure 4 illustrates a method 400 according to an embodiment. The method may be carried out by a processing arrangement, in a similar manner to the method 200 of Figure 2.

The method 400 comprises a step 410 of obtaining a first input dataset 405. The first input dataset 405 comprises imaging data at a desired/target point of a motion of a region of interest in the imaging data.

The method 400 also comprises a step 420 of obtaining a plurality of further input datasets 407. Each further input dataset comprises imaging data obtained at a different, other point of the motion of the region of interest. The term “other point” has been previously clarified.

The method then performs a process 430 of processing the input datasets to provide a plurality of output datasets 409. The plurality of output datasets 409 includes a first output dataset, containing reconstructed medical imaging data at the desired/target point of the motion of the region of interest, as well as one or more further output dataset, each containing reconstructed medical imaging data at a respective other point of the motion of the region of interest. In a similar manner to the process 230 of method 200, process 430 may comprise inputting the first input dataset and the further input dataset(s) into the machine-learning algorithm in a step 431. The process 430 may then process the input datasets using the machine-learning algorithm, in a step 432, to produce the output datasets. The process 433 may then output, e.g. to a user interface or memory arrangement, the produced output datasets 409.

Method 400 differs from method 200 in that the machine-learning algorithm produces a plurality of output datasets, each corresponding to a different input dataset and providing medical imaging data at the same point of the motion of the region of interest as the medical imaging data contained in the corresponding input dataset. The medical imaging data of each output dataset may be denoised version and/or a version having other improved imaging characteristics and/or transformed/converted/registered to some predetermined coordinate system of the medical image data of its corresponding input dataset. The predetermined coordinate system may be a coordinate system defined by the imaging data of one of the input datasets, e.g. the first input dataset, so that each output dataset is registered to the same coordinate system.

By way of further example, Figure 5 illustrates an example of a device 50. alternatively labelled “a processing arrangement,” within which one or more parts of an embodiment may be employed. Various operations discussed above may utilize the capabilities of the device 50. For example, one or more parts of a system for processing an image with a CNN may be incorporated in any element, module, application, and/or component discussed herein. In this regard, it is to be understood that system functional blocks can run on a single computer or may be distributed over several computers and locations, e.g. connected via Internet.

The device 50 includes, but is not limited to, processors, PCs, workstations, laptops, PDAs, palm devices, servers, storages, cloud computing devices, distributed processing systems and the like. Generally, in terms of hardware architecture, the device 50 may include one or more processing circuitries 51, memory 52, and one or more I/O devices 53 that are communicatively coupled via a local interface (not shown). The local interface can be, for example but not limited to, one or more buses or other wired or wireless connections, as is known in the art. The local interface may have additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communications. Further, the local interface may include address, control, and/or data connections to enable appropriate communications among the aforementioned components. The processing circuitry 51 is a hardware device for executing software that can be stored in the memory 52. The processing circuitry 51 can be virtually any custom made or commercially available processing circuitry, a central processing unit (CPU), a digital signal processing circuitry (DSP), or an auxiliary processing circuitry among several processing circuitries associated with the device 50, and the processing circuitry 51 may be a semiconductor based microprocessing circuitry in the form of a microchip or a microprocessing circuitry.

The memory 52 can include any one or combination of volatile memory elements, e.g., random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and non-volatile memory elements (e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), tape, compact disc read only memory (CD-ROM), disk, diskette, cartridge, cassette or the like, etc. Moreover, the memory 52 may incorporate electronic, magnetic, optical, and/or other types of storage media. Note that the memory 52 can have a distributed architecture, where various components are situated remote from one another, but can be accessed by the processing circuitry 51.

The software in the memory 52 may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. The software in the memory 52 includes a suitable operating system (O/S) 54, compiler 55, and one or more applications 56 in accordance with exemplary embodiments. As illustrated, the application 56 comprises numerous functional components for implementing the features and operations of the exemplary embodiments. The application 56 of the device 50 may represent various applications, computational units, logic, functional units, processes, operations, virtual entities, and/or modules in accordance with exemplary embodiments, but the application 56 is not meant to be a limitation.

The O/S 54 controls the execution of other computer programs, and provides scheduling, input-output control, file and data management, memory management, and communication control and related services. It is contemplated by the inventors that the application 56 for implementing exemplary embodiments may be applicable on all commercially available operating systems.

Application 56 may be a source program, executable program (object code), script, or any other entity comprising a set of instructions to be performed. When a source program, then the program is usually translated via a compiler, such as the compiler 55, assembler, interpreter, or the like, which may or may not be included within the memory 52, so as to operate properly in connection with the O/S 54. Furthermore, the application 56 can be written as an object oriented programming language, which has classes of data and methods, or a procedure programming language, which has routines, subroutines, and/or functions, for example but not limited to, C, C++, C#, Pascal, BASIC, API calls, HTML, XHTML, XML, ASP scripts, JavaScript, FORTRAN, COBOL, Perl, Java, ADA, .NET, and the like.

The I/O devices 53 may include input devices such as, for example but not limited to, a mouse, keyboard, scanner, microphone, camera, etc. Furthermore, the I/O devices 53 may also include output devices, for example but not limited to a printer, display, etc. Finally, the I/O devices 53 may further include devices that communicate both inputs and outputs, for instance but not limited to, a modulator/demodulator (for accessing remote devices, other files, devices, systems, or a network), a radio frequency (RF) or other transceiver, a telephonic interface, a bridge, a router, etc. The I/O devices 53 also include components for communicating over various networks, such as the Internet or intranet.

If the device 50 is a PC, workstation, intelligent device or the like, the software in the memory 52 may further include a basic input output system (BIOS) (omitted for simplicity). The BIOS is a set of essential software routines that initialize and test hardware at startup, start the O/S 54, and support the transfer of data among the hardware devices. The BIOS is stored in some type of read-only-memory, such as ROM, PROM, EPROM, EEPROM or the like, so that the BIOS can be executed when the device 50 is activated.

When the device 50 is in operation, the processing circuitry 51 is configured to execute software stored within the memory 52, to communicate data to and from the memory 52, and to generally control operations of the device 50 pursuant to the software. The application 56 and the O/S 54 are read, in whole or in part, by the processing circuitry 51, perhaps buffered within the processing circuitry 51, and then executed.

When the application 56 is implemented in software it should be noted that the application 56 can be stored on virtually any computer readable medium for use by or in connection with any computer related system or method. In the context of this document, a computer readable medium may be an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program for use by or in connection with a computer related system or method.

The application 56 can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer- based system, processing circuitry-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium.

In the context of the present disclosure, the I/O devices 53 may be configured to receive the input datasets from a medical imaging device 510. In another example, the input datasets are obtained from a memory arrangement or unit (not shown).

The I/O devices 53 may be configured to provide the output dataset(s) to a user interface 520. The user interface may be configured to provide a visual representation of the output dataset(s), e.g. display an image or images corresponding to the medical imaging data contained in the output dataset(s).

There is also proposed an image processing system 500 comprising the device 50.

The image processing system may further comprise the user interface 520 and/or the medical imaging device 510.

The skilled person would be readily capable of developing a processing arrangement for carrying out any herein described method. Thus, each step of the flow chart may represent a different action performed by a processing arrangement, and may be performed by a respective module of the processing arrangement.

Embodiments may therefore make use of a processing arrangement. The processing arrangement can be implemented in numerous ways, with software and/or hardware, to perform the various functions required. A processor is one example of a processing arrangement which employs one or more microprocessors that may be programmed using software (e.g., microcode) to perform the required functions. A processing arrangement may however be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.

Examples of processing arrangement components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). In various implementations, a processor or processing arrangement may be associated with one or more storage media such as volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and/or processing arrangements, perform the required functions. Various storage media may be fixed within a processor or processing arrangement or may be transportable, such that the one or more programs stored thereon can be loaded into a processor or processing arrangement.

It will be understood that disclosed methods are preferably computer-implemented methods. As such, there is also proposed the concept of a computer program comprising code means for implementing any described method when said program is run on a processing arrangement, such as a computer. Thus, different portions, lines or blocks of code of a computer program according to an embodiment may be executed by a processing arrangement or computer to perform any herein described method. In some alternative implementations, the functions noted in the block diagram(s) or flow chart(s) may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. If a computer program is discussed above, it may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. If the term "adapted to" is used in the claims or description, it is noted the term "adapted to" is intended to be equivalent to the term "configured to". Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. A computer-implemented method of processing medical imaging data, comprising: obtaining a first input dataset comprising medical imaging data generated by a medical imaging device at a desired point during a motion of a region of interest; obtaining at least one further input dataset comprising the medical imaging data generated by the medical imaging device at a respective at least one other point of the motion of the region of interest, wherein a difference between the desired point of the motion of the region of interest and each respective other point of the motion of the region of interest is predetermined; inputting the first input dataset and the at least one further input dataset to a machine- learning algorithm, wherein the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate an output dataset, wherein the output dataset is different to the first input dataset and comprises output medical imaging data of the region of interest at the desired point of the motion of the region of interest; and processing, using the machine-learning algorithm, the first input data set and the at least one further input dataset to generate the output dataset.

2. The computer-implemented method according to claim 1, further comprising using the machine learning algorithm to reduce, based on the first input dataset and the at least one further input dataset, noise and/or artefacts in the medical imaging data of the first input dataset to generate the output dataset.

3. The computer-implemented method according to claim 1 or 2, wherein: the motion of the region of interest cycles according to an anatomical cycle; the first input dataset comprises the medical imaging data generated at the desired point of the anatomical cycle; each of the at least one further input dataset comprises the medical imaging data generated at a respective other point of the anatomical cycle; and the difference between the desired point of the anatomical cycle and each respective other point of the anatomical cycle is predetermined.

4. The computer-implemented method according to claim 3, wherein: a point of the anatomical cycle is represented by a percentage indicating a progress of the region of interest through the anatomical cycle; and for each of the at least one further input dataset, the difference between the percentage representing the desired point of the anatomical cycle and the percentage representing the other point of the anatomical cycle is a multiple of a first predetermined percentage value.

5. The computer-implemented method according to claim 4, wherein the first predetermined percentage value is no greater than 5% or 10%.

6. The computer-implemented method according to claims 1 to 5, wherein the at least one further input dataset comprises no more than four further input datasets or ten further input datasets.

7. The computer-implemented method according to claims 1 to 6, wherein the at least one further input dataset contains the medical imaging data from: a first number of points of the motion of the region of interest that precede the desired point of the motion of the region of interest; and a second number of points of the motion of the region of interest that follow the desired point of the motion of the region of interest, wherein the first number of points and the second number of points are equal.

8. The computer-implemented method according to claims 1 to 7, wherein: the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate a plurality of output datasets, the plurality of output datasets comprising: a first output dataset, different to the first input dataset, comprising the medical imaging data of the region of interest at the desired point of the motion of the region of interest; and at least one further output dataset including, for each of the at least one further input dataset, a respective further output dataset different to the at least one further input dataset, the at least one further output dataset comprising the medical imaging data of the region of interest at the other point of the motion of the region of interest of the at least one further input dataset; and using the machine-learning algorithm to process the first input data set and at least one further input dataset to generate the first output dataset and the at least one further output dataset.

9. The computer-implemented method according to claim 8, further comprising using the machine learning algorithm to reduce, based on the first input dataset and the at least one further input dataset, noise and/or artefacts in the medical imaging data of the first input dataset and each of the at least one further input dataset to generate a plurality of output datasets.

10. The computer-implemented method according to claims 1 to 9, wherein the machine- learning algorithm is a neural network.

11. The computer-implemented method according to claims 1 to 10, wherein the region of interest is a heart, and wherein the motion of the region of interest is a cardiac cycle of the heart.

12. The computer-implemented method according to claims 1 to 11, wherein the medical imaging data of the first input dataset and the at least one further input dataset is generated during a same anatomical cycle of the region of interest.

13. The computer-implemented method according to claims 1 to 12, wherein the medical imaging data comprises computed tomography (CT) data.

14. A non-transitory computer-readable medium for storing executable instructions that, when executed by processing circuitry, cause the processing circuitry to perform the method according to claims 1-13.

15. A device configured to process medical imaging data, comprising: processing circuitry; and a memory containing instructions that, when executed by the processing circuitry, configure the processing circuitry to: obtain a first input dataset comprising the medical imaging data generated by a medical imaging system at a desired point of a motion of a region of interest; obtain at least one further input dataset comprising the medical imaging data generated by the medical imaging system at a respective at least one other point of the motion of the region of interest, wherein a difference between the desired point of the motion of the region of interest and each respective other point of the motion of the region of interest is predetermined; input the first input dataset and the at least one further input dataset to a machine-learning algorithm, wherein the machine-learning algorithm is configured to process the first input dataset and the at least one further input dataset to generate an output dataset, wherein the output dataset is different to the first input dataset and comprises output medical imaging data of the region of interest at the desired point of the motion of the region of interest; and process, using the machine-learning algorithm, the first input data set and the at least one further input dataset to generate the output dataset.