JP2024515595A - Determining confidence indicators for deep learning image reconstruction in computed tomography - Google Patents

Abstract

A method and system for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT) is provided. The method comprises (S1) acquiring energy resolved X-ray data, and (S2) processing the energy resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution of at least one reconstructed basis image or image features thereof. The method further comprises (S3) generating one or more confidence indications for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image based on the representation of the posterior probability distribution.

Description

The proposed techniques relate to X-ray technology, X-ray imaging, and corresponding imaging reconstructions and imaging tasks. In particular, the proposed techniques relate to methods and systems for determining confidence indications for deep-learning image reconstruction in Computed Tomography (CT), methods and systems for generating uncertainty maps for deep-learning image reconstruction in Spectral CT, and corresponding image reconstruction and X-ray imaging systems, as well as related computer programs and computer program products.

Radiological images, such as X-ray images, have been used for many years for medical applications and non-destructive testing.

Typically, an x-ray imaging system includes an x-ray source and an x-ray detector array that includes multiple detectors, each containing one or many detector elements (independent means of measuring x-ray intensity/influence). The x-ray source emits x-rays, which pass through the subject or object being imaged and are registered by the detector array. Some materials absorb x-rays more than others, forming an image of the subject or object.

The challenge of an X-ray imaging detector is to extract the maximum amount of information from the detected X-rays and enter it into an image of the object or subject.

In a typical medical x-ray imaging system, x-rays are generated by an x-ray tube. A typical medical x-ray tube has a broad energy spectrum, ranging from zero to 180 keV. Therefore, the detector typically detects x-rays of a range of energies.

It may be useful to provide a brief overview of an exemplary overall X-ray imaging system with reference to FIG. 1. In this exemplary but non-limiting example, the X-ray imaging system 100 essentially consists of an X-ray source 10, an X-ray detector system 20, and an associated image processing system or device 30. In general, the X-ray detector system 20 is configured to register radiation from the X-ray source 10, optionally focused by X-ray optics, and passing through an object, subject, or part thereof. The X-ray detector system 20 is connectable to the image processing system 30 via suitable analog and readout electronics that are at least partially integrated into the X-ray detector system 20 to enable image processing and/or image reconstruction by the image processing system 30.

As an example, an X-ray computed tomography (CT) system includes an X-ray source and an X-ray detector arranged to allow projection images of a subject or object to be acquired at different view angles covering at least 180 degrees. This is most commonly achieved by mounting the source and detector on a support that can be rotated around the subject or object. An image containing projections registered onto different detector elements for different view angles is called a sinogram, and in the following, even if the detector is two-dimensional, the collection of projections registered onto different detector elements for different view angles is called a sinogram, and a sinogram is a three-dimensional image.

A further development of X-ray imaging is energy-resolved X-ray imaging, also known as spectral X-ray imaging. This can be achieved by having the source rapidly switch between two different emission spectra, by using two or more X-ray sources that emit different X-ray spectra, or, more prominently, by using an energy-discriminating detector that measures the incident radiation at two or more energy levels. One example of such a detector is a multi-bin photon-counting detector, where each registered photon produces a current pulse that is compared to a set of thresholds, thereby counting the number of photons incident on each of a number of energy bins.

In spectral X-ray projection measurements, projection images are typically obtained for each energy level. A weighted sum of these can be made to optimize the contrast-to-noise ratio (CNR) for a given imaging task, as described in Tapsovaara and Wagner, "SNR and DQE analysis of broadspectrum X-ray imaging," Rhys. Med. Biol. 30, 519.

Another technique that energy-resolved X-ray imaging makes possible is basis material decomposition, which exploits the fact that all materials composed of elements with low atomic numbers, such as human tissue, have a linear attenuation coefficient (E) whose energy dependence can be approximately expressed as a linear combination of two basis functions.

where _f1 and _f2 are basis functions and _a1 and _a2 are corresponding basis coefficients. More generally, _f1 is a basis function and _a1 is the corresponding basis coefficient. If there are one or more elements in the imaging volume with sufficiently high atomic number that their K-absorption edges lie in the energy range used for imaging, one basis function must be added for each such element. In the field of medical imaging, such K-edge elements might typically be iodine or gadolinium, which are substances used as contrast agents.

Basis material decomposition is described in Alvarez and Macovski, "Energy-selective reconstructions in X-ray computed tomography," Phys. Med. Biol. 21, 733, in which the integral of each basis coefficient Ai = ∫ _l a _i dl, for i = 1...N, where N is the number of basis functions, is estimated from the measured data at each projection line l from the source to the detector element. In one implementation, this is accomplished by first expressing the expected registration counts at each energy bin as a function of _Ai .

where λ _i is the expected count in energy bin i, E is the energy, and S _i is a response function that depends on the spectral shape incident on the object, the quantum efficiency of the detector, and the sensitivity of energy bin i to x-rays with energy E. Although the term "energy bin" is most commonly used for photon counting detectors, this formula can also describe other energy resolving x-ray systems such as multi-layer detectors and kVp switching sources.

Then, under the assumption that the counts in each bin are Poisson-distributed random variables, Ai can be estimated using the maximum likelihood method. This is accomplished by minimizing the negative log-likelihood function. See Roessl and Proksa, "K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors," Phys. Med. Biol. 52 (2007), 4679-4696.
where m _i is the measurement count for energy bin i and M _b is the number of energy bins.

As a result, arranging the estimated basis coefficient line integrals ai (with upper ring ai) for each projection line into an image matrix gives material-specific projection images (also called basis images) for each basis i. These basis images can be viewed directly (e.g., projection x-ray imaging) or taken as input to a reconstruction algorithm that forms a map of the basis coefficients _ai inside the object (e.g., CT). In either case, the result of the basis decomposition can be viewed as one or more basis image representations, such as line integrals of the basis coefficients or the basis coefficients themselves.

The map of basis coefficients _ai inside an object is called a basis material image, a basis image, a material image, a material-specific image, a material map or a basis map.

However, a well-known limitation of this and other techniques is that the variance of the estimated line integrals typically increases with the number of bases used in the basis decomposition. Among other things, this results in an unfortunate trade-off between improved tissue quantification and increased image noise.

Furthermore, performing accurate basis decompositions using more than two basis functions is difficult in practice and may introduce artifacts, bias or excessive noise. Such basis decompositions may also require extensive calibration measurements and data preprocessing to obtain accurate results.

Since many image reconstruction tasks are inherently complex, artificial intelligence (AI) and machine learning such as deep learning have begun to be used for general image reconstruction, with satisfactory results. However, a current problem with image reconstruction using machine learning such as deep learning is its low explainability. Even if an image appears to have a very low noise level at first glance, it actually contains errors due to the bias of the neural network estimator.

Therefore, there is a need to improve reliability and/or explainability in machine learning image reconstruction, such as deep learning image reconstruction for computed tomography (CT).

Tapsovaara and Wagner, "SNR and DQE analysis of broadspectrum X-ray imaging," Rhys. Med. Biol. 30,519 Alvarez and Macovski, "Energy-selective reconstructions in X-ray computed tomography," Phys. Med. Biol. 21,733 Roessl and Proksa, "K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors," Phys. Med. Biol. 52 (2007), 4679-4696

In general, it is desirable to provide improvements in image reconstruction for x-ray imaging applications.

The object of the present invention is to provide a method for determining a confidence indication of a machine learning image reconstruction, such as a deep learning image reconstruction in computed tomography (CT).

The present invention aims to provide a method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT.

It is also an object of the present invention to provide a system for determining a reliability indication for machine learning image reconstructions, such as deep learning image reconstructions in computed tomography (CT).

Another object is to provide a system for generating uncertainty maps for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT.

Yet another object is to provide a corresponding image reconstruction system.

Yet another object is to provide an overall X-ray imaging system.

It is a further object to provide corresponding computer programs and computer program products.

These and other objectives may be achieved by one or more embodiments of the proposed technology.

The inventors have realized that to be able to trust images resulting from machine learning image reconstruction, such as deep learning image reconstruction, it is highly desirable to quantify the degree of confidence in the reconstructed images (values) or otherwise determine an indication or representation of confidence. This may be particularly important in photon-counting spectral CT, where it is theoretically possible to generate quantitatively accurate maps of material composition, but where high noise levels, especially in three-basis decompositions, may necessitate the use of machine learning image reconstruction, such as deep learning reconstruction methods, as a key component of the image reconstruction chain.

The basic idea of the present invention is to provide radiologists with a confidence indication, such as an uncertainty map or a confidence map, for each image generated by machine learning image reconstruction, such as deep learning image reconstruction.

It will be appreciated that a set of training data, e.g., a measured energy-resolved X-ray data set and a corresponding set of ground truth or reconstructed basis material maps specifically selected for training a machine learning system such as a neural network, can be used to identify or approximate a probability distribution of one or more reconstructed basis material images. Such a distribution prior to the new measurements being evaluated is called a prior distribution. If one or more measurements of a representation of the X-ray image data are further performed, the probability distribution of the possible basis material images with additional knowledge of the measurements is known as a posterior probability distribution.

According to a first aspect, there is provided a method for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT).
acquiring energy resolved x-ray data;
processing the energy-resolved X-ray data based on at least one machine learning system to generate at least one reconstructed basis image or a representation of a posterior probability distribution of image features;
and generating, based on the representation of the posterior probability distribution, at least one reconstructed base image, or at least one derived image derived from the at least one reconstructed base image, or one or more confidence indications for image features of the at least one reconstructed base image or the at least one derived image.

By way of example, the confidence indication may include one or more uncertainty or reliability maps. Such uncertainty or reliability maps may be presented together with associated images or image features in various ways to provide additional useful information to the radiologist.

According to a second aspect, a system for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT) is provided. The system is configured to acquire energy resolved X-ray data. The system is further configured to process the energy resolved X-ray data based on at least one machine learning system to obtain a representation of a posterior probability distribution of at least one reconstructed basis image or image features thereof. The system is also configured to generate one or more confidence indications for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image based on the representation of the posterior probability distribution.

According to a third aspect, there is provided a corresponding image reconstruction system including such a system for determining a confidence indication.

According to a fourth aspect, there is provided an overall X-ray imaging system including such an image reconstruction system.

According to a fifth aspect, a corresponding computer program and computer program product are provided.

In this way, improved reliability and/or explainability in machine learning image reconstruction for computed tomography (CT) may be obtained.

The present embodiment, together with further objects and advantages thereof, will be best understood by reference to the following description taken together with the accompanying drawings.

1 is a schematic diagram showing an example of an entire X-ray imaging device. FIG. 13 is a schematic diagram showing another example of an X-ray imaging system. 1 is a schematic block diagram of a CT system as an example of an X-ray imaging system. FIG. 2 is a schematic diagram illustrating another example of relevant components of an optical imaging system. 1 is a schematic diagram of a photon counting circuit and/or device according to the prior art; 1 is a schematic flow diagram illustrating an example of a method for determining a confidence indication of a machine learning image reconstruction, such as a deep learning image reconstruction in computed tomography (CT). FIG. 1 is a schematic flow diagram illustrating an example of a method for determining a confidence indication for a machine learning image reconstruction, the method additionally including performing an actual image reconstruction in accordance with an exemplary embodiment. FIG. 1 is a schematic flow diagram illustrating an example method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT. FIG. 2 is a schematic diagram illustrating an example of an uncertainty map according to an embodiment. FIG. 1 is a schematic diagram illustrating an example of a Bayesian or probabilistic neural network that can be used to solve a material decomposition task or problem. FIG. 1 is a schematic diagram illustrating an example of a neural network estimator that takes a material concentration map obtained from a deep learning based material decomposition and generates a confidence map. FIG. 1 is a schematic diagram showing a neural network for sinogram spatial material decomposition based on an unrolled iterative material decomposition method. FIG. 1 is a schematic diagram illustrating an example of a neural network that takes an energy bin sinogram as input and generates a reconstructed basis material image. FIG. 1 is a schematic diagram illustrating an example of a probabilistic neural network that takes an energy bin sinogram as input and generates a reconstructed basis material image based on random dropout. FIG. 1 is a schematic diagram illustrating an example of a probabilistic neural network that takes energy bin sinograms as input and generates reconstructed basis material images based on additive noise insertion. FIG. 1 is a schematic diagram of an example of a probabilistic neural network that takes an energy bin sinogram as input and generates a reconstructed basis material image based on a variational autoencoder. The probabilistic neural network is composed of an encoder, a random feature generator, and a decoder. FIG. 1 is a schematic diagram illustrating an example implementation of a computer according to an embodiment.

For a better understanding, it may be useful to continue with an introductory description of a non-limiting example of an overall X-ray imaging system.

2 is a schematic diagram showing an example of an X-ray imaging system 100 composed of an X-ray source 10 that emits X-rays and an X-ray detector system 20 having an X-ray detector that detects the X-rays. FIG. 2 is a schematic diagram showing an example of an X-ray imaging system 100 comprising an X-ray source 10 that emits X-rays, an X-ray detector system 20 having an X-ray detector that detects the X-rays after passing through an object, an analog processing circuit 25 that processes and digitizes raw electrical signals from the X-ray detector, a digital processing circuit 40 that can perform further processing operations on the measured data, such as applying corrections, temporary storage, or filtering, and a computer 50 that can store the processed data and perform further post-processing and/or image reconstruction. According to the present invention, all or part of the analog processing circuit 25 can be implemented in the X-ray detector system 20.

The overall X-ray detector can be considered as the X-ray detector system 20, or the X-ray detector system 20 combined with associated analog processing circuitry 25.

The digital part including the digital processing circuit 40 and/or the computer 50 can be considered as an image processing system 30 that performs image reconstruction based on image data from the X-ray detector. The image processing system 30 may be defined as the computer 50, or the image processing system 35 (digital image processing) may be defined as a combined system of the digital processing circuit 40 and the computer 50, or as the digital processing circuit 40 alone, if the digital processing circuit is further specialized for image processing and/or reconstruction.

One example of a commonly used x-ray imaging system is the x-ray computed tomography (CT) system, which includes an x-ray tube that produces a fan-like cone beam of x-rays and an opposing array of x-ray detectors that measure the percentage of the x-rays that pass through the patient or object. The x-ray tube and detector array are mounted on a gantry that rotates around the object.

3 is a schematic block diagram of a CT system as an example of an X-ray imaging system. The CT system consists of a computer 50 that receives commands and scanning parameters from an operator via an operator console 60, which may have a display and some form of operator interface, such as a keyboard and mouse. The commands and parameters provided by the operator are used by the computer 50 to provide control signals to the X-ray controller 41, the gantry controller 42 and the table controller 43. In particular, the X-ray controller 41 provides power and timing signals to the X-ray source 10, which controls the emission of X-rays to an object or patient lying on the table 12. The gantry controller 42 controls the rotational speed and position of the gantry 11, which comprises the X-ray source 10 and the X-ray detector 20. The table controller 43 controls and determines the position of the patient table 12 and the scan range of the patient. There is also a detector controller 44, which is configured to control and/or receive data from the detector 20.

In one embodiment, computer 50 also performs post-processing and image reconstruction of image data output from the x-ray detector. The computer thereby corresponds to image processing system 30 shown in Figures 1 and 2. An associated display allows an operator to observe the reconstructed image and other data from the computer.

An X-ray source 10 arranged on a gantry 11 emits X-rays. An X-ray detector 20, for example in the form of a photon-counting detector, detects the X-rays after passing through the patient. The X-ray detector 20 is formed by a number of pixels, also called sensors or detector elements, for example, and associated processing circuits, such as ASICs arranged on a detector module. Part of the analog processing is implemented in the pixels, and the remaining processing is implemented in the ASICs for example. In one embodiment, the processing circuit (ASIC) digitizes the analog signals from the pixels. The processing circuit (ASIC) may also constitute a digital processing unit that may perform further processing operations on the measurement data, such as applying corrections, temporary storage, and/or filtering. During a scan to acquire X-ray projection data, the gantry and the components mounted thereon rotate about an iso-center.

Modern X-ray detectors usually require the conversion of incident X-rays into electrons. This is usually done by the photoelectric effect or Compton interaction, and the resulting electrons usually produce secondary visible light until they lose energy, which can be detected by a light-sensitive material. There are also semiconductor-based detectors, where the electrons generated by the X-rays create a charge, an electron-hole pair, which is collected by an applied electric field.

A detector operating in energy integration mode exists in the sense that it provides a signal integrated from a large number of x-rays. The output signal is proportional to the total energy deposited by the detected x-rays.

Photon-counting and energy-resolving X-ray detectors are becoming more common in medical X-ray applications. Photon-counting X-ray detectors essentially measure the energy of each X-ray, which has the advantage of providing additional information about the composition of the subject, which can be used to improve image quality and reduce radiation dose.

Typically, photon counting x-ray detectors determine the energy of a photon by comparing the height of the electrical pulse generated by the interaction of the photon in the detector material to a series of comparator voltages. These comparator voltages are also called energy thresholds. Typically, the analog voltages of the comparators are set by a digital-to-analog converter (DAC), which converts the digital settings sent from the controller into analog voltages against which the height of the photon pulse can be compared.

Photon counting detectors count the number of photons that interact within the detector during a measurement time. A new photon is typically identified by an electrical pulse whose height exceeds the comparator voltage of at least one comparator. Once a photon is identified, the event is stored by incrementing a digital counter associated with the channel.

When using several different thresholds, so-called energy-discriminating photon counting detectors are obtained, which can sort the detected photons into energy bins corresponding to the various thresholds. This type of photon counting detector is sometimes also called a multi-bin detector. In general, the energy information allows to create new kinds of images, where new information is available and image artifacts inherent to the prior art can be eliminated. In other words, in an energy-discriminating photon counting detector, the pulse height is compared to several programmable thresholds (T1-TN) in a comparator and classified according to the pulse height. In other words, a photon counting detector consisting of two or more comparators is called a multi-bin photon counting detector. For a multi-bin photon counting detector, the photon counts are typically stored in a set of counters, one for each energy threshold. For example, a counter can be assigned to correspond to the highest energy threshold crossed by the photon pulse. In another example, a counter records the number of times the photon pulse crosses each energy threshold.

As an example, edge-on is a special, non-limiting design for photon-counting detectors in which the x-ray sensors, such as x-ray detection elements or pixels, are edge-on to the incident x-rays.

For example, such a photon counting detector may have pixels in at least two directions, one of the directions of the edge-on photon counting detector having a component in the direction of the x-rays. Such an edge-on photon counting detector may be referred to as a depth-segment photon counting detector, having pixels in two or more depth segments in the direction of the incident x-rays.

Alternatively, the pixels may be arranged as an array (non-depth-segmented) in a direction substantially perpendicular to the direction of the incident x-rays, and each of the pixels may be oriented edge-on with respect to the incident x-rays. In other words, the photon-counting detector may be non-depth-segmented while still being positioned edge-on with respect to the incident x-rays.

To increase the absorption efficiency, it is also possible to place the edge-on photon counting detector on the edge, in which case the absorption depth can be chosen to be any length, and the edge-on photon counting detector can still be fully depleted without having to be at very high voltages,

The conventional mechanism for detecting X-ray photons directly through a semiconductor detector basically works as follows: the energy of the X-ray interaction in the detector material is converted into electron-hole pairs inside the semiconductor defects, the number of electron-hole pairs being generally proportional to the photon energy. The electrons and holes drift towards the electrodes and backside of the detector (or vice versa). During this drift, the electrons and holes induce a current in the electrodes, and this current can be measured.

As shown in FIG. 4, one or more signals are sent from the detector elements 21 of the X-ray detector to the input, 27, of an analog processing circuit (e.g., an ASIC) 25. It should be understood that the term Application Specific Integrated Circuit (ASIC) is broadly interpreted as a general circuit used and configured for a specific application. The ASIC processes the charge generated from each X-ray and converts it to digital data that can be used to obtain measurement data such as photon count and/or estimated energy. The ASIC is configured to connect to the digital processing circuit so that the digital data is sent to further digital processing circuitry 40 and/or one or more memories 45, and finally the data is input to the image processing circuitry 30 for generating a reconstructed image.

Since the number of electrons and holes arising from an X-ray event is proportional to the energy of the X-ray photon, the total charge contained in an induced current pulse is proportional to this energy. After the filtering step of the ASIC, the pulse amplitude is proportional to the total charge of the current pulse and therefore to the X-ray energy. The amplitude of the pulse can be measured by comparing its value with one or more thresholds (THR) of one or more comparators (COMP), and a counter is introduced that records the number of cases when the pulse is greater than the threshold. In this way, it is possible to count and/or record the number of X-ray photons detected within a certain time frame with energies exceeding the energy corresponding to the respective threshold (THR).

The ASIC typically samples the analog photon pulse once per clock cycle and registers the outputs of the comparators. The comparators (thresholds) output a 1 or 0 depending on whether the analog signal is above or below the comparator voltage. The information available at each sample is, for example, a 1 or 0 for each comparator representing whether the comparator was triggered (the photon pulse was above the threshold) or not.

In a photon counting detector there is usually a photon counting logic that determines if a new photon has been registered and registers the photon in a counter. In the case of a multi-bin photon counting detector there are usually several counters, for example one for each comparator, and the photon count is registered in the counters according to an estimate of the photon energy. The logic can be implemented in several different ways. The two most common categories of photon counting logic are the so-called non-paralyzable counting modes and paralyzable counting modes. Other photon counting logic includes, for example, local maxima detection, which counts detected local maxima in a voltage pulse and possibly also registers the height of that pulse.

Photon counting detectors have many advantages, including but not limited to high spatial resolution, low electronic noise, energy resolution, material separation capability, and spectral imaging ability. However, energy integrating detectors have the advantage of high count-rate tolerance. Count-rate tolerance comes from the fact/realization that since the total energy of the photons is measured, adding one additional photon will always increase the output signal (within reasonable limits), regardless of the amount of photons currently registered by the detector. This crucial advantage is one of the main reasons why energy integrating detectors are the standard in medical CT today.

For a better understanding, it may be useful to start with a brief system overview and/or analysis of some technical issues. To this end, reference is made to FIG. 5, which provides a schematic diagram of a photon counting circuit and/or device according to the prior art.

When photons interact in the semiconductor material, a cloud of electron-hole pairs is generated. When an electric field is applied to the detector material, the charge carriers are collected at electrodes attached to the detector material. A signal is sent from the detector element to the input of an analog processing circuit (such as an ASIC). It should be understood that the term application specific integrated circuit (ASIC) is to be broadly interpreted as a general circuit that is used and configured for a specific application. The ASIC can be used to process the charge generated from each x-ray, convert it to digital data, and obtain measurement data such as photon count and/or estimated energy, in one example, the ASIC can process the charge such that a voltage pulse is generated having a maximum height proportional to the amount of energy deposited by the photon in the detector material.

The ASIC may include a set of comparators 302, each of which compares the magnitude of the voltage pulse with a reference voltage. The comparator output is typically either zero or one (0/1) depending on which of the two voltages compared is greater. We assume here that the comparator output is one if the voltage pulse is higher than the reference voltage, and zero if the reference voltage is higher than the voltage pulse. A digital-to-analog converter, DAC, 301 may be used to convert a digital setting, which may be supplied by a user or a control program, into a reference voltage that may be used by the comparators 302. If the height of the voltage pulse exceeds the reference voltage for a particular comparator, we will refer to that comparator as triggered. Each comparator is typically associated with a digital counter 303, which is incremented based on the comparator output according to photon counting logic.

In general, basis material decomposition takes advantage of the fact that all materials composed of elements with low atomic numbers, such as human tissue, have a linear attenuation coefficient (E) whose energy dependence can be approximately expressed as a linear combination of two (or more) basis functions.

where _f1 and _f2 are basis functions and _a1 and _a2 are corresponding basis coefficients. More generally, f _i are basis functions and a _i are corresponding basis coefficients. If there are one or more elements in the imaging volume with a sufficiently high atomic number that their k-edges lie in the energy range used for imaging, then one basis function must be added for each such element. In the field of medical imaging, such k-edge elements are typically iodine or gadolinium, which are substances used as contrast agents.

As mentioned above, the line integral A _i of each basis coefficient a ₁ is estimated from the measurement data of each projection ray _l from the source to the detector element. The line integral A _i can be expressed as:

where N is the number of basis functions. In one embodiment, decomposition of the basis material is accomplished by first expressing the expected registration number of counts in each energy bin as a function of A _i . Typically, such a function can take the form:

where _λi is the expected count in energy bin t, E is the energy, and s _i is a response function that depends on the spectral shape incident on the imaged object, the quantum efficiency of the detector, and the sensitivity of energy bin i to x-rays of energy if. Although the term "energy bin" is most commonly used for photon counting detectors, this formula can also describe other energy resolving x-ray systems, such as multi-layer detectors and kVp switching sources.

Maximum likelihood can then be used to estimate A _i under the assumption that the counts in each bin are Poisson distributed random variables. This is accomplished by minimizing the negative log-likelihood function. See Roessl and Proksa, "K-edge imaging in X-ray computed tomography using multi-bin photon counting detectors," Phys. Biol. 52 (2007), 4679-4696.

where m _i is the measurement count for energy bin i and M b is the number of energy bins.

From the line integrals A, a tomographic reconstruction can be performed to obtain the basis coefficients _ai . This step can be viewed as a separate tomographic reconstruction or as part of the overall basis decomposition.

As mentioned above, the estimated basis coefficient line _integrals for each projection line are placed in an image matrix, and the result is a material-specific projection image (also called a basis image) for each basis i. This basis image can be viewed directly (e.g., in projection X-ray imaging) or can be taken as input to a reconstruction algorithm to form a map of the basis coefficients a _i inside the object (e.g., in CT). In either case, the result of the basis decomposition can be viewed as one or more basis image representations, such as the line integrals of the basis coefficients or the basis coefficients themselves.

In the field of X-ray imaging, the representation of image data consists, for example, of sinograms, projection images or reconstructed CT images. Such a representation of image data may be energy resolved if it consists of multiple channels, so-called multi-channel or multi-bin energy information, where the data in different channels relate to the measured X-ray data in different energy intervals.

Through a process of material decomposition that takes as input a representation of the energy-resolved x-ray image data, a set of basis image representations can be generated. Such a set is a collection of multiple basis image representations, each basis image representation being associated with the contribution of a particular basis function to the total x-ray attenuation. Such a set of basis image representations may be a set of basis sinograms, a set of reconstructed basis CT images, or a set of projection images. It will be understood that "image" in this context may mean, for example, a two-dimensional image, a three-dimensional image, or a time-resolved image sequence.

For example, a representation of energy resolved x-ray image data can consist of a collection of energy bin sinograms, each containing the counts measured in one energy bin. This collection of energy bin sinograms can be input to a material decomposition algorithm to generate a set of basis sinograms. These basis sinograms can be taken as input to a reconstruction algorithm to generate a reconstructed basis image.

In a two-basis decomposition, a two-basis image representation is generated. This is based on the approximation that the attenuation of any material in the imaged object can be expressed as a linear combination on two basis functions. In a three-basis decomposition, a three-basis image representation is generated, based on the approximation that the attenuation of any material in the imaged object can be expressed as a linear combination on three basis images. Similarly, four-basis decompositions, five-basis decompositions, and similar higher order decompositions can be defined. It is also possible to perform a one-basis decomposition by approximating all materials in the imaged object as having X-ray attenuation coefficients with similar energy dependence up to the density scale factor.

A two-basis decomposition results in a set of basis sinograms including, for example, water sinograms and iodine sinograms, with basis functions corresponding to the linear attenuation coefficients of water and iodine, respectively. Alternatively, the basis functions may represent the attenuation of water and calcium, calcium and iodine, polyvinyl chloride and polyethylene. A three-basis decomposition results in a set of basis sinograms including, for example, water sinograms, calcium sinograms and iodine sinograms. Alternatively, the basis functions represent the attenuation of water, iodine and gadolinium, or polyvinyl chloride, polyethylene and iodine.

As mentioned above, artificial intelligence (AI) and machine learning such as deep learning are beginning to be used for general image reconstruction, and have achieved some satisfactory results. However, the current problem with image reconstruction using machine learning, such as deep learning, is its limited empirical nature. Even if an image appears to have little noise at first glance, it actually contains errors due to the bias of the neural network estimator.

Deep learning generally refers to a machine learning method using representation learning based on artificial neural networks and similar architectures. Learning can be supervised, semi-supervised or unsupervised. Deep learning systems such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied to a variety of technology areas, including computer vision, speech recognition, natural language processing, social network filtering, machine translation and board game programs, producing results that match and in some cases exceed the performance of human experts.

The adjective "deep" in deep learning comes from the use of multiple layers in the network. Early work showed that while a linear perceptron cannot be a universal classifier, a network with a single hidden layer with a non-polynomial activation function and unbounded width can. Deep learning is the latest variation, with a theoretically unlimited number of layers and finite size, allowing practical applications and optimized implementations while still preserving theoretical universality under mild conditions. In deep learning, layers are heterogeneous and allowed to deviate significantly from biologically informed connectionist models for efficiency, ease of training, and ease of understanding.

The inventors have recognized a need for improving reliability and/or explainability in machine learning image reconstruction, such as deep learning image reconstruction, particularly in computed tomography (CT).

The proposed technique is generally applicable to provide an indication of the reliability of reconstructed images and/or image features based on machine learning such as neural networks and deep learning.

As mentioned above, the inventors have realized that in order to be able to trust images (as described above) resulting from machine learning image reconstruction, such as deep learning image reconstruction, it is highly desirable to quantify the degree of confidence in the reconstructed image (values) or otherwise determine an indication or expression of confidence. This may be particularly important in photon counting spectral CT, where while it is theoretically possible to generate quantitatively accurate maps of material composition, the high noise levels, especially in the 3-basis decomposition, mean that machine learning, such as deep learning image reconstruction, must or should be used as a key component of the image reconstruction chain.

In a sense, the basic idea of the present invention is to provide radiologists with a reliability indication, such as an uncertainty map, for each image or image feature generated by a machine learning image reconstruction, such as a deep learning image reconstruction.

According to a first main aspect, a non-limiting example of a method for determining a confidence indication for machine learning image reconstruction, such as deep learning image reconstruction in computed tomography (CT), is provided.

FIG. 6A is a schematic flow diagram illustrating an example method for determining a confidence indication for a machine learning image reconstruction, such as a deep learning image reconstruction in computed tomography (CT).

Essentially, the method includes the following steps:
Energy resolved X-ray data is acquired (S1).
The energy-resolved x-ray data is processed based on at least one machine learning system, such as a neural network, to generate at least one reconstructed basis image or a representation of the posterior probability distribution of image features (S2).
Based on the representation of the posterior probability distribution, one or more confidence indications are generated for the at least one reconstructed image, or at least one derived image derived from the at least one reconstructed base image, or image features of the at least one reconstructed base image or the at least one derived image (S3).

In other words, this can be expressed as processing the energy-resolved x-ray data based on at least one neural network or similar machine learning system to obtain a representation of at least one posterior probability distribution of at least one basis image or its image features. This representation can then be processed to form a confidence indication of one or more images or image features.

It will be appreciated that a set of training data, e.g. a measured energy resolved X-ray data set and a corresponding set of ground truth or reconstructed basis material maps specifically selected for training a machine learning system such as a neural network, can be used to identify or approximate a probability distribution of one or more reconstructed basis material images. Such a distribution prior to the new measurements being evaluated is called a prior distribution. If one or more measurements of a representation of the X-ray image data are further performed, the probability distribution of the possible basis material images with additional knowledge of the measurements is known as a posterior probability distribution.

In other words, prior information about what a CT image is likely to look like is typically specified by a training dataset that includes a set of training input image data and output image data. Such input image data and output image data can take the form of images with different contents, such as sinograms or bin images or sinograms or basis images or sinograms. By training a mapping to map the input data in each pair to output image data that is as similar as possible to the corresponding output image data in the input/output training pair, a mapping is obtained that can perform noise removal, decomposition into basis images, or image reconstruction from the measured image data. The training output image data in each pair is also called a label. In a preferred embodiment, such a mapping can take the form of a convolutional neural network (CNN), but there are other embodiments, such as support vector machines or decision trees, that can realize/configure this mapping. In order to find a mapping that gives the best match between the network output and the training output image data, a data discrepancy function, also called a loss function, is usually used to calculate the data discrepancy between the network output and the training output image data. In a preferred embodiment of the present invention, said mapping can be probabilistic, meaning that it gives different outputs when applied multiple times to the same input data. In this embodiment, the loss function can take the form of, for example, the Kullback-Leibler distance or Wasserstein distance between the distribution of output image data generated by the network and the distribution of training output image data.

As an example, a convolutional neural network is trained by minimizing this data mismatch using an optimization technique such as ADAM. Once the mapping is trained, it can be applied at run time by mapping measured image data to generate output image data. For example, the probabilistic mapping can be applied to the input image data multiple times to generate an ensemble of output image data. The mean and standard deviation of the output image data can be calculated over this ensemble, and the mean output image can be used as an estimate of the denoised, decomposed or reconstructed image, and the standard deviation can be used as an estimate of the uncertainty of the denoised, decomposed or reconstructed image.

In another embodiment of the invention, two separate neural networks are used, one network trained to generate an estimate of the output image data, e.g., the reconstructed basis image, and a second network trained to generate an estimate of the uncertainty of the output image data, e.g., a map of the uncertainty of the reconstructed basis image. For example, one way to train such networks is to first train a single probabilistic neural network to generate samples from the posterior distribution of the output image data as described above, and then train two neural networks to predict the mean and standard deviation of the posterior distribution.

In yet another embodiment of the invention, a network can be directly trained to predict the mean and standard deviation of the output image data. This is achieved by assuming an output probability distribution parameterized by the mean and standard deviation, and minimizing a data discrepancy measure, such as the Lullback-Leibler diference or the Wasserstein diference, between the output probability distribution with parameters predicted by the network and an approximation of the posterior distribution of the output image data based on the training dataset.

In yet another embodiment of the invention, a neural network estimator implemented according to one of the above methods can be trained to predict the uncertainty of a non-neural network based CT data processing method, e.g. a reconstruction, decomposition or denoising method. To this end, the uncertainty of said CT data processing method can be predicted by repeatedly applying said method to noisy data, e.g. simulated or measured data, and a neural network can be trained to predict such noisy data.

In an exemplary embodiment of the present invention, the energy resolved X-ray data is acquired using a photon counting X-ray detector or from an intermediate memory that stores the energy resolved X-ray data.

The fact that energy-resolved X-ray data are used means that multi-channel energy information is employed. Furthermore, the consideration of one or more basis images, also called basis material images or material-specific images or material-selected images, means that multiple materials (i.e. at least two basis materials) are involved in the overall analysis. This leads to a higher dimensional context.

The confidence indication may be any suitable indication of the confidence of the image(s) or image feature(s) ultimately reconstructed by the machine learning image reconstruction, such as deep learning image reconstruction, and may, for example, be a related quantification of the confidence and/or trust of the reconstructed image(s) and/or image feature(s). The confidence indication may also be a complex representation of the confidence, such as an uncertainty map, as exemplified in more detail below.

One example of a confidence indication is a map showing the uncertainty of the basis image estimate, e.g., the standard deviation of the estimated iodine concentration. This results in an image that highlights areas of high iodine concentration uncertainty in the reconstructed iodine basis image.

Another example of a confidence indication is a confidence map that indicates the confidence that a substance is present at various locations. As an example, such a map may highlight areas in the image where iodine is definitely present, while leaving dark areas where there is a high degree of confidence that iodine is not present. Such a confidence map may be calculated, for example, by dividing the estimated iodine concentration by the estimated standard deviation of the iodine concentration. In another example, such a map may be calculated by calculating the posterior probability that the map contains iodine at a particular location. Yet another example of a confidence indication is a confidence interval for the concentration of one or more basis substances at each point in the image.

For example, the machine learning image reconstruction is deep learning image reconstruction, and the at least one machine learning system includes at least one neural network.

In certain examples, the representation of the posterior probability distribution includes at least one of a mean variance, a covariance, a standard deviation, a skewness, and a kurtosis.

Optionally, the one or more confidence indications may include an error estimate or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image, and/or an error estimate or a measure of statistical uncertainty for at least one image measurement derivable from the at least one reconstructed basis image.

For example, the error estimate or statistical uncertainty measure can include at least one of an upper bound for an error, a lower bound for an error, a standard deviation, a variance or a mean absolute error.

As an example, the at least one image measurement can include at least one of a dimensional measure of a feature, an area, a volume, a degree of inhomogeneity, a measure of shape or irregularity, a measure of composition, and a measure of concentration of a substance.

As illustrated below, the one or more confidence indications may include one or more uncertainty maps for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features thereof.

In a particular embodiment, the step S3 of generating one or more confidence indications includes generating a confidence map of the reconstructed material-selected X-ray image for computed tomography (CT).

As an example, a confidence map can be generated to highlight portions of the reconstructed material-selected X-ray image that the machine learning image reconstruction was able to determine with a confidence level above a predefined threshold, i.e., with high confidence.

For example, step S3 of generating one or more confidence indications may include generating one or more confidence maps using a neural network that receives as input a material concentration map obtained from the deep learning-based material decomposition.

In a particular embodiment, as shown diagrammatically in FIG. 6B, the method further includes performing S2a material decomposition based image reconstruction and/or machine learning image reconstruction to generate the at least one reconstructed basis image or image features thereof based on the acquired energy resolved x-ray data.

As an example, step S2a of performing material decomposition-based image reconstruction and/or machine learning image reconstruction may include generating the at least one reconstructed basis image or image feature by a neural network that receives as input an energy bin sinogram.

In an optional embodiment, the step S3 of generating one or more confidence indices may include determining an uncertainty or confidence map for each basis material image and a covariance between different basis material images. A formula or algorithm for uncertainty propagation may then be used to propagate the uncertainty or confidence map to generate an uncertainty map for the derived image.

In certain embodiments, the at least one basis material image may be generated along with at least one uncertainty map, the uncertainty map being a representation of the uncertainty or error estimate of the at least one basis material image, and the at least one basis material image and the at least one uncertainty map may be presented to a user as separate images/maps or in combination.

For example, the at least one uncertainty map may be presented as an overlay on the at least one base material image, or the at least one uncertainty map may be presented by a distorting filter on the at least one base material image,

As an example, step S2 (FIG. 6A) or S2b (FIG. 6B) of processing the energy resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution comprises generating samples of the posterior probability distribution given the acquired energy resolved X-ray data by a neural network, and step S3 of generating one or more confidence indications comprises generating an uncertainty map as a standard deviation over the samples.

In any embodiment, step S2 or S2b of processing the energy-resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution comprises applying a neural network implemented as a variational autoencoder to encode the input data vectors into parameters of a probability distribution of the latent random variables and extracting from this probability distribution a collection of posterior samples of the latent random variables for (subsequent) processing by a corresponding decoder to obtain posterior observations.

In certain embodiments, the step S3 of generating one or more confidence indications comprises generating at least one map of the covariance or correlation coefficient of at least one basis coefficient and/or at least one map of the covariance or correlation coefficient of at least one set of basis functions associated with the at least one reconstructed basis image.

In an exemplary embodiment of the present invention, the representation of the posterior probability distribution is defined by the mean and variance of multiple image features.

In an exemplary embodiment of the invention, the representation of the posterior probability distribution can be given by a number of Monte Carlo samples from the distribution.

In an exemplary embodiment of the present invention, the neural network is a convolutional neural network (CNN) having at least five layers.

In an exemplary embodiment of the present invention, the neural network-based processing may include processing using a probabilistic neural network.

In an exemplary embodiment of the present invention, the neural network is configured to operate based on random dropout, noise insertion, a variational autoencoder or noisy stochastic gradient de-scent.

In an exemplary embodiment of the present invention, the neural network-based processing may include processing with a deterministic neural network that provides a measure of the posterior probability distribution.

In an exemplary embodiment of the present invention, the neural network based processing may include processing with a deterministic neural network that provides a measure of uncertainty of a reconstructed image or image feature.

In an exemplary embodiment of the present invention, the neural network-based processing is based on a neural network based on one or more inputs calculated from at least one physical model of data acquisition.

In an exemplary embodiment of the invention, the at least one input calculated from a physical model of data acquisition is a gradient of a data discrepancy function, an estimate of a scattered photon distribution, or a representation of cross-talk between detector pixels, or a representation of pile-up.

In an exemplary embodiment of the present invention, the process is based on a neural network that includes an unrolled optimization neural network architecture.

In an exemplary embodiment of the invention, the process is based on a neural network that takes as input at least one standard deviation, variance or covariance map in image space or sinogram space based on the Cramer-Rao lower bound.

In an exemplary embodiment of the invention, the process may be based on a neural network that performs the following steps:
At least one representation of energy resolved X-ray image data
performing at least two basis material decompositions on the image representation, resulting in at least two sets of original basis image representations;
obtaining or selecting at least two basis image representations from at least two of the original sets of basis image representations;
Processing the obtained or selected basis image representations with the neural network based data processing to result in a representation of a posterior probability distribution over a set of basis image representations.

In an exemplary embodiment of the invention, the process is based on a neural network that is trained by minimizing a loss function that is calculated as a disparity measure in image or sinogram space between the labels, i.e., given outputs corresponding to the network inputs in the training set, and the network outputs.

In an exemplary embodiment of the present invention, the loss function is based on weighted mean squared error, Kullback-Leibler distance, or Wasserstein distance.

In an exemplary embodiment of the present invention, the loss function incorporates at least two different basis material components incorporated with different weighting factors.

In an exemplary embodiment of the present invention, the loss function is calculated based on a set of basis coefficients in a basis transformed relative to the original basis.

In an exemplary embodiment of the present invention, the algorithm is trained on image data that is generated with deliberately introduced model error. This makes the neural network estimator more robust to model error and model uncertainty. This approach also allows the image uncertainty due to unknown model error to be incorporated into the probabilistic neural network.

According to a complementary aspect, a non-limiting example of a method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT, is provided.

Figure 7 is a schematic flow diagram illustrating an example of a method for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT.

The method includes the following steps.
The method includes the steps of: acquiring (S11) energy resolved X-ray data; processing (S12) said energy resolved X-ray data based on at least one neural network to obtain a representation of a posterior probability distribution for at least one reconstructed basis image or image feature thereof; generating (S13) one or more uncertainty maps for at least one reconstructed image, or derived image, or image feature thereof based on said representation of a posterior probability distribution.

In an exemplary embodiment of the invention, generating one or more uncertainty maps includes generating at least one map of variance or standard deviation of at least one basis coefficient and/or at least one map of covariance or correlation coefficient of at least one set of basis functions.

As an example, the energy resolved X-ray data is obtained from, or by, a photon counting X-ray detector, or from an intermediate memory that stores the energy resolved X-ray data.

In an exemplary embodiment of the present invention, the neural network is a convolutional neural network (CNN) having at least five layers.

In an exemplary embodiment of the present invention, the representation of the posterior probability distribution is defined by the mean and variance of multiple image features.

In an exemplary embodiment of the invention, the representation of the posterior probability distribution can be given by a number of Monte Carlo samples from the distribution.

In an exemplary embodiment of the present invention, the neural network-based processing may include processing using a stochastic neural network.

In an exemplary embodiment of the present invention, the neural network is configured to operate based on random dropout, noise insertion, a variational autoencoder or noisy stochastic gradient de-scent.

In an exemplary embodiment of the present invention, the neural network-based processing may include processing with a deterministic neural network that provides a measure of probability distribution.

In an exemplary embodiment of the invention, the neural network based processing may include processing with a deterministic neural network that provides a measure of uncertainty in the reconstructed image or image features.

In an exemplary embodiment of the present invention, the neural network-based processing is based on a neural network based on one or more inputs calculated from at least one physical model of data acquisition.

In an exemplary embodiment of the invention, the at least one input calculated from a physical model of data acquisition is a gradient of a data discrepancy function, an estimate of a scattered photon distribution, or a representation of cross-talk between detector pixels, or a representation of pile-up.

In an exemplary embodiment of the present invention, the process is based on a neural network that includes an unrolled optimization neural network architecture.

In an exemplary embodiment of the invention, the process is based on a neural network that takes as input at least one standard deviation, variance or covariance map in image space or sinogram space based on the Cramer-Rao lower bound.

In an exemplary embodiment of the invention, the process is based on a neural network that is trained by minimizing a loss function that is calculated as a discrepancy measure in image space or sinogram space between the labels and the network output.

In an exemplary embodiment of the present invention, the loss function is based on a weighted mean square error in which at least two different basis material components are incorporated with different weighting factors.

To provide an illustrative framework to facilitate understanding of the proposed technique, we next present a concrete example of deep learning-based image reconstruction in the specific context of CT image reconstruction.

However, it should be understood that the proposed technique for providing an indication of the reliability of deep learning image reconstruction in CT applications is generally applicable to deep learning based image reconstruction for CT and is not limited to the following specific example of deep learning based image reconstruction.

As an example, the disclosed invention can provide a confidence map of a reconstructed material-selective X-ray CT image. Such a confidence map can highlight parts of the image that the reconstruction algorithm was able to determine with high confidence.

In particular, such images can provide images of the distribution of contrast agents such as iodine. It is understood that quantifying iodine in a 3-basis decomposition is very sensitive to noise and therefore reconstruction algorithms such as deep learning algorithms may need to make heavy use of prior information to obtain this image. A confidence map for iodine concentration is therefore useful to enable an observer such as a radiologist to interpret the image, as for example shown diagrammatically in FIG. 8.

Furthermore, it is understood that the noise in the decomposed basis images and sinograms is usually highly correlated between different basis material images. Furthermore, it is understood that features of one basis image, such as an area containing iodine, may appear as an artifact in another basis image, such as a bone basis image, if the image reconstruction algorithm is imperfect. Therefore, it is important to predict not only the uncertainty of the individual material maps, but also the covariance between different material maps. This allows the uncertainty propagation formula or algorithm to be used to propagate the confidence map and obtain a derived image, such as an uncertainty map of a virtual non-contrast image, a virtual non-calcium image, a virtual monoenergetic image, or a syn-thetic Hounsfield unit image.

In a non-limiting embodiment of the disclosed invention, a method is provided for generating at least one basis material image together with at least one uncertainty map, the uncertainty map being a representation of the uncertainty or error estimate of the basis material image. Such at least one basis material image together with the at least one uncertainty map can be presented to a user as separate images or in combination, for example as a color overlay. Another possibility is that the at least one uncertainty map can be presented in the form of a distortion filter of the basis material map, for example by a blur filter.

Figure 9 shows a non-limiting example of a probabilistic neural network that generates an average material image along with a variance map or uncertainty map.

Figure 10 shows a deep neural network that maps material concentration maps to uncertainty maps. Such maps can be presented together with the underlying material maps or separately (Bone, Soft Tissue, Iodine).

It is also understood that to generate highly accurate CT images, detectors with good energy resolution, such as photon counting detectors, are required. It is also understood that to generate highly accurate CT images from energy resolved measurement data, accurate physical models are beneficial. Such physical models can be incorporated into deep learning image processing or reconstruction algorithms, for example by unrolling an iterative optimization loop.

FIG. 11 illustrates an exemplary embodiment of such an unrolled iterative loop for sinogram space basis material decomposition. Thereby, an input sinogram is processed through a series of neural network blocks, each of which may include one or more neural network layers. As previously mentioned, an energy sinogram may include, for example, the counts measured in a particular energy bin. Each block fakes as input the output from the preceding block along with an estimate of the gradient of an objective function, for example a likelihood function or log-likelihood function. In one embodiment of the present invention, the likelihood function may be a likelihood function for detecting a particular combination of measurement counts given an estimate of the basis material sinogram. In a preferred embodiment, the layers of the neural network may be layers of a convolutional neural network, i.e., convolutional layers. In a preferred embodiment, the neural network is implemented using a graphics processing unit (GPU). It should be understood that this is a non-limiting example, and that projection and backprojection operations within the neural network allow the network to convert material images or energy bin count images to material images, or convert bin counts or material sinograms to material images.

Apart from the unrolled gradient descent algorithm mentioned above, other iterative algorithms can be unrolled, such as a Newton method, a conjugate gradient method, a Nesterov- accelerated method or a pri-mal-dual method, resulting in different network architectures or different functions of the image estimate as input to each network layer.

In an exemplary embodiment of the invention, the neural network architecture can be based on physics modes based on models of the focal spot shape, the x-ray spectrum shape, charge sharing, scatter in the patient, scatter inside the detector or pile-up! or material decomposition methods that take into account correction terms. These allow different functions to be applied to the estimated output at one or more steps in the neural network.

The combination of photon-counting detectors and careful physics modeling allows the generation of highly accurate photon-counting images. It is appreciated that the benefit of this high accuracy can be enhanced by providing reliable error estimation. The proposed technique is based on the finding that the combined use of spectral CT and neural network-based error estimation allows the generation of highly accurate quantitative images together with error estimation. It is further appreciated that both the image estimation and the uncertainty estimation can be further improved by incorporating at least one model of the physics of image acquisition.

As an example, a method for generating an uncertainty map is disclosed. A probabilistic neural network, such as a Bayesian neural network, can be used to generate samples from a posterior probability distribution of one or more images given observed/measured data and a training set. As an example, the training set can include a set of input-output pairs, where each training input is a set of bin sinograms and the training output (labels) is a set of basis images. Such pairs can be generated, for example, through simulation of CT imaging of a numerical phantom or through measurement of physical phantoms with known composition. In another example, such training pairs are generated by CT imaging of a patient, where the training output is obtained as a reconstructed image and the training input can be obtained as a measured sinogram, as a modified sinogram with extra noise added, or as a sinogram from another CT imaging of the same object acquired at a lower dose. By using training inputs with increased noise compared to the training output, the resulting trained neural network can achieve the ability to reduce noise, and in another embodiment of the present invention, the input data can be a set of basis sinograms, a set of reconstructed bin images, or a set of basis images, and in yet another embodiment of the present invention, the input data can be a set of basis sinograms, a set of reconstructed bin images, or a set of basis images, and in this way, neural networks can be constructed that operate in either the sinogram domain or the image domain and perform basis decomposition or denoising of the basis or bin images or sinograms.

Once trained, the neural network is ready to process the observed/measured data to generate, during "run-time" in, for example, a clinical setting, a reliability representation such as an uncertainty map or confidence map for each image considered. This kind of neural network provides the network with outputs that are random variables that depend on the inputs. By feeding the same data to this network, the posterior probability distribution can be sampled. For example, an uncertainty map can be generated as the standard deviation of a large number of such samples.

Such neural networks generating the uncertainty map of the basis material images need to be specially designed for processing multi-energy channel images and/or sinogram data. In particular, such neural networks can take as input at least two representations of energy resolved measurement data, e.g. two energy bin images or two pre-resolved basis material images. Such neural networks can also generate at least one uncertainty map for at least one basis material image. Such neural networks can process different material maps jointly, separately, or separately for some layers and then jointly for at least one layer.

It will be appreciated that the neural network estimator for generating the basis material images or the uncertainty map, or both, may incorporate a smoothing filter having an adjustable filter size to adjust the spatial resolution of the resulting images. Such an adjustable filter or filters may take the form of Gaussian smoothing, for example, in at least one layer. The neural network may be trained to generate a series of images having varying resolution characteristics when one or more parameters of said adjustable filter or filters are varied. Once trained, the neural network may be used to generate images of varying resolution by adjusting at least one parameter of said adjustable filter. Such adjustable filters may be applied with different filter characteristics, e.g., kernel sizes, to different basis material images to achieve a desired spatial resolution and noise characteristics in each material image.

A Bayesian neural network can be trained by minimizing the discrepancy between the output distribution given training input images and the distribution of training output images (also called training labels). Such discrepancy can be measured by a mean squared error, a Kullback-Leibler divergence or a Wasserstein distance. It should be understood that the concepts of "training input images" and "training output images" are non-limiting and refer to representations of image data, which can be, for example, bin images or sinograms, or basis images or sinograms.

Such a probabilistic neural network can be based, for example, on random dropout, where the network connects with a probability that can be fixed or learned from data (Figure 13). In another embodiment of the invention, the probabilistic neural network can be based on additive noise insertion after at least one network layer (Figure 14). This additive noise insertion can also be replaced by multiplicative noise insertion or other types of noise insertion.

In another embodiment of the present invention, the probabilistic neural network is implemented as a variational autoencoder (Figure 15). This variational autoencoder first encodes the data input vector into parameters of a probability distribution of the latent random variables, such as normal distribution parameters. A set of posterior samples of the latent variables are then drawn from the latent distribution and processed by a decoder to obtain posterior observations of the outcome. These posterior observations are used to compute the posterior mean (the final outcome) and posterior variance (an uncertainty map of the outcome).

In particular, the discrepancy or loss function used to train the neural network is optimally adapted to the context of spectral CT material decomposition, by treating different basis components differently, reflecting their different noise levels and potentially different clinical importance. As an example, the basis images may be transformed by basis modification before the data discrepancy is calculated. For example, the data discrepancy may be calculated by comparing the basis images from the training set with the basis projections generated as output from the network, and in another example, the data discrepancy may be calculated by transforming the basis images into a set of monoenergetic images and comparing these between the training set and the network output. Depending on which type of image is used to calculate the data discrepancy, the performance of the neural network denoising method may be optimized for the type of image that is of interest to show to the end user. In another example, the mathematical function that calculates the data discrepancy may weight different linear combinations of the basis images differently to obtain greater noise suppression in types of images where low noise is more important compared to types of images where unbiasedness is more important. For example, to accurately assess the material composition of a sample, it may be preferable to minimize noise in the 70 keV monochromatic images, while achieving unbiasedness in the effective atomic number map is more important.

As part of or in addition to generating the uncertainty map, the disclosed methods can be used to generate uncertainty estimates for one or more derived image features, e.g., radiological image features. Examples of such features include the volume of a lesion, the average density of a region, the average effective atomic number of a region, or the standard deviation or another measure of heterogeneity across a region. To generate error estimates for such derived features, a probabilistic neural network can be used to generate a set of image realizations. The uncertainty of the feature can then be determined, for example, as the standard deviation of these realizations.

For example, performing an accurate basis decomposition using more than two basis functions can be difficult in practice and can introduce artifacts, bias, and excessive noise. Also, such basis decompositions may require extensive calibration measurements and data preprocessing to obtain accurate results. In general, a basis decomposition into a larger number of basis functions can be technically more challenging than a decomposition into a smaller number of basis functions.

For example, it may be difficult to calibrate accurately enough to produce a 3-basis decomposition with less image bias and artifacts compared to a 2-basis decomposition. Also, it may be difficult to find a material decomposition algorithm that can perform a 3-basis decomposition on noisy data without producing overly noisy basis images. That is, it may be difficult to achieve the theoretical lower bound on the basis image noise given by the Cramer-Rao lower bound, whereas it may be easier to achieve this lower bound when performing a 2-basis decomposition.

As an example, the amount of information required to generate a larger number of basis image representations may be possible to extract from a set of multiple basis image representations, each having a smaller number of basis image representations. For example, the information required to generate a three basis decomposition into a sinogram of water, calcium, and iodine may be possible to extract from a set of three two-basis decompositions (water-calcium decomposition, water-iodine decomposition, calcium-iodine decomposition).

It may be easier to accurately perform multiple 2-basis decompositions than to accurately perform a single 3-basis decomposition. This observation can be used, for example, to solve the problem of performing an accurate 3-basis decomposition. As an example, the energy-resolved image data is first used to perform water-calcium, water-iodine, and calcium-iodine decompositions. A convolutional neural network can then be used to map the resulting six basis images, or a subset thereof, to a set of three output images including water, calcium, and iodine images. Such a network can be trained with multiple 2-basis image representation sets as input data and 3-basis image representation sets as output data, the 2-basis and 3-basis image representation sets being generated from measured patient or phantom image data, or from simulated image data based on a numerical phantom.

The above method can significantly reduce bias, artifacts, or noise in the 3-basis image representation set compared to a 3-basis decomposition performed directly on the energy-resolved image data. Alternatively, higher resolution images can be generated.

As an alternative or complement to neural networks, the machine teaming system or method applied to the original basis images can include another machine learning system or method, such as a support vector machine or a decision tree-based system or method.

The basis material decomposition step used to generate the original basis image representation can include prior information, such as volume or mass preservation constraints or nonnegativity constraints. Alternatively, such prior information can take the form of a prior image representation, such as an image from a previous examination or a reconstructed image from the aggregate counts of all energy bins, and the algorithm can penalize deviations of the decomposed basis image representation from this prior image representation. Another option is to use prior information learned, for example from a training image set represented as a trained dictionary or a pretrained convolutional neural network, or a trained subspace, i.e. a subspace of the vector space of possible images in which the reconstructed image is expected to reside.

Material decomposition can be performed on the projection image data or sinogram data, for example, by processing each measured projection ray independently. This processing can take the form of a maximum likelihood decomposition, or a maximum a posteriori decomposition, where a prior probability distribution on the material composition in the imaged object is assumed. It can also take the form of a linear or affine transformation from the set of input counts to the set of output counts, a low-order polynomial approximation, e.g., as exemplarily described by Lee et al. (IEEE Transactions on Medical Imaging (Volume: 36, Issue: 2, Feb. 2017: 560-573)), a neural network estimator, or a look-up table, as exemplarily described by Alvarez (https://arxiv.org/abs/1702.01006). Alternatively, the material decomposition method may process multiple rays jointly, and may constitute a one-stage or two-stage reconstruction algorithm.

A paper by Chen and Li in Optical Engineering 58(1), 013104, describes a method for multi-material decomposition of spectral CT data using deep neural networks.

A paper by Poirot et al. published in Scientific Reports, Vol. 9, Paper No. 17709 (2019) discloses a method for generating non-contrast single-energy CT images from dual-energy CT images using a convolutional neural network.

Figure 8 is a schematic diagram illustrating an example of an uncertainty map according to an embodiment. Whereas an iodine map provides an estimate of iodine concentration, an iodine confidence map provides an indication of the confidence with which the algorithm can predict that iodine is present at each given pixel of the image. The resulting image highlights areas where iodine is highly likely to be present, and darker areas are areas where iodine is likely not present.

Figure 9 is a schematic diagram illustrating an example of a Bayesian or probabilistic neural network that can be used to solve the material decomposition problem. This exemplary neural network takes as input eight energy bin sinograms and produces as output a number T of material sinograms. The neural network is represented by a mapping Ψθ, where θ is a random parameter vector. Because this mapping depends on the random parameters, applying the network multiple times to the same input data will result in different outputs. The mean of the collection of such outputs is used as an estimate of the material map, while the variance is used as an estimate of the uncertainty map.

Figure 10 is a schematic diagram of an example of a neural network estimator that takes material concentration maps obtained from a deep learning-based material decomposition and generates confidence maps. These confidence maps highlight areas of the image where bone, soft tissue, and iodine are likely to be present, respectively. As can be seen in the image, the iodine confidence map highlights areas where there is a tumor taking up iodine, but also gives a low but non-zero value to the spine region since the algorithm cannot completely deny the presence of iodine in this region. The application of a neural network to material concentration maps obtained from a deep learning-based material decomposition is exemplary and in alternative embodiments of the invention, the neural network can be applied to reconstructed images using other methods such as filtered backprojection.

Figure 11 is a schematic diagram showing a neural network for sinogram space material decomposition based on an unrolled iterative material decomposition method. The method is based on an iterative denoising method with a predefined number of iterations, where the update step at each iteration is replaced by a neural network. In this exemplary embodiment, the gradient descent algorithm is unrolled, meaning that at each iteration step the gradient is calculated and taken as an additional input to the next network, thereby providing the network with information about the physics and statistical models underlying the denoising.

Figure 12 is a schematic diagram showing an example of a neural network that takes energy bin sinograms as input and generates reconstructed basis material images. As an example, a detector that generates eight energy bin sinograms can be used, which are fed into the neural network as eight input channels. In this example, the three output channels correspond to the three basis images (bone, soft tissue, iodine).

Figure 13 is a schematic diagram of an example of a probabilistic neural network that takes energy bin sinograms as input and generates reconstructed basis material images based on random dropout. Each time the network is applied to a set of input sinograms, a random selection of the network weights is randomly set to zero, giving a random network output.

Figure 14 is a schematic diagram showing an example of a probabilistic neural network that takes an energy bin sinogram as input and generates basis material images reconstructed based on the insertion of additive noise. By adding noise values to the nodes of each layer of the network, the output basis images are random functions of the input bin sinograms. In this way, applying the network multiple times to the same input image can provide a random distribution of output images, and the network can be trained to match this distribution to the posterior distribution of images given the input data.

Figure 15 is a schematic diagram of an example of a probabilistic neural network that takes an energy bin sinogram as input and generates reconstructed basis material images based on a variational autoencoder. The network is composed of an encoder, a random feature generator, and a decoder. The encoder converts the energy bin sinogram into a feature mean vector and a variance vector. These vectors are used as parameters to randomly generate a random feature vector. This vector can be chosen, for example, as a sample from a multivariate normal distribution with the mean and variance given by the output vector from the encoder. This random feature vector is used as the input of the decoder network, which generates the bone, soft tissue, and iodine basis images. In this way, the entire variational autoencoder functions as a probabilistic neural network that maps the input sinogram to a set of output basis images, which are not deterministic but are sampled from a statistical distribution. The network can be trained so that this distribution matches the posterior distribution of the image given the input data.

Non-limiting example features of mapping uncertainty using deep neural networks include:
a. The neural network is trained to approximate the posterior probability distribution of the solution.
b. The neural network acts as a random function, i.e. for the same observation "y" (network input), the network can provide K different solutions (network outputs, X ₁ , X ₂ , ..., X _K ).
- In Bayesian networks, this is achieved by, for example, random dropout.
In the post-processing of the variational encoder, there is a random latent parameter Z.
In a generative network, there are random input parameters Z.
c. A statistical distance is used in the training loss of the neural network to learn the posterior probability distribution of the solution (KL divergence, Wasserstein distance, etc.).

According to a second main aspect, a non-limiting example of a corresponding system for determining a confidence indication for machine learning image reconstruction, such as deep learning image reconstruction in computed tomography (CT), is provided, where the system for determining the confidence indication is configured to acquire energy-resolved X-ray data. The system is further configured to process the energy-resolved X-ray data based on at least one machine learning system, such as one or more neural networks, to obtain a representation of a posterior probability distribution of at least one reconstructed basis image or image features thereof. The system is also configured to generate one or more confidence indications for the at least one reconstructed image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image, based on the representation of the posterior probability distribution.

As previously mentioned, the machine learning image reconstruction may be, for example, deep learning image reconstruction, and the at least one machine learning system may include at least one neural network.

As an example, the one or more confidence indications may include an error estimate or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image, and/or an error estimate or a measure of statistical uncertainty for at least one image measurement derivable from the at least one reconstructed basis image.

Optionally, the system may be configured to generate the one or more confidence indications in the form of one or more uncertainty maps for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features thereof,

In certain embodiments, the system may be configured to generate the one or more reliability indications in the form of a reliability map for a reconstructed material-selected X-ray image for computed tomography (CT).

As an example, the system is further configured to perform material decomposition based image reconstruction and/or machine learning image reconstruction based on an energy bin sinogram as input to generate the at least one reconstructed basis image or image features thereof.

Optionally, the system may be configured to generate a confidence map to highlight portions of the reconstructed material-selected X-ray image where the machine learning image reconstruction was able to determine with a confidence level above a threshold.

According to a complementary aspect, a non-limiting example of a corresponding system for generating an uncertainty map for machine learning image reconstruction, such as deep learning image reconstruction in spectral CT, is provided. The system for generating an uncertainty map is configured to acquire energy-resolved X-ray data. The system is further configured to process the energy-resolved X-ray data based on at least one machine learning system, such as one or more neural networks, to obtain a representation of a posterior probability distribution of at least one basis image or image feature thereof. The system is also configured to generate one or more uncertainty maps for at least one reconstructed image, or derived image, or image feature thereof, based on the representation of the posterior probability distribution.

According to additional aspects, there is provided a corresponding image reconstruction system including such a system for determining a confidence indication and/or such a system for generating an uncertainty map for deep learning image reconstruction.

According to another aspect, an overall X-ray imaging system is provided that includes such an image reconstruction system.

According to yet another aspect, a corresponding computer program and computer program product are provided.

In an exemplary embodiment of the present invention, the step or configuration of acquiring or obtaining energy resolved x-ray (image) data is performed by a CT imaging system.

In an exemplary embodiment of the present invention, the step or configuration of acquiring or obtaining energy resolved X-ray (image) data is performed by an energy resolved photon counting detector, also referred to as a multi-bin photon counting X-ray detector.

Alternatively, the step or configuration of acquiring energy resolved x-ray (image) data is performed by multi x-ray tube acquisition, slow or fast kV switching acquisition, multi-layer detector or split filter acquisition.

In an exemplary embodiment of the present invention, the machine learning includes a machine learning architecture and/or algorithm that may be based on a convolutional neural network. Alternatively, the machine learning architecture and/or algorithm may be based on a support vector machine or a decision tree-based method.

In an exemplary embodiment of the present invention, the convolutional neural network may be based on a residual network (ResNet), a residual encoder-decoder, a U-Net, an AlexNet, or a LeNet architecture. Alternatively, the machine learning algorithm based on the convolutional neural network may be based on an unrolled optimization method based on gradient descent algorithm, a pri-mal-dual algorithm or an alternating direction method of multipliers (ADMM) algorithm.

In an exemplary embodiment of the present invention, the convolutional neural network includes at least one forward projection or at least one back projection as part of the network architecture.

For better understanding, we now provide an illustrative and non-limiting example of the proposed technology.

As an example, it is possible to determine a reliability indication, such as an uncertainty map or a confidence map, by introducing separate machine learning based estimators to generate estimates of, for example, the bias, variance and/or covariance of the different reconstruction basis images. These estimates are propagated to generate uncertainty maps of derived images, such as a virtual monochromatic image or a virtual non-contrast image.

There are various ways to generate these maps. One method is based on bootstrapping, where a neural network is trained on a resampled training data set. For example, a random set of training samples, including input and output training data, each sampled with replacement, can be used to train the neural network. By repeating this procedure, an ensemble of neural networks can be obtained, and each of these networks can be used to process the input image data to obtain an output image or an output image data representation. The variability or uncertainty in this ensemble of output images can be measured, for example, as a pixel-wise standard deviation over the distribution of images. A second neural network can then be trained to map the measured image data to the resulting uncertainty or resulting distribution of image values. However, a less computationally intensive method is the variational autoencoder. This neural network architecture, which maps data to a low-dimensional feature space in an intermediate layer, can be trained to sample the posterior probability distribution of the image results of a machine learning image reconstruction procedure, such as the deep learning reconstruction method under study. The posterior probability distribution of the low-dimensional intermediate latent features of the variational autoencoder can be obtained from the encoder function. The decoder can then be used to determine the corresponding posterior probability distribution for the reconstructed image.

One way to represent a probability distribution is to provide a random sample, also called a Monte Carlo sample, from the distribution.

One way to process an image representation to obtain a representation of a probability distribution is to apply a probabilistic neural network to the image representation. A probabilistic neural network is a neural network that contains random elements or components, such that the output of the probability distribution is a random function that depends on the input.

By way of example, the probabilistic neural network can provide one or more Monte Carlo samples of a probability distribution.

In another exemplary embodiment, a deterministic neural network can be trained to provide a measure of the probability distribution of a posterior random variable, e.g., an image or an image feature.

For example, the measure of the probability distribution of a posterior random variable can be a mean variance, a covariance, a standard deviation, a skewness, a kurtosis, or a combination of these.

For example, a statistical estimator of the image uncertainty or posterior probability distribution, such as a Monte Carlo estimator, a Markov Chain Monte Carlo estimator, a bootstrap estimator or a stochastic neural network estimator, can be first created and then used to generate training data for training a deterministic neural network that predicts one or more measures of the posterior probability distribution.

By way of example, the basis image may be a map of the density of a physical material, such as water, soft tissue, calcium, iodine, gadolinium, or gold. The basis image may also be a map of a fictitious or hypothetical material that represents a physical property, such as a map of the Compton scatter cross-section, photoelectric absorption cross-section, density or effective atomic number.

For example, the confidence indication can be an error estimate or a measure of statistical uncertainty for at least one point in one or more reconstructed images. The confidence indication can also be an error estimate or a measure of statistical uncertainty for at least one image measurement derivable from at least one image.

For example, error estimates or measures of statistical uncertainty can be upper error limits, error bounds, standard deviations, variances, or mean absolute error.

For example, an image measurement that can be derived from at least one image can be a measurement of a feature's size, area, volume, degree of non-uniformity, shape or irregularity, a measurement of composition, or a measurement of a substance's concentration.

For example, the image measurement that can be derived from at least one image can be a radiology feature, such as a standardized radiology feature.

In an exemplary embodiment of the present invention, processing the energy resolved x-ray data includes forming at least one basis sinogram or reconstructed basis image and processing said image based on a neural network.

For example, the neural network can be a convolutional neural network.

For example, the neural network can be a deep neural network with at least five layers to allow sufficient flexibility in adapting to the training data.

Example methods for estimating error maps include Markov chain Monte Carlo methods and approximate Bayesian estimation methods based on variational dropout.

The paper by Tanno et al., "Uncertainty modelling in deep learning for safer neuroimage enhancement: Demonstration in diffusion MRI," Neuroimage 225, October 9, 2020, is about a method using Bayesian neural networks with variational dropout to generate uncertainty maps for diffusion magnetic resonance image enhancement.

The paper "Uncertainty Quantification in Deep MRI Reconstruction" by Edupuganti et al., IEEE Transactions on Medical Imaging, Vol. 40, No. 1, January 2021, describes a method to use a variational autoencoder as a post-processing step to generate resulting posterior samples and use them to build an uncertainty map for magnetic resonance image (MRI) reconstruction from undersampled data. However, the authors require a preprocessed reconstruction that is computed without deep learning, and is further related to MRI.

The paper "Deep posterior sampling: Uncertainty quantification for large scale inverse problems" by Adler and Oktem, Medical Imaging with Deep Learning 2019, is about a method to quantify the uncertainty in X-ray computed tomography images, considering a post-processing generative neural network that samples the posterior probability distribution of the results. However, the authors require pre-processed reconstructions, and the paper does not disclose how to quantify the error of a specific material density map. The paper is more about MRI.

The US20200294284A1 specification is directed to a method for generating uncertainty information for a reconstructed image. However, the paper does not disclose a method for quantifying the error of a specific material density map and considers energy-integrated CT without any energy resolved data. This further means that a material basis decomposition cannot be effectively performed to generate a basis image.

The approach in this application primarily considers energy-resolved (spectral) CT with multi-energy and multi-material results.

For example, photon-counting CT implies a high-dimensional problem, requiring appropriate scaling, and cross-material and cross-energy information influences posterior sampling.

The above three articles are based on post-processing neural networks and do not solve the reconstruction problem with deep learning.

The proposed technique allows basis image reconstruction and uncertainty mapping to be solved using machine learning techniques such as deep learning.

It will be appreciated that the features and arrangements described herein can be implemented, combined, and rearranged in various ways.

For example, the embodiments may be implemented in hardware, or at least partially in software for execution by suitable processing circuitry, or a combination thereof.

The steps, functions, procedures, and/or blocks described herein may be implemented in hardware using conventional techniques, including both general purpose electronic circuitry and application specific circuitry, such as discrete or integrated circuit technology.

Alternatively, or as a complement, at least some of the steps, functions, procedures and/or blocks described herein may be implemented in software, such as a computer program, for execution by suitable processing circuitry, such as one or more processors or processing units.

This can be implemented, for example, as part of a computer-based image reconstruction system.

16 is a schematic diagram illustrating an example of a computer implementation according to an embodiment. In this particular example, the system 200 comprises a processor 210 and a memory 220, the memory including instructions executable by the processor, such that the processor is operable to perform the steps and/or actions described herein. The instructions are typically configured as computer programs 225, 235 and may be preconfigured in the memory 220 or downloaded from an external memory device 230. Optionally, the system 200 comprises an input/output interface 240 that may be interconnected to one or more processors 210 and/or memory 220 to allow input and/or output of relevant data, such as input parameters and/or result output parameters.

In certain embodiments, the memory includes such a set of instructions executable by the processor, whereby the processor is operable to generate a confidence representation, such as an uncertainty map, for deep learning based image reconstruction in CT imaging.

The term "processor" should be interpreted in a general sense as any system or device capable of executing program code or computer program instructions to perform a particular processing, decision-making, or computational task.

Thus, a processing circuit including one or more processors is configured, upon execution of a computer program, to perform well-defined processing tasks as described herein.

The processing circuitry need not be specialized solely to perform the steps, functions, procedures and/or blocks described above, but may perform other tasks.

The present technology also provides a computer program product including a computer-readable medium 220, 230 having such a computer program stored thereon.

As an example, the software or computer program 225, 235 can be realized as a computer program product, typically carried or stored on a computer-readable medium 220, 230, particularly a non-volatile medium. The computer-readable medium can include one or more removable or non-removable memory devices, including but not limited to read-only memory (ROM), random access memory (RAM), compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs, universal serial bus (USB) memory, hard disk drive (HDD) storage, flash memory, magnetic tape, or any other conventional memory device. Thus, the computer program can be loaded into the operating memory of a computer or equivalent processing device for execution by its processing circuitry.

A process flow can be considered as a computation flow when it is executed by one or more processors. The corresponding device, system and/or apparatus can be defined as a group of functional modules, where each step executed by the processor corresponds to a functional module. In this case, the functional modules are implemented as computer programs executed on the processor. Thus, the device, system and/or apparatus may alternatively be defined as a group of functional modules, where the functional modules are implemented as computer programs executed on at least one processor.

Thus, the computer programs resident in the memory can be organized as suitable functional modules configured to perform at least some of the steps and/or tasks described herein when executed by the processor.

Alternatively, most of the modules can be implemented as hardware modules or can be replaced by hardware. Software or hardware is purely an implementation choice.

10: X-ray source 11: Gantry 12: Table 13: Isocenter 15: CT 20: X-ray detector system 21: Detector element 25: Analog processing circuit 30: Image processing system or device 35: Image processing system 40: Digital processing circuit 41: X-ray control device 42: Gantry control device 43: Table control device 44: Detector controller 45: Memory 50: Computer 60: Operator console 100: X-ray imaging system 200: System 210: Processor 220: Memory 225, 235: Computer program 230: External memory device 240: Input/output interface 301: Digital-to-analog converter 302: Comparator 303: Digital counter

Claims (30)

1. A method for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT), comprising:
acquiring energy resolved X-ray data (S1);
- processing the energy resolved X-ray data based on at least one machine learning system to generate at least one reconstructed basis image or a representation of a posterior probability distribution of image features (S2, S2b);
and (S3) generating one or more confidence indications for image features of the at least one reconstructed base image or the at least one derived image based on the representation of the posterior probability distribution. The method of claim 1, wherein the machine learning image reconstruction is deep learning image reconstruction and the at least one machine learning system includes at least one neural network. The method of claim 1 or 2, wherein the representation of the posterior probability distribution includes at least one of the mean, variance, covariance, standard deviation, skewness, and kurtosis. A method according to any one of claims 1 to 3, wherein the one or more confidence indications comprise an error estimate or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image and/or an error estimate or a measure of statistical uncertainty for at least one image measurement derivable from the at least one reconstructed basis image. The method of claim 4, wherein the error estimate or statistical uncertainty measure comprises at least one of an upper error limit, a lower error limit, a standard deviation, a variance, or a mean absolute error. The method of claim 4 or 5, wherein the at least one image measurement comprises at least one selected from a feature dimension measurement, an area measurement, a volume measurement, a degree of non-uniformity measurement, a shape or irregularity measurement, a composition measurement, and a substance concentration measurement. The method of any of claims 1 to 6, wherein the one or more confidence indications include one or more uncertainty maps for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features thereof. 8. The method of claim 1, wherein the step (S3) of generating one or more confidence indications includes generating a confidence map of a reconstructed material-selected X-ray image for computed tomography (CT). The method of claim 8, wherein the confidence map is generated to highlight portions of the reconstructed material-selected x-ray image where the machine learning image reconstruction was able to determine with a confidence level equal to or greater than a predetermined threshold. A method according to any one of claims 1 to 9, wherein the step (S3) of generating one or more confidence indications comprises a step of generating one or more confidence maps by a neural network that receives as input a material concentration map obtained from a deep learning-based material decomposition. The method according to any one of claims 1 to 10, further comprising (S2a) performing material decomposition-based image reconstruction and/or machine learning image reconstruction to generate the at least one reconstructed basis image or image features thereof based on the acquired energy-resolved X-ray data. The method of claim 11, wherein the step (S2a) of performing material decomposition based image reconstruction and/or machine learning image reconstruction comprises a step of generating the at least one reconstructed basis image or image feature by a neural network receiving an energy bin sinogram as input. A method according to any of claims 1 to 12, wherein the step (S3) of generating the one or more confidence indications comprises a step of determining an uncertainty or confidence map of each individual basis material image and a step of determining a covariance between different basis material images, and a step of propagating the uncertainty or the confidence map using a formula or algorithm for the propagation of the uncertainty, making it possible to obtain an uncertainty map of a derived image. A method according to any one of claims 1 to 13, in which at least one basis material image is generated together with at least one uncertainty map, the uncertainty map being a representation of an uncertainty or error estimate of the at least one basis material image, and the at least one basis material image and the at least one uncertainty map being presentable to a user as separate images or in combination. The method of claim 13 or 14, wherein the at least one uncertainty map can be presented as an overlay on the at least one basis material image, or the at least one uncertainty map can be presented by a distortion filter on the at least one basis material image. 16. The method of any of claims 1 to 15, wherein the step (S2; S2b) of processing the energy resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution comprises: generating samples of the posterior probability distribution given the acquired energy resolved X-ray data by a neural network; and the step (S3) of generating one or more confidence indications comprises generating an uncertainty map as a standard deviation over a number of samples. A method according to any of claims 1 to 16, wherein the step (S2; S2b) of processing the energy-resolved X-ray data based on at least one machine learning system to generate a representation of a posterior probability distribution comprises applying a neural network implemented as a variational autoencoder to encode input data vectors into parameters of a probability distribution of latent random variables, and extracting a set of posterior samples of the latent random variables from this probability distribution for processing by a corresponding decoder to obtain posterior observations. A method according to any one of claims 1 to 17, wherein the step of generating (S3) the one or more confidence indications comprises the step of generating at least one map of variance or standard deviation of at least one basis coefficient associated with the at least one reconstructed basis image and/or at least one map of covariance or correlation coefficients of a set of at least one basis function associated with the at least one reconstructed basis image. A method according to any one of claims 1 to 18, wherein the representation of the posterior probability distribution is specified by the mean and variance of multiple image features. A system (30, 40, 50, 200) for determining one or more confidence indications for machine learning image reconstruction in computed tomography (CT), the system being configured to acquire energy resolved X-ray data, the system being further configured to process the energy resolved X-ray data based on at least one machine learning system to acquire a representation of a posterior probability distribution of at least one reconstructed basis image or image features thereof, the system also being configured to generate one or more confidence indications for the at least one reconstructed basis image, or at least one derived image derived from the at least one reconstructed basis image, or image features of the at least one reconstructed basis image or the at least one derived image, based on the representation of the posterior probability distribution. The system (30, 40, 50, 200) of claim 20, wherein the machine learning image reconstruction is deep learning image reconstruction and the at least one machine learning system includes at least one neural network. The system (30, 40, 50, 200) of claim 20 or 21, wherein the one or more confidence indications include an error estimate or a measure of statistical uncertainty for at least one point in the at least one reconstructed basis image and/or an error estimate or a measure of statistical uncertainty for at least one image measurement derivable from the at least one reconstructed basis image. The system (30, 40, 50, 200) according to any of claims 20 to 22, configured to generate the one or more confidence indications in the form of one or more uncertainty maps for the at least one reconstructed basis image, or for at least one derived image derived from the at least one reconstructed basis image, or for image features thereof. The system (30, 40, 50, 200) according to any of claims 20 to 23, wherein the system (30, 40, 50, 200) is configured to generate the one or more reliability indications in the form of a reliability map for a reconstructed material-selected X-ray image for computed tomography (CT). The system (30, 40, 50, 200) according to any of claims 20 to 24, further configured to perform material decomposition based image reconstruction and/or machine learning image reconstruction based on an energy bin sinogram as input to generate the at least one reconstructed basis image or image features thereof. The system (30, 40, 50, 200) of any of claims 20 to 25, wherein the system (30, 40, 50, 200) is configured to generate the confidence map to highlight portions of the reconstructed material-selected X-ray image where the machine learning image reconstruction was determined with a confidence level equal to or greater than a threshold. An image reconstruction system including a system (30, 40, 50, 200) according to any one of claims 20 to 26 for determining one or more confidence indications for machine learning image reconstruction in CT. An X-ray imaging system (100) comprising an image reconstruction system according to claim 27. A computer program (225, 235) comprising instructions that, when executed by at least one processor (30, 40, 50, 210) associated with a computed tomography system (100), cause the at least one processor (30, 40, 50, 210) to perform a method according to any one of claims 1 to 19. A computer program product comprising a non-transitory computer readable storage medium (220, 230) carrying a computer program (225, 235) according to claim 29.