WO2022268618A1

WO2022268618A1 - Multi-energy x-ray imaging with anatomical intelligence

Info

Publication number: WO2022268618A1
Application number: PCT/EP2022/066409
Authority: WO
Inventors: Grzegorz Andrzej TOPOREK; Leili SALEHI; Ayushi Sinha; Ramon Quido ERKAMP; Ashish Sattyavrat PANSE; Daniel Simon Anna Ruijters
Original assignee: Koninklijke Philips N.V.
Priority date: 2021-06-24
Filing date: 2022-06-15
Publication date: 2022-12-29

Abstract

System and methods for improved spectral imaging. Input spectral data is combined by a combiner (CMB) based on combination parameters. The combination parameters are, or are derived from, feature maps. The feature maps may be generated by processing the input spectral data by a feature extractor (FE). The feature extractor may be a machine learning model (M).

Description

MULTI-ENERGY X-RAY IMAGING WITH ANATOMICAL INTELLIGENCE

FIELD OF THE INVENTION

The invention relates broadly to image processing for spectral imaging. Specifically, the invention relates to a system for image processing, to a method of image processing, to a method of training a machine learning model for use in the system or method, to a computer program element, and to a computer readable medium.

BACKGROUND OF THE INVENTION

Conventional computed tomography (CT) scanners that utilize a single X-ray energy spectrum can discriminate some tissue types based on their attenuation coefficients (Hounsfield scale).

However, tissue types or materials with different elemental composition can be still be represented by the same Hounsfield unit “HU” value as they may have similar linear attenuation coefficients over one X-ray spectrum. This makes it difficult to differentiate these materials in a CT image. For example, it is difficult to differentiate between calcified plaques and iodine contrast in vessels. Even though calcium and iodine are different materials, depending on the mass density and iodine concentration, the contrast agent, calcified plaque and bone in the vicinity appear similar.

As the linear attenuation coefficients are energy spectrum dependent, it is possible to differentiate between elements or materials by imaging with plural different energy spectra, as is done in spectral imaging. For example, DECT (“Dual Energy CT”) may be used by clinicians to improve the characterization of tissue types. Superior tissue characterization enables multiple applications such as disease severity monitoring (e.g., highlighting lesions, bleeding detection, etc.), as well as quantitative assessment of structures (segmentation, edema monitoring, underperfused area monitoring in the brain, etc.).

Typical spectral CT systems may be source-based. Such source-based solutions may include DECT as mentioned above, but may also include other hardware solutions, such as rapid kilovoltage (kVp) switching, or dual-spin technologies. In addition, it is in particular compact detector-based spectral CT systems (IQon Spectral CT, by Philips) that have gained popularity of late. In such detector-based systems, the spectral separation occurs at detector level.

In general, in CT, measurements of line integrals of the function to be reconstructed are collected at the detector. The function is that of linear attenuation coefficients. It is known that the linear attenuation coefficient varies with energy: J m(c, y; £^')ds. It is also known that the attenuation coefficient of any material can be expressed as a linear combination of photoelectric and Compton scatter coefficients, as per m(E) = a_x — + C ₂†KN (E) Solving for weights ₁ and a₂ for each ray projection acquired at two different energies effectively decomposes the raw data into photoelectric and Compton scatter basis components (referred to as “basis pair images”), which is a main principle of DECT.

One way of visualizing dual-energy image data is by linear blending of the basis pair images. For instance, a linear mixing ratio may be 70/30, i.e., 70% of high-energy image data, and 30% of low-energy data. Other relatively more complex methods, such as described in US 2012/0321164A1, propose to blend two or more energy sources using a non-linear function. Parameters of the function are either predetermined or adjusted by the user based on the organ of interest.

Despite these advancements in hardware technology, currently available visualization and reconstruction methods for spectral CT imaging are still very basic in some sense: first, the main challenge is that different energy levels may impact contrast resolution and/or noise characteristics of the respective image data differently. For example, 80 kV image data may provide greater contrast resolution than 140 kV image data. But, the 80 kV image data may be noisier than the 140 kV image. In addition, low energy data may lack sharp contours but have better contrast resolution for a wide range of soft tissues. Second, additional material decomposition images that are generated from photoelectric and Compton scatter basis images are at present simply color-coded and displayed along with, or blended with, high-resolution grayscale images (for example, poly-energetic or virtual monoenergetic images).

SUMMARY OF THE INVENTION

There may be a need for improved spectral imaging.

An object of the present invention is achieved by the subject matter of the independent claims where further embodiments are incorporated in the dependent claims. It should be noted that the following described aspect of the invention equally applies to the image processing method, to the method of training the machine learning model, to the computer program element and to the computer readable medium.

According to a first aspect of the invention there is provided a system for image processing, comprising: an input interface for receiving plural spectral input data generated by a spectral X-ray imaging system; at least one feature extractor configured to process one type of the plural spectral input data and generate a feature map corresponding to the processed type of spectral input data; a combiner configured to combine, in a combination operation, the plural spectral input data into combined data, the combination operation controlled by a combiner parameter that is based on feature maps previously generated by the one or more feature extractor; and a graphics display generator to cause visualizing the combined data on a display device. In embodiments, the plural spectral input data is of one of plural types of spectral data, the one or more feature extractor corresponding to that type of spectral data.

In embodiments, the system comprises a selector configured to select a corresponding one of the one or more feature extractor, based on the type of the spectral data.

In embodiments, the combiner implements a reconstruction algorithm or a blender algorithm.

In embodiments, the one or more feature extractor is implemented as a machine learning model.

In embodiments, the machine learning model is of the artificial neural network type.

In embodiments, the feature map includes output generated at a hidden/intermediate layer of the artificial neural network type model. The feature map is internal output of the model. In particular, the output may be provided by any intermediate layer in an artificial neural network type mode. The feature map may be an entity form latent space of the network.

In embodiments the combiner parameter is multi -dimensional.

In embodiments, the system includes a user interface allowing a user to modify the combiner parameters.

In embodiments, the graphics display generator is to effect visualizing the feature map on the or on a different display device.

In embodiments, the feature extractor is configured to account for prior knowledge such as anatomical information.

According to another aspect there is provided a method of image processing, comprising: receiving plural spectral input data generated by a spectral X-ray imaging system; processing at least one type of the plural spectral input data by one or more feature extractors; combining, in a combination operation, the plural spectral input data into combined data, the combination operation being controlled by combiner based on feature maps generated in the processing step; and visualizing the combined data on a display device.

According to another aspect there is provided a method of training the machine learning model.

According to another aspect there is provided a computer program element, which, when being executed by at least one processing unit, is adapted to cause the at least one processing unit to perform the method of image processing or the training method.

According to another aspect there is provides at least one computer readable medium having stored thereon the program element, or having stored thereon the machine learning model.

The proposed system and method address a clinical need for generating high resolution grayscale images in combination with good visualization of material properties beyond simple color-coding. Improved processing as proposed herein can enable more accurate registration, segmentation, and disease visualization. The processing may be one of pre-processing (in image domain), or one of post-processing (image domain). The processing using the computed combiner parameters may also happen “in-between”, during and/or in the reconstruction algorithm.

The proposed system and method allow combining, in a computationally efficient manner, types of spectral imagery so that, for a given image location, the type with improved contrast is chosen. The combination may be done in a fast and automated manner. Machine learning (“ML”) may be used for improved robustness and accuracy. Unsupervised learning may be used to train ML model(s), thus reducing setup overhead.

The proposed system and method allow for improved multi -energy X-ray imaging that incorporates anatomical information via for example self-learned, generalizable, spatial feature maps. A plurality of images at different energies may be combined so that the respective benefit of a particular aspect of image information in each spectral input image is maintained and visualized in a single resulting image. The feature maps are also highly tune-able: the user interface allows user to adjust preponderance/importance of high frequency (edges) vs. low frequency (soft tissue) image information for each energy source by changing a single parameter. A feature map is a spatially resolved data structure. Elements of the feature map may be indicative of or related to parameters of a model, such as in a machine learning model. Such feature maps have been found to be correlated to structures within the spectral data processed by the feature extractor using the model. In embodiments it is in particular such correlations that is harnessed herein in the combiner parameters.

Whilst the above has been described with main reference to X-ray based spectral imagery, other types of multi -energy data generated by other imaging modalities may also be envisaged herein for processing and/or visualization in accordance with the principles described herein. The term “spectral data” as used herein may refer to image data in projection or image domain. Spectral data of different types may relate to spectral data at different energies or may relate to different data type, including imagery derived from multi -energy data in image or projection domain.

More specially, the proposed visualization scheme allows computing locally bespoke imagery, with good contrast, by combining, in a combination operation, different types of spectral data. Different types of spectral data may give rise to different contrast for different materials/tissues. Thus, some spectral data type may be more suitable in terms of improved contrast for imaging of certain materials/tissue (or quantities thereof) than it is for imaging other materials/tissue (or quantities thereof). The combination operation is controlled by the combiner data. The combiner data includes intermediate output of the ML model generated in response to the input data to be combined. Spatial variation of the combiner data modulates the combiner operation. To this end, the combiner data is preferably multi-dimensional, with dimension 2, 3 or higher, corresponding to the dimension of the input spectral (image) data. The variation allows combining contributions from the spectral data to emphasize in the combined image certain local information from one spectral data type to be combined over others, to so arrive at the bespoke contrast. In preferred embodiments, the combiner parameters are configured so that at each location in the combined image, the spectral data best suited for contrast at that location is emphasized or amplified over other spectral data types. The combiner, thanks to the specially configured combiner data, may thus be said to have “anatomy awareness”.

Whilst feature maps (derived or not) may be visualized, they are not spectral data or (spectral) image data as such. In the latter, pixels/voxels represent intensities of matter versus X-ray interaction, whilst pixel/voxels of the feature maps represent spatially resolved data generated within the ML models. As mentioned, the data so generated is in response to the machine learning model processing imagery, such as the input spectral data. The feature maps thus represent the manner of operation of the machine model when the spectral imagery is processed by the machine learning model.

Whilst imaging of animal or human patients is mainly envisaged herein, imaging of objects is not excluded herein. The principles described herein are well applicable beyond the medical realm, such as in screening items of baggage in security checks or of a product in non-destructive testing, or other tasks that may call for material type specific imaging.

The combiner parameters for different spectral image types may be normalized as an option. In general, two or more parameters may be used. However, when only two items of spectral data/spectral images are to be combined, only a single (multi -dimensional) parameter may be required because of optional normalization.

In embodiments, the machine learning model may be based on an encoder-decoder architecture as once example of an ML model envisaged herein. Specifically, such an encoder- decoder architecture may be used to train the machine learning model. Once trained, and when used in the feature extractor, only the encoder portion may be required in deployment. In the alternative, or in addition, it is the decoder part that is used in the model for the feature extractor. Using the encoder- decoder architecture allows unsupervised learning However other ML models are also envisaged herein, and so are supervised learning setups.

“User” relates to a person, such as medical personnel or other, operating the imaging apparatus or overseeing the imaging procedure. In other words, the user is in general not the patient.

In general, the term “ machine learning ” includes a computerized arrangement that implements a machine learning (“ML”) algorithm. Some such ML algorithms operate to adjust a machine learning model that is configured to perform (“learn”) a task. Other ML algorithms operate directly on training data, not necessarily using such a model. This adjusting or updating of model parameters based on a training data (corpus) is called “training”. In general task performance by the ML model may improve measurably, with training experience. Training experience may include suitable training data and exposure of the model to such data. Task performance may improve the better the data represents the task to be learned. “ Training experience helps improve performance if the training data well represents a distribution of examples over which the final system performance is measured”. The performance may be measured by objective tests based on output produced by the modeling response to feeding the model with test data. The performance may be defined in terms of a certain error rate to be achieved for the given test data. See for example, T. M. Mitchell, “ Machine Learning ”, page 2, section 1.1, page 6 1.2.1, McGraw-Hill, 1997.

“ Spectral ” (image) data in image- or projection domain is provided. The data is or may be based on projection data acquired by an imaging system configured for spectral imaging. The imaging system is spectral if it implements a spectral resolution capability of the spectrum of radiation into at least two energy levels.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the invention will now be described with reference to the following drawings, which, unless stated otherwise, are not to scale, wherein:

Figure 1 shows a block diagram of a spectral X-ray imaging system;

Figure 2 shows a block diagram of a computer-implemented spectral processing system for generating spectral data such as spectral imagery for display;

Figures 3 show block diagrams of a visualization support system for combining spectral data;

Figure 4 shows a block diagram illustrating details of a feature extractor as may be used in the visualization support system of Figure 3;

Figures 5 show embodiments of a training system for training machine learning models as may be used in the feature extractor of Figure 4;

Figure 6 illustrates data including exemplary feature maps as may be generated by a machine learning model in response to processing illustrated exemplary spectral input imagery; and

Figure 7 shows a flow chart of a method of computer-implemented support of visualizing spectral data.

DETAIFED DESCRIPTION OF EMBODIMENTS

Figure 1 a schematic block diagram of a spectral imaging system SIS. Broadly, the system includes the imaging apparatus IA having a detector system DT, and a data processing system DPS configured to process data generated by the detector system DT during data acquisition.

The detector system DT includes spatially arranged X-ray sensitive detector pixels. The detector system DT generates projection data A that represents radiation intensities as detected at the detector pixels.

The imaging apparatus SIS is configured for spectral imaging. Spectral imaging allows acquiring multi -energy projection data AEI, E2 at least two energy levels El, E2, also referred to as energy ranges or “(energy jwindows”. The acquired projection data A_{EI, E}2 1S thus multi dimensional, in that to each detector pixel there corresponds at least two different intensities, one for each energy window. Operation of the imaging apparatus is now described in more detail, before turning to the data processing section DPS.

The imaging apparatus IA may be a CT scanner. The scanner IA may include a stationary gantry and a rotating gantry. The rotating gantry is rotatably supported by the stationary gantry and rotates around an examination region ER and a portion of an object or subject therein about a Z-axis. A radiation source XS, such as an X-ray tube, is supported by and rotates with the rotating gantry around the examination region ER. The radiation source XS emits in general wideband polychromatic X-ray radiation XB that may be optionally collimated to form a generally fan, wedge, or cone shaped X-ray radiation beam that traverses the examination region ER.

Alternatively, the imaging apparatus IA is a C-arm scanner wherein a C-arm imaging device rotates around the examination region ER.

The radiation sensitive detector array of pixels of detector system DT may subtend an angular arc opposite the radiation source XS across the examination region ER. The detector array includes one or more rows of the detector pixels that are arranged with respect to each other along the Z-axis and operate to detects X-ray radiation traversing the examination region ER and hence parts of the patient PAT. The detector pixels generate the projection (raw) data.

A subject support SP, such as a couch, supports a subject PAT or object (e.g., a phantom) in the examination region ER. The subject support PC is movable in coordination with performing an imaging procedure. This allows moving the subject PAT or object with respect to the examination region ER for loading, scanning, and/or unloading the subject or object.

An operator console (not shown) may include a human readable output device such as a display monitor DD, etc. and a user input device such as a keyboard, mouse, etc. The console OC further includes a processor (e.g., a central processing unit (CPU), a microprocessor, etc.) and computer readable storage medium (which excludes transitory medium) such as physical memory.

The operator console OC allows user to control the imaging procedure.

Whilst the principles disclosed herein are described with main reference to CT or other volumetric/rotational imaging modalities such as C-arm imagers or similar, they are of equal application to projection imaging in radiography.

During imaging/acquisition operation, the patient, or at least a region of interest (“ROI”) of patient, resides in the examination region ER. For example, the patient may lie on the patient couch SP arranged at least partly inside a donut shaped CT examination region ER.

The X-ray source XS is energized. X-rays XB emerge from the source XS, traverse the examination region ER and the RO I/patient PAT, and are then registered at the far end at the X- ray sensitive pixels of the X-ray detector D.

The impinging X-ray XB causes the X-ray sensitive pixels to respond with electrical signals. The electrical signals are processed by data acquisition circuitry DAS of the scanner IA to produce digital projection raw data. As briefly mentioned, the imaging apparatus IA, such as the CT scanner in Figure 1, is configured for spectral imaging. The X-ray imaging apparatus SIA thus produces sets of spectral projection raw data 2y>2 for two or more energy windows.

In this manner, spectral imaging allows resolving image contrast into plural energy windows. Resolving into two such energy windows, high E2 and low El, is sufficient for present purposes and is usually referred to as dual energy imaging. The low energy (about < 30 keV) data IEI represents attenuation mainly by photoelectric absorption, whilst the higher energy data l._E2, (E2 > 30 keV) represents attenuation mainly by Compton scattering. Thus, each energy data /._{/ /}. /^captures different aspects of attenuation. However, resolving in more than two such energy windows, as into three, four or more such energy windows is also envisaged in spectral imaging.

Spectral imaging capability may be implemented by detector-sided or source-sided solution. A detector-sided solution for dual imaging is shown in Figure 1. The detector DT comprises two layers, a (relative to the X-ray source XS) top layer TL and a lower bottom layer BL of X-ray sensors. This “dual energy” arrangement allows detecting high energy E2 projection data at the lower layer BL and lower energy projection data at the top layer TL. More than two such layers (multi-layer detector) can be used if resolution into more than two energy windows is required. In embodiments, resolving into two or more than two energy levels may be obtained at the detector-side by having a specially configured detector DT that is equipped with counting circuitry (not shown). The counting circuitry classifies incoming intensities in projection domain into different energy bins against a set of energy thresholds. Source-side solution are also envisaged, such as dual-or multi-source imagers, or those with a single source XS equipped with fast kVp switching circuitry.

Although we will mainly refer below to dual imaging with two energy windows, E1,E2, the principles of the present disclosure are applicable to all form of spectral imaging with more than two energy windows E_{j >}2, and such are specifically envisaged herein in embodiments.

Spectral imaging is useful for diagnostic purposes in particular as it allows extracting more information than traditional energy integrating imaging would allow. In the latter, the spectral information described above is usually lost. In traditional polychromatic imaging, the energy is integrated and it is the total energy deposited at the detector that confers image contrast. The different energy projection images y_j>2inay be of interest in themselves, but the spectral projection imagery may be processed into other imagery still, such as a reconstructed spectral imagery, virtual monochromatic image, contrast agent quantitative maps, a virtual non-contrast image, an electron density image, and/or other spectral imagery.

The data processing section DPS processes the projection data into said spectral imagery of different type(s). The projection data or other types of spectral imagery may be stored in a data repository MEM, may be displayed on a display device DD or may be otherwise processed.

Operation of the data processing system DPS is now explained in more detail. The data processing system DPS may include a spectral processing sub-system SPS configured to process the multi energy projection imagery A EI_, E2 into imagery IEI, E2 in image domain. Specifically, the data processing section DPS may include a visualization system SYS-V to process the imagery for display on the display device DD. As will become apparent further below, components of the systems SPS and SYS-V may overlap.

Before explaining in more detail the visualization (sub-)system SYS-V reference is now first made to the spectral processing system SPS. This is described in more detail in the block diagram of Figure 2. The spectral processing system SPS receives as input the multi energy projection data A_{EI, E}2. The set-up in Figure 2 corresponds to a dual energy set-up for two energy windows Ei, E2 but, as said, more than two energy windows for the projection data Ey>2 are also envisaged herein.

The spectral processing system SPS may be operable in projection domain PD and/or in image domain ID. The spectral processing system SPS may include one or more combiner components CMB (“combiner”) as shown in Figure 2. The combiner CMB combines the multi energy projection imagery, or imagery derived there-from, to obtain spectral data of different types. The spectral data of different types may include spectral imagery in projection or image domain with contrast coding for different functions, effects, materials/tissues, structures etc.

The different spectral data types may include at least two or more image data items acquired at or based on at least two energy levels. Examples of image data types include the raw projections lei_{, E}2 acquired at least two energy windows. Other types of spectral data include basis- component images, based on photoelectric and Compton data, generateable from projection-based decomposition algorithms known in the art. Other spectral data includes the said basis images - such as photoelectric and Compton images - obtained by operation of any spectral reconstruction algorithm known in the art. Other spectral data type includes virtual monoenergetic images generated by combining basis images at different energy levels (typically 40 to 200 keV). Other spectral data types include material maps MM, such as for Iodine density I i_d (or other contrast agent), virtual non contrast imagery VNC, uric acid pair imagery, effective atomic number imagery, etc. generated from any material decomposition algorithm known in the art. Conventional, poly-energetic image may be used in combination with any of the above listed spectral data type. Such conventional energy integrated data is derivable from spectral data if required.

In embodiments, the combiner CMB is, or may include, a basis-components-de- compositioner BD. The basis-components de-compositioner BD combines projection imagery at least two energy levels lki_{. E}2 into projection imagery representative of Compton effects and photo electric effects, respectively. The combiner CMB may thus be part of a pre-processor stage PP.

If transformation into image domain ID is envisaged, the combiner CMB may include instead, or in addition, one or more re-constructors, in particular a spectral re -constructor SR, at re- constructer section RS. As said (tomographic) re-constructor SR transforms projection imagery of whatever type into one or more types of tomographic (cross-sectional) imagery I in the image domain ID. The reconstructed cross-sectional imagery can be thought of as image values that are assigned by the re-constructer to grid points, referred to as voxels, in the 3D portion that makes up the examination region ER. There may be a plurality of such cross-sectional images in different section planes of the image domain ID. The plurality of cross-sectional images in different such planes may form a 3D image volume. Location for image values in a given cross sectional image may also be referred to herein as (image) pixels instead of voxels.

If required, a conventional reconstructor CR may also be included. The conventional reconstructor RC performs energy integrating reconstruction and may be based on fdtered back- projection (FBP) or other that combines the set of high and low energy projection data into a conventional image lev. This may be displayed then in Hounsfield units (HU) as described elsewhere.

The spectral reconstructor RS, as one embodiment of the combiner CMB, combines the Compton scatter and photo electric projection imagery Ac, Ap and/or the set of projection imagery at different levels AEI, E2 as detected at the detector into cross-sectional imagery IQ_P in image domain ID. In Compton images Ic and photo electric images I_p, contrast is conferred by the respective physical phenomena of Compton scatter or photo electric absorption. The spectral re -constructor SR may include or utilize a noise model, such as an anti-correlated noise model or other. The spectral re constructor SR may be implemented as an iterative re-construction algorithm, such as those based on (regularized) (log) -likelihood approaches or on gradient synchronization or other still. Spectral re construction algorithms have been described elsewhere in detail, for example by JFessler et al in “ Multi-Material Decomposition Using Statistical Image Reconstruction for Spectral CT IEEE Transactions on Medical Imaging, vol 33, issue(8), (2014).

The Compton and photo electric effect images Ic, Ip may be processed by another embodiment of the combiner CMB, namely by a material de-composition algorithm MD, to yield material specific maps. Such maps confer contrast to a specific material of interest. In this manner, a VNC (virtual non contrast image) image or iodine map can be generated if contrast agent has been administered before or during projection data acquisition. In VNC imagery, contrast is conferred by matter other than contrast agent.

In an additional or alternative embodiment, the combiner CMB may arranged as a linear combiner LC that linearly combines the photo electric or the Compton image to generate virtual mono-energetic images h at different energy levels E as shown to the right of Figure 2. In addition or instead, the combiner CMB may combine the virtual mono-energetic images themselves at prescribed blend ratios. A combiner (not shown) that combines non-linearly is also envisaged in embodiments. The virtual mono-energetic/monochromatic images, approximate contrast that may be obtainable if a mono-chromatic energy source with a spectrum restricted in the given energy window had been used in the acquisition of the projection data. However, oftentimes the X-ray source used XS has preferably a polychromatic spectrum, not restricted to the desired spectral energy window. Thus, virtual mono-energy imagery ME may be generated despite having merely a polychromatic X-ray source at one’s disposal. The combiner LC may operate in projection domain PD to linearly or otherwise combine the projection imagery _/./,.

In sum, the spectral processing section SPS described above generates a plurality of different types of spectral imagery (in projection and/or image domain), referred to herein as “spectral data” of different types in general and collectively.

The proposed visualization section SYS-v is configured to control the combiner CMB to combine the spectral data of the same or of different type in an advantageous manner for visualization. The visualization processing section SYS-v in interaction with the combiner CMB is now described in more detail with reference to Figure 3A. The combiner CMB may be part of the spectral processing system SPS and/or the visualizer SYS-v may be an external computational entity.

Different types of the above-described spectral data types confer different contrast to different parts of tissue materials. The spectral data of different types can thus be combined so that locally, in the combined image X+, contrast is bespoke or tailored to respective tissue material type at a respective location. The above-described combiner CMB in any one, more than one, or all embodiments operates to combine the spectral data in a novel manner using specially configured combiner parameters, to be described more fully below. The combiner parameters oy drive or control the combination operation of combiner CMB.

Broadly, the visualization system SYS-v may include, or may cooperate with, the above-described combiner CMB in any one or more of the embodiments as described above in Figure 2. The visualization system SYS-V may include a feature extractor FE, to be described in more detail below.

The different types of spectral data generatable by the spectral processing system SPS will be generically referred to herein as Xj. Index j may thus refer to the different energy levels/ranges/windows, but may refer more generally to the different types of spectral data, or to the same type but at different energy levels, etc.

At least two different types of spectral data X /. / >1 are received at one or more input port IN of the system SYS-v.

A selector SL selects from a bank of pre-configured machine learning models M_j, the ones that fit to the respective data types j. Specifically, selector SL selects, based on each spectral data type /=1.2. ... , a respective applicable machine learning model M_j from the bank of pre-configured models. The one or more models M_j implement the feature extractor FE. The respective model is then applied to the respective data type Aj. Herein, the machine learning model M_j is said be “applicable” or “corresponding”, if it has been trained on spectral data of the same type j. The selector may be implemented as a look-up table (LUT) or the like. The models may be assigned with suitable identifiers j to facilitate selection by selector SL.

Application of the machine learning model My to the respective data type j yields a respectively associated combination parameter a j. In other words, the output of the selected models M_j is a set of different combiner parameter a y, one for each data type j. The models can be so applied in parallel or in sequence.

The combiner CMB in any one or all of the above -described embodiments, combines the spectral data X_j using the set of combiner parameters a , to produce the (improved) combined image X+. X+ can be thus be written formally as X+=C(X y, a y), where C() is a combiner function. The function may be linear, or non-linear, depending on the combiner CMB. The spectral information as captured in the spectral data of different types may thus be consolidated into a single image that is the combined image X+.

Graphics output interface g-OUT uses the combined image X+ to drive graphics circuitry. The graphics circuitry, based on image X+, controls display device DD to display a visualization of the improved combined image X+. Thus, a graphics display is generated to visualize, in particular, combined image X+. Alternatively, or instead of so displaying, the combined image X+ may be stored in a memory MEM or may be otherwise processed.

The set of combiner parameters a j may be adjusted by a user through a suitable user interface UI. For example, this may be done if the ML-based combined image X+ is not to the user’s liking for some reason. In this manner the user may adjust the contrast in the combined imagery X+ to emphasis certain spectral data types, and thus related tissue and/or materials.

The combiner CMB combines the input spectral dataX_j based on the computed parameter combiners j to modulate the combination operation. This modulation may be done globally, in which case there is one single constant qj for some or each data type y, or may be done regionally, in which same or all the /., s differ regionally across the respective spectral input data lj. Preferably, however, the two or more sets of combiner parameters qj generated by the feature extractor FE allow point-wise local variation. In other words, at least one, more than one, or all combiner parameter(s) qj is a data structure that corresponds in dimension and size to the respective spectral data to which it is applied by the combiner CMB. Thus, each image location in the spectral data of type j has its respective local parameter which is applied at the said respective location. Thus, a given parameter a, for a given spectral data type j may be multi-dimensional, and may be written as ay = ay (xi. ...x_\j. where N corresponds to the dimension of the respective spectral data type y, and ( xi, ... X_N) are the coordinates of an image location (such as pixel or voxel, as the case may be). N may be 2 or 3, or even higher, such as 4 as may be the case for a times series of 3D volumes for example. Because of its multi -dimensional nature (matrix or tensor), the combiner parameter a j for a given type j may act differently on different portions of the spectral image/data to which it is applied. A respective entry of the multidimensional parameter acts on a respective value of the spectral data at a respective different spatial position (pixel/voxel).

The combiner CMB may part of the spectral processing system SPS described above However, this is not necessarily required as the combiner CMB may also be embodied as an image blender algorithm that linearly or non-linearly combines spectral data as generated by the spectral processing section SPS. Operation of a linear blender CMB may be written formally as C(a_jX_j) = oC_jX_j. For example, user selected ones of the virtual mono-energetic images ME may be linearly combined. However, combiner functions C( y) other than linear are also envisaged.

During training process, the feature extractor FE has inherently leamt a rich representation of the spectral data by incorporating most prominent features, such as edges, comers, ridges, blobs, gradients, etc. These features will be typically very good descriptors of anatomical features represented by the spectral data. In addition and optionally, feature extractor FE may take as a-priori knowledge additional sources of information such as segmentation masks, contours automatically or manually selected by user, or detection results. This may be implemented for instance by combining feature extractor FE with a segmentation mask extracted from a segmentation model, or by training feature extractor FE’s model on a segmentation task rather than classification task. The segmentation information or other anatomical context information may be used as contextual data, in addition to the spectral image data. Both, the contextual data and the input spectral data may be co-processed by the ML model M_j during training. In this manner, anatomy awareness may be “fed” into the feature extractor FE. The contextual data may include the said segmentation map (e.g. denoting which spatial part belong to which anatomy or tissue type), or may include more generic information about a body region. Anatomy awareness may be exploited when creating the feature map. Another example of such contextual data may include specifications of a-priori material compositions (or ranges thereof), such as chemical or molecular composition tables may be used instead of or in addition to segmentation maps. For example: bone tissue has certain composition characteristics that are different from muscle tissue or blood. Such compositional properties can be exploited as boundary conditions or regularization mechanisms in then learning phase in which the feature maps are build up as the material composition are related to certain photoelectric and Compton scatter ratios.

In embodiments, the combiner CMB may be part of the spectral processing system SPS. Specifically, the combiner CMB may be incorporated in the spectral reconstructor SR. The spectral reconstructor SR may implement a spectral -reconstruction algorithm. The algorithm may proceed iteratively from an initial estimate for the to be reconstructed spectral image in image domain. Voxels in image domain are pre-populated with constant/random values to define the initial estimate for example. A forward-projector is used to map at some or each iteration the current image estimate into projection domain. An updater function processes a deviation, if any, of the forward- projected, estimated, projection imagery from the original projection imagery l_Eΐ,_E2, as acquired. Based on the deviation, if any, current values at voxels in image domain are adjusted, and so on, until a stopping condition is fulfilled. The estimate at this iteration cycle is then output as the final reconstructed spectral image. In embodiments where the combiner CMB is part of the spectral processing system SPS, the combiner parameters a, may be incorporated weights, factors or as other parameters into the re-construction algorithm itself. For example, the combiner parameters may be used in the forward-projector and/or in the update function. The combiner parameters may be incorporated as weights in a weighted fdtered back-projection algorithm from raw projections L_EI,E2.

Other algorithms in which the computed combiner parameters a, may be used include any one or more of: decomposition algorithms from basis components images, weighted spectral reconstruction algorithms, and noise modelling algorithms from basis images. In addition or instead, the combiner parameters may be used in image blending algorithms such as those based on monoenergetic, poly-energetic, and material decomposition images.

The various reconstruction algorithms may operate as a mapping from projection domain to image domain, whilst blending type algorithms remain in the respective data spaces, so map from projection domain to projection domain or from image domain to image domain. The combiner CMB may be implemented herein in the context of a (tomographic) reconstructor or a blender algorithm.

Referring now in more detail to the manner in which the combiner parameters a, are generated, reference is now made to inset Figure 3B. Figure 3B confers a schematic view of details of the inner workings of a given one of the machine learning models Mj .

In embodiments, each machine learning model Mj has been trained previously, as will be described in more detail below, on training image data procured from an existing stock of medical images of prior patients for example. The training data TD may be queried from various medical image repositories, such as a PACS or other. In training, model parameters Q of such ML models are adapted based on an optimization procedure implemented by a computerized training system TS. In certain types of such machine learning models, such as neural network type models etc, the machine learning parameters are sometimes referred to as weights. The weights are arranged in cascaded fashion in different layers Li as illustrated in Figure 3B.

The number of intermediate hidden layers is at least one, such as 1-5, or any other number. Only two layers LI, L2 are illustrated in Figure 3B as an example. The number may run into double-digit figures and may depend on the complexity of the task.

Preferably, some or all of the intermediate layers Li are convolutional layers, that is, include one or more convolutional filters CV which process an input feature map from an earlier layer into intermediate output, sometimes referred to as logits. An optional bias term may be applied by addition for example. An activation layer processes in a non-linear manner the logits into a next generation feature map which is then output and passed as input to the next layer, and so forth. The non-linear activation layers may be implemented as a rectified linear unit ReLU as shown, or as a soft-max- function, a sigmoid-function, to/r /7-function or any other suitable non-linear function. Optionally, there may be other functional layers such as pooling layers or batch normalization layer, or drop-out layers to foster more robust learning. The pooling layers reduce dimension of output whilst drop-out layer sever connections between nodes from different layers to regularize the training process.

Each hidden L_m layer and optionally the input layer IL implements one or more such convolutional operators CV. Each layer L_m may implement the same number of convolution operators CV or the number may differ for some or all layers.

The convolutional operator CV implements a convolutional operation to be performed on its respective input. The convolutional operator may be conceptualized as a convolutional kernel. It may be implemented as a matrix including entries that form fdter elements (the said weights). The fdter weights may be part of the model parameters Q. It is in particular these weights that are adjusted in the training phase. The first layer IL processes, by way of its one or more convolutional operators, the input data such as the spectral data Xj . Feature maps are the outputs of convolutional layers, one feature map for each convolutional operator in a layer. The feature map of an earlier layer is then input into the next layer to produce feature maps of a higher generation, and so forth until the last layer OL represents the final output. The final output is of lesser interest herein, as the combiner parameters are, or are derived from, the feature map at a given intermediate layer Li. Which intermediate layer is used is user adjustable. Preferably, intermediate layers within 1 -3 layers of the central layer is/are used. The central layer defines latent space (to be described in more detail below at Figure 5). The feature map may be collected from latent space for example, or from other intermediate layers, upstream or downstream latent space.

As proposed herein, the combiner parameters may be derived by a layer output extractor LOE from intermediate output generated by a certain one (or more) of the intermediate layers Li. The (or a different) user interface UI may be provided that may allow user to control from which of the layers in the cascade of layers the extraction should take place. The intermediate output includes one or more of the feature maps FM. Thus, the feature maps are generated as intermediate output of the intermediate layers as the input spectral data X_j of the applicable type is applied to the input layer and propagates though the intermediate layer to the output layer OL. The respective spectral input data may be applied in parallel to their corresponding machine leaning model M_j, to obtain the respective feature map FM_j.

If the dimension of the intermediate output corresponds to the size and dimension of the input data XJ, the extractor LOE may operate as a simple pick-up device that copies the output generated at the desired layer as is to provide the feature map FM. In some, preferred embodiments however, the dimension/size of the feature maps are generally different from dimension/size of the input Xj. This is because data dimension is changed during processing across the layers so that the dimension/size of the feature map may increase or decrease relative to dimension/size of input XJ.

For example, the spectral input may have size n x m, whilst the feature map FM at a selected layer / has a smaller or bigger dimension kx l , with k ¹ m and/or l ¹ n. In such a situation, the layer output extractor LOE operates to scale the feature map FM so as to match the size and/or dimension of the input spectral data Xj . The scaling may include interpolation when scaling for increased size, or may include averaging when decreasing size/scale.

Each feature map is in general of matricial or tensor data type, with corresponding dimension such as 2D or 3D, or higher. The magnitude of the feature map entries may represent fdter output at that layer. The magnitude of the feature map entries may represent the importance for the respective learning task that the model has learned to attach to a particular spatial feature. The entries may represent the logits of a given intermediate layer. The entries may represent output of a convolution filter CV of that layer. Alternatively the entries may represent the logits processed by the respective non-linear activation layer of the given convolution layer Li.

As each model M_j has been trained from respective different spectral datatypes j, the feature maps FMj will in general differ because features of different materials and/or structures, tissues etc will present in better contrast in one type of spectral data than they do in others. In this way, each feature map represents the importance of features or image structures to which the respective spectral data part is particularly suited for. The combiner parameters may be based on the said features maps FMj as generated by the different ML models My on processing the respective spectral data type. Combining the input spectral data X j according to combiner parameters j based on the feature maps FM_j may thus allow local spatial adaptation of contrast to the particular tissue/material represented at a particular image location.

The respective combiner parameter a j may be represented by the respective feature map FM_j, possibly suitably scaled to match dimension/size of spectral data type X j. In other embodiments, a post processing is applied to the feature maps to obtain derived feature maps such as heat maps. Such post-processing can may be done by gradient based analysis, or by a reshaping of the feature map followed by a non-linearity operation. In gradient based analysis, each entry in the derived feature map (such as the heat map) may represent the rate of change across parameters in the respective layer.

Specifically, 2D methods have been previously described to create activation/heat maps from intermediate feature maps, including the Grad-CAM method and methods based on deconvolutional operations as described for example in R. R. Selvaraju, et al in “ Grad-CAM : Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization ,” Comput. Vis., (2016) and Zeiler M.D., Fergus R. “Visualizing and Understanding Convolutional Networks” in. Fleet D et al (eds), “Computer Vision - ECCV 2014. ECCV 2014. Lecture Notes in Computer Science”, vol 8689, Springer, pp 818-833(2014).

For example, the Grad-CAM method is based on calculating a gradient of a class with respect to the last multi-dimensional convolutional layer. Weights of some or each filter output at a given layer Li is then obtained by averaging, such as global average pooling. The heatmap may be generated by applying the weights to the corresponding filter outputs and then applying a non- linearity layer (e.g., ReLU), sigmoid, other other). The result may then be normalized and/or scaled to the desired size.

The feature maps , or the derived feature maps, may be normalized by LOE across data types j so that the combiner parameters

cc_j = 1 add up to unity over spectral data type j.

The block diagram in Figure 4 illustrates operation of feature constructor FE. Only two types of spectral image data XI, X2 are combined for illustration. Spectral data of different types X j, {j >1) (two types are shown as an example, but there may be more) are received at input port IN. Each type of data X j is applied to their respective/corresponding machine learning model MJ as selected by selector SL. The models were previously trained on spectral data of the same respective typej.

Processing of the respective data type XI, X2 by the respective model Ml, M2 generates respective feature maps f, w, at the same or at different hidden layers. The feature maps may then be used by layer extractor LEO to derive the respective combined parameter al, a2 for the

LOE LOE respective data types XI, X2: f — > al, w — > a2. The layer extractor LOE may apply a scaling and/or normalization. Instead of using the “bare” feature maps themselves as combiner parameters, a feature map processor FMP may be used to compute derived feature maps as described above, such as heat maps, other gradient or non-gradient-based maps, etc. The derived maps may also need to be normalized by extractor LEO or feature map processor before they can be used as combiner parameters.

Natively, or thanks to scaling, the respective set of combined parameters al, a2 each have the same dimension/size as the respective input data XI, X2. The combiner parameters al, a2 are used by the combiner CMB to combine the input spectral data XI, X2 to derive the improved image X+. The combined image X+ may then be visualized. For example, the combiner CMB may implement an image blending algorithm. The combination operation using the combiner parameters al, a2 may be implemented by combining the image data based on a mapping function C( ^'·,·). For instance, image data can be blended using weight-based image blending:

X+ = alXl+ 0.2X2 (1) wherein a2=l- al, and al is a preferably pixel-wise feature map. As before, X1,X2 are the input spectral data to be combined. Other combination functions C( y ) than of the linear type as in (1) include weighted Laplacian/Gaussian pyramid blending. In Laplacian/Gaussian pyramids, the “weights” are the features maps or derived feature maps, suitably represented at different scales and preferably normalized. Selective, region-based weighted blending may also be contemplated, as opposed to a fine-grained localized approach such as pixel/voxel-wise weighting. The earlier mentioned user interface UI may be arranged as a mouse, touch screen, augmented reality goggles/headsets, or of any other type. For instance, user may wish to change a blending ratio as per a computed feature map FM. The blending ratio refers to the importance of a feature in a feature map that is applied for instance to blend two image data sets. In some examples, a blending ratio may be represented as a number from 0 to 1 or percentage and can be applied to rescale the feature map. The blending ratio may be a single scalar value or a vector of values representing the importance of features in different regions of the images or at a per-pixel level as mentioned earlier. Users may use a single slider of the user interface USI to adjust this ratio which could automatically control how the values change in each region or pixel. Alternatively, users could have multiple sliders, for instance, one per region of interest. The regions of interest may be identified by the user using the UI, such as by touch screen or pointer tool action or AR interaction. For example, once the FM or derived heat map is calculated, user may wish to change a global blending ratio or modify the localized blending ratio. Specifically, user may use interface UI to change the blending ratio for highly activated features that may represent edges, or user may change the blending ratio for features that are activating on circular/round structures. The changes can be adjusted independently, thanks to using multi-dimensional (2D or 3D, or higher) blending maps/combiner parameters j.

Reference is now made to Figure 5 which illustrates operation of the training systems TS. The training system TS may train the bank of machine learning models for each spectral data type Xj. Training may be formulated as an optimization procedure based on an objective function F. The objective function F may be configured as a utility or loss function. In embodiments, the objective function A is arranged as a loss function and measures the deviation, if any, between the expected output of the machine learning model and the actually computed output. Based on this deviation, as quantified by the loss function, the machine learning parameters Q of the machine learning model are adapted, preferably in iterations, until an iteration stopping condition is met at which point the model is considered sufficiently trained. The training is run separately for each data type j to obtain the different models Mj . Training can be done one-by-one over the training data set, or can be done in batches preferably. Training may be done as a one-off operation initially for a given set of training data, but may be repeated in re-training cycles, when new training data becomes available so as to improve the performance of the models Mj, Training may be done in unsupervised or supervised fashion. Preferably unsupervised learning is used as it is easier to administer.

The machine learning models in Figure 5 are symbolically represented as a turned over triangle/frustum to illustrate the change of dimensions of the feature maps generated whilst processing the input data across the various cascaded layers.

Specific reference is now made to Figure 5A which shows an embodiment of an unsupervised learning system TS, using an auto-encoder or encoder/decoder (E/D)-machine learning model set-up. More specifically, an encoder-decoder type network is schematically shown in Figure 5A. The encoder EC takes in training input data Xj and compresses it to a low-dimensional representation in latent space £. In the training, the decoder part DC of the network is made to learn reconstructing the input image, using the low-dimensional representation in latent space. Parameters of both networks parts, the encoder EC and decoder DC, are adjusted by minimizing the reconstruction error:

F^e = D(Xj, M^e(Xj)) (2)

In (2). is the objective function, in this case a cost function, and D is distance measure, and M the model, in this case the encoder-decoder. For a given model Mj, the training input Xj. s are of the same spectral data type j, and the learning is done separately per model Mj and spectral data type X\ . The parameters Q to be learned may include the fdter weights of the convolutional fdter in the various layers Li. Overall internal setup of the layers Li are as in the convolutional NN architecture as discussed above in connection with Figure 3B.

The encoder-decoder setup Figure 5A and (2) forces the encoder EC, during optimization, to discover high level salient features that can represent the input data X\ in a multi dimensional feature space represented by the feature map as generated at each intermediate layer. Preferably, the feature map representation of the training input is selective and sparse at the same time in order to compress the input Xj into such a low-dimensional representation. Suitable regularization functions R may be used in (2) , F = D+R.

After training, the feature maps may be collected at latent space £, but may be collected at any other intermediate layer /., before or after the (central) layer that outputs a representation in latent space £. The learning is preferably done in batches. The parameters Q in (2) are adjusted so that the sum of reconstructions errors of the training data set is considered, such as minimized. In Figure 5A, a variational encoder-decoder may be used instead, which (additionally) learns a distribution over the learned latent space £.

After training, it is the encoder part EC that is of main interest herein and it is this part that may be used as the respective model MJ for the feature extractor FE for spectral type j . The decoder DC part may be disregarded for this purpose. Memory space may be so conserved. For re training however, the decoder DC may still be required. In the alternative, it is the decoder DC part that used in the feature extractor FE. If memory savings are of minor concern, both the decoder DC and encoder EC are used in the feature extractor FE.

Referring now to a supervised learning embodiment, reference is now made to Figure 5B. The neural network Mj may be trained on any arbitrary task. The network may be chosen as a CNN as mentioned earlier in Figure 3B. As in Figure 5 A, training is done separately for each spectral data type j, to so obtain at least one trained model Mj for each spectral data type j of interest. The learning procedure may be formalized in terms of an objective function as:

F« = D(?_j, M^e(¾)) (3) where the Ij’s are the targets associated with each training input specimen Xj. As in Figure 5 A, after training, output from the intermediate convolutional layers Li results in an intermediate feature map.

The feature map describes the input in a feature space, that is used by the network to make a prediction. The prediction can be any task, such as a classification of the input image into high- or low-quality. Other tasks may include any one of i) binary or multi-class classification (malignant vs. benign tumor), ii) object detection, localization or segmentation (tumor localization), iii) image quality or similarity assessment (good vs. low-quality acquisition), iv) image denoising or artefact removal using generative networks such as variational autoencoder (VAE), v) style transfer using generative adversarial networks (GAN). The localization/segmentation in option ii) is one example of introducing anatomical information, anatomical awareness, into FM extractor as mentioned above.

The distance function D in (3) is configured based on the task, so may include cross entropy for classification tasks, or LI - or L2 norms such as Euclidean Distance, for regression type tasks. As in (2), the objective function Ain (3) may include summation over the training data specimens and their targets. Generally, the task layer OL can be configured for any task, so long as the task performance is dependent on image structures as encoded by the image. A segmentation task may be preferred ii) however, to foster anatomy awareness.

Whilst the above has been explained with main reference to feature maps in the context of artificial neural network type models, other models are also envisaged herein, where spatially associable/distributed parameters are learned.

Figure 6A is an illustration of feature maps as may be derived as intermediate layers of a machine learning model, in particular of the neural network type.

Figure 6A illustrates a two-dimensional example, that is, the feature map is a 2D matrix, whereas Figure 6C illustrates a 3D example with feature map being a 3D matrix with entries indexed by three numbers.

In more detail, pane A shows a 2D feature map. Figure 6B shows the input spectral image Xj that was fed into the model and gave rise to the feature map in pane A. Specifically, feature map of pane A was generated in response to processing by ML model, the input image B. As can be seen, spatial image structures are clearly correlated to modulations in the feature map. A similar phenomenon can be observed in the 3D example of Figures 6C, D. Pane D is a 3D input spectral data/image type, such as a re -constructed spectral volume, whilst pane C illustrates the respective feature map, again with correlating image structures in D represented by modulations in the feature map C.

Reference is now made to Figure 7 which shows a flow-chart of a method that may be used to implement the above-described visualizations system for spectral data. It will be understood however that the described method steps are not necessarily tied to the architecture described above, but may be understood instead as a teaching in its own right.

At step S710, spectral data Xj is received, previously generated by a spectral imaging system or arrangement, for example as described above in Figures 1 and 2.

Consistent with the above used notation, / may represent different energy levels, ranges or others, or may index different spectral datatypes as described above in Figure 2. Two or more datatypes are received at Step S710.

At an optional step S720, based on the respective spectral data type j, a pre-trained machine learning model is identified and selected in the bank of machine learning models including differently trained models, at least one for each data type j .

At step S730 the spectral data Xj is applied to a corresponding, trained machine learning model for the respective spectral datatype j and as selected at step S720. The respective model processes the input data to generate a respective feature map as described above.

At optional step S740, the one or more feature maps so generated are processed into respective sets of combiner parameters j. The processing may include suitable scaling and/or normalization The feature maps themselves may be used as combiner parameters qj, so step S740 is optional. Other processing may be used in step S740 to generate derived feature maps, such as activation maps (for example, heat maps), based on gradient-based processing of the feature maps as desired.

At step S750, the input data Xj is then combined using the combiner parameters and a combiner function C() to obtain a combined image X+. The combination operation may be embedded in a spectral reconstruction algorithm or in other data processing algorithms applicable to spectral data.

At step S76, the combined image X+ is then output. Output may include using the combined image to drive a graphics display generator to generate visualization of the improved image X+ on a display device. In addition, or instead, output may include storing or processing the improved image X+.

The components of the data processing system DPS including the visualizer SYS-v may be implemented as one or more software modules, run on one or more general-purpose processing units PU such as a workstation associated with the imager IA, or on a server-computer associated with a group of imagers.

Alternatively, some or all components of the visualizer SY S-v may be arranged in hardware such as a suitably programmed microcontroller or microprocessor, such an FPGA (field- programmable-gate-array) or as a hardwired IC chip, an application specific integrated circuitry (ASIC), integrated into the imaging system SIS. In a further embodiment still, the data processing system DPS including the visualizer SYS-v may be implemented in both, partly in software and partly in hardware.

The different components of the DPS including the visualizer SYS-v may be implemented on a single data processing unit PU. Alternatively, some or more components are implemented on different processing units PU, possibly remotely arranged in a distributed architecture and connectable in a suitable communication network such as in a cloud setting or client- server setup, etc.

One or more features described herein can be configured or implemented as or with circuitry encoded within a computer-readable medium, and/or combinations thereof. Circuitry may include discrete and/or integrated circuitry, a system-on-a-chip (SOC), and combinations thereof, a machine, a computer system, a processor and memory, a computer program.

In another exemplary embodiment of the present invention, a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.

The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention. This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above-described apparatus. The computing unit can be adapted to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method of the invention.

This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.

Further on, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above.

According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.

A computer program may be stored and/or distributed on a suitable medium (in particular, but not necessarily, a non-transitory medium), such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.

It has to be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters is considered to be disclosed with this application. However, all features can be combined providing synergetic effects that are more than the simple summation of the features.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.

In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items re-cited in the claims. The mere fact that certain measures are re cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. System (SYS-v) for image processing, comprising an input interface (IN) for receiving plural spectral input data generated by a spectral X-ray imaging system (SIS); at least one feature extractor (FE) configured to process one type of the plural spectral input data and generate a feature map corresponding to the processed type of spectral input data; a combiner (CMB) configured to combine, in a combination operation, the plural spectral input data into combined data, the combination operation controlled by a combiner parameter that is based on feature maps previously generated by the one or more feature extractor (FE); and a graphics display generator (g-OUT) to cause visualizing the combined data on a display device (DD).

2. System of claim 1, comprising a selector (SL) configured to select a corresponding one of the one or more feature extractor (FE), based on the type of the spectral data.

3. System of any one of previous claims, wherein the combiner (CMB) implements a reconstruction algorithm or a blender algorithm.

4. System of any one of previous claims, wherein the one or more feature extractor (FE) is implemented as a machine learning model (M).

5. System of claim 4, wherein the machine learning model (M) is of the artificial neural network type.

6. System of claim 5, wherein the feature map includes output generated at a hidden layer (Li) of the artificial neural network type model.

7. System of any one of the previous claims, wherein the combiner parameter is multi dimensional.

8. System of any one of the previous claims, including a user interface (UI) allowing a user to modify the combiner parameters.

9. System of any one of the previous claims, wherein the graphics display generator (g-

OUT) is configured to effect visualizing the feature map on the display device or on a different display device.

10. System of any one of the previous claims, wherein the feature extractor (FE) is configured to account for prior knowledge such as anatomical information.

11. Method of image processing, comprising: receiving (S710) plural spectral input data generated by a spectral X-ray imaging system (SIS); processing at least one type of the plural spectral input data by one or more feature extractors (FE); combining (S750), in a combination operation, the plural spectral input data into combined data, the combination operation being controlled by combiner based on feature maps generated in the processing step; and visualizing (S760) the combined data on a display device (DD).

12. A computer program element, which, when being executed by at least one processing unit, is adapted to cause the at least one processing unit (PU) to perform the method as per claim 11.

13. At least one computer readable medium having stored thereon the program element of claim 12