CN117083629A

CN117083629A - Machine learning in the field of contrast-enhanced radiology

Info

Publication number: CN117083629A
Application number: CN202180096431.0A
Authority: CN
Inventors: M·伦伽; M·普特奥拉布
Original assignee: Bayer AG
Current assignee: Bayer AG
Priority date: 2021-03-09
Filing date: 2021-11-29
Publication date: 2023-11-17

Abstract

The present invention relates to the field of creating artificial contrast enhanced radiological images by means of machine learning methods.

Description

Machine learning in the field of contrast-enhanced radiology

The present invention relates to the technical field of generating artificial contrast enhanced radiological images by means of a machine learning method.

Radiology is a medical field involving imaging for diagnostic and therapeutic purposes.

X-rays and films sensitive to X-rays have been used primarily for medical imaging, while radiology today includes a number of different imaging methods such as, for example, computed Tomography (CT), magnetic resonance imaging (MRT), or ultrasound imaging.

For all of these methods, substances may be utilized to help delineate or define certain structures in the examination object. Said substance is called a contrast agent.

In computed tomography, an iodine-containing solution is generally used as a contrast agent. In magnetic resonance imaging (MRT), superparamagnetic substances (e.g. iron oxide nanoparticles, superparamagnetic Iron Platinum Particles (SIPPs)) or paramagnetic substances (e.g. gadolinium chelates, manganese chelates) are often used as contrast agents.

The contrast agents can be broadly classified into the following categories according to their diffusion pattern in tissue: extracellular contrast agent, intracellular contrast agent, and intravascular contrast agent.

Extracellular MRT contrast agents include, for example, gadolinium chelates, gadobutrolGadolinium terrloGadoterate->Gadofoshan->And gadolinium bis-amineEtc. The high hydrophilicity of the gadolinium chelates and their low molecular weight allow rapid diffusion into the interstitial space after intravenous injection. After a relatively short period of time in the blood circulation system, they are excreted through the kidneys.

Intracellular contrast agents are taken up to some extent by tissue cells and then excreted.

Intracellular MRT contrast based on gadocetetic acidAgents are for example characterized in that they are taken up in a specific proportion by liver cells (hepatic parenchymal cells), accumulate in functional tissue (parenchyma), enhance the contrast of healthy liver tissue, and subsequently they are excreted into the faeces through the gall bladder. An example of a gadolinium-based contrast agent is described in US6,039,931A; they can be given, for example, under the trade nameOr->Are purchased commercially. An additional MRT contrast agent with a lower rate of hepatic parenchymal cell uptake is gadobenate dimeglumine +>

//>Is mediated by the stable gadolinium complex Gd-EOB-DTPA (disodium gadocetetate). DTPA forms a complex with extremely high thermodynamic stability with paramagnetic gadolinium ions. Ethoxybenzyl (EOB) radicals are mediators of hepatobiliary uptake of contrast agents.

Intravascular contrast agents are distinguished by a significantly longer residence time in the blood circulation system compared to extracellular contrast agents. For example, gadolinium fossilizid is a gadolinium-based intravascular MRT contrast agent. It is in the form of trisodium salt monohydrateIs used. It binds to serum albumin, thus achieving a long residence time of the contrast agent in the blood circulation system (half-life in blood of about 17 hours).

There are a variety of radiological examinations in which a contrast agent is administered to a patient and the dynamic diffusion of the contrast agent in the body is tracked by means of an imaging method. One example that may be mentioned is the detection and differential diagnosis of focal liver lesions by means of dynamic contrast enhanced magnetic resonance imaging.

Can be used for detecting tumors in the liver. The blood supply of healthy liver tissue is mainly achieved via the portal vein (Vena portae), while the hepatic artery (Arteria hepatica) supplies most primary tumors. After intravenous injection of the contrast agent in bolus form, a time delay between the signal rise of healthy liver parenchyma and the signal rise of the tumor can be observed accordingly.

In addition to malignant tumors, benign lesions such as cysts, hemangiomas and Focal Nodular Hyperplasia (FNH) are often found in the liver. Proper treatment planning requires distinguishing these benign lesions from malignant tumors. Can be used for distinguishing benign and malignant focal liver lesions. By T1 weighting the MRT, information about the characteristics of the lesion is provided. Differentiation is achieved by utilizing different blood supply of the liver from the tumor and a time cross-sectional line of contrast enhancement.

By passing throughThe contrast enhancement achieved can be divided into at least two periods: dynamic phase (including so-called arterial phase, portal phase and late phase) and hepatobiliary phase (in which hepatic parenchymal cell pairs have occurred)Is significantly ingested).

During the distribution period byIn the example of contrast enhancement achieved, it is observed that a typical perfusion providing information for characterizing lesionsAnd (5) a mode of injection. Delineating vascularization helps to characterize lesion type and determine spatial relationship between tumor and blood vessel.

In the example of a T1 weighted MRT image,10-20 minutes after injection (in hepatobiliary phase) resulted in a significant signal enhancement in healthy liver parenchyma, whereas lesions containing no or only small amounts of liver parenchymal cells, such as metastatic or medium to poorly differentiated hepatocellular carcinoma (HCCs), were shown as darker areas.

Thus, tracking the diffusion of contrast agents over time in the dynamic and hepatobiliary phases provides a good possibility for detection and differential diagnosis of focal liver lesions; however, the examination extends over a relatively long time span. During the time span, movement of the patient should be largely avoided, minimizing movement artifacts in the MRT image. Long movement restrictions can be uncomfortable for the patient.

The present invention solves this and other problems. The present invention provides an apparatus that enables the composite generation of one or more radiological images. The synthetically generated radiological images are predicted by means of a machine learning model based on a time series showing the change over time of contrast enhancement of the measured radiological images. This has the advantage that the radiological examination can be accelerated since there is no need to measure all radiological images important for diagnosis; one or more radiological images may be predicted (calculated) based on the measured radiological images; the inspection time can be shortened. In addition, the prediction of the radiological image is done in frequency space (rather than in real space as is usual). As a result, contrast information can be separated from detail information in the radiological characterization of the examination region, such that training and prediction is limited to the contrast information, which is then reintroduced after prediction. This procedure reduces e.g. computational complexity. Furthermore, operating in frequency space means having a higher tolerance to image alignment defects.

In a first aspect, the present invention provides a computer-implemented method comprising the steps of:

Receiving a plurality of first representations of an examination region of an examination subject in a frequency space, wherein at least some of the first representations represent the examination region during a first time span after administration of a contrast agent,

feeding a plurality of first characterizations to a predictive model, wherein the predictive model has been trained based on a first reference characterization and a second reference characterization of an examination region of a plurality of examination objects, such that one or more second reference characterizations are generated from the first reference characterization, wherein at least some of the first reference characterizations represent examination regions in frequency space during a first time span after administration of a contrast agent, the one or more second reference characterizations represent examination regions in frequency space during a second time span,

receiving one or more predictive characterizations of the examination region in frequency space from the predictive model, wherein the one or more predictive characterizations represent the examination region during a second time span,

transforming the one or more predictive characterizations into one or more characterizations of the examination region in real space,

-outputting and/or storing one or more characterizations of the examination region in the real space.

The invention also provides a computer system comprising:

a receiving unit which receives the signal from the receiving unit,

control and calculation unit, and

an output unit which outputs the output signal of the first output unit,

wherein the control and calculation unit is configured to:

causing the receiving unit to receive a plurality of first representations of an examination region of an examination subject in a frequency space, wherein at least some of the first representations represent the examination region during a first time span after administration of a contrast agent,

feeding the plurality of first characterizations to a predictive model, wherein the predictive model has been trained based on first reference characterizations of an examination region of a plurality of examination objects and second reference characterizations, such that one or more second reference characterizations are generated from the first reference characterizations, at least some of the first reference characterizations representing examination regions in frequency space during a first time span after administration of a contrast agent, the one or more second reference characterizations representing examination regions in frequency space during a second time span,

-causing the output unit to output and/or store one or more representations of the examination region in the real space.

The present invention also provides a computer program product comprising a computer program, the computer program being loadable into the internal memory of a computer system, wherein the computer program causes the computer system to perform the steps of:

The invention also provides the use of a contrast agent in a method for predicting at least one radiological image, wherein the method comprises the steps of:

generating a plurality of first representations of an examination region of an examination object in a frequency space, wherein at least some of the first representations represent the examination region during a first time span after administration of a contrast agent,

Also provided is a contrast agent for use in a method for predicting at least one radiological image, wherein the method comprises the steps of:

administering a contrast agent, wherein the contrast agent diffuses in an examination region of an examination subject,

Also provided is a kit comprising a contrast agent according to the invention and a computer program.

The invention is explained in more detail below without distinguishing the subject matter of the invention (method, computer system, computer program, use, contrast agent used, kit of parts). Rather, the following description is intended to apply analogously to all subject matter of the present invention, regardless of the context in which they appear (method, computer system, computer program, use, contrast agent used, kit of parts).

If steps are recited in one order in the present specification or claims, this does not necessarily mean that the present invention is limited to the recited order. Rather, it is contemplated that the steps may also be performed in a different order or in parallel with one another, unless one step is built on another, which would absolutely require the built steps to be performed later (however, this would be clear in the individual case). Thus, the order stated may be a preferred embodiment.

The invention makes it possible to shorten the time span for performing a radiological examination of an examination object.

The "subject" is usually a living body, preferably a mammal, particularly preferably a human.

An "examination region" is typically a part of an examination object, for example an organ or an organ part. The "examination region" is also referred to as the image volume (field of view, FOV), in particular the volume imaged in the radiological image. The examination region is usually determined by the radiologist, for example on an overview image (in english: localizer). Of course, the examination region may also alternatively or additionally be defined automatically, for example based on a selected protocol.

The term "radiological examination" is understood to mean all imaging methods which allow a thorough knowledge of the examination object by means of electromagnetic radiation, particle radiation or mechanical waves, for diagnostic, therapeutic and/or scientific purposes. In the context of the present invention, the term "radiology" specifically encompasses the following examination methods: computed tomography, magnetic resonance imaging, ultrasound imaging, positron emission tomography, echocardiography, and scintigraphy.

In a preferred embodiment of the invention, the radiological examination is a magnetic resonance imaging examination.

Magnetic resonance imaging, MRT or MR (english: MRI: magnetic Resonance Imaging) is an imaging method which is used, inter alia, in medical diagnostics for delineating the structure and function of tissues and organs in the human or animal body.

In MR imaging, the magnetic moments of protons in an examination object are aligned in a basic magnetic field, so that macroscopic magnetization exists along the longitudinal direction. This is then deflected from the rest position by the incident radiation of a high-frequency (HF) pulse (excitation). The recovery (relaxation) or magnetization dynamics from the excited state to the rest position is then detected as a relaxation signal by one or more HF receiver coils.

For spatial encoding, a rapidly switching magnetic gradient field is superimposed on the basic magnetic field. The acquired relaxation signals or the detected and spatially resolved MR data are initially presented as raw data in a spatial frequency space and can be transformed into a physical space (image space) by a subsequent inverse fourier transformation.

For the original MRT, tissue contrast is generated from different relaxation times (T1 and T2) and proton densities. T1 relaxation describes the transition of the longitudinal magnetization to its equilibrium state, T1 being the time required to reach 63.21% of the equilibrium magnetization before resonance excitation. It is also known as the longitudinal relaxation time or spin lattice relaxation time. Similarly, T2 relaxation describes the transition of transverse magnetization to its equilibrium state.

In radiology, contrast agents are often used to enhance contrast.

A "contrast agent" is a substance or mixture of substances used in imaging methods such as X-ray diagnosis, magnetic resonance imaging and ultrasound imaging to improve the delineation of the structure and function of the human body.

Examples of contrast agents can be found in the literature (see, e.g., A.S.L.Jascith et al: contrast Agents in computed tomography: A Review, journal of Applied Dental and Medical Sciences,2016, volume 2, 2 nd, 143-149; H.Lucic et al: X-ray-Computed Tomography Contrast Agents, chem.Rev,2013, 113,3, 1641-166; https:// www.radiology.wisc.edu/wp-content/uplads/2017/10/content-agents-tutorial. Pdf, M.R.Nough et al: radiographic and magnetic resonances contrast agents: essentials and tips for safe practices, world J radio, 2017, 9 (9): 339-349; L.C.Abanyi et al: intravascular Contrast Media in Radiography: historical Development & Review of Risk Factors for Adverse Reactions, south American Journal of Clinical Research,2016, volume 3, 1 st, 1-10;ACR Manual on Contrast Media,2020,ISBN:978-1-55903-012-0; A.Ignee et al: ultrasound contrast agents, endosc Ultrasound,2016, 5, 355-355, 5 (6).

MRContrast agents exert their effect by modifying the relaxation time of the structure in which the contrast agent is taken up. A distinction can be made between two groups of substances: paramagnetic substances and superparamagnetic substances. Both groups of substances have unpaired electrons that induce a magnetic field around a single atom or molecule. Superparamagnetic contrast agents lead to a significant reduction of T2, whereas paramagnetic contrast agents mainly lead to a reduction of T1. The effect of the contrast agent is indirect in that the contrast agent itself does not emit a signal, but only affects the signal intensity of the hydrogen protons surrounding it. One example of a superparamagnetic contrast agent is iron oxide nanoparticles (SPIO, english: superparamagnetic iron oxide (superparamagnetic iron oxide)). An example of a paramagnetic contrast agent is a gadolinium chelate such as gadofoshan (trade name:etc.), gadoteric acid->Gadolinium diamine->Gadoteridol->And gadobutrol->

By means of the invention, one or more synthetic radiological images of the examination region can be predicted. The prediction is done by means of a prediction model.

The prediction model is a computer-aided model configured to predict one or more second characterizations of an examination region of the examination object in frequency space based on the plurality of first characterizations of the examination region of the examination object in frequency space. At least some of the plurality of first characterizations represent an examination region during a first time span after administration of a contrast agent. The at least one second characterization represents an examination region during a second time span.

The term "plurality" means a number of at least two. The plurality of first characterizations used for prediction typically does not exceed ten.

The second time span may be before the first time span or after the first time span. It is conceivable that the time spans overlap at least partially, or that one time span is within another time span.

Fig. 1 (a), 1 (b), 1 (c) and 1 (d) are for illustration purposes only. Fig. 1 (a), 1 (b), 1 (c) and 1 (d) depict time lines, respectively. The defined time points are marked on the time line. Time point t ₀ The time point at which the contrast agent is administered to the subject is indicated. The dashed box shows the time span over which the examination object is subjected to a radiological examination, i.e. the time the examination object stays in a magnetic resonance imaging system or a computed tomography system, for example.

Fig. 1 (a) schematically illustrates a typical overview of a radiological examination. The examination object is introduced into a tomographic scanner. At time point t _-1 Where the examination object is located in a tomography scanner. At time point t _-1 At this point, a first radiological image is generated, i.e. a representation of the examination region of the examination object is generated. At the time point (t _-1 ) Here, no contrast agent has been administered to the examination subject, i.e. the characterization is a contrast agent-free (original) characterization. At time point t ₀ A contrast agent is administered to an examination object located in a tomographic scanner. At time point t ₁ 、t ₂ And t ₃ A further characterization of the examination object is generated. After that, the inspection object leaves the tomographic scanner. The object under examination is located in the tomographic scanner for a relatively long time span T _t . During this time, four characterizations of the examination region are generated based on the measurements.

Fig. 1 (b) schematically illustrates an overview of a radiological examination according to the present invention. The examination object is introduced into a tomographic scanner. At time point t _-1 Where the examination object is located in a tomography scanner. At time point t _-1 At which a first characterization of an inspection area of an inspection object is generated. At the time point (t _-1 ) At the position of the first part,no contrast agent has been administered to the subject, i.e. the characterization is a contrast agent-free (original) characterization. At time point t ₀ Where a contrast agent is applied to an examination object located in a tomographic scanner. At time point t ₁ And t ₂ At which a further characterization of the examination object is generated. After that, the inspection object leaves the tomographic scanner. In the overview example shown in FIG. 1 (b), the example based on FIG. 1 (a) is shown at time t ₃ The characterization of the measurement generation at that location is predicted (calculated). The object under examination being located in the tomographic scanner for a time span T _a Said time span T _a Shorter than the time span T in FIG. 1 (a) _t . Thus, in the example of fig. 1 (b), the duration of the radiological examination is less than in the example of fig. 1 (a), and therefore the examination subject is more comfortable; however, in both cases, the generated representation of the examination region is indicative of the time t _-1 、t ₁ 、t ₂ And t ₃ An examination region at the location. Based on the time point t _-1 、t ₁ And t ₂ The characterization generated by the measurement results at is a first characterization of the examination region of the examination object in the context of the present invention, wherein at least some represent the examination region during a first time span after administration of the contrast agent, i.e. the point in time t ₁ 、t ₂ And t ₃ Characterization of the site; at time point t _-1 The characterization at this point represents the examination region in a time span prior to administration of the contrast agent. At time point t ₃ The characterization at is based on the characterization t by the predictive model according to the invention _-1 、t ₁ And t ₂ Predicted by the user. It represents the examination region during a second time span, which in the example of fig. 1 (b) follows the first time span.

Fig. 1 (c) schematically illustrates another overview of a radiological examination according to the present invention. At time point t ₀ Where a contrast agent is administered to the subject. At the time point (t ₀ ) At this point, the inspection object is not yet located in the tomographic scanner. Only after administration of the contrast agent, the examination object is introduced into the tomographic scanner. At time point t ₁ 、t ₂ And t ₃ (after administration of contrast agent)In a first time span), a characterization of an examination region of the examination object is generated. These represent the examination region in a first time span after administration of the contrast agent. At time point t ₃ After generating the characterization, the examination object can leave the tomographic scanner; the radiological examination ends. According to time point t ₁ 、t ₂ And t ₃ Characterization at time t can be predicted _-1 Characterization of the site. Time point t _-1 The characterization at represents the examination region during a second time span, which precedes the first time span. The object under examination being located in the tomographic scanner for a time span T _b Said time span T _b Shorter than the time span T in FIG. 1 (a) _t . Thus, in the example of fig. 1 (c), the duration of the radiological examination is less than in the example of fig. 1 (a), and therefore the examination subject is more comfortable; however, in both examples, generated is a representation of the examination region, representing at the point in time t _-1 、t ₁ 、t ₂ And t ₃ Is provided.

Fig. 1 (d) schematically illustrates another overview of a radiological examination according to the present invention. This example is intended to make clear that the invention is not limited to a single administration of contrast agent. For example, it is contemplated that there may be two or more administrations. In the case of a single administration, it is also not necessary to administer the same contrast agent; alternatively, a different contrast agent may be administered. At time point t ₀ Where a (first) contrast agent is administered to the examination subject. At the time point (t ₀ ) At this point, the inspection object is not yet located in the tomographic scanner. Only after the contrast agent has been applied, the examination object is introduced into the tomographic scanner. At time point t ₂ At which a first representation of an examination region of an examination object is generated. At time point t ₂ The characterization at this point represents the examination region in a first time span after administration of the (first) contrast agent. At time point t ₃ Where a (second) contrast agent is administered. At the time point (t ₃ ) Where the examination object is located in a tomography scanner. After administration of the (second) contrast agent, two further characterizations of the examination region are generated, one at the time point t ₄ Characterization at the other is at the point in timet ₂ Characterization of the site. At time point t ₂ 、t ₄ And t ₅ The characterization generated here is a characterization representing the examination region during a first time span after administration of the contrast agent. These characterizations can be used to predict the time point t _-1 Characterization of the examination region at and/or at time t ₁ Characterization of the examination region at. At time point t _-1 And t ₁ The characterization of the examination region at is indicative of the examination region during a second time span, which is prior to the first time span.

Combinations of the profiles shown in fig. 1 (b), 1 (c) and 1 (d) and other profiles/variants are equally possible.

In a preferred embodiment of the invention, the first time span starts before or at the time of administration of the contrast agent. It is advantageous when the generated one or more representations of the examination region show the examination region (original image) to which no contrast agent has been applied, since the radiologist is already able to obtain important information about the health of the examination subject from such an image. For example, the radiologist can identify bleeding in the original MRT image.

In order for the prediction model according to the present invention to be able to make the predictions described herein, the prediction model must be appropriately configured in advance.

Here, the term "predictive" means that at least one characterization of the examination region during the second time span in the frequency space, representing the examination region, is calculated using a plurality of first characterizations of the examination region in the frequency space, wherein at least some of the plurality of first characterizations represent the examination region during the first time span after administration of the contrast agent.

In a supervised machine learning process, a predictive model is preferably created (configured, trained) by means of a self-learning algorithm. The training data is used for learning. For each of a plurality of inspection objects, the training data includes a plurality of characterizations of an inspection region. The examination region is typically the same for all examination objects (e.g. a part of a human body, or an organ, or a part of an organ). In this specification, the characterization of the training dataset is also referred to as a reference characterization. The term "plurality" means preferably more than 10, even more preferably more than 100.

For each examination object, the training data includes: i) A plurality of first reference characterizations of the examination region in frequency space, at least some of the first reference characterizations representing the examination region during a first time span after administration of the contrast agent; and ii) one or more second reference representations of the examination region in the frequency space representing the examination region during a second time span.

The predictive model is trained to predict (calculate) one or more second reference characterizations for each inspection object based on the plurality of first reference characterizations.

During machine learning, a self-learning algorithm generates a statistical model based on training data. This means that the embodiments do not simply dead-mark, but rather enable the algorithm to "recognize" patterns and rules in the training data. Thus, the predictive model may also evaluate unknown data. Verification data may be used to test the quality of the assessment of the unknown data.

The predictive model may be trained by means of supervised learning (english: supervised learning), i.e. the algorithm is presented with pairs of data sets (first and second characterization) consecutively, respectively. The algorithm then learns the relationship between the first and second characterizations.

Self-learning systems trained by means of supervised learning are widely described in the prior art (see, e.g., C.Perez: machine Learning Techniques: supervised Learning and Classification, amazon Digital Services LLC-Kdp Print Us,2019,ISBN 1096996545, 9781096996545).

Preferably, the predictive model is or comprises an artificial neural network.

The artificial neural network comprises at least three layers of processing elements: a first layer with input neurons (nodes), an nth layer with at least one output neuron (node), and N-2 inner layers, where N is a natural number greater than 2.

The input neuron is for receiving a first characterization. The output neuron is for outputting one or more second tokens for the plurality of first tokens.

The processing elements of the layer between the input and output neurons are connected to each other in a predetermined pattern with predetermined connection weights.

Preferably, the artificial neural network is a so-called convolutional neural network (abbreviated as CNN).

Convolutional neural networks are capable of processing input data in the form of a matrix. CNNs basically comprise alternating layers of filtering (convolutional layers) and polymeric (pooling) layers, eventually comprising one or more layers of "normal" fully-connected neurons (dense/fully-connected layers).

Training of the neural network may be performed, for example, by means of a back propagation method. The purpose of the network is to maximize the reliability of the mapping of a given input data to a given output data. The mapping quality is described by a loss function (in english: loss function). The goal is to minimize the loss function. In the case of the back propagation method, the artificial neural network is taught by altering the connection weights.

In the training state, the connection weights between the processing elements contain information about a relationship between the first characterization and the one or more second characterizations, which information can be used to predict the one or more second characterizations showing the examination region during the second time span for a new plurality of first characterizations (e.g., a new plurality of first characterizations of a new examination object), wherein at least some of the new plurality of first characterizations show the examination region during the first time span after administration of the contrast agent.

A cross-validation method may be used to divide the data into a training data set and a validation data set. The training dataset is used for back-propagation training of network weights. The validation data set is used to check the accuracy of predictions that the trained network is applied to the unknown data.

Further details of the construction and training of artificial neural networks can be gleaned from the prior art (see, e.g., S.Khan et al: A Guide to Convolutional Neural Networks for Computer Vision, morgan & Claypool Publishers 2018,ISBN 1681730227,9781681730226,WO2018/183044A1, WO2018/200493, WO2019/074938A1, WO2019/204406A1, WO2019/241659A 1).

Preferably, the predictive model is a Generative Antagonism Network (GAN) (see, e.g., http://3dgan. Csail. Mit. Edu /).

In addition to characterization, other information about the examination object, examination region, examination conditions and/or radiological examination method can also be used for training, verification and prediction.

Examples of information about the examination object are: sex, age, weight, height, medical history, pharmaceutical properties, duration and intake of medication, blood pressure, central venous pressure, respiratory rate, serum albumin, total bilirubin, blood glucose, iron content, respiratory capacity, etc. The information about the examination object can also be read from a database or a patient electronic file, for example.

Examples of information about the examination region are: history of past disease, surgery, partial excision, liver transplantation, iron liver, fatty liver, etc.

As described above, the characterization of the examination region for training, validation and prediction is a characterization of the examination region in frequency space (also referred to as spatial frequency space or fourier space or frequency domain or fourier characterization).

In magnetic resonance imaging, the raw data usually appear as so-called k-space data due to the measurement method described above. The k-space data are delineations of the examination region in frequency space, i.e. these k-space data can be used for training, verification and prediction. If representations in real space are present, these can be transformed (transformed) into representations in frequency space, for example by fourier transformation; conversely: the representation in frequency space may be converted (transformed) into a representation in real space, for example by an inverse fourier transform.

Thus, if the radiological image of the examination region is present in the form of a two-dimensional image in real space, this representation of the examination region can be converted into a two-dimensional representation of the examination region in frequency space by means of a 2D fourier transformation.

The three-dimensional image (volume rendering) of the examination region can be regarded as a stack of two-dimensional images. Furthermore, it is conceivable that the three-dimensional image is converted into a three-dimensional representation of the examination region in frequency space by means of a 3D fourier transformation.

It is also conceivable to use a transform other than a fourier transform to transform the real-space representation into a frequency-space representation. Three main characteristics that this transformation must satisfy are as follows:

a) There is an explicit inverse transformation (there is an explicit link between the real space depiction and the frequency space depiction),

b) The location of the contrast information is referred to as,

c) Robustness to defective image alignment.

Details about the transformation from one depiction to another are described in many textbooks and publications (see, e.g., W.Burger, M.J.Burge: digital Image Processing: an Algorithmic Introduction Using Java, texts in Computer Science, 2 nd edition, springer-Verlag,2016,ISBN:9781447166849;W.Birkfellner:Applied Medical Image Processing, 2 nd edition: A Basic Coure, verlag Taylor & Francis,2014,ISBN:9781466555570;R.Bracewell:Fourier Analysis and Imaging,Verlag Springer Science&Business Media,2004,ISBN:9780306481871).

Fig. 2 shows schematically and exemplarily a link between the characterization of the examination region in real space as well as in frequency space.

Fig. 2 depicts a timeline. At three different time points t ₁ 、t ₂ And t ₃ At, a characterization of the examination region is generated based on the measurement results. The examination region is a lung of a person. At time point t ₁ A first characterization is generated. This may be a representation (O1) of the examination region (lung) in real space or may be a representation (F1) of the examination region (lung) in frequency space. The representation (O1) of the examination region in real space can be converted into a representation (F1) of the examination region in frequency space by means of a fourier transformation FT. Characterization (F1) of the examination region in frequency space can be performed by inverse FourierThe transformation iFT is converted into a representation (O1) of the examination region in real space. The two representations (O1) and (F1) comprise the same information about the examination region, only in different depictions. At time point t ₂ Another characterization is generated. This may be a representation (O2) of the examination region (lung) in real space or may be a representation (F2) of the examination region (lung) in frequency space. The representation (O2) of the examination region in real space can be converted into a representation (F2) of the examination region in frequency space by means of a fourier transformation FT. The characterization of the examination region in frequency space (F2) can be converted into a characterization of the examination region in real space (O2) by an inverse fourier transform iFT. The two representations (O2) and (F2) comprise the same information about the examination region, only in different depictions. At time point t ₃ Another characterization is generated. This may be a representation (O3) of the examination region (lung) in real space or may be a representation (F3) of the examination region (lung) in frequency space. The representation (O3) of the examination region in real space can be converted into a representation (F3) of the examination region in frequency space by means of a fourier transformation FT. The characterization of the examination region in frequency space (F3) can be converted into a characterization of the examination region in real space (O3) by an inverse fourier transform iFT. The two representations (O3) and (F3) contain the same information about the examination region, but are depicted differently.

The characterization of the examination region in real space (O1), (O2) and (O3) is a familiar characterization; they can be immediately understood by people. Characterization of (O1), (O2) and (O3) shows how the contrast agent dynamically diffuses in the vein. The representations (F1), (F2) and (F3) contain the same information, but are more difficult for humans to understand.

Fig. 3 schematically and exemplarily shows how the characterizations (F1), (F2) and (F3) of the examination region in frequency space as generated in fig. 2 can be used to train a Predictive Model (PM). The representations (F1), (F2) and (F3) constitute a training dataset of the examination object. Training is accomplished by using multiple training data sets for multiple examination subjects.

The representations (F1) and (F2) are a plurality of (in this example two) first reference representations of the examination region in frequency space, at least some of the plurality of first representations representing the examination region during a first time span after administration of the contrast agent. The representation (F3) is a second reference representation of the examination region in frequency space, representing the examination region during a second time span. In fig. 3, the predictive model is trained to predict the characterization of the examination region in frequency space (F3) from the characterizations of the examination region in frequency space (F1) and (F2). The characterizations (F1) and (F2) are input into a Predictive Model (PM), and the predictive model calculates a characterization (f3) from the characterizations (F1) and (F2). Asterisks indicate that the characterization (f3) is a predictive characterization. The calculated characterization (F3) is compared with the characterization (F3). The bias may be used in a back propagation method to train a predictive model to reduce the bias to a defined minimum. If the predictive model has been trained based on multiple training data sets for multiple examination objects, and if the prediction has reached a defined accuracy, the trained predictive model may be used for the prediction. This is illustrated and schematically in fig. 4.

Fig. 4 shows the Prediction Model (PM) trained in fig. 3. The predictive model is used to generate one or more second representations of the examination region in frequency space based on a plurality of first representations of the examination region in frequency space, at least some of the plurality of first representations representing the examination region in a first time span after administration of the contrast agent, the one or more second representations representing the examination region in a second time span.

In this embodiment, two characterizations of the examination region in frequency space are performedAnd +.>Is input into a predictive model and said predictive model generates (calculates) a third characterization +.>The wave symbol (-) representation is a representation of a new inspection object at which the inspection is performedThe characterization of the object is typically not present in the characterization used in the training method to train the predictive model. Asterisks indicate the characterization->Is a predictive characterization. Characterization of the examination region in the frequency space +.>Can be converted into a representation of the examination region in real space, for example by means of an inverse fourier transformation iFT +.>

The use of a characterization of the examination region in frequency space has advantages over the use of a characterization of the examination region in real space (also referred to as image space). When using a characterization of the examination region in frequency space, contrast information important for training and prediction can be separated from detail information (micro-structures). Thus, in the case of training, the information to be learned by the predictive model may be focused on, and in the case of prediction, the information to be predicted by the predictive model may be focused on: contrast information.

However, the contrast information of the representation of the examination region in real space (each pixel/voxel inherently carries information about contrast) is typically distributed throughout the representation, and the contrast information of the representation of the examination region in frequency space is then encoded in the center of and around the frequency space. In other words: the low frequency of the characterization in frequency space determines the contrast, while the high frequency contains information of the microstructure.

Thus, contrast information can be separated, limiting training and prediction to contrast information, and information about minute structures can be reintroduced after training/prediction.

In a preferred embodiment, the method according to the invention comprises the steps of:

specifying a region in the first representation, wherein the specified region comprises a center of the frequency space,

downscaling the first representation to the specified region, wherein a plurality of downscaled first representations are obtained,

feeding a plurality of reduced first representations to a predictive model,

Receiving one or more second representations of the examination region in frequency space from the predictive model, wherein the one or more second representations represent the examination region during a second time span,

augmenting the one or more second representations with one or more regions of the received first representations that lie outside the specified region, wherein one or more augmented second representations are obtained,

transforming the one or more augmented second characterizations into one or more characterizations of the examination region in real space,

The designation of the region in the first characterization may be achieved, for example, by: a user of a computer system according to the invention enters one or more parameters into the computer system according to the invention and/or selects from a list defining the shape and/or size of the area. However, it is also conceivable that the designation is performed automatically, for example by a computer system according to the invention, which is suitably configured to select a predetermined region in the characterization of the examination region.

The specified area is typically smaller than the frequency space filled by the first characterization, but in any case includes the center of the frequency space.

The region of the frequency space comprising the center of the frequency space (also called origin or zero point) contains contrast information related to the method according to the invention. If the specified region is smaller than the frequency space filled by the first characterization, the result is a lower computational complexity of the subsequent predictions (this also applies in particular to the training of the prediction model). Thus, the choice of region size has a direct impact on computational complexity.

In principle, it is also possible to specify a region corresponding to the entire frequency space filled by the first characterization; in this case, the sub-region of the frequency space is not reduced and the computational complexity is maximized.

Thus, by specifying the region around the center of the frequency space, the user of the computer system according to the invention can decide himself whether he wants to examine the complete characterization of the region in the frequency space to form the basis of training and prediction, or whether he wants to reduce the computational complexity. Here, he can directly influence the required computational complexity by the size of the specified region.

The designated region typically has the same dimensions as the frequency space: in the example of 2D characterization in 2D frequency space, the designated region is typically an area; in the example of 3D characterization in 3D frequency space, the designated region is typically a volume.

The specified area may in principle have any shape; thus, it may be circular and/or angular, concave and/or convex, for example. Preferably, in the example of a 3D frequency space in a cartesian coordinate system, the region is cuboid or cubic, and in the example of a 2D frequency space in a cartesian coordinate system, the region is rectangular or square. However, it may also be spherical, annular or have other shapes.

Preferably, the geometric center of gravity of the designated region coincides with the center of the frequency space.

The characterization for training, validation and prediction is reduced to the specified region. The term "reduced" means here that all parts of the representation that are not located within the specified region are cut off (discarded) or covered by a mask. In the case of a mask, those areas outside the specified area are covered by the mask, with the result that only the specified area remains uncovered; when covered with a mask, the color value of the corresponding pixel/voxel may be set to zero (black), for example.

The characterization thus obtained is also referred to as a reduced characterization in this specification.

The first characterization obtained after downscaling (downscaled first characterization) is fed to the predictive model: the predictive model has been trained in advance in a training method to learn the dynamic effect of the amount of contrast agent on the characterization of the examination region in frequency space. The training preferably also utilizes the reduced representation (reduced first representation and reduced second representation).

Thus, the predictive model has learned a dynamic image of the characterization of the examination region by the contrast agent, and this learned "knowledge" may be applied to predict one or more (reduced) second characterizations based on the (reduced) first characterization.

The one or more predicted second characterizations represent an examination region in frequency space during a second time span.

A predictive model is calculated based on the (reduced) first representation and outputs a second representation of the one or more predictions.

If at least one predicted second representation has been generated based on the reduced first representation, it is appropriate to later re-add the previously discarded (excised or covered with a mask) portions so that as little information as possible about the micro-structures is lost in the final artificially generated image.

To reuse previously discarded (excised or covered with a mask) portions, the at least one predicted second characterization may be superimposed with the at least one received first characterization such that the at least one predicted second characterization replaces a corresponding superimposed frequency region of the at least one initially received first characterization. Preferably, the predicted second characterization replaces a corresponding frequency region in the initially received first characterization that represents the examination region when no contrast agent is applied.

The replacement superimposed frequency regions correspond to one or more regions in the frequency space that are omitted by the received first characterization when the first characterization is reduced to the specified region.

In other words: the frequency space of at least one predicted second representation of the examination region is filled by those regions of at least one initially received first representation, wherein the at least one initially received first representation is larger than the predicted second representation.

By using a characterization of the examination region in frequency space, it is thus possible to separate the contrast information from the detail information region, to limit the training and prediction to the contrast information, and to reintroduce the detail information after the training and/or prediction. As already described, this procedure reduces the computational complexity during training, validation and prediction.

However, operating in frequency space has another advantage over operating in real space: the registration of individual characterizations in frequency space is less important than in real space. "registration" (also referred to in the art as "image alignment") is an important process in digital image processing and is used to match two or more images of the same scene, or at least similar scenes, to one another in the best possible manner. One of the images is defined as a reference image, while the other is called a target image. In order to optimally match the target image with the reference image, a compensation transformation is calculated. The images to be aligned differ from each other in that they are acquired from different locations, different points in time and/or by different sensors.

In an example of the invention, the individual characterizations of the plurality of first characterizations of the examination region are: first, at different points in time; secondly, the contrast agent content in the examination region and the contrast agent diffusion in the examination region are different.

Thus, using the characterization of the examination region in frequency space has the advantage over using the characterization of the examination region in real space that: the training, validation and prediction methods have a higher tolerance to errors in image alignment. In other words: a representation in frequency space that is not superimposed with precision will have less impact than a representation in real space that is not superimposed with pixel/voxel precision. This is because of the properties of the fourier transform: as already described, the contrast information of the fourier transform image is always mapped near the origin of the fourier space. Flipping or rotation in image space (real space) may cause image information (e.g., visible structures) to be located in different areas of the image after transformation. However, in fourier space, these transforms do not change the areas encoded with contrast information relevant to the present invention.

Fig. 5 illustrates schematically and schematically the steps of training a predictive model according to a preferred embodiment of the invention.

Received are two first representations (F1) and (F2) of the examination object in the frequency space, and a second representation (F3) of the examination object in the frequency space. In the characterization (F1), (F2) and (F3), the same region a is specified in each case. Region a includes the center of the frequency space, in this example a square shape, with the geometric center of gravity of the square coinciding with the center of the frequency space. The representations (F1), (F2) and (F3) are reduced to correspond to the specified region a: the result is three reduced characterizations (F1 _red )、(F2 _red ) And (F3) _red ). The reduced representation is used for training. The predictive model is trained to follow the reduced representation (F1 _red ) And (F2) _red ) To predict the reduced representation (F3 _red ). The reduced representation (F1 _red ) And (F2) _red ) Is fed to a Prediction Model (PM) and the prediction model computes a reduced representation (f3#) _red ) The reduced representation (f3#) _red ) As close as possible to the reduced representation (F3 _red )。

Fig. 6 illustrates schematically and schematically how the prediction model trained in fig. 5 may be used for prediction.

In this embodiment, two first characterizations of the examination region in frequency space are received And +.>And reduce them to the designated area a, respectively. The result is two reduced first characterizations +.>And->The reduced first characterization +.>And +.>Feed to a trained Predictive Model (PM). A trained Predictive Model (PM) is based on the reduced first characterization +.>And +.>To calculate a reduced second representationIn a further step, the reduced second characterization +.>By the first characterization received->In (a) reducing the received first token +.>The area discarded during this time->To supplement. As described, replace the received first token +.>Or in addition to the received first representation +.>Besides the parts of (2), the received second representation can also be taken in->Is added to the reduced third representation +.>

From the supplemented characterizationThe characterization of the examination region in real space can be generated by means of an inverse fourier transformation>

It should be noted that other methods may also be used to transform the frequency-space depiction into a real-space depiction, such as iterative reconstruction methods.

The method according to the invention can be performed by means of a computer system. The invention also provides a computer system configured to perform the method according to the invention (e.g. by means of a computer program according to the invention).

FIG. 7 schematically and exemplarily illustrates one embodiment of a computer system according to the present invention. The computer system (10) comprises a receiving unit (11), a control and calculation unit (12) and an output unit (13).

A "computer system" is an electronic data processing system that processes data by means of programmable algorithms. Such systems typically include control and computing units, often referred to as "computers," that include a processor for performing logical operations and memory for loading computer programs, and also peripheral devices.

In computer technology, "peripheral devices" refer to all devices connected to a computer and used to control the computer and/or as input and output devices. Examples of such peripheral devices are displays (screens), printers, scanners, mice, keyboards, joysticks, drivers, cameras, microphones, speakers, etc. Internal ports and expansion cards are also considered peripheral devices in computer technology.

Modern computer systems are generally divided into desktop PCs, portable PCs, laptops, notebooks, netbooks and tablet PCs and so-called handheld devices (e.g. smart phones); all of these systems may be used to practice the present invention.

Input to the (e.g., user-controlled) computer system is accomplished through an input device, such as through a keyboard, mouse, microphone, touch display screen, and/or other device. The output is achieved by an output unit (13), which may be, in particular, a display (screen), a printer and/or a data storage medium.

The computer system (10) according to the invention is configured to predict one or more second characterizations of the examination region of the examination subject in frequency space from a plurality of first characterizations of the examination region in frequency space representing the examination region during a first time span after administration of the contrast agent, wherein the one or more second characterizations represent the examination region during the second time span.

A control and calculation unit (12) is used for controlling the receiving unit (11) and the output unit (13), coordinating the data and signal flow between the various units, processing the characterization of the examination area and generating the artificial radiological image. It is conceivable that there are a plurality of said control and calculation units.

The receiving unit (11) is used for receiving the characterization of the examination region. The characterization may be transmitted, for example, from a magnetic resonance imaging system, or from a computed tomography system, or read from a data storage medium. The magnetic resonance imaging system or the computer tomography system may be a component of the computer system according to the invention. However, it is also conceivable that the computer system according to the invention is a component of a magnetic resonance imaging system or a computer tomography system. The characterization may be transmitted over a network connection or over a direct connection. The characterization may be transmitted over radio (WLAN, bluetooth, mobile communication, and/or other means) and/or cable. It is conceivable that there are a plurality of receiving units. The data storage medium may also be a component of, or be connected to, a computer system according to the invention, e.g. through a network. It is contemplated that there are multiple data storage media.

The receiving unit receives the characterization and possibly other data, such as, for example, information about the examination object, image acquisition parameters and/or other information, and transmits them to the control and calculation unit.

The control and calculation unit is configured to generate an artificial radiological image based on the received data.

Via an output unit (13), the artificial radiological image may be displayed (e.g., on a monitor), output (e.g., via a printer), and/or stored into a data storage medium. It is envisaged that there may be a plurality of output units.

Fig. 8 shows an exemplary preferred embodiment of a method for training a predictive model according to the invention in the form of a flow chart.

The method (100) comprises the steps of:

(110) Receiving training data, wherein for each of a plurality of inspection objects, the training data comprises: i) A plurality of first reference characterizations of the examination region in frequency space, at least a portion of the plurality of first reference characterizations representing the examination region during a first time span after administration of the contrast agent; and ii) one or more second reference characterizations of the examination region in frequency space, the one or more second reference characterizations representing the examination region during a second time span,

(120) For each inspection object: feeding the plurality of first reference representations to a predictive model, wherein the predictive model is trained to generate one or more second reference representations based on the plurality of first reference representations, wherein the training comprises minimizing a loss function, wherein the loss function quantifies a deviation between the generated second reference representations and one or more received second reference representations,

(130) Methods of outputting and/or storing a trained predictive model and/or providing a trained predictive model to one or more characterizations of an examination region for predicting a new examination object.

Fig. 9 shows in flow chart form an exemplary further preferred embodiment of the method according to the invention for training a predictive model.

The method (200) comprises the steps of:

(210) Receiving training data, wherein for each of a plurality of inspection objects, the training data comprises: i) A plurality of first reference characterizations of the examination region in frequency space, at least a portion of the plurality of first reference characterizations representing the examination region during a first time span after administration of the contrast agent; and ii) one or more second reference characterizations of the examination region in frequency space, the one or more second reference characterizations representing the examination region during a second time span,

(220) Designating a region in the first reference representation, wherein the designated region comprises a center of a frequency space,

(230) Downscaling the reference representation to the specified region, wherein a plurality of downscaled first reference representations and one or more downscaled second reference representations for each inspection object are obtained,

(240) For each inspection object: feeding the plurality of reduced first reference representations to a predictive model, wherein the predictive model is trained to generate one or more reduced second reference representations based on the plurality of reduced first reference representations, wherein the training comprises minimizing a loss function, wherein the loss function quantifies a deviation between the generated reduced second reference representations and one or more reduced second reference representations of training data,

(250) Methods of outputting and/or storing a trained predictive model and/or providing a trained predictive model to one or more characterizations of an examination region for predicting a new examination object.

Fig. 10 illustrates, in flow chart form, one preferred embodiment of a method for predicting one or more characterizations in accordance with the present invention.

The method (300) comprises the steps of:

(310) Providing a predictive model, wherein the predictive model has been trained according to the method (100) described above,

(320) Receiving a plurality of first representations of an examination region of an examination subject in a frequency space, wherein at least a portion of the first representations represent the examination region during a first time span after administration of a contrast agent,

(330) The plurality of first representations is fed to a predictive model,

(340) Receiving one or more predictive characterizations of the examination region in frequency space from the predictive model, wherein the one or more predictive characterizations represent the examination region during a second time span,

(350) Transforming the one or more predictive representations into one or more representations of the examination region in real space,

(360) One or more representations of the examination region in the real space are output and/or stored.

Fig. 11 illustrates, in flow chart form, another preferred embodiment of a method for predicting one or more characterizations in accordance with the present invention.

The method (400) comprises the steps of:

(410) Providing a predictive model, wherein the predictive model has been trained according to the method (200) described above,

(420) Receiving a plurality of first representations of an examination region of an examination subject in a frequency space, wherein at least a portion of the first representations represent the examination region during a first time span after administration of a contrast agent,

(430) Designating an area in the first characterization, wherein the designated area comprises a center of a frequency space,

(440) Downscaling the first representation to the specified region, wherein a plurality of downscaled first representations are obtained,

(450) The plurality of reduced first representations is fed to a predictive model,

(460) Receiving one or more second representations of the examination region in frequency space from the predictive model, wherein the one or more second representations represent the examination region during a second time span,

(470) Augmenting the one or more second representations with one or more regions of the received first representations that are outside the specified region, wherein one or more augmented second representations are obtained,

(480) Transforming the one or more augmented second characterizations into one or more characterizations of the examination region in real space,

(490) One or more representations of the examination region in the real space are output and/or stored.

Listed below are several embodiments of how the present invention can be used to predict an artificial radiological image.

Example 1

In one embodiment, the present invention is used to simulate an intravascular contrast agent (also known as a blood pool agent, english: blood pool agent).

When generating radiological images with relatively long acquisition times/scan times, for example, acquisition images of the chest and abdomen under free breathing conditions to delineate the vascular system (e.g., diagnosing pulmonary embolism under free breathing conditions in MRT), extracellular contrast agent may be cleared from the vascular system relatively quickly, which means that the contrast is rapidly degraded. However, it would be advantageous to be able to maintain contrast over a longer period of time.

To solve this problem, a plurality of first representations of the examination region in frequency space are generated/received in a first step, wherein at least some of the first representations represent the examination region after administration of the contrast agent.

The administered contrast agent may be an extracellular contrast agent and/or an intracellular contrast agent.

The contrast agent is preferably introduced into a blood vessel (e.g., arm vein) of the subject, and the dose used is based on body weight. Since then, the contrast agent moves along the blood circulation system together with the blood.

The "blood circulation system" is the path followed by blood in the body of a human being and most animals. It is a flow system of blood formed by the heart and the vascular network (cardiovascular system, vascular system).

The period of time during which the extracellular contrast agent circulates in the blood circulation system depends on the examination subject, the contrast agent and the amount of administration, while the contrast agent is continuously cleared from the blood circulation system via the kidneys.

At least one first characterization of the vascular system or part thereof is captured while the contrast agent diffuses and/or circulates in the vascular system of the examination subject. A plurality of first representations representing different periods of diffusion of the contrast agent in the vascular system or portion thereof (e.g., distribution period, arterial period, venous period, and/or other periods, etc.) may be captured. Capturing multiple images allows for later differentiation of vessel types.

The measured characterization represents the vascular system or portion thereof with contrast enhancement relative to surrounding tissue. Preferably, at least one characterization shows a contrast enhanced artery (arterial phase) and at least another characterization shows a contrast enhanced vein (venous phase).

An artificial representation is generated based on the measured representation. The manual characterization preferably shows the same examination area as the measured characterization. If a plurality of measured images of the examination region are captured at different points in time after the administration of the contrast agent, the later characterization shows in particular vessels with a gradually decreasing contrast with respect to the surrounding tissue, as the contrast agent is gradually cleared from the vessels. In contrast, artificial characterization shows vessels with continuously high contrast relative to surrounding tissue.

This is achieved by feeding the measured characterizations into a predictive model according to the invention, which has been trained in advance, so as to predict a plurality of characterizations showing constant contrast enhancement of the blood vessel over time, based on characterizations showing the change in contrast enhancement of the measured blood vessel over time.

The reference data used to train and validate such predictive models typically include a characterization of the examination region measured after administration of an extracellular or intracellular contrast agent. The reference data may also include a characterization of the examination region after administration of the blood pool contrast agent. Such reference data may be determined, for example, in a clinical study. An intravascular contrast agent that may be used in such clinical studies is, for example, nano-iron oxide. Nano-iron oxide is a colloidal iron-carbohydrate complex that has been approved for use in parenteral treatment of iron deficiency in chronic kidney disease in situations where oral treatment is not available. The nano-iron oxide is administered by intravenous injection. The nano ferric oxide can be named as the commodityOr->Solutions for intravenous injection are commercially available. The iron-carbohydrate complex shows superparamagnetism and thus can be used for contrast enhancement in MRT examinations (off label) (see, e.g., L.P. Smits et al: evaluation of ultrasmall superparamagnetic iron-oxide (USPIO) enhanced MRI with ferumoxytol to quantify arterial wall inflammation, atherosclerosis 2017,263: 211-218).

Similarly contemplated for use in administering intravascular contrast agentsThe subsequent characterization serves as training data.

It is similarly conceivable to synthetically generate a reference representation showing that the blood vessels in the examination region have a constant contrast enhancement over time, for example by means of a segmentation method based on the first representation. Segmentation methods are widely described in the literature. The following publications are given as examples: conversano et al: hepatic Vessel Segmentation for 3D Planning of Liver Surgery,Acad Radiol 2011, 18:461-470; s. moccia et al: blood vessel segmentation algorithms-Review of methods, datasets and evaluation metrics, computer Methods and Programs in Biomedicine 158 (2018) 71-91; marcan et al: segmentation of hepatic vessels from MRI images for planning of electroporation-based treatments in the liver, radio Oncol 2014;48 (3): 267-281; T.A. hope et al: improvement of Gadoxetate Arterial Phase Capture With a High Spatio-Temporal Resolution Multiphase Three-Dimensional SPGR-Dixon Sequence, journal ofMagnetic Resonance Imaging 38:938-945 (2013); WO 2009/1359233 a1, us6754376b1, WO 2014/1622793 a1, WO2017/139110a1, WO2007/053676a2, ep2750102a 1).

Based on having fed the first characterization, a second characterization is generated after the trained predictive model, which shows a constant contrast enhancement in the blood vessel over time.

Example 2

In a preferred embodiment, the invention is used to generate (predict) one or more artificial MRT images in dynamic contrast enhanced magnetic resonance imaging.

In the text that follows, the term "image" is used. In the context of the present invention, an "image" is a representation. The image may be a representation in real space or a representation in frequency space. For training the predictive model and for prediction, always use is made of the characterization in frequency space; i.e. for example k-space data. However, if tokens in real space are generated based on the measurements, they can be converted into tokens in frequency space, e.g. by means of fourier transformation, after which they are introduced into training and/or prediction.

The examination region is introduced into the basic magnetic field. The examination region is subjected to an MRT method, and in the process a plurality of MRT images are generated, which show the examination region during a first time span. These MRT images generated based on the measurement results during the first time span are also referred to as first MRT images in this specification.

The term "plurality" means that at least two (first) MRT images, preferably at least three (first) MRT images, particularly preferably at least four (first) MRT images are generated.

The contrast agent is applied to the examination object and diffuses in the examination region. The contrast agent is preferably administered intravenously (e.g., into an arm vein) in bolus form, and the dose administered is based on body weight.

The contrast agent is preferably a hepatobiliary contrast agent, such as Gd-EOB-DTPA or Gd-BOPTA. In a particularly preferred embodiment, the contrast agent is a substance or mixture of substances having gadoteric acid or gadoteric acid salt as contrast-enhancing active substance. Very particular preference is given to the disodium salt of gadofostip (Gd-EOB-DTPA disodium).

The first time span preferably comprises the distribution of contrast agent in the examination region. Preferably, the first time span comprises arterial and/or portal venous phases and/or late phases in dynamic contrast enhanced magnetic resonance imaging of the liver or liver portion of the examination subject. The stated periods are for example defined and described in the following publications: magn. Reson. Imaging,2012, 35 (3): 492-511, doi:10.1002/jmri.22833; clujul Medical,2015, volume 88, phase 4: 438-448, doi:10.15386/cjmed-414; journal of hepatology, 2019, volume 71: 534-542, http:// dx.doi.org/10.1016/j.jhep.2019.05.005.

Fig. 12 schematically shows the time-domain cross-hatching of the concentration of contrast agent in the hepatic artery (a), hepatic vein (V) and healthy liver cells (P) after intravenous administration of hepatobiliary contrast agent to a human arm. The concentration is depicted in the form of the signal intensity I in the region stated in the magnetic resonance measurement (hepatic artery, hepatic vein, hepatic cell) as a function of time t. Once an intravenous bolus is made, the concentration of contrast agent first rises in the hepatic artery (a) (dashed line). The concentration goes through a maximum and then drops. The concentration in the hepatic vein (V) rises more slowly than in the hepatic artery and reaches its maximum later (dashed line). The concentration of contrast agent in healthy liver cells (P) rises slowly (continuous curve) and reaches its maximum only at a very late point in time (maximum not depicted in fig. 12). Several characteristic points in time may be defined: at time point TP0, the contrast agent is administered intravenously as a bolus. At time point TP1, the concentration of the contrast agent (signal intensity) in the hepatic artery reaches its maximum value. At time point TP2, the curves of the signal intensities of the hepatic artery and hepatic vein intersect. At time point TP3, the concentration of contrast agent (signal intensity) in the hepatic vein reaches through its maximum value. At time point TP4, the curves of the signal intensities of the hepatic artery and the hepatic cells intersect. At time point T5, the concentrations in the hepatic artery and vein have fallen to a level at which they no longer cause a measurable contrast enhancement.

In a preferred embodiment, the first time span is selected such that such an MRT image of the liver or liver portion of the examination subject is generated:

(i) The examination region is shown without the contrast agent applied,

(ii) Showing an examination region during an arterial phase, in which the contrast agent diffuses through the artery in the examination region,

(iii) Showing an examination region during a portal period in which the contrast agent enters the examination region via a portal vein, an

(iv) An examination region is shown during an advanced stage in which the concentration of the contrast agent in the artery and vein decreases and the concentration of the contrast agent in liver cells increases.

Preferably, the first time span starts within a time span of 1 minute to 1 second before the administration of the contrast agent, or a time span of 2 minutes to 15 minutes from the administration of the contrast agent, preferably a time span of 2 minutes to 13 minutes, more preferably a time span of 3 minutes to 10 minutes. Since the contrast agent is excreted very slowly through the kidneys and/or bile, the second time span may extend to two hours or more after administration of the contrast agent.

In a preferred embodiment, the first time span comprises at least time points TP0, TP1, TP2, TP3 and TP4.

In a preferred embodiment, at least the MRT images (based on measurements) for all of the following periods are generated: in the time span before TP0, in the time span from TP0 to TP1, in the time span from TP1 to TP2, in the time span from TP2 to TP3, and in the time span from TP3 to TP4.

It is conceivable that one or more MRT images are generated (based on the measurement results) separately, before TP0, from TP0 to TP1, from TP1 to TP2, from TP2 to TP3, and from TP3 to TP4, in a time span.

Based on the (first) MRT image during the first time span generated (based on the measurement results), a second MRT image or a plurality of second MRT images showing the examination region during a second time span are predicted. In this specification, the MRT image predicted for the second time span is also referred to as a second MRT image.

In a preferred embodiment of the invention, the second time span is after the first time span.

The second time span is preferably a time span within the hepatobiliary period; preferably a time span starting at least 10 minutes after administration of the contrast agent, preferably a time span starting at least 20 minutes after administration of the contrast agent.

By means of the predictive model according to the invention, at least one second characterization representing the examination region during a second time span is predicted. The predictive model has been trained in advance to predict one or more MRT images showing the examination region during a second time span based on a plurality of first MRT images showing the examination region during a first time span.

The embodiment described herein is also schematically depicted in fig. 1 (b).

Example 3

In another preferred embodiment of the invention, the invention is used to distinguish lesions in the liver from blood vessels. In the example of a T1 weighted MRT image,resulting in a significant signal enhancement in healthy liver parenchyma 10-20 minutes after injection (in hepatobiliary phase) without liver parenchyma refinementLesions of cells or containing only small amounts of hepatocytes, such as metastatic or medium to poorly differentiated hepatocellular carcinoma (HCCs), appear as darker areas. However, in the hepatobiliary phase, the blood vessels also appear as dark areas, meaning that in the MRT image generated during the hepatobiliary phase, it is not possible to distinguish between liver lesions and blood vessels based on contrast alone.

The invention can be used to generate artificial MRT images of the liver or liver parts of an examination object, wherein the contrast between blood vessels and liver cells in the liver has been artificially minimized, so that liver lesions are more easily identified.

In the context of the present invention, an "image" is a representation. The image may be a representation in real space or a representation in frequency space. For training the predictive model and for prediction, the characterization in frequency space is always utilized. However, tokens in real space may be generated based on the measurements, and then they may be converted into tokens in frequency space, e.g. by means of fourier transformation, after which they are introduced into training and/or prediction.

The plurality of first characterizations includes at least one characterization of the examination region that identifies a blood vessel, preferably depicted as contrast enhancement due to a contrast agent (blood vessel characterization).

When paramagnetic contrast agents are used, the vessels in this characterization are characterized by high signal intensity due to contrast enhancement (high signal delineation). Those (continuous) structures in such characterization that have signal intensities within an empirically determinable range are assigned to the blood vessel. This means that by this characterization, there is information about where in the real-space depiction the blood vessel is depicted, or which structures in the real-space depiction can be attributed to the blood vessel (artery and/or vein).

The plurality of first characterizations also includes at least one characterization of the examination region, wherein healthy liver cells are delineated by contrast enhancement (liver cell characterization), such as a characterization of the examination region acquired during the hepatobiliary phase.

Combining information of at least one vessel characterization via a vessel with information of at least one liver cell characterization. This involves (manually) generating (calculating) at least one characterization in which contrast differences between structures attributable to blood vessels and structures attributable to healthy liver cells are leveled.

Here, the term "leveling … …" means "unifying" or "minimizing". The purpose of leveling is to make the boundaries between the blood vessels and healthy liver cells in the artificially generated representation disappear and to make the blood vessels and healthy liver cells in the artificially generated representation appear as uniform tissue, in contrast to which liver lesions are structurally prominent due to different contrasts.

Typically, a (number=1) artificial characterization is predicted based on a (number=1) vascular characterization and a (number=1) liver cell characterization.

It is contemplated that in addition to the at least one vascular characterization and the at least one liver cell characterization, at least one native characterization is additionally used, thereby predicting at least one artificial characterization.

In one embodiment, the generation of the artificial characterization includes the steps of:

feeding at least one vessel representation and at least one liver cell representation to a predictive model, wherein the predictive model has been trained on the basis of a reference representation by means of supervised learning, so as to generate at least one artificial representation from the at least one reference vessel representation and the at least one reference liver cell representation, wherein contrast differences between structures attributable to the vessel and structures attributable to healthy liver cells are leveled in the at least one artificial representation,

-receiving at least one artificial representation as output from the predictive model.

Example 4

In another embodiment, the invention is used to generate native MRT images of the liver. Here, one or more artificial MRT images of the liver or liver part of the examination object are generated, which show the liver or liver part without contrast enhancement caused by the contrast agent. An artificial MRT image is created based on the MRT image, all acquired with contrast enhancement by contrast agent.

The examination region is introduced into the basic magnetic field. The contrast agent is applied to the examination object and diffuses in the examination region. The contrast agent is preferably administered intravenously (e.g., into an arm vein) in bolus form, and the dose administered is based on body weight. Preferably, the contrast agent is a hepatobiliary contrast agent, such as Gd-EOB-DTPA or Gd-BOPTA. In a particularly preferred embodiment, the contrast agent is a substance or mixture of substances having gadoteric acid or gadoteric acid salt as contrast-enhancing active substance. Very particular preference is given to the disodium salt of gadofostip (Gd-EOB-DTPA disodium).

A plurality of first representations of the examination region are generated, the plurality of first representations representing the examination region during a first time span after administration of the contrast agent. Preferably, the plurality of first characterizations are T1 weighted depictions.

Preferably, the plurality of first representations comprises at least one representation of the examination region, the at least one representation representing the examination region during a dynamic period, e.g. at least one representation representing the examination region during an arterial period, a venous period and/or a late period (see e.g. fig. 12 and the explanation relating to embodiment 2). When paramagnetic contrast agents are used, the vessels in this characterization are characterized by high signal intensities due to contrast enhancement (high signal delineation).

Preferably, the plurality of first representations further comprises at least one representation of the examination region during a liver and gall phase. During the hepatobiliary phase, healthy liver tissue (parenchyma) is depicted with contrast enhancement.

MRT examination of the dynamic and hepatobiliary phases can extend a relatively long time span. During the time span, patient movement should be avoided, thereby minimizing movement artifacts in the radiological image. Long-term movement restriction is uncomfortable for the patient. Thus, a shortened MRT procedure is now established, in which a contrast agent is administered to the examination subject a certain time span (i.e. 10 to 20 minutes) before the acquisition of the MRT image, so that the MRT image in the hepatobiliary phase can be acquired directly. Subsequently, after administration of the second dose of contrast agent, a dynamic phase MRT image is acquired during the same MRT procedure. The result is a significantly shorter residence time of the patient or subject in the MRT compared to conventional MRT procedures. Thus, according to the invention, preferably at least one characterization of the liver or of a portion of the liver in the hepatobiliary phase is recorded after (first) administration of a first contrast agent to the subject, and at least another characterization of the same liver or of a portion of the same liver in the dynamic phase is recorded after (second) administration of a second contrast agent to the same subject. The first contrast agent is a paramagnetic hepatobiliary contrast agent. The second contrast agent may also be an extracellular paramagnetic contrast agent.

Subsequently, a first characterization of the examination region is fed to the predictive model according to the invention. The predictive model has been trained in advance for predicting one or more second characterizations based on the received first characterization, the one or more second characterizations showing a liver or liver portion of the examination subject without contrast enhancement by the contrast agent. The predictive model is preferably created during supervised machine learning by means of a self-learning algorithm. For learning is training data comprising a plurality of representations of the examination areas of the liver or liver portions of a plurality of examination subjects during a dynamic phase as well as during a hepatobiliary phase. Furthermore, the training data also comprises a representation of the examination region where no contrast enhancement is present, i.e. a representation generated without the administration of contrast agent.

The embodiments described herein are also schematically depicted in fig. 1 (c).

Example 5

In another preferred embodiment, the invention is used to reduce patient examination time in dynamic contrast-enhanced magnetic resonance imaging of the liver.

Here, the contrast agent is administered in the form of two boluses. The first administration is completed at a point in time when the examination subject is not yet located in the MRT scanner. In the case of the first application, a first contrast agent is applied. The first contrast agent is preferably administered intravenously (e.g., into an arm vein) in a bolus, and the dose administered is based on body weight. The first contrast agent is preferably a hepatobiliary contrast agent, such as Gd-EOB-DTPA or Gd-BOPTA. In a particularly preferred embodiment, the first contrast agent is a substance or mixture of substances having gadoleracetic acid or gadoleracete as contrast-enhancing active substance. Very particular preference is given to the disodium salt of gadofostip (Gd-EOB-DTPA disodium).

After the application of the first contrast agent, a time span may be waited, after which the examination object is introduced into the MRT scanner and a first MRT image is generated at a first point in time.

The time span between the first application and the generation of the first MRT image is preferably in the range of 5 minutes to 1 hour, more preferably in the range of 10 minutes to 30 minutes, most preferably in the range of 8 minutes to 25 minutes.

The first MRT image represents the liver or liver portion of the subject during the hepatobiliary phase after administration of the first contrast agent. In the first MRT image, healthy liver cells are depicted as having contrast enhancement due to the administration of the first contrast agent.

In this specification, the liver and gall stage in which the first MRT image is generated is also referred to as the first liver and gall stage. The first contrast agent has reached healthy liver cells resulting in contrast enhancement and in the case of paramagnetic contrast agents, in signal enhancement of healthy liver cells. During arterial, portal and late phases that occur after administration of the first contrast agent, no MRT image is generated. In this specification, arterial phase, portal venous phase and advanced phase that occur after administration of the first imaging agent are also referred to as first arterial phase, first portal venous phase and first advanced phase.

It is contemplated that multiple MRT images are generated during the first liver biliary phase.

After the one or more first MRT images are generated at the first stage of the liver and gall, a second administration of contrast agent is performed. The second administration is of a second contrast agent. The second contrast agent may be the same contrast agent as the first contrast agent; however, the second contrast agent may also be a different contrast agent, preferably an extracellular contrast agent. Likewise, the second contrast agent is preferably administered intravenously (e.g., into an arm vein) in a bolus, and the dose administered is based on body weight.

In this specification, the application of the first contrast agent is also referred to as a first application; in this specification, administration of a second contrast agent is also referred to as a second administration. If the first contrast agent is the same as the second contrast agent, a first administration of the hepatobiliary contrast agent occurs and at a later point in time, a second administration of the hepatobiliary contrast agent occurs. If the first contrast agent and the second contrast agent are different, a first administration of the first contrast agent, which is a hepatobiliary contrast agent, occurs, and a second administration of the second (different) contrast agent at a later point in time, occurs thereafter.

At the point in time of the second administration (or at the point in time of the administration of the second contrast agent), the examination subject is preferably already located in the MRT scanner. After administration of the second contrast agent, arterial phase, portal phase and late phase are again passed. In this specification, the arterial phase, portal venous phase, and late phase are also referred to as a second arterial phase, a second portal venous phase, and a second late phase. One or more MRT images are generated during the second arterial phase and/or the second portal venous phase and/or the second late phase. The MRT images are also referred to as second, third, fourth, etc. in terms of their acquisition order.

In a preferred embodiment, a second MRT image is generated during a second arterial phase, a third MRT image is generated during a second portal venous phase, and a fourth MRT image is generated during a second advanced phase. Such a second MRT image shows in particular arteries with contrast enhancement; such a third MRT image shows in particular veins with contrast enhancement.

It is also conceivable to generate more than one MRT image during the stated period.

From the MRT images generated during one or more periods after the administration of the first and second contrast agents, an artificial MRT image can be calculated.

The purpose of generating artificial MRT images from measured MRT images is to increase the contrast between healthy liver tissue and other regions. When using a hepatobiliary paramagnetic contrast agent as the first contrast agent, the signal intensity of healthy liver tissue during the second arterial phase, the second portal venous phase and the second late phase may still be increased by administration of the first contrast agent. The second contrast agent that diffuses during the stated second period also causes an increase in the signal of the tissue in which the second contrast agent diffuses. This means that in the MRT image there is only a low contrast between healthy liver tissue and the rest of the tissue, which has contrast enhancement due to the action of the (second) contrast agent. To increase this contrast, at least one artificial MRT image is generated by means of a predictive model, which will show an examination region as during the dynamic phase after administration of the first contrast agent, or as after administration of the second contrast agent only: blood vessels are depicted as having contrast enhancement due to administration of the second contrast agent, but healthy liver cells are not depicted as having contrast enhancement due to administration of the first contrast agent. In other words, an artificial MRT image similar to the second MRT image is generated, except that the contrast enhancement of healthy liver cells due to the administration of the first contrast agent is subtracted (eliminated) from the second MRT image.

The embodiments described herein are also schematically depicted in fig. 1 (d).

Claims

1. A computer-implemented method comprising the steps of:

feeding a plurality of first characterizations to a predictive model, wherein the predictive model has been trained based on first reference characterizations of an examination region of a plurality of examination objects, thereby generating one or more second reference characterizations from the first reference characterizations, wherein at least some of the first reference characterizations represent examination regions in frequency space during a first time span after administration of a contrast agent, the one or more second reference characterizations representing examination regions in frequency space during a second time span,

-outputting one or more characterizations of an examination region in the real space.

2. The method of claim 1, wherein the plurality of first characterizations comprises:

-at least one characterization of the examination region in frequency space, representing the examination region prior to administration of the contrast agent, and

at least one characterization of the examination region in frequency space representing the examination region in a first time span after administration of the contrast agent,

and wherein the one or more predictive characterizations represent the examination region in a second time span, wherein the second time span is subsequent to the first time span.

3. The method of claim 1, wherein the plurality of first characterizations includes at least two characterizations of the examination region in frequency space that represent the examination region in a first time span after administration of the contrast agent, and wherein the one or more predictive characterizations represent the examination region in a second time span, wherein the second time span precedes the first time span.

4. The method of claim 1, wherein the plurality of first characterizations comprises:

at least one characterization of the examination region in the frequency space representing the examination region in a first time span after the application of the first contrast agent,

At least one characterization of the examination region in frequency space representing the examination region in a first time span after administration of a second contrast agent, wherein the first contrast agent and the second contrast agent are identical or different, wherein the second contrast agent is administered after the first contrast agent,

and wherein the one or more predictive characterizations represent an examination region in a second time span, wherein the second time span precedes the first time span.

5. The method of claim 1, wherein the plurality of predictive characterizations represent an examination region in the second time span in which contrast enhancement is constant over time.

6. The method of any one of claims 1 to 5, wherein the predictive model is or comprises an artificial neural network.

7. The method of any of claims 1 to 6, wherein the first characterization of the examination region in the frequency space is k-space data of a magnetic resonance imaging examination.

8. The method according to any one of claims 1 to 6, wherein in a first step a plurality of radiological images of the examination region in real space are received and these received radiological images are converted into a first representation of the examination region in frequency space by fourier transformation.

9. The method according to any one of claims 1 to 8, comprising the steps of:

downscaling the first representation to the specified region,

feeding a plurality of reduced first representations to a predictive model,

augmenting the one or more second representations with one or more regions of the received first representations that lie outside the specified region,

10. The method according to any of claims 1 to 9, wherein the one or more augmented second characterizations are transformed into one or more characterizations of the examination region in real space by means of an inverse fourier transform.

11. A system, comprising:

a receiving unit which receives the signal from the receiving unit,

control and calculation unit, and

an output unit which outputs the output signal of the first output unit,

wherein the control and calculation unit is configured to:

feeding the plurality of first characterizations to a predictive model, wherein the predictive model has been trained based on first reference characterizations of an examination region of a plurality of examination objects, thereby generating one or more second reference characterizations from the first reference characterizations, wherein at least some of the first reference characterizations represent examination regions in frequency space during a first time span after administration of a contrast agent, the one or more second reference characterizations representing examination regions in frequency space during a second time span,

-causing the output unit to output one or more representations of an examination region in the real space.

12. A computer program product comprising a computer program loadable into the internal memory of a computer system, wherein the computer program causes the computer system to perform the steps of:

13. Use of a contrast agent in a method for predicting at least one radiological image, wherein the method comprises the steps of:

14. A contrast agent for use in a method for predicting at least one radiological image, wherein the method comprises the steps of:

15. A kit comprising a contrast agent and a computer program product according to claim 12.