CN117422784A

CN117422784A - Image reconstruction method and system based on rapid sampling score generation model

Info

Publication number: CN117422784A
Application number: CN202311388868.9A
Authority: CN
Inventors: 伍伟文; 王堰阳; 李子荣
Original assignee: Sun Yat Sen University
Current assignee: Sun Yat Sen University
Priority date: 2023-10-24
Filing date: 2023-10-24
Publication date: 2024-01-19

Abstract

The invention discloses an image reconstruction method and system based on a rapid sampling score generation model, wherein the method comprises the following steps: obtaining a score generation model and a limited angle CT image; carrying out noise injection processing on the limited angle CT image through a forward process of the score generation model to generate Gaussian noise characteristics; and carrying out image reconstruction on the limited-angle CT image through a backward process of the score generation model according to Gaussian noise characteristics, and generating a CT reconstructed image. The invention greatly reduces the sampling steps of image reconstruction, remarkably accelerates the sampling process, improves the efficiency and the speed of image reconstruction, can effectively relieve the negative influence of directional artifacts generated by limited angle CT scanning in the image reconstruction process, can maintain the clear edge and detail characteristics of the image while realizing quick reconstruction, and improves the quality and the effect of image reconstruction. The invention is applied to the technical field of image reconstruction.

Description

Image reconstruction method and system based on rapid sampling score generation model

Technical Field

The invention relates to the technical field of image reconstruction, in particular to an image reconstruction method and an image reconstruction system based on a rapid sampling score generation model.

Background

Limited-angle computed tomography (LACT) reconstruction is a classical inverse problem, and the resulting reconstructed images exhibit edge divergence and artifacts, and these degraded images present significant challenges for clinical diagnosis, so obtaining high quality images from Limited-angle data is a difficult task. In recent years, a Score model-based generation model (Score-Based Generative Models, SGM) has been attracting attention due to its excellent complex data distribution modeling capability, and a typical SGM employs a two-stage generation scheme including a forward stage of introducing noise into original data and a reverse stage of recovering the original data from the noise, and SGM shows remarkable achievement in application tasks of limited-angle CT reconstruction.

In the application task of limited angle CT reconstruction, the existing SGM technology generally has a problem: the sampling rate of SGM is slow. Taking the latest SGM models DOLCE and DPS as examples, the sampling step sizes of these models are typically set to 1000 or 2000, which results in about 40 minutes for reconstructing one image, which severely limits the application of SGM models in clinical diagnosis. In order to improve the efficiency of the sampling process, the related art adopts a method of increasing the sampling interval to realize skip sampling, thereby accelerating the sampling rate. For example, the de-noising diffusion implicit model (Denoising Diffusion Implicit Models, DDIM) improves the sampling process by employing a skip sampling technique that achieves fast sampling at fixed intervals and continuously increases speed by a constant coefficient. However, when skip sampling is implemented, the noise variance range of the skip sampling is too wide, the number of sampling points of the skip sampling is small, part of complex details are lost due to the large noise range, and the model is high in instability due to the reduction of the number of the sampling points, so that the effect of image reconstruction is reduced, and an unstable and low-quality sampling result is generated.

Disclosure of Invention

The present invention aims to solve at least one of the technical problems existing in the prior art to a certain extent.

To this end, it is an object of the present invention to provide an image reconstruction method and system that generates a model based on a fast sampling score.

In order to achieve the technical purpose, the technical scheme adopted by the embodiment of the invention comprises the following steps:

in one aspect, an embodiment of the present invention provides an image reconstruction method for generating a model based on a rapid sampling score, including the steps of:

obtaining a score generation model and a limited angle CT image;

performing noise injection processing on the limited angle CT image through the forward process of the score generation model to generate Gaussian noise characteristics;

and carrying out image reconstruction on the limited angle CT image through a backward process of the score generation model according to the Gaussian noise characteristics, and generating a CT reconstructed image.

In addition, the image reconstruction method based on the rapid sampling score generation model according to the above embodiment of the present invention may further have the following additional technical features:

further, in one embodiment of the present invention, the performing, by the forward process of the score generation model, noise injection processing on the limited angle CT image, generating gaussian noise features includes:

Defining a forward process of the score generation model by a stochastic differential equation;

and injecting noise features into the limited angle CT image through the forward process to generate Gaussian noise features.

Further, in an embodiment of the present invention, the generating a CT reconstructed image by performing image reconstruction on the limited angle CT image according to the gaussian noise feature through a backward process of the score generation model includes:

acquiring time sequence information of a backward process, dividing the backward process into a plurality of time sequence reconstruction stages according to the time sequence information, wherein the Gaussian noise characteristic is used as an input characteristic of a first time sequence reconstruction stage;

determining a current time sequence reconstruction stage, sampling input features of the current time sequence reconstruction stage, and generating image sampling features of the current time sequence reconstruction stage;

taking the image sampling characteristic of the current time sequence reconstruction stage as an input characteristic of a next time sequence reconstruction stage, taking the next time sequence reconstruction stage as the current time sequence reconstruction stage, and returning to the step of determining the current time sequence reconstruction stage;

when the sampling of all time sequence reconstruction stages is completed, taking the image sampling characteristic of the last time sequence reconstruction stage as a CT reconstructed image and outputting the CT reconstructed image.

Further, in an embodiment of the present invention, the sampling the input feature of the current time sequence reconstruction stage to generate an image sampling feature of the current time sequence reconstruction stage includes:

performing skip sampling on the input features of the current time sequence reconstruction stage to obtain first sampling features;

performing time backtracking sampling on the first sampling feature to obtain a second sampling feature, wherein the time sequence of the second sampling feature is earlier than that of the first sampling feature;

resampling the second sampling feature to obtain a third sampling feature, wherein the time sequence of the third sampling feature is the same as the time sequence of the first sampling feature;

and performing compressed sensing processing on the third sampling characteristic to generate an image sampling characteristic of the current time sequence reconstruction stage.

Further, in an embodiment of the present invention, the step of skip sampling the input feature of the current timing reconstruction stage to obtain a first sampled feature includes:

acquiring a noise interval of skip sampling, and calculating to obtain a first noise scale according to the noise interval;

and performing skip sampling on the input features of the current time sequence reconstruction stage according to the first noise scale to obtain first sampling features.

Further, in an embodiment of the present invention, the performing time retrospective sampling on the first sampling feature to obtain a second sampling feature includes:

acquiring step length information of time backtracking, and calculating to obtain a second noise scale according to the step length information;

and performing time retrospective sampling on the first sampling feature according to the second noise scale, so that the time sequence of the first sampling feature is gradually close to the time sequence of the input feature in the current time sequence reconstruction stage, and generating a second sampling feature.

Further, in an embodiment of the present invention, the resampling the second sampling feature to obtain a third sampling feature includes:

acquiring a sampler for resampling, wherein the sampler comprises a predictor and a corrector;

and resampling the second sampling feature by the predictor, correcting the resampled feature by the corrector, and further obtaining a third sampling feature.

Further, in an embodiment of the present invention, the performing compressed sensing processing on the third sampling feature to generate an image sampling feature of the current time sequence reconstruction stage includes:

Acquiring a sampling operator;

and performing compressed sensing processing on the third sampling feature through the sampling operator to generate the image sampling feature of the current time sequence reconstruction stage.

Further, in one embodiment of the present invention, the sampling operator is a sampling operator based on a diagonal total regularization term.

In another aspect, an embodiment of the present invention provides an image reconstruction system for generating a model based on a rapid sampling score, including:

the acquisition module is used for acquiring the score generation model and the limited angle CT image;

the forward processing module is used for carrying out noise injection processing on the limited-angle CT image through the forward process of the score generation model to generate Gaussian noise characteristics;

and the image reconstruction module is used for reconstructing the image of the Gaussian noise characteristic through the backward process of the score generation model to generate a CT reconstructed image.

The beneficial effects of the invention are as follows: the image reconstruction method and the system based on the rapid sampling score generation model are provided, so that the sampling steps of image reconstruction can be greatly reduced, the sampling process is remarkably accelerated, the efficiency and the speed of image reconstruction are improved, and rapid reconstruction is realized; meanwhile, in the image reconstruction process, the negative influence of orientation artifact generated by limited angle CT scanning can be effectively relieved, the clear edge and detail characteristics of the image can be maintained while the image is rapidly reconstructed, and the quality and effect of image reconstruction are improved.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. The objectives and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description is made with reference to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art, and it should be understood that the drawings in the following description are only for convenience and clarity of describing some embodiments in the technical solutions of the present invention, and other drawings may be obtained according to these drawings without the need of inventive labor for those skilled in the art.

FIG. 1 is a flow chart of an image reconstruction method based on a rapid sample score generation model provided by the present invention;

FIG. 2 is a schematic diagram of an image reconstruction method based on a rapid sample score generation model provided by the present invention;

FIG. 3 is a schematic diagram of a rapid sampling provided by the present invention;

FIG. 4 is a flow chart of the fast sampling of any timing reconstruction stage provided by the present invention;

FIG. 5 is a distortion contrast plot of a limited angle CT image provided by the present invention;

FIG. 6 is a graph showing a comparison of the TIFA method on a reconstruction task of a 90-angle CT image with other comparison methods;

FIG. 7 is a graph showing another effect comparison between the TIFA method and other comparison methods on the reconstruction task of 90-angle CT images provided by the invention;

FIG. 8 is a graph showing a comparison of the TIFA method on a 60-angle CT image reconstruction task and other comparison methods;

FIG. 9 is a graph showing another effect comparison between the TIFA method and other comparison methods on the reconstruction task of the 60-angle CT image provided by the invention;

FIG. 10 is a graph showing a further effect comparison of the TIFA method on a 90-angle CT image reconstruction task and other comparison methods provided by the present invention;

FIG. 11 is a graph showing the comparison of the TIFA method on the reconstruction task of 90-angle CT image with DPS and DOLCE methods.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The present application is further described below with reference to the drawings and specific examples. The described embodiments should not be construed as limitations on the present application, and all other embodiments, which may be made by those of ordinary skill in the art without the exercise of inventive faculty, are intended to be within the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with one another without conflict.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.

Limited-angle computed tomography (LACT) reconstruction is a classical inverse problem, and the resulting reconstructed images exhibit edge divergence and artifacts, and these degraded images present significant challenges for clinical diagnosis, so obtaining high quality images from Limited-angle data is a difficult task. In recent years, a Score model-based generation model (Score-Based Generative Models, SGM) has been attracting attention due to its excellent complex data distribution modeling capability, such as a denoising diffusion probability model (Denoising Diffusion Probabilistic Models, DDPM), a random differential equation (Stochastic Differential Equations, SDE), a denoising diffusion implicit model (Denoising Diffusion Implicit Models, DDIM), and the like, and SGM has shown remarkable achievement in application tasks of limited angle CT reconstruction. A typical SGM employs a two-stage generation scheme that includes a forward stage to introduce noise into the original data and a reverse stage to recover the original data from the noise, which is typically implemented using a parameterized deep neural network, e.g., the SGM can be fine-tuned with training data.

In the application task of limited angle CT reconstruction, the existing SGM technology generally has a problem: the sampling rate of SGM is slow. Taking the latest SGM models DOLCE and DPS as examples, the sampling step sizes of these models are typically set to 1000 or 2000, which results in about 40 minutes for reconstructing one image, which severely limits the application of SGM models in clinical diagnosis. In order to improve the efficiency of the sampling process, the related art adopts a method of increasing the sampling interval to realize skip sampling, thereby accelerating the sampling rate. For example, DDIM improves the sampling process by employing a skip sampling technique that achieves fast sampling at fixed intervals and continuously increases speed by a constant factor.

However, in order to achieve such accelerated sampling, a trade-off between reconstruction quality and sampling rate is typically required. Specifically, when skip sampling is realized, the noise variance range of the skip sampling is too wide, the number of sampling points of the skip sampling is small, the noise distribution of the skip sampling and the distribution condition of the number of the sampling points can effectively accelerate the sampling process, however, a part of complex details can be lost due to the large noise range, and a model is high in instability due to the reduction of the number of the sampling points, so that the effect of image reconstruction is reduced, and an unstable and low-quality sampling result is generated. Since skip sampling does not balance between sampling rate and reconstruction quality, merely pursuing accelerated sampling by expanding the sampling interval has proven to be unsuitable.

Therefore, how to achieve rapid image reconstruction while maintaining quality and effect of image reconstruction in the application task of limited angle CT reconstruction becomes one of the problems to be solved.

Aiming at the problems and defects existing in the related art, the embodiment of the invention provides an image reconstruction method and an image reconstruction system based on a rapid sampling score generation model, wherein a generation model (Fractional Derivative-Based Generative Model, SDE) based on differential derivatives is introduced, noise is gradually injected into a limited angle CT image to be reconstructed by utilizing a forward process of the SDE, and then the noisy data is gradually denoised by utilizing a backward process of the SDE, so that a CT reconstruction image is obtained, and further the reconstruction of the limited angle CT image is realized. In order to accelerate coarse noise and finely sample fine noise, the invention introduces large-scale skip sampling, resampled time backtracking sampling and compressed sensing sampling in the reverse process, can effectively capture rough structural details under significant noise level and obtain finer characteristics through the large-scale skip sampling and resampled time backtracking sampling, and can reduce the negative influence of directional artifacts on diagonals of a limited angle CT image through introducing a diagonal total variation (Diagonal Total Variation, DTV) regularization term in the compressed sensing sampling.

First, an implementation procedure of the image reconstruction method based on the rapid sampling score generation model according to the embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The method in the embodiment of the invention can be applied to the terminal, the server, software running in the terminal or the server and the like. The terminal may be, but is not limited to, a tablet computer, a notebook computer, a desktop computer, etc. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligent platforms.

Referring to fig. 1 and 2, fig. 1 is a flowchart of an image reconstruction method based on a rapid sampling score generation model provided by the present invention, fig. 2 is a schematic diagram of an image reconstruction method based on a rapid sampling score generation model provided by the present invention, in which the training process is a forward process, which follows the training of a random differential equation SDE, the testing process is a reverse process, which introduces a large-scale skip sampling, a time backtracking sampling with resampling, and a compressed sensing sampling, the method may include, but is not limited to, the steps of:

S100, acquiring a score generation model and a limited angle CT image.

Note that the score generation model is an improved SDE model, which is a typical example of the generation model field, and is widely used in the medical image field to solve the inverse problem.

Specifically, on the basis of a typical SDE model, the embodiment of the invention maintains the original forward process of the SDE, introduces large-scale skip sampling, time backtracking sampling with resampling and compressed sensing sampling based on a DTV regularization term in the backward process of the SDE, further constructs an improved SDE model, and realizes image reconstruction by utilizing the improved SDE model.

S200, performing noise injection processing on the limited-angle CT image through a forward process of the score generation model, and generating Gaussian noise characteristics.

In the step, the forward process is the training process, and in the forward process of the score generation model, noise is gradually injected into a clean limited angle CT image, and the training process of the neural network is molded, so that Gaussian noise characteristics are generated.

S300, performing image reconstruction on the limited-angle CT image through a backward process of the score generation model according to Gaussian noise characteristics, and generating a CT reconstructed image.

In this step, the backward process of score generation model, also called iterative denoising process or test process, represents the image reconstruction stage in image reconstruction, which depicts a perturbation process, reversing the initial forward process. In the backward process of the score generation model, the backward process is divided into a plurality of ordered reconstruction stages, and in each stage, large-scale skip sampling, time backtracking sampling with resampling and compressed sensing sampling are sequentially carried out on noise data, noise images are gradually denoised through sampling operations of the stages, and then CT reconstruction images are generated.

In some embodiments of the present invention, referring to fig. 2, in step S200, the step of generating gaussian noise features by performing noise injection processing on the limited angle CT image through a forward process of the score generation model includes:

s210, defining a forward process of the score generation model through a random differential equation.

In this step, in the whole forward process, the input data x is defined as x (t) =xt, and the time t e [0,1], and the forward process of the score generation model can be represented by the following formula (1):

dx＝f(x，t)dt+g(t)dw，(1)

where f (·, t) denotes the linear drift function, g (t) denotes the scalar diffusion coefficient, vector w follows the standard brownian motion, and dt denotes an infinitesimal time step. Intuitively, given a small time step dt, data x adds normal distributed random value noise with mean f (x, t) and variance g (t) dw.

Alternatively, different combinations of linear drift functions and scalar diffusion coefficients may generate different types of SDEs. The embodiment of the invention adopts an SDE model of typical explosion variation, the linear drift function and the scalar diffusion coefficient can be respectively set as f=0,where σ (t) represents a proportional function in which the temporal variation of noise gradually increases. It should be noted, however, that in other embodiments of the invention, the linear drift function and the targetThe quantity diffusion coefficient may also be set to other parameters, which are not particularly limited in the present invention.

S220, injecting noise features into the limited-angle CT image through a forward process, and generating Gaussian noise features.

In this step, the noise characteristics are gradually injected into the limited angle CT image x through the forward process defined by the formula (1) ₀ In, gaussian noise feature x is generated _T As shown in fig. 2. Wherein x is ₀ Representing the distribution of raw data, i.e. limited angle CT images, x _T Approximately spherical gaussian, T representing the number of sampling steps.

In some embodiments of the present invention, given a forward procedure as defined in equation (1) above, a corresponding reverse procedure may be constructed, the reverse procedure being represented by the reverse procedure of equation (1), as shown in equation (2) below:

Wherein,is a time dependent scoring function. Optionally, an estimator s of this scoring function _θ Obtained by training by a noise-removal score matching method, as shown in the following formula (3):

wherein p (x) _t |x ₀ ) Corresponds to a disturbance kernel for converting data x ₀ Disturbance to noisy sample x _T Score estimator s _θ (x _t T) may replace the scoring function in equation (2).

Referring to fig. 3, fig. 3 is a schematic diagram of the fast sampling provided by the present invention, and on the basis of the reverse process defined by the formula (2), the embodiment of the present invention introduces large-scale skip sampling, time backtracking sampling with resampling, and compressed sensing sampling in the reverse process, so as to perform fast sampling through coarse noise scale and fine noise scale.

Specifically, in step S300, the limited-angle CT image is reconstructed by the backward process of the score generation model according to the gaussian noise characteristics, and the process of generating the CT reconstructed image may include, but is not limited to, the following steps:

s310, acquiring time sequence information of a backward process, and dividing the backward process into a plurality of time sequence reconstruction stages according to the time sequence information.

It should be noted that, the gaussian noise characteristic is used as the input characteristic of the first timing reconstruction stage.

In the step, the whole image reconstruction stage is divided into a plurality of time sequence reconstruction stages with a precedence relationship and a time sequence relationship according to the time sequence of the backward process. Specifically, the output of the forward process, i.e., gaussian noise characteristics, is taken as the input of the first sequential reconstruction stage, the output of the last sequential reconstruction stage is taken as the CT reconstructed image, and the output of the last sequential reconstruction stage is taken as the input of the next sequential reconstruction stage for other sequential reconstruction stages than the first sequential reconstruction stage.

S320, determining a current time sequence reconstruction stage, sampling input features of the current time sequence reconstruction stage, and generating image sampling features of the current time sequence reconstruction stage.

In the step, for the ith time sequence reconstruction stage, large-scale skip sampling, time backtracking sampling with resampling and compressed sensing sampling are carried out on the input features of the ith time sequence reconstruction stage, so that the image sampling features of the ith time sequence reconstruction stage are generated.

S330, taking the image sampling characteristic of the current time sequence reconstruction stage as the input characteristic of the next time sequence reconstruction stage, taking the next time sequence reconstruction stage as the current time sequence reconstruction stage, and returning to the step S320.

In this step, when the sampling of the ith timing reconstruction stage is completed, the image sampling feature of the ith timing reconstruction stage is used as the input of the (i+1) th timing reconstruction stage, the (i+1) th timing reconstruction stage is used as the (i) th timing reconstruction stage, i.e. let i=i+1, and then step S320 is returned.

S340, when sampling of all time sequence reconstruction stages is completed, taking the image sampling characteristic of the last time sequence reconstruction stage as a CT reconstructed image and outputting.

In this step, the Gaussian noise characteristic x is sequentially calculated by a plurality of time sequence reconstruction stages _T Sampling and denoising such that the Gaussian noise signature x _T Gradually transition to x ₀ Thereby obtaining CT reconstructed image x ₀ 。

In some embodiments of the present invention, referring to fig. 3 and 4, fig. 4 is a flowchart of a fast sampling of any one of the time series reconstruction stages provided in the present invention, and in step S320, the process of sampling the input feature of the current time series reconstruction stage to generate the image sampling feature of the current time series reconstruction stage may include, but is not limited to, the following steps:

s321, performing skip sampling on the input features of the current time sequence reconstruction stage to obtain first sampling features.

In this step, coarse noise β is realized by skip sampling _t The accelerated sampling on the scale, namely, one-step sampling on the large-scale noise is equivalent to m-step sampling on the small-scale noise, so as to obtain a first sampling characteristic x' _t 。

S322, for the first sampling feature x' _t Performing time backtracking sampling to obtain a second sampling characteristic x' _t+L 。

It should be noted that the timing of the second sampling feature is earlier than the timing of the first sampling feature.

S323, for the second sampling feature x' _t+L Resampling to obtain a third sampling characteristic

The timing of the third sampling feature is the same as the timing of the first sampling feature.

In the steps, after one-step sampling is carried out on a larger scale, time backtracking is carried out on a smaller scale, and a reconstructed image is obtainedLike from x' _t Return to x' _t+L Wherein L represents a satisfied time backtracking step, L<m. Subsequently, image resampling and reconstruction is performed under small scale noise, from x' _t+L Transition toWherein (1)>And x' _t Is the same. Through the process, time backtracking provides refinement correction for large-scale reconstruction, and the speed and stability of the image reconstruction process are improved.

S324, for the third sampling featurePerforming compressed sensing processing to generate image sampling characteristics x of the current time sequence reconstruction stage _t 。

In this step, in order to further improve the stability of image reconstruction, to achieve faster sampling efficiency and more stable image reconstruction, a diagonal total variation regularization term is introduced into compressed sensing, and a third sampling feature is obtained by compressed sensingProcessing to obtain image sampling characteristic x of current time sequence reconstruction stage _t A limited angle CT reconstruction is achieved.

Referring to FIG. 3, a conventional score-based reconstruction method, such as VE-SDE, uses sequential sampling to make Gaussian noise signatures x _T Gradually transition to CT reconstructed image x ₀ . In practical applications, the choice of T is very important, and T is typically set to be 2000 or 4000 equivalent, which results in a large computational resource overhead when solving the inverse problem. Therefore, in order to reduce the overhead of computing resources and improve sampling efficiency, methods such as DDIM implement skip sampling by increasing the sampling interval. However, the noise size selection of the skip samples spans a wide range and is not corrected in the case of large strides, which results inDownsampling fails to achieve high quality image reconstruction.

In a conventional T-step VE-SDE, the noise variance is selected to satisfyAnd->And +.>Wherein (1)>Representing the disturbance data distribution. In the image reconstruction and sampling process, the span of noise is defined as α _t ＝σ _t+1 ² -σ _t ² . In the skip-sampling score model, the noise span is larger, and by taking m-step skip sampling as an example, the noise span can be expressed as beta _t ＝σ _t+m ² -σ _t ² . Obviously, regardless of the value of t, beta _t Always greater than alpha _t This means that the noise is coarse when downsampling is achieved, typically coarse noise will lead to poor reconstruction results. In order to obtain a better image reconstruction result, the embodiment of the invention introduces large-scale skip sampling, time backtracking sampling with resampling and compressed sensing sampling in a backward process.

The implementation of large scale skip sampling, time backtracking sampling with resampling, and compressed sensing sampling introduced in the backward process by embodiments of the present invention will be described in detail below with reference to fig. 2-5.

1. The implementation process of the large-scale skip sampling is as follows:

in the embodiment of the invention, the fast sampling is based on large-scale skip sampling. In particular, large scale skip sampling, i.e. sampling at coarse noise levels, is achieved by increasing the noise interval during the sampling process. In step S321, the process of skip sampling the input feature of the current timing reconstruction stage to obtain the first sampling feature mainly includes the following steps:

The first step, obtaining a noise interval of skip sampling, and calculating according to the noise interval to obtain a first noise scale.

In a common scoring model, T is typically set to 2000 or 4000, with a corresponding sampling noise scale of α _t ＝σ _t+1 ² -σ _t ² . In the large-scale jump of the embodiment of the invention, a noise interval m is acquired, and a first noise scale beta is calculated according to the interval m _t ＝σ _t+m ² -σ _t ² This increases the sampling rate by a factor of m compared to the traditional scoring model.

Step two, input characteristics x of the current time sequence reconstruction stage are calculated according to the first noise scale _t+m Skip sampling is carried out to obtain a first sampling characteristic x' _t 。

Optionally, the input feature x at the current timing reconstruction stage _t+m The step of performing skip sampling may include:

obtaining predictors and correctors for skip sampling, input features x of current timing reconstruction stage by predictors _t+m Predicting, correcting the output of the predictor by a corrector to obtain a first sampling characteristic x' _t 。

In the skip sampling process, a data consistency method is adopted to carry out iteration constraint on the large-scale skip sampling, so that forward iteration in the backward process is realized. Alternatively, a joint iterative reconstruction method (Simultaneous Iterative Reconstruction Technique, SIRT) is employed as the data consistency method.

In this step, the sampler is preset with iteration times, and at least one iteration is performed according to the iteration times. In a single iteration, the predictor inputs the feature x for the current timing reconstruction stage _t+m Performing prediction, namely performing skip sampling, then performing data consistency constraint on the output of the predictor through SIRT, inputting the updated output of the predictor into a corrector for correction, performing data consistency constraint on the output of the corrector through SIRT, and updatingThe output of the corrector is returned to the predictor for iteration, the last iteration outputting the first sampled characteristic x' _t 。

2. The implementation process of the time backtracking sampling with resampling is as follows:

in the embodiment of the invention, the time backtracking sampling process is related to forward noise injection. During the time backtracking process, from x' _t To x' _t+1 The image of (2) may be represented by the following formula (4):

based on this, the recursive derivation continues to x' _t+L The following formula (5) can be obtained:

by re-parameterizing the above equation (5), the following equation (6) can be obtained:

this means that the average value of x 'can be directly passed through' _t Sum covariance matrix ofIs used to implement the time-retrospective sampling.

Based on this, in step S322, the step of performing time trace sampling on the first sampling feature to obtain the second sampling feature mainly includes:

Step one, acquiring step length information of time backtracking, and calculating to obtain a second noise scale according to the step length information.

In this step, step information l of time backtracking is obtained, and a second noise scale sigma is calculated according to the step information l _t+l ² -σ _t+l-1 ² 。

And secondly, performing time retrospective sampling on the first sampling feature according to a second noise scale, so that the time sequence of the first sampling feature is gradually close to the time sequence of the input feature in the current time sequence reconstruction stage, and generating a second sampling feature.

In this step, according to the second noise scale sigma _t+l ² -σ _t+l-1 ² The foregoing formula (6) is obtained, and the mean value shown by the formula (6) is x' _t Sum covariance matrix ofIs used for the distribution of the first sampling characteristic x' _t Time backtracking sampling is performed such that a first sampled feature x' _t Reverse transition to second sample feature x' _t+L 。

Second sampling feature x 'obtained after time backtracking' _t+L It is necessary to go through a resampling inverse to return to x' _t . In the embodiment of the invention, a PC sampler is adopted to sample the second sampling characteristic x' _t+L Resampling and reconstruction are performed. The PC sampler is composed of two components: a predictor for numerically solving the SDE and a corrector for refining the results using a score-based approach. Specifically, discretizing the sampling process based on the contrast-driven score model, VE-SDE, can yield equation (7) that characterizes the predictor's prediction process:

Wherein the initial value L is equal to L.

Regarding the step of the corrector, its update follows formula (8):

based on this, in step S323, the second sampled feature is resampled to obtain a third sampled feature, including:

the method comprises the steps that firstly, a sampler for resampling is obtained, wherein the sampler comprises a predictor and a corrector;

second step, the second sampling feature x is processed by the predictor _t+L Resampling is carried out, and the feature obtained by resampling is corrected by a corrector, so as to obtain a third sampling feature x _t 。

Optionally, in the resampling process, the data consistency method is adopted to carry out iteration constraint on the resampling process, so that forward iteration in the backward process is realized.

Alternatively, SIRT is used as a data consistency method.

In this step, the sampler is preset with iteration times, and at least one iteration is performed according to the iteration times. In a single iteration, the predictor samples the second sample characteristic x _t+L Prediction, i.e. resampling, then data consistency constraint is carried out on the output of the predictor through SIRT, the updated output of the predictor is input into a corrector for correction, data consistency constraint is carried out on the output of the corrector through SIRT, the updated output of the corrector is returned into the predictor for iteration, and the third sampling characteristic x is output in the last iteration _t 。

Illustratively, the output of the corrector is constrained by SIRT for data consistency, as shown in equation (9) below:

wherein d=diag {1/||b } ₁ ||，1/||b ₂ ||，…，1/||b _q The || } represents a diagonal matrix, b _q Representation matrixIs the q-th row vector of (c).

3. The implementation process of compressed sensing sampling is as follows:

referring to fig. 5, fig. 5 is a distortion contrast diagram of a limited angle CT image provided by the present invention, and it can be seen from fig. 5 that the distortion of the limited angle CT image is mainly caused by stretching of a diagonal, and that a diagonal position shown by a circle can be provided with obvious artifacts, and that the artifacts on the diagonal are significantly more than other regions. In compressed sensing techniques, conventional TV regularization uses gradient operators only in the horizontal and vertical directions to sparsely characterize the image, while shape distortions caused by limited angles are predominantly reflected in the diagonal direction. Therefore, computing the TV operator along the diagonal direction of the image is more consistent with the reconstruction task of limited angle CT images. Furthermore, a large noise span will lead to an unstable and low quality sampling result during a large jump from m steps. Therefore, the embodiment of the invention introduces a regularization term of total variation of the diagonal, namely a regularization term of the DTV, in compressed sensing sampling by combining the noise span problem caused by large-scale jump and the diagonal artifact problem of the limited-angle CT image.

Specifically, in step S324, the compressed sensing processing is performed on the third sampling feature, and the process of generating the image sampling feature of the current time sequence reconstruction stage may include, but is not limited to, the following steps:

first, a sampling operator is obtained.

It should be noted that, the sampling operator is a sampling operator based on the DTV regularization term.

In this step, as t decreases, the amplitude of the image change decreases accordingly, so that the weight μ (t) defining the DTV regularization term adjusts as t changes. The operation of the sampling operator based on the DTV regularization term is shown in the following formula (10):

wherein,representing a sampling operator, x, based on a regularization term of a DTV _i,j Representing the pixel value x.

And secondly, performing compressed sensing processing on the third sampling feature through a sampling operator to generate an image sampling feature of the current time sequence reconstruction stage.

In this step, the artifacts in the diagonal region are corrected by DTV transformation, thereby realizing image reconstruction in the current time-series reconstruction stage.

Then, another implementation step of the image reconstruction method based on the rapid sampling score generation model provided by the embodiment of the present invention will be described in detail below.

The problem of limited angle reconstruction in limited angle CT imaging can be expressed as Wherein (1)>Representing an underdetermined matrix, y representing the measured limited angle projection and x representing the image to be reconstructed. In the context of the reconstruction task, p is different from the sampling process shown in equation (2) due to the observability as a priori information y _t (x) The influence of the observed a priori information y results in a change in the sampling process shown in equation (2), as shown in equation (11) below:

wherein p is _t (x|y) represents a posterior probability term affected by y. Expanding p in a formula using bayesian theorem _t (x|y) the result shown in the following formula (12) can be obtained:

wherein the super parameter lambda balances prior information of the model and posterior distribution terms of measured dataTo implement the sampling process outlined in equation (12), embodiments of the present invention are implemented by constructing an initial objective function as shown in equation (13):

the initial objective function represents an optimization equation that solves the finite angle inverse problem using a scored model, where u represents a priori information of the data, solved using an SDE model, λ ₁ Weights representing a priori information. Typically, the sampling process of u requires 2000 and more iterative steps, requiring a significant amount of time and computational resources.

In order to reduce the sampling time and improve the effect and quality of image reconstruction, the embodiment of the invention introduces large-scale skip sampling, time backtracking sampling with resampling and compressed sensing sampling in the backward process, and optimizes the initial objective function shown in the formula (13) based on the sampling time.

Considering the DTV regularization term, large-scale skip sampling, time-retrospective sampling with resampling, the initial objective function shown in equation (13) above can be converted into a final objective function shown in equation (15) below:

/>

wherein u is _t Represents terms optimized using large scale skip sampling and time trace-back sampling with resampling based on score model, μ _t Is the weight of the regularization term of the DTV,is a regularization term.

Based on this, the image reconstruction method based on the rapid sampling score generation model provided by the embodiment of the invention can also be the following steps:

a100, obtaining a score generation model and a limited angle CT image.

Step a100 is performed in the same manner as step S100 of the previous embodiment.

And A200, performing noise injection processing on the limited angle CT image through a forward process of the score generation model, and generating Gaussian noise characteristics.

Step a200 is performed in the same manner as step S200 of the previous embodiment.

A300, constructing an objective function used for representing a backward process of the score generation model, solving the objective function, and performing image reconstruction on the limited-angle CT image to generate a CT reconstructed image.

Unlike the implementation of step S300 in the previous embodiment, this step further integrates the whole process of reconstructing the limited-angle CT image into one objective function, as shown in the above formula (15), and then solves the objective function, so as to achieve the reconstruction of the limited-angle CT image.

For the objective function shown in equation (15) above, the embodiment of the present invention uses the initial Gaussian noise signature x _T And (5) carrying out iterative solution. Starting from large scale skip sampling, the objective function shown in equation (15) can be converted to equation (16) as follows:

equation (16) represents the predictor in the PC sampler employed in large scale skip sampling, its numerical solution score model.

Subsequently, we need to correct the solution using a corrector, the correction procedure is shown in equation (17) below:

in order to impose constraints on image generation, the embodiment of the invention introduces a SIRT iterative method, which is applied to correction of model prior reconstruction by using a limited angle sinusoidal image map, and the correction is shown in the following formula (18):

Finally, since large-scale skip sampling has instability, the DTV regularization term constructed previously is used to correct the results, as shown in equation (19) below:

through the processes shown in the above formulas (16) to (19), the embodiment of the present invention has realized one-step reconstruction under large-scale noise δ. Using large scale noise delta for reconstruction allows us to reduce the number of parameters T from conventional 2000 or 4000 to 200 or 100. Compared with the traditional SDE model, the method and the device of the embodiment of the invention remarkably reduce the number of graphic reconstruction steps. However, since the skip reconstruction of the large-scale noise δ passes over the small-scale noise, a problem arises in that the image reconstruction result is unstable and inaccurate. To address this problem, embodiments of the present invention implement time backtracking and resampling processesThe small-scale noise sigma is reconstructed. As in the previous examples, the examples of the present invention are directed to x' _t Performing time backtracking to x' _t+L Then backtracking the result x 'with time' _t+L Resampling is performed to return to x' _t At the timing, this process is entirely under small scale noise σ. By iteratively executing the optimization process proposed by the embodiment of the invention, the noise characteristic x can be obtained from the pure Gaussian noise _T Downsampling a clear reconstructed image x ₀ 。

In summary, in the reconstruction task of the limited angle CT image, the invention introduces a rapid sampling mechanism in the SDE, and can realize rapid reconstruction and simultaneously maintain clear edges and detail characteristics of the image. The reconstruction process is as follows: and gradually injecting noise into the limited-angle CT image to be reconstructed by utilizing the forward process of the SDE, and then gradually denoising the noisy data by utilizing the backward process of the SDE to obtain a CT reconstructed image, thereby realizing the reconstruction of the limited-angle CT image.

The mechanism of fast sampling is centrally embodied in the backward process of the SDE, and the whole sampling process follows the principles of robust optimization theory and can be built on a complete data model (as the objective function shown in equation (15) above). The fast sampling mechanism includes large scale skip sampling, time backtracking sampling with resampling, and compressed sensing sampling implemented in each stage. First, in an initial large-scale jump phase, initial results are obtained by skipping multiple sampling steps. Then, in the time backtracking sampling step, the initial result is destroyed controllably by introducing small scale noise. Thereafter, in the resampling step, the initially compromised results are carefully fine-tuned using specialized resampling techniques. Finally, fine tuning is carried out on the fine result by introducing a DTV regularization term into the compressed sensing sampling, and the sampling of the current stage is completed. Image reconstruction can be achieved through multiple stages of sampling iterations.

The invention can greatly reduce the sampling steps of image reconstruction, remarkably quicken the sampling process, improve the efficiency and the speed of image reconstruction and realize quick reconstruction; meanwhile, in the image reconstruction process, the negative influence of orientation artifact generated by limited angle CT scanning can be effectively relieved, the clear edge and detail characteristics of the image can be maintained while the image is rapidly reconstructed, and the quality and effect of image reconstruction are improved.

The image reconstruction method based on the rapid sampling score generation model proposed by the embodiment of the present invention is verified by the following embodiment. The method proposed by the embodiments of the present invention is referred to as the TIFA method in the following embodiments.

1. And (3) selecting a comparison model: the present embodiment selects a filtered backprojection algorithm (Filtered Back Projection, FBP), a fast iterative threshold contraction algorithm (Fast Iterative Shrinkage-Thresholding Algorithm, FISTA), a deep convolutional neural network (FBPConvNet), a DPS model, and a DOLCE model as comparison methods.

2. Construction of a data set: the analog dataset from the AAPM2016CT low dose challenge was selected as the first dataset for the image reconstruction task. In order to generate 90 degree and 60 degree limited angle CT images, the present embodiment employs an equiangular fan beam projection geometry algorithm to process image data of a first dataset such that images of the dataset generate image data from both 60 degree and 90 degree angles, and the generated image data is added to the first dataset. The first data set is then split into a training set and a test set in a 10:1 ratio. Furthermore, the real cardiac clinical dataset is selected as the second dataset for the image reconstruction task.

3. Evaluation index of image reconstruction task: with Peak Signal-To-Noise Ratio (PSNR) and structural similarity (Structural Similarity Index, SSIM), the rise in PSNR and SSIM corresponds To a higher level of reconstruction quality.

4. Experimental environment: experimental studies were performed using the pyrerch framework, all performed on a high performance computing system equipped with NVIDIA RTX a6000 48GB graphics processing unit. An Adam optimization algorithm is adopted during training, and the learning rate is set to be 2 multiplied by 10 ^-4 . In configuring the noise variance, a fixed value sigma is used _min =0.01 and σ _max ＝378。

5. Description of experimental data:

1) For image reconstruction tasks on the AAPM dataset:

the present embodiment first compares the performance of the image reconstruction task on the AAPM dataset by the method of the present invention and other comparison methods, where the image reconstruction task includes two aspects, namely, reconstruction of a 60-angle CT image on the one hand and reconstruction of a 90-angle CT image on the other hand.

Referring to fig. 6 and 7, fig. 6 is a graph comparing one effect of the TIFA method on the reconstruction task of 90-angle CT image provided by the present invention with that of other contrast methods, fig. 7 is a graph comparing another effect of the TIFA method on the reconstruction task of 90-angle CT image provided by the present invention with that of other contrast methods, fig. 6 and 7, "Label" in fig. 6 and 7 represents a Label image, i.e., ground Truth (Ground Truth) commonly used in model evaluation, the first row in fig. 6 and 7 is a reconstructed CT image, and the second row is a region of interest (Region Of Interest, ROI) with an angle range of [0 °,90 ° ]. As can be seen from fig. 6 and 7:

The image reconstruction results of FBP and FISTA show significant image distortion, and it is difficult for FBP and FISTA to extract information outside the skeleton structure. FBPConvNet is a typical example of a supervised deep learning approach, which yields significant improvements in the quality of image reconstruction, which can be focused on larger structures and edges. However, FBPConvNet suffers from significant drawbacks in accurately predicting reconstruction details due to the inherent limitations of limited angle measurements. Generating models DPS and DOLCE based on scores is widely considered as a benchmark approach compared to other approaches, which achieve praise reconstruction results on limited angle CT reconstruction tasks. However, when the ROI is detected, it is apparent that the region indicated by the arrow cannot be accurately restored, and that the DPS and the DOLCE cannot faithfully capture the details of the image, the difference between the image reconstruction effects of the different methods further highlights that the DPS and the DOLCE have defects in restoring the boundary structure. In contrast, the TIFA method provided by the invention can still maintain important structural details under the condition of sampling acceleration, and the comparison of the results of the different methods in FIG. 6 and FIG. 7 highlights the superiority of the TIFA method relative to the current mainstream baseline method.

Referring to table 1 below, "Method" of table 1 indicates an image reconstruction Method, "90 Limited-angle" indicates a 90-angle CT image reconstruction task, and "60 Limited-angle" indicates a 60-angle image reconstruction task. As can be seen from table 1: the TIFA method provided by the invention is superior to other methods in PSNR and SSIM aspects.

TABLE 1 TIFA and comparative method Performance on AAPM datasets

Similarly, the present embodiment continues to evaluate the effectiveness of the image reconstruction method over the 60-angle CT image task. Referring to fig. 8, fig. 8 is a graph comparing an effect of the TIFA method on a reconstruction task of a 60-angle CT image provided by the present invention with that of other comparison methods, where "Label" in fig. 8 represents a Label image, a first line in fig. 8 is a reconstructed image, and a second line is an ROI with an angle range of [0 °,60 ° ]. As can be seen from fig. 8:

under these highly sparse view conditions, there is significant noise in the image reconstruction results of the DPS, which is limited in the ability to recover fine structural details as seen by the DPS. The result of image reconstruction by the DOLCE makes it difficult for the DOLCE to accurately maintain the fidelity of the structure, which is further confirmed by examining the extracted ROI. This suggests that both reference methods of generating a model based on the score, DPS and DOLCE, fail to ensure the accuracy of each sampling step in the iterative sampling process. In order to solve the challenge, the TIFA method provided by the invention adopts time backtracking and resampling to optimize the sampling step, so that the sampling process can be accelerated, the texture complexity of the image can be captured, and compared with other methods, the TIFA method can minimize the difference between the reconstructed image and the reference true value.

2) Image reconstruction task for a real clinical cardiac dataset:

in order to verify that the TIFA method provided by the invention has stable generalization performance, the embodiment uses a model trained by an AAPM simulation data set to reconstruct heart data in clinic. Referring to fig. 9 and 10, fig. 9 is a graph comparing another effect of the TIFA method on the reconstruction task of the 60-angle CT image with other contrast methods provided by the present invention, fig. 10 is a graph comparing still another effect of the TIFA method on the reconstruction task of the 90-angle CT image with other contrast methods provided by the present invention, fig. 9 and 10, "Label" represents a Label image, "our" represents the TIFA method, the first line in fig. 9 and 10 is a reconstructed image, the second line in fig. 9 is an ROI with an angle range of [0 °,60 ° ], and the second line in fig. 10 is an ROI with an angle range of [0 °,90 ° ]. As can be seen from fig. 9 and 10:

under the condition of less limited angles, serious distortion occurs in image reconstruction results of other comparison methods, and the image reconstruction results cannot accurately reconstruct the structure of the image, so that the TIFA method provided by the invention has greater advantage in structure fidelity.

Referring to table 2, it can be seen from table 2: the TIFA method provided by the embodiment of the invention is superior to other methods in PSNR and SSIM aspects, and has good image reconstruction performance.

Table 2 expression of TIFA and comparative methods on real clinical cardiac dataset

Referring to fig. 11, fig. 11 is a graph showing the comparison of the effects of the TIFA method and the DPS and DOLCE methods on the reconstruction task of 90-angle CT images provided by the present invention, and it can be seen from fig. 11: the sampling steps of DPS and DOLCE are typically set to 2000 or 4000, which results in an image reconstruction by DPS or DOLCE taking about 40 minutes to reconstruct an image, which is inefficient. The method provided by the invention adopts 200 sampling steps, and realizes rapid image reconstruction while guaranteeing the image reconstruction quality. In a typical scenario, we propose a TIFA method that takes only 50 steps (1.35 minutes) to achieve a significant image reconstruction compared to the 2000 steps method DPS (27.67 minutes) and DOLCE.

The verification data prove that the image reconstruction method based on the rapid sampling score model provided by the invention only uses 200 sampling steps, so that the 90 DEG and 60 DEG reconstruction performance is obviously superior to other methods adopting 2000 or 4000 sampling steps. In addition, the experimental result shows that the image reconstruction method based on the rapid sampling score model provided by the invention can still provide high-quality image reconstruction functions even though the method has only 100 steps.

Therefore, compared with the method in the prior art, the method can realize rapid image reconstruction, ensure the clear edge and detail characteristics of the image, improve the quality and effect of image reconstruction, and have high generalization capability and high usability.

In addition, an embodiment of the present invention provides an image reconstruction system for generating a model based on a rapid sampling score, including:

and the image reconstruction module is used for reconstructing the image of the Gaussian noise characteristics through a backward process of the score generation model to generate a CT reconstructed image.

The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.

While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.

While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the embodiments described above, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention, and these equivalent modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.

Claims

1. The image reconstruction method for generating the model based on the rapid sampling score is characterized by comprising the following steps of:

obtaining a score generation model and a limited angle CT image;

2. The method for reconstructing an image based on a rapid sampling score generation model according to claim 1, wherein said performing a noise injection process on said limited angle CT image by a forward process of said score generation model generates gaussian noise features, comprising:

3. The method for reconstructing an image based on a rapid sampling score generation model according to claim 1, wherein said reconstructing an image of said limited angle CT image from said gaussian noise characteristics by a backward process of said score generation model, generating a CT reconstructed image, comprises:

4. The method for reconstructing an image based on a rapid sampling score generation model according to claim 3, wherein said sampling input features of said current time series reconstruction stage to generate image sampling features of said current time series reconstruction stage comprises:

5. The method for reconstructing an image based on a rapid sampling score generation model according to claim 4, wherein said skip sampling of the input features of the current time series reconstruction stage to obtain first sampled features comprises:

6. The method for reconstructing an image based on a rapid sampling score generation model according to claim 4, wherein said performing a time retrospective sampling on said first sampling feature to obtain a second sampling feature comprises:

7. The method for reconstructing an image based on a rapid sampling score generation model according to claim 4, wherein resampling said second sampled feature to obtain a third sampled feature comprises:

8. The method for reconstructing an image based on a rapid sampling score generation model according to claim 4, wherein said performing compressed sensing processing on said third sampling feature to generate an image sampling feature of said current time-series reconstruction stage comprises:

acquiring a sampling operator;

9. The rapid sampling score generation model based image reconstruction method of claim 8, wherein the sampling operator is a sampling operator based on a diagonal total score regularization term.

10. An image reconstruction system for generating a model based on a rapid sampling score, comprising: