CN113327305B - Model-driven deep learning fluorescence molecule tomography method and system - Google Patents

Model-driven deep learning fluorescence molecule tomography method and system Download PDF

Info

Publication number
CN113327305B
CN113327305B CN202110598228.5A CN202110598228A CN113327305B CN 113327305 B CN113327305 B CN 113327305B CN 202110598228 A CN202110598228 A CN 202110598228A CN 113327305 B CN113327305 B CN 113327305B
Authority
CN
China
Prior art keywords
model
distribution
fluorescence
molecular tomography
fluorophore
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110598228.5A
Other languages
Chinese (zh)
Other versions
CN113327305A (en
Inventor
邓勇
华泳州
刘锴贤
蒋宇轩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN202110598228.5A priority Critical patent/CN113327305B/en
Publication of CN113327305A publication Critical patent/CN113327305A/en
Application granted granted Critical
Publication of CN113327305B publication Critical patent/CN113327305B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0071—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Pathology (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

The invention discloses a model-driven deep learning fluorescent molecule tomography method and system. The method comprises the following steps: acquiring the distribution of surface detection fluorescence to be tested; inputting the surface detection fluorescence distribution to be detected into a fluorescence molecular tomography model to obtain the distribution of the fluorophore to be detected; the fluorescence molecular tomography model is formed by training a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescence molecular tomography image reconstruction, a residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm. The invention adopts a fluorescence molecular tomography model to establish the end-to-end mapping relation between the surface detection fluorescence distribution and the fluorophore distribution, avoids the disadvantages of the traditional model-based image reconstruction and improves the quality of the image reconstruction.

Description

Model-driven deep learning fluorescence molecule tomography method and system
Technical Field
The invention relates to the technical field of molecular imaging, in particular to a model-driven deep learning fluorescence molecular tomography method and system.
Background
Fluorescence Molecular Tomography (FMT) is a living optical Molecular imaging technology based on diffused light, has the characteristics of high sensitivity, large imaging field of view, deep imaging depth, non-invasive detection and the like, and has great application prospects in the aspects of gene expression, protein interaction, tumorigenesis and development, pharmacokinetics and the like.
The traditional FMT image reconstruction is based on model reconstruction and is characterized by being clear in physical meaning and strong in interpretability, but due to the complexity of a biological tissue structure, modeling errors are inevitably caused to forward modeling of photon propagation in a living body, and due to the high scattering characteristic of biological tissues to photons in a fluorescence waveband, the FMT reverse problem has high morbidity, so that the problems of multiple reconstructed image artifacts, insufficient positioning and quantifying precision and the like are caused, and therefore, the FMT image reconstruction quality at the present stage is difficult to break through substantially by following the traditional image reconstruction research thinking. How to improve the image reconstruction quality is a problem which needs to be solved at present.
Disclosure of Invention
The invention aims to provide a model-driven deep learning fluorescence molecule tomography method and system, which avoids the disadvantages of the traditional model-based image reconstruction and improves the quality of the image reconstruction by establishing an end-to-end mapping relation between surface detection fluorescence distribution and fluorophore distribution to be detected.
In order to achieve the purpose, the invention provides the following scheme:
a model-driven deep-learning fluorescence molecular tomography method, the method comprising:
acquiring surface detection fluorescence distribution;
inputting the surface detection fluorescence distribution into a fluorescence molecular tomography model, and reconstructing to obtain fluorophore distribution; the fluorescence molecular tomography model is trained by using a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescence molecular tomography image reconstruction, the residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm.
Optionally, the method for determining the fluorescence molecular tomography model includes:
acquiring a test set; the test set comprises a surface-detected fluorescence profile to be trained and a corresponding true fluorophore profile;
determining a regularized optimization objective function based on a diffusion approximation model of a radiation transfer equation;
calculating a gradient of the regularized optimization objective function;
developing the gradient descent algorithm according to the gradient to obtain a calculation graph under each iteration number;
carrying out parameterization processing on the regular term gradient in the calculation graph by adopting a residual block structure of a multilayer three-dimensional convolution neural network to obtain a parameterized calculation graph under each iteration number;
determining the parameterized computation graph under each iteration number as a layer of network structure, and cascading all the network structures to obtain the hierarchical network model;
and inputting the test set into the hierarchical network model for training to obtain the fluorescent molecular tomography model.
Optionally, the inputting the test set into the hierarchical network model for training to obtain the fluorescence molecular tomography model specifically includes:
inputting the distribution of the surface detection fluorescence to be trained into the hierarchical network model under the current training times to obtain the distribution of the fluorophore under the current training times;
calculating a current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution;
judging whether the current loss function difference value is smaller than a preset threshold value or not;
if so, determining the hierarchical network model under the current training times as the fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
Optionally, the regularized optimization objective function is:
Figure BDA0003091977180000031
wherein Ω (x) is an objective function to be minimized during fluorescent molecular tomography image reconstruction, Φ is surface detection fluorescence distribution, a is a forward matrix obtained by solving the diffusion approximation model based on the radiation transmission equation by finite elements, x is fluorophore distribution, λ is the regularization parameter, and m (x) is a regularization term.
Optionally, the expression of the computation graph is:
Figure BDA0003091977180000032
wherein the content of the first and second substances,
Figure BDA0003091977180000033
the gradient of the k-th iteration of the objective function is optimized for regularization,
Figure BDA0003091977180000034
k is the iteration number of the calculation graph, N is the maximum iteration number of the calculation graph, and xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations ηkFor the iteration step length of the kth iteration, A is a forward matrix obtained by using finite elements to solve the diffusion approximation model based on the radiation transmission equation, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the k-th iteration'k(xk-1) Is the regular term gradient for the kth iteration.
A model-driven deep-learning fluorescent molecular tomography system, the system comprising:
the surface detection fluorescence distribution acquisition module is used for acquiring surface detection fluorescence distribution;
the fluorophore distribution output module is used for inputting the surface detection fluorescence distribution into a fluorescence molecular tomography model and reconstructing to obtain fluorophore distribution; the fluorescence molecular tomography model is trained by using a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescence molecular tomography image reconstruction, the residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm.
Optionally, the system further includes: a fluorescence molecular tomography model building module;
the fluorescence molecular tomography model building module comprises:
a test set acquisition unit for acquiring a test set; the test set includes the surface-detected fluorescence profile to be trained and the corresponding true fluorophore profile;
the objective function determining unit is used for determining a regularized optimization objective function based on a diffusion approximation model of a radiation transmission equation;
a gradient calculation unit for calculating a gradient of the regularized optimization objective function;
a calculation map obtaining unit, configured to develop the gradient descent algorithm according to the gradient to obtain a calculation map for each iteration number;
the parameterized computation graph acquisition unit is used for carrying out parameterized processing on the regular term gradient in the computation graph by adopting a residual block structure of a multilayer three-dimensional convolutional neural network to obtain a parameterized computation graph under each iteration number;
the hierarchical network model building unit is used for determining the parameterized computation graph under each iteration number as a layer of network structure and cascading all the network structures to obtain the hierarchical network model;
and the fluorescent molecular tomography model construction unit is used for inputting the test set into the hierarchical network model for training to obtain the fluorescent molecular tomography model.
Optionally, the fluorescence molecular tomography model building unit specifically includes:
a fluorophore distribution acquisition subunit, configured to, under the current training frequency, input the distribution of the surface detection fluorescence to be trained in the test set into the hierarchical network model, so as to obtain the distribution of fluorophores under the current training frequency;
the loss function difference value operator unit is used for calculating the current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution;
the judging subunit is used for judging whether the current loss function difference value is smaller than a preset threshold value;
if so, determining the hierarchical network model under the current training times as the fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
Optionally, the regularized optimization objective function in the objective function determination unit is:
Figure BDA0003091977180000041
wherein Ω (x) is an objective function to be minimized during fluorescent molecular tomography image reconstruction, Φ is surface detection fluorescence distribution, a is a forward matrix obtained by solving the diffusion approximation model based on the radiation transmission equation by finite elements, x is fluorophore distribution, λ is the regularization parameter, and m (x) is a regularization term.
Optionally, the expression of the computation graph in the computation graph obtaining unit is:
Figure BDA0003091977180000051
wherein the content of the first and second substances,
Figure BDA0003091977180000052
the gradient of the k-th iteration of the objective function is optimized for regularization,
Figure BDA0003091977180000053
k is the iteration number of the calculation graph, N is the maximum iteration number of the calculation graph, and xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations ηkFor the iteration step length of the kth iteration, A is a forward matrix obtained by using finite elements to solve the diffusion approximation model based on the radiation transmission equation, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the k-th iteration'k(xk-1) Is the regular term gradient for the kth iteration.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention discloses a model-driven deep learning fluorescence molecule tomography method and a system, wherein the method is used for acquiring the distribution of surface detection fluorescence to be tested; inputting the surface detection fluorescence distribution to be detected into a fluorescence molecular tomography model to obtain the distribution of the fluorophore to be detected; the fluorescence molecular tomography model is trained by using a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescence molecular tomography image reconstruction, a residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm. The invention adopts a fluorescence molecular tomography model to establish the end-to-end mapping relation between the surface detection fluorescence distribution and the fluorophore distribution, avoids the disadvantages of the traditional model-based image reconstruction and improves the quality of the image reconstruction.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flowchart of a model-driven deep learning fluorescence molecular tomography method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a model-driven deep learning fluorescence molecular tomography system according to an embodiment of the present invention;
FIG. 3 is a three-dimensional view of a simulated heterogeneous cylindrical model;
FIG. 4 is a slice view of a simulated heterogeneous cylindrical model;
FIG. 5 is a three-dimensional view of the reconstruction result;
fig. 6 is a slice diagram of the reconstruction result.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a model-driven deep learning fluorescence molecule tomography method, aims to improve the quality of image reconstruction of fluorescence molecule tomography, and can be applied to the technical field of molecular imaging.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a flowchart of a model-driven deep learning fluorescence molecular tomography method according to an embodiment of the present invention. As shown in fig. 1, the present invention provides a model-driven deep learning fluorescence molecular tomography method, which comprises:
step 101: and acquiring the fluorescence distribution of the surface detection.
Step 102: inputting the surface detection fluorescence distribution into a fluorescence molecular tomography model, and reconstructing to obtain fluorophore distribution; the fluorescence molecular tomography model is trained by using a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescence molecular tomography image reconstruction, a residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm.
As an alternative embodiment, the method for determining the fluorescence molecular tomography model comprises:
acquiring a test set; the test set includes the detected fluorescence distribution of the surface to be trained and the corresponding true fluorophore distribution. Specifically, a simulation dataset is obtained in which the test set is a heterogeneous model, the simulation dataset includes a plurality of data pairs, and the inner data pairs include a surface-detected fluorescence distribution to be trained and a corresponding true fluorophore distribution.
A regularized optimization objective function is determined based on a diffusion approximation model of the radiation transport equation.
The gradient of the regularized optimization objective function is calculated.
And developing the gradient descent algorithm according to the gradient to obtain a calculation map under each iteration number.
And carrying out parameterization processing on the regular term gradient in the calculation graph by adopting a residual block structure of the multilayer three-dimensional convolution neural network to obtain a parameterized calculation graph under each iteration number.
And determining the parameterized computation graph under each iteration number as a layer of network structure, and cascading all the network structures to obtain a hierarchical network model. Specifically, a layer of network structure is constructed by the parameterized calculation graph under the successive iteration times, and the corresponding layers of network structures are cascaded according to the sequence of the iteration times from small to large to obtain the hierarchical network model.
And inputting the test set into a hierarchical network model for training to obtain a fluorescent molecular tomography model.
As an optional implementation manner, inputting the test set into a hierarchical network model for training to obtain a fluorescence molecular tomography model, specifically including:
and inputting the distribution of the fluorescence of the surface detection to be trained in the test set into the hierarchical network model under the current training times to obtain the distribution of the fluorophore under the current training times.
Calculating a current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution.
And judging whether the current loss function difference value is smaller than a preset threshold value.
If so, determining the hierarchical network model under the current training times as a fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
As an optional implementation, the regularized optimization objective function is:
Figure BDA0003091977180000071
wherein, omega (x) is an objective function to be minimized when reconstructing a fluorescence molecular tomography image, phi is surface detection fluorescence distribution, A is a forward matrix obtained by solving a diffusion approximation model based on a radiation transmission equation by using a finite element, x is fluorophore distribution, lambda is a regularization parameter, and M (x) is a regularization term,
Figure BDA0003091977180000072
to pair, proceed L2And (5) carrying out norm operation.
As an alternative implementation, the expression of the computation graph is:
Figure BDA0003091977180000073
wherein the content of the first and second substances,
Figure BDA0003091977180000081
the gradient of the k-th iteration of the objective function is optimized for regularization,
Figure BDA0003091977180000082
k is the number of iterations of the computation graph, N is the maximum number of iterations of the computation graph, xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations, etakFor the iteration step length of the kth iteration, A is a forward matrix obtained by using a finite element to solve a diffusion approximation model based on a radiation transmission equation, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the k-th iteration'k(xk-1) Is the regular term gradient for the kth iteration.
Adopting a residual block structure of a multilayer three-dimensional convolution neural network to carry out parameterization processing on the regular term gradient in the calculation graph to obtain a parameterized regular term gradient
Figure BDA0003091977180000083
Replacing the regular term gradient in the calculation graph by using the parameterized regular term gradient to obtain a parameterized calculation graph as follows:
Figure BDA0003091977180000084
wherein R is the maximum layer number of the multilayer three-dimensional convolution neural network,
Figure BDA0003091977180000085
the convolution kernel parameters of the R-th layer three-dimensional convolution neural network in the regular term gradient of the k-th layer network structure,
Figure BDA0003091977180000086
for the fluorophore distribution of the regularization term gradient input to the kth iterationThe output result of the R-1 layer three-dimensional convolution neural network,
Figure BDA0003091977180000087
is the bias parameter of the R-th layer three-dimensional convolution neural network in the regular term gradient of the k-th iteration.
In particular, the method comprises the following steps of,
Figure BDA0003091977180000088
Figure BDA0003091977180000089
wherein r is the number of layers of the multilayer three-dimensional convolution neural network, M'k(xk-1) For the regular term gradient of the k-th iteration,
Figure BDA00030919771800000810
the output of the distribution of fluorophores through the r-layer three-dimensional convolutional neural network for the regularization term gradient input to the kth iteration,
Figure BDA00030919771800000811
for the convolution kernel parameters of the layer r three-dimensional convolution neural network in the regular term gradient of the layer k network structure,
Figure BDA00030919771800000812
the output of the r-1 layer three-dimensional convolutional neural network for the fluorophore distribution of the regularization term gradient input to the kth iteration,
Figure BDA00030919771800000813
for the bias parameters of the r-th layer three-dimensional convolutional neural network in the regularization term gradient of the k-th iteration,
Figure BDA00030919771800000814
the output result of the 0-layer three-dimensional convolution neural network in the fluorophore distribution of the regular term gradient input to the k-th iteration, namely the fluorophore distribution output by the k-1 iterations, namely the fluorescence of the regular term gradient of the k-th iteration without any treatmentThe distribution of the optical mass.
Examples
Optical parameters of different tissues in the heterogeneous cylindrical model constructed in the embodiment are shown in table 1, the positions of real fluorescent rods in an x-y plane are shown by white dots in fig. 4, each fluorescent rod has a diameter of 2mm and a length of 6mm, the axial direction is from-3 mm to 3mm, and the concentrations of the three fluorescent rods are the same and are all set to be 1; the total number of the light sources is 36, the light sources are arranged into 3 layers, the corresponding Z coordinates are 3mm, 0mm and-3 mm, and 12 light source incidence points are uniformly arranged on each layer; 144 detectors are arranged in 4 layers, the corresponding Z coordinates are-4 mm, -1mm, 2mm and 5mm, and 36 detection points are uniformly arranged on each layer; the total detection data (i.e., the distribution of fluorescence detected from the surface) was 36 × 144 to 5184. And (3) carrying out finite element subdivision on the simulation model to finally obtain 13926 nodes, 68689 tetrahedral units, a reconstruction grid is a 1mm cubic voxel grid, and the total number of reconstruction voxels is 4815. The dimension of the forward matrix a obtained when solving the forward problem is therefore 5184 × 4815.
TABLE 1 optical parameters of different tissues in heterogeneous cylindrical model
Figure BDA0003091977180000091
The total detection data (i.e. the surface detection fluorescence distribution) is input into the fluorescence molecular tomography model to obtain the reconstruction result, as shown in fig. 5 and 6, comparing fig. 5 with fig. 3, and comparing fig. 6 with fig. 4, the visible reconstruction result is better in accordance with the real fluorescence distribution, the positioning accuracy is high, the shadow is less, and the image reconstruction quality is higher.
The invention also provides a model-driven deep learning fluorescent molecule tomography system, and fig. 2 is a structural diagram of the model-driven deep learning fluorescent molecule tomography system provided by the embodiment of the invention. As shown in fig. 2, a model-driven deep learning fluorescence molecular tomography system in the present embodiment includes:
a surface detection fluorescence distribution acquisition module 201, configured to acquire a surface detection fluorescence distribution.
The fluorophore distribution output module 202 is configured to input the surface detection fluorescence distribution into a fluorescence molecular tomography model, and reconstruct the fluorescence distribution; the fluorescence molecular tomography model is formed by training a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization objective function of the fluorescent molecular tomography image reconstruction, a residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm.
As an optional implementation, the system further comprises: and a fluorescent molecular tomography model building module.
The fluorescent molecular tomography model building module comprises:
a test set acquisition unit for acquiring a test set; the test set includes the detected fluorescence distribution of the surface to be trained and the corresponding true fluorophore distribution.
And the objective function determination unit is used for determining a regularized optimization objective function based on the diffusion approximation model of the radiation transfer equation.
And the gradient calculation unit is used for calculating the gradient of the regularized optimization objective function.
And the calculation map acquisition unit is used for developing the gradient descent algorithm according to the gradient to obtain a calculation map under each iteration number.
And the parameterized calculation map acquisition unit is used for carrying out parameterized processing on the regular term gradient in the calculation map by adopting a residual block structure of the multilayer three-dimensional convolutional neural network to obtain the parameterized calculation map under each iteration number.
And the hierarchical network model building unit is used for determining the parameterized calculation graph under each iteration number as a hierarchical network structure and cascading all the network structures to obtain the hierarchical network model. Specifically, a layer of network structure is constructed by the parameterized calculation graph under the successive iteration times, and the corresponding layers of network structures are cascaded according to the sequence of the iteration times from small to large to obtain the hierarchical network model.
And the fluorescent molecular tomography model construction unit is used for inputting the test set into the hierarchical network model for training to obtain the fluorescent molecular tomography model.
As an optional implementation manner, the fluorescence molecular tomography model building unit specifically includes:
and the fluorophore distribution acquisition subunit is used for inputting the distribution of the surface detection fluorescence to be trained in the test set into the hierarchical network model under the current training times to obtain the distribution of the fluorophores under the current training times.
The loss function difference value operator unit is used for calculating the current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution.
And the judging subunit is used for judging whether the current loss function difference value is smaller than a preset threshold value.
If so, determining the hierarchical network model under the current training times as a fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
As an optional implementation, the regularized optimization objective function in the objective function determination unit is:
Figure BDA0003091977180000111
wherein, omega (x) is an objective function which needs to be minimized when the fluorescence molecular tomography image is reconstructed, phi is the surface detection fluorescence distribution, A is a forward matrix obtained by solving a diffusion approximate model based on a radiation transmission equation by using finite elements, x is the fluorophore distribution, lambda is a regularization parameter, M (x) is a regularization item,
Figure BDA0003091977180000112
to pair, proceed L2And (5) carrying out norm operation.
As an optional implementation, the expression of the computation graph in the computation graph obtaining unit is:
Figure BDA0003091977180000113
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003091977180000114
the gradient of the k-th iteration of the objective function is optimized for regularization,
Figure BDA0003091977180000115
k is the number of iterations of the computation graph, N is the maximum number of iterations of the computation graph, xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations, etakFor the iteration step length of the kth iteration, A is a forward matrix obtained by solving a diffusion approximation model based on a radiation transmission equation by using finite elements, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the k-th iteration'k(xk-1) Is the regular term gradient for the kth iteration.
The technical effects of the invention comprise:
(1) compared with the traditional deep learning method based on model driving, the method and the system improve the quality of image reconstruction.
(2) Compared with the existing data-driven deep learning method, the method and the system of the invention improve the network interpretability. The existing data-driven deep learning method brings a new research paradigm for FMT research, and is characterized in that nonlinear mapping from detection data to a target can be directly established by avoiding modeling errors of a forward problem and ill-posed nature of a reverse problem, and a plurality of research groups focus on deep learning-based optical molecular imaging method research. A convolution neural network of an end-to-end three-dimensional codec structure for FMT imaging improves the three-dimensional reconstruction positioning precision and reduces the processing time of a reconstruction algorithm; the multilayer perceptron model is applied to biological self-luminescence Tomography (BLT), traditional forward modeling is abandoned, the mapping relation between detection data and a reconstructed target is directly established, and higher positioning accuracy is achieved; a multilayer convolutional neural network is provided to be applied to Diffusion Optical Tomography (DOT), and a result which is more accurate and faster than that obtained by a traditional method is obtained. The deep learning method based on data driving obtains the mapping relation from the projection image to the imaging target in a learning mode from the data set, and avoids the disadvantage of traditional image reconstruction based on a model. However, the deep learning method based on data driving lacks theoretical understanding of the relationship between the network topology and the performance, no corresponding physical model corresponds to the network topology, the design of the network architecture tends to be empiric, the interpretability is poor, meanwhile, a large amount of training data is needed to support network training, and the network generalization capability is difficult to guarantee, so that the deep learning method based on data driving does not have wide applicability in FMT practical application.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are presented solely to aid in the understanding of the apparatus and its core concepts; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the foregoing, the description is not to be taken in a limiting sense.

Claims (6)

1. A model-driven deep-learning fluorescence molecular tomography method, the method comprising:
acquiring surface detection fluorescence distribution;
inputting the surface detection fluorescence distribution into a fluorescence molecular tomography model, and reconstructing to obtain fluorophore distribution; the fluorescence molecular tomography model is trained by using a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization target function of the fluorescence molecular tomography image reconstruction, the residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm;
the method for determining the fluorescent molecular tomography model comprises the following steps:
acquiring a test set; the test set comprises a surface-detected fluorescence profile to be trained and a corresponding true fluorophore profile;
determining a regularized optimization objective function based on a diffusion approximation model of a radiation transfer equation;
calculating a gradient of the regularized optimization objective function;
developing the gradient descent algorithm according to the gradient to obtain a calculation graph under each iteration number;
carrying out parameterization processing on the regular term gradient in the calculation graph by adopting a residual block structure of a multilayer three-dimensional convolution neural network to obtain a parameterized calculation graph under each iteration number;
determining the parameterized computation graph under each iteration number as a layer of network structure, and cascading all the network structures to obtain the hierarchical network model;
inputting the test set into the hierarchical network model for training to obtain the fluorescent molecular tomography model;
inputting the test set into the hierarchical network model for training to obtain the fluorescence molecular tomography model, which specifically comprises:
inputting the distribution of the surface detection fluorescence to be trained into the hierarchical network model under the current training times to obtain the distribution of the fluorophore under the current training times;
calculating a current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution;
judging whether the current loss function difference value is smaller than a preset threshold value or not;
if so, determining the hierarchical network model under the current training times as the fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
2. The model-driven deep learning fluorescence molecular tomography method of claim 1, wherein the regularized optimization objective function is:
Figure FDA0003631073920000021
wherein Ω (x) is an objective function to be minimized during fluorescent molecular tomography image reconstruction, Φ is surface detection fluorescence distribution, a is a forward matrix obtained by solving the diffusion approximation model based on the radiation transmission equation by finite elements, x is fluorophore distribution, λ is the regularization parameter, and m (x) is a regularization term.
3. The model-driven deep-learning fluorescence molecular tomography method according to claim 1, wherein the expression of the computation graph is:
xk＝ReLU[xk-1-ηk▽Ω(xk-1))]，k＝1,2,…，N,N＝15，
wherein ∑ Ω (x)k-1) Optimizing a gradient of objective function kth iteration for regularization, [ omega ] (x)k-1)＝ATAxk-1-ATΦ+λkM'k(xk-1) K is the number of iterations of the computation graph, N is the maximum number of iterations of the computation graph, xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations ηkFor the iteration step length of the kth iteration, A is a forward matrix obtained by using finite elements to solve the diffusion approximation model based on the radiation transmission equation, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the kth iteration'k(xk-1) Is the regular term gradient for the kth iteration.
4. A model-driven deep-learning fluorescence molecular tomography system, the system comprising:
the surface detection fluorescence distribution acquisition module is used for acquiring surface detection fluorescence distribution;
the fluorophore distribution output module is used for inputting the surface detection fluorescence distribution into a fluorescence molecular tomography model and reconstructing to obtain fluorophore distribution; the fluorescence molecular tomography model is formed by training a test set pair hierarchical network model; the hierarchical network model is constructed based on the gradient of a regularization optimization target function of the fluorescence molecular tomography image reconstruction, the residual block structure of a multilayer three-dimensional convolution neural network and a gradient descent algorithm;
a fluorescence molecular tomography model building module;
the fluorescence molecular tomography model building module comprises:
a test set acquisition unit for acquiring a test set; the test set includes the surface-detected fluorescence profile to be trained and the corresponding true fluorophore profile;
the target function determining unit is used for determining a regularized optimization target function based on a diffusion approximation model of a radiation transfer equation;
a gradient calculation unit for calculating a gradient of the regularized optimization objective function;
the calculation map acquisition unit is used for developing the gradient descent algorithm according to the gradient to obtain a calculation map under each iteration number;
the parameterized computation graph acquisition unit is used for carrying out parameterized processing on the regular term gradient in the computation graph by adopting a residual block structure of a multilayer three-dimensional convolutional neural network to obtain a parameterized computation graph under each iteration number;
the hierarchical network model building unit is used for determining the parameterized calculation graph under each iteration number as a layer of network structure and cascading all the network structures to obtain the hierarchical network model;
the fluorescent molecular tomography model building unit is used for inputting the test set into the hierarchical network model for training to obtain the fluorescent molecular tomography model;
the fluorescent molecular tomography model construction unit specifically comprises:
a fluorophore distribution acquisition subunit, configured to, under the current training frequency, input the distribution of the surface detection fluorescence to be trained in the test set into the hierarchical network model, so as to obtain the distribution of fluorophores under the current training frequency;
the loss function difference value operator unit is used for calculating the current loss function difference value; the current loss function difference value is the absolute value of the difference between the value of the loss function under the current training times and the value of the loss function under the last training times; the loss function is the mean square error between the fluorophore distribution output by the hierarchical network model and the corresponding true fluorophore distribution;
the judging subunit is used for judging whether the current loss function difference value is smaller than a preset threshold value;
if so, determining the hierarchical network model under the current training times as the fluorescence molecular tomography model; if not, adjusting the learnable parameters in the hierarchical network model under the current training times according to the loss function under the current training times, and carrying out next training; the learnable parameters include: the step length of each layer of network structure, the regularization parameter of each layer of network structure, the convolution kernel parameter of the three-dimensional convolution neural network in each layer of network structure and the bias parameter of the three-dimensional convolution neural network in each layer of network structure.
5. The model-driven deep learning fluorescence molecular tomography system of claim 4, wherein the regularized optimization objective function in the objective function determination unit is:
Figure FDA0003631073920000041
wherein Ω (x) is an objective function to be minimized during fluorescent molecular tomography image reconstruction, Φ is surface detection fluorescence distribution, a is a forward matrix obtained by solving the diffusion approximation model based on the radiation transmission equation by finite elements, x is fluorophore distribution, λ is the regularization parameter, and m (x) is a regularization term.
6. The model-driven deep-learning fluorescence molecule tomography system according to claim 4, wherein the expression of the computation graph in the computation graph acquiring unit is:
xk＝ReLU[xk-1-ηk▽Ω(xk-1))]，k＝1,2,…，N,N＝15，
▽Ω(xk-1)＝ATAxk-1-ATΦ+λkM'k(xk-1)，
wherein ∑ Ω (x)k-1) Optimizing a gradient of objective function kth iteration for regularization, [ omega ] (x)k-1)＝ATAxk-1-ATΦ+λkM'k(xk-1) K is the number of iterations of the computation graph, N is the maximum number of iterations of the computation graph, xkFor the distribution of the fluorophore output over the kth iteration, ReLU [ ·]Is a linear rectification function, xk-1Distribution of fluorophore over k-1 iterations ηkFor the iteration step length of the kth iteration, A is a forward matrix obtained by using finite elements to solve the diffusion approximation model based on the radiation transmission equation, phi is the surface detection fluorescence distribution, and lambda iskRegularization parameter, M 'for the k-th iteration'k(xk-1) Is the regular term gradient for the kth iteration.
CN202110598228.5A 2021-05-31 2021-05-31 Model-driven deep learning fluorescence molecule tomography method and system Active CN113327305B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110598228.5A CN113327305B (en) 2021-05-31 2021-05-31 Model-driven deep learning fluorescence molecule tomography method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110598228.5A CN113327305B (en) 2021-05-31 2021-05-31 Model-driven deep learning fluorescence molecule tomography method and system

Publications (2)

Publication Number Publication Date
CN113327305A CN113327305A (en) 2021-08-31
CN113327305B true CN113327305B (en) 2022-06-24

Family

ID=77422672

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110598228.5A Active CN113327305B (en) 2021-05-31 2021-05-31 Model-driven deep learning fluorescence molecule tomography method and system

Country Status (1)

Country Link
CN (1) CN113327305B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830565A (en) * 2024-03-06 2024-04-05 华中科技大学 Multi-operator self-supervision fluorescent molecular tomography method and system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101396262B (en) * 2008-10-31 2010-06-23 清华大学 Fluorescent molecule tomography rebuilding method based on linear relationship
WO2013159250A1 (en) * 2012-04-28 2013-10-31 清华大学 4d fluorescence molecular tomography image reconstruction method
CN102871646B (en) * 2012-08-16 2014-09-10 清华大学 Large-data-size fluorescent molecule tomography reconstruction method
CN109191564B (en) * 2018-07-27 2020-09-04 中国科学院自动化研究所 Depth learning-based three-dimensional reconstruction method for fluorescence tomography
US11373390B2 (en) * 2019-06-21 2022-06-28 Adobe Inc. Generating scene graphs from digital images using external knowledge and image reconstruction
CN112734872B (en) * 2020-12-30 2022-10-04 天津工业大学 Fluorescence molecule tomography method and system based on multi-wavelength concurrent reconstruction

Also Published As

Publication number Publication date
CN113327305A (en) 2021-08-31

Similar Documents

Publication Publication Date Title
JP6853273B2 (en) Systems and methods incorporating neural networks and forward physical models for semiconductor applications
CN109191564B (en) Depth learning-based three-dimensional reconstruction method for fluorescence tomography
Dong et al. A deep learning reconstruction framework for X-ray computed tomography with incomplete data
Christen et al. Markov chain Monte Carlo using an approximation
Mary et al. Kappa (κ): analysis of curvature in biological image data using B-splines
Ming et al. Rapid reconstruction of 3D neuronal morphology from light microscopy images with augmented rayburst sampling
CN111292386B (en) CT projection metal trace complement metal artifact correction method based on U-net
Varga et al. Direction-dependency of binary tomographic reconstruction algorithms
CN113327305B (en) Model-driven deep learning fluorescence molecule tomography method and system
CN106097441A (en) Compound regularization Bioluminescence tomography reconstruction method based on L1 norm Yu TV norm
CN115868923A (en) Fluorescence molecule tomography method and system based on expanded cyclic neural network
CN108095686B (en) Fluorescence molecular tomography target feasible region selection method
CN111488498A (en) Node-graph cross-layer graph matching method and system based on graph neural network
Kandel et al. Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm
Johnston et al. Organised randoms: Learning and correcting for systematic galaxy clustering patterns in KiDS using self-organising maps
Bardsley et al. Tikhonov regularized Poisson likelihood estimation: theoretical justification and a computational method
CN110785709B (en) Generating high resolution images from low resolution images for semiconductor applications
Mustafa et al. Sparse-view spectral CT reconstruction using deep learning
CN116503506B (en) Image reconstruction method, system, device and storage medium
CN117197349A (en) CT image reconstruction method and device
CN115311380A (en) Reflective fluorescent molecular tomography method and system based on ADMM network
CN112037300B (en) Optical reconstruction method and device based on alternate direction multiplier subnetwork
CN115294300A (en) Multi-branch attention prior parameterized finite projection fast fluorescence tomography reconstruction method
CN117830565A (en) Multi-operator self-supervision fluorescent molecular tomography method and system
Arridge et al. Deep learning and inverse problems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant