CN112734814B - Three-dimensional craniofacial cone beam CT image registration method - Google Patents

Three-dimensional craniofacial cone beam CT image registration method Download PDF

Info

Publication number
CN112734814B
CN112734814B CN201911029712.5A CN201911029712A CN112734814B CN 112734814 B CN112734814 B CN 112734814B CN 201911029712 A CN201911029712 A CN 201911029712A CN 112734814 B CN112734814 B CN 112734814B
Authority
CN
China
Prior art keywords
image
registration
phi
deformation field
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911029712.5A
Other languages
Chinese (zh)
Other versions
CN112734814A (en
Inventor
裴玉茹
张云庚
郭玉珂
查红彬
许天民
马赓宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Priority to CN201911029712.5A priority Critical patent/CN112734814B/en
Publication of CN112734814A publication Critical patent/CN112734814A/en
Application granted granted Critical
Publication of CN112734814B publication Critical patent/CN112734814B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The invention discloses a three-dimensional craniofacial cone beam CT image registration method, which establishes a registration full convolution neural network based on unsupervised learning, effectively realizes the image registration of three-dimensional craniofacial CBCT by correspondingly solving dense consistent voxel levels among paired CBCT images, meets the requirement of reversible consistency, realizes the efficient online registration of three-dimensional craniofacial Cone Beam CT (CBCT) images, and greatly reduces the registration time.

Description

Three-dimensional craniofacial cone beam CT image registration method
Technical Field
The invention relates to the technical fields of oral clinical medicine and computer vision, in particular to a three-dimensional craniofacial cone beam computed tomography (Cone Beam Computed Tomography, CBCT) image registration method.
Background
The solution of the dense voxels of the three-dimensional image is a key technology in medical image analysis and is widely applied to the tasks of statistical shape analysis, feature point and segmentation migration, preoperative and postoperative efficacy evaluation and the like.
In the prior art, the traditional three-dimensional image non-rigid registration method reduces the apparent difference of the dense voxel level of the three-dimensional image through online nonlinear iterative optimization, and commonly used measures comprise mean square error, normalized cross correlation, mutual information and the like. Deformation models in non-rigid registration include elastomer models, demons, differential stratospheric maps, B-spline free deformation models, and the like. Considering large-scale parameters to be solved in three-dimensional image registration, the optimization solution of the traditional registration method usually has great calculation time cost and is easy to trap into local minima. Some methods use downsampling and statistical deformation models to reduce parameters of the problem solving, but still rely on iterative parameter optimization solutions. The non-rigid registration method based on numerical solution requires manual key point and segmentation map labeling. The one-to-one mapping that keeps topology, deformation fields free of folds, reversible smoothing is an important property that anatomical image registration should have, for which registration methods using differential-homoembryo deformation models including LDDMM, syN, differential-homoembryo demons, etc. are used in anatomical structure registration.
In recent years, regression models are introduced into the non-rigid registration of three-dimensional images, and the learning-based three-dimensional image non-rigid registration method avoids online iterative optimization, such as support vector machines, random forests and deep neural networks, by finding the regression models of the input images and the registration deformation fields, and compared with the traditional optimization method, the time consumption of the registration is reduced by orders of magnitude. The supervised deep learning-based three-dimensional image registration method requires truth values of key points and deformation fields in the training process, and the manual labeling difficulty of the three-dimensional image can limit the acquisition of training data. Unsupervised deep learning-based three-dimensional image registration methods use spatial transformation networks to reduce apparent differences at the voxel level. The existing three-dimensional image registration method based on deep learning reduces the required memory occupation through three-dimensional image block registration, or only solves the unidirectional mapping from one image to a reference image. In order to solve the reversible one-to-one mapping, the existing three-dimensional image registration method based on deep learning using differential homoembryo mapping requires a re-parameterization layer to convert the velocity field predicted by the neural network into a bidirectional deformation field. Some methods utilize reversible constraints to directly infer the bi-directional deformation field, avoiding the estimation of intermediate layers, but relying on manual extraction of features. Thus, no effective solution currently exists to directly predict reversibly uniform deformation fields using neural networks.
Disclosure of Invention
The invention provides a three-dimensional craniofacial Cone Beam CT (CBCT) image registration method, which aims to overcome the defects, perform high-efficiency online registration on three-dimensional images and meet the requirement of reversible consistency.
In order to achieve the purpose, the registration framework based on unsupervised learning is established, and the full convolution neural network is utilized, so that the voxel level correspondence solution with dense consistency between paired CBCT images is realized, and compared with the traditional iterative optimization registration method, the registration time is greatly reduced. The invention uses the convolution neural network to regress the two-way mapping of voxel level between the input image V and the template image T, guarantees the forward deformation field and the reverse deformation field to meet the requirement of reversible consistency by introducing combined reversible constraint, and strengthens the closed loop consistency of the deformation field by using any binary group and ternary group in the data set through the constraint of the deformation field in the network training process. In registration network training, the system allows a user to specify a structure of interest (SOI), enhancing the registration effect of anatomical structures with higher registration difficulty, such as the skull base with lower bone density and blurred boundary with surrounding soft tissue. For online registration of any pair of given images, the trained convolutional neural network returns forward and directional deformation fields of each image and the template image T respectively, and a pair of forward and reverse deformation fields which are reversibly consistent with each other are obtained through combination of the deformation fields.
In the present invention, a computed tomography is: CT (Computed Tomography); cone beam computed tomography: cone Beam Computed Tomography (CBCT).
The technical scheme provided by the invention is as follows:
a three-dimensional craniofacial Cone Beam CT (CBCT) image registration method establishes a registration full convolution neural network based on unsupervised learning, and the registration time can be greatly reduced by solving the correspondence of voxel levels with consistency between pairs of CBCT images, thereby effectively realizing the image registration of the three-dimensional craniofacial Cone Beam CT (CBCT); the method comprises the following steps:
1) Defining a template image, and obtaining a template image T after the template image is deformed by an average deformation field;
a set of three-dimensional image data sets v= { V i I=1,.. N registration to a randomly selected reference image V r Obtaining a deformation field phi i N is the number of images of the dataset. The template image T is an image obtained by deforming the average deformation field.
In the formula 1 of the present invention,representing the utilization of deformation fields->For image V r Deformation is performed.
2) The construction and pre-training of the registration network comprises the following steps:
21 Solving a smooth one-to-one mapping of voxel levels between a three-dimensional image V and a template image T by using a bidirectional registration network based on a fixed template image, wherein V and T are three-dimensional single-channel gray images;
22 Using convolutional neural networks to regress the input image and the output bi-directional deformation field:
h Θ，T ：V→[φ f ，φ b ]
wherein phi is f For deforming the input image V into the deformation field of the template image T b For the deformation field of the template image T into the input image V, Θ is a learnable parameter of the registration network h (e.g. a convolution kernel weight of the convolutional neural network). Setting image pairs And +.>With similar anatomical appearance (the degree of similarity can be measured by equation 4).
23 Convolutional neural network using symmetrical codec structure with long residual connection, estimating six-channel forward and reverse deformation fields [ phi ] from the input single-channel three-dimensional image V f ，φ b ]. The coder-decoder is composed of coder and decoder, the characteristic image of decoding stage is composed of the characteristic image output by previous decoding layer and the characteristic imageThe co-dimensional feature images of the encoding stage are composed to facilitate feature propagation and training convergence.
The invention uses a symmetrical codec structure with long residual connection to infer six-channel forward and reverse deformation fields [ phi ] from an input single-channel three-dimensional image V f ，φ b ]. The encoder comprises six convolutional layers of 3 x 3 step size 1, each convolution layer is followed by an instance regularization (instance normalization), a leak ReLU activation function, and a 2 x 2 max pooling layer. The decoder contains six convolutional layers of convolution kernel size 3 x 3 steps of 1. Each convolution layer is preceded by a x 2 upsampling layer and followed by an instance regularization and ReLU activation function layer.
24 Pre-training the convolutional neural network by using the synthetic data, generating a deformation field by using the randomly sampled B-spline deformation parameters, and ensuring that the generated deformation field is not folded; deforming the template image T by using the generated deformation field to generate a composite image V syn And the reverse deformation field is calculated. The neural network is pre-trained using the composite image and corresponding forward and reverse deformation fields.
3) Training the registration network constructed in the step 2) in an unsupervised mode to obtain a trained registration network; the method comprises the following steps:
for having distributionV= { V i I=1,..n, N is the number of images of the training dataset, the loss function is minimized by finding the optimal network optimizable parameter Θ.
Wherein h is a registration network; Θ is a learnable parameter of the registration network h, V is a training image, T is a template image,is the distribution of the training data set.
The loss function of registration network training is:
wherein L is sim Is a similarity measure loss, L reg Is a deformation field regularization term, L inv Is a deformation field combined reversible constraint term, L str Is the registration loss of the structure of interest (SO 1), Θ is the convolutional neural network optimizable parameter, α sim 、α reg 、α inv 、α str The weights of the loss terms are respectively.
Each loss function is described in turn as follows:
31 Registration network predicts forward and reverse deformation fields [ phi ] f ，φ b ]Similarity measure L sim Metrology using forward deformation field phi f，i Deformed input imageDifference from template image T at voxel level and using inverse deformation field phi b，i Deformed template image +.>And input image V i M is the number of images of one batch of data. Similarity metric loss L sim Expressed as equation 4:
32 Deformation field regularization term L reg To ensure the smoothness of the forward and reverse deformation fields, the spatial gradient of the deformation field is used as a regularization term, expressed as equation 5.
Wherein phi is f，i ，φ b，i For both the forward and reverse deformation fields, I.I F Is the Frobenius norm and m is the number of images of a batch of data.
33 Deformation field combination reversible constraint term L inv ：
The invention uses the combined reversible constraint to lead the forward and reverse deformation fields to meet the conditions of forward and reverse consistency and closed loop consistency, and the deformation field is combined with the reversible constraint item L inv Expressed as equation 6:
L inv the first term of the two-way conversion [ phi ] f ，φ b ]Is reciprocal, m is the number of images in a batch in training. I.I F Is the Frobenius norm. L (L) inv Requires k images { V }, second term of i I=0,..Is an identity transformation. For a training set with N three-dimensional images, the number of closed loops with the length of more than or equal to 2 is +.>To simplify training, only the closed loop formed by two or three volumes of data may be considered, K being set to 3.L (L) inv The second term of (2) requires that the forward and reverse deformation fields of any two images are reciprocal and the deformation fields of any three images satisfy the closed loop agreement.
In equation 6, phi f，i ，φ b，i For both the forward and reverse deformation fields, I.I F Is the Frobenius norm, m is the number of images of a batch of data,a combination of k deformation fields along the closed loop c. For the combination of deformation fields, the standard coordinate field Q is changed into Q' after being deformed by a plurality of deformation fields' Q is the combined result of the deformation fields.
34 Loss of registration L for a structure of interest (SOI) str ：
In registration of three-dimensional craniofacial images, some anatomical structures such as the skull base are relatively blurred from the surrounding soft tissue boundaries due to lower bone density and device-dependent noise, and the registration difficulty is high, the present invention uses a structure of interest (SOI) registration loss function L in network training str To improve registration of the fine structure, expressed as equation 7.
Where M is a mask for a region of interest (SOI), where a voxel value of 1 represents the region of interest and the remaining voxel values are 0.m is the number of images of a batch of data, V i Is an input image of a network, T is a template image, phi f，i Is the forward deformation field of network output, phi b，i Is the reverse deformation field of the network output. The operator ∈ represents element-by-element matrix multiplication.
4) On-line registration of three-dimensional images to be registered (paired CBCT images):
in-line registration stage, for any pair of floating images V M And a fixed image V F Is denoted as (V) M ，V F ) Returning V respectively through the trained registration network in the step 3) M And V F Forward and reverse deformation fields of (a)And obtaining a final registration deformation field phi through deformation field combination M→F And phi F→M ：
Wherein phi is used M→F Representing that the floating image can be deformed to a fixed image, using phi F→M The fixed image may be deformed to a floating image,should be in accordance with V M Has similar apparent characteristics->Should be in accordance with V F Has similar apparent characteristics. And obtaining a deformation field, namely registering the images. After the floating image is deformed by using the deformation field generated by registration, the more similar the obtained image is to the appearance of the fixed image, the better the registration effect is.
Through the steps, registration of three-dimensional craniofacial Cone Beam CT (CBCT) images is realized.
The invention has the beneficial effects that:
by using the three-dimensional craniofacial Cone Beam CT (CBCT) image registration method provided by the invention, the dense correspondence of a pair of three-dimensional image voxel levels can be solved, the requirement of reversible consistency is met, and the efficient online registration of the three-dimensional craniofacial Cone Beam CT (CBCT) image is realized.
Drawings
FIG. 1 is a diagram of a registration network training framework corresponding to the method of the present invention;
fig. 2 is a block diagram of an online registration flow of the present invention.
FIG. 3 is a schematic representation of a registration deformation field and registration results for a pair of images in an embodiment of the present invention;
wherein (a) is the input image pair (V) M ，V F ) The method comprises the steps of carrying out a first treatment on the surface of the (b) And (c) are each V M And V F Forward and reverse deformation fields of (a); (d) Is the forward deformation field phi M→F And a reverse deformation field phi F→M (e) A deformed image;
FIG. 4 is an illustration of the overlap of the results of a pair of three-dimensional images prior to registration and after forward registration and after reverse registration in the sagittal, coronal, and axial planes in an embodiment of the present invention;
wherein, the liquid crystal display device comprises a liquid crystal display device,(a) Is a sagittal plane; (b) is a coronal plane; (c) the axial surfaces overlap. (d) Deformation field phi f And phi b Is a slice of the same.
Detailed Description
The invention is further described by way of examples in the following with reference to the accompanying drawings, but in no way limit the scope of the invention.
The three-dimensional craniofacial Cone Beam CT (CBCT) image registration method provided by the invention uses a convolutional neural network to regress an input image and a bidirectional deformation field, reduces the apparent difference of voxel levels of a target image and a deformed image by using a space transformation network, ensures the reversible consistency of a forward deformation field and a reverse deformation field by combining reversible constraint, avoids iterative optimization of the traditional registration method during online registration, and can greatly reduce registration time.
The following is a further description of embodiments of the invention with reference to the drawings and examples.
Step 1: defining template images
A group of acquired oral craniofacial three-dimensional cone beam CT image data sets v= { V i I=1,..n } is registered to a randomly selected reference image V r Obtaining a deformation field phi i N is the number of images of the dataset. The template image T is an image obtained by deforming the average deformation field.
Step 2: registration network construction and pre-training
The invention solves the smooth one-to-one mapping of voxel level between a three-dimensional image V and a template image T by using a bidirectional registration network based on a fixed template image, V and T are three-dimensional single-channel gray level images, and a convolution neural network is used for regressing an input image and an output bidirectional deformation field h Θ，T ：V→[φ f ，φ b ]，φ f : v- & gt T is the deformation field from the input image V to the template image T, phi b : T-V is the deformation field of the template image T deformed to the input image V, and Θ is the learnable parameter of the registration network hNumbers, such as convolution kernel weights of a convolutional neural network. Image pairAnd +.>Is expected to have a similar anatomical appearance. The optimal parameter theta is obtained in training by measuring the registration similarity in the forward direction and the reverse direction.
The invention uses a symmetrical codec structure with long residual connection to infer six-channel forward and reverse deformation fields [ phi ] from an input single-channel three-dimensional image V f ，φ b ]. The encoder comprises six convolutional layers of 3 x 3 step size 1, each convolution layer is followed by an instance regularization (instance normalization), a leak ReLU activation function, and a 2 x 2 max pooling layer. The decoder contains six convolutional layers of convolution kernel size 3 x 3 steps of 1. Each convolution layer is preceded by a x 2 upsampling layer and followed by an instance regularization and ReLU activation function layer. The feature image of the decoding stage is composed of the feature image output by the previous decoding layer and the feature image with the same size of the encoding stage, so as to promote the propagation of the features and the convergence of training.
The method uses the synthetic data to pretrain the convolutional neural network, uses the randomly sampled B-spline deformation parameters to generate a deformation field, ensures that the generated deformation field is not folded, utilizes the generated deformation field to deform the template image T to generate a synthetic image V, and calculates a reverse deformation field. The neural network is pre-trained using the composite image and corresponding forward and reverse deformation fields.
Step 3: unsupervised mode training registration network
The present invention uses combined reversible constraints to ensure reversible and closed-loop consistency of the deformation field. For having distribution D v V= { V i I=1,..n, N is the number of images of the training dataset, the loss function is minimized by finding the optimal network optimizable parameter Θ.
Where Θ is the learnable parameter of the registration network h, V is the training image, T is the template image,is the distribution of the training data set.
The loss function of registration network training is:
wherein L is sim Is a similarity measure loss, L reg Is a deformation field regularization term, L inv Is a deformation field combined reversible constraint term, L str Is the registration loss of the structure of interest (SOI), Θ is the convolutional neural network optimizable parameter, α sim 、α reg 、α inv 、α str The weights of the loss terms are respectively.
Each loss function is described in turn as follows:
1) Registration network predicts forward and reverse deformation fields [ phi ] f ，φ b ]Similarity measure L sim Metrology using forward deformation field phi f，i Deformed input imageDifference from template image T at voxel level and using inverse deformation field phi b，i Deformed template image +.>And input image V i M is the number of images of one batch of data.
2) Regularization ofItem L reg The method is used for guaranteeing the smoothness of the forward deformation field and the backward deformation field, and the spatial gradient of the deformation field is used as a regularization term.
Wherein phi is f，i ，φ b，i For both the forward and reverse deformation fields, I.I F Is the Frobenius norm and m is the number of images of a batch of data.
3) The invention uses the combined reversible constraint to ensure that the forward deformation field and the reverse deformation field meet the conditions of forward and reverse consistency and closed loop consistency, L inv The first term of the two-way conversion [ phi ] f ，φ b ]Is reciprocal, m is the number of images in a batch in training. I.I F Is the Frobenius norm. L (L) inv Requires k images { V }, second term of i I=0,..Is an identity transformation. For a training set with N three-dimensional images, the number of closed loops with the length of more than or equal to 2 is +.>To simplify training, only the closed loop formed by two or three volumes of data may be considered, K being set to 3.L (L) inv The second term of (2) requires that the forward and reverse deformation fields of any two images are reciprocal and the deformation fields of any three images satisfy the closed loop agreement.
Wherein phi is f，i ，φ b，i For both the forward and reverse deformation fields, I.I F Is the Frobenius norm, m is the number of images of a batch of data,a combination of k deformation fields along the closed loop c. For the combination of deformation fields, the standard coordinate field Q is utilized to change into Q ', Q' -Q after being deformed by a plurality of deformation fields, and the combination result of the deformation fields is obtained.
4) In registration of three-dimensional craniofacial images, some anatomical structures such as the skull base are relatively blurred from the surrounding soft tissue boundaries due to lower bone density and device-dependent noise, and the registration difficulty is high, the present invention uses a structure of interest (SOI) registration loss function L in network training str To improve registration of the fine structures.
Where M is a mask for a region of interest (SOI), where a voxel value of 1 represents the region of interest and the remaining voxel values are 0.m is the number of images of a batch of data, V i Is an input image of a network, T is a template image, phi f，i Is the forward deformation field of network output, phi b，i Is the reverse deformation field of the network output. The operator ∈ represents element-by-element matrix multiplication.
Step 4: online registration of paired CBCT images
In the online registration stage, given any pair of floating and fixed images (V M ，V F ) The registration network returns to V respectively M And V F Forward and reverse deformation fields of (a)And obtaining a final deformation field through deformation field combination
In order to verify the registration accuracy of the registration network on paired CBCT images, for composite data, the mean square distance between the forward deformation field and the reverse deformation field predicted by the network and the true value of the deformation field is calculated, the average error is smaller than 0.25mm, meanwhile, the deformation field has better reversibility, and the average value of the number of voxels with the jacobian less than or equal to 0 is 7.44, which indicates that the deformation field predicted by the registration method has better reversibility.
With the method of the present invention, pairs of images (e.g., cone beam CT images) can be rapidly and efficiently registered non-rigidly. The system utilizes convolutional neural network regression to input images and bidirectional deformation fields, avoids iterative solution of the traditional registration method, and greatly reduces registration time. The system utilizes the combined reversible constraint to further require that for any binary group and ternary group in the training data set, the deformation field meets the requirement of closed loop consistency, so that the forward deformation field and the reverse deformation field predicted by the registration network have better reversibility. The system uses region of interest (SOI) registration loss to enhance registration of fine anatomical structures. The system effectively solves the problems that the traditional non-rigid registration method is time-consuming to calculate and easy to sink into local minimum, and meets the requirement of reversible consistency of deformation fields.
The above embodiments are only for illustrating the present invention and not for limiting the same, and various changes and modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention, and therefore all equivalent technical solutions are also within the scope of the present invention, and the scope of the present invention is defined by the claims.

Claims (10)

1. A three-dimensional craniofacial cone beam CT image registration method, wherein the cone beam CT is called CBCT for short; the registration method establishes a registration full convolution neural network based on unsupervised learning, and effectively realizes image registration of three-dimensional craniofacial CBCT by correspondingly solving voxel levels with consistent density among paired CBCT images; the method comprises the following steps:
1) Defining a template image, and obtaining a template image T after the template image is deformed by an average deformation field;
a set of three-dimensional image data sets v= { V i I=1,.. N registration to randomly selected reference image V r Obtaining a deformation field phi i The method comprises the steps of carrying out a first treatment on the surface of the Where N is the number of images of the dataset;
the template image T is an image obtained by deforming the average deformation field, and is expressed by formula 1:
in the formula 1 of the present invention,representing the utilization of deformation fields->For image V r Deforming;
2) The registration network is built and pre-training is carried out, and the method comprises the following steps:
21 Using a bi-directional registration network based on the fixed template image to solve a smooth one-to-one mapping of voxel levels between the three-dimensional image and the template image; the three-dimensional image and the template image are three-dimensional single-channel gray scale images;
22 Using convolutional neural networks to regress the input image and the output bi-directional deformation field:
h Θ，T ：V→[φ f ，φ b ]
wherein V is an input image; phi (phi) f A deformation field for deforming the input image V into the template image T; phi (phi) b A deformation field for deforming the template image T into the input image V; Θ is a learnable parameter of the registration network h;
23 Using a symmetrical codec structure with long residual connection, six-channel forward and reverse deformation fields [ phi ] are extrapolated from the single-channel input image V f ，φ b ]The method comprises the steps of carrying out a first treatment on the surface of the The characteristic image of the decoding stage consists of the characteristic image output by the previous decoding layer and the characteristic image with the same size of the encoding stage;
24 Pre-training the convolutional neural network using the composite image and corresponding forward and reverse deformation field data, generating a deformation field using randomly sampled B-spline deformation parameters, and leaving the generated deformation field unfolded; using the generated variationsThe shape field deforms the template image T to generate a composite image V syn And calculating a reverse deformation field;
3) Training the registration network of the step 2) in an unsupervised mode to obtain a trained registration network; the method comprises the following steps:
for having distributionV= { V i I=1,..a., N }, N is the number of images of the training dataset, minimizing the loss function by finding the optimal network optimizable parameter Θ, expressed as equation 2:
wherein h is a registration network; Θ is a learnable parameter of the registration network h; v is a training image; t is the template image;is the distribution of the training dataset;
the loss function for registration network training is expressed as equation 3:
wherein L is sim Measure loss for similarity; l (L) reg Is a deformation field regularization term; l (L) inv Is a deformation field combination reversible constraint term; l (L) str SOI registration loss for the structure of interest; Θ is the convolutional neural network optimizable parameter; alpha sim 、α reg 、α inv 、α str Weights of the corresponding penalty terms, respectively;
4) Performing online registration on the three-dimensional CBCT image to be registered:
for any pair of floating images V M And a fixed image V F Is denoted as (V) M ，V F )，Returning V respectively through the trained registration network in the step 3) M And V F Forward and reverse deformation fields of (a)And obtaining a final registration deformation field phi through deformation field combination M→F And phi F→M Expressed as:
wherein phi is used M→F Representing that the floating image can be deformed to a fixed image, using phi F→M The fixed image may be deformed to a floating image,and V is equal to M Has similar apparent characteristics->And V is equal to F Has similar apparent characteristics;
obtaining a deformation field to realize image registration; the floating image is deformed by using the deformation field generated by registration, and the obtained image is the registered image;
through the steps, the registration of the three-dimensional craniofacial CBCT image is realized.
2. The method of three-dimensional craniofacial cone-beam CT image registration according to claim 1 wherein in step 22), the learnable parameter Θ of the registration network h is a convolution kernel weight of the convolutional neural network.
3. The method for three-dimensional craniofacial cone-beam CT image registration of claim 1 wherein the steps of22 Specifically setting an image pairAnd->With similar anatomical appearance, the optimal parameters Θ are obtained in training by measuring the forward and reverse registration similarity.
4. The method of three-dimensional craniofacial cone-beam CT image registration according to claim 1 wherein in step 23) the encoder comprises six convolution layers of 3 x 3 and 1 step size, each followed by an instance regularization, a leak ReLU activation function, and a 2 x 2 max pooling layer.
5. The method of three-dimensional craniofacial cone-beam CT image registration according to claim 1 wherein in step 23) the decoder comprises six convolution layers of 3 x 3 convolution kernel size with a step size of 1; each convolution layer is preceded by a x 2 upsampling layer and followed by an instance regularization and ReLU activation function layer.
6. The method for registration of three-dimensional craniofacial cone-beam CT images of claim 1 wherein step 3) registers a loss function of network trainingIn the similarity measure loss L sim Specifically expressed as formula 4:
where m is the number of images; phi (phi) f，i Is a forward deformation field; phi (phi) b，i Is a reverse deformation field;representing the use of a forward deformation field phi f，i A deformed input image; />Representing the use of the reverse deformation field phi b，i And (5) a deformed template image.
7. The method for registration of three-dimensional craniofacial cone-beam CT images of claim 1 wherein step 3) registers a loss function of network trainingIn, deformation field regularization term L reg For ensuring the smoothness of the forward and reverse deformation fields, spatial gradients of the deformation fields are used as regularization terms, expressed as equation 5:
wherein phi is f，i 、φ b，i For both the forward and reverse deformation fields, I.I F Is the Frobenius norm and m is the number of images.
8. The method for registration of three-dimensional craniofacial cone-beam CT images of claim 1 wherein step 3) registers a loss function of network trainingIn the deformation field combination reversible constraint term L inv Expressed as equation 6:
L inv the first term of the two-way conversion [ phi ] f ，φ b ]Is reciprocal, m is the number of images in a batch in training; I.I F Is the Frobenius norm;
L inv in the second item of (2),a combination of k deformation fields along the closed loop c; k images { V i I=0,..>Is an identity transformation, for a training set with N three-dimensional images, the number of closed loops with the length of more than or equal to 2 is +.>
9. The method of three-dimensional craniofacial cone-beam CT image registration according to claim 8, wherein only two or three closed loops formed by volume data are considered, and k has a value of 3.
10. The method for registration of three-dimensional craniofacial cone-beam CT images of claim 1 wherein step 3) registers a loss function of network trainingIn the SOI registration loss L of the structure of interest str For improving registration of fine structures, expressed as formula 7:
wherein M is a mask of the region of interest SOI, wherein a voxel value of 1 represents the region of interest and the rest voxel values are 0; m is the number of images; v (V) i An input image for a network; t is a template image; phi (phi) f，i The forward deformation field is output by the network; phi (phi) b，i The reverse deformation field is output by the network; the operator ∈ represents element-by-element matrix multiplication.
CN201911029712.5A 2019-10-28 2019-10-28 Three-dimensional craniofacial cone beam CT image registration method Active CN112734814B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911029712.5A CN112734814B (en) 2019-10-28 2019-10-28 Three-dimensional craniofacial cone beam CT image registration method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911029712.5A CN112734814B (en) 2019-10-28 2019-10-28 Three-dimensional craniofacial cone beam CT image registration method

Publications (2)

Publication Number Publication Date
CN112734814A CN112734814A (en) 2021-04-30
CN112734814B true CN112734814B (en) 2023-10-20

Family

ID=75589463

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911029712.5A Active CN112734814B (en) 2019-10-28 2019-10-28 Three-dimensional craniofacial cone beam CT image registration method

Country Status (1)

Country Link
CN (1) CN112734814B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344876B (en) * 2021-06-08 2023-05-12 安徽大学 Deformable registration method between CT and CBCT
CN116416289B (en) * 2023-06-12 2023-08-25 湖南大学 Multimode image registration method, system and medium based on depth curve learning

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106096616A (en) * 2016-06-08 2016-11-09 四川大学华西医院 A kind of nuclear magnetic resonance image feature extraction based on degree of depth study and sorting technique
EP3246875A2 (en) * 2016-05-18 2017-11-22 Siemens Healthcare GmbH Method and system for image registration using an intelligent artificial agent
CN107507234A (en) * 2017-08-29 2017-12-22 北京大学 Cone beam computed tomography image and x-ray image method for registering
CN107730542A (en) * 2017-08-29 2018-02-23 北京大学 Cone beam computed tomography image corresponds to and method for registering
US10304193B1 (en) * 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3246875A2 (en) * 2016-05-18 2017-11-22 Siemens Healthcare GmbH Method and system for image registration using an intelligent artificial agent
CN106096616A (en) * 2016-06-08 2016-11-09 四川大学华西医院 A kind of nuclear magnetic resonance image feature extraction based on degree of depth study and sorting technique
CN107507234A (en) * 2017-08-29 2017-12-22 北京大学 Cone beam computed tomography image and x-ray image method for registering
CN107730542A (en) * 2017-08-29 2018-02-23 北京大学 Cone beam computed tomography image corresponds to and method for registering
US10304193B1 (en) * 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
锥形束计算机断层扫描中牙齿的分割精度;刘怡;James MAH;许天民;;北京大学学报(医学版)(第01期);全文 *

Also Published As

Publication number Publication date
CN112734814A (en) 2021-04-30

Similar Documents

Publication Publication Date Title
Pishchulin et al. Building statistical shape spaces for 3d human modeling
CN108416802B (en) Multimode medical image non-rigid registration method and system based on deep learning
CN110599528B (en) Unsupervised three-dimensional medical image registration method and system based on neural network
Wu et al. SharpMean: groupwise registration guided by sharp mean image and tree-based registration
CN107358648A (en) Real-time full-automatic high quality three-dimensional facial reconstruction method based on individual facial image
Cifor et al. Hybrid feature-based diffeomorphic registration for tumor tracking in 2-D liver ultrasound images
CN112734814B (en) Three-dimensional craniofacial cone beam CT image registration method
US20140161352A1 (en) Iterative method for determining a two-dimensional or three-dimensional image on the basis of signals arising from x-ray tomography
Wang et al. 3D brain surface matching based on geodesics and local geometry
WO2024021523A1 (en) Graph network-based method and system for fully automatic segmentation of cerebral cortex surface
Zhao et al. S3Reg: superfast spherical surface registration based on deep learning
Ghanei et al. A 3D deformable surface model for segmentation of objects from volumetric data in medical images
Ma et al. Cortexode: Learning cortical surface reconstruction by neural odes
CN115830016B (en) Medical image registration model training method and equipment
CN114359360A (en) Two-way consistency constraint medical image registration algorithm based on countermeasure
CN115512110A (en) Medical image tumor segmentation method related to cross-modal attention mechanism
Jia et al. A novel dynamic multilevel technique for image registration
Yushkevich Continuous medial representation of brain structures using the biharmonic PDE
CN115439478B (en) Pulmonary lobe perfusion intensity assessment method, system, equipment and medium based on pulmonary perfusion
WO2024051018A1 (en) Pet parameter image enhancement method and apparatus, device, and storage medium
Osher et al. Variational problems and partial differential equations on implicit surfaces: Bye bye triangulated surfaces?
Anas et al. Ct scan registration with 3d dense motion field estimation using lsgan
CN111798500B (en) Differential synblast non-rigid registration algorithm based on hierarchical neighborhood spectral features
CN112991406B (en) Method for constructing brain map based on differential geometry technology
CN107767409A (en) Consistent point drift method for registering based on higher-dimension expression

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant