CN111784755A - Brain magnetic resonance image registration method fusing multi-scale information - Google Patents
Brain magnetic resonance image registration method fusing multi-scale information Download PDFInfo
- Publication number
- CN111784755A CN111784755A CN202010727073.6A CN202010727073A CN111784755A CN 111784755 A CN111784755 A CN 111784755A CN 202010727073 A CN202010727073 A CN 202010727073A CN 111784755 A CN111784755 A CN 111784755A
- Authority
- CN
- China
- Prior art keywords
- network
- data
- training
- magnetic resonance
- registration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 210000004556 brain Anatomy 0.000 title claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 25
- 238000002474 experimental method Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 238000013508 migration Methods 0.000 claims description 4
- 230000005012 migration Effects 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000012952 Resampling Methods 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 210000003625 skull Anatomy 0.000 claims description 2
- 238000012795 verification Methods 0.000 claims description 2
- 230000004927 fusion Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000011218 segmentation Effects 0.000 abstract description 2
- 230000007547 defect Effects 0.000 abstract 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 5
- 238000002591 computed tomography Methods 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 4
- 208000014644 Brain disease Diseases 0.000 description 2
- 210000000988 bone and bone Anatomy 0.000 description 2
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 210000004872 soft tissue Anatomy 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000003925 brain function Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
The existing 3D brain magnetic resonance image registration technology has defects in segmentation precision and speed, and has poor response capability to large deformation in an image to be registered. In order to realize efficient and rapid registration of brain magnetic resonance images, a brain magnetic resonance image registration method based on fusion of multi-scale information is provided. The extraction of multi-scale information is integrated in the characteristic extraction process of the image. Multi-scale information is a high-level feature with image space information so that we can make an accurate measure of similarity between images. The extraction of multi-scale features of the data is realized by using a U-shaped convolution network, and the generalization capability of the model is enhanced by automatically learning the inherent intrinsic relation between the same data sets. The invention can realize accurate and rapid registration.
Description
Technical Field
The invention belongs to the technical field of medical image processing, and particularly relates to a method for registering 3D brain magnetic resonance images.
Background
Both Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) use non-invasive imaging modalities to demonstrate high-definition images of patients without cranial bone artifacts. The diagnosis of brain diseases generally uses CT and MRI images, and the CT images can clearly observe slight changes of bone tissue morphology but are insensitive to changes of soft tissues. MRI-generated medical images have higher contrast and spatial resolution for soft tissue, and therefore MRI is typically used for brain function imaging. MRI is an essential auxiliary diagnostic tool in clinical brain disease diagnosis, and provides an important judgment basis for clinical diagnosis of doctors.
Registration (Registration) of medical images refers to a process of matching two images in spatial positions through a series of spatial transformations, and is mainly applied to the aspects of lesion detection, disease diagnosis, surgical planning, surgical navigation, curative effect evaluation and the like in clinical treatment. Currently, manual registration is generally performed by experienced experts in the clinical diagnosis process based on existing clinical medical knowledge, personal experience, and spatial imagination. The manual registration method is time-consuming and labor-consuming, and the registration result is highly subjective from person to person. Doctors can also experience significant errors due to fatigue during long-term work. Therefore, by using the technology of replacing manual registration with modern computers, more medical resources can be saved, more real-time, accurate and efficient image registration can be realized, and the lives of more patients can be saved.
The traditional medical image registration method mainly comprises a gray-scale-based method which is easily interfered by noise; the method based on the region transformation has large calculation amount and long time consumption; feature-based approaches, the registration result depends on the choice of feature points. The traditional registration method uses a mode of iterative registration, and because each registration uses a mode of iterative optimization from zero, the inherent relation between the same data sets is ignored, and therefore the traditional registration method needs a great deal of time.
The brain magnetic resonance image registration method based on fusion of multi-scale information can extract the distribution characteristics of global spatial information in the image, and has a good effect on large deformation in the image. In addition, the method trains the network by using a transfer learning method, avoids training network parameters from zero, can effectively improve the training speed of the network, learns the inherent internal relation of data, and can enable the model to have stronger generalization capability. Therefore, the present invention is mainly focused on dealing with large deformations occurring in the data and improving the registration speed of the images.
Disclosure of Invention
The invention provides a method for automatically and accurately registering a 3D brain magnetic resonance image by using a convolutional neural network, aiming at solving the problems that misjudgment and missed diagnosis exist in manual registration in medical image registration, and the traditional registration method cannot meet the real-time property of medical image registration.
The invention is realized by the following technical scheme: a brain magnetic resonance image registration method fusing multi-scale information. By passing
By adding the multi-scale feature layer in the coding and decoding path of the U-shaped convolutional network, the extracted features have spatial information capability, and a new convolutional block structure is used in the network, so that jump connection is added, and the extracted multi-scale feature information can be conveniently transmitted to a deeper layer of the model. By using the feature migration, the network can learn the intrinsic features among data, the network is prevented from restarting training every time, and the registration speed is improved.
(1) Data preprocessing: the preprocessing of the 3D brain magnetic resonance image has the effects of mapping data to a consistent weight space, reducing noise interference under different conditions, and facilitating network feature extraction and analysis processing after normalization sampling. Skull removing and segmenting the data, affine and normalization transformation of the data and the like, resampling the data to the same resolution, and cutting the data to the same size;
(2) and (3) loading model feature migration: the network can realize faster registration compared with the traditional method by learning the internal relation among data. According to the method, when training is started, the transferred initial weight is loaded, so that the situation that the parameters of the network start from zero is avoided, and the training speed of the network model can be improved.
(3) Training a network to extract features: multi-scale features are extracted using convolution operations. The network adopts a spatial network pooling layer formed by convolution kernels with different sizes to realize the extraction of multi-scale information. Using kernel size in the encoding path as
Extended convolution of (1) instead of size
Max-pooling layer of (2). After the characteristic diagram size is reduced through 4 layers of convolution in the coding path, a multi-scale convolution layer is connected, and the height of data is extractedA stage feature. The multi-scale convolutional layer is kernel size of
、
、
And residual (Res) structure. And generating a registered deformation field after continuously reducing the size of the feature map through a decoding path.
(4) Experiments were performed in Pycharm using the LPBA40 data pre-processed in (1) and images were randomly divided into a 24:8:8 ratio as training, validation and test sets for the experiments, respectively. A total of 1500 epochs were trained with one epoch every 100 times. Thus, the head office performed 150000 exercises. And adjusting network parameters in the training process until the network converges, and stopping training.
The method has the advantages that the convolution network with the multi-scale features is used, so that the deformation field generated by the network is closer to the real deformation field, the network is fast to register by using the migration features, the network learns the internal connection features of the data, the network has strong generalization capability, and the network can realize efficient and fast 3D brain magnetic resonance image registration.
Drawings
As shown in the drawings, fig. 1 is a modified convolution block structure. Fig. 2 is a network training flow chart of a brain magnetic resonance image registration method fusing pair-scale information.
Detailed Description
In order to verify the registration performance of the invention in the 3D brain magnetic resonance image, an LPBA40 public data set is selected for training, verification and testing.
The method comprises the following steps: preprocessing 40 brain magnetic resonance images, and realizing affine and normalization of data by using FreeSenr software.
Step two: training an improved U-shaped convolution network in Pycharm development software, optimizing by using an Adam optimizer, and setting the learning rate to be 0.0001. The batch size is set to 2, one epoch per 100 trains, 1500 epochs. The 40 images were randomly divided into a 24:8:8 ratio for training, validation and testing. And training the network until the network converges, and stopping training.
Step three: the experiment uses LPBA40 public data set network untrained (Unscreen) data for testing, and the experiment result is evaluated by a coincidence degree coefficient (Dice). The experimental results are shown below:
TABLE 1 brain MRI registration results with fusion of multi-scale information
| 1. Method of producing a composite material | 2.Dice | 3.GPU(s) | 4.CPU(s) |
| 5. Multi-scale U-net | 6. 0.762 | 7. 0.45 | 8. 57 |
Experimental results show that the brain magnetic resonance image registration method fusing multi-scale information can effectively realize rapid and accurate segmentation of the 3D brain magnetic resonance image.
Claims (1)
1. The invention discloses a registration method for a 3D brain magnetic resonance image, which comprises the following steps:
(1) data preprocessing: the preprocessing of the 3D brain magnetic resonance image has the effects of mapping data to a consistent weight space, reducing noise interference under different conditions, and facilitating network feature extraction and analysis processing after normalization sampling; skull removing and segmenting the data, affine and normalization transformation of the data and the like, resampling the data to the same resolution, and cutting the data to the same size;
(2) feature extraction: extracting the characteristics of the data by using a U-shaped convolution network, training a network model by using the extracted characteristics with global spatial information, and continuously adjusting model parameters in the training process; extracting higher-level features from the data, so that the network can more efficiently and quickly realize accurate registration among images; the process is as follows:
a. and (3) loading model feature migration: the network can realize faster registration compared with the traditional method by learning the internal relation among data; when training starts, the method loads a transferred initial weight, avoids starting network parameters from zero, and can improve the training speed of a network model;
b. training a network to extract features: extracting multi-scale features of the data by using a U-shaped convolution network; in order to make the extracted features more spatial, the network adopts a spatial network pooling layer consisting of convolution kernels with different sizes to realize the extraction of multi-scale information; replacing a maximum pooling layer with an expanded convolution in a coding path of the network, connecting a multi-scale convolution layer after reducing the size of a feature map through 4 layers of convolution in the coding path, extracting high-level features of data, and generating a registered deformation field after continuously reducing the size of the data through a decoding path;
c. the experiment is carried out in Pycharm, an LPBA40 public data set subjected to data preprocessing in the step (1) is used, images are randomly divided into a ratio of 24:8:8, and the ratio is respectively used as a training set, a verification set and a test set of the experiment; training for 1500 epochs in total according to one epoch every 100 times; thus, a total of 150000 training sessions were performed; and adjusting network parameters in the training process until the network converges, and stopping training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010727073.6A CN111784755A (en) | 2020-07-26 | 2020-07-26 | Brain magnetic resonance image registration method fusing multi-scale information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010727073.6A CN111784755A (en) | 2020-07-26 | 2020-07-26 | Brain magnetic resonance image registration method fusing multi-scale information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111784755A true CN111784755A (en) | 2020-10-16 |
Family
ID=72763979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010727073.6A Pending CN111784755A (en) | 2020-07-26 | 2020-07-26 | Brain magnetic resonance image registration method fusing multi-scale information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111784755A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113808182A (en) * | 2021-11-19 | 2021-12-17 | 首都医科大学附属北京安贞医院 | 2D and 3D image registration method and device |
-
2020
- 2020-07-26 CN CN202010727073.6A patent/CN111784755A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113808182A (en) * | 2021-11-19 | 2021-12-17 | 首都医科大学附属北京安贞医院 | 2D and 3D image registration method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3432784B1 (en) | Deep-learning-based cancer classification using a hierarchical classification framework | |
| CN111833359B (en) | Brain tumor segmentation data enhancement method based on generation of confrontation network | |
| Charron et al. | Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network | |
| CN105741251B (en) | A kind of blood vessel segmentation method of Hepatic CT A sequence images | |
| Zhang et al. | Multi‐needle localization with attention U‐net in US‐guided HDR prostate brachytherapy | |
| CN107563434B (en) | Brain MRI image classification method and device based on three-dimensional convolutional neural network | |
| Wang et al. | Learning methods of convolutional neural network combined with image feature extraction in brain tumor detection | |
| CN109753997B (en) | Automatic accurate robust segmentation method for liver tumor in CT image | |
| CN113284126B (en) | Method for predicting hydrocephalus shunt operation curative effect by artificial neural network image analysis | |
| CN106651874B (en) | Space domain splitting method after brain tumor surgery based on multi-modal MRI data | |
| Arai et al. | Significant dimension reduction of 3D brain MRI using 3D convolutional autoencoders | |
| CN109215035B (en) | Brain MRI hippocampus three-dimensional segmentation method based on deep learning | |
| Wang et al. | EE-Net: An edge-enhanced deep learning network for jointly identifying corneal micro-layers from optical coherence tomography | |
| Hao et al. | Magnetic resonance image segmentation based on multi-scale convolutional neural network | |
| CN115311193A (en) | Abnormal brain image segmentation method and system based on double attention mechanism | |
| CN113362944B (en) | Assessment method of functional dyspepsia and needling curative effect prediction model based on machine learning | |
| CN111784755A (en) | Brain magnetic resonance image registration method fusing multi-scale information | |
| Derikvand et al. | Brain tumor segmentation in MRI images using a hybrid deep network based on patch and pixel | |
| Memiş et al. | Computerized 2D detection of the multiform femoral heads in magnetic resonance imaging (MRI) sections with the integro-differential operator | |
| WO2023133929A1 (en) | Ultrasound-based human tissue symmetry detection and analysis method | |
| CN111951219B (en) | Thyroid eye disease screening method, system and equipment based on orbit CT image | |
| CN113744234A (en) | Multi-modal brain image registration method based on GAN | |
| CN113744207A (en) | Medical image registration method fusing multi-scale information | |
| Astaraki et al. | Fully Automatic Segmentation of Gross Target Volume and Organs-at-Risk for Radiotherapy Planning of Nasopharyngeal Carcinoma | |
| Hsu et al. | A comprehensive study of age-related macular degeneration detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20201016 |
|
| WD01 | Invention patent application deemed withdrawn after publication |