CN112750097A - Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network - Google Patents

Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network Download PDF

Info

Publication number
CN112750097A
CN112750097A CN202110050993.3A CN202110050993A CN112750097A CN 112750097 A CN112750097 A CN 112750097A CN 202110050993 A CN202110050993 A CN 202110050993A CN 112750097 A CN112750097 A CN 112750097A
Authority
CN
China
Prior art keywords
gabor
fusion
image
layer
fuzzy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110050993.3A
Other languages
Chinese (zh)
Other versions
CN112750097B (en
Inventor
王丽芳
张晋
王蕊芳
张炯
米嘉
刘阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
North University of China
Original Assignee
North University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by North University of China filed Critical North University of China
Priority to CN202110050993.3A priority Critical patent/CN112750097B/en
Publication of CN112750097A publication Critical patent/CN112750097A/en
Application granted granted Critical
Publication of CN112750097B publication Critical patent/CN112750097B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/043—Architecture, e.g. interconnection topology based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20192—Edge enhancement; Edge preservation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Computational Mathematics (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention belongs to the field of medical image fusion, and particularly relates to multi-modal medical image fusion based on multi-CNN combination and a fuzzy neural network. In order to represent the texture detail of a focus part in a multi-modal medical image more fully and to make the edge clearer, the method provided by the invention mainly comprises two parts: 1) constructing a G-CNN group (G-CNNs); 2) G-CNNs fusion based on fuzzy neural networks. The first part firstly obtains different Gabor representation pairs of CT and MR through a group of Gabor filters with different proportions and directions, and then trains corresponding CNN by using each pair of different Gabor representations respectively, thereby generating G-CNNs; and the second part fuses a plurality of outputs of the G-CNNs by using a fuzzy neural network to obtain a final fused image.

Description

Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network
Technical Field
The invention belongs to the field of medical image fusion, and particularly relates to multi-modal medical image fusion based on multi-CNN combination and a fuzzy neural network.
Background
Image fusion has a wide range of applications, such as medical imaging, remote sensing, machine vision, biometric identification, and military applications. The goal of fusion is to obtain better contrast and perception experience. In recent years, with the increasing demand for clinical applications, research on multi-modal medical image fusion has been receiving attention. The objective of multi-modal medical image fusion is to provide a better medical image to assist the surgeon in surgical intervention.
Nowadays, medical images have multiple modalities, such as Magnetic Resonance (MR) images, Computed Tomography (CT) images, Positron Emission Tomography (PET) images, X-ray images, and the like, and images of different modalities have their own advantages and limitations, for example, CT can well display bone information, but cannot clearly display structural information such as soft tissue; the MR image can fully display soft tissue information, but has great defects on the detection of bone information; the PET image can provide abundant human body metabolism information for clinic, but the resolution is low. Therefore, the medical image information of multiple modes is combined to complete multi-mode image fusion, and the complementary advantages can be realized. The multi-modal fusion image not only retains the characteristics of the original image, but also makes up the defects of the single-modal medical image, shows richer detail information and provides comprehensive information for clinical diagnosis and treatment and image-guided surgery.
The image fusion method is divided into three layers: pixel level, feature level, decision level. The image fusion at pixel level is to combine the pixel values of each point corresponding to two or more source images by a certain fusion method to calculate a new pixel value, so that each point pixel is fused to form a new fusion image. Common pixel-level image fusion methods include a spatial domain-based fusion method and a transform domain-based fusion method. The image fusion based on the spatial domain is mainly divided into block-based fusion and region-based fusion, including a logic filtering method, a weighted average method, a mathematical morphology method, an image algebra method, a simulated annealing method and the like, and the method has the advantages of small calculated amount, easiness in realization, poor accuracy and unsuitability for the field of medical images. The method based on the transform domain is to decompose the source image, combine the decomposed source images by using different fusion rules, and finally execute inverse transformation operation to reconstruct the fusion image, and comprises a pyramid image fusion method, a wavelet transform image fusion method, a multi-scale decomposition method and the like, so that the method not only can maintain the contrast and reduce the blocking effect, but also has unique advantages in describing the local characteristics of signals.
The feature level image fusion method is to extract feature information which is interesting to an observer, such as edges, contours, shapes, local features and other information from a source image, and then analyze, process and integrate the feature information to obtain fused image features. The currently common methods are: weighted average method, Bayesian estimation method, cluster analysis method.
The decision-level image fusion method is to analyze, reason, recognize, judge and the like the characteristic information of each image to form corresponding results, and then further fuse, wherein the final fusion result is a global optimal decision. The method has the advantages of good real-time performance, high flexibility and certain fault-tolerant capability, but the preprocessing cost is higher, and the loss of original information in the image is more.
In recent years, with the rise of deep learning, a Convolutional Neural Network (CNN) has stronger feature extraction capability than that of a conventional method as an important branch of deep learning, and is suitable for image fusion.
The fusion method based on the spatial domain can distort the spectrum and the space of the fused image, which is not beneficial for the doctor to observe the pathological change condition of the focus area. Pyramid transformation in the transform domain based fusion method fails to introduce spatial direction selectivity in the decomposition process, thereby producing images with blocking artifacts, and introduces many artifacts in the edges of the fused images to affect the fusion result; the wavelet transform can only capture limited direction information, and the information acquisition in the edge and texture area is limited, so that the edge of the image cannot be clearly represented.
Liu et al propose a multi-focus image fusion method based on a Convolutional Neural Network (CNN), which classifies focus regions by using the CNN to obtain a decision graph. The decision graph is combined with the source image to generate a fused image, but the CNN model proposed by liu has lower feature dimensionality of final output of convolution and downsampling, and information loss of the fused image is caused. zhang et al propose a general image fusion framework FCNN based on a full convolution neural network, which solves the problem of information loss by using the full convolution neural network, but the convolution kernel of the convolution layer is too single in setting, and the extracted features can not well represent the texture information of the focus part, so that the fused image is difficult to be directly used for clinical diagnosis
In order to obtain a fused image with abundant textures, clear edges and high information content, the invention provides a method for fusing multi-modal medical images based on Gabor expressed multi-CNN combinations (Gabor-CNNs, G-CNNs) and a fuzzy neural network. The proposed method mainly consists of two parts: 1) constructing a G-CNN group (G-CNNs); 2) G-CNNs fusion based on fuzzy neural networks. The first part firstly obtains different Gabor representation pairs of CT and MR through a group of Gabor filters with different proportions and directions, and then trains corresponding CNN by using each pair of different Gabor representations respectively, thereby generating G-CNNs; and the second part fuses a plurality of outputs of the G-CNNs by using a fuzzy neural network to obtain a final fused image. The method utilizes Gabor to represent the pair: gaborCTAnd GaborMRThe method is used as the input of the CNN to enhance and reserve the texture features and edge details of the source image, then the CNN is used for extracting the depth features of the source image by utilizing the advantage that the CNN can extract effective information from a complex background, finally the fuzzy neural network is used for solving the problems of fuzziness and uncertainty in the fusion process, the obtained fusion image reserves the rich texture features and clear edge information of the CT and MR of the medical source image, and the visual characteristics and the information content are also obviously improved.
Disclosure of Invention
In order to fully represent texture details of a focus part in a multi-modal medical image and make edges clearer, the multi-modal medical image is fused based on a method of combining Gabor-CNNs (G-CNNs) and fuzzy neural networks represented by Gabor.
In order to achieve the purpose, the invention adopts the following technical scheme:
the multi-modal medical image fusion based on the multi-CNN combination and the fuzzy neural network comprises the following steps:
step 1, filtering the CT and MR images of the training data set by a filter bank consisting of 16 Gabor filters with different proportions and directions to obtain 16 different Gabor representation pairs of the CT and the MR:
Figure BDA0002899099800000041
and
Figure BDA0002899099800000042
1.. 16, the representation has detail texture information in a plurality of different directions to enhance the texture features of the source image, and then each Gabor representation pair is used:
Figure BDA0002899099800000043
and
Figure BDA0002899099800000044
1.. 16 to train the respective 16G-CNNs, the 16G-CNNs are combined into G-CNNs, and since the data sets for training each G-CNN are all in different texture directions, the information that each G-CNN characterizes texture is different.
And 2, putting each pair of Gabor representations into corresponding G-CNNs for preliminary fusion, and then fusing a plurality of outputs of the G-CNNs by using a fuzzy neural network to obtain a final fused image.
Further, Gabor representation of the medical image can enhance the texture characteristics of the source image and obtain more edge details, so that the fused image contains more texture and edge information about the relevant lesion site. To obtain Gabor representations of CT and MR for a given medical image, a set of Gabor filter banks of different dimensions and orientations needs to be designed. Both the CT image and the MR image are two-dimensional gray scale images, so a group of two-dimensional Gabor filters are defined, the filter bank in the step 1 is a group of two-dimensional Gabor filters, and the formula is as follows:
Figure BDA0002899099800000045
in the formula (1), U represents the direction of the filter bank and is selected to be 0 degrees, 90 degrees, 180 degrees and 270 degrees; v denotes the filter bank dimensions set to 4, 8, 16 and 32; z ═ x, y denotes the position of the pixel,
Figure BDA0002899099800000046
kv＝kmax/fv，φu＝πu/8，kmaxat maximum frequency, f is the spacing factor between filters in the frequency domain. According to the formula (1), the filter can set different U directions and V scales, and different Gabor representations of the medical image are displayed.
Still further, the step 1 yields 16 different Gabor representation pairs of CT and MR:
Figure BDA0002899099800000047
and
Figure BDA0002899099800000051
1.. 16, Gabor representations of CT and MR are obtained for specific direction u and v scales, in particular by the following formula:
GaborCT (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (2)
GaborMR (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (3)
formula (2) in formula (3), I (z) represents a given image, #u,v(z) denotes a filter, denotes a convolution operation, GaborCT (u,v)(z) and GaborMR (u,v)(z) is the Gabor representation of CT and MR for a given image. Thus, a 4 × 4 filter bank is obtained, and after the images CT and MR are convolved by the Gabor filter bank, 16 groups of Gabor filters with different directions and different scales are obtainedCTAnd GaborMRAnd (4) showing. These watchesThe characteristics of the focus part are characterized from different directions, so that the key information is strengthened and is used as a training set of the G-CNNs.
To realize to GaborCTAnd GaborMRImage fusion, wherein a model structure of G-CNN is designed as shown in FIG. 2, and the structure consists of three parts: feature extraction, feature fusion and image reconstruction.
Since the downsampling feature mapping inevitably loses the source information of the input image and affects the quality of the image, no downsampling operation is performed in the G-CNN structure, so that the size of the feature map output by each layer and the input Gabor are reducedCTAnd GaborMRThe image remains consistent. The network structure is composed of 4 convolutional layers CONV1, CONV2, CONV3 and CONV4, as shown in fig. 2. Firstly, extracting information characteristics of an image by adopting two convolutional layers CONV1 and CONV 2; secondly, fusing the convolution characteristics of the input image by using a characteristic fusion FUSE; finally, the fused features are reconstructed by two convolutional layers CONV3 and CONV4 to obtain a fused image.
Further, the G-CNN in the step 2 consists of three parts of feature extraction, feature fusion and image reconstruction:
feature extraction: 3X 224 GaborCTAnd GaborMRInputting an image into a G-CNN, extracting depth features from an input Gabor-based representation source image by adopting two convolutional layers, and adopting a first convolutional layer of an advanced ResNet101 pre-trained on ImageNet as a first convolutional layer CONV1, wherein the CONV1 comprises 64 convolutional kernels with the size of 7 multiplied by 7, and the step size and the filling parameter are respectively set to be 1 and 3; in order to extract deeper features and adjust convolution features of CONV1, a second convolution layer CONV2 is added, the number of kernels and the kernel size of CONV2 are respectively set to be 64 and 3 x 3, the step size and the filling parameter of CONV2 are set to be 1, in order to overcome the overfitting problem and accelerate network convergence, the CONV2 sets a ReLU activation function and a batch processing normalization function, and 64 Gabor of 224 x 224 are obtained through convolution operations of two convolution layersCTAnd GaborMRA feature map of the image;
feature fusion: in the fusion process, the convolution characteristics of multiple inputs are fused by using an element-level fusion rule, as shown in formula (4):
Figure BDA0002899099800000061
in the formula (4), N represents the number of input images, N is not less than 2,
Figure BDA0002899099800000062
a feature map representing the i-th input image extracted by CONV2,
Figure BDA0002899099800000063
representing a fusion feature graph generated by a feature fusion module, wherein fuse represents an element level fusion rule;
image reconstruction: after feature fusion, reconstructing a fused image from the fused convolution features by adopting two convolution layers CONV3 and CONV4, wherein the CONV3 parameter setting is the same as that of CONV2 and can play a role in adjusting the fused convolution features, so that the kernel number and the kernel size of CONV3 are respectively set to be 64 and 3 multiplied by 3, and a ReLU activation function and a batch processing normalization function which are the same as those of CONV2 are also set; the CONV4 reconstructs and outputs the feature mapping through element weighted average, the kernel number and the kernel size of the CONV4 are set to be 3 and 1 multiplied by 1 respectively, and the step size and the filling parameter are set to be 0; after convolution operation of two convolution layers through image reconstruction, 3 × 224 × 224 Gabor is obtainedCTAnd GaborMRThe size of the characteristic graph in the whole process is consistent with the size of the input source image in the model, and the Gabor of the source image is well keptCTAnd GaborMRThe obtained fused image has rich texture features and clear edges of the focus part, but has the problems of uncertain and fuzzy boundary of the diffuse focus area.
Still further, the element-level fusion rule in step 2.2 is an element maximum fusion rule.
Further, the fuzzy neural network in the step 2 is composed of an input layer, a fuzzy partition layer, a forward combination layer, an inference layer and an output layer;
in the first input layer of the first layer,sixteen neurons x1，x2，...，x16Is a specific pixel value represented by the same position of 16 Gabor fusion images obtained by G-CNNs;
in the second layer of blur segmentation layer, due to the different gray levels of different tissues in the medical image, the pixels are segmented into five blur sets according to pixel values: very dark, normal, light and very light, and the membership function is used to represent the fuzzy set and is expressed as a gaussian function, as shown in equation (8):
Figure BDA0002899099800000071
in the formula (8), c and σ are the center and width of a gaussian function, i is 1, 2,. 16, j is 1, 2,. 5, different fuzzy sets correspond to different membership functions, five different membership functions sequentially correspond to 'very dark', 'normal', 'bright' and 'very bright', for a specific pixel value, the five membership functions have different output μ values, and the fuzzy set corresponding to the maximum μ value is selected as the fuzzy set of the pixel value;
in the third forward combination layer, which is the premise of forming rules, nodes of the same fuzzy set are arranged together;
in the fourth layer of inference layer, it is to form rule-based inference, its node is that sixteen nodes representing the same fuzzy set in the third layer are connected to form, five fuzzy rules are set up according to five fuzzy sets divided (as shown in fig. 6): when x is1，x2，...x16The dark and bright pixel values are all very dark, and the output rule is
Figure BDA0002899099800000072
When x is1，x2，...x16The dark and bright pixel values belong to the dark degree, and the output rule is
Figure BDA0002899099800000073
When x is1，x2，...x16The pixel values are dark and brightDegree of normality, the output rule is
Figure BDA0002899099800000074
When x is1，x2，...x16The dark and bright pixel values belong to the brightness degree, and the output rule is
Figure BDA0002899099800000081
When x is1，x2，...x16The dark and bright pixel values belong to the very bright degree, and the output rule is
Figure BDA0002899099800000082
In the fifth output layer, the neuron represents the pixel value of the fused image based on the same position as the input image, and in order to obtain the fused image, a connection weighting factor V between the inference layer and the output layer is set as follows:
Figure BDA0002899099800000083
in the formula (9), p0、p1...p16Is a weighting factor, l ═ 1, 2, 3,. 5;
the pixel values of 16 Gabor fused images were calculated by the deblurring function as follows:
Figure BDA0002899099800000084
in the formula (10), L is 5, RlIs the rule output value.
16 Gabor fusion images obtained by G-CNNs are finally fused into a fusion image through a five-layer fuzzy neural network, the problem of fuzzy characteristic uncertainty in the fusion process is solved, the CT/MR fusion image is finally obtained, the edges and the textures of the fusion image are clear, and the content is rich.
Compared with the prior art, the invention has the following advantages:
the invention provides multi-modal medical image fusion of multi-CNN combination and fuzzy neural network based on Gabor representation. According to the method, the Gabor representation is used as the input of the CNN to enhance and retain the texture features and edge details of the focus part in the source image, then the CNN is used for extracting the depth features of the source image from the complex background, and the integrated thought in machine learning is used for training a plurality of G-CNNs, wherein each G-CNN has different texture information representing capability, the fuzzy and uncertain problems in the fusion process are solved by using the fuzzy neural network, and the obtained fusion image retains the rich texture features and clear edge information of the focus part in the medical source image.
Drawings
FIG. 1 is a frame diagram of multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network of the present invention;
FIG. 2 is a diagram of a model structure of G-CNN of the present invention;
FIG. 3 is a diagram of a model neural network architecture of the present invention;
FIG. 4 is a fusion diagram of G-CNNs based on the fuzzy neural network of the present invention;
FIG. 5 is a fuzzy set corresponding to the membership function of the present invention;
FIG. 6 shows five fusion rules for fuzzy neural network fusion according to the present invention;
FIG. 7 is a graph of the fusion result of CT and MR images of cerebrovascular disease;
FIG. 8 is a graph of CT/MR fusion results for cerebral infarction disease;
FIG. 9 is a graph of CT/MR fusion results for brain tumor disease.
Detailed Description
As shown in fig. 1, the multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network of the present invention comprises the following steps:
step 1, filtering the CT and MR images of the training data set by a filter bank consisting of 16 two-dimensional Gabor filters with different proportions and directions, wherein the formula is as follows:
Figure BDA0002899099800000091
in the formula (1), U represents the direction of the filter bank and is selected to be 0 degrees, 90 degrees, 180 degrees and 270 degrees; v denotes the filter bank dimensions set to 4, 8, 16 and 32; z ═ x, y denotes the position of the pixel,
Figure BDA0002899099800000092
kv＝kmax/fv，φu＝πu/8，kmaxat maximum frequency, f is the spacing factor between filters in the frequency domain. According to the formula (1), the filter can set different U directions and V scales, and different Gabor representations of the medical image are displayed.
Gabor representations of CT and MR for specific orientation u and v dimensions are obtained by the following formula:
GaborCT (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (2)
GaborMR (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (3)
formula (2) in formula (3), I (z) represents a given image, #u,v(z) denotes a filter, denotes a convolution operation, GaborCT (u,v)(z) and GaborMR (u,v)(z) is the Gabor representation of CT and MR for a given image, resulting in 16 different Gabor representation pairs for CT and MR:
Figure BDA0002899099800000101
and
Figure BDA0002899099800000102
i＝1,...,16；
each pair of Gabor is then used to represent a pair:
Figure BDA0002899099800000103
and
Figure BDA0002899099800000104
1.. 16 to train the respective 16G-CNNs, the 16G-CNNs consisting of G-CNNs;
step 2, putting each pair of Gabor representations into corresponding G-CNN for preliminary fusion, designing the structure of the G-CNN as shown in figure 2, and consisting of three parts: the method comprises the following steps of characteristic extraction, characteristic fusion and image reconstruction, and specifically comprises the following steps:
feature extraction: 3X 224 GaborCTAnd GaborMRInputting an image into a G-CNN network, extracting depth features from an input Gabor-based representation source image by adopting two convolutional layers, and adopting a first convolutional layer of an advanced ResNet101 pre-trained on ImageNet as a first convolutional layer CONV1, wherein the CONV1 comprises 64 convolutional cores with the size of 7 multiplied by 7, and the step size and the filling parameter are respectively set to be 1 and 3; adding a second convolutional layer CONV2, setting the kernel number and the kernel size of CONV2 to be 64 and 3 multiplied by 3 respectively, setting the step size and the filling parameter of CONV2 to be 1, setting the ReLU activation function and the batch normalization function of CONV2, and respectively obtaining 64 Gabor of 224 multiplied by 224 through convolution operation of the two convolutional layersCTAnd GaborMRA feature map of the image;
feature fusion: in the fusion process, the convolution characteristics of multiple inputs are fused by using an element maximum fusion rule, as shown in formula (4):
Figure BDA0002899099800000105
in the formula (4), N represents the number of input images, N is not less than 2,
Figure BDA0002899099800000106
a feature map representing the i-th input image extracted by CONV2,
Figure BDA0002899099800000111
representing a fusion feature graph generated by a feature fusion module, wherein fuse represents an element level fusion rule, and an element maximum fusion rule is selected;
image reconstruction: after feature fusion, a fused image is reconstructed from the fused convolution features using two convolution layers CONV3 and CONV4, the number of kernels and the kernel size of CONV3 are set to 64 and 3 × 3, respectively, and the same ReL as that of CONV2 is set in the same mannerA U activation function and a batch processing normalization function; the CONV4 reconstructs and outputs the feature mapping through element weighted average, the kernel number and the kernel size of the CONV4 are set to be 3 and 1 multiplied by 1 respectively, and the step size and the filling parameter are set to be 0; after convolution operation of two convolution layers through image reconstruction, 3 × 224 × 224 Gabor is obtainedCTAnd GaborMRThe fused image of (1).
When CT and MR images are fused, different Gabor representations are obtained from the CT and MR images through a filter bank, the corresponding Gabor representations are put into the corresponding G-CNN for fusion, and Gabor fusion images are obtained, and as shown in figure 3, the fusion images have the problems of uncertain and fuzzy boundary of a diffuse focus boundary region. The fuzzy neural network can effectively solve the defects of fuzziness, uncertainty and the like in the fusion process through the fuzzy set and the membership function, so that the G-CNNs fusion images are fused together through the fuzzy neural network to obtain the final fusion image.
Then, fusing a plurality of outputs of the G-CNNs by using a fuzzy neural network to obtain a final fused image, wherein the fuzzy neural network consists of an input layer, a fuzzy partition layer, a forward combination layer, a reasoning layer and an output layer as shown in a dotted line frame in figure 4;
in the first input layer, sixteen neurons x1，x2，...，x16Is a specific pixel value represented by the same position of 16 Gabor fusion images obtained by G-CNNs;
in the second-layer blur division layer, pixels are divided into five blur sets according to pixel values: very dark, normal, light and very light, and the membership function is used to represent the fuzzy set and is expressed as a gaussian function, as shown in equation (8):
Figure BDA0002899099800000121
in equation (8), c and σ are the center and width of the gaussian function, i is 1, 2,. 16, j is 1, 2,. 5, different fuzzy sets correspond to different membership functions, the logical order is as shown in fig. 5, five different membership functions correspond to "very dark", "normal", "bright", and "very bright" in sequence, the abscissa represents the image pixel value, and the ordinate represents the output value of the membership function μ. The specific process is as follows: for a specific pixel value, the five membership functions have different output mu values, and a fuzzy set corresponding to the maximum mu value is selected as a fuzzy set of the pixel value;
arranging nodes of the same fuzzy set together in a third forward combination layer;
in the fourth layer of inference layer, the nodes are formed by connecting sixteen nodes representing the same fuzzy set in the third layer, and five fuzzy rules are set according to the five fuzzy sets (as shown in fig. 6): when x is1，x2，...x16The dark and bright pixel values are all very dark, and the output rule is
Figure BDA0002899099800000122
When x is1，x2，...x16The dark and bright pixel values belong to the dark degree, and the output rule is
Figure BDA0002899099800000123
When x is1，x2，...x16The pixel values are all in normal degree, and the output rule is
Figure BDA0002899099800000124
When x is1，x2，...x16The dark and bright pixel values belong to the brightness degree, and the output rule is
Figure BDA0002899099800000125
When x is1，x2，...x16The dark and bright pixel values belong to the very bright degree, and the output rule is
Figure BDA0002899099800000126
In the fifth output layer, the neuron represents the pixel value of the fused image based on the same position as the input image, and in order to obtain the fused image, the connection weighting factor v between the inference layer and the output layer is set as follows:
Figure BDA0002899099800000127
in the formula (9), p0、p1...p16Is a weighting factor, l ═ 1, 2, 3,. 5;
the pixel values of 16 Gabor fused images were calculated by the deblurring function as follows:
Figure BDA0002899099800000131
in the formula (10), L is 5, RlIs the rule output value.
16 Gabor fusion images obtained by G-CNNs are finally fused into a fusion image through a five-layer fuzzy neural network, the problem of fuzzy characteristic uncertainty in the fusion process is solved, the CT/MR fusion image is finally obtained, the edges and the textures of the fusion image are clear, and the content is rich.
To verify the effectiveness of the method of the invention, the following three sets of data were used:
1. CT/MR image fusion experiment for cerebrovascular disease
TABLE 1 comparison of CT/MR image fusion results for cerebrovascular disorders
Figure BDA0002899099800000132
Fig. 7 shows the fusion result of CT and MR images of cerebrovascular diseases, which should preserve the marginal details of the vascular lesion as much as possible to facilitate the diagnosis and observation of the cerebral cortical structure by the doctor. Wherein a and b in fig. 8 represent CT and MR source images, respectively, of cerebrovascular disease and c-h in fig. 8 are the fusion results of DWT, LAP, NSST-PCNN, CNN, GFF and the proposed method, respectively. To more visually show the differences in the comparison methods, the experimental results are marked with yellow rectangles. From the fusion result region of the yellow mark, the fusion obtained by the DWT methodThe resultant results lack edge information and have poor visual effect; the fusion result obtained by the LAP method loses many edges in the MR image to cause image blurring; the NSST-PCNN fusion image has high brightness, and the edge information of the focus part is unclear; the CNN method has the problem of low edge information contrast, and the GFF method and the proposed method have good fusion effect. From the quantitative data in Table 1, it can be seen that the methods DWT, LAP, NSST-PCNN, CNN are applied to MI index and QAB/FThe low index indicates that the marginal information storage capacity of the methods is low compared with the proposed method, which indicates that the method provided by the invention can keep the complete information of the focus margin, has good fusion quality, and is beneficial to doctors to accurately diagnose and observe the boundary information of the focus area.
2. CT/MR image fusion experiment for cerebral infarction disease
TABLE 2 CT/MR image fusion result Performance comparison for cerebral infarction diseases
Figure BDA0002899099800000141
Fig. 8 shows the CT/MR image fusion result of the cerebral infarction disease, and the fusion result should fully characterize the textural features of the lesion site as much as possible so that the doctor can accurately determine the infarct site and the infarct degree of the cerebral infarction disease. In fig. 8 a and b represent CT and MR source images of a cerebral infarction disease, respectively, and in fig. 8 c-h are the fusion results of DWT, LAP, NSST-PCNN, CNN, GFF and the proposed method, respectively. The similarity of the result obtained by the DWT method and the source image is low, and the texture characteristics of the pathological change part in the source image cannot be well reserved; the image obtained by the LAP method has low contrast and large noise, so that the texture features of the lesion area have partial blurring; the fusion result obtained by NSSI-PCNN has higher brightness in the lesion region, so that the texture information in the image is weakened; the texture feature contrast of the lesion region of the fused image obtained by the CNN method is low; the texture features of the GFF fused image in the lesion region are not sufficiently characterized. It can be seen from table 3 that the fusion methods proposed in the four indexes are higher than other fusion methods. From the perspective of the fusion result and the quantitative index, the method of the invention can fully and clearly keep the textural features of the cerebral infarction lesion part, has good fusion quality, and is beneficial to diagnosis of doctors.
3. CT/MR image fusion experiment for brain tumor diseases
TABLE 3 brain CT/MR image fusion result Performance comparison
Figure BDA0002899099800000151
Fig. 9 shows the fusion result of CT/MR images of brain tumor diseases, and the fusion result should reserve abundant texture features and clear edge information of the brain tumor region as much as possible to help doctors to accurately determine the grade and boundary of the brain tumor, so as to facilitate the diagnosis and excision of the brain tumor by the doctors. In fig. 9 a and b represent CT and MR source images, respectively, of a brain tumor disease, and in fig. 9 c-h are the fusion results of DWT, LAP, NSST-PCNN, CNN, GFF and the proposed method, respectively. The fused image obtained by the DWT method has lower definition at the edge of a yellow mark area and fuzzy texture characteristics; the fusion image obtained by LAP in the image d retains more CT image information, but some details of the MR image are lost, so that the texture features and edge details of the lesion site are partially lost; the fused image obtained by NSST-PCNN has higher brightness, and the texture characteristics and edge details of the source image are weakened; the fusion result obtained by the CNN method has very low contrast on the whole, and the texture characteristics and edge details of the brain tumor part are not obvious enough; in terms of visual impact, GFF performs similarly to the methods presented herein, but the lesion area is somewhat smooth, the texture features are not sufficient, and the edge details are not clear enough. As can be seen from the quantization indices in Table 3, the comparison method is applied to MI index and Q indexAB/FThe indexes are all low, which indicates that the texture features and the edge details of the film have defects and are consistent with subjective vision. The method provided by the invention is superior to the comparison method in the aspect of keeping texture characteristics and edge details.

Claims (6)

1. The multi-modal medical image fusion based on the multi-CNN combination and the fuzzy neural network is characterized by comprising the following steps:
step 1, training data is obtainedAfter the CT and MR images are filtered by a filter bank consisting of 16 Gabor filters of different proportions and orientations, 16 different Gabor representation pairs of CT and MR are obtained:
Figure FDA0002899099790000015
and
Figure FDA0002899099790000016
1.., 16, then each pair of Gabor is used to represent a pair:
Figure FDA0002899099790000017
and
Figure FDA0002899099790000018
1.. 16 to train the respective 16G-CNNs, the 16G-CNNs consisting of G-CNNs;
and 2, putting each pair of Gabor representation pairs into corresponding G-CNNs for preliminary fusion, and then fusing a plurality of outputs of the G-CNNs by using a fuzzy neural network to obtain a final fused image.
2. The multimodal medical image fusion based on multi-CNN combination and fuzzy neural network as claimed in claim 1, wherein the filter bank in step 1 is a set of two-dimensional Gabor filters, and its formula is:
Figure FDA0002899099790000011
in the formula (1), U represents the direction of the filter bank and is selected to be 0 degrees, 90 degrees, 180 degrees and 270 degrees; v denotes the filter bank dimensions set to 4, 8, 16 and 32; z ═ x, y denotes the position of the pixel,
Figure FDA0002899099790000012
kv＝kmax/fv，φu＝πu/8，kmaxf is the spacing between filters in the frequency domain, at maximum frequencyA factor.
3. The multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network as claimed in claim 2, wherein the step 1 obtains 16 different Gabor representation pairs of CT and MR:
Figure FDA0002899099790000013
and
Figure FDA0002899099790000014
gabor representations of CT and MR at specific orientation u and v scales are obtained in particular by the following formula:
GaborCT (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (2)
GaborMR (u,v)(z)＝I(z)*ψu,v(z)for 0≤u≤U-1,0≤v≤V-1 (3)
formula (2) in formula (3), I (z) represents a given image, #u,v(z) denotes a filter, denotes a convolution operation, GaborCT (u,v)(z) and GaborMR (u,v)(z) is the Gabor representation of CT and MR for a given image.
4. The multimodal medical image fusion based on multi-CNN combination and fuzzy neural network of claim 1, wherein the G-CNN is composed of three parts of feature extraction, feature fusion and image reconstruction;
the characteristic extraction: 3X 224 GaborCTAnd GaborMRInputting an image into a G-CNN, extracting depth features from an input Gabor-based representation source image by adopting two convolutional layers, and adopting a first convolutional layer of an advanced ResNet101 pre-trained on ImageNet as a first convolutional layer CONV1, wherein the CONV1 comprises 64 convolutional kernels with the size of 7 multiplied by 7, and the step size and the filling parameter are respectively set to be 1 and 3; adding a second convolutional layer CONV2, setting the number of cores and the core size of CONV2 to 64 and 3 × 3, respectively, setting the step size and the fill parameter of CONV2 to 1, setting the ReLU activation function and the batch normalization function of CONV2, and passing through twoConvolution operation of the convolution layer to obtain 64 Gabor of 224 × 224CTAnd GaborMRA feature map of the image;
the characteristics are fused: in the fusion process, the convolution characteristics of multiple inputs are fused by using an element-level fusion rule, as shown in formula (4):
Figure FDA0002899099790000021
in the formula (4), N represents the number of input images, N is not less than 2,
Figure FDA0002899099790000022
a feature map representing the i-th input image extracted by CONV2,
Figure FDA0002899099790000023
representing a fusion feature graph generated by a feature fusion module, wherein fuse represents an element level fusion rule;
and (3) image reconstruction: after feature fusion, reconstructing a fused image from the fused convolution features by adopting two convolution layers CONV3 and CONV4, wherein the number of kernels and the size of the kernels of CONV3 are respectively set to be 64 and 3 multiplied by 3, and a ReLU activation function and a batch processing normalization function which are the same as those of CONV2 are also set; the CONV4 reconstructs and outputs the feature mapping through element weighted average, the kernel number and the kernel size of the CONV4 are set to be 3 and 1 multiplied by 1 respectively, and the step size and the filling parameter are set to be 0; after convolution operation of two convolution layers through image reconstruction, 3 × 224 × 224 Gabor is obtainedCTAnd GaborMRThe fused image of (1).
5. The multimodal medical image fusion based on multi-CNN combination and fuzzy neural network as claimed in claim 4, wherein the element level fusion rule in step 2.2 is element maximum fusion rule.
6. The multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network of claim 1, wherein the fuzzy neural network in step 2 is composed of an input layer, a fuzzy partition layer, a forward combination layer, an inference layer and an output layer;
in the first input layer, sixteen neurons x1，x2，...，x16Is a specific pixel value represented by the same position of 16 Gabor fusion images obtained by G-CNNs;
in the second-layer blur division layer, pixels are divided into five blur sets according to pixel values: very dark, normal, light and very light, and the membership function is used to represent the fuzzy set and is expressed as a gaussian function, as shown in equation (8):
Figure FDA0002899099790000031
in the formula (8), c and σ are the center and width of a gaussian function, i is 1, 2,. 16, j is 1, 2,. 5, different fuzzy sets correspond to different membership functions, five different membership functions sequentially correspond to 'very dark', 'normal', 'bright' and 'very bright', for a specific pixel value, the five membership functions have different output μ values, and the fuzzy set corresponding to the maximum μ value is selected as the fuzzy set of the pixel value;
arranging nodes of the same fuzzy set together in a third forward combination layer;
in the fourth layer of inference layer, the nodes are formed by connecting sixteen nodes representing the same fuzzy set in the third layer, and five fuzzy rules are set according to the divided five fuzzy sets: when x is1，x2，...x16The dark and bright pixel values are all very dark, and the output rule is
Figure FDA0002899099790000032
When x is1，x2，...x16The dark and bright pixel values belong to the dark degree, and the output rule is
Figure FDA0002899099790000041
When x is1，x2，...x16The pixel values are all in normal degree, and the output rule is
Figure FDA0002899099790000042
When x is1，x2，...x16The dark and bright pixel values belong to the brightness degree, and the output rule is
Figure FDA0002899099790000043
When x is1，x2，...x16The dark and bright pixel values belong to the very bright degree, and the output rule is
Figure FDA0002899099790000044
In the fifth output layer, the neuron represents the pixel value of the fused image based on the same position as the input image, and in order to obtain the fused image, the connection weighting factor v between the inference layer and the output layer is set as follows:
Figure FDA0002899099790000045
in the formula (9), p0、p1...p16Is a weighting factor, l ═ 1, 2, 3,. 5;
the pixel values of 16 Gabor fused images were calculated by the deblurring function as follows:
Figure FDA0002899099790000046
in the formula (10), L is 5, RlIs the rule output value.
CN202110050993.3A 2021-01-14 2021-01-14 Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network Active CN112750097B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110050993.3A CN112750097B (en) 2021-01-14 2021-01-14 Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110050993.3A CN112750097B (en) 2021-01-14 2021-01-14 Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network

Publications (2)

Publication Number Publication Date
CN112750097A true CN112750097A (en) 2021-05-04
CN112750097B CN112750097B (en) 2022-04-05

Family

ID=75652065

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110050993.3A Active CN112750097B (en) 2021-01-14 2021-01-14 Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network

Country Status (1)

Country Link
CN (1) CN112750097B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114974518A (en) * 2022-04-15 2022-08-30 浙江大学 Multi-mode data fusion lung nodule image recognition method and device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529667A (en) * 2016-09-23 2017-03-22 中国石油大学(华东) Logging facies identification and analysis method based on fuzzy depth learning in big data environment
CN106971174A (en) * 2017-04-24 2017-07-21 华南理工大学 A kind of CNN models, CNN training methods and the vein identification method based on CNN
CN108714026A (en) * 2018-03-27 2018-10-30 杭州电子科技大学 The fine granularity electrocardiosignal sorting technique merged based on depth convolutional neural networks and on-line decision
CN109544492A (en) * 2018-10-25 2019-03-29 东南大学 A kind of multi-focus image fusion data set production method based on convolutional neural networks
CN109801250A (en) * 2019-01-10 2019-05-24 云南大学 Infrared and visible light image fusion method based on ADC-SCM and low-rank matrix expression
CN109919960A (en) * 2019-02-22 2019-06-21 西安工程大学 A kind of image continuous boundary detection method based on Multiscale Gabor Filters device
CN110555458A (en) * 2019-07-24 2019-12-10 中北大学 Multi-band image feature level fusion method for generating countermeasure network based on attention mechanism
US20200134821A1 (en) * 2018-10-26 2020-04-30 National Cheng Kung University Convolutional neural network and associated method for identifying basal cell carcinoma
CN111882514A (en) * 2020-07-27 2020-11-03 中北大学 Multi-modal medical image fusion method based on double-residual ultra-dense network

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529667A (en) * 2016-09-23 2017-03-22 中国石油大学(华东) Logging facies identification and analysis method based on fuzzy depth learning in big data environment
CN106971174A (en) * 2017-04-24 2017-07-21 华南理工大学 A kind of CNN models, CNN training methods and the vein identification method based on CNN
CN108714026A (en) * 2018-03-27 2018-10-30 杭州电子科技大学 The fine granularity electrocardiosignal sorting technique merged based on depth convolutional neural networks and on-line decision
CN109544492A (en) * 2018-10-25 2019-03-29 东南大学 A kind of multi-focus image fusion data set production method based on convolutional neural networks
US20200134821A1 (en) * 2018-10-26 2020-04-30 National Cheng Kung University Convolutional neural network and associated method for identifying basal cell carcinoma
CN109801250A (en) * 2019-01-10 2019-05-24 云南大学 Infrared and visible light image fusion method based on ADC-SCM and low-rank matrix expression
CN109919960A (en) * 2019-02-22 2019-06-21 西安工程大学 A kind of image continuous boundary detection method based on Multiscale Gabor Filters device
CN110555458A (en) * 2019-07-24 2019-12-10 中北大学 Multi-band image feature level fusion method for generating countermeasure network based on attention mechanism
CN111882514A (en) * 2020-07-27 2020-11-03 中北大学 Multi-modal medical image fusion method based on double-residual ultra-dense network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
SUBHASH RAJPUROHIT等: "Identification of Acute Lymphoblastic Leukemia in Microscopic Blood Image Using Image Processing and Machine Learning Algorithms", 《网页在线公开：HTTPS://IEEEXPLORE.IEEE.ORG/STAMP/STAMP.JSP?TP=&ARNUMBER=8554576》 *
王丽芳等: "基于联合图像块聚类自适应字典学习的多模态医学图像融合", 《计算机科学》 *
马淑兰等: "基于四元数小波变换和自适应神经网络的图像融合处理", 《西安石油大学学报（自然科学版）》 *

Also Published As

Publication number Publication date
CN112750097B (en) 2022-04-05

Similar Documents

Publication Publication Date Title
Yi et al. Generative adversarial network in medical imaging: A review
YİĞİT et al. Applying deep learning models to structural MRI for stage prediction of Alzheimer's disease
CN111882514B (en) Multi-mode medical image fusion method based on double-residual ultra-dense network
CN109934887B (en) Medical image fusion method based on improved pulse coupling neural network
CN110444277B (en) Multi-mode brain MRI image bidirectional conversion method based on multi-generation and multi-confrontation
WO2022121100A1 (en) Darts network-based multi-modal medical image fusion method
Wang et al. Multimodal medical image fusion based on Gabor representation combination of multi-CNN and fuzzy neural network
Liu et al. MSDF-Net: Multi-scale deep fusion network for stroke lesion segmentation
CN110660063A (en) Multi-image fused tumor three-dimensional position accurate positioning system
Liu et al. Robust spiking cortical model and total-variational decomposition for multimodal medical image fusion
CN107292858A (en) A kind of multimode medical image fusion method based on low-rank decomposition and rarefaction representation
Liang et al. Alzheimer’s disease classification using 2d convolutional neural networks
Jiang et al. CT image super resolution based on improved SRGAN
CN113436128B (en) Dual-discriminator multi-mode MR image fusion method, system and terminal
CN112750097B (en) Multi-modal medical image fusion based on multi-CNN combination and fuzzy neural network
Yang et al. Retinal image enhancement with artifact reduction and structure retention
Panda et al. A 3D wide residual network with perceptual loss for brain MRI image denoising
Hao et al. Magnetic resonance image segmentation based on multi-scale convolutional neural network
CN112562058B (en) Method for quickly establishing intracranial vascular simulation three-dimensional model based on transfer learning
CN116630272A (en) Cerebral hemorrhage pseudo-health image generation method based on pathological decoupling
Yang et al. Hierarchical progressive network for multimodal medical image fusion in healthcare systems
CN114926383A (en) Medical image fusion method based on detail enhancement decomposition model
Zhang et al. Medical image fusion based a densely connected convolutional networks
Tajammal et al. Deep Learning-Based Ensembling Technique to Classify Alzheimer's Disease Stages Using Functional MRI
Nguyen et al. Comparative study on super resolution techniques for upper gastrointestinal endoscopic images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant