CN110322402A - Medical image super resolution ratio reconstruction method based on dense mixing attention network - Google Patents

Medical image super resolution ratio reconstruction method based on dense mixing attention network Download PDF

Info

Publication number
CN110322402A
CN110322402A CN201910361678.5A CN201910361678A CN110322402A CN 110322402 A CN110322402 A CN 110322402A CN 201910361678 A CN201910361678 A CN 201910361678A CN 110322402 A CN110322402 A CN 110322402A
Authority
CN
China
Prior art keywords
network
resolution
medical image
super
resolution ratio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910361678.5A
Other languages
Chinese (zh)
Other versions
CN110322402B (en
Inventor
刘可文
马圆
熊红霞
刘朝阳
房攀攀
李小军
陈亚雷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University of Technology WUT
Original Assignee
Wuhan University of Technology WUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University of Technology WUT filed Critical Wuhan University of Technology WUT
Priority to CN201910361678.5A priority Critical patent/CN110322402B/en
Publication of CN110322402A publication Critical patent/CN110322402A/en
Application granted granted Critical
Publication of CN110322402B publication Critical patent/CN110322402B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4023—Decimation- or insertion-based scaling, e.g. pixel or line decimation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
    • G06T3/4076—Super resolution, i.e. output image resolution higher than sensor resolution by iteratively correcting the provisional high resolution image using the original low-resolution image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/22—Cropping
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The invention discloses a kind of medical image super resolution ratio reconstruction methods based on dense mixing attention network.The present invention introduces mixing attention mechanism on the basis of dense neural network, and neural network is made to focus more on channel and region containing abundant high-frequency information, the precision for accelerating network convergence, further promoting super-resolution.It mainly comprises the steps that and designs and build based on dense neural network and the network for mixing attention mechanism；Preprocessed data collection, data enhancing, constructs training sample；Training network model is lost using L2 until network model reaches convergence；In the super-resolution rebuilding stage, the medical image of low resolution is inputted, goes out final high-definition picture using trained network model Super-resolution Reconstruction.Method of the present invention is higher compared to the super-resolution method precision of mainstream, is a kind of effective medical image super resolution ratio reconstruction method.

Description

Medical image super resolution ratio reconstruction method based on dense mixing attention network
Technical field
The present invention relates to medical image super-resolution rebuilding technologies, and in particular to one kind is based on dense mixing attention network Medical image super resolution ratio reconstruction method, belong to digital image processing field.
Background technique
Medical image is widely used in clinical diagnosis and treatment, but during obtaining medical image, due to hardware Limitation, environment influence, and missing high-frequency information causes medical image resolution ratio low, fuzzy.From hardware point of view improve the above problem by The limitation of manufacturing process and cost is improved from software respective, using image super-resolution method to the medical image of low resolution Corresponding high-definition picture can efficiently be obtained by carrying out super-resolution rebuilding.
Present image super-resolution method mainly has three classes, is based on interpolation, the side based on modeling and based on study respectively Method, the method based on study can be divided into the method based on rarefaction representation and the method based on convolutional neural networks.Based on interpolation Method have the characteristics that computational efficiency is high, but be easily lost high frequency texture detailed information.Model-based method is using first Information constrained solution space is tested, effect has a certain upgrade compared to the method based on interpolation, but when input image size is smaller, energy The prior information of effective use is less, and super-resolution efect is poor.Method based on study is by learning low, high resolution graphics The internal relation as between realizes super-resolution.In recent years, the super-resolution method based on convolutional neural networks achieves higher Precision.However, the convolution kernel of convolutional neural networks coequally treats each channel and region of characteristic pattern, reduces network and contain The channel of abundant high-frequency information and the feature representation ability in region.In addition to this, conventional convolution neural network is in propagated forward Recall info can be lost, the thought of dense neural network can be introduced, a large amount of jump is added and links structure, repeatedly used features, further Promote network performance.In conclusion the performance of the image super-resolution method based on study still has the space of promotion.
Summary of the invention
The technical problems to be solved by the present invention are: during obtaining medical image, due to hardware limitation, environment shadow It rings, lacks caused by high-frequency information that medical image resolution ratio is low, fuzzy problem.
The present invention solves its technical problem, and the following technical solution is employed:
Medical image super resolution ratio reconstruction method provided by the invention is a kind of doctor based on dense mixing attention network Image super-resolution rebuilding method is learned, this method is to introduce mixing attention mechanism on the basis of dense neural network, addition Mixing attention mechanism unit makes neural network focus more on channel and region containing abundant high-frequency information, increases the spy of network Ability to express is levied, the precision for accelerating network convergence, further promoting super-resolution.
The medical image super resolution ratio reconstruction method, comprising the following steps:
Step 1: designing and builds based on dense neural network and the network for mixing attention mechanism；
Step 2: pre-processing input picture, and data enhancing constructs training sample；
Step 3: training network model is lost using L2 until network model reaches convergence；
Step 4: in the super-resolution rebuilding stage, the medical image of low resolution is inputted, utilizes trained network model Super-resolution Reconstruction goes out final high-definition picture.
The mixing attention mechanism refers to that network indicates with Enhanced feature while paying close attention to have abundant high-frequency information Channel and region ability, have 2 cascade convolutional layers, active coatings in the mixing attention mechanism unit.
The dense neural network includes N (N >=8) a basic unit, there is N (N >=8) a cascade in each basic unit Convolutional layer, active coating, one mixing attention mechanism unit of last cascade of each basic unit；In each basic unit with Between basic unit, a large amount of dense jumps of addition link structure, extract deeper character representation.
In the above method, dense neural network can divide five stages, respectively feature extraction, feature Nonlinear Mapping, Feature Dimension Reduction, deconvolution up-sampling, convolution obtain final output, in which: feature extraction phases are swashed using cascade convolution sum Layer living, feature Nonlinear Mapping stage use bottleneck layer using dense neural network described in the above method, Feature Dimension Reduction stage Dimensionality reduction is up-sampled using deconvolution, and convolution obtains final output.
In the above method, pretreatment described in step 2, data enhancing, are to cut to input picture, to cutting The subgraph arrived carries out down-sampling operation, obtains corresponding low-resolution image, obtains more training samples using data enhancing.
In above method step 3, using the loss L based on L2 norm2The obtained high-definition picture of quantization super-resolution and The similarity degree of true high-definition picture, training process are learnt using small lot, the expression formula of the loss function of use are as follows:
In formula: IHRFor true high-definition picture；ISRTo execute the high-definition picture that super-resolution obtains；H,W,C The respectively size (length and width) and port number of input picture, n are the number of small lot study, and v is v spies in small lot n Sign figure, k are k-th of channel of v characteristic patterns, and (i, j) is characterized the coordinate position in figure, Iv,i,j,kFor v characteristic patterns Kth channel position be (i, j) pixel value.
In above method step 4, the medical image that the super-resolution rebuilding obtains is amplified by low-resolution imageIt obtains again,
Methods and techniques process proposed by the present invention extends realized software systems including method.
More convolution, activation can be arranged in the method for the present invention by the more basic units of setting or in basic unit Layer, further increases super-resolution precision.It is all this method propose network foundation on be arranged more basic units or The more convolution of setting, active coating in basic unit, and then the method for improving super-resolution precision is all inconsistent with the present invention 's.
The experimental results showed that (specific experiment data are referring to the description in specific implementation method), the method for proposition compares mainstream Image super-resolution method in Y-PSNR (Peak Signal-to-Noise Ratio, PSNR) and structural similarity There are the promotion of 0.146db-5.874dB and 0.1%-7.66% on (Structural similarity, SSIM) respectively.
The present invention has the advantages that following main compared with prior art:
Mixing attention mechanism is introduced, a kind of new mixing attention mechanism unit is proposed, adds in the network architecture The mechanism unit can accelerate network convergence, and Enhanced feature indicates ability, further promotes the performance of network, the mixing attention The structure design of mechanism unit has initiative；
Based on dense neural network, a large amount of dense jump connection structures of addition make full use of the spy of different phase, different scale Sign makes gradient information be transmitted directly to network front layer, improves conventional neural networks and loses recall info, gradient in propagated forward The problem of disappearance, network are degenerated, further promotes network performance；
The present invention constructs a kind of medical image super-resolution side based on mixing attention mechanism and dense neural network Method improves network performance compared to the method for mainstream, and mixing attention mechanism is merged with dense neural network with pioneering Property.
Detailed description of the invention
Fig. 1 is the structure chart of mixing attention mechanism unit proposed by the present invention.
Fig. 2 is the basic cell structure figure proposed by the present invention based on dense mixing attention network.
Fig. 3 is that the present invention is based on the network structures of the medical image super resolution ratio reconstruction method of dense mixing attention network Figure.
Fig. 4 is the effect for the medical image super-resolution rebuilding that the present invention obtains again with other three kinds of method super-resolutions 2 Comparison diagram.
Specific embodiment
It is disclosed by the invention it is a kind of based on it is dense mixing attention network medical image super resolution ratio reconstruction method, be Mixing attention mechanism is introduced on the basis of dense neural network, focuses more on neural network containing abundant high-frequency information Channel and region, the precision for accelerating network convergence, further promoting super-resolution.It mainly comprises the steps that and designs and build Based on dense neural network and the network for mixing attention mechanism；Preprocessed data collection, data enhancing, constructs training sample；Make Training network model is lost with L2 until network model reaches convergence；In the super-resolution rebuilding stage, the doctor of low resolution is inputted Image is learned, goes out final high-definition picture using trained network model Super-resolution Reconstruction.Method phase of the present invention It is higher than the super-resolution method precision of mainstream, it is a kind of effective medical image super resolution ratio reconstruction method.
Below with reference to embodiment and attached drawing, the invention will be further described, but does not limit the present invention.
Medical image super resolution ratio reconstruction method provided by the invention based on dense mixing attention network, is dense The basis of neural network introduces mixing attention mechanism, and the mixing attention mechanism unit of addition focuses more on neural network to contain There are the channel and region of abundant high-frequency information, increase the feature representation ability of network, accelerates network convergence, further promotes oversubscription The precision of resolution.
The medical image super resolution ratio reconstruction method of above-mentioned dense mixing attention network, comprising the following steps:
Step 1: designing and builds based on dense neural network and the network for mixing attention mechanism；
Step 2: pre-processing input picture, and data enhancing constructs training sample；
Step 3: training network model is lost using L2 until network model reaches convergence；
Step 4: in the super-resolution rebuilding stage, the medical image of low resolution is inputted, utilizes trained network model Super-resolution Reconstruction goes out final high-definition picture.
As shown in Figure 1, mixing attention mechanism unit described in step 1 is by the mixing attention mechanism unit There are 2 cascade convolutional layers, active coatings.The image that attention mechanism unit dimension is H*W*C is inputted, wherein H, W are image Long and wide, C is port number, and process cascade convolution, activation twice obtain dimension as the descriptor τ of H*W*C:
τ=f (W2δ(W1x)),τ∈RC,
In formula: x is input, W1For the parameter of first layer convolution, first layer convolution executes the characteristic pattern channel that the factor is 16 Number dimensionality reduction, obtains the characteristic pattern that dimension is H*W*C/16, and δ (g) is RELU activation operation, W2For the parameter of second layer convolution, Secondary convolution executes several litres of the characteristic pattern channel dimension that the factor is 16, and f (g) is sigmoid activation operation.Convolution, activation pair twice Channel dimension carries out port number dimensionality reduction and several litres of channel dimension, the Description Matrix τ in the different channels of C correspondence of studyi, wherein i=0, 1,2...C, adaptive is assigned to the channel containing bulk redundancy low-frequency information for more sparse Description Matrix, so that neural network Focus more on the channel containing abundant high-frequency information.Each Description Matrix τiSize be H*W, corresponding original input image i-th is logical Each element in road.By convolution twice, activation, the region that original input image contains abundant high-frequency information is retained, is contained The region of bulk redundancy low-frequency information is suppressed, the descriptor τ that will be obtainediHadamard product is carried out with the i-th channel of original input, So that neural network focuses more on the region containing abundant high-frequency information in the i-th channel.By obtained descriptor and input picture Hadamard is multiplied, and obtains the characteristic pattern by mixing attention mechanism unit.
The convolutional layer is the basic unit of convolutional neural networks, for extracting the different characteristic of input picture, first layer Convolutional layer can only extract some rudimentary features such as levels such as edge, lines and angle, and the network of deeper can be from low-level features The more complicated feature of iterative extraction.
The active coating is the basic unit of convolutional neural networks, for enhancing the non-of decision function and entire neural network Linear characteristic, itself can't change convolutional layer, and common activation primitive has Sigmoid (S) function, RELU line rectification function Deng.
In Fig. 3, design described in step 1 and build based on dense neural network and the network for mixing attention mechanism It is divided into five stages, respectively feature extraction, feature Nonlinear Mapping, Feature Dimension Reduction, deconvolution up-sampling, convolution obtains finally Output:
Stage one: feature extraction phases refer to the image contract data from input, construct the characteristic information of nonredundancy, are used for Promote it is subsequent study with it is extensive, the feature of extraction in some cases can better than mankind's manual extraction feature.As shown in figure 3, The present invention completes feature extraction, convolutional layer using cascade convolutional layer and RELU active coating after low-resolution image input Parameter is set as 128 × 3 × 3 × 3, i.e. 128 sizes are the convolution kernel that 3 port numbers are 3.
Stage two: the feature Nonlinear Mapping stage refers to using the operator for being unsatisfactory for linear conditions, for completing vector sky Between mapping (including the abstract vector space being made of function).As shown in figure 3, the present invention uses the dense mind in the middle part of network Nonlinear Mapping is completed through network, the dense neural network used includes n basic unit, and each basic unit there are n Cascade convolution, active coating, one mixing attention block of last cascade of basic unit, the parameter of convolutional layer is set in basic unit Being set to 16 × 3 × 3, i.e. the convolution kernel that 16 sizes are 3 × 3, activation primitive uses RELU function, and the step-length of convolution operation is 1, It is operated at edge using zero padding (zero padding), keeps the size of characteristic pattern consistent.Due to being added to a large amount of dense jumps Link structure, different phase, different scale characteristic pattern be aggregating, with the intensification of the network number of plies, input the logical of convolution kernel Road number is linearly increasing, and the port number of each convolution kernel of first layer convolution is 16 in each basic unit, later the volume of every layer of convolution The port number of product core increases by 16 on the basis of upper one layer.
Stage three: the Feature Dimension Reduction stage, which refers to from the characteristic information comprising bulk redundancy or extraneous features, finds out main feature Information, for reducing computation complexity.As shown in figure 3, the present invention uses bottleneck layer dimensionality reduction, bottleneck layer is one layer of convolutional layer, is used In completing the mapping of higher dimensional space to lower dimensional space, the convolution kernel parameter of bottleneck layer is set as 256 × 1152 × 1 × 1, i.e., 256 A size is the convolution kernel that 1 port number is 1152, and the step-length of convolution operation is 1.
Stage four: the convolution kernel parameter of deconvolution up-sampling is set as 256 × 256 × 2 × 2, and the step-length of convolution operation is 2, it is operated at edge using zero padding (zero padding).
Stage five: convolution obtains final output, and the parameter of convolution kernel is set as 3 × 256 × 3 × 3, the step of convolution operation A length of 1, it is operated at edge using zero padding (zero padding).
Pretreatment described in step 2, data enhancing, specific embodiment are to cut to input picture, are cut to big The small subgraph for being 96 × 96 carries out bicubic down-sampling behaviour to the subgraph that cutting obtains using the imresize function of Matlab Make, obtain the low-resolution image that corresponding size is 48 × 48, using data enhancing such as rotation, mirror image, obtains more Training sample.
Experiment L2 described in step 3 loses training network model until network model is restrained, and specific embodiment is to use Loss L based on L2 norm2The similarity degree of high-definition picture and true high-definition picture that quantization super-resolution obtains, instruction Practice training process to learn using small lot (mini-batch), the number of small lot study is set as 16.The loss function of use Expression formula are as follows:
In formula: IHRFor true high-definition picture；ISRTo execute the high-definition picture that super-resolution obtains；H,W,C The respectively size (length and width) and port number of input picture, n are the number of small lot study, and v is v spies in small lot n Sign figure, k are k-th of channel of v characteristic patterns, and (i, j) is characterized the coordinate position in figure, Iv,i,j,kImage is opened for v The position in kth channel is the pixel value of (i, j).
Super-resolution rebuilding described in step 4, specific embodiment are that the medical image of low resolution is inputted network, The high-definition picture exported, the medical image that the super-resolution rebuilding obtains are equivalent to the low-resolution image Amplification It obtains again.
By experimental verification, dense mixing attention network of the invention can sufficiently be multiplexed different phase, different scale Feature, further promote network performance；Mixing attention mechanism can be such that neural network focuses more on containing abundant high frequency letter The channel and region of breath inhibit channel and region containing bulk redundancy information, accelerate network convergence, further promote internetworking Energy.
In order to prove effectiveness of the invention, 400 are picked out clearly from the public data at American National lung cancer center concentration The CT image that clear degree is high, details size abundant is 512 × 512 picks out 100 images as test set as training set. Described in summary of the invention step 2, input picture is cut, the subgraph that size is 96 × 96 is cut to, uses Matlab's Imresize function carries out the operation of bicubic down-sampling to the subgraph that cutting obtains, and obtains low point that corresponding size is 48 × 48 Resolution image obtains more training samples using data enhancing such as rotation, mirror image.
In experiment, chooses bicubic interpolation method and two kinds of representative methods based on convolutional neural networks and carry out pair Than.For the fairness for guaranteeing comparison, each method is tested under identical hardware environment.
The two kinds of representative methods based on convolutional neural networks chosen are as follows:
The method that method 1:Kim et al. is proposed, bibliography are as follows: Kim J, Kwon Lee J, Mu Lee K.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016:1646- 1654.
The method that method 2:Tong et al. is proposed, bibliography are as follows: Tong Tong, Gao Qinquan one kind are based on intensive connection network The Fujian image super-resolution method [P]: CN106991646A, 2017-07-28.
The hardware environment parameter setting of experiment:
Table 1
The evaluation index of selection:
The objective indicator for being widely used in evaluation image super-resolution effect has Y-PSNR (Peak Signal-to- Noise Ratio, PSNR) and structural similarity (Structural similarity, SSIM), the present invention select PSNR and SSIM is as the index objectively evaluated.In addition to this, time also conduct needed for the present invention will complete single image super-resolution Reference objectively evaluates one of index.
Each super-resolution method objectively evaluates comparison:
Table 2
It can be seen that the present invention in Y-PSNR (PSNR) and structural similarity from the experimental data of 2 each method of table (SSIM) have the promotion of 5.874dB and 7.66% respectively compared to bicubic interpolation method, compared to method 1 have respectively 0.259dB and 0.37% promotion, compared to the promotion that method 2 has 0.146dB and 0.1%.The time-consuming aspect of single-frame images super-resolution, bicubic Interpolation method is most fast, and method 1, method 2, time-consuming of the invention are above Bicubic method, but in 0.5s, real-time is preferable.
The effect for each super-resolution method that Fig. 4 is shown has chosen four groups of grain details than more rich image, shows each The effect of super-resolution method is the CT image of aorta, apex pulmonis portion, lung, the lobe of the lung respectively.The high-resolution that each method obtains The comparison of image and true high-definition picture (Ground Truth, GT) as indicated, objectively evaluate index value mark accordingly Below image.Compared to control methods, the image of super-resolution rebuilding of the present invention has preferable image sharpness, visual perception effect Fruit is best, and image clearly, details be true, brightness uniformity, and the image that super-resolution obtains is closest to true picture.It is commented in conjunction with objective Valence and subjective evaluation result, the present invention are a kind of effective medical image super resolution ratio reconstruction methods.
The medical image that the present invention is applicable in includes but is not limited to CT image, nuclear magnetic resonance (MRI) image, X-ray (X-ray) figure Picture and Positron emission computed tomography (PET) image.
The present invention utilizes the advantages of dense neural network, sufficiently multiplexing different phase, the feature of different scale, further mentions Network performance is risen, parameter amount is reduced, improves gradient and disappears and network degenerate problem；It the advantages of using mixing attention mechanism, mentions A kind of new mixing attention mechanism unit is gone out, has added the mechanism unit in the network architecture and focus more on neural network and contain There are the channel and region of abundant high-frequency information, accelerates network convergence, further promotes network performance；In conjunction with dense neural network and Attention mechanism is mixed, a kind of effective medical image super-resolution reconstruction method is constructed.
The content being not described in detail in this specification belongs to the prior art well known to professional and technical personnel in the field.

Claims (10)

1. a kind of medical image super resolution ratio reconstruction method, it is characterized in that a kind of medicine figure based on dense mixing attention network As super resolution ratio reconstruction method, this method is to introduce mixing attention mechanism, the mixing of addition on the basis of dense neural network Attention mechanism unit makes neural network focus more on channel and region containing abundant high-frequency information, increases the mark sheet of network Danone power, the precision for accelerating network convergence, further promoting super-resolution.
2. medical image super resolution ratio reconstruction method according to claim 1, it is characterised in that the following steps are included:
Step 1: designing and builds based on dense neural network and the network for mixing attention mechanism；
Step 2: pre-processing input picture, and data enhancing constructs training sample；
Step 3: training network model is lost using L2 until network model reaches convergence；
Step 4: in the super-resolution rebuilding stage, inputting the medical image of low resolution, utilizes trained network model oversubscription It distinguishes and reconstructs final high-definition picture.
3. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that the mixing attention Mechanism refers to that network indicates with Enhanced feature while paying close attention to the ability in the channel and region with abundant high-frequency information, this is mixed Closing in attention mechanism unit has 2 cascade convolutional layers, active coatings.
4. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that: the dense nerve net Network includes N number of basic unit, there is N number of cascade convolutional layer, active coating in each basic unit, N >=8, each basic unit Finally cascade a mixing attention mechanism unit；In each basic unit between basic unit, a large amount of dense jumps of addition Link structure, extracts deeper character representation.
5. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that dense neural network point five A stage, respectively feature extraction, feature Nonlinear Mapping, Feature Dimension Reduction, deconvolution up-sampling, convolution obtains final defeated Out, in which: feature extraction phases use cascade convolution sum active coating, and the feature Nonlinear Mapping stage uses claim 4 institute The dense neural network stated, Feature Dimension Reduction stage are used bottleneck layer dimensionality reduction, are up-sampled using deconvolution, and convolution obtains final defeated Out.
6. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that pre- place described in step 2 Reason, data enhancing, are cut to input picture, are carried out down-sampling operation to the subgraph that cutting obtains, are obtained corresponding low Image in different resolution obtains more training samples using data enhancing.
7. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that in step 3, using being based on The loss L of L2 norm2The similarity degree of high-definition picture and true high-definition picture that quantization super-resolution obtains, was trained The study of Cheng Caiyong small lot, the expression formula of the loss function of use are as follows:
In formula: IHRFor true high-definition picture；ISRTo execute the high-definition picture that super-resolution obtains；H, W, C difference For the size (length and width) and port number of input picture, n is the number of small lot study, and v is v features in small lot n Figure, k are k-th of channel of v characteristic patterns, and (i, j) is characterized the coordinate position in figure, Iv,i,j,kCharacteristic pattern is opened for v The position in kth channel is the pixel value of (i, j).
8. medical image super resolution ratio reconstruction method according to claim 2, it is characterised in that in step 4: described is super The medical image that resolution reconstruction obtains is amplified by low-resolution imageIt obtains again,
9. according to claim 1 to any medical image super resolution ratio reconstruction method in 8, it is characterised in that the side of proposition Method and technical process extend realized software systems including method.
10. according to claim 1 to any medical image super resolution ratio reconstruction method in 8, it is characterized in that passing through setting More convolution, active coating are arranged in more basic units in basic unit, and then improve super-resolution precision.
CN201910361678.5A 2019-04-30 2019-04-30 Medical image super-resolution reconstruction method based on dense mixed attention network Active CN110322402B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910361678.5A CN110322402B (en) 2019-04-30 2019-04-30 Medical image super-resolution reconstruction method based on dense mixed attention network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910361678.5A CN110322402B (en) 2019-04-30 2019-04-30 Medical image super-resolution reconstruction method based on dense mixed attention network

Publications (2)

Publication Number Publication Date
CN110322402A true CN110322402A (en) 2019-10-11
CN110322402B CN110322402B (en) 2023-07-25

Family

ID=68113374

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910361678.5A Active CN110322402B (en) 2019-04-30 2019-04-30 Medical image super-resolution reconstruction method based on dense mixed attention network

Country Status (1)

Country Link
CN (1) CN110322402B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445390A (en) * 2020-02-28 2020-07-24 天津大学 Wide residual attention-based three-dimensional medical image super-resolution reconstruction method
CN111461987A (en) * 2020-04-01 2020-07-28 中国科学院空天信息创新研究院 Network construction method, image super-resolution reconstruction method and system
CN111861880A (en) * 2020-06-05 2020-10-30 昆明理工大学 Image super-fusion method based on regional information enhancement and block self-attention
CN112529825A (en) * 2020-12-11 2021-03-19 平安科技（深圳）有限公司 Face image resolution reconstruction method, device and equipment and storage medium
CN113222823A (en) * 2021-06-02 2021-08-06 国网湖南省电力有限公司 Hyperspectral image super-resolution method based on mixed attention network fusion
WO2022217746A1 (en) * 2021-04-13 2022-10-20 湖南大学 High-resolution hyperspectral calculation imaging method and system, and medium
CN116071239A (en) * 2023-03-06 2023-05-05 之江实验室 CT image super-resolution method and device based on mixed attention model

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108710830A (en) * 2018-04-20 2018-10-26 浙江工商大学 A kind of intensive human body 3D posture estimation methods for connecting attention pyramid residual error network and equidistantly limiting of combination
CN108765290A (en) * 2018-05-29 2018-11-06 天津大学 A kind of super resolution ratio reconstruction method based on improved dense convolutional neural networks
CN108765291A (en) * 2018-05-29 2018-11-06 天津大学 Super resolution ratio reconstruction method based on dense neural network and two-parameter loss function
CN109035233A (en) * 2018-07-24 2018-12-18 西安邮电大学 Visual attention network and Surface Flaw Detection method
CN109584161A (en) * 2018-11-29 2019-04-05 四川大学 The Remote sensed image super-resolution reconstruction method of convolutional neural networks based on channel attention
CN109636721A (en) * 2018-11-29 2019-04-16 武汉大学 Video super-resolution method based on confrontation study and attention mechanism

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108710830A (en) * 2018-04-20 2018-10-26 浙江工商大学 A kind of intensive human body 3D posture estimation methods for connecting attention pyramid residual error network and equidistantly limiting of combination
CN108765290A (en) * 2018-05-29 2018-11-06 天津大学 A kind of super resolution ratio reconstruction method based on improved dense convolutional neural networks
CN108765291A (en) * 2018-05-29 2018-11-06 天津大学 Super resolution ratio reconstruction method based on dense neural network and two-parameter loss function
CN109035233A (en) * 2018-07-24 2018-12-18 西安邮电大学 Visual attention network and Surface Flaw Detection method
CN109584161A (en) * 2018-11-29 2019-04-05 四川大学 The Remote sensed image super-resolution reconstruction method of convolutional neural networks based on channel attention
CN109636721A (en) * 2018-11-29 2019-04-16 武汉大学 Video super-resolution method based on confrontation study and attention mechanism

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
HANG ZHAO 等: "Loss Functions for Image Restoration with Neural Networks", 《IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING》 *
HANG ZHAO 等: "Loss Functions for Image Restoration with Neural Networks", 《IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING》, 30 December 2016 (2016-12-30) *
JINGHUI CHU 等: "A Novel Multiconnected Convolutional Network for Super-Resolution", 《HTTPS://DOI.ORG/10.1109/LSP.2018.2820057》 *
JINGHUI CHU 等: "A Novel Multiconnected Convolutional Network for Super-Resolution", 《HTTPS://DOI.ORG/10.1109/LSP.2018.2820057》, 27 March 2018 (2018-03-27) *
王东飞;: "基于通道注意力的卷积神经网络在图像超分辨率重建中的应用", 广播与电视技术, no. 06 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445390A (en) * 2020-02-28 2020-07-24 天津大学 Wide residual attention-based three-dimensional medical image super-resolution reconstruction method
CN111445390B (en) * 2020-02-28 2022-03-25 天津大学 Wide residual attention-based three-dimensional medical image super-resolution reconstruction method
CN111461987A (en) * 2020-04-01 2020-07-28 中国科学院空天信息创新研究院 Network construction method, image super-resolution reconstruction method and system
CN111461987B (en) * 2020-04-01 2023-11-24 中国科学院空天信息创新研究院 Network construction method, image super-resolution reconstruction method and system
CN111861880A (en) * 2020-06-05 2020-10-30 昆明理工大学 Image super-fusion method based on regional information enhancement and block self-attention
CN111861880B (en) * 2020-06-05 2022-08-30 昆明理工大学 Image super-fusion method based on regional information enhancement and block self-attention
CN112529825A (en) * 2020-12-11 2021-03-19 平安科技（深圳）有限公司 Face image resolution reconstruction method, device and equipment and storage medium
WO2022121256A1 (en) * 2020-12-11 2022-06-16 平安科技（深圳）有限公司 Facial image resolution reconstruction method, apparatus and device, and storage medium
WO2022217746A1 (en) * 2021-04-13 2022-10-20 湖南大学 High-resolution hyperspectral calculation imaging method and system, and medium
CN113222823A (en) * 2021-06-02 2021-08-06 国网湖南省电力有限公司 Hyperspectral image super-resolution method based on mixed attention network fusion
CN116071239A (en) * 2023-03-06 2023-05-05 之江实验室 CT image super-resolution method and device based on mixed attention model

Also Published As

Publication number Publication date
CN110322402B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN110322402A (en) Medical image super resolution ratio reconstruction method based on dense mixing attention network
CN108460726B (en) Magnetic resonance image super-resolution reconstruction method based on enhanced recursive residual network
CN109064396A (en) A kind of single image super resolution ratio reconstruction method based on depth ingredient learning network
CN108830813A (en) A kind of image super-resolution Enhancement Method of knowledge based distillation
CN107492071A (en) Medical image processing method and equipment
CN109949255A (en) Image rebuilding method and equipment
CN107464216A (en) A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks
CN108416821B (en) A kind of CT Image Super-resolution Reconstruction method of deep neural network
Du et al. Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network
CN108805814A (en) Image Super-resolution Reconstruction method based on multiband depth convolutional neural networks
CN108647775A (en) Super-resolution image reconstruction method based on full convolutional neural networks single image
CN110706214B (en) Three-dimensional U-Net brain tumor segmentation method fusing condition randomness and residual error
CN111899165A (en) Multi-task image reconstruction convolution network model based on functional module
CN111487573B (en) Enhanced residual error cascade network model for magnetic resonance undersampling imaging
CN111598964B (en) Quantitative magnetic susceptibility image reconstruction method based on space adaptive network
CN111951164B (en) Image super-resolution reconstruction network structure and image reconstruction effect analysis method
CN106157249A (en) Based on the embedded single image super-resolution rebuilding algorithm of optical flow method and sparse neighborhood
Li et al. A multiscale double-branch residual attention network for anatomical–functional medical image fusion
CN114565816A (en) Multi-modal medical image fusion method based on global information fusion
CN111127487B (en) Real-time multi-tissue medical image segmentation method
CN109903225A (en) A kind of medical image Enhancement Method based on deep learning
Saeed et al. Contrastive pretraining for echocardiography segmentation with limited data
CN114821259A (en) Zero-learning medical image fusion method based on twin convolutional neural network
CN110084750A (en) Single image super-resolution method based on multilayer ridge regression
Lu et al. Fine-grained calibrated double-attention convolutional network for left ventricular segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant