CN107464216A - A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks - Google Patents
A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks Download PDFInfo
- Publication number
- CN107464216A CN107464216A CN201710655267.8A CN201710655267A CN107464216A CN 107464216 A CN107464216 A CN 107464216A CN 201710655267 A CN201710655267 A CN 201710655267A CN 107464216 A CN107464216 A CN 107464216A
- Authority
- CN
- China
- Prior art keywords
- image
- medical image
- resolution
- neural networks
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 32
- 238000013507 mapping Methods 0.000 claims abstract description 23
- 239000013598 vector Substances 0.000 claims description 31
- 230000004913 activation Effects 0.000 claims description 15
- 210000002569 neuron Anatomy 0.000 claims description 15
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 claims description 8
- 238000003384 imaging method Methods 0.000 claims description 8
- 239000000203 mixture Substances 0.000 claims description 4
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 238000012545 processing Methods 0.000 description 15
- 238000002474 experimental method Methods 0.000 description 6
- 230000003044 adaptive effect Effects 0.000 description 5
- 210000000988 bone and bone Anatomy 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 5
- 210000003928 nasal cavity Anatomy 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 230000000052 comparative effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001225 therapeutic effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks, comprises the following steps：A) by the low resolution medical image of acquisitionIt is input in convolutional neural networks；B) lead to and carry out first time convolution operation；C) operation of first time overlapping poolization is carried out；D) secondary convolution operation is carried out；E) secondary overlapping poolization operation is carried out；F) Nonlinear Mapping is carried out；G) convolution operation is carried out；H) articulamentum is established, reconstructs another piece image；I) image of reconstruct is overlapped, obtains high-definition picture.Finer feature extraction is carried out to medical image by the Further Feature Extraction layer in model, reduce the characteristic dimension of extraction by the overlapping pool layer in model, make the feature of extraction more representative, the loss of each layer convolution operation is made up by the articulamentum in model, finally gives more preferable super-resolution reconstruction effect.
Description
Technical field
The present invention relates to image resolution ratio reconfiguration technique field, and in particular to a kind of doctor based on multilayer convolutional neural networks
Learn image super-resolution reconstructing method.
Background technology
With the development of information, we are in a digital Age.Due to main way of the image as people's acquisition information
One of footpath, so application of the people to digital picture is also more and more therewith.Among many fields, we are to high image quality
Image has required.And high-definition picture is compared with normal image, then there are higher picture element density, finer and smoother image quality and more
More information, it can more meet the demand that we are applied to the information in image.The super-resolution rebuilding technology of image can dash forward
Break the physical limit of the intrinsic resolution of existing image-forming component, the low-resolution image second-rate from a width or multiframe imaging
Middle acquisition high-definition picture.
Medical image improves the imaging resolution of medical imaging device for disease as a kind of important means to diagnose the illness
The judgement of disease and the formulation of therapeutic scheme are extremely significant.Moreover, the medical image with higher resolution ratio exists for software
It is very helpful in automatic segmentation and reconstruction threedimensional model details.Medical imaging devices and in general image imaging device phase
Than there is several different parts.First, the radiant of equipment is highly limited, and reduces the time that patient is radiated.Second,
Imaging rate is fast.Shorten patient's discomfort time.3rd, equipment design object is to check and diagnose the illness.In imaging process
Many artifacts can be produced.
Image super-resolution reconstruct is broadly divided into four types：Based on interpolation, based on reconstruct, based on enhancing and based on study
Method.In recent years, due to machine learning and the development of deep learning, the image super-resolution method based on study is to develop into
For burning hoter research field, such method utilizes image data base or image in itself, passes through and learns to obtain high-resolution and low-resolution
Association between image, and utilize it as prior-constrained condition generation high-definition picture.Relative to other it is traditional based on
For the super-resolution method of study, the method based on deep learning has simple in construction, fireballing advantage, and due to instructing
Practice the stage, the method based on deep learning optimizes all operations simultaneously, so the method based on deep learning reconstruct come
High-definition picture is better than traditional method based on study in quality.But when handling medical image, existing depth
Degree study convolutional neural networks framework can not improve reconstructed image definition well, reduce the image when carrying out convolution operation and believe
The loss of breath.For medical image, there is this one kind the image of specific use to use more reasonable network structure to turn into influence reconstruct image
As an important factor for quality.
The content of the invention
The present invention is in order to overcome the shortcomings of above technology, there is provided a kind of Further Feature Extraction layer by model is to doctor
The characteristic dimension that image carries out finer feature extraction, reduces extraction by the overlapping pool layer in model is learned, makes extraction
Feature is more representative, made up by the articulamentum in model the loss of each layer convolution operation based on multilayer convolutional neural networks
Medical image ultra-resolution ratio reconstructing method.
Technical scheme is used by the present invention overcomes its technical problem：
A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks, comprises the following steps：
A) the low resolution medical image Y of acquisition is input in convolutional neural networks；
B) first time convolution operation is carried out to low resolution medical image Y by computer, carried from low resolution medical image Y
Partly overlapping block of pixels is taken, and each block of pixels is expressed as high dimensional feature vector, output differentiates medical image Y's to be low
All feature F1(Y)；
C) using computer to characteristic pattern F1(Y) operation of first time overlapping poolization is carried out, obtains characteristic image F1'(Y)；
D) using computer to characteristic image F1' the secondary convolution operation of (Y) progress, to characteristic image F1' (Y) carry out it is secondary
Feature extraction, the feature of second extraction is formed into characteristic image F2(Y)；
E) computer characteristic image F is utilized2(Y) secondary overlapping poolization operation is carried out, is reduced feature graph parameter and dimension
Spend the image F of feature2'(Y)；
F) to image F2' the low resolution feature in (Y) carries out Nonlinear Mapping, by the low resolution characteristic vector of acquisition
High-resolution features vector is mapped to, obtains high-resolution global characteristics mapping ensemblen F3(Y)；
G) using computer to global characteristics mapping ensemblen F3(Y) convolution operation, the image F'(Y reconstructed are carried out)；
H) image F is established using computer afterwards in step e)2' (Y) articulamentum, image F is utilized on articulamentum2'(Y)
Reconstruct another piece image F " (Y)；
I) by the image F'(Y of reconstruct in step g)) it is overlapped with F " (Y), obtain high-definition picture F (Y).Enter one
Step, feature extraction is carried out using the convolution kernel that 32 sizes are 9 × 9 in step b), convolution kernel moving step length is 1, computer
ReLU is selected to pass through formula F as activation primitive1(Y)=max (0, W1*Y+B1) calculate characteristic pattern F1(Y), wherein Y is low point
Distinguish medical image, W1For convolution kernel, B1The neuron bias vector for being 32 for dimension.
Further, overlapping pool layer is used in step c) to characteristic pattern F1(Y) carry out first time overlapping poolization to operate, one
Individual pond layer is formed by being separated by the grid that the pond unit that s pixel occupies forms, and each unit is responsible for adjacent z*z models
The central area summation enclosed, wherein s=1, z=2.
Further, the convolution kernel that 32 sizes are 5 × 5 is used in step d) to characteristic image F1' the secondary spy of (Y) progress
Sign extraction, convolution kernel moving step length are 1, and computer selects ReLU to pass through formula F as activation primitive2(Y)=max (0, W2*
F1'(Y)+B2) calculate characteristic image F2(Y), wherein W2For convolution kernel, B2The neuron bias vector for being 32 for dimension.
Further, overlapping pool layer is used in step e) to characteristic image F2(Y) carry out secondary overlapping poolization to operate, one
Individual pond layer is formed by being separated by the grid that the pond unit that s pixel occupies forms, and each unit is responsible for adjacent z*z models
The central area summation enclosed, wherein s=1, z=2.
Further, the convolution collecting image F that 32 sizes are 7 × 7 is used in step f)2' the low resolution in (Y) is special
Sign carries out Nonlinear Mapping, and convolution kernel moving step length is 1, and computer selects ReLU to pass through formula F as activation primitive3(Y)=
max(0,W3*F2'(Y)+B3) calculate high-resolution global characteristics mapping ensemblen F3(Y), wherein W3For convolution kernel, B3It is 32 for dimension
Neuron bias vector.
Further, the convolution kernel that 1 size is 5 × 5 is used in step g) to global characteristics mapping ensemblen F3(Y) rolled up
Product operation, convolution kernel moving step length are 1, and computer selects ReLU as activation primitive, by formula F ' (Y)=W4*F3(Y)+B4
Reconstructed image F'(Y), wherein W4For convolution kernel, B4The neuron bias vector for being 1 for dimension.
Further, the convolution collecting image F that 1 size is 11 × 11 is used in step h)2' (Y) progress convolution operation,
Convolution kernel moving step length is 1, and computer selects ReLU as activation primitive, " (Y)=W by formula F5*F2'(Y)+B5Reconstruct image
As F " (Y), wherein W5For convolution kernel, B5The neuron bias vector for being 1 for dimension.
Further, by formula F (Y)=F'(Y in step g))+F " (Y) progress imaging importings, obtain high resolution graphics
As F (Y).
The beneficial effects of the invention are as follows：The present invention provides a multilayer convolutional neural networks model, passes through two in model
Secondary feature extraction layer carries out finer feature extraction to medical image, reduces the spy of extraction by the overlapping pool layer in model
Dimension is levied, makes the feature of extraction more representative, the loss of each layer convolution operation is made up by the articulamentum in model, it is final to obtain
To more preferable super-resolution reconstruction effect.
Brief description of the drawings
Fig. 1 is the flow chart of the present invention；
Fig. 2 is of the invention and existing 6 kinds of super-resolution methods to basin bone CT Image Reconstruction comparative result figures；
Fig. 3 is of the invention and existing 6 kinds of super-resolution methods to oesophagus CT Image Reconstruction comparative result figures；
Fig. 4 is of the invention and existing 6 kinds of super-resolution methods to nasal cavity CT Image Reconstruction comparative result figures；
In figure, (a) original CT image (b) is that the CT images (c) after the processing of bicubic differential technique are the processing of sparse coding method
CT images (d) afterwards are the choosing of adaptive sparse domain for the CT images (e) after the Statistical Prediction Model method processing based on rarefaction representation
It is based on volume that the CT images (f) after method processing, which are selected, as the CT images (g) after the deep layer network Model Method processing based on sparse prior
CT images (h) after the image super-resolution method processing of product network are the medical image of the invention based on multilayer convolutional neural networks
CT images after ultra-resolution ratio reconstructing method processing.
Embodiment
1 to accompanying drawing 4, the present invention will be further described below in conjunction with the accompanying drawings.
A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks, comprises the following steps：
A) the low resolution medical image Y of acquisition is input in convolutional neural networks.B) low differentiate is cured by computer
Learn image Y and carry out first time convolution operation, extract partly overlapping block of pixels from low resolution medical image Y, and by each picture
Plain block is expressed as high dimensional feature vector, all feature F for low resolution medical image Y of output1(Y).C) computer pair is utilized
Characteristic pattern F1(Y) operation of first time overlapping poolization is carried out, obtains characteristic image F1'(Y).D) using computer to characteristic image F1'
(Y) secondary convolution operation is carried out, to characteristic image F1' (Y) progress Further Feature Extraction, by the feature composition characteristic of second extraction
Image F2(Y).E) computer characteristic image F is utilized2(Y) secondary overlapping poolization operation is carried out, is reduced feature graph parameter and dimension
Spend the image F of feature2'(Y).F) to image F2' the low resolution feature in (Y) carries out Nonlinear Mapping, by the low resolution of acquisition
Rate maps feature vectors obtain high-resolution global characteristics mapping ensemblen F into high-resolution features vector3(Y).G) computer is utilized
To global characteristics mapping ensemblen F3(Y) convolution operation, the image F'(Y reconstructed are carried out).H) computer is utilized afterwards in step e)
Establish image F2' (Y) articulamentum, image F is utilized on articulamentum2' (Y) reconstruct another piece image F " (Y).I) by step
G) the image F'(Y of reconstruct in) it is overlapped with F " (Y), obtain high-definition picture F (Y).The present invention provides a multilayer volume
Product neural network model, finer feature extraction is carried out to medical image by the Further Feature Extraction layer in model, passed through
Overlapping pool layer in model reduces the characteristic dimension of extraction, makes the feature of extraction more representative, passes through the connection in model
Layer makes up the loss of each layer convolution operation, finally gives more preferable super-resolution reconstruction effect.
Embodiment 1：
Feature extraction is carried out using the convolution kernel that 32 sizes are 9 × 9 in step b), convolution kernel moving step length is 1, is calculated
Machine selects ReLU to pass through formula F as activation primitive1(Y)=max (0, W1*Y+B1) calculate characteristic pattern F1(Y), wherein Y is low
Differentiate medical image, W1For convolution kernel, B1The neuron bias vector for being 32 for dimension.W1Size be：1×9×9×32.Its
In the corresponding convolution kernel of each element.The major function of feature extraction layer is the extraction (portion from step a) input picture
Divide overlapping) block of pixels, and each block of pixels is expressed as high dimensional feature vector.These vectors include a whole set of Feature Mapping,
The number of mapping is equal to the dimension of vector.The output of this layer is all features of input picture.
Embodiment 2：
Using overlapping pool layer to characteristic pattern F in step c)1(Y) operation of first time overlapping poolization, a pond layer are carried out
The grid that the pond unit occupied by being separated by s pixel forms is formed, and each unit is responsible for the center to adjacent z*z scopes
Region is summed, if setting s=z, we can just obtain the pond layer commonly used in other network structures.But the main pin of the present invention
To medical image, due to medical image specific use in processing we in order to more accurate, so we select it is overlapping
Pond makes s<z.Here it is the method that we use in the network frame of oneself, therefore make s=1, z=2.The purpose is to improve
The feature fineness extracted in step 2, dimension is reduced, but the number of input feature vector figure will not change, it is simply each
The size of output characteristic figure can reduce.Overlapping pool layer can also effectively reduce number of parameters simultaneously, be advantageous to simplify network, carry
The efficiency of high parameter training.
Embodiment 3：
Using the convolution kernel that 32 sizes are 5 × 5 to characteristic image F in step d)1' (Y) progress Further Feature Extraction, volume
Product core moving step length is 1, and computer selects ReLU to pass through formula F as activation primitive2(Y)=max (0, W2*F1'(Y)+B2)
Calculate characteristic image F2(Y), wherein W2For convolution kernel, B2The neuron bias vector for being 32 for dimension.W2Size be 32 × 5
×5×32.Because by step c), the dimension of characteristic pattern reduces, so the convolution kernel of 5 × 5 sizes just has been able to cover institute
There is feature.Different from other super-resolution network frames, for the Further Feature Extraction of medical image feature can be made more to represent
Property.
Embodiment 4：
Using overlapping pool layer to characteristic image F in step e)2(Y) secondary overlapping poolization operation, a pond layer are carried out
The grid that the pond unit occupied by being separated by s pixel forms is formed, and each unit is responsible for the center to adjacent z*z scopes
Sum in region.If setting s=z, we can just obtain the pond layer commonly used in other network structures.But the main pin of the present invention
To medical image, due to medical image specific use in processing we in order to more accurate, so we select it is overlapping
Pond makes s<z.Here it is the method that we use in the network frame of oneself, therefore make s=1, z=2.The purpose is to improve
The feature fineness extracted in step 2, dimension is reduced, but the number of input feature vector figure will not change, it is simply each
The size of output characteristic figure can reduce.Overlapping pool layer can also effectively reduce number of parameters simultaneously, be advantageous to simplify network, carry
The efficiency of high parameter training.
Embodiment 5：
The convolution collecting image F that 32 sizes are 7 × 7 is used in step f)2' the low resolution feature in (Y) carry out it is non-
Linear Mapping, convolution kernel moving step length are 1, and computer selects ReLU to pass through formula F as activation primitive3(Y)=max (0, W3*
F2'(Y)+B3) calculate high-resolution global characteristics mapping ensemblen F3(Y), wherein W3For convolution kernel, B3The neuron for being 32 for dimension is inclined
Put vector.Wherein W3Size be 32 × 5 × 5 × 32.The purpose is to the characteristic vector that will be obtained in step e) from low resolution
Space is converted into high resolution space.Vector after each mapping represents a high-resolution pixel block.
Embodiment 6：
Using the convolution kernel that 1 size is 5 × 5 to global characteristics mapping ensemblen F in step g)3(Y) convolution operation, volume are carried out
Product core moving step length is 1, and computer selects ReLU as activation primitive, by formula F ' (Y)=W4*F3(Y)+B4Reconstructed image
F'(Y), wherein W4For convolution kernel, B4The neuron bias vector for being 1 for dimension.Wherein W4Size is：32 × 5 × 5 × 1, W4's
Effect is similar to：First then these coefficient mappings to image area are averaged.Whole restructuring procedure is a linear operation.
Embodiment 7：
The convolution collecting image F that 1 size is 11 × 11 is used in step h)2' (Y) progress convolution operation, convolution kernel shifting
Dynamic step-length is 1, and computer selects ReLU as activation primitive, " (Y)=W by formula F5*F2'(Y)+B5Reconstructed image F " (Y),
Wherein W5For convolution kernel, B5The neuron bias vector for being 1 for dimension.Wherein W5Size be：32 × 11 × 11 × 1, we make
It is that make use of medical image that there is most of characteristic for having obvious repetitive structure with articulamentum, is made up so as to reach in network structure
The feature lost for no reason in training.Ensure there is no unnecessary falseness in the high-resolution medical image finally reconstructed with this
Information, reduce the error between the high-definition picture and true picture after reconstruct.Diagnosis to doctor is with checking that offer is credible
Spend high image.
Embodiment 8：
By formula F (Y)=F'(Y in step g))+F " (Y) progress imaging importings, obtain high-definition picture F (Y).
In order to better illustrate effectiveness of the invention, the present invention is shown reconstruct effect using the method for contrast experiment
Fruit.In a large amount of basin bones, each position selects the test that two CT images are tested as a comparison in the CT images of oesophagus and nasal cavity
Figure, part design sketch displaying such as Fig. 2, Fig. 3 of contrast experiment 2, shown in Fig. 4.(a) original CT image (b) is bicubic differential technique
CT images (c) after processing are that the CT images (d) after the processing of sparse coding method are the Statistical Prediction Model method based on rarefaction representation
CT images (e) after processing are that the CT images (f) after the back-and-forth method processing of adaptive sparse domain are the deep layer net based on sparse prior
CT images (h) after CT images (g) after the processing of network modelling are handled for the image super-resolution method based on convolutional network are this
CT images after medical image ultra-resolution ratio reconstructing method processing of the invention based on multilayer convolutional neural networks.Using this (b)-
(g) this 6 representative single image ultra-resolution ratio reconstructing methods are compared with the experimental result of the present invention.
Contrast experiment's content is as follows：
Experiment 1, respectively with bicubic differential technique, sparse coding method, the deep layer network Model Method based on sparse prior, be based on
The image super-resolution method of convolutional network, the Statistical Prediction Model method based on rarefaction representation, adaptive sparse domain back-and-forth method and
For the present invention to two CT images of basin bone, two CT images of oesophagus and two CT images of nasal cavity carry out 2 times of Super-resolution reconstructions
Structure.Its objective evaluation index PSNR is as shown in Table 1.
Table one
Experiment 2, respectively with bicubic differential technique, sparse coding method, the deep layer network Model Method based on sparse prior, be based on
The image super-resolution method of convolutional network, the Statistical Prediction Model method based on rarefaction representation, adaptive sparse domain back-and-forth method and
For the present invention to two CT images of basin bone, two CT images of oesophagus and two CT images of nasal cavity carry out 3 times of Super-resolution reconstructions
Structure.Its objective evaluation index PSNR is as shown in Table 2.
Table two
Experiment 3, respectively with bicubic differential technique, sparse coding method, the deep layer network Model Method based on sparse prior, be based on
The image super-resolution method of convolutional network, the Statistical Prediction Model method based on rarefaction representation, adaptive sparse domain back-and-forth method and
For the present invention to two CT images of basin bone, two CT images of oesophagus and two CT images of nasal cavity carry out 4 times of Super-resolution reconstructions
Structure.Its objective evaluation index PSNR is as shown in Table 3.
Table three
From table one, two, three as can be seen that the present invention has the average objective evaluation index of highest.With newest based on volume
The image super-resolution method of product network is compared, and average PSNR is than its 6 high 2.1dB on 2 times of CT images, in 3 times of CT images
Upper averagely PSNR is than its high 0.6dB, and average PSNR is than its high 1dB on 4 times of CT images.
In summary, the high-resolution medical image that present invention reconstruct obtains has obvious advantage on subjective effect,
And there is higher objective evaluation index for other control methods.Therefore the present invention is a kind of effective single width medical image
Ultra-resolution ratio reconstructing method.
Claims (9)
1. a kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks, it is characterised in that including as follows
Step：
A) the low resolution medical image Y of acquisition is input in convolutional neural networks；
B) first time convolution operation is carried out to low resolution medical image Y by computer, the extraction unit from low resolution medical image Y
Divide overlapping block of pixels, and each block of pixels is expressed as high dimensional feature vector, the owning for low resolution medical image Y of output
Feature F1(Y)；
C) using computer to characteristic pattern F1(Y) operation of first time overlapping poolization is carried out, obtains characteristic image F1'(Y)；
D) using computer to characteristic image F1' the secondary convolution operation of (Y) progress, to characteristic image F1' (Y) progress quadratic character
Extraction, the feature of second extraction is formed into characteristic image F2(Y)；
E) computer characteristic image F is utilized2(Y) secondary overlapping poolization operation is carried out, is reduced feature graph parameter and dimensional characteristics
Image F2'(Y)；
F) to image F2' the low resolution feature in (Y) carries out Nonlinear Mapping, by the low resolution maps feature vectors of acquisition
Into high-resolution features vector, high-resolution global characteristics mapping ensemblen F is obtained3(Y)；
G) using computer to global characteristics mapping ensemblen F3(Y) convolution operation, the image F'(Y reconstructed are carried out)；
H) image F is established using computer afterwards in step e)2' (Y) articulamentum, image F is utilized on articulamentum2' (Y) reconstruct
Go out another piece image F " (Y)；
I) by the image F'(Y of reconstruct in step g)) it is overlapped with F " (Y), obtain high-definition picture F (Y).
2. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：Feature extraction is carried out using the convolution kernel that 32 sizes are 9 × 9 in step b), convolution kernel moving step length is 1, is calculated
Machine selects ReLU to pass through formula F as activation primitive1(Y)=max (0, W1*Y+B1) calculate characteristic pattern F1(Y), wherein Y is low
Differentiate medical image, W1For convolution kernel, B1The neuron bias vector for being 32 for dimension.
3. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：Using overlapping pool layer to characteristic pattern F in step c)1(Y) carry out first time overlapping poolization operation, a pond layer by
The grid for being separated by the pond unit composition that s pixel occupies is formed, and each unit is responsible for the center to adjacent z*z scopes
Domain is summed, wherein s=1, z=2.
4. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：Using the convolution kernel that 32 sizes are 5 × 5 to characteristic image F in step d)1' (Y) progress Further Feature Extraction, volume
Product core moving step length is 1, and computer selects ReLU to pass through formula F as activation primitive2(Y)=max (0, W2*F1'(Y)+B2)
Calculate characteristic image F2(Y), wherein W2For convolution kernel, B2The neuron bias vector for being 32 for dimension.
5. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：Using overlapping pool layer to characteristic image F in step e)2(Y) carry out the operation of secondary overlapping poolization, a pond layer by
The grid for being separated by the pond unit composition that s pixel occupies is formed, and each unit is responsible for the center to adjacent z*z scopes
Domain is summed, wherein s=1, z=2.
6. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：The convolution collecting image F that 32 sizes are 7 × 7 is used in step f)2' the low resolution feature in (Y) carry out it is non-thread
Property mapping, convolution kernel moving step length be 1, computer select ReLU as activation primitive, pass through formula F3(Y)=max (0, W3*
F2'(Y)+B3) calculate high-resolution global characteristics mapping ensemblen F3(Y), wherein W3For convolution kernel, B3The neuron for being 32 for dimension is inclined
Put vector.
7. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：Using the convolution kernel that 1 size is 5 × 5 to global characteristics mapping ensemblen F in step g)3(Y) convolution operation, volume are carried out
Product core moving step length is 1, and computer selects ReLU as activation primitive, by formula F ' (Y)=W4*F3(Y)+B4Reconstructed image
F'(Y), wherein W4For convolution kernel, B4The neuron bias vector for being 1 for dimension.
8. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：The convolution collecting image F that 1 size is 11 × 11 is used in step h)2' (Y) progress convolution operation, convolution kernel movement
Step-length is 1, and computer selects ReLU as activation primitive, " (Y)=W by formula F5*F2'(Y)+B5Reconstructed image F " (Y), its
Middle W5For convolution kernel, B5The neuron bias vector for being 1 for dimension.
9. the medical image ultra-resolution ratio reconstructing method according to claim 1 based on multilayer convolutional neural networks, it is special
Sign is：By formula F (Y)=F'(Y in step g))+F " (Y) progress imaging importings, obtain high-definition picture F (Y).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710655267.8A CN107464216A (en) | 2017-08-03 | 2017-08-03 | A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710655267.8A CN107464216A (en) | 2017-08-03 | 2017-08-03 | A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107464216A true CN107464216A (en) | 2017-12-12 |
Family
ID=60548361
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710655267.8A Pending CN107464216A (en) | 2017-08-03 | 2017-08-03 | A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107464216A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108259994A (en) * | 2018-01-15 | 2018-07-06 | 复旦大学 | A kind of method for improving video spatial resolution |
| CN108304821A (en) * | 2018-02-14 | 2018-07-20 | 广东欧珀移动通信有限公司 | Image-recognizing method and device, image acquiring method and equipment, computer equipment and non-volatile computer readable storage medium storing program for executing |
| CN108416821A (en) * | 2018-03-08 | 2018-08-17 | 山东财经大学 | A kind of CT Image Super-resolution Reconstruction methods of deep neural network |
| CN108447020A (en) * | 2018-03-12 | 2018-08-24 | 南京信息工程大学 | A kind of face super-resolution reconstruction method based on profound convolutional neural networks |
| CN109035356A (en) * | 2018-07-05 | 2018-12-18 | 四川大学 | A kind of system and method based on PET pattern imaging |
| CN109118428A (en) * | 2018-06-07 | 2019-01-01 | 西安电子科技大学 | A kind of image super-resolution rebuilding method based on feature enhancing |
| CN109191376A (en) * | 2018-07-18 | 2019-01-11 | 电子科技大学 | High-resolution terahertz image reconstruction method based on SRCNN improved model |
| CN109325931A (en) * | 2018-08-22 | 2019-02-12 | 中北大学 | Based on the multi-modality images fusion method for generating confrontation network and super-resolution network |
| CN109394205A (en) * | 2018-09-30 | 2019-03-01 | 安徽心之声医疗科技有限公司 | A kind of more illness analysis methods of electrocardiosignal based on deep neural network |
| CN109543548A (en) * | 2018-10-26 | 2019-03-29 | 桂林电子科技大学 | A kind of face identification method, device and storage medium |
| CN109800746A (en) * | 2018-12-05 | 2019-05-24 | 天津大学 | A kind of hand-written English document recognition methods based on CNN |
| CN109815809A (en) * | 2018-12-19 | 2019-05-28 | 天津大学 | A kind of English handwriting identification method based on CNN |
| CN110826467A (en) * | 2019-11-22 | 2020-02-21 | 中南大学湘雅三医院 | Electron microscope image reconstruction system and method |
| WO2020056791A1 (en) * | 2018-09-21 | 2020-03-26 | 五邑大学 | Method and apparatus for super-resolution reconstruction of multi-scale dilated convolution neural network |
| CN110942425A (en) * | 2019-11-26 | 2020-03-31 | 贵州师范学院 | Reconstruction method and reconstruction system of super-resolution image and electronic equipment |
| CN111260566A (en) * | 2020-01-09 | 2020-06-09 | 复旦大学 | Image restoration method based on sparse prior |
| CN111258467A (en) * | 2020-01-07 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Interface display method and device, computer equipment and storage medium |
| CN112907441A (en) * | 2020-12-29 | 2021-06-04 | 中央财经大学 | Space downscaling method based on super-resolution of underground water satellite image |
| CN113114541A (en) * | 2021-06-15 | 2021-07-13 | 上海兴容信息技术有限公司 | Method and system for judging whether network connection can be established between network nodes |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106096616A (en) * | 2016-06-08 | 2016-11-09 | 四川大学华西医院 | A kind of nuclear magnetic resonance image feature extraction based on degree of depth study and sorting technique |
| CN106156807A (en) * | 2015-04-02 | 2016-11-23 | 华中科技大学 | The training method of convolutional neural networks model and device |
| CN106709875A (en) * | 2016-12-30 | 2017-05-24 | 北京工业大学 | Compressed low-resolution image restoration method based on combined deep network |
| US20170193680A1 (en) * | 2016-01-04 | 2017-07-06 | Kla-Tencor Corporation | Generating high resolution images from low resolution images for semiconductor applications |
-
2017
- 2017-08-03 CN CN201710655267.8A patent/CN107464216A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106156807A (en) * | 2015-04-02 | 2016-11-23 | 华中科技大学 | The training method of convolutional neural networks model and device |
| US20170193680A1 (en) * | 2016-01-04 | 2017-07-06 | Kla-Tencor Corporation | Generating high resolution images from low resolution images for semiconductor applications |
| CN106096616A (en) * | 2016-06-08 | 2016-11-09 | 四川大学华西医院 | A kind of nuclear magnetic resonance image feature extraction based on degree of depth study and sorting technique |
| CN106709875A (en) * | 2016-12-30 | 2017-05-24 | 北京工业大学 | Compressed low-resolution image restoration method based on combined deep network |
Non-Patent Citations (4)
| Title |
|---|
| JIWON KIM等: "Accurate Image Super-Resolution Using Very Deep Convolutional Networks", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
| KRIZHEVSKY A等: "ImageNet classification with deep convolutional neural networks", 《NEURAL INFORMATION PROCESSING SYSTEMS》 * |
| SUN Y等: "Deep learning face representation from predicting 10,000 classes", 《2014 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 肖进胜等: "改进的基于卷积神经网络的图像超分辨率算法", 《光学学报》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108259994B (en) * | 2018-01-15 | 2020-10-30 | 复旦大学 | Method for improving video spatial resolution |
| CN108259994A (en) * | 2018-01-15 | 2018-07-06 | 复旦大学 | A kind of method for improving video spatial resolution |
| CN108304821B (en) * | 2018-02-14 | 2020-12-18 | Oppo广东移动通信有限公司 | Image recognition method and device, image acquisition method and device, computer device and non-volatile computer-readable storage medium |
| CN108304821A (en) * | 2018-02-14 | 2018-07-20 | 广东欧珀移动通信有限公司 | Image-recognizing method and device, image acquiring method and equipment, computer equipment and non-volatile computer readable storage medium storing program for executing |
| CN108416821A (en) * | 2018-03-08 | 2018-08-17 | 山东财经大学 | A kind of CT Image Super-resolution Reconstruction methods of deep neural network |
| CN108447020A (en) * | 2018-03-12 | 2018-08-24 | 南京信息工程大学 | A kind of face super-resolution reconstruction method based on profound convolutional neural networks |
| CN109118428B (en) * | 2018-06-07 | 2023-05-19 | 西安电子科技大学 | Image super-resolution reconstruction method based on feature enhancement |
| CN109118428A (en) * | 2018-06-07 | 2019-01-01 | 西安电子科技大学 | A kind of image super-resolution rebuilding method based on feature enhancing |
| CN109035356A (en) * | 2018-07-05 | 2018-12-18 | 四川大学 | A kind of system and method based on PET pattern imaging |
| CN109035356B (en) * | 2018-07-05 | 2020-07-10 | 四川大学 | System and method based on PET (positron emission tomography) graphic imaging |
| CN109191376B (en) * | 2018-07-18 | 2022-11-25 | 电子科技大学 | High-resolution terahertz image reconstruction method based on SRCNN improved model |
| CN109191376A (en) * | 2018-07-18 | 2019-01-11 | 电子科技大学 | High-resolution terahertz image reconstruction method based on SRCNN improved model |
| CN109325931A (en) * | 2018-08-22 | 2019-02-12 | 中北大学 | Based on the multi-modality images fusion method for generating confrontation network and super-resolution network |
| WO2020056791A1 (en) * | 2018-09-21 | 2020-03-26 | 五邑大学 | Method and apparatus for super-resolution reconstruction of multi-scale dilated convolution neural network |
| CN109394205B (en) * | 2018-09-30 | 2022-06-17 | 安徽心之声医疗科技有限公司 | Electrocardiosignal analysis method based on deep neural network |
| CN109394205A (en) * | 2018-09-30 | 2019-03-01 | 安徽心之声医疗科技有限公司 | A kind of more illness analysis methods of electrocardiosignal based on deep neural network |
| CN109543548A (en) * | 2018-10-26 | 2019-03-29 | 桂林电子科技大学 | A kind of face identification method, device and storage medium |
| CN109800746A (en) * | 2018-12-05 | 2019-05-24 | 天津大学 | A kind of hand-written English document recognition methods based on CNN |
| CN109815809A (en) * | 2018-12-19 | 2019-05-28 | 天津大学 | A kind of English handwriting identification method based on CNN |
| CN110826467B (en) * | 2019-11-22 | 2023-09-29 | 中南大学湘雅三医院 | Electron microscope image reconstruction system and method thereof |
| CN110826467A (en) * | 2019-11-22 | 2020-02-21 | 中南大学湘雅三医院 | Electron microscope image reconstruction system and method |
| CN110942425A (en) * | 2019-11-26 | 2020-03-31 | 贵州师范学院 | Reconstruction method and reconstruction system of super-resolution image and electronic equipment |
| CN111258467A (en) * | 2020-01-07 | 2020-06-09 | 腾讯科技（深圳）有限公司 | Interface display method and device, computer equipment and storage medium |
| CN111260566B (en) * | 2020-01-09 | 2022-08-09 | 复旦大学 | Image restoration method based on sparse prior |
| CN111260566A (en) * | 2020-01-09 | 2020-06-09 | 复旦大学 | Image restoration method based on sparse prior |
| CN112907441A (en) * | 2020-12-29 | 2021-06-04 | 中央财经大学 | Space downscaling method based on super-resolution of underground water satellite image |
| CN112907441B (en) * | 2020-12-29 | 2023-05-30 | 中央财经大学 | Space downscaling method based on super-resolution of ground water satellite image |
| CN113114541B (en) * | 2021-06-15 | 2021-09-14 | 上海兴容信息技术有限公司 | Method and system for judging whether network connection can be established between network nodes |
| CN113114541A (en) * | 2021-06-15 | 2021-07-13 | 上海兴容信息技术有限公司 | Method and system for judging whether network connection can be established between network nodes |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107464216A (en) | A kind of medical image ultra-resolution ratio reconstructing method based on multilayer convolutional neural networks | |
| Du et al. | Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network | |
| JP2023025056A (en) | Dose reduction for medical imaging using deep convolutional neural network | |
| CN109214989B (en) | Single image super resolution ratio reconstruction method based on Orientation Features prediction priori | |
| CN108460726A (en) | A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network | |
| CN105046672B (en) | A kind of image super-resolution rebuilding method | |
| CN107369189A (en) | The medical image super resolution ratio reconstruction method of feature based loss | |
| CN105631807B (en) | The single-frame image super-resolution reconstruction method chosen based on sparse domain | |
| CN109671023A (en) | A kind of secondary method for reconstructing of face image super-resolution | |
| CN113344799A (en) | System and method for reducing colored noise in medical images using deep neural networks | |
| Sood et al. | An application of generative adversarial networks for super resolution medical imaging | |
| Du et al. | Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network | |
| CN106157244A (en) | A kind of QR Code Image Super-resolution Reconstruction method based on rarefaction representation | |
| Zhou et al. | Volume upscaling with convolutional neural networks | |
| CN108416821A (en) | A kind of CT Image Super-resolution Reconstruction methods of deep neural network | |
| Bai et al. | Probabilistic self‐learning framework for low‐dose CT denoising | |
| CN109360148A (en) | Based on mixing random down-sampled remote sensing image ultra-resolution ratio reconstructing method and device | |
| CN111899165A (en) | Multi-task image reconstruction convolution network model based on functional module | |
| CN110322402A (en) | Medical image super resolution ratio reconstruction method based on dense mixing attention network | |
| Guan et al. | Srdgan: learning the noise prior for super resolution with dual generative adversarial networks | |
| Lai et al. | Joint sparse reconstruction of multi-contrast MRI images with graph based redundant wavelet transform | |
| CN110349085A (en) | A kind of single image super-resolution feature Enhancement Method based on generation confrontation network | |
| CN110211193A (en) | Three dimensional CT interlayer image interpolation reparation and super-resolution processing method and device | |
| CN114792287A (en) | Medical ultrasonic image super-resolution reconstruction method based on multi-image fusion | |
| Liu et al. | An edge enhanced srgan for mri super resolution in slice-selection direction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20171212 |