CN109727197A - A kind of medical image super resolution ratio reconstruction method - Google Patents
A kind of medical image super resolution ratio reconstruction method Download PDFInfo
- Publication number
- CN109727197A CN109727197A CN201910004840.8A CN201910004840A CN109727197A CN 109727197 A CN109727197 A CN 109727197A CN 201910004840 A CN201910004840 A CN 201910004840A CN 109727197 A CN109727197 A CN 109727197A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- medical image
- resolution ratio
- residual error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
A kind of medical image super resolution ratio reconstruction method extracts the characteristic information of image by convolutional layer and is encapsulated in a capsule structure；Then, pass through routing infrastructure layer operation input and the high-resolution details of forecast image；Finally, it carries out the capsule residual error characteristic image that obtained routing infrastructure is predicted to merge reconstruct with interpolation low-resolution image, the high-definition picture that resolution ratio is substantially improved is obtained, the invention proposes a kind of new use routing infrastructure residual error networks to carry out super-resolution rebuilding to medical image.It is better than tradition CNN method such as SRCNN in terms of accuracy, real-time, picture quality, this method precision is high, reconstruction speed is fast, robustness is good, has broad application prospects in fields such as the computer-aided diagnosis systems such as medical conditions.
Description
Technical field
The present invention is medical image super resolution ratio reconstruction method, is suitable at machine learning, pattern-recognition and medical image
Manage technical field.
Background technique
Medical image is widely used in computer-aided diagnosis, as radioscopic image, computed tomography, nuclear-magnetism are total
Vibration image etc. changes modern medicine.Therefore the medical image for obtaining high quality plays to Guan Chong Precise Diagnosis illness reason
The effect wanted, but the hardware device of high-definition picture it is expensive and and imaging technique in certain circumstances limit
System, it is necessary to the image of high-resolution and clarity is obtained by way of software.In order to more preferably identify and determine disease
Exact position, it is desirable to obtain as far as possible clearly CT, MRI image provides better auxiliary diagnosis for doctor, related convenient for observation
Feature even subtle texture.Before the study of data-driven machine is widely used, traditional image rebuilding method is mostly used greatly
The form of optimization problem, it will be observed that LR image and model estimation between cost minimization, generally use some form of
Regularization term.However, these non-learning methods usually have a limitation, they need the priori indicated about good data to know
Know, the precision that can be improved is also limited.
With the development of deep learning, the super-resolution of image shows the raising that is greatly improved.Wherein, super-resolution convolution
Network (SRCNN) is concerned due to its network structure is simple, recovery precision is high.But the required training time is longer, dependent on small
The contextual information of image-region, training convergence is too slow, and network is used only for single scale.Designing a kind of can reach real-time
It is required that possess it is higher reconstruction signal-to-noise ratio and structural similarity super resolution ratio reconstruction method have in clinical application it is of crucial importance
Meaning, better booster action played to diagnosis various diseases, the super-resolution rebuilding of medical image is computer view
One research hotspot in feel field.
Summary of the invention
The purpose of the present invention is to medical image super-resolution rebuildings, and in the presence of rebuilding, speed is slow, reconstructed image quality is low
The disadvantages of, existing method for reconstructing is unable to reach high-precision while meeting requirement of real time.It is proposed a kind of routing infrastructure residual error net
The medical image super resolution ratio reconstruction method of network.
The technical solution adopted by the present invention is that: a kind of medical image super resolution ratio reconstruction method, comprising:
1) data set source The Cancer Imaging Archive (TCIA), the database are the medicine of cancer research
The Open Access Journals database of image；
2) pretreatment such as it is normalized, rotates, expanding to medical image；
3) pass through Training mode with capsule residual error characteristic pattern of the depth network structure based on routing to original image
Learn its deeper feature；
4) after precisely obtaining low resolution to high-resolution mapping relations, the capsule that obtained routing infrastructure is predicted is residual
Poor characteristic image carries out merging reconstruct with interpolation low-resolution image；
5) image after reconstructing has obtained higher resolution ratio and more preferably PSNR, the indexs such as SSIM.
Invention herein is, using Tensorflow as back-end engine, to operate in GPU work based on deep learning frame Keras
On standing, the test platform that uses: processor is Intel i7-7700 CPU, inside saves as 16GB, video card NVIDIA GeForce
GTX 1070.Since GPU video memory is little, so larger neural network can not be run in existing machine, this is also
This experiment uses small-scale the reason of training neural network after doing image preprocessing.In order to existing deep learning method into
One quantitative comparison of row, herein assesses the method for proposition using identical TCIA medical images data sets.
There is the method for many reconstruction and study for medical image super-resolution rebuilding at present, but rebuilds speed and again
Building quality always is one of the significant challenge faced in super-resolution rebuilding task.Selected from TCIA data set parts of images into
Row experiment, the detection effect figure of the networks and this paper network such as comparison SRCNN, it can be seen that context of methods is to the super-resolution in image
Rate reconstruction has better PSNR, SSIM and lower reconstruction time.
Effect of the invention is: the new use routing infrastructure residual error network of one kind being proposed to carry out oversubscription to medical image
Resolution is rebuild.Firstly, extracting the characteristic information of image by convolutional layer and being encapsulated in a capsule structure；Then, pass through road
By structure sheaf operation input and the high-resolution details of forecast image.Finally, the capsule residual error that obtained routing infrastructure is predicted
Characteristic image carries out merging reconstruct with interpolation low-resolution image, obtains the high-definition picture that resolution ratio is substantially improved.We
Method is better than tradition CNN method such as SRCNN in terms of accuracy, real-time, picture quality, this method precision is high, it is fast to rebuild speed,
Robustness is good, has broad application prospects in fields such as the computer-aided diagnosis systems such as medical conditions.
Detailed description of the invention
Fig. 1 is the network topology structure figure that the present invention uses；
Fig. 2 is the capsule residual error characteristic pattern that the present invention uses；
Fig. 3 is the routing infrastructure flow through a network figure that the present invention uses；
Fig. 4 (a) is the reconstruction effect picture original image that present invention amplification scale is 3；
Fig. 4 (b) is the reconstruction effect picture SRCNN reconstruction figure that present invention amplification scale is 3；
Fig. 4 (c) is that the reconstruction effect picture DRCN that present invention amplification scale is 3 is rebuild；
Fig. 4 (d) is the reconstruction effect picture VDSR reconstruction figure that present invention amplification scale is 3；
Fig. 4 (e) is the reconstruction effect picture reconstruction figure of the present invention that present invention amplification scale is 3.
Specific implementation method
See that Fig. 1, Fig. 2, Fig. 3, Fig. 4 (a)-Fig. 4 (e), a kind of medical image super resolution ratio reconstruction method are mentioned by convolutional layer
It takes the characteristic information of image and is encapsulated in a capsule structure；Then, pass through routing infrastructure layer operation input and forecast image
High-resolution details.Finally, by the capsule residual error characteristic image and interpolation low-resolution image of the prediction of obtained routing infrastructure
Fusion reconstruct is carried out, the high-definition picture that resolution ratio is substantially improved is obtained；Method includes:
1) data set source The Cancer Imaging Archive (TCIA), the database are the medicine of cancer research
The Open Access Journals database of image；
2) pretreatment such as it is normalized, rotates, expanding to medical image；
3) pass through Training mode with capsule residual error characteristic pattern of the depth network structure based on routing to original image
Learn its deeper feature；
4) after precisely obtaining low resolution to high-resolution mapping relations, the capsule that obtained routing infrastructure is predicted is residual
Poor characteristic image carries out merging reconstruct with interpolation low-resolution image；
5) image after reconstructing has obtained higher resolution ratio and more preferably PSNR, the indexs such as SSIM.
Target of the present invention is medical image super-resolution rebuilding, which is considered as one
The transition problem of a mode, i.e. medical image are first mode, and capsule residual error characteristic pattern is second mode；It is improved using one
Convolutional neural networks simulate the mapping relations between first mode and second mode.
Medical image super resolution ratio reconstruction method of the present invention is: using an improved convolutional neural networks simulation
Mapping relations between first mode and second mode；It is indicated between Current Situation of Neural Network reconstruct image and standard drawing with loss function
Error；The error that iterates in the training process loss function, when loss function is as small as possible, training obtains model and can
It is enough effectively to extract to medical image low resolution to high-resolution mapping principle, HR is gone out by the regular accurate reconstruction acquired
Medical image.Entire medical image super-resolution rebuilding process by the feature extraction of image capsule residual error, neural network mapping study,
The synthesis three parts of reconstruct image form.
The present invention is based on the minds that routing infrastructure residual error neural network architecture design one can effectively extract characteristics of image
Through network；The network includes extraction and feature reconstruction two parts of feature；The thought of the network is to propose that has a routing knot
The residual error neural network of structure replaces pondization to operate, improves effective benefit to feature with dynamic routing operation in the network architecture
With the capsule residual error characteristic image and interpolation for reducing the loss of a large amount of characteristic informations of image, and obtained routing infrastructure being predicted
Low-resolution image carries out fusion reconstruct, based on the reconstruction image after comprehensive characteristics information have higher resolution ratio and
PSNR,SSIM。
The depth network that the present invention uses be one can predicted characteristics detection probability repetition framework, each layer of convolutional layer it
There are multiple same routing infrastructure networks to connect afterwards, convolution kernel size is 9x9, and has one after each convolutional layer
Activation primitive ReLU；It is a size after several pieces continuous for 3x3, the routing layer that step-length is 1, effect is to reduce image point
Resolution obtains the depth characteristic for extracting capsule residual image；The glue for the routing infrastructure prediction that feature reconstruction part is
Capsule residual error characteristic image carries out merging reconstruct with interpolation low-resolution image；The input of depth network is the low resolution after interpolation
Medical image, output are then the high-resolution HR medical images after reconstruct.
Claims (5)
1. a kind of medical image super resolution ratio reconstruction method, it is characterised in that: extract the characteristic information of image simultaneously by convolutional layer
It is encapsulated in a capsule structure；Then, pass through routing infrastructure layer operation input and the high-resolution details of forecast image；Most
Afterwards, it carries out the capsule residual error characteristic image that obtained routing infrastructure is predicted to merge reconstruct with interpolation low-resolution image, obtain
The high-definition picture that resolution ratio is substantially improved；Method includes:
1) data set source The Cancer ImagingArchive, the database are the openings of the medical image of cancer research
Obtain database；
2) pretreatment such as it is normalized, rotates, expanding to medical image；
3) learnt with capsule residual error characteristic pattern of the depth network structure based on routing to original image by Training mode
Its deeper feature；
4) after precisely obtaining low resolution to high-resolution mapping relations, the capsule residual error of obtained routing infrastructure prediction is special
Sign image carries out merging reconstruct with interpolation low-resolution image；
5) image after reconstructing has obtained higher resolution ratio and more preferably PSNR, SSIM index.
2. a kind of medical image super resolution ratio reconstruction method according to claim 1, which is characterized in that by the medical image
Super resolution ratio reconstruction method is considered as the transition problem of a mode, i.e. medical image is first mode, and capsule residual error characteristic pattern is
Second mode；Using the mapping relations between an improved convolutional neural networks simulation first mode and second mode.
3. a kind of medical image super resolution ratio reconstruction method according to claim 1, which is characterized in that described to use receptive field
Bigger capsule residual error depth network substitutes traditional CNN network；The first mould is simulated using an improved convolutional neural networks
Mapping relations between formula and second mode；The mistake between current depth network reconfiguration figure and standard drawing is indicated with loss function
Difference；The error that iterates in the training process loss function, when loss function is as small as possible, training obtains model can have
Effect is extracted to medical image low resolution to high-resolution mapping principle, goes out HR medicine by the regular accurate reconstruction acquired
Image；Entire medical image super-resolution rebuilding process is by the feature extraction of image capsule residual error, depth network mapping study, reconstruct
The synthesis three parts of figure form.
4. a kind of medical image super resolution ratio reconstruction method according to claim 1 or 2 or 3, which is characterized in that be based on road
The neural network of characteristics of image can be effectively extracted by structural residual neural network architecture design one；The network includes feature
Extraction and feature reconstruction two parts；The thought of the network is to propose the residual error neural network with routing infrastructure, that is, is existed
It replaces pondization to operate with dynamic routing operation in network structure, improves the effective use to feature, reduce the big measure feature letter of image
The loss of breath, and carry out the capsule residual error characteristic image of obtained routing infrastructure prediction to merge weight with interpolation low-resolution image
Structure is having higher resolution ratio and PSNR, SSIM based on the reconstruction image after comprehensive characteristics information.
5. a kind of medical image super resolution ratio reconstruction method according to claim 1 or 2 or 3, which is characterized in that use
Depth network is the repetition framework of an energy predicted characteristics detection probability, has multiple same routings after each layer of convolutional layer
Structural network connects, and convolution kernel size is 9x9, and have an activation primitive ReLU after each convolutional layer；Continuous
It is a size after several pieces for 3x3, the routing layer that step-length is 1, effect is to reduce image resolution ratio, and it is residual to obtain extraction capsule
The depth characteristic of difference image；The capsule residual error characteristic image and interpolation for the routing infrastructure prediction that feature reconstruction part is
Low-resolution image carries out fusion reconstruct；The input of depth network is the low resolution medical image after interpolation, and output is then weight
High-resolution HR medical image after structure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004840.8A CN109727197B (en) | 2019-01-03 | 2019-01-03 | Medical image super-resolution reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910004840.8A CN109727197B (en) | 2019-01-03 | 2019-01-03 | Medical image super-resolution reconstruction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109727197A true CN109727197A (en) | 2019-05-07 |
| CN109727197B CN109727197B (en) | 2023-03-14 |
Family
ID=66299605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910004840.8A Active CN109727197B (en) | 2019-01-03 | 2019-01-03 | Medical image super-resolution reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109727197B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866870A (en) * | 2019-10-29 | 2020-03-06 | 中山大学 | Super-resolution processing method for amplifying medical image by any multiple |
| CN112082915A (en) * | 2020-08-28 | 2020-12-15 | 西安科技大学 | Plug-and-play type atmospheric particulate concentration detection device and detection method |
| CN115115727A (en) * | 2022-05-18 | 2022-09-27 | 首都医科大学附属北京友谊医院 | Nuclear magnetic image processing method, system, device and storage medium |
| CN115272084A (en) * | 2022-09-27 | 2022-11-01 | 成都信息工程大学 | High-resolution image reconstruction method and device |
| CN115545110A (en) * | 2022-10-12 | 2022-12-30 | 中国电信股份有限公司 | High resolution data reconstruction method for generating countermeasure network and related method and device |
| CN112082915B (en) * | 2020-08-28 | 2024-05-03 | 西安科技大学 | Plug-and-play type atmospheric particulate concentration detection device and detection method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108647775A (en) * | 2018-04-25 | 2018-10-12 | 陕西师范大学 | Super-resolution image reconstruction method based on full convolutional neural networks single image |
| CN108985316A (en) * | 2018-05-24 | 2018-12-11 | 西南大学 | A kind of capsule network image classification recognition methods improving reconstructed network |
| WO2019202292A1 (en) * | 2018-04-20 | 2019-10-24 | DrugAI Limited | Interaction property prediction system and method |
-
2019
- 2019-01-03 CN CN201910004840.8A patent/CN109727197B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019202292A1 (en) * | 2018-04-20 | 2019-10-24 | DrugAI Limited | Interaction property prediction system and method |
| CN108647775A (en) * | 2018-04-25 | 2018-10-12 | 陕西师范大学 | Super-resolution image reconstruction method based on full convolutional neural networks single image |
| CN108985316A (en) * | 2018-05-24 | 2018-12-11 | 西南大学 | A kind of capsule network image classification recognition methods improving reconstructed network |
Non-Patent Citations (2)
| Title |
|---|
| SANJIB KUMAR SAHU，PANKAJ KUMAR: "Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules", 《2018 UKSIM-AMSS 20TH INTERNATIONAL CONFERENCE ON MODELLING & SIMULATION》 * |
| 朱应钊，胡颖茂，李嫚: "胶囊网络技术及发展趋势研究", 《广东通信技术》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866870A (en) * | 2019-10-29 | 2020-03-06 | 中山大学 | Super-resolution processing method for amplifying medical image by any multiple |
| CN110866870B (en) * | 2019-10-29 | 2023-04-28 | 中山大学 | Super-resolution processing method for amplifying medical image by any multiple |
| CN112082915A (en) * | 2020-08-28 | 2020-12-15 | 西安科技大学 | Plug-and-play type atmospheric particulate concentration detection device and detection method |
| CN112082915B (en) * | 2020-08-28 | 2024-05-03 | 西安科技大学 | Plug-and-play type atmospheric particulate concentration detection device and detection method |
| CN115115727A (en) * | 2022-05-18 | 2022-09-27 | 首都医科大学附属北京友谊医院 | Nuclear magnetic image processing method, system, device and storage medium |
| CN115115727B (en) * | 2022-05-18 | 2023-08-01 | 首都医科大学附属北京友谊医院 | Nuclear magnetic image processing method, system, equipment and storage medium |
| CN115272084A (en) * | 2022-09-27 | 2022-11-01 | 成都信息工程大学 | High-resolution image reconstruction method and device |
| CN115272084B (en) * | 2022-09-27 | 2022-12-16 | 成都信息工程大学 | High-resolution image reconstruction method and device |
| CN115545110A (en) * | 2022-10-12 | 2022-12-30 | 中国电信股份有限公司 | High resolution data reconstruction method for generating countermeasure network and related method and device |
| CN115545110B (en) * | 2022-10-12 | 2024-02-02 | 中国电信股份有限公司 | High resolution data reconstruction method for generating an antagonism network and related method and apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109727197B (en) | 2023-03-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111709953B (en) | Output method and device in lung lobe segment segmentation of CT (computed tomography) image | |
| CN109727197A (en) | A kind of medical image super resolution ratio reconstruction method | |
| CN110288609B (en) | Multi-modal whole-heart image segmentation method guided by attention mechanism | |
| Mall et al. | A comprehensive review of deep neural networks for medical image processing: Recent developments and future opportunities | |
| Tong et al. | RIANet: Recurrent interleaved attention network for cardiac MRI segmentation | |
| Banerjee et al. | A completely automated pipeline for 3D reconstruction of human heart from 2D cine magnetic resonance slices | |
| CN109636802A (en) | Pulmonary parenchyma based on depth convolutional neural networks is through CT image partition method | |
| Tian et al. | Multi-step medical image segmentation based on reinforcement learning | |
| CN110517238A (en) | CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system | |
| Shen et al. | Smart health of ultrasound telemedicine based on deeply represented semantic segmentation | |
| WO2022032824A1 (en) | Image segmentation method and apparatus, device, and storage medium | |
| Chen et al. | Generative adversarial U-Net for domain-free medical image augmentation | |
| CN107146263B (en) | A kind of dynamic PET images method for reconstructing based on the constraint of tensor dictionary | |
| Du et al. | Segmentation and visualization of left atrium through a unified deep learning framework | |
| Yong et al. | Automatic ventricular nuclear magnetic resonance image processing with deep learning | |
| Zhao et al. | AGMN: Association graph-based graph matching network for coronary artery semantic labeling on invasive coronary angiograms | |
| Qi et al. | Cascaded conditional generative adversarial networks with multi-scale attention fusion for automated bi-ventricle segmentation in cardiac MRI | |
| CN113689441A (en) | DeepLabV3 network-based left ventricle ultrasonic dynamic segmentation method | |
| Li et al. | RSU-Net: U-net based on residual and self-attention mechanism in the segmentation of cardiac magnetic resonance images | |
| CN109637629A (en) | A kind of BI-RADS hierarchy model method for building up | |
| Dong et al. | A novel end‐to‐end deep learning solution for coronary artery segmentation from CCTA | |
| Beetz et al. | Mesh U-Nets for 3D cardiac deformation modeling | |
| Wen et al. | Analysis on SPECT myocardial perfusion imaging with a tool derived from dynamic programming to deep learning | |
| Lu et al. | AugMS-Net: Augmented multiscale network for small cervical tumor segmentation from MRI volumes | |
| Yang et al. | A lightweight fully convolutional network for cardiac MRI segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |