CN110827275B - Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning - Google Patents
Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning Download PDFInfo
- Publication number
- CN110827275B CN110827275B CN201911157464.2A CN201911157464A CN110827275B CN 110827275 B CN110827275 B CN 110827275B CN 201911157464 A CN201911157464 A CN 201911157464A CN 110827275 B CN110827275 B CN 110827275B
- Authority
- CN
- China
- Prior art keywords
- nuclear magnetic
- liver
- layer
- image
- liver nuclear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000004185 liver Anatomy 0.000 title claims abstract description 86
- 210000001367 artery Anatomy 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 32
- 240000007651 Rubus glaucus Species 0.000 title claims abstract description 23
- 235000011034 Rubus glaucus Nutrition 0.000 title claims abstract description 23
- 235000009122 Rubus idaeus Nutrition 0.000 title claims abstract description 23
- 238000013135 deep learning Methods 0.000 title claims abstract description 18
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 22
- 238000003908 quality control method Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims abstract description 20
- 238000012800 visualization Methods 0.000 claims abstract description 16
- 238000011176 pooling Methods 0.000 claims abstract description 15
- 238000012216 screening Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 230000007704 transition Effects 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 238000010586 diagram Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 6
- 238000005481 NMR spectroscopy Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 3
- 238000013145 classification model Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 abstract description 2
- 238000003745 diagnosis Methods 0.000 description 10
- 208000014018 liver neoplasm Diseases 0.000 description 10
- 201000007270 liver cancer Diseases 0.000 description 9
- 238000002595 magnetic resonance imaging Methods 0.000 description 8
- 239000002872 contrast media Substances 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 5
- 238000002347 injection Methods 0.000 description 5
- 239000007924 injection Substances 0.000 description 5
- 208000019423 liver disease Diseases 0.000 description 5
- 238000013136 deep learning model Methods 0.000 description 3
- 238000000586 desensitisation Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000003759 clinical diagnosis Methods 0.000 description 2
- 210000005229 liver cell Anatomy 0.000 description 2
- 206010006322 Breath holding Diseases 0.000 description 1
- 206010019695 Hepatic neoplasm Diseases 0.000 description 1
- 206010067125 Liver injury Diseases 0.000 description 1
- 239000002616 MRI contrast agent Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 230000036770 blood supply Effects 0.000 description 1
- 201000011510 cancer Diseases 0.000 description 1
- 208000019425 cirrhosis of liver Diseases 0.000 description 1
- 238000013170 computed tomography imaging Methods 0.000 description 1
- 230000034994 death Effects 0.000 description 1
- 231100000517 death Toxicity 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000013399 early diagnosis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011010 flushing procedure Methods 0.000 description 1
- 210000002767 hepatic artery Anatomy 0.000 description 1
- 230000002440 hepatic effect Effects 0.000 description 1
- 231100000753 hepatic injury Toxicity 0.000 description 1
- 210000002989 hepatic vein Anatomy 0.000 description 1
- 210000003494 hepatocyte Anatomy 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000003902 lesion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000001208 nuclear magnetic resonance pulse sequence Methods 0.000 description 1
- 239000002504 physiological saline solution Substances 0.000 description 1
- 210000003240 portal vein Anatomy 0.000 description 1
- 238000011158 quantitative evaluation Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30056—Liver; Hepatic
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Epidemiology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a liver nuclear magnetic artery phase image quality grading method based on raspberry pie and deep learning, which comprises the following steps: step one, acquiring a liver nuclear magnetic artery phase image and preprocessing the image to obtain a gray image, so as to obtain a training sample data set with graded quality; classifying the training sample data set and constructing a convolutional neural network model; step three, performing feature visualization operation by using global average pooling operation to obtain a feature visualization heat map; step four, screening a characteristic visual heat map; inputting the depth abstract features of the feature map into a classifier for secondary training to obtain a common Mei-show liver nuclear magnetic quality control grading model; step six, inputting the liver nuclear magnetic arterial phase images of the patient to be classified to obtain the grading prediction result of the liver nuclear magnetic arterial phase images of the common display.
Description
Technical Field
The invention relates to the field of medical image processing and analysis, in particular to a liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning.
Background
China is a country with high incidence of liver diseases. Liver cancer, called liver cancer for short, is a common malignant tumor of Chinese people. It is counted that about 33 ten thousand people die of liver cancer every year in China, accounting for nearly 50% of the world's liver cancer deaths. The method can realize early screening, early diagnosis and early treatment of liver cancer patients, and can greatly prolong the life cycle of the liver cancer patients. Various imaging examinations including computed tomography and Magnetic Resonance Imaging (MRI) can monitor tissue morphology and focus characteristics of liver diseases noninvasively, and become one of important means for early screening of liver cancer.
Primepick is a novel MRI contrast agent. About 50% of the common display contrast agent in the blood can be absorbed by liver cells, so that MRI can accurately provide liver focus blood supply information and liver cell information in the diagnosis process. The clinical trial of the American multicenter 3 phase has proved in 2010 that the common display can improve the qualitative diagnosis rate of liver lesions and has good safety. And thus, plectrum has been considered by clinicians as a valuable liver and gall specific contrast agent. With the wide application of the common Meyer in clinical liver disease diagnosis, the defects of the common Meyer are gradually revealed. Clinicians find that liver MRI arterial phase images often suffer from artifacts. The imaging performance of the artifacts is different and the causes are different. These artifacts can be a significant contributor to the imaging quality of MRI, and in severe cases can lead to missed diagnosis, misdiagnosis, or even failure to diagnose.
The acquisition of high-quality medical images is a precondition of accurate image diagnosis, and the quality control of liver common-display MRI arterial phase images can improve the quality of MRI images and the accuracy of clinical diagnosis. The western countries have already implemented medical image quality control systems, which are mainly manually inspected by medical physicists and engineering technicians in radiology. But the clinical quality control in China starts late, and the related system is not perfect. The medical physicist or engineering technician of radiology department lacks clinical diagnosis experience and knowledge, and cannot accurately grasp the gist of image quality control.
In recent years, with the rapid development of artificial intelligence technology, a great number of imaging-based automatic diagnosis or quantitative evaluation of liver diseases are emerging, such as automatic segmentation of liver and liver tumor areas based on a full convolutional neural network FCN, liver cirrhosis classification based on a convolutional neural network CNN, liver cancer classification, liver cancer survival prediction based on an imaging group and machine learning model, liver injury classification, and the like. These prior studies demonstrated that the way of mining digital features of medical images and training artificial intelligence models could be used for assisted analysis of liver disease.
In summary, it is necessary to fully automatically complete the MRI quality control of the common liver by using the deep learning model, optimize the existing workflow, improve the MRI quality of the common liver in clinic, actually help the clinician to reduce the working pressure, improve the diagnosis efficiency, and benefit the patient.
Disclosure of Invention
The invention provides a liver nuclear magnetic arterial phase image quality grading method based on raspberry pie and deep learning, which takes the image data of the liver nuclear magnetic arterial phase of the common display marked by a professional radiologist as a training sample to construct a convolutional neural network model, and can accurately grade the quality of the liver nuclear magnetic arterial phase image of the common display.
The invention also aims to introduce a characteristic dense connection strategy, increase the characteristic input of each layer, distinguish the nuclear magnetic arterials, and improve the accuracy of quality control grading.
A liver nuclear magnetic artery phase image quality grading method based on raspberry pie and deep learning comprises the following steps:
firstly, acquiring a liver nuclear magnetic artery phase image, preprocessing the liver nuclear magnetic artery phase image to obtain a gray level image, and respectively marking the grading result of the liver nuclear magnetic artery phase image to obtain a quality grading training sample data set;
classifying the training sample data set, constructing a convolutional neural network model, extracting hidden features in a gray level image, and obtaining a pre-trained convolutional neural network model;
step three, utilizing a gradient weighting type activation mapping method to perform feature visualization on all convolution layers in the pre-training convolution neural network model to obtain a feature visualization heat map;
screening the feature visualization heat map, selecting a feature layer with a highlight capturing artifact region, and extracting the depth abstract feature of the feature map;
inputting the depth abstract features of the feature map into a classifier for secondary training to obtain a training-completed common display liver nuclear magnetic quality control grading model;
step six, setting up a common display liver nuclear magnetic quality control grading model in raspberry pie equipment, and inputting the patient liver nuclear magnetic arterial phase images to be classified to obtain grading prediction results of the common display liver nuclear magnetic arterial phase images.
Preferably, the liver nuclear magnetic resonance arterial phase image preprocessing process in the first step includes:
firstly, carrying out signal normalization on the acquired liver nuclear magnetic artery phase image, wherein the calculation formula is as follows:
wherein I is i Is the signal value of the ith liver nuclear magnetic artery phase image, I' i For the normalized image signal value of the liver nuclear magnetic artery phase,sigma is the signal mean value of all acquired nuclear magnetic images I Signal standard deviation representing all nuclear magnetic images;
and then, carrying out gray processing on the liver nuclear magnetic artery phase image and carrying out pixel point segmentation to obtain a gray image.
Preferably, the second step includes:
inputting the gray level image of the liver nuclear magnetic artery phase image as an input layer vector into a convolutional neural network model; the convolution god sets the output layer of the network model as a quality grading label of the image of the magnetic artery phase of the liver nuclear of the common display;
the convolutional neural network model comprises a first dense connection module, a second dense connection module and a third dense connection module;
a first transition module is arranged between the first dense connection module and the second dense connection module;
a second transition module is arranged between the second dense connection module and the third dense connection module;
the dense connection modules comprise 6 convolution layers with convolution kernels of 3 multiplied by 3, and can extract features of gray images of the liver nuclear magnetic arterial phase images;
the transition modules each include 1 transition convolutional layer with a convolutional kernel size of 1 x 1 and 1 average pooling layer with a kernel size of 2 x 2, which enable the convolutional layer characteristics to be compressed and selected.
Preferably, the operation process of the convolution layer is as follows:
sliding on a gray image of an input layer by utilizing a convolution kernel, and performing convolution operation on pixel points (i, j) on the gray image to obtain an output feature map, wherein the convolution operation formula is as follows:
wherein x is l (i, j) is a feature of arbitrary layer l, x l ＝H l (x 0 ，x 1 ,…x i …,x l-1 )，x i Is characteristic of any preamble layer, H l () For batch normalization operations, consisting of an activation function and a convolution operation of size 3 x 3, x l+1 (i, j) is the output feature of arbitrary layer l, w l+1 (i, j) is the weight parameter of layer 1+1, b l Is the bias value of the first layer;
the size of the output feature map is:
wherein L is l+1 For the output feature map size of arbitrary layer L, L l For the feature size of any layer l of any layer, p is the corresponding filling parameter, f is the corresponding convolution kernel size, and s is the corresponding convolution step size.
Preferably, the convolution kernel size is 3×3.
Preferably, the operation formula of the pooling layer is:
wherein x is l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size.
Preferably, the feature visualization heat map calculation formula is:
where H is the feature visualization heat map, relu is the activation function,gradient weight for class c for nth feature map, ++> For the mean value, y of the feature map with n dimensions of i x j c Scoring class c by nth feature map, +.> Weight of category c for nth feature map, ++>Where c=3.
Preferably, the classifier in the fifth step has a calculation formula as follows:
wherein the method comprises the steps of，a k (X) is an activation function operation at the characteristic channel k and X ε Ω pixel positions, P k (X) the output value at the pixel position of characteristic channel k and X.epsilon.omega.
Preferably, the common display liver nuclear magnetic quality control grading model is stored in a raspberry pi device.
The beneficial effects of the invention are that
The invention provides a supervised learning mode using a deep learning model, which is trained on a large-scale common display liver nuclear magnetic data set marked by a professional radiologist, and integrates human knowledge and experience to realize feature screening, thereby effectively compressing features. The obtained common Mei-show liver nuclear magnetic quality control grading model can be used for more specifically distinguishing nuclear magnetic arterial phase artifacts, and the accuracy of quality control grading is improved.
Drawings
Fig. 1 is a flowchart of a method for grading the image quality of a liver nuclear magnetic artery phase based on raspberry group and deep learning.
Fig. 2 is a network structure diagram of the deep learning neural network model according to the present invention.
Fig. 3 is a schematic diagram of a sample with 1-part image quality classification in the nuclear magnetic artery phase of the common-display liver.
Fig. 4 is a schematic diagram of a 2-part sample of the image quality classification of the nuclear magnetic artery phase of the common-display liver.
Fig. 5 is a schematic diagram of a common-display liver nuclear magnetic artery phase image quality grading 3-segment sample.
Fig. 6 is a diagram showing a connection structure of a raspberry pi device according to the present invention.
Detailed Description
The present invention is described in further detail below with reference to the drawings to enable those skilled in the art to practice the invention by referring to the description.
As shown in fig. 1, the method for grading the quality of the liver nuclear magnetic artery phase image based on raspberry group and deep learning provided by the invention comprises the following steps:
step S110, acquiring liver nuclear magnetic artery phase images, respectively marking grading results, and preprocessing the liver nuclear magnetic artery phase images to obtain gray images so as to obtain a training sample data set with graded quality;
the liver nuclear magnetic artery phase image acquisition method comprises the following steps:
the patients all received breath-hold training prior to examination. And the patient was asked to disable water on the morning of the test and to empty stomach for 4 hours prior to the test.
The inspection equipment adopts a super-high field nuclear magnetic scanner, the maximum gradient field intensity is 40mT/m in the X axis and 40mT/m in the Y axis, 45mT/m in the Z axis, the gradient field switching rate is 200mT/m/m, and the surface coil adopts an 8-channel phased array body coil.
Before and after the contrast agent is injected, T1WI of a rapid three-dimensional phase-disturbing gradient echo pulse sequence is respectively carried out, the specific parameters are TR/TE=4.19/4.17 ms, the turnover angle is 9 degrees, the visual field is 300 multiplied by 400mm, the matrix is 168 multiplied by 320, and the recombination minimum layer thickness is 2.5-3.5 mm.
The common Mei-showing contrast agent is injected by pushing through an artificial vein high-pressure injection cylinder, the injection dosage is 0.1mL/kg, the flow rate is 2mL/s, the contrast agent is immediately flushed by 20mL of physiological saline after injection, the flushing flow rate is 2mL/s, the hepatic artery phase 16-25 s, the portal vein phase 46-55 s, the hepatic vein phase 86-100 s, the dynamic late 150-180 s and the hepatic multi-phase dynamic scanning of 20min after the hepatocyte phase injection are respectively carried out after the injection, and the single breath-holding time is 16+/-1 s.
The desensitization and pretreatment process of the image data comprises the following steps: the acquired liver nuclear magnetic artery phase image data format enhanced by the common Meinamide contrast agent is a DICOM file, and desensitization operation is carried out on text information in the DICOM header file. Personal information such as patient phones, addresses and the like and hospital information in the personal information are erased so as to ensure privacy. Since the age and sex information of the patient have reference significance for disease diagnosis, the information is reserved.
The image data in all collected DICOM is subjected to signal normalization, and the calculation formula is as follows:
wherein I is i Is the ithSignal value of image of nuclear magnetic artery phase of liver, I' i For the normalized image signal value of the liver nuclear magnetic artery phase,sigma is the signal mean value of all acquired nuclear magnetic images I Signal standard deviation representing all nuclear magnetic images;
as shown in fig. 3 to 5, the image data is marked: at least three clinicians are selected to form a panel of specialities. The panel of experts is required to cover different levels of experience, consisting essentially of: at least one physician with a working experience of 8 years or more; at least one resident, the working experience is 5 years or more; at least one physician has a working experience of 3 years and more. The grading mode adopts a 3-point method recommended by the American radiology institute (ACR), as shown in FIG. 3, the 3-point method represents images with good image quality and is suitable for diagnosis; 2 represents an image of a general but diagnosable quality; score 1 represents an image of poor quality that cannot be used for diagnosis. And (3) carrying out back-to-back quality control grading on the data in the images by the expert group members, if opinion divergence occurs, holding the discussion to determine grading results, and if opinion cannot be unified after the discussion, taking the doctor opinion with the highest job level as the reference.
The image cases subjected to data desensitization, preprocessing and data labeling are arranged in time sequence, the liver nuclear magnetic artery phase image is subjected to gray processing and pixel point segmentation, and a gray image is obtained, wherein the resolution of the gray image is preferably 512 multiplied by 512.
As shown in fig. 2, step S120, selecting 20% of data closest to the acquisition time as test set data, and the remaining 80% of data as training set data, and constructing a convolutional neural network model, extracting hidden features in a gray scale image, and obtaining a pre-trained convolutional neural network model;
the convolutional neural network model introduces a feature dense connection strategy based on the hierarchical connection of the traditional convolutional neural network, increases the feature input of each layer, and aims at the first layer to realize the feature x of l Can be expressed as:
x l ＝H l (x 0 ，x 1 ,…x i …,x l-1 )
wherein x is i For the characteristics of any preamble layer, the deep learning network is constructed to be 24 layers in total, wherein the deep learning network comprises 3 dense connection modules and 2 transition modules. Each dense connection module contains 6 convolution layers with a convolution kernel of 3 x 3. Each dense connection module is connected by 1 transition module. The transition module contains 1 convolution layer with a convolution kernel size of 1×1 and 1 average pooling layer with a kernel size of 2×2.
The gray level image of the liver nuclear magnetic artery phase image is used as an input layer vector to be input into a convolutional neural network model; the convolution god sets the output layer of the network model as a quality grading label of the image of the liver nuclear magnetic artery phase of the common display;
the convolutional neural network model comprises a first dense connection module, a second dense connection module and a third dense connection module; a first transition module is arranged between the first dense connection module and the second dense connection module; a second transition module is arranged between the second dense connecting module and the third dense connecting module;
the dense connection modules comprise 6 convolution layers with convolution kernels of 3 multiplied by 3, and can extract characteristics of gray images of liver nuclear magnetic artery images;
the operation process of the convolution layer is as follows:
the convolution kernel is utilized to slide on the gray level image of the input layer, convolution operation is carried out on the pixel points (i, j) on the gray level image, an output characteristic diagram is obtained, and a convolution operation formula is as follows:
wherein x is l (i, j) is a feature of arbitrary layer l, x l ＝H l (x 0 ，x 1 ,…x i …,x l-1 )，x i Is characteristic of any preamble layer, H l () For batch normalization operations, consisting of an activation function and a convolution operation of size 3 x 3, x l+1 (i, j) is an arbitrary layerOutput characteristics of l, w l+1 (i, j) is the weight parameter of layer 1+1, b l Is the bias value of the first layer;
the size of the output feature map is:
wherein L is l+1 For the output feature map size of arbitrary layer L, L l For the feature size of any layer l of any layer, p is the corresponding filling parameter, f is the corresponding convolution kernel size, and s is the corresponding convolution step size.
The transition modules each include 1 transition convolutional layer with a convolutional kernel size of 1 x 1 and 1 average pooling layer with a kernel size of 2 x 2, which enable the convolutional layer characteristics to be compressed and selected.
Step S130, performing feature visualization on all convolution layers in the pre-training convolution neural network model by using global average pooling operation to obtain a feature visualization heat map;
the operation formula of the pooling layer is as follows:
wherein x is l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size.
Preferably, the feature visualization heat map calculation formula is:
where H is the feature visualization heat map, relu () is the activation function,gradient weight for class c for nth feature map, ++> For the mean value, y of the feature map with n dimensions of i x j c Scoring class c by nth feature map, +.> Weight of category c for nth feature map, ++>Where c=3.
Step S140, screening the feature visualization heat map, selecting a feature layer with a highlight capturing artifact region, and extracting the depth abstract feature of the feature map;
step S150, inputting the deep abstract features of the feature map into a classifier for secondary training to obtain a convolutional neural network model after training, namely a common display liver nuclear magnetic quality control hierarchical model;
the classifier in the fifth step has the following calculation formula:
wherein a is k (X) is an activation function operation at the characteristic channel k and X ε Ω pixel positions, P k (X) represents the output values at the pixel locations of characteristic channel k and X ε Ω.
And step 160, inputting the images of the nuclear magnetic arterial phase of the liver of the patient to be classified into a common display liver nuclear magnetic quality control classification model to obtain a classification prediction result of the images of the nuclear magnetic arterial phase of the common display liver.
In another embodiment, the method also comprises raspberry pi 4, and the deep learning common display liver nuclear magnetic arterial phase image quality grading neural network model of the shutdown method can be transplanted into a hardware device raspberry pi. The raspberry pie is connected with a computer module and a display of the nuclear magnetic resonance apparatus respectively. The method comprises the steps of directly inputting the image of the nuclear magnetic artery phase of the common MEI display liver acquired and reconstructed by a computer module of the nuclear magnetic resonance apparatus into raspberry pie equipment, and then transmitting the quality grading result of the image of the nuclear magnetic artery phase of the common MEI display liver judged by a neural network model in the raspberry pie to a display screen, so as to realize the direct quality control of a data generating end, optimize clinical workflow and improve the working efficiency of doctors and technicians.
The invention provides a supervised learning mode using a deep learning model, which is trained on a large-scale common display liver nuclear magnetic data set marked by a professional radiologist, and integrates human knowledge and experience to realize feature screening, thereby effectively compressing features. The obtained common Mei-show liver nuclear magnetic quality control grading model can be used for more specifically distinguishing nuclear magnetic arterial phase artifacts, and the accuracy of quality control grading is improved.
The invention provides a liver nuclear magnetic arterial phase image quality grading method based on raspberry pie and deep learning, which takes the image data of the liver nuclear magnetic arterial phase of the common display marked by a professional radiologist as a training sample to construct a convolutional neural network model, so that the image of the liver nuclear magnetic arterial phase of the common display can be accurately graded, a characteristic dense connection strategy is introduced, the characteristic input of each layer is increased, the nuclear magnetic arterial phase artifact can be distinguished, and the accuracy of quality control grading is improved.
Although embodiments of the present invention have been disclosed above, it is not limited to the details and embodiments shown and described, it is well suited to various fields of use for which the invention would be readily apparent to those skilled in the art, and accordingly, the invention is not limited to the specific details and illustrations shown and described herein, without departing from the general concepts defined in the claims and their equivalents.
Claims (5)
1. The liver nuclear magnetic artery phase image quality grading method based on raspberry pie and deep learning is characterized by comprising the following steps of:
firstly, acquiring a liver nuclear magnetic artery phase image, preprocessing the liver nuclear magnetic artery phase image to obtain a gray level image, and respectively marking the grading result of the liver nuclear magnetic artery phase image to obtain a quality grading training sample data set;
classifying the training sample data set, constructing a convolutional neural network model, extracting hidden features in a gray level image, and obtaining a pre-trained convolutional neural network model;
the second step comprises the following steps:
inputting the gray level image of the liver nuclear magnetic artery phase image as an input layer vector into a convolutional neural network model; the output layer of the convolutional neural network model is a common display liver nuclear magnetic arterial phase image quality grading label;
the convolutional neural network model comprises a first dense connection module, a second dense connection module and a third dense connection module;
a first transition module is arranged between the first dense connection module and the second dense connection module;
a second transition module is arranged between the second dense connection module and the third dense connection module;
the dense connection modules comprise 6 convolution layers with convolution kernels of 3 multiplied by 3, and can extract features of gray images of the liver nuclear magnetic arterial phase images;
the transition modules comprise 1 transition convolution layer with a convolution kernel size of 1 multiplied by 1 and 1 average pooling layer with a kernel size of 2 multiplied by 2, and can compress and select the characteristics of the convolution layers;
the operation process of the convolution layer is as follows:
and sliding on the gray level image of the input layer by utilizing a convolution kernel, and sequentially carrying out convolution operation on pixel points on the gray level image to obtain an output feature map, wherein the convolution operation formula is as follows:
wherein x is l (i, j) is arbitraryCharacteristics of layer l, x l ＝H l (x 0 ，x 1 ,…x i …,x l-1 )，x i Is characteristic of any preamble layer, H l () For batch normalization operations, consisting of an activation function and a convolution operation of size 3 x 3, x l+1 (i, j) is the output feature of arbitrary layer l, w l+1 (i, j) is the weight parameter of layer 1+1, b l Is the deviation value of the first layer; the size of the output characteristic diagram is as follows:wherein L is l+1 For the output feature map size of arbitrary layer L, L l For the characteristic size of any layer l of any layer, p is a corresponding filling parameter, f is a corresponding convolution kernel size, and s is a corresponding convolution step length;
step three, utilizing a gradient weighting type activation mapping method to perform feature visualization on all convolution layers in the pre-training convolution neural network model to obtain a feature visualization heat map;
screening the feature visualization heat map, selecting a feature layer with a highlight capturing artifact region, and extracting the depth abstract feature of the output feature map;
inputting the depth abstract features of the output feature map into a classifier for secondary training to obtain a training-completed common display liver nuclear magnetic quality control grading model;
inputting the images of the nuclear magnetic arteries of the liver of the patient to be classified into a common display liver nuclear magnetic quality control classification model to obtain a classification prediction result of the images of the nuclear magnetic arteries of the common display liver.
2. The method for grading the image quality of the liver nuclear magnetic arterial phase based on raspberry pie and deep learning according to claim 1, wherein the liver nuclear magnetic arterial phase image preprocessing process in the first step comprises the following steps:
firstly, carrying out signal normalization on the acquired liver nuclear magnetic artery phase image, wherein the calculation formula is as follows:
wherein I is i Is the signal value of the ith liver nuclear magnetic artery phase image, I' i For the normalized image signal value of the liver nuclear magnetic artery phase,sigma is the signal mean value of all acquired nuclear magnetic images I Signal standard deviation representing all nuclear magnetic images;
and then, carrying out graying treatment on the liver nuclear magnetic artery phase image and carrying out pixel point segmentation to obtain a gray image.
3. The method for grading the image quality of the liver nuclear magnetic artery phase based on raspberry pie and deep learning according to claim 1, wherein the calculation formula of the pooling layer is as follows:
wherein x is l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size.
4. The method for grading the image quality of the liver nuclear magnetic artery phase based on raspberry pie and deep learning according to claim 1 or 3, wherein the gradient weighted category activation mapping method in the third step comprises:
where H is the feature visualization heat map, relu is the activation function,for the gradient weight of the nth feature map to category c, for the mean value, y of the feature map with n dimensions of i x j c Scoring class c by the nth feature map,
weight of category c for nth feature map, ++>Where c=3.
5. The method for grading the image quality of the liver nuclear magnetic resonance arterial phase based on raspberry pie and deep learning according to claim 1, wherein the common display liver nuclear magnetic resonance quality control grading model is stored in raspberry pie equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911157464.2A CN110827275B (en) | 2019-11-22 | 2019-11-22 | Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911157464.2A CN110827275B (en) | 2019-11-22 | 2019-11-22 | Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110827275A CN110827275A (en) | 2020-02-21 |
| CN110827275B true CN110827275B (en) | 2023-12-22 |
Family
ID=69558656
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911157464.2A Active CN110827275B (en) | 2019-11-22 | 2019-11-22 | Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110827275B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111554402A (en) * | 2020-04-24 | 2020-08-18 | 山东省立医院 | Machine learning-based method and system for predicting postoperative recurrence risk of primary liver cancer |
| CN112257769B (en) * | 2020-10-19 | 2022-10-21 | 中国科学院自动化研究所 | Multilayer nuclear magnetic image classification method and system based on reinforcement learning type brain reading |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651830A (en) * | 2016-09-28 | 2017-05-10 | 华南理工大学 | Image quality test method based on parallel convolutional neural network |
| CN107506797A (en) * | 2017-08-25 | 2017-12-22 | 电子科技大学 | One kind is based on deep neural network and multi-modal image alzheimer disease sorting technique |
| CN109472774A (en) * | 2018-10-11 | 2019-03-15 | 上海中医药大学 | A kind of tongue picture picture quality detection method based on deep learning |
| CN109523514A (en) * | 2018-10-18 | 2019-03-26 | 西安电子科技大学 | To the batch imaging quality assessment method of Inverse Synthetic Aperture Radar ISAR |
| CN109767429A (en) * | 2018-12-28 | 2019-05-17 | 上海联影智能医疗科技有限公司 | A kind of image screening method and device |
| US10304193B1 (en) * | 2018-08-17 | 2019-05-28 | 12 Sigma Technologies | Image segmentation and object detection using fully convolutional neural network |
| WO2019133538A2 (en) * | 2017-12-29 | 2019-07-04 | Leica Biosystems Imaging, Inc. | Processing of histology images with a convolutional neural network to identify tumors |
| CN110246126A (en) * | 2019-06-14 | 2019-09-17 | 吉林大学第一医院 | A method of extracting terminal bronchi tree from lung CT image |
| CN110335259A (en) * | 2019-06-25 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of medical image recognition methods, device and storage medium |
| CN110414494A (en) * | 2019-01-25 | 2019-11-05 | 西安电子科技大学 | SAR image classification method with ASPP deconvolution network |
-
2019
- 2019-11-22 CN CN201911157464.2A patent/CN110827275B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106651830A (en) * | 2016-09-28 | 2017-05-10 | 华南理工大学 | Image quality test method based on parallel convolutional neural network |
| CN107506797A (en) * | 2017-08-25 | 2017-12-22 | 电子科技大学 | One kind is based on deep neural network and multi-modal image alzheimer disease sorting technique |
| WO2019133538A2 (en) * | 2017-12-29 | 2019-07-04 | Leica Biosystems Imaging, Inc. | Processing of histology images with a convolutional neural network to identify tumors |
| US10304193B1 (en) * | 2018-08-17 | 2019-05-28 | 12 Sigma Technologies | Image segmentation and object detection using fully convolutional neural network |
| CN109472774A (en) * | 2018-10-11 | 2019-03-15 | 上海中医药大学 | A kind of tongue picture picture quality detection method based on deep learning |
| CN109523514A (en) * | 2018-10-18 | 2019-03-26 | 西安电子科技大学 | To the batch imaging quality assessment method of Inverse Synthetic Aperture Radar ISAR |
| CN109767429A (en) * | 2018-12-28 | 2019-05-17 | 上海联影智能医疗科技有限公司 | A kind of image screening method and device |
| CN110414494A (en) * | 2019-01-25 | 2019-11-05 | 西安电子科技大学 | SAR image classification method with ASPP deconvolution network |
| CN110246126A (en) * | 2019-06-14 | 2019-09-17 | 吉林大学第一医院 | A method of extracting terminal bronchi tree from lung CT image |
| CN110335259A (en) * | 2019-06-25 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of medical image recognition methods, device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 陈寿宏 ; 柳馨雨 ; 马峻 ; 康怀强 ; .深度卷积神经网络胸片肺结节分类识别研究.计算机工程与应用.2018,(第24期),181-186页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110827275A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108573490B (en) | Intelligent film reading system for tumor image data | |
| Zhou et al. | A radiomics approach with CNN for shear-wave elastography breast tumor classification | |
| CN106372390B (en) | A kind of self-service healthy cloud service system of prevention lung cancer based on depth convolutional neural networks | |
| CN110517238B (en) | AI three-dimensional reconstruction and human-computer interaction visualization network system for CT medical image | |
| CN112086197B (en) | Breast nodule detection method and system based on ultrasonic medicine | |
| JP7218215B2 (en) | Image diagnosis device, image processing method and program | |
| CN110946619B (en) | Intravascular ultrasound automatic image histology analysis system and analysis method | |
| CN109255354B (en) | Medical CT-oriented computer image processing method and device | |
| CN105934199A (en) | Apparatus and method for processing a medical image of a body lumen | |
| CN108766555A (en) | The computer diagnosis method and system of Pancreatic Neuroendocrine Tumors grade malignancy | |
| CN112489788B (en) | Multi-mode image analysis method and system for cancer diagnosis | |
| CN110827275B (en) | Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning | |
| CN110292396A (en) | The prediction of quantitative imaging uses | |
| CN111312373B (en) | Automatic labeling method for PET/CT image fusion | |
| CN114241187A (en) | Muscle disease diagnosis system, device and medium based on ultrasonic bimodal images | |
| CN114398979A (en) | Ultrasonic image thyroid nodule classification method based on feature decoupling | |
| Zhang et al. | Comparison of multiple feature extractors on Faster RCNN for breast tumor detection | |
| CN113889261A (en) | Pathological feature assistance-based PET/CT automatic lung cancer diagnosis classification model training method | |
| Zeng et al. | Machine Learning-Based Medical Imaging Detection and Diagnostic Assistance | |
| CN117562583B (en) | Artificial intelligence aided heart function detection system and method | |
| Kumar et al. | A Novel Approach for Breast Cancer Detection by Mammograms | |
| CN112819740A (en) | Medical image fusion method based on multi-component low-rank dictionary learning | |
| Hussein et al. | Myositis Detection From Muscle Ultrasound Images Using a Proposed YOLO-CSE Model | |
| CN117017310A (en) | Acoustic-electric dual-mode congenital heart disease prediction device based on knowledge distillation | |
| CN112132790B (en) | DAC-GAN model construction method and application thereof in mammary gland MR image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |