CN110827275B - Liver nuclear magnetic artery image quality grading method based on raspberry pie and deep learning - Google Patents

Abstract

The present invention provides a method for grading the quality of liver MRI arterial phase images based on Raspberry Pi and deep learning, which includes: Step 1. Collect liver MRI arterial phase images and perform preprocessing to obtain grayscale images, and obtain quality graded training sample data. Set; Step 2, classify the training sample data set and build a convolutional neural network model; Step 3, use the global average pooling operation to perform feature visualization operation to obtain the feature visualization heat map; Step 4, filter the feature visualization heat map; Step 5 , input the deep abstract features of the feature map into the classifier for secondary training, and obtain the grading model of the quality control of the liver MRI of the PMU; Step 6: Input the arterial phase MRI image of the liver of the patient to be classified, and obtain the arterial phase image of the liver MRI of the patient to be classified. According to the grading prediction results, the convolutional neural network model constructed by the present invention can accurately grade the quality of the arterial phase images of the liver magnetic resonance imaging.

Description

Quality grading method of liver MRI arterial phase images based on Raspberry Pi and deep learning

Technical field

The invention relates to the field of medical image processing and analysis, and in particular to a liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning.

Background technique

my country is a country with a high incidence of liver disease. Among them, hepatocellular carcinoma, or liver cancer for short, is a common malignant tumor in the Chinese population. According to statistics, about 330,000 people die from liver cancer in my country every year, accounting for nearly 50% of liver cancer deaths in the world. Early screening, early diagnosis, and early treatment of liver cancer patients can greatly extend their survival period. Various imaging examinations, including computed tomography and magnetic resonance imaging (MRI), can non-invasively monitor the tissue morphology and lesion characteristics of liver diseases, and have become one of the important means for early screening of liver cancer.

Primax is a new type of MRI contrast agent. About 50% of the contrast agent in the blood can be absorbed by liver cells, allowing MRI to accurately provide blood supply information of liver lesions and liver cell information during the diagnosis process. A multi-center phase 3 clinical trial in the United States has proven in 2010 that Primax can improve the qualitative diagnosis rate of liver lesions and has good safety. Therefore, Premexan has been regarded by clinicians as a valuable hepatobiliary-specific contrast agent. With the widespread application of Promexan in the diagnosis of clinical liver diseases, its shortcomings have gradually emerged. Clinicians have found that arterial phase images of liver MRI often exhibit artifacts. The imaging manifestations of artifacts vary and their causes are different. These artifacts will greatly affect the imaging quality of MRI, and in serious cases may lead to missed diagnosis, misdiagnosis, or even inability to diagnose.

Obtaining high-quality medical images is the prerequisite for accurate imaging diagnosis. Quality control of the liver's MRI arterial phase images can improve the quality of MRI images and improve the accuracy of clinical diagnosis. Western countries have long implemented a medical imaging quality control system, which is mainly completed by manual review by medical physicists and engineering technicians in the radiology department. However, my country's clinical quality control work started late and the relevant systems are not yet perfect. Medical physicists or engineering technicians in the radiology department lack clinical diagnostic experience and knowledge and cannot accurately grasp the key points of image quality control.

In recent years, with the rapid development of artificial intelligence technology, a large number of automated diagnosis or quantitative assessments of liver diseases based on imaging have emerged, such as automatic segmentation of liver and liver tumor areas based on fully convolutional neural network FCN, and automatic segmentation of liver and liver tumor areas based on convolutional neural network. CNN's liver cirrhosis classification, liver cancer classification, liver cancer survival prediction, liver injury classification based on radiomics and machine learning models, etc. These preliminary studies have proven that mining digital features of medical images and training artificial intelligence models can be used to assist in the analysis of liver diseases.

To sum up, it is necessary to use deep learning models to fully automatically complete the quality control of primary and secondary liver MRI, optimize the existing workflow, improve the quality of primary and secondary liver MRI in clinical practice, and effectively help clinicians reduce work pressure and improve diagnostic efficiency. , benefiting patients.

Contents of the invention

The present invention provides a method for grading the quality of liver MRI arterial phase images based on Raspberry Pi and deep learning. It uses the PUMEX liver MRI arterial phase image data marked by professional radiologists as training samples to build a convolutional neural network model. It can accurately grade the quality of the arterial phase images of liver magnetic resonance imaging with general imaging.

Another purpose of the present invention is to introduce a feature dense connection strategy, increase the feature input of each layer, be able to distinguish nuclear magnetic arterial phase artifacts, and improve the accuracy of quality control grading.

A liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning, including:

Step 1: Collect liver MRI arterial phase images and perform preprocessing to obtain grayscale images, and mark the liver MRI arterial phase image grading results respectively to obtain a quality-graded training sample data set;

Step 2: Classify the training sample data set and construct a convolutional neural network model, extract hidden features in the grayscale image, and obtain a pre-trained convolutional neural network model;

Step 3: Use the gradient weighted category activation mapping method to visualize the features of all convolutional layers in the pre-trained convolutional neural network model to obtain a feature visualization heat map;

Step 4: Screen the feature visualization heat map, select the feature layer with highlighted capture artifact areas, and extract the deep abstract features of the feature map;

Step 5: Input the deep abstract features of the feature map into the classifier for secondary training to obtain the trained PUMIX liver MRI quality control grading model;

Step 6: Build a quality control grading model for the quality control of the Liver MRI in the Raspberry Pi device, and input the patient's liver MRI arterial phase images to be classified to obtain the grading prediction results of the Liver MRI arterial phase images of the Primex.

Preferably, the liver MRI arterial phase image preprocessing process in step one includes:

First, the acquired liver MRI arterial phase images are signal normalized, and the calculation formula is:

Among them, I _i is the signal value of the i-th liver MRI arterial phase image, I′ _i is the signal value of the normalized liver MRI arterial phase image, is the signal mean of all acquired nuclear magnetic images, and σ _I represents the signal standard deviation of all nuclear magnetic images;

Then, the liver MRI arterial phase image is grayscaled and segmented into pixels to obtain a grayscale image.

Preferably, the second step includes:

The grayscale image of the liver MRI arterial phase image is input into the convolutional neural network model as an input layer vector; the output layer of the convolutional neural network model is the PUMIX liver MRI arterial phase image quality grading label;

The convolutional neural network model includes a first dense connection module, a second dense connection module and a third dense connection module;

There is a first transition module between the first densely connected module and the second densely connected module;

There is a second transition module between the second densely connected module and the third densely connected module;

Among them, the dense connection modules each include 6 convolution layers with a convolution kernel of 3×3, which can perform feature extraction on the grayscale image of the liver MRI arterial phase image;

The transition modules each include a transition convolution layer with a convolution kernel size of 1×1 and an average pooling layer with a kernel size of 2×2, which can compress and select convolution layer features.

Preferably, the operation process of the convolution layer is:

Use the convolution kernel to slide on the grayscale image of the input layer, and perform a convolution operation on the pixel points (i, j) on the grayscale image to obtain the output feature map. The convolution operation formula is:

Among them, x _l (i, j) is the feature of any layer l, x _l =H _l (x ₀ , x ₁ ,...x _i ...,x _l-1 ), x _i is the feature of any pre-sequence layer, H _l () is a batch normalization operation, consisting of an activation function and a convolution operation of size 3×3, x _l+1 (i, j) is the output feature of any layer l, w _l+1 (i, j) is The weight parameter of layer l+1, b _l is the bias value of layer l;

The size of the output feature map is:

Among them, L _l+1 is the output feature map size of any layer l, L _l is the feature size of any layer l, p is the corresponding filling parameter, f is the corresponding convolution kernel size, and s is the corresponding convolution step length.

Preferably, the convolution kernel size is 3×3.

Preferably, the operation formula of the pooling layer is:

Among them, x _l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size.

Preferably, the calculation formula of the feature visualization heat map is:

Among them, H is the feature visualization heat map, relu is the activation function, is the gradient weight of the nth feature map to category c,/> is the mean value of n feature maps with dimensions i×j, y ^c is the score of the nth feature map for category c,/> is the weight of category c to the nth feature map,/> where c=3.

Preferably, the classifier calculation formula in step five is:

Among them, a _k (X) is the activation function operation at the feature channel k and X∈Ω pixel position, and P _k (X) is the output value at the feature channel k and X∈Ω pixel position.

Preferably, the PUMEX liver MRI quality control grading model is stored in the Raspberry Pi device.

Beneficial effects of the invention

The present invention provides a supervised learning model using a deep learning model, which is trained on a large-scale general imaging liver MRI data set marked by professional radiologists, and integrates human knowledge and experience to implement feature screening, thereby effectively compressing features. The trained PUMEX liver MRI quality control grading model can more specifically distinguish MRI arterial phase artifacts and improve the accuracy of quality control grading.

Description of the drawings

Figure 1 is a flow chart of the liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning according to the present invention.

Figure 2 is a network structure diagram of the deep learning neural network model of the present invention.

Figure 3 is a schematic diagram of a 1-point sample of the arterial phase image quality grading of the liver magnetic resonance imaging using PUMEX according to the present invention.

Figure 4 is a schematic diagram of a 2-point sample of the arterial phase image quality classification of the liver magnetic resonance imaging performed by PUMEX according to the present invention.

Figure 5 is a schematic diagram of a 3-point sample of the arterial phase image quality classification of the liver magnetic resonance imaging performed by PUMEX according to the present invention.

Figure 6 is a connection structure diagram of the Raspberry Pi device according to the present invention.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings, so that those skilled in the art can implement it with reference to the text of the description.

As shown in Figure 1, the liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning provided by the present invention includes:

Step S110: Collect liver MRI arterial phase images and mark the grading results respectively, preprocess the liver MRI arterial phase images to obtain grayscale images, and obtain a quality-graded training sample data set;

The liver MRI arterial phase image acquisition method is:

All patients received breath-holding training before examination. Patients are also required to abstain from water in the early morning of the examination day and fast for 4 hours before the examination.

The inspection equipment uses a superconducting ultra-high field nuclear magnetic scanner. The maximum gradient field strengths are: 40mT/m for the X-axis and Y-axis, 45mT/m for the Z-axis, and the gradient field switching rate is 200mT/m/m. The surface coil Using 8-channel phased array body coil.

Before and after contrast agent injection, T1WI of fast three-dimensional spoiled gradient echo pulse sequence was performed. Specific parameters: TR/TE=4.19/4.17ms, flip angle 9°, field of view 300×400mm, matrix 168×320, reorganization The minimum layer thickness is 2.5~3.5mm.

The Promax contrast agent is injected through an artificial intravenous high-pressure syringe with an injection dose of 0.1 mL/kg and a flow rate of 2 mL/s. Immediately after the injection of the contrast agent, 20 mL of normal saline is used for flushing. The flush flow rate is 2 mL/s. Injection After the drug was administered, multi-phase dynamic scans of the liver were performed for 16 to 25 seconds in the hepatic artery phase, 46 to 55 seconds in the portal venous phase, 86 to 100 seconds in the hepatic venous phase, 150 to 180 seconds in the late dynamic phase, and 20 minutes after drug injection in the hepatocellular phase. The single breath-holding time was 16 ±1s.

The desensitization and preprocessing process of the image data is as follows: the collected Pulmax contrast agent-enhanced liver MRI arterial phase image data format is a DICOM file, and the text information in the DICOM header file is desensitized. Erase personal information such as patient phone number, address, and hospital information to ensure privacy. Because the patient's age and gender information are of reference significance for disease diagnosis, they are retained.

The signal is normalized for all collected image data in DICOM. The calculation formula is:

Among them, I _i is the signal value of the i-th liver MRI arterial phase image, I′ _i is the signal value of the normalized liver MRI arterial phase image, is the signal mean of all acquired nuclear magnetic images, and σ _I represents the signal standard deviation of all nuclear magnetic images;

As shown in Figures 3 to 5, image data annotation: select at least three clinical radiologists to form an expert group. The expert panel is required to cover different levels of experience, and its general composition is as follows: at least one attending physician with 8 years or more of work experience; at least one resident physician with 5 years or more of work experience; at least one physician with 5 years of work experience or more In 3 years and above. The grading method adopts the 3-point system recommended by the American College of Radiology (ACR), as shown in Figure 3. A score of 3 represents an image with good image quality and suitable for diagnosis; a score of 2 represents an image with average image quality but still diagnostic; 1 A score of 10 points represents images of poor quality that cannot be used for diagnosis. Members of the expert panel conduct back-to-back quality control grading of the data in the images. If there are differences of opinion, a discussion will be held to determine the grading results. If consensus cannot be reached after the discussion, the opinion of the highest-ranking doctor shall prevail.

The imaging cases that have undergone data desensitization, preprocessing and data annotation are arranged in chronological order. The liver MRI arterial phase images are grayscaled and segmented into pixels to obtain a grayscale image. As a preferred option, the grayscale image The resolution is 512×512.

As shown in Figure 2, step S120 selects the 20% of the data closest to the collection time as the test set data, and the remaining 80% of the data as the training set data, and builds a convolutional neural network model to extract hidden features in the grayscale image. , get the pre-trained convolutional neural network model;

Based on the hierarchical connections of traditional convolutional neural networks, the convolutional neural network model introduces a dense feature connection strategy and increases the feature input of each layer. For the lth layer, its feature x _l can be expressed as:

x _l =H _l (x ₀ , x ₁ ,...x _i ...,x _l-1 )

Among them, x _i is the feature of any preorder layer, and the constructed deep learning network has a total of 24 layers, including 3 dense connection modules and 2 transition modules. Each densely connected module contains 6 convolutional layers with a convolution kernel of 3×3. Each densely connected module is connected by a transition module. The transition module includes a convolutional layer with a convolution kernel size of 1×1 and an average pooling layer with a kernel size of 2×2.

The grayscale image of the liver MRI arterial phase image is input into the convolutional neural network model as the input layer vector; the output layer of the convolutional neural network model is the PUMIX liver MRI arterial phase image quality classification label;

The convolutional neural network model includes a first dense connection module, a second dense connection module and a third dense connection module; there is a first transition module between the first dense connection module and the second dense connection module; a second dense connection module There is a second transition module between the third densely connected module;

Among them, the dense connection modules include 6 convolution layers with a convolution kernel of 3×3, which can extract features from grayscale images of liver MRI arterial phase images;

The operation process of the convolution layer is:

Use the convolution kernel to slide on the grayscale image of the input layer, and perform a convolution operation on the pixel points (i, j) on the grayscale image to obtain the output feature map. The convolution operation formula is:

Among them, x _l (i, j) is the feature of any layer l, x _l =H _l (x ₀ , x ₁ ,...x _i ...,x _l-1 ), x _i is the feature of any pre-sequence layer, H _l () is a batch normalization operation, consisting of an activation function and a convolution operation of size 3×3, x _l+1 (i, j) is the output feature of any layer l, w _l+1 (i, j) is The weight parameter of layer l+1, b _l is the bias value of layer l;

The size of the output feature map is:

Among them, L _l+1 is the output feature map size of any layer l, L _l is the feature size of any layer l, p is the corresponding filling parameter, f is the corresponding convolution kernel size, and s is the corresponding convolution step length.

The transition modules each include a transition convolution layer with a convolution kernel size of 1×1 and an average pooling layer with a kernel size of 2×2, which can compress and select convolution layer features.

Step S130: Use the global average pooling operation to perform feature visualization on all convolutional layers in the pre-trained convolutional neural network model to obtain a feature visualization heat map;

The operation formula of the pooling layer is:

Among them, x _l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size.

Preferably, the calculation formula of the feature visualization heat map is:

Among them, H is the feature visualization heat map, relu() is the activation function, is the gradient weight of the nth feature map to category c,/> is the mean value of n feature maps with dimensions i×j, y ^c is the score of the nth feature map for category c,/> is the weight of category c to the nth feature map,/> where c=3.

Step S140: Screen the feature visualization heat map, select feature layers with highlighted capture artifact areas, and extract deep abstract features of the feature map;

Step S150: Input the deep abstract features of the feature map into the classifier for secondary training to obtain the trained convolutional neural network model, which is the PUMEX liver MRI quality control grading model;

The classifier calculation formula in step five is:

Among them, a _k (X) is the activation function operation at the feature channel k and X∈Ω pixel position, and P _k (X) represents the output value at the feature channel k and X∈Ω pixel position.

Step S160: Input the patient's liver MRI arterial phase images to be classified into the Premex liver MRI quality control grading model to obtain the grading prediction results of the Premex liver MRI arterial phase images.

In another embodiment, a Raspberry Pi 4 is also included, and the deep learning PMU arterial phase image quality classification neural network model of the present invention can be transplanted to the hardware device Raspberry Pi. Connect the Raspberry Pi to the computer module and display of the nuclear magnetic resonance instrument respectively. The arterial phase images of the liver MRI collected and reconstructed by the computer module of the MRI machine are directly input into the Raspberry Pi device, and then the arterial phase images of the liver MRI identified by the neural network model in the Raspberry Pi are The image quality grading results are transmitted to the display screen to achieve direct quality control at the data generation end, optimize clinical workflow, and improve the work efficiency of doctors and technicians.

The present invention provides a supervised learning model using a deep learning model, which is trained on a large-scale general imaging liver MRI data set marked by professional radiologists, and integrates human knowledge and experience to implement feature screening, thereby effectively compressing features. The trained PUMEX liver MRI quality control grading model can more specifically distinguish MRI arterial phase artifacts and improve the accuracy of quality control grading.

The present invention provides a method for grading the quality of liver MRI arterial phase images based on Raspberry Pi and deep learning. It uses the PUMEX liver MRI arterial phase image data marked by professional radiologists as training samples to build a convolutional neural network model. It can accurately grade the quality of the arterial phase images of liver MRI with general imaging. It also introduces a feature dense connection strategy, increases the feature input of each layer, can distinguish the arterial phase artifacts of MRI, and improves the accuracy of quality control grading.

Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and embodiments. They can be applied to various fields suitable for the present invention. For those familiar with the art, they can easily Additional modifications may be made, and the invention is therefore not limited to the specific details and illustrations shown and described herein without departing from the general concept defined by the claims and equivalent scope.

Claims (5)

1. A liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning, which is characterized by including: Step 1: Collect liver MRI arterial phase images and perform preprocessing to obtain grayscale images, and mark the liver MRI arterial phase image grading results respectively to obtain a quality-graded training sample data set; Step 2: Classify the training sample data set and construct a convolutional neural network model, extract hidden features in the grayscale image, and obtain a pre-trained convolutional neural network model; The second step includes: The grayscale image of the liver MRI arterial phase image is input into the convolutional neural network model as an input layer vector; the output layer of the convolutional neural network model is the PUMEX liver MRI arterial phase image quality grading label; The convolutional neural network model includes a first dense connection module, a second dense connection module and a third dense connection module; There is a first transition module between the first densely connected module and the second densely connected module; There is a second transition module between the second densely connected module and the third densely connected module; Wherein, the dense connection modules each include 6 convolution layers with a convolution kernel of 3×3, which can perform feature extraction on the grayscale image of the liver MRI arterial phase image; The transition modules each include a transition convolution layer with a convolution kernel size of 1×1 and an average pooling layer with a kernel size of 2×2, which can compress and select convolution layer features; The operation process of the convolution layer is: Use the convolution kernel to slide on the grayscale image of the input layer, and perform convolution operations on the pixels on the grayscale image in sequence to obtain the output feature map. The convolution operation formula is: Among them, x _l (i, j) is the feature of any layer l, x _l =H _l (x ₀ , x ₁ ,...x _i ...,x _l-1 ), x _i is the feature of any pre-sequence layer, H _l () is a batch normalization operation, consisting of an activation function and a convolution operation of size 3×3, x _l+1 (i, j) is the output feature of any layer l, w _l+1 (i, j) is The weight parameter of the l+1th layer, b _l is the bias value of the lth layer; the size of the output feature map is: Among them, L _l+1 is the output feature map size of any layer l, L _l is the feature size of any layer l, p is the corresponding filling parameter, f is the corresponding convolution kernel size, and s is the corresponding convolution step length; Step 3: Use the gradient weighted category activation mapping method to visualize the features of all convolutional layers in the pre-trained convolutional neural network model to obtain a feature visualization heat map; Step 4: Screen the feature visualization heat map, select the feature layer with highlighted capture artifact areas, and extract the deep abstract features of the output feature map; Step 5: Input the deep abstract features of the output feature map into the classifier for secondary training to obtain the trained PUMIX liver MRI quality control grading model; Step 6: The patient's liver MRI arterial phase images to be classified are input into the Premex liver MRI quality control grading model to obtain the grading prediction results of the Premex liver MRI arterial phase images. 2. The liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning according to claim 1, characterized in that the liver MRI arterial phase image preprocessing process in step one includes: First, the acquired liver MRI arterial phase images are signal normalized, and the calculation formula is: Among them, I _i is the signal value of the i-th liver MRI arterial phase image, I′ _i is the signal value of the normalized liver MRI arterial phase image, is the signal mean of all acquired nuclear magnetic images, and σ _I represents the signal standard deviation of all nuclear magnetic images; Then, the liver MRI arterial phase image is grayscaled and pixel segmented to obtain a grayscale image. 3. The liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning according to claim 1, characterized in that the calculation formula of the pooling layer is: Among them, x _l (i, j) is the feature of any layer l, d is the corresponding pooling kernel size, and r is the corresponding pooling step size. 4. The liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning according to claim 1 or 3, characterized in that the gradient weighted category activation mapping method in step three includes: Among them, H is the feature visualization heat map, relu is the activation function, is the gradient weight of the nth feature map to category c, is the mean value of n feature maps with dimensions i×j, y ^c is the score of the nth feature map for category c, is the weight of category c to the nth feature map,/> where c=3. 5. The liver MRI arterial phase image quality grading method based on Raspberry Pi and deep learning according to claim 1, characterized in that the PUMEX liver MRI quality control grading model is stored in the Raspberry Pi device.