CN116630324B - Method for automatically evaluating adenoid hypertrophy by MRI (magnetic resonance imaging) image based on deep learning - Google Patents

Abstract

The invention is applicable to the technical field of medical image processing, and provides a method for automatically evaluating adenoid hypertrophy by MRI images based on deep learning, which comprises the following steps: converting the acquired MRI image in DICOM format into PNG format, selecting one frame with nasal septum and several frames around the frame as data set from sagittal image sequence, and preprocessing the image; image enhancement, expanding a data set; making a label; extracting features of the image; four landmarks are automatically located in the image and the ratio of the adenoid thickness to the nasopharyngeal cavity gap (AN ratio) is calculated to evaluate whether the image corresponds to a patient with adenoid hypertrophy. According to the invention, by automatically positioning four landmarks and further calculating the AN ratio, the automatic evaluation of whether the patient has adenoid hypertrophy or not is realized, the repeated and time-consuming measurement work of doctors is reduced, and the doctors are assisted to evaluate and diagnose the adenoid hypertrophy.

Description

Method for automatically evaluating adenoid hypertrophy by MRI (magnetic resonance imaging) image based on deep learning

Technical Field

The invention belongs to the technical field of medical image processing, and particularly relates to a method for automatically evaluating adenoid hypertrophy by using MRI images based on deep learning.

Background

The adenoids are a mass of lymphoid tissue located in the posterior wall of the top of the nasopharynx. The adenoids can be physiologically hypertrophic at the age of 2-10 years, but local infection and inflammatory stimulus can cause the pathological hypertrophy of the adenoids. For children, the frequency of upper respiratory tract infections increases due to the fact that adenoid hypertrophy encroaches on the airways, resulting in a relatively small nasopharyngeal volume. Other problems that may be caused by adenoid hypertrophy include maxillofacial dysplasia, excessive daytime sleepiness, impaired cognitive function, poor learning performance, and the like. Depending on the symptoms and course of disease in children, mild symptoms can inhibit continued hypertrophy of the adenoids by symptomatic conservative treatment, and the like, which self-atrophy, but when the adenoids are severely hypertrophic, surgical excision may be required. The adenoid hypertrophy has great harm to children and the incidence rate is remarkably increased in recent years, and the timely detection and diagnosis have important significance for early treatment, disease course control and the like.

Currently, the examination of childhood adenoid hypertrophy mainly includes nasopharyngeal lateral images and flexible nasopharyngeal microscopy. However, the invasiveness of flexible nasopharyngeal mirrors makes many children difficult to cooperate with doctors in preoperative glandular assessment, thereby limiting their use in clinical diagnosis. Therefore, nasopharyngeal side imaging is the most commonly used examination tool for infants suffering from adenoid hypertrophy, and the degree of adenoid hypertrophy and nasopharyngeal cavity obstruction is judged by measuring the adenoid/nasopharyngeal cavity (A/N) ratio on the nasopharyngeal side image, so that a basis is provided for targeted treatment. There are difficulties in manually measuring the various indices on the image, which results in a large number of errors and individual differences between measuring physicians, the accuracy of the identification is largely dependent on the clinical experience of the doctors, inaccurate identification may lead to incorrect evaluation results, and such evaluation is time-consuming and requires repeated work. For this reason we propose a method for automatically assessing adenoid hypertrophy based on deep learning MRI images.

Disclosure of Invention

The invention aims to provide a method for automatically evaluating adenoid hypertrophy based on a deep learning MRI image, which aims to solve the problems in the background art.

In order to achieve the above purpose, the present invention provides the following technical solutions:

a method for automatically assessing adenoid hypertrophy based on deep learning MRI images, comprising the steps of:

step A, converting all acquired MRI images from a DICOM format to a PNG format, selecting one frame of a nasal septum and a plurality of left and right frames of images thereof from a coronal bit sequence, carrying out gray scale normalization preprocessing, unifying pixel values to a [0,1] interval, cutting into a picture of 560pix x 640pix, and marking a data set;

step B, enhancing the image processed in the step A, expanding a data set, and dividing the expanded data set into three parts, namely a training set, a verification set and a test set;

step C, constructing an adenoidet adeno sample evaluation network model based on a convolutional neural network;

step D, training the Adenoidenet by using the training set and the verification set obtained in the step B to generate a training model;

and E, testing the training model generated in the step D by using the test set obtained in the step B.

Further, in the step B, the specific operation of expanding the data set is as follows:

the image data is flipped horizontally, rotated by different angles and scaled.

Further, in the step C, the CNN-based adenoidenet adenovirus evaluation network model includes:

an encoder module for extracting local features by convolution;

a decoder module for recovering image resolution and capturing long-range dependencies;

and the landmark detection module is used for realizing landmark positioning.

Further, the specific operation of the step C is as follows: the input PNG image is subjected to information loss caused by downsampling through a coder module and a decoder module, the final characteristics are extracted and obtained, the characteristics are sent into a landmark detection module, the coordinates of four finally predicted landmarks are obtained, and finally the AN ratio is calculated.

Further, in the step C:

in the encoder, input data firstly passes through two convolution layers, namely a BN layer and a ReLU layer, the convolution kernel size of each convolution layer is 3*3, the step length is 1, the output of each convolution layer enters the BN layer and the ReLU layer, the output of the last ReLU layer enters a pooling layer, the pooling mode is maximum pooling, and the pooling window size is 2; then three repeated sub-modules based on depth separable convolution are adopted, each module comprises a convolution layer, an LN layer and two full connection layers, the convolution kernel size of the convolution layer is 7*7, the step length is 1, the input of the convolution layer enters the LN layer, then the input of the convolution layer sequentially passes through the full connection layers and the GELU layers, the output of the last GELU layer in each module enters a pooling layer, the pooling mode is the maximum pooling, the pooling window size is 2, and the local characteristics are finally obtained; the local feature is then input into a decoder;

in the decoder, the input features pass through three sub-modules based on adaptive convolution, each module comprises a convolution layer, a BN layer, a ReLU layer, an adaptive convolution layer and an upsampling layer, the convolution kernel of the convolution layer has a size of 3*3, the step size is 1, the output of the convolution layer enters the BN layer and the ReLU layer, and then enters the adaptive convolution layer, and the adaptive convolution layer comprises two branches: one branch is a standard 3*3 convolution layer, a BN layer and a ReLU layer, the other branch is a deformable convolution, the outputs of the two branches are added to obtain a final output, the final output enters an up-sampling layer, the up-sampling uses transposed convolution, the convolution kernel size is 3*3, and the step length is 1; the output of the last up-sampling layer enters a landmark detection module;

in the landmark detection module, the feature obtains a heat map through a convolution layer, and then the convolution layer outputs and enters a DSNT layer to obtain a final coordinate value; the convolution kernel size of the convolution layer is 1*1; and finally, calculating according to the coordinates to obtain AN AN ratio.

Further, the specific operation of the step D is as follows:

and C, training the Adenoidenet by using the training set obtained in the step B, setting the initial learning rate to be 0.001, the batch_size to be 16, the loss function to be a mean square loss function, and the optimizer to be an Adam optimizer.

Further, the specific operation of the step E is as follows:

and C, testing the trained model by using the test set obtained in the step B, predicting coordinates of four landmarks, calculating AN AN ratio, and judging whether the model is adenoid hypertrophy.

Compared with the prior art, the invention has the beneficial effects that:

according to the method for automatically evaluating adenoid hypertrophy based on the MRI image of deep learning, the encoder module in the adenoid evaluation network model based on the convolutional neural network can efficiently extract local features through convolution, the decoder module can recover image resolution and capture long-distance dependence, the local features and global features are mutually fused by the adenoid, so that more-scale and richer features are obtained, and accurate landmark positioning is finally realized in the landmark detection module. The method can realize accurate landmark positioning and evaluation of adenoid hypertrophy, reduces the cost of manual participation, and provides a powerful auxiliary means for the evaluation of adenoid hypertrophy.

Drawings

FIG. 1 is a flow chart of the present invention.

FIG. 2 is a block diagram of an adenoidenet of the invention.

Detailed Description

The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Specific implementations of the invention are described in detail below in connection with specific embodiments.

The invention provides a method for automatically evaluating adenoid hypertrophy based on a deep learning MRI image, which comprises the following steps:

step A, converting all acquired MRI images from a DICOM format to a PNG format, selecting one frame of a nasal septum and a plurality of left and right frames of images thereof from a coronal bit sequence, carrying out gray scale normalization preprocessing, unifying pixel values to a [0,1] interval, cutting into a picture of 560pix x 640pix, and marking a data set;

step B, enhancing the image processed in the step A, expanding a data set, and dividing the expanded data set into three parts, namely a training set, a verification set and a test set;

step C, constructing an adenoidet adeno sample evaluation network model based on a convolutional neural network;

step D, training the Adenoidenet by using the training set and the verification set obtained in the step B to generate a training model;

and E, testing the training model generated in the step D by using the test set obtained in the step B.

As a preferred embodiment of the present invention, in the step B, the specific operation of expanding the data set is as follows:

all image data is flipped horizontally, rotated through different angles and scaled.

As a preferred embodiment of the present invention, in the step C, the CNN-based adenoidet adenoid evaluation network model includes:

an encoder module for extracting local features by convolution;

a decoder module for recovering image resolution and capturing long-range dependencies;

and the landmark detection module is used for realizing landmark positioning.

As a preferred embodiment of the present invention, the specific operation of the step C is as follows: the input PNG image passes through AN encoder module and a decoder module, information loss caused by downsampling is compensated through skip connection, final characteristics are extracted, the characteristics are sent into a landmark detection module, final predicted landmark coordinates are obtained, and AN AN ratio is calculated.

As a preferred embodiment of the present invention, in the step C:

in the encoder, input data firstly passes through two convolution layers, namely a BN layer and a ReLU layer, the convolution kernel size of each convolution layer is 3*3, the step length is 1, the output of each convolution layer enters the BN layer and the ReLU layer, the output of the last ReLU layer enters a pooling layer, the pooling mode is maximum pooling, and the pooling window size is 2; then three repeated sub-modules based on depth separable convolution are adopted, each module comprises a convolution layer, an LN layer and two full connection layers, the convolution kernel size of the convolution layer is 7*7, the step length is 1, the input of the convolution layer enters the LN layer, then the input of the convolution layer sequentially passes through the full connection layers and the GELU layers, the output of the last GELU layer in each module enters a pooling layer, the pooling mode is the maximum pooling, the pooling window size is 2, and the local characteristics are finally obtained; the local feature is then input into a decoder;

in the decoder, the input features pass through three sub-modules based on adaptive convolution, each module comprises a convolution layer, a BN layer, a ReLU layer, an adaptive convolution layer and an upsampling layer, the convolution kernel of the convolution layer has a size of 3*3, the step size is 1, the output of the convolution layer enters the BN layer and the ReLU layer, and then enters the adaptive convolution layer, and the adaptive convolution layer comprises two branches: one branch is a standard 3*3 convolution layer, a BN layer and a ReLU layer, the other branch is a deformable convolution, the outputs of the two branches are added to obtain a final output, the final output enters an up-sampling layer, the up-sampling uses transposed convolution, the convolution kernel size is 3*3, and the step length is 1; the output of the last up-sampling layer enters a landmark detection module;

in the landmark detection module, the feature obtains a heat map through a convolution layer, and then the convolution layer outputs and enters a DSNT layer to obtain a final coordinate value; the convolution kernel size of the convolution layer is 1*1; and finally, calculating according to the coordinates to obtain AN AN ratio.

In the embodiment of the invention, preferably, the deformable convolution can increase the standard position of the sampling point by any offset so as to enlarge the sampling area and capture the deformation characteristic.

As a preferred embodiment of the present invention, the specific operation of the step D is as follows:

and C, training the Adenoidenet by using the training set obtained in the step B, setting the initial learning rate to be 0.001, the batch_size to be 16, the loss function to be a mean square loss function, and the optimizer to be an Adam optimizer.

As a preferred embodiment of the present invention, the specific operation of step E is as follows:

and C, testing the trained model by using the test set obtained in the step B, predicting coordinates of four landmarks, calculating AN AN ratio, and judging whether the model is adenoid hypertrophy.

Embodiment 1, a method for automatically evaluating adenoid hypertrophy based on deep learning MRI images according to an embodiment of the present invention, includes the steps of:

A. image preprocessing

Collecting 300 samples, wherein the coronal position of each sample comprises 96-134 frames which are different, firstly converting the samples from a DICOM format into PNG images in batches through a pydicom library in Python, then selecting one frame of images with the nasal septum and left and right frames of images with the image size of 737pix×901pix, firstly carrying out gray scale normalization treatment on the images, normalizing pixel values to be in a [0,1] interval, and then cutting pictures to be in a size of 560pix×640 pix; the poorly imaged sample was removed, according to 7:1:2, dividing the sample into a training set, a verification set and a test set according to the proportion, and finally obtaining: training set 350, verification set 50, test set 100.

Enhancement of the original data: (a) rotating by-10 degrees to 10 degrees with a random probability of 0.5; (b) flipping with a probability level of 0.5; (c) randomly scaling 90% of the original length and width with a probability of 0.5.

B. Network construction and training

Constructing a CNN-based network, wherein the input of the Adenoidenet is a PNG image, the input image passes through an encoder module and a decoder module, and compensates information loss caused by downsampling through skip connection, extracts to obtain final characteristics, and then sends the characteristics to a landmark detection module to obtain final predicted landmark coordinates;

in the encoder, input data firstly passes through two convolution layers, namely a BN layer and a ReLU layer, the convolution kernel size of each convolution layer is 3*3, the step length is 1, the output of each convolution layer enters the BN layer and the ReLU layer, the output of the last ReLU layer enters a pooling layer, the pooling mode is maximum pooling, and the pooling window size is 2; then three repeated sub-modules based on depth separable convolution are adopted, each module comprises a convolution layer, an LN layer and two full connection layers, the convolution kernel size of the convolution layer is 7*7, the step length is 1, the input of the convolution layer enters the LN layer, then the input of the convolution layer sequentially passes through the full connection layers and the GELU layers, the output of the last GELU layer in each module enters a pooling layer, the pooling mode is the maximum pooling, the pooling window size is 2, and the local characteristics are finally obtained; the local feature is then input into a decoder;

in the decoder, the input features pass through three sub-modules based on adaptive convolution, each module comprises a convolution layer, a BN layer, a ReLU layer, an adaptive convolution layer and an upsampling layer, the convolution kernel of the convolution layer has a size of 3*3, the step size is 1, the output of the convolution layer enters the BN layer and the ReLU layer, and then enters the adaptive convolution layer, and the adaptive convolution layer comprises two branches: one branch is a standard 3*3 convolution layer, a BN layer and a ReLU layer, the other branch is a deformable convolution, the outputs of the two branches are added to obtain a final output, the final output enters an up-sampling layer, the up-sampling uses transposed convolution, the convolution kernel size is 3*3, and the step length is 1; the output of the last up-sampling layer enters a landmark detection module;

in the landmark detection module, the feature obtains a heat map through a convolution layer, and then the output of the convolution layer enters a DSNT layer to obtain the final coordinate value of the predicted landmark. The convolution kernel size of the convolution layer is 1*1. And finally, calculating according to the coordinates to obtain AN AN ratio, and further evaluating whether the adenoid hypertrophy exists.

C. Training of a network

Training the Adenoidenet by using the training set obtained in the A, and setting the initial learning rate to be 0.001 and the batch_size to be 16. The loss function is set as a mean square loss function and the optimizer selects the Adam optimizer. Setting training epoch as 100, setting learning rate attenuation strategy, wherein the learning rate is stepwise reduced according to the increase of training times to [0.0001,0.00001,0.000001], the corresponding training epoch is [25,50,75], and once for each training round, verifying, and stopping training when the testing effect of the verification set reaches convergence.

D. The test data verifies the trained model and determines the test effect

And predicting landmark positions of the test set. Loading the model and the weight stored in the training stage, inputting test data into the trained model to obtain a test result, wherein the test result comprises the coordinate and the AN ratio of each landmark, and calculating the average radial error and the AN error of the model predictive landmark according to the predictive value and the true value given by the model.

The foregoing is merely a preferred embodiment of the present invention, and it should be noted that modifications and improvements can be made by those skilled in the art without departing from the spirit of the present invention, and these should also be considered as the scope of the present invention, which does not affect the effect of the implementation of the present invention and the utility of the patent.

Claims (5)

1. A method for automatically assessing adenoid hypertrophy based on deep learning MRI images, comprising the steps of:

step A, converting all acquired MRI images from a DICOM format to a PNG format, selecting one frame of a nasal septum and a plurality of left and right frames of images thereof from a coronal bit sequence, carrying out gray scale normalization preprocessing, unifying pixel values to a [0,1] interval, cutting into a picture of 560pix x 640pix, and marking a data set;

step B, enhancing the image processed in the step A, expanding a data set, and dividing the expanded data set into three parts, namely a training set, a verification set and a test set;

step C, constructing an adenoidet adeno sample evaluation network model based on a convolutional neural network;

step D, training the Adenoidenet by using the training set and the verification set obtained in the step B to generate a training model;

step E, testing the training model generated in the step D by using the test set obtained in the step B;

in the step C, the CNN-based adenoidet adenoid evaluation network model includes:

an encoder module for extracting local features by convolution;

a decoder module for recovering image resolution and capturing long-range dependencies;

the landmark detection module is used for realizing landmark positioning;

in the step C:

in the encoder, input data firstly passes through two convolution layers, namely a BN layer and a ReLU layer, the convolution kernel size of each convolution layer is 3*3, the step length is 1, the output of each convolution layer enters the BN layer and the ReLU layer, the output of the last ReLU layer enters a pooling layer, the pooling mode is maximum pooling, and the pooling window size is 2; then three repeated sub-modules based on depth separable convolution are adopted, each module comprises a convolution layer, an LN layer and two full connection layers, the convolution kernel size of the convolution layer is 7*7, the step length is 1, the input of the convolution layer enters the LN layer, then the input of the convolution layer sequentially passes through the full connection layers and the GELU layers, the output of the last GELU layer in each module enters a pooling layer, the pooling mode is the maximum pooling, the pooling window size is 2, and the local characteristics are finally obtained; the local feature is then input into a decoder;

in the decoder, the input features pass through three sub-modules based on adaptive convolution, each module comprises a convolution layer, a BN layer, a ReLU layer, an adaptive convolution layer and an upsampling layer, the convolution kernel of the convolution layer has a size of 3*3, the step size is 1, the output of the convolution layer enters the BN layer and the ReLU layer, and then enters the adaptive convolution layer, and the adaptive convolution layer comprises two branches: one branch is a standard 3*3 convolution layer, a BN layer and a ReLU layer, the other branch is a deformable convolution, the outputs of the two branches are added to obtain a final output, the final output enters an up-sampling layer, the up-sampling uses transposed convolution, the convolution kernel size is 3*3, and the step length is 1; the output of the last up-sampling layer enters a landmark detection module;

in the landmark detection module, the feature obtains a heat map through a convolution layer, and then the convolution layer outputs and enters a DSNT layer to obtain a final coordinate value; the convolution kernel size of the convolution layer is 1*1; and finally, calculating according to the coordinates to obtain AN AN ratio.

2. The method for automatically assessing adenoid hypertrophy based on deep learning MRI images as claimed in claim 1 wherein in said step B the specific operation of expanding the data set is as follows:

the image data is flipped horizontally, rotated by different angles and scaled.

3. The method for automatically assessing adenoid hypertrophy based on deep learning MRI images as set forth in claim 1 wherein said step C comprises the specific operations of: the input PNG image is subjected to information loss caused by downsampling through a coder module and a decoder module, the final characteristics are extracted and obtained, the characteristics are sent into a landmark detection module, the coordinates of four finally predicted landmarks are obtained, and finally the AN ratio is calculated.

4. The method for automatically assessing adenoid hypertrophy based on deep learning MRI images as set forth in claim 1 wherein said step D comprises the specific operations of:

and C, training the Adenoidenet by using the training set obtained in the step B, setting the initial learning rate to be 0.001, the batch_size to be 16, the loss function to be a mean square loss function, and the optimizer to be an Adam optimizer.

5. The method for automatically assessing adenoid hypertrophy based on deep learning MRI images as set forth in claim 1 wherein said step E is specifically performed by:

and C, testing the trained model by using the test set obtained in the step B, predicting coordinates of four landmarks, calculating AN AN ratio, and judging whether the model is adenoid hypertrophy.