CN115311311A - Image description algorithm for lumbar intervertebral disc and construction method and application thereof - Google Patents
Image description algorithm for lumbar intervertebral disc and construction method and application thereof Download PDFInfo
- Publication number
- CN115311311A CN115311311A CN202211243529.7A CN202211243529A CN115311311A CN 115311311 A CN115311311 A CN 115311311A CN 202211243529 A CN202211243529 A CN 202211243529A CN 115311311 A CN115311311 A CN 115311311A
- Authority
- CN
- China
- Prior art keywords
- image
- disc
- magnetic resonance
- lumbar
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
- G06T2207/30012—Spine; Backbone
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Public Health (AREA)
- Data Mining & Analysis (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
An image description algorithm for lumbar intervertebral disc and a construction method and application thereof relate to the field of deep learning and medical image processing, and the invention comprises the following steps: the semantic segmentation positioning model is used for performing semantic segmentation positioning facing to lumbar disc pixels on the three-dimensional spine magnetic resonance image sagittal position central slice and providing lumbar disc pixel positioning information; based on a three-dimensional spine magnetic resonance image dimension reduction model of semantic segmentation positioning, performing region division on all slices of the three-dimensional spine magnetic resonance image according to lumbar disc pixel positioning information so as to reserve a rectangular region related to lumbar disc pixels; and generating a network model for the three-dimensional spine magnetic resonance image-oriented text, coding the image after dimension reduction, and obtaining an image science suggested text mapped by the image through training. The invention can output rich information under low computational power consumption, and enhances the performance of the image description algorithm.
Description
Technical Field
The invention relates to the technical field of deep learning and medical image processing, in particular to an image description algorithm for lumbar intervertebral disc and a construction method and application thereof.
Background
Magnetic Resonance (MR) imaging is an important imaging diagnostic tool, and is widely used for diagnosing lesions of human tissues such as spine, cranium and brain, heart blood vessels, joint bones and the like. With the gradual development of artificial intelligence algorithms, medical image technologies such as tissue segmentation based on three-dimensional images, image registration of medical images, positioning of target tissues in high-dimensional images, and the like, have received more and more attention from researchers. On the one hand, however, these existing popular image description algorithms have a problem of large calculation amount, and the large calculation amount consumes equipment with strong calculation power, which significantly reduces the practicability and the generalization of the algorithms; on the other hand, the imaging rules of Magnetic Resonance (MR) images are different for different organ tissues, especially for spinal and lumbar intervertebral discs, which have some unique properties that rich information density is included in sagittal slices, but information is sparse in the vertical dimension of sagittal planes, and this characteristic of information imbalance greatly affects the performance of the algorithm. Some researchers provide image description algorithms facing sparse spine images, such as target tissue segmentation and detection algorithms, but besides the problem of computational power consumption, the image description algorithms also have the problems of poor richness of output information and low intuition degree.
Therefore, there is an urgent need to develop a high-performance imaging suggestion text generation algorithm suitable for the lightweight of the lumbar intervertebral disc of the spine.
Disclosure of Invention
The invention provides a lumbar intervertebral disc-oriented image description algorithm and a construction method and application thereof, and aims to solve the problems of high computational power consumption, poor output information richness and low intuition degree of the existing image description algorithm.
The technical scheme adopted by the invention for solving the technical problem is as follows:
the invention relates to an image description algorithm for lumbar intervertebral discs, which comprises the following steps:
the semantic segmentation positioning model is used for performing semantic segmentation positioning facing to lumbar disc pixels on the three-dimensional spine magnetic resonance image sagittal position central slice and providing lumbar disc pixel positioning information;
based on a three-dimensional spine magnetic resonance image dimension reduction model of semantic segmentation positioning, performing region division on all slices of the three-dimensional spine magnetic resonance image according to lumbar disc pixel positioning information so as to reserve rectangular regions related to lumbar disc pixels and eliminate pixels unrelated to tasks;
and generating a network model for the three-dimensional spine magnetic resonance image-oriented text, coding the image after dimension reduction, and obtaining an iconography suggestion text mapped by the image through training.
Furthermore, the semantic segmentation positioning model for the three-dimensional spine magnetic resonance image sagittal position center slice adopts a context-guided segmentation network CGNet, and the total network parameter amount is 500K.
Furthermore, the three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning comprises an inter-disc pixel extraction module and an image dimension reduction module; the disc pixel extraction module is used for extracting a disc pixel matrix from all slices in the three-dimensional spine magnetic resonance image, wherein the disc pixel matrix comprises pixel information of five key lumbar discs; and the image dimension reduction module is used for merging key information of the obtained inter-disc pixel matrix and removing redundant pixels.
Further, the text generation network model facing the three-dimensional spine magnetic resonance image is of a two-stage structure and comprises an image coding structure and a text decoding structure; the image coding structure is a variant of a VGG16 classification network, and a softmax layer is removed from the structure so as to save pixel-level characteristic information of an image; the text decoding structure is a lightweight GRU training network.
Furthermore, the three-dimensional spine magnetic resonance image-oriented text generation network model has ten layers, and is formed by sequentially connecting an image feature input layer, an image feature Dropout layer, an image feature Dense layer, a text label input layer, a text label Embedding layer, a text label feature Dense layer, a GRU layer, an input merging layer, a merging Dense layer and an output Dense layer in a linear manner; the GRU layer is a core layer and is used for learning the mapping relation between the pixel-level feature information and the text label information.
Further, the total parameter number of the text generation network model facing the three-dimensional spine magnetic resonance image is 1.52M.
The invention relates to a construction method of an image description algorithm for lumbar intervertebral discs, which comprises the following steps:
setting a semantic segmentation positioning model facing a sagittal position center slice of a three-dimensional spine magnetic resonance image;
the semantic segmentation positioning model facing the three-dimensional spine magnetic resonance image sagittal position center slice adopts a context-guided segmentation network CGNet, and the total network parameter amount is 500K;
constructing a three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning;
the three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning comprises an inter-disc pixel extraction module and an image dimension reduction module; the disc pixel extraction module is used for extracting a disc pixel matrix from all slices in the three-dimensional spine magnetic resonance image, wherein the disc pixel matrix comprises pixel information of five key lumbar discs; the image dimensionality reduction module is used for merging key information of the obtained inter-disc pixel matrix and removing redundant pixels;
establishing a text generation network model facing the three-dimensional spine magnetic resonance image;
the three-dimensional spine magnetic resonance image-oriented text generation network model is of a two-section structure and comprises an image coding structure and a text decoding structure; the image coding structure is a variant of a VGG16 classification network, and a softmax layer is removed from the structure so as to save pixel-level characteristic information of an image; the text decoding structure is a lightweight GRU training network.
Furthermore, the three-dimensional spine magnetic resonance image-oriented text generation network model has ten layers, and is formed by sequentially connecting an image feature input layer, an image feature Dropout layer, an image feature Dense layer, a text label input layer, a text label Embedding layer, a text label feature Dense layer, a GRU layer, an input merging layer, a merging Dense layer and an output Dense layer in a linear manner; the GRU layer is a core layer and is used for learning the mapping relation between the pixel-level feature information and the text label information.
The application of the lumbar disc-oriented image description algorithm in generation of the iconography suggestion text for the lumbar disc comprises the following steps of:
firstly, semantic segmentation and positioning of a sagittal position center slice of a three-dimensional spinal magnetic resonance image;
selecting a sagittal central slice of a three-dimensional spine magnetic resonance image, and performing semantic segmentation and positioning on the sagittal central slice by using a context-guided segmentation network CGNet to obtain a disc positioning picture and provide lumbar disc pixel positioning information for three-dimensional spine magnetic resonance image dimension reduction processing;
step two, performing dimensionality reduction processing on the three-dimensional spine magnetic resonance image based on semantic segmentation positioning;
selecting all slices of the three-dimensional spine magnetic resonance image, and performing region division on all slices of the three-dimensional spine magnetic resonance image by using a three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning according to positioning information of lumbar disc pixels so as to reserve a rectangular region only related to the lumbar disc pixels and eliminate pixels irrelevant to a task;
generating a text oriented to the three-dimensional spine magnetic resonance image;
and carrying out image coding processing on the spine magnetic resonance image subjected to dimensionality reduction through an image coding structure, transmitting the acquired pixel-level feature information to a text decoding structure, carrying out character decoding processing on the pixel-level feature information through the text decoding structure, calculating text label information mapped by the lumbar disc pixels subjected to dimensionality reduction, and outputting a corresponding iconography suggestion text.
Further, the specific operation steps of the second step are as follows:
acquiring boundary coordinates of the upper, lower, left and right sides of five intervertebral discs at the lumbar vertebra part according to the positioning information of the lumbar intervertebral disc pixels, and dividing each slice in the three-dimensional spine magnetic resonance image into 32 sub-areas according to the boundary coordinates; in the 32 sub-regions, 10 key sub-regions cover five lumbar disc pixels, and numbering is performed according to the traversal sequence of the third-order hilbert curve, so that the disc L1-L2 is distributed in the key sub-regions 18 and 31, the disc L2-L3 is distributed in the key sub-regions 13 and 12, the disc L3-L4 is distributed in the key sub-regions 14 and 9, the disc L4-L5 is distributed in the key sub-regions 3 and 8, and the disc L5-S1 is distributed in the key sub-regions 4 and 5; in the 10 key sub-regions, assuming that the input three-dimensional spine magnetic resonance image has i slices, the classification is performed according to the following rules: key subregions 4, 5, 3 and 8 distributed by the intervertebral disc L5-S1 and the intervertebral disc L4-L5 are reserved in the 4 th to the i-1 th slices, key subregions 18, 31, 13, 12, 14 and 9 distributed by the intervertebral discs L1-L2, L2-L3 and L3-L4 are reserved in the 3 rd to the i-th slices, and the rest key subregions are discarded; and calculating the average value of the pixel values of the reserved key subregions according to the slicing sequence, and finally obtaining the dimension-reduced images corresponding to the five inter-discal plates.
The invention has the beneficial effects that:
according to the image description algorithm for the lumbar intervertebral disc and the construction method and application thereof, the lumbar intervertebral disc pixels are segmented for positioning, and the lumbar intervertebral disc pixel positioning information is provided for the subsequent three-dimensional spine Magnetic Resonance (MR) image dimension reduction processing; performing dimensionality reduction processing on the input three-dimensional spine Magnetic Resonance (MR) image according to the lumbar disc pixel positioning information; and carrying out image coding and character decoding processing based on the dimension reduction image to obtain an iconography suggestion text of the spinal and lumbar intervertebral discs of the patient. Compared with the prior art, the invention has the following advantages:
(1) The image description algorithm for the lumbar intervertebral disc is composed of three light-weight designed sub-algorithms, and the light-weight design based on the algorithm can remarkably reduce the consumption of the image description algorithm on computing resources and reduce the computing power consumption;
(2) According to the image description algorithm for the lumbar intervertebral disc, the problems of poor richness and low intuition degree of output information in the existing algorithm can be effectively solved by outputting the iconography suggestion text information of the spinal and lumbar intervertebral discs;
(3) The invention has the advantage of outputting rich information under low computational power consumption, enhances the performance of the image description algorithm, and has extremely high application and popularization values in the technical fields of deep learning and medical image processing.
Drawings
Fig. 1 is a flowchart of an image description algorithm for lumbar intervertebral disc according to the present invention.
Fig. 2 is a three-dimensional spine Magnetic Resonance (MR) image sagittal center slice.
Fig. 3 is a picture obtained after semantic segmentation and positioning of a sagittal position center slice of a three-dimensional spine Magnetic Resonance (MR) image.
Fig. 4 is an image of five intervertebral discs of the lumbar region after dimensionality reduction.
Fig. 5 is a sagittal Magnetic Resonance (MR) image of a portion of the human back in experiment 1.
Figure 6 is a manually labeled iconography text label depicting the status of each lumbar disc in trial 1.
Figure 7 is a final pictorial recommendation text for the lumbar disc of the patient obtained in trial 1.
Fig. 8 is a sagittal Magnetic Resonance (MR) image of a portion of the human back in trial 2.
Figure 9 is a pictorial text label manually labeled in trial 2 describing the status of each lumbar disc.
Figure 10 is a final pictorial recommendation text for the lumbar disc of the patient obtained in trial 2.
Detailed Description
The following is a description of the illustrated embodiments of the invention using the accompanying drawings. The drawings illustrate only one embodiment of the invention and are therefore not to be considered limiting of its scope. It is obvious to a person skilled in the art that other relevant figures can also be derived from these figures without inventive effort.
The invention relates to an image description algorithm for lumbar intervertebral discs, which can output an iconography suggestion text about the lumbar intervertebral discs of a patient after inputting a three-dimensional spine Magnetic Resonance (MR) image of the patient.
The invention relates to an image description algorithm for lumbar intervertebral discs, which consists of three sub-algorithms, and specifically comprises the following steps: the method comprises a semantic segmentation positioning model facing a sagittal position center slice of a three-dimensional spine Magnetic Resonance (MR) image, a three-dimensional spine Magnetic Resonance (MR) image dimension reduction model based on semantic segmentation positioning, and a text generation network model facing the three-dimensional spine Magnetic Resonance (MR) image.
The three sub-algorithms are all light, and after a semantic segmentation positioning model facing to a sagittal position central slice of a three-dimensional spine Magnetic Resonance (MR) image and a text generation network model facing to the three-dimensional spine Magnetic Resonance (MR) image are supervised and trained, an obtained storage model can be operated on low-computing-force equipment in a redeployment mode.
The semantic segmentation positioning model facing the three-dimensional spine Magnetic Resonance (MR) image sagittal position center slice is mainly used for performing semantic segmentation positioning facing lumbar disc pixels on the three-dimensional spine Magnetic Resonance (MR) image sagittal position center slice, and providing lumbar disc pixel positioning information for a subsequent three-dimensional spine Magnetic Resonance (MR) image dimensionality reduction model based on semantic segmentation positioning.
The three-dimensional spine Magnetic Resonance (MR) image dimension reduction model based on semantic segmentation positioning is mainly used for carrying out dimension reduction processing on the three-dimensional spine Magnetic Resonance (MR) image so as to eliminate pixels irrelevant to tasks; specifically, the three-dimensional spine Magnetic Resonance (MR) image dimension reduction model based on semantic segmentation positioning can perform region division on all slices of a three-dimensional spine Magnetic Resonance (MR) image according to positioning information of lumbar disc pixels so as to reserve a rectangular region only related to the lumbar disc pixels.
The text facing the three-dimensional spine Magnetic Resonance (MR) image generates a network model for coding the image after dimension reduction, and an imaging suggestion text mapped by the text is obtained through training.
The invention relates to a construction method of an image description algorithm for lumbar intervertebral discs, which specifically comprises the following steps:
step one, setting a semantic segmentation positioning model facing a three-dimensional spine Magnetic Resonance (MR) image sagittal position center slice; the method comprises the following specific steps:
constructing a lightweight encoder-decoder structure segmentation network, preferably adopting a lightweight medical segmentation network CGNet (context-guided segmentation network), wherein the total network parameter amount is 500K, and the requirement of rapid deployment on low-computing-power equipment can be met;
constructing a three-dimensional spine Magnetic Resonance (MR) image dimension reduction model based on semantic segmentation positioning;
the model comprises two modules, namely an inter-disc pixel extraction module and an image dimension reduction module, wherein the inter-disc pixel extraction module is used for extracting inter-disc pixel matrixes from all slices in a three-dimensional spine Magnetic Resonance (MR) image, and the inter-disc pixel matrixes comprise pixel information of five key lumbar discs; the image dimensionality reduction module is used for merging key information of the obtained disc pixel matrix and removing redundant pixels;
establishing a text generation network model facing to a three-dimensional spine Magnetic Resonance (MR) image;
the text generation network model for the three-dimensional spine Magnetic Resonance (MR) image is a two-section structure and consists of an image coding structure and a text decoding structure, wherein the image coding structure is a variant of a VGG16 classification network, and a softmax layer is removed from the structure of the image coding structure so as to store pixel-level characteristic information of an image; the text decoding structure is a lightweight GRU training network, and the network has two inputs, namely pixel-level feature information and text label information;
specifically, the three-dimensional spine Magnetic Resonance (MR) image-oriented text generation network model has ten layers, and is formed by sequentially connecting an image feature input layer, an image feature Dropout layer, an image feature Dense layer, a text label input layer, a text label Embedding layer, a text label feature Dense layer, a GRU layer, an input merging layer, a merging Dense layer and an output Dense layer in a linear manner, wherein the GRU layer is a core level and is used for learning a mapping relationship between pixel-level feature information and text label information; the total parameter number of the text generation network model facing the three-dimensional spine Magnetic Resonance (MR) image is 1.52M.
The image description algorithm for the lumbar intervertebral disc can realize the generation of an iconography suggestion text of the lumbar intervertebral disc, is deployed on low-computing-force equipment, and can obtain the iconography suggestion text corresponding to the lumbar intervertebral disc of a patient as long as a three-dimensional spine Magnetic Resonance (MR) image of the patient is input.
The application of the lumbar disc-oriented image description algorithm in generation of the iconography suggestion text for the lumbar disc disclosed by the invention is shown in fig. 1, and specifically comprises the following steps of:
firstly, semantic segmentation and positioning of a sagittal position center slice oriented to a three-dimensional spine Magnetic Resonance (MR) image;
selecting a sagittal center slice of a three-dimensional spine Magnetic Resonance (MR) image, performing semantic segmentation and positioning on the sagittal center slice by applying CGNet (context-guided segmentation network) as shown in FIG. 2 to obtain a disc positioning picture, and providing lumbar disc pixel positioning information for subsequent three-dimensional spine Magnetic Resonance (MR) image dimension reduction processing as shown in FIG. 3;
step two, performing three-dimensional spine Magnetic Resonance (MR) image dimensionality reduction processing based on semantic segmentation positioning;
selecting all slices of a three-dimensional spine Magnetic Resonance (MR) image, and performing region division on all slices of the three-dimensional spine Magnetic Resonance (MR) image by using a three-dimensional spine Magnetic Resonance (MR) image dimension reduction model based on semantic segmentation positioning according to positioning information of lumbar disc pixels so as to reserve a rectangular region only related to the lumbar disc pixels and eliminate pixels irrelevant to a task; the method comprises the following specific steps:
acquiring upper, lower, left and right boundary coordinates of five intervertebral discs at a lumbar vertebral part according to positioning information of lumbar intervertebral disc pixels, and dividing each slice in a three-dimensional spine Magnetic Resonance (MR) image into 32 sub-regions according to the boundary coordinates; in the 32 sub-regions, 10 key sub-regions cover five lumbar disc pixels, and numbering is performed according to the traversal sequence of the third-order hilbert curve, so that the disc L1-L2 is distributed in the key sub-regions 18 and 31, the disc L2-L3 is distributed in the key sub-regions 13 and 12, the disc L3-L4 is distributed in the key sub-regions 14 and 9, the disc L4-L5 is distributed in the key sub-regions 3 and 8, and the disc L5-S1 is distributed in the key sub-regions 4 and 5; in the 10 key sub-regions, assuming that the input three-dimensional spine Magnetic Resonance (MR) image has i slices, the classification can be performed according to the following rules: key subregions 4, 5, 3 and 8 distributed by the intervertebral disc L5-S1 and the intervertebral disc L4-L5 are reserved in the 4 th to the i-1 th slices, key subregions 18, 31, 13, 12, 14 and 9 distributed by the intervertebral discs L1-L2, L2-L3 and L3-L4 are reserved in the 3 rd to the i-th slices, and the rest key subregions are discarded; calculating the average value of the pixel values of the reserved key subregions according to the slicing sequence, and finally obtaining the reduced-dimension images corresponding to the five inter-discal disks, as shown in fig. 4;
generating a text oriented to a three-dimensional spine Magnetic Resonance (MR) image;
a text generation network model facing a three-dimensional spine Magnetic Resonance (MR) image is used for coding the image after dimension reduction, and an iconography suggestion text mapped by the image is obtained through training; the method comprises the following specific steps:
and carrying out image coding processing on the spine Magnetic Resonance (MR) image subjected to dimensionality reduction through an image coding structure, transmitting the acquired pixel-level feature information to a text decoding structure, carrying out character decoding processing on the pixel-level feature information through the text decoding structure, calculating text label information mapped by the lumbar disc pixels subjected to dimensionality reduction, and outputting a corresponding iconography suggestion text.
In order to test the application effect of the lumbar intervertebral disc-oriented image description algorithm in the generation of the imaging suggestion text of the lumbar intervertebral disc, the following tests are carried out:
test 1
Firstly, training a semantic segmentation positioning model facing a sagittal position central slice of a three-dimensional spine Magnetic Resonance (MR) image, namely a lightweight encoder-decoder structure segmentation network, wherein a data set used for training is 759 human back sagittal position Magnetic Resonance (MR) image maps (figure 5), and intervertebral discs among 7 lower spines (T11/T12-L5/S1) are calibrated corresponding to each tissue label; training data accounts for 90% of all data sets, test data accounts for 10% of all data sets; learning rate of 10 -6 Training is performed for 10 periods, each comprising 2000 iterations.
Then, a text generation network model facing to a three-dimensional spine Magnetic Resonance (MR) image, that is, a two-stage structure composed of an image coding structure and a text decoding structure is trained, a data set used for the training is 448 three-dimensional spine Magnetic Resonance (MR) images, wherein about 10% of data shows that a subject is healthy, and the rest subjects have lumbar disc lesions of different degrees, and according to pathological degrees, the lumbar discs are classified into normal discs, bulging discs, herniation discs and stenosis discs, and an iconography text label describing the state of each lumbar disc is artificially labeled, as shown in fig. 6, the obtained iconography text label may include the following information: normal lumbar 1-2, 2-3, 3-4, 4-5, 5-1 disc herniation; the training period is 50 periods, each period comprising 1000 iterations.
Finally, the two trained models are stored in the image description algorithm facing to the lumbar intervertebral disc, so that the deployment is convenient; when the lumbar disc-oriented image description algorithm is deployed on a low-computing-power device and is operated, a three-dimensional spine Magnetic Resonance (MR) image of a patient is directly input, so that an iconography suggestion text corresponding to the lumbar disc of the patient can be obtained, as shown in fig. 7, an obtained iconography text label may include the following information: normal disc of lumbar vertebra 1-2, normal disc of lumbar vertebra 2-3, protrusion of disc of lumbar vertebra 3-4, protrusion of disc of lumbar vertebra 4-5, protrusion of disc of lumbar vertebra 5-1.
Test 2
The experimental procedure is the same as experiment 1, wherein the input three-dimensional spine Magnetic Resonance (MR) image test data is shown in fig. 8, the artificially labeled iconography text labels describing the states of the lumbar intervertebral discs are shown in fig. 9, and the obtained iconography text labels may include the following information: 1-2 disc herniation of lumbar vertebra, 2-3 disc herniation of lumbar vertebra, 3-4 disc normal of lumbar vertebra, 4-5 disc herniation of lumbar vertebra, 5-1 disc normal of lumbar vertebra; the final algorithm results in an imaging suggestion text of the lumbar disc of the patient, as shown in fig. 10, and the obtained imaging text label may include the following information: bulging of lumbar 1-2 lumbar intervertebral discs, bulging of lumbar 2-3 lumbar intervertebral discs, normal lumbar 3-4 lumbar intervertebral discs, lumbar 4-5 lumbar intervertebral discs, and lumbar 5-1 lumbar intervertebral discs.
Through the two tests, the application effect of the lumbar disc-oriented image description algorithm in the generation of the lumbar disc photographical suggestion text is good, the ideal lumbar disc photographical suggestion text is generated, the final text generation evaluation index BLEU-1 value is 0.6873, the abnormal disc classification rate is 0.8901, and the abnormal disc classification accuracy rate is 0.4503.
The image description algorithm for the lumbar intervertebral disc has the advantages of remarkably reducing the consumption of auxiliary algorithms on computing resources and being rich in output information, enhances the performance of the image description algorithm, and has high application value and popularization value in the technical fields of deep learning and medical image processing.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. The image description algorithm for the lumbar intervertebral disc is characterized by comprising the following steps:
the semantic segmentation positioning model is used for performing semantic segmentation positioning facing to lumbar disc pixels on the three-dimensional spine magnetic resonance image sagittal position central slice and providing lumbar disc pixel positioning information;
based on a semantic segmentation positioning-based three-dimensional spine magnetic resonance image dimension reduction model, performing region division on all slices of the three-dimensional spine magnetic resonance image according to lumbar disc pixel positioning information so as to reserve a rectangular region related to lumbar disc pixels and eliminate pixels unrelated to tasks;
and generating a network model for the three-dimensional spine magnetic resonance image-oriented text, coding the image after dimension reduction, and obtaining an iconography suggestion text mapped by the image through training.
2. The lumbar disc-oriented image description algorithm of claim 1, wherein the semantic segmentation positioning model for the three-dimensional spine magnetic resonance image sagittal center slice adopts a context-guided segmentation network CGNet, and the total network parameter is 500K.
3. The lumbar disc-oriented image description algorithm of claim 1, wherein the semantic segmentation location-based three-dimensional spine magnetic resonance image dimension reduction model comprises a disc pixel extraction module and an image dimension reduction module; the disc pixel extraction module is used for extracting a disc pixel matrix from all slices in the three-dimensional spine magnetic resonance image, wherein the disc pixel matrix comprises pixel information of five key lumbar discs; and the image dimension reduction module is used for merging key information of the obtained inter-disc pixel matrix and removing redundant pixels.
4. The lumbar disc-oriented image description algorithm of claim 1, wherein the three-dimensional spine magnetic resonance image-oriented text generation network model is a two-segment structure comprising an image coding structure and a text decoding structure; the image coding structure is a variant of a VGG16 classification network, and a softmax layer is removed from the structure so as to save pixel-level characteristic information of an image; the text decoding structure is a lightweight GRU training network.
5. The lumbar disc-oriented image description algorithm according to claim 1, wherein the three-dimensional spine magnetic resonance image-oriented text generation network model has ten layers, and is composed of an image feature input layer, an image feature Dropout layer, an image feature depth layer, a text label input layer, a text label Embedding layer, a text label feature depth layer, a GRU layer, an input merging layer, a merging depth layer and an output depth layer which are sequentially connected by a linear connection; the GRU layer is a core layer and is used for learning the mapping relation between the pixel-level feature information and the text label information.
6. The lumbar disc-oriented image description algorithm of claim 1, wherein the total parameter number of the three-dimensional spine magnetic resonance image-oriented text generation network model is 1.52M.
7. The construction method of the image description algorithm for lumbar intervertebral disc according to claim 1, comprising the steps of:
setting a semantic segmentation positioning model facing a sagittal position center slice of a three-dimensional spine magnetic resonance image;
the semantic segmentation positioning model facing the three-dimensional spine magnetic resonance image sagittal position center slice adopts a context-guided segmentation network CGNet, and the total network parameter amount is 500K;
constructing a three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning;
the three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning comprises an inter-disc pixel extraction module and an image dimension reduction module; the disc pixel extraction module is used for extracting a disc pixel matrix from all slices in the three-dimensional spine magnetic resonance image, wherein the disc pixel matrix comprises pixel information of five key lumbar discs; the image dimensionality reduction module is used for merging key information of the obtained inter-disc pixel matrix and removing redundant pixels;
establishing a text generation network model facing the three-dimensional spine magnetic resonance image;
the three-dimensional spine magnetic resonance image-oriented text generation network model is of a two-section structure and comprises an image coding structure and a text decoding structure; the image coding structure is a variant of a VGG16 classification network, and a softmax layer is removed from the structure so as to save pixel-level characteristic information of an image; the text decoding structure is a lightweight GRU training network.
8. The method for constructing an image description algorithm for lumbar intervertebral disc according to claim 7, wherein the text generation network model for the three-dimensional spine magnetic resonance image has ten layers, and is composed of an image feature input layer, an image feature Dropout layer, an image feature sense layer, a text label input layer, a text label Embedding layer, a text label feature sense layer, a GRU layer, an input merging layer, a merging sense layer and an output sense layer which are sequentially connected by a linear connection; the GRU layer is a core layer and is used for learning the mapping relation between the pixel-level feature information and the text label information.
9. The application of the lumbar disc-oriented image description algorithm in generating the visual suggestion text of the lumbar disc as claimed in claim 1, characterized by comprising the following steps:
firstly, semantic segmentation and positioning of a sagittal position center slice of a three-dimensional spine magnetic resonance image;
selecting a sagittal central slice of a three-dimensional spine magnetic resonance image, and performing semantic segmentation and positioning on the sagittal central slice by using a context-guided segmentation network CGNet to obtain a disc positioning picture and provide lumbar disc pixel positioning information for three-dimensional spine magnetic resonance image dimension reduction processing;
step two, performing dimensionality reduction processing on the three-dimensional spine magnetic resonance image based on semantic segmentation positioning;
selecting all slices of the three-dimensional spine magnetic resonance image, and performing region division on all slices of the three-dimensional spine magnetic resonance image by using a three-dimensional spine magnetic resonance image dimension reduction model based on semantic segmentation positioning according to positioning information of lumbar disc pixels so as to reserve a rectangular region only related to the lumbar disc pixels and eliminate pixels irrelevant to a task;
generating a text oriented to the three-dimensional spine magnetic resonance image;
and carrying out image coding processing on the spine magnetic resonance image subjected to dimensionality reduction through an image coding structure, transmitting the acquired pixel-level characteristic information to a text decoding structure, carrying out character decoding processing on the pixel-level characteristic information through the text decoding structure, calculating text label information mapped by the lumbar disc pixels subjected to dimensionality reduction, and outputting a corresponding iconography suggestion text.
10. The application of the lumbar disc-oriented image description algorithm in generation of the visual suggestion text for the lumbar disc as claimed in claim 9, wherein the specific operation steps of the second step are as follows:
acquiring boundary coordinates of the upper, lower, left and right sides of five intervertebral discs at the lumbar vertebra part according to the positioning information of the lumbar intervertebral disc pixels, and dividing each slice in the three-dimensional spine magnetic resonance image into 32 sub-areas according to the boundary coordinates; in the 32 sub-areas, 10 key sub-areas cover five lumbar disc pixels, and numbering is performed according to the traversal sequence of the third-order hilbert curve, so that the disc L1-L2 are distributed in the key sub-areas 18 and 31, the disc L2-L3 are distributed in the key sub-areas 13 and 12, the disc L3-L4 are distributed in the key sub-areas 14 and 9, the disc L4-L5 are distributed in the key sub-areas 3 and 8, and the disc L5-S1 are distributed in the key sub-areas 4 and 5; in the 10 key sub-regions, if the input three-dimensional spine magnetic resonance image has i slices, the classification is performed according to the following rules: key subregions 4, 5, 3 and 8 distributed by the intervertebral disc L5-S1 and the intervertebral disc L4-L5 are reserved in the 4 th to the i-1 th slices, key subregions 18, 31, 13, 12, 14 and 9 distributed by the intervertebral discs L1-L2, L2-L3 and L3-L4 are reserved in the 3 rd to the i-th slices, and the rest key subregions are discarded; and calculating the average value of the pixel values of the reserved key subregions according to the slicing sequence, and finally obtaining the dimension-reduced images corresponding to the five inter-discal plates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211243529.7A CN115311311B (en) | 2022-10-12 | 2022-10-12 | Image description method for lumbar intervertebral disc and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211243529.7A CN115311311B (en) | 2022-10-12 | 2022-10-12 | Image description method for lumbar intervertebral disc and application thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115311311A true CN115311311A (en) | 2022-11-08 |
| CN115311311B CN115311311B (en) | 2022-12-20 |
Family
ID=83868217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211243529.7A Active CN115311311B (en) | 2022-10-12 | 2022-10-12 | Image description method for lumbar intervertebral disc and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115311311B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116797457A (en) * | 2023-05-20 | 2023-09-22 | 北京大学 | Method and system for simultaneously realizing super-resolution and artifact removal of magnetic resonance image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111192248A (en) * | 2019-12-30 | 2020-05-22 | 山东大学 | Multi-task relation learning method for positioning, identifying and segmenting vertebral body in nuclear magnetic resonance imaging |
| CN112465771A (en) * | 2020-11-25 | 2021-03-09 | 平安科技（深圳）有限公司 | Method and device for analyzing spine nuclear magnetic resonance image and computer equipment |
| CN113506308A (en) * | 2021-07-06 | 2021-10-15 | 同济大学 | Deep learning-based vertebra positioning and spine segmentation method in medical image |
| CN114693719A (en) * | 2022-03-30 | 2022-07-01 | 南京航空航天大学 | Spine image segmentation method and system based on 3D-SE-Vnet |
| CN115082680A (en) * | 2022-06-28 | 2022-09-20 | 南京佗道医疗科技有限公司 | Spine segmentation method based on three-dimensional image |
-
2022
- 2022-10-12 CN CN202211243529.7A patent/CN115311311B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111192248A (en) * | 2019-12-30 | 2020-05-22 | 山东大学 | Multi-task relation learning method for positioning, identifying and segmenting vertebral body in nuclear magnetic resonance imaging |
| CN112465771A (en) * | 2020-11-25 | 2021-03-09 | 平安科技（深圳）有限公司 | Method and device for analyzing spine nuclear magnetic resonance image and computer equipment |
| CN113506308A (en) * | 2021-07-06 | 2021-10-15 | 同济大学 | Deep learning-based vertebra positioning and spine segmentation method in medical image |
| CN114693719A (en) * | 2022-03-30 | 2022-07-01 | 南京航空航天大学 | Spine image segmentation method and system based on 3D-SE-Vnet |
| CN115082680A (en) * | 2022-06-28 | 2022-09-20 | 南京佗道医疗科技有限公司 | Spine segmentation method based on three-dimensional image |
Non-Patent Citations (7)
| Title |
|---|
| HUA-DONG ZHENG，ET AL.: "Deep learning-based high-accuracy quantitation for lumbar intervertebral disc degeneration from MRI", 《NATURE COMMUNICATIONS》 * |
| QI LI，ET AL.: "Segmentation of Intervertebral Disc based on Semi-supervised Conditional Generative Adversarial Network", 《2022 3RD INTERNATIONAL CONFERENCE ON CVIDL & ICCEA）》 * |
| WAFA MBARKI,ET AL.: "A novel method based on deep learning for herniated lumbar disc segmentation", 《2020 4TH INTERNATIONAL CONFERENCE ON ADVANCED SYSTEMS AND EMERGENT TECHNOLOGIES》 * |
| YANG XI，ET AL.: "2020 4th International Conference on Advanced Systems and Emergent Technologies semantic integration and modulation of attention: An fMRI and graph-based study", 《PLOS ONE》 * |
| YU-KAI CHENG，ET AL.: "Automatic Segmentation of Specific Intervertebral Discs through a Two-Stage MultiResUNet Model", 《J. CLIN. MED.》 * |
| 李强等: "一种改进的医疗文本分类模型:LS-GRU", 《东北大学学报(自然科学版)》 * |
| 汤阳杰，等.: "腰椎间盘早期退变的智能诊断方法研究", 《BIOMEDRXIV》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116797457A (en) * | 2023-05-20 | 2023-09-22 | 北京大学 | Method and system for simultaneously realizing super-resolution and artifact removal of magnetic resonance image |
| CN116797457B (en) * | 2023-05-20 | 2024-05-14 | 北京大学 | Method and system for simultaneously realizing super-resolution and artifact removal of magnetic resonance image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115311311B (en) | 2022-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112465827B (en) | Contour perception multi-organ segmentation network construction method based on class-by-class convolution operation | |
| Zhou et al. | Efficient skeletonization of volumetric objects | |
| CN111461232A (en) | Nuclear magnetic resonance image classification method based on multi-strategy batch type active learning | |
| Du et al. | Accelerated super-resolution MR image reconstruction via a 3D densely connected deep convolutional neural network | |
| CN116129141B (en) | Medical data processing method, apparatus, device, medium and computer program product | |
| CN114399510B (en) | Skin focus segmentation and classification method and system combining image and clinical metadata | |
| WO2022059315A1 (en) | Image encoding device, method and program, image decoding device, method and program, image processing device, learning device, method and program, and similar image search device, method and program | |
| CN115311311B (en) | Image description method for lumbar intervertebral disc and application thereof | |
| Chuang et al. | Efficient triple output network for vertebral segmentation and identification | |
| CN115272295A (en) | Dynamic brain function network analysis method and system based on time domain-space domain combined state | |
| CN113436128B (en) | Dual-discriminator multi-mode MR image fusion method, system and terminal | |
| CN113538209A (en) | Multi-modal medical image registration method, registration system, computing device and storage medium | |
| CN112489048B (en) | Automatic optic nerve segmentation method based on depth network | |
| Xu et al. | RUnT: A network combining residual U-Net and transformer for vertebral edge feature fusion constrained spine CT image segmentation | |
| CN117475268A (en) | Multimode medical image fusion method based on SGDD GAN | |
| CN111798452A (en) | Carotid artery handheld ultrasonic image segmentation method, system and device | |
| CN110570416A (en) | Method for visualization and 3D printing of multi-modal cardiac images | |
| CN116309754A (en) | Brain medical image registration method and system based on local-global information collaboration | |
| CN115565671A (en) | Atrial fibrillation auxiliary analysis method based on cross-model mutual teaching semi-supervision | |
| CN114842020A (en) | Lightweight tumor image segmentation method | |
| CN116802512A (en) | Method for training CNN and for processing an input perfusion sequence using said CNN | |
| CN112419267A (en) | Brain glioma segmentation model and method based on deep learning | |
| CN118261909B (en) | Method, system and equipment for dividing paraspinal muscles | |
| CN113781636B (en) | Pelvic bone modeling method and system, storage medium, and computer program product | |
| Unal et al. | A comparison of feature extraction techniques for diagnosis of lumbar intervertebral degenerative disc disease |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |