CN114305473A - Body composition automatic measuring system based on abdomen CT image and deep learning - Google Patents
Body composition automatic measuring system based on abdomen CT image and deep learning Download PDFInfo
- Publication number
- CN114305473A CN114305473A CN202210068304.6A CN202210068304A CN114305473A CN 114305473 A CN114305473 A CN 114305473A CN 202210068304 A CN202210068304 A CN 202210068304A CN 114305473 A CN114305473 A CN 114305473A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- image
- lumbar vertebra
- muscle
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a body composition automatic measuring system based on abdominal CT images and deep learning. The system comprises a positioning module for all axial slices corresponding to the third lumbar vertebra on a CT image, a four-class skeletal muscle segmentation module for axial slices corresponding to the third lumbar vertebra on the CT image, and an automatic body composition measurement module based on the CT image. The first two modules are utilized to train a deep learning model to position and segment four skeletal muscles of the third lumbar vertebra, and the proportion of body components such as fat, muscle and the like is automatically calculated. The invention is trained and verified on CT data of clinical liver cirrhosis patients, the average Dice of the four skeletal muscle segmentation results is 0.9283, and the average surface distance is 0.6779 mm. The present invention can also obtain body components such as subcutaneous fat and intra-abdominal fat by threshold treatment. The time for obtaining the body components corresponding to the third lumbar vertebra in batches is 2-3 seconds, and the method has important significance for clinically diagnosing complications of liver cirrhosis patients.
Description
Technical Field
The invention belongs to the technical field of medical diagnosis equipment, and particularly relates to an automatic body composition measuring system based on an abdominal CT image.
Background
Body composition analysis is the quantification of muscle and adipose tissue. The measurement is usually based on two types of methods, biological parameters or medical images. In particular, measurement of skeletal muscle mass on axial CT slices through The center of The third lumbar vertebra (The third lumbar vertibra, L3) proved to be The most accurate [1 ]. Measurement of skeletal muscle in L3 (or L4) axial CT slices cuts muscle and fat primarily according to the HU value of the CT image, in two ways: a method divides HU value intervals of all muscles and fat, such as muscle, subcutaneous and intramuscular fat, and intraabdominal fat, to [ -29,150], [ -190, -30] and [ -150, -50] [2], respectively. Another method measures only the psoas [3 ]. The drawbacks of both of these methods are: the method is difficult to distinguish tissues under certain special pathological conditions, so that the segmentation result is inaccurate; no matter all skeletal muscles are analyzed as a whole, or only the psoas muscles are analyzed, the CT image is not fully utilized; more importantly, the use of only skeletal muscle on single axial CT slices of L3 or L4 resulted in a small number of samples that failed to accurately describe muscle mass.
(1) Conventional segmentation method
The medical image segmentation is a key factor for determining whether the medical image can provide reliable basis in clinical diagnosis and treatment. Currently, several semi-automatic skeletal muscle area measurement software, such as ImageJ, CoreCleaner, Slice-O-matic, 3D Slicer, Horos, OsiriX, Mimics, and FatSeg, are widely used for image segmentation [4] [5 ].
In addition to software segmentation, machine learning based methods are also widely used for muscle segmentation. For example, researchers use shape-prior modeling to segment the muscle in the L3 corresponding slice or use a new finite element deformation model to segment the muscle [6 ]. However, segmenting different types of skeletal muscle on L3 axial CT slices using conventional methods is challenging because: 1) the shape and size difference of different types of skeletal muscles is large, and the imbalance-like problem exists during optimization, as shown in four skeletal muscle labels in fig. 3; 2) the boundary between different skeletal muscles and between the skeletal muscles and surrounding tissues is not clear, the boundary has rough edges, and the existing software segmentation method cannot actively learn the useful information, so that the accuracy is not high, as shown in labels 2 and 3 of fig. 3; 3) the same type of skeletal muscle has morphological differences among different individuals, which affects the segmentation result.
(2) Deep learning segmentation method
Deep learning can bring low muscle mass and mass from theoretical studies to clinical practice because it can achieve automatic batch segmentation of relevant organs, tissues and tumors in patient images in existing various disease databases. The deep learning related studies on abdominal CT images are: segmenting skeletal muscles in a third or fourth lumbar axial CT image for body composition analysis [7 ]; automatically positioning an axial slice at the center of L3 from a whole body or part of a body CT image, and then segmenting body components such as skeletal muscle [8 ]; analysis of skeletal muscle segmentation was performed using axial CT slices corresponding to the 3 rd lumbar inferior endplate [9 ]. However, these methods all segment the entire skeletal muscle in a single slice to take into account the effects of the physiological pathology on the entire skeletal muscle, and do not take into account the effects of the physiological pathology on different kinds of muscle.
The existing research also has a plurality of muscle groups which are fully automatically divided into a third lumbar vertebra and a fourth lumbar vertebra and are used for detecting central sarcopenia [10 ]; automatic segmentation of paraspinal muscles on axial magnetic resonance imaging of different vertebras [11] has also been studied; researchers also segmented the left paraspinal muscles automatically in the twelfth thoracic axial direction [12 ]. These methods, while having focused on different kinds of muscle groups, are still limited to a single slice, not analyzed over the entire volume, and do not cover all muscle groups. The recently emerging paper, however, demonstrates that by segmenting the entire pelvic skeletal muscle, the mean difference in skeletal muscle volume is significantly lower than the mean difference for the corresponding region of a single CT slice [13 ].
Malnutrition in patients with cirrhosis often leads to sarcopenia. Currently, however, diagnosing sarcopenia depends on a measure of skeletal muscle mass and number on a single axial slice through the Third lumbar vertebra (L3) on an abdominal or pelvic CT image. This measurement lacks correlation analysis of different types of skeletal muscle with sarcopenia and uses only skeletal muscle on a single axial CT slice, with a smaller sample size.
Accordingly, the invention divides the skeletal muscles on all axial CT slices related to L3 into four types, and automatically calculates the number of the skeletal muscles and the average muscle attenuation; according to the thresholding, the amount of fat and the average HU value are calculated, and thus the present system can automatically measure the body composition of the abdominal CT image.
Disclosure of Invention
The invention aims to provide a body composition automatic measuring system based on abdominal CT images and deep learning, which has high measuring speed and high automation degree.
The automatic body composition measuring system based on the abdominal CT image and the deep learning can automatically acquire the parameters of muscles, fat and the like of all layers corresponding to the third lumbar vertebra, and comprises three modules: (I) the positioning module is used for positioning all axial slices corresponding to the third lumbar vertebra on the CT image; (II) a segmentation module for segmenting four types of skeletal muscles of the axial slice corresponding to the third lumbar vertebra on the CT image; (III) an automatic measurement module for automatic measurement of body composition of clinical CT images. The overall architecture of the system is shown in fig. 1. Wherein:
the positioning module firstly performs CT examination on the abdomen or the abdomen and pelvis of a patient to obtain a CT image including a third lumbar vertebra and other vertebral bodies of the spine; a third lumbar location is located and provided by an experienced physician. And selecting the section corresponding to the third lumbar vertebra as a positive sample input of the positioning network training, and selecting one half of the number of the remaining sections as a negative sample input of the positioning network training. Here, the positioning network outputs the full-link layer setting as class 2 using the pre-training network EfficientNet-b2, and judges whether it is the third lumbar vertebra corresponding slice. Training is carried out to obtain an optimal positioning model, and the positioning module can automatically screen out all axial slices corresponding to the third lumbar vertebra;
(1) the segmentation module extracts four types of skeletal muscles from all axial slices corresponding to the automatically screened third vertebral body, wherein the four types of skeletal muscles are divided according to the spatial distribution and the muscle texture characteristics of the skeletal muscles; taking the axial slice of the third lumbar vertebra segment of the abdominal CT image as the input of a training segmentation network, and outputting a class 4 skeletal muscle label of the segment region; training to obtain an optimal segmentation model; here, the segmentation network adopts a Residual structure enhanced U-Net (2D Residual U-Net) framework widely used for medical image segmentation, and uses the coding and decoding path design for reference.
The automatic measurement module utilizes a positioning model and a segmentation model obtained by training of the former two modules, and takes a new abdomen or abdomen-pelvis CT image and a corresponding third lumbar vertebra position as input, on one hand, according to a deep learning model, the number of four types of skeletal muscles and average muscle attenuation (namely HU value of the CT image) can be calculated, on the other hand, threshold processing is carried out according to HU value intervals of subcutaneous fat and intra-abdominal fat which are [ -29,150], [ -150, -50] respectively, and the fat number and the average muscle attenuation are obtained. The third lumbar volume for each patient is calculated and the third lumbar proportion can be calculated. Finally, automatic measurement of body composition can be achieved.
The three modules are described in further detail below.
-said positioning module (I), the content of which comprises:
(1) the abdomen or abdomen and pelvic cavity CT image comprises a larger abdomen and waist area, so that firstly, a third lumbar vertebra is positioned, and a doctor selects all axial slices of the third lumbar vertebra to form position labels required by a training depth model, and the total number of the position labels is 4-8;
(2) positioning pretreatment: after marking the position of the third lumbar vertebra, selecting a section corresponding to the third lumbar vertebra as a positive sample input, and setting a label to be 1; one half of the number of remaining slices is selected as negative sample input with the label set to 0. And normalizing the HU value to a [0,1] interval;
(3) the CT data of dry cirrhosis patients (216 cases in the embodiment) are used for training a positioning network, the positioning network uses a pre-training network EfficientNet-b2, the full-link layer setting is output as class 2, and whether the slice corresponds to the third lumbar vertebra or not is judged. As shown in fig. 2, all slices corresponding to the third lumbar vertebra are selected, and the model with the highest accuracy is saved as the optimal model.
(II) the segmentation module (II) comprises the following contents:
(1) the section of the third lumbar vertebra acquired by the positioning module is divided into four types of skeletal muscles by experienced doctors, so that the analysis of the multiple types of skeletal muscles on the volume is realized: including the rectus abdominis, the external oblique abdominis, the internal oblique abdominis, the transverse abdominis of L3 abdominal anterior periphery (see Label1 of fig. 3); l3 psoas major, psoas minor, psoas quadratus (see Label2 and Label3 of fig. 3) on the left and right sides of the vertebral body; and paraspinal muscle groups such as the erector spinae muscle posterior to L3 (see Label4 of fig. 3);
(2) segmentation pretreatment: normalizing each CT data, unifying the size to the depth, the height and the width of 8, 512 and 512 respectively, and normalizing the HU value to a [0,1] interval;
(3) a four-class skeletal muscle segmentation network is constructed, and the method is based on a Residual structure enhanced U-Net (2D Residual U-Net) framework widely used for medical image segmentation, and the coding and decoding path design is used for reference. The input to the segmentation network is the abdominal CT image third lumbar segment axial slice, and the output is the class 4 skeletal muscle label for that segment region. Wherein:
1)2D encoding-decoding backbone network
This is a segmentation network for axial slice images. The network is provided with four layers in total and consists of an Encoder branch and a Decoder branch which are provided with jump links. Each layer of the Encoder branch is made up of Block2D, down sampled by a factor of 2 by the maximum pooling layer (max _ pooling), reduced in feature size and passed on to the next layer. Each layer of the Decoder branch is firstly subjected to 2 times of upsampling (Upsample), and after a feature map with the same size as that of a layer corresponding to the Encoder branch is subjected to jump connection of channel direction addition operation, Block2D processing is carried out. In the last layer of the Decoder branch, the features are restored to the same size as the input image, and after the channel direction addition operation is performed with the features output by the 3D coding branch, the segmentation labels are output through a 1x1 convolution kernel.
2) Three-dimensional coding branch structure
This is a contextual feature extraction network of volumetric regions composed of multiple layers of axial slices related to L3, as shown in fig. 4. Each layer of the three-dimensional Encoder branch is made up of Block3D, max pooling layer, and attention enhancement mechanism. The feature graph output by the Block3D is subjected to 2-time down-sampling and halving through a maximum pooling layer (max pooling), the obtained features are transmitted to the next layer on one hand, and on the other hand, after passing through a fuzzy region attention enhancement mechanism, the up-sampling is restored to the original image size, and the feature graph is fused with the output features of the two-dimensional branch circuit through channel addition operation. In the invention, an input L3 axial slice three-dimensional image I is directly input into a network in a channel × depth × height × width mode, and the size is 1 × 8 × 512 × 512. After three times of Block3D and downsampling, feature maps of different sizes are enhanced by an attention mechanism, and channel-add connection is carried out on the feature maps and the two-dimensional features.
3) Block2D and Block3D
The basic structure of the split network is the residual structure in the 2D ResU-Net, but the structure is different, namely improved. Firstly, a convolutional layer is cascaded with InstanceNorm and LeakyReLU to form a basic block; then, three groups of basic blocks are connected by cascade and jump to form Block2D or Block3D with residual structure, respectively, as shown in fig. 5. The convolution kernel of Block2D is 3x3 and the convolution kernel of Block3D is 3x 3. The two structural blocks do not change the number of channels, but can deepen the model, contribute to more precise edge information extraction and have better correction effect on skeletal muscle refinement. In particular, since the three-dimensional encoding branch contains more trainable spatial and texture information and requires more convolutional layers to extract spatial information, positions of residual connection of Block3D and Block2D are different, and many Block3D convolutional layers are provided, so that the extracted information can be sufficiently trained.
4) Texture attention enhancement mechanism
In order to better fuse the 3D features and the 2D features, the system constructs a three-dimensional Texture Attention Enhancement mechanism (Texture Attention Enhancement mechanism) for performing feature compression in a channel direction on a three-dimensional network and enhancing a blurred edge region based on a conventional Standard Enhancement Block (SE Block), as shown in fig. 6. The feature map extracted by Block3D and having the size of channel × depth × height × width is input into the attention mechanism. First, it is subjected to global average pooling operation to obtain a feature map with size of C × 1 × 1 × 1. Then, the neuron number of the first full-connection layer is C/16 (similar to SE Block, the setting ratio is 16), and the original neuron number C is restored by the second full-connection layer. This operation adds non-linear processing and can fit complex correlations between channels. Then generating a probability chart capable of inputting Texture Enhance Block through a Sigmoid function. The proposed Texture Enhance Block can increase the range of attention, as shown in equation (1),
TextureEnhance(x)＝x(1-x) (1)
where x represents the probability map of the input. This formula gives higher weight to edge regions with probability close to 0.5 and reduces weight to regions far from 0.5. By adding this part, the network no longer focuses only on the middle part of the skeletal muscle, but also enhances the thinning of the edges, in particular the shape of the bundle of skeletal muscle fibers. The generated information is weighted by multiplying each pixel of the input feature map. Finally, compressing the channels by Squeeze and Upsample restores the feature map to the original size of 512 × 512 in the height and width directions for merging with the two-dimensional network.
5) Constructing a loss function
For a 2D abdomen CT image I with the size of H multiplied by W, a medical image segmentation task is used, a common multi-class cross entropy loss function is combined with a Dice loss function to segment and obtain four classes of skeletal muscles, as shown in a formula (2),
where C ═ 5 denotes four types of skeletal muscle and background. OmegacRepresents the weight of the c-th skeletal muscle,
a gold standard value indicating that pixel i belongs to a class c label,
indicating that pixel i is predicted as the prediction result of the class c label, and H and W respectively represent the height and width of the two-dimensional axial plane image.
The size of the four types of skeletal muscles differs greatly from the background, which means that there is a problem of imbalance in the types that causes the segmentation frame to be unstable. For this reason, in the training process, penalties need to be made to low confidence predictions (such as Label2 and Label3) by setting weights in the loss function. Specifically, in the training image, the pixel ratio of the four skeletal muscles to the background is counted, and then the skeletal muscle with the smaller ratio is set to have a smaller weight and the skeletal muscle with the larger ratio is set to have a larger weight, as shown in formula (3),
where H, W and D represent the height, width and depth of a two-dimensional image, NcRepresenting the pixel count for the class c label. Thus, from ωcThe prior statistics of (a) ensure a class-balanced optimization of the loss function.
(4) The segmentation network was trained using CT data from several patients with liver cirrhosis (216 in the embodiment) to automatically segment four types of skeletal muscle. And saving the model with the highest Dice index as the optimal segmentation model.
(III) the automatic measurement module (III) comprises the following contents:
(1) after the new abdomen or abdomen-pelvis CT image is subjected to positioning pretreatment, the new abdomen or abdomen-pelvis CT image is input into a positioning optimal model obtained by training, and 4-8 slices corresponding to the third lumbar vertebra are obtained;
(2) after the obtained 4-8 slices corresponding to the third lumbar vertebra are subjected to segmentation pretreatment, inputting the slices into an optimal segmentation model to obtain four skeletal muscles;
(3) meanwhile, carrying out threshold processing on the 4-8 slices corresponding to the obtained third lumbar vertebra to obtain the volume of the third lumbar vertebra and corresponding regions of intra-abdominal fat and subcutaneous fat;
(4) and calculating the quantity and average muscle attenuation of the obtained four types of skeletal muscles, intra-abdominal fat and subcutaneous fat, and realizing automatic measurement of body components.
The invention is trained and verified on CT data of 216 clinical patients with liver cirrhosis, the average Dice of the four types of skeletal muscle segmentation results reaches 0.9283, and the average surface distance is 0.6779 mm. Further generalization on CT data of 101 non-cirrhosis patients showed that the mean Dice was 0.9277 and the mean surface distance was 0.8279 mm. The invention can measure four types of skeletal muscles corresponding to the third lumbar vertebra in the CT image, and can obtain other body components such as subcutaneous fat, intra-abdominal fat and the like in the region by threshold processing. The invention obtains the body components corresponding to the third lumbar vertebra in batches, the use time is 2-3 seconds, and the invention has important significance for clinically assisting the diagnosis of the complications of the liver cirrhosis patients.
The invention has the advantages that:
(1) the automatic body composition measuring system based on the abdominal CT image and the deep learning can automatically measure four types of skeletal muscles corresponding to the third lumbar vertebra, intra-abdominal fat and subcutaneous fat, and is beneficial to diagnosis and treatment of liver cirrhosis complications (such as sarcopenia).
(2) The positioning method of the invention uses the pre-training network, and can quickly and accurately position the third lumbar vertebra. The segmentation method is based on an advanced U-Net framework and is enhanced; considering the characteristics of different types of skeletal muscle sizes, fiber bundle deformation and the like, the segmentation method also combines a texture attention mechanism for enhancing the fuzzy region of the skeletal muscle edge; extracting a 3D coding branch of the fiber bundle characteristic; reducing the loss function of the class imbalance. The system accurately segments multiple types of skeletal muscles of all axial slices related to L3 on more than 300 abdominal or pelvic CT images.
The automatic measurement system uses an optimal model for positioning and dividing the network, and has the advantages of high measurement speed, high automation degree and no need of occupying a large amount of memory compared with other traditional methods.
Drawings
FIG. 1 is a general framework diagram of the system of the present invention.
Fig. 2 is a schematic view of all slices corresponding to the screened third lumbar vertebra.
FIG. 3 is a schematic diagram of four types of skeletal muscle divisions.
FIG. 4 is a schematic diagram of a skeletal muscle segmentation network.
FIG. 5 is a schematic diagram of residual modules in a skeletal muscle segmentation network.
FIG. 6 is a texture feature enhancement module in a skeletal muscle segmentation network.
FIG. 7 is a graph showing the results of muscle fat segmentation in four patients with liver cirrhosis.
Fig. 8 is a flow chart of the application of the system of the present invention.
Detailed Description
The invention is further described below in conjunction with examples and a system overall block diagram.
Taking a clinical raw image acquired from a clinic as an example, the application process of the invention for automatically measuring body components is shown in fig. 8.
Module 1 screens the axial slice corresponding to the third lumbar vertebra. Firstly, positioning preprocessing is carried out on a clinical CT image set of 101 patients with non-cirrhosis, namely, a three-dimensional data set is stored as a picture with an extension name of 'png format', and then random vertical inversion and affine transformation are carried out. All slices were input into the localization model trained from 216 cirrhosis datasets to obtain the prediction result for the third lumbar slice, as shown in table 1. In table 1, "0" indicates that the slice is not an axial slice corresponding to the third lumbar vertebra, and "1" indicates an axial slice corresponding to the third lumbar vertebra. Accuracy represents Accuracy.
Module 2 is a skeletal muscle segmentation process. Firstly, the section corresponding to the third lumbar positioned by the module 1 is subjected to segmentation pretreatment, the size is filled to the depth, the height and the width of 8, 512 and 512 respectively, and the numerical value is normalized to the interval of [0,1 ]. Then, the images are input into a segmentation model obtained by 216 cases of liver cirrhosis data set training, and four types of skeletal muscle segmentation results are obtained.
Tables 1 and 2 show the results of 101 cases of non-cirrhosis data on the localization model and the segmentation model. In the positioning model, the section corresponding to the third lumbar vertebra is used as the golden standard for training, and the higher the accuracy rate obtained in table 1 is, the better the positioning effect is. In the segmentation model, four types of skeletal muscles are used as golden standard training, the segmentation indexes Dice, Sensitivity, Precision and Specificity are higher and better, the ASSD (mm) is smaller and better, and the segmentation indexes and the average value of the four types of skeletal muscles are respectively calculated in Table 2. Here, Dice is a commonly used index for evaluating the image segmentation accuracy, Sensitivity represents Sensitivity, Precision represents accuracy, Specificity represents Specificity, and ASSD represents average surface distance.
TABLE 1.101 statistical results of non-cirrhosis patient CT images in positioning network experiment
TABLE 2.101 statistical results of non-cirrhosis CT images in segmentation network experiment
TABLE 3.101 average body composition of non-cirrhotic CT images
Reference to the literature
[1]Schweitzer L,Geisler C,Pourhassan M,et al.What is the best reference site for a single MRI slice to assess whole-body skeletal muscle and adipose tissue volumes in healthy adults？[J].The American journal of clinical nutrition,2015,102(1):58-65.
[2]Chang C D,Wu J S,Mhuircheartaigh J N,et al.Effect of sarcopenia on clinical and surgical outcome in elderly patients with proximal femur fractures[J].Skeletal radiology,2018,47(6):771-777.
[3]Fuchs G,Chretien Y R,Mario J,et al.Quantifying the effect of slice thickness,intravenous contrast and tube current on muscle segmentation:Implications for body composition analysis[J].European radiology,2018,28(6):2455-2463.
[4]Janne W,Thobias R,Sofia T,et al.Precision of MRI-based body composition measurements of postmenopausal women[J].Plos One,2018,13(2):e0192495.
[5]Lenchik L,Boutin R.Sarcopenia:Beyond Muscle Atrophy and into the New Frontiers of Opportunistic Imaging,Precision Medicine,and Machine Learning[J].Seminars in musculoskeletal radiology,2018,22(03):307-322.
[6]Popuri K,Cobzas D,Esfandiari N,et al.Body Composition Assessment in Axial CT Images Using FEM-Based Automatic Segmentation of Skeletal Muscle[J].IEEE Transactions on Medical Imaging,2016,35(2):512-520.
[7]Dabiri S,Popuri K,Feliciano E M C,et al.Muscle segmentation in axial computed tomography(CT)images at the lumbar(L3)and thoracic(T4)levels for body composition analysis[J].Computerized Medical Imaging and Graphics,2019,75:47-55.
[8]Castiglione J,Somasundaram E,Gilligan L A,et al.Automated segmentation of abdominal skeletal muscle on pediatric CT scans using deep learning[J].Radiology:Artificial Intelligence,2021,3(2):e200130.
[9]Park H J,Shin Y,Park J,et al.Development and validation of a deep learning system for segmentation of abdominal muscle and fat on computed tomography[J].Korean journal of radiology,2020,21(1):88-100.
[10]Burns J E,Yao J,Chalhoub D,et al.Amachine learning algorithm to estimate sarcopenia on abdominal CT[J].Academic radiology,2020,27(3):311-320.
[11]Jiawei H,Shen H,Chen B,et al.Segmentation of Paraspinal Muscles at Varied Lumbar Spinal Levels by Explicit Saliency-Aware Learning[C]//23th International Conference on Medical Image Computing and Computer Assisted Intervention(MICCAI).2020.
[12]Barnard R,Tan J,Roller B,et al.Machine learning for automatic paraspinous muscle area and attenuation measures on low-dose chest CT scans[J].Academic radiology,2019,26(12):1686-1694.
[13]Borrelli P,Kaboteh R,Enqvist O,et al.Artificial intelligence-aided CT segmentation for body composition analysis:a validation study[J].European Radiology Experimental,2021,5(1):1-6。
Claims (6)
1. The utility model provides a body composition automatic measuring system based on belly CT image and deep learning which characterized in that can obtain the muscle of third lumbar vertebrae corresponding all levels, fat parameter automatically, includes three module:
the positioning module is used for positioning all axial slices corresponding to the third lumbar vertebra on the CT image;
the segmentation module is used for segmenting four types of skeletal muscles of axial slices corresponding to a third lumbar vertebra on the CT image;
an automatic measurement module for automatic measurement of body composition of clinical CT images;
wherein:
the positioning module firstly performs CT examination on the abdomen or the abdomen and pelvis of a patient to obtain a CT image including a third lumbar vertebra and other vertebral bodies of the spine; a third lumbar location is located and provided by an experienced physician; selecting the section corresponding to the third lumbar vertebra as a positive sample input of the positioning network training, and selecting one half of the number of the remaining sections as a negative sample input of the positioning network training; here, the positioning network uses a pre-training network EfficientNet-b2 to output the full-link layer settings as class 2 and judge whether the slice corresponds to the third lumbar vertebra; training to obtain an optimal positioning model;
(1) the segmentation module extracts four types of skeletal muscles from all axial slices corresponding to the automatically screened third vertebral body, wherein the four types of skeletal muscles are divided according to the spatial distribution and the muscle texture characteristics of the skeletal muscles; taking the axial slice of the third lumbar vertebra segment of the abdominal CT image as the input of a training segmentation network, and outputting a class 4 skeletal muscle label of the segment region; training to obtain an optimal segmentation model; the segmentation network adopts a residual error structure enhanced U-Net framework for medical image segmentation, and the coding and decoding path design is used for reference;
the automatic measurement module utilizes a positioning model and a segmentation model obtained by training of the first two modules, and takes a new abdomen or abdomen-pelvis CT image and a corresponding third lumbar vertebra position as input, on one hand, the number of four types of skeletal muscles and average muscle attenuation, namely an HU value of the CT image, are calculated according to a deep learning model; on the other hand, threshold processing is carried out according to HU value intervals of subcutaneous fat and intra-abdominal fat of [ -29,150], [ -150 and-50 ] respectively to obtain fat quantity and average muscle attenuation; and calculating the volume of the third lumbar vertebra of each patient, calculating the proportion of the third lumbar vertebra, and finally realizing automatic measurement of body components.
2. The system of claim 1, wherein the positioning module, its contents comprise:
(1) because the abdomen or abdomen-pelvic cavity CT image contains a larger abdomen-waist area, firstly, a third lumbar vertebra is positioned, and a doctor selects all axial slices of the third lumbar vertebra to form position labels required by a training depth model, wherein the number of the position labels is 4-8;
(2) positioning pretreatment: after marking the position of the third lumbar vertebra, selecting a section corresponding to the third lumbar vertebra as a positive sample input, and setting a label to be 1; one half of the number of remaining slices is selected as negative sample input with the label set to 0. And normalizing the HU value to a [0,1] interval;
(3) training a positioning network by utilizing CT data of patients with cirrhosis, wherein the positioning network uses a pre-training network EfficientNet-b2, the full connecting layer is set and output to be 2 types, and whether the slice corresponds to a third lumbar vertebra is judged; and screening all the slices corresponding to the third lumbar vertebra, and storing the model with the highest accuracy as the optimal model.
3. The system of claim 2, wherein the segmentation module, its contents comprise:
(1) the section of the third lumbar vertebra acquired by the positioning module is divided into four types of skeletal muscles by experienced doctors, so that the analysis of the multiple types of skeletal muscles on the volume is realized: including the abdominal rectus muscle, abdominal external oblique muscle, abdominal internal oblique muscle and abdominal transverse muscle of the front and the periphery of the abdomen of L3; l3 major, minor, and quadratus lumborum on the left and right sides of the vertebral body; and paraspinal muscle groups such as the erector spinae muscle posterior to L3;
(2) segmentation pretreatment: normalizing each CT data, unifying the size to the depth, the height and the width of 8, 512 and 512 respectively, and normalizing the HU value to a [0,1] interval;
(3) constructing four types of skeletal muscle segmentation networks, wherein the four types of skeletal muscle segmentation networks are based on a U-Net framework which is widely used for residual structure enhancement of medical image segmentation, and coding and decoding path design of the four types of skeletal muscle segmentation networks is used for reference; the input to the segmentation network is the abdominal CT image third lumbar segment axial slice, and the output is the class 4 skeletal muscle label for that segment region.
(4) Training a segmentation network by utilizing CT data of a plurality of patients with liver cirrhosis, and automatically segmenting four types of skeletal muscles; and saving the model with the highest Dice index as the optimal segmentation model.
4. The system of claim 3, wherein the segmentation module is configured to:
1)2D encoding-decoding backbone network
A segmentation network for axial slice images; the network is provided with four layers in total and consists of an Encoder branch and a Decoder branch which are provided with jump links; each layer of the Encoder branch consists of Block2D, 2 times of down sampling is carried out through the maximum pooling layer, the size of the feature map is reduced, and the feature map is transmitted to the next layer; each layer of the Decoder branch is firstly subjected to 2 times of upsampling, and after jump connection of channel direction addition operation is carried out on feature maps with the same size as the corresponding layer of the Encoder branch, Block2D processing is carried out; in the last layer of the Decoder branch, the features are restored to be the same as the size of the input image, and after the addition operation in the channel direction is carried out on the features and the features output by the 3D coding branch, a segmentation label is output through a 1x1 convolution kernel;
2) three-dimensional coding branch structure
The three-dimensional Encoder branch is a context feature extraction network of a body region consisting of L3 related multi-layer axial slices, and each layer of the three-dimensional Encoder branch consists of a Block3D, a maximum pooling layer and an attention enhancement mechanism; the feature graph output by Block3D is subjected to 2-time down-sampling and halving through a maximum pooling layer, the obtained features are transmitted to the next layer on one hand, and on the other hand, after passing through a fuzzy region attention enhancement mechanism, the up-sampling is recovered to the original image size, and the feature graph is fused with the output features of the two-dimensional branch circuit through channel direction addition operation;
3) block2D and Block3D
As a basic structure of a segmentation network, a residual error structure in the 2D ResU-Net is used for reference; wherein, the convolution layer is cascaded with InstanceNorm and LeakyReLU to form a basic block; three groups of basic blocks respectively form a Block2D or a Block3D with a residual error structure through cascade connection and jump connection; the convolution kernel of Block2D is 3x3, and the convolution kernel of Block3D is 3x 3;
4) texture attention enhancement mechanism
In order to better fuse the 3D features and the 2D features, a three-dimensional texture attention enhancing mechanism for compressing the features in the channel direction on a three-dimensional network and enhancing a fuzzy edge region is constructed based on a traditional standard enhancing Block (SE Block); inputting a feature diagram with the size of channel multiplied by depth multiplied by height multiplied by width extracted by Block3D into an attention mechanism; firstly, carrying out global average pooling operation on the feature map to obtain a feature map with the size of C multiplied by 1; then passing through two full-connection layers, wherein the number of neurons in the first full-connection layer is C/16, and the number of original neurons in the second full-connection layer is restored to C; this operation adds non-linear processing and can fit complex correlations between channels; then generating a probability chart capable of inputting Texture Enhance Block through a Sigmoid function; the Texture Enhance Block can increase the range of attention, as shown in equation (1),
TextureEnhance(x)＝x(1-x) (1)
wherein x represents a probability map of the input; this formula gives higher weight to edge regions with probability close to 0.5, and reduces weight for regions far from 0.5; by adding this part, the network no longer focuses only on the middle part of the skeletal muscle, but also enhances the thinning of the edges, in particular the shape of the skeletal muscle fiber bundles; the generated information is weighted by multiplying each pixel of the input feature map; finally, compressing the channels by Squeeze and Upsample restores the feature map to the original size of 512 × 512 in the height and width directions for merging with the two-dimensional network.
5. The system of claim 4, wherein the loss function trained by the segmentation module is constructed to:
for a 2D abdomen CT image I with the size of H multiplied by W, in a medical image segmentation task, combining a plurality of types of cross entropy loss functions with a Dice loss function to segment and obtain four types of skeletal muscles, wherein a loss function formula is shown as (2),
wherein, C ═ 5, represents four types of skeletal muscle and background; omegacRepresents the weight of the c-th skeletal muscle,
a gold standard value indicating that pixel i belongs to a class c label,
the prediction result of the pixel i as a class c label is represented, and H and W respectively represent the height and the width of the two-dimensional axial plane image; weight ωcThe following were determined:
firstly, in a training image, the pixel proportion of four skeletal muscles and a background is counted, then the skeletal muscle with small proportion is set with small weight, the skeletal muscle with large proportion is set with large weight, as shown in a formula (3),
where H, W and D represent the height, width and depth of a two-dimensional image, NcIndicates class cAnd counting the pixel number of the label.
(5) The CT data of a plurality of patients with liver cirrhosis are used for training a segmentation network, and four types of skeletal muscles are automatically segmented. And saving the model with the highest Dice index as the optimal segmentation model.
6. The system of claim 5, wherein the automated measurement module comprises:
(1) after the new abdomen or abdomen-pelvis CT image is subjected to positioning pretreatment, the new abdomen or abdomen-pelvis CT image is input into a positioning optimal model obtained by training, and 4-8 slices corresponding to the third lumbar vertebra are obtained;
(2) after the obtained 4-8 slices corresponding to the third lumbar vertebra are subjected to segmentation pretreatment, inputting the slices into an optimal segmentation model to obtain four skeletal muscles;
(3) meanwhile, carrying out threshold processing on the 4-8 slices corresponding to the obtained third lumbar vertebra to obtain the volume of the third lumbar vertebra and corresponding regions of intra-abdominal fat and subcutaneous fat;
(4) and calculating the quantity and average muscle attenuation of the obtained four types of skeletal muscles, intra-abdominal fat and subcutaneous fat, and realizing automatic measurement of body components.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210068304.6A CN114305473A (en) | 2022-01-20 | 2022-01-20 | Body composition automatic measuring system based on abdomen CT image and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210068304.6A CN114305473A (en) | 2022-01-20 | 2022-01-20 | Body composition automatic measuring system based on abdomen CT image and deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114305473A true CN114305473A (en) | 2022-04-12 |
Family
ID=81028053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210068304.6A Pending CN114305473A (en) | 2022-01-20 | 2022-01-20 | Body composition automatic measuring system based on abdomen CT image and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114305473A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116309385A (en) * | 2023-02-27 | 2023-06-23 | 之江实验室 | Abdominal fat and muscle tissue measurement method and system based on weak supervision learning |
| CN116758087A (en) * | 2023-08-22 | 2023-09-15 | 邦世科技(南京)有限公司 | Lumbar vertebra CT bone window side recess gap detection method and device |
-
2022
- 2022-01-20 CN CN202210068304.6A patent/CN114305473A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116309385A (en) * | 2023-02-27 | 2023-06-23 | 之江实验室 | Abdominal fat and muscle tissue measurement method and system based on weak supervision learning |
| CN116309385B (en) * | 2023-02-27 | 2023-10-10 | 之江实验室 | Abdominal fat and muscle tissue measurement method and system based on weak supervision learning |
| CN116758087A (en) * | 2023-08-22 | 2023-09-15 | 邦世科技(南京)有限公司 | Lumbar vertebra CT bone window side recess gap detection method and device |
| CN116758087B (en) * | 2023-08-22 | 2023-10-31 | 邦世科技(南京)有限公司 | Lumbar vertebra CT bone window side recess gap detection method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ahammad et al. | Image processing based segmentation techniques for spinal cord in MRI | |
| US7813536B2 (en) | Image measuring apparatus and method, and image measuring system for glomerular filtration rate | |
| CN114305473A (en) | Body composition automatic measuring system based on abdomen CT image and deep learning | |
| Bhadoria et al. | Comparison of segmentation tools for multiple modalities in medical imaging | |
| CN111429474B (en) | Mammary gland DCE-MRI image focus segmentation model establishment and segmentation method based on mixed convolution | |
| Marzola et al. | Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment | |
| CN110288698B (en) | Meniscus three-dimensional reconstruction system based on MRI | |
| CN113409309B (en) | Muscle CT image sketching method, system, electronic equipment and machine storage medium | |
| CN110991254B (en) | Ultrasonic image video classification prediction method and system | |
| CN113222951B (en) | Osteoporosis artificial intelligent diagnosis device for recognizing hip joint X-ray | |
| CN107292858A (en) | A kind of multimode medical image fusion method based on low-rank decomposition and rarefaction representation | |
| CN116228624A (en) | Multi-mode constitution component marking and analyzing method based on artificial intelligence technology | |
| Hess et al. | Deep learning for multi-tissue segmentation and fully automatic personalized biomechanical models from BACPAC clinical lumbar spine MRI | |
| Le et al. | Automatic segmentation of mandibular ramus and condyles | |
| CN110738633A (en) | organism tissue three-dimensional image processing method and related equipment | |
| CN116758087B (en) | Lumbar vertebra CT bone window side recess gap detection method and device | |
| Ramos et al. | Fast and smart segmentation of paraspinal muscles in magnetic resonance imaging with CleverSeg | |
| CN112967254A (en) | Lung disease identification and detection method based on chest CT image | |
| CN116433620A (en) | CT image-based bone mineral density prediction and osteoporosis intelligent screening method and system | |
| CN115953416A (en) | Automatic knee bone joint nuclear magnetic resonance image segmentation method based on deep learning | |
| Michopoulou | Image analysis for the diagnosis of MR images of the lumbar spine | |
| CN115908449A (en) | 2.5D medical CT image segmentation method and device based on improved UNet model | |
| CN115294023A (en) | Liver tumor automatic segmentation method and device | |
| Yan et al. | Segmentation of pulmonary parenchyma from pulmonary CT based on ResU-Net++ model | |
| Shen et al. | Deep Learning on Knee CT Scans from Osteoarthritis Patients for Joint Space Assessment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |