CN112802028A - Image processing method and device for mediastinal organ segmentation - Google Patents
Image processing method and device for mediastinal organ segmentation Download PDFInfo
- Publication number
- CN112802028A CN112802028A CN201911111737.XA CN201911111737A CN112802028A CN 112802028 A CN112802028 A CN 112802028A CN 201911111737 A CN201911111737 A CN 201911111737A CN 112802028 A CN112802028 A CN 112802028A
- Authority
- CN
- China
- Prior art keywords
- segmentation
- image data
- organ
- image
- semantic segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Abstract
The application discloses an image processing method and device for mediastinal organ segmentation, computer equipment and a readable storage medium. The method comprises the following steps: inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion; performing example segmentation on image data of tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result to obtain an example segmentation result; and outputting an image segmentation result of the image data of the tomography of the mediastinum organ according to the semantic segmentation result and the example segmentation result. The method and the device solve the technical problem that image detection of the mediastinum total disease species in the related technology lacks a detection method with good detection performance. Through the method and the device, the purpose of image detection of the whole disease species of mediastinum is achieved, and therefore the technical effect of improving the image detection efficiency of the whole disease species of mediastinum is achieved.
Description
Technical Field
The present application relates to the field of computer vision, and in particular, to an image processing method and apparatus, a computer device, and a readable storage medium for mediastinal organ segmentation.
Background
The deep learning method is widely applied to the field of computer vision, in particular to the field of more basic semantic segmentation and example segmentation, and is successfully applied to the segmentation and detection tasks of medical CT images. In the field of semantic segmentation, Ronneberger et al use an encoder-decoder structure to optimize the extraction of the network to the texture detail information in the input image and obtain a more accurate segmentation result; milletari et al replace 2D operations with operations such as 3D convolution, pooling, and the like, can realize end-to-end image semantic segmentation of 3D images, and are suitable for medical image segmentation of three-dimensional structures. In the example segmentation field, He et al introduces a branching network for segmentation based on an object detection framework, achieving excellent example segmentation effects. Chen et al propose a multitasking multi-stage hybrid cascading instance partitioning framework that achieves superior results over Mask R-CNN, and adds a branch of semantic partitioning to enhance context information.
At present, no related technology relates to detection of the whole disease species of mediastinum, because the related disease species are more, and the detection scheme for each disease is more. For example, aortic, cardiac, lymphatic, and thyroid disorders can all be detected using case segmentation, or cardiac disorders can be detected individually using 3D semantic segmentation. Each solution has advantages and disadvantages, and the performance difference is large.
Aiming at the problem that the image detection of the mediastinum total disease species in the related technology lacks a detection method with good detection performance, an effective solution is not provided at present.
Disclosure of Invention
The present application mainly aims to provide an image processing method and apparatus for mediastinal organ segmentation, a computer device, and a readable storage medium, so as to solve the problem that detection of a total disease category of mediastinal is lack of a detection method with good detection performance in the related art.
To achieve the above object, according to a first aspect of the present application, an image processing method for mediastinal organ segmentation is provided.
The image processing method for mediastinal organ segmentation according to the application comprises the following steps: inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion; performing example segmentation on image data of tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result to obtain an example segmentation result; and outputting an image segmentation result of the image data of the tomography of the mediastinum organ according to the semantic segmentation result and the example segmentation result.
Further, the preset semantic segmentation model comprises a first semantic segmentation model, and the method for performing semantic segmentation on image data of tomography of the mediastinum organ by inputting the image data into the preset semantic segmentation model and obtaining a semantic segmentation result by fusion comprises the following steps: preprocessing the marked image data to obtain a plurality of slice image data; grouping a plurality of image data of the slices to obtain a plurality of groups of training data images; and respectively training the multiple groups of training data images to obtain the first semantic segmentation model.
Further, the preset semantic segmentation model comprises a plurality of sub-network models; the method comprises the following steps of inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion, wherein the semantic segmentation result comprises the following steps: training the image data after data segmentation according to a first preset rule to obtain a plurality of sub-network models; inputting the image data into a plurality of sub-network models respectively to obtain segmentation results of the plurality of sub-network models; and carrying out average processing on the segmentation results of the plurality of sub-network models to obtain the semantic segmentation result.
Further, the performing, based on the semantic segmentation result, an example segmentation on the image data of the tomography of the mediastinal organ through a preset example segmentation model to obtain an example segmentation result includes: acquiring segmentation label information of the semantic segmentation result; and concatenating the segmentation label information in the convolution layer of the preset instance segmentation model to obtain the instance segmentation result.
Further, the performing, based on the semantic segmentation result, an example segmentation on the image data of the tomography of the mediastinal organ through a preset example segmentation model to obtain an example segmentation result includes: performing data segmentation on the marked image data according to a second preset rule to obtain a plurality of groups of training data images; respectively training the multiple groups of training data images to obtain the preset example segmentation model; and performing data segmentation on the image data which is not marked according to a second preset rule and inputting the image data into the preset example segmentation model to obtain the example segmentation result.
Further, the outputting an image segmentation result of the image data of the tomography of the mediastinal organ according to the semantic segmentation result and the example segmentation result comprises: measuring a mediastinum organ image in the image segmentation result to obtain a measurement result of the mediastinum organ image; and comparing the measurement result of the image of the mediastinum organ with a preset threshold value of the image of the mediastinum organ to obtain a detection result of the image of the mediastinum organ.
In order to achieve the above object, according to a second aspect of the present application, there is provided an image processing apparatus for mediastinal organ segmentation.
An image processing apparatus for mediastinal organ segmentation according to the present application includes: the first segmentation module is used for inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation and obtaining a semantic segmentation result through fusion; the second segmentation module is used for carrying out example segmentation on the image data of the tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result so as to obtain an example segmentation result; and the output module is used for outputting an image segmentation result of the image data of the tomography of the mediastinum organ according to the semantic segmentation result and the example segmentation result.
Further, the preset semantic segmentation model comprises a first semantic segmentation model, and the device further comprises: the segmentation module is used for preprocessing the marked image data to obtain a plurality of slice image data; the grouping module is used for grouping the plurality of image data of the slices to obtain a plurality of groups of training data images; and the training module is used for respectively training the multiple groups of training data images to obtain the first semantic segmentation model.
To achieve the above object, according to a third aspect of the present application, there is provided a computer apparatus comprising: one or more processors; storage means for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as previously described.
In order to achieve the above object, according to a fourth aspect of the present application, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method as described above.
In the embodiment of the application, image data of tomography of the mediastinum organ is input into a preset semantic segmentation model for semantic segmentation, and a semantic segmentation result is obtained through fusion; based on the semantic segmentation result, the image data of the tomography of the mediastinal organ is subjected to instance segmentation through a preset instance segmentation model so as to obtain an instance segmentation result, and the image segmentation result of the image data of the tomography of the mediastinal organ is output according to the semantic segmentation result and the instance segmentation result, so that the purpose of image detection of the whole mediastinal disease is achieved, the technical effect of improving the image detection efficiency of the whole mediastinal disease is achieved, and the technical problem that the image detection of the whole mediastinal disease lacks a detection method with better detection performance in the related technology is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, serve to provide a further understanding of the application and to enable other features, objects, and advantages of the application to be more apparent. The drawings and their description illustrate the embodiments of the invention and do not limit it. In the drawings:
FIG. 1 is a schematic flow diagram of an image processing method for mediastinal organ segmentation according to a first embodiment of the present application;
FIG. 2 is a schematic flow chart of an image processing method for mediastinal organ segmentation according to a second embodiment of the present application;
FIG. 3 is a schematic flow chart of an image processing method for mediastinal organ segmentation according to a third embodiment of the present application;
FIG. 4 is a flow diagram illustrating semantic segmentation of a mediastinal organ image according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of an image processing method for mediastinal organ segmentation according to a fourth embodiment of the present application;
FIG. 6 is a schematic flow chart of an image processing method for mediastinal organ segmentation according to a fifth embodiment of the present application;
FIG. 7 is a schematic flow chart of an image processing method for mediastinal organ segmentation according to a sixth embodiment of the present application;
FIG. 8 is a schematic diagram of a full flow of image detection and measurement of a mediastinal organ according to an embodiment of the present application; and
fig. 9 is a schematic structural diagram of the components of an image processing apparatus for mediastinal organ segmentation according to the first embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
According to an embodiment of the present invention, there is provided an image processing method for mediastinal organ segmentation, as shown in fig. 1, the method including steps S101 to S103 as follows:
step S101, inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion.
During specific implementation, firstly, massive labeled mediastinum organ flat-scan CT image data are collected to be used as training samples and input into a preset network model for training so as to obtain the preset semantic segmentation model, and then unlabeled mediastinum organ flat-scan CT image data are used as samples to be tested and input into the trained preset semantic model for semantic segmentation of images. The flat-scan CT image data of the mediastinal organ may include image data of mediastinal organs such as aorta, pulmonary artery, trachea, esophagus, heart, thyroid nodule and lymph nodule.
Preferably, a plurality of preset semantic segmentation models are provided, for example, a 2.5D network model and a 3D network model may be adopted, the unlabeled CT image data of the mediastinal organ under flat scan is respectively input into the 2.5D network model and the 3D network model to perform semantic segmentation of the image, and finally, the segmentation results of the 2.5D network model and the 3D network model are weighted and fused to obtain a final semantic segmentation result.
And S102, carrying out example segmentation on image data of tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result to obtain an example segmentation result.
The core of semantic segmentation is to determine the category of pixels, and the core of example segmentation is to determine the category and specific entity of pixels, so the example segmentation is a further identification process performed on the basis of the semantic segmentation, and therefore, when image segmentation is performed on mediastinal organs such as thyroid nodules and lymph nodules, in the embodiment of the application, based on the semantic segmentation result of the images of the mediastinal organs such as thyroid nodules and lymph nodules, example segmentation is further performed on the CT image data of the mediastinal organ flat scan by using a pre-trained example segmentation model such as a Mask R-CNN network model, so as to obtain an example segmentation result.
Step S103, outputting an image segmentation result of image data of tomography scanning of the mediastinum organ according to the semantic segmentation result and the example segmentation result.
In specific implementation, the image pixel classification results of the aorta, the pulmonary artery, the trachea, the esophagus, the heart, the thyroid nodule, the lymph nodule and other mediastinal organs are obtained through the semantic segmentation results, the example segmentation results of the thyroid nodule and the lymph nodule are further obtained based on the semantic segmentation results, and the segmentation results are summarized and output to serve as a basis for judging whether the mediastinal organs have lesions or not according to the measurement results of the images of the mediastinal organs. Through the process, the purpose of image segmentation and detection of the image data of the whole disease species of the mediastinum is achieved, the good detection performance of each detection method is integrated, and the accuracy and the efficiency of image detection of the whole disease species of the mediastinum are improved.
As a preferred implementation manner of the embodiment of the present application, as shown in fig. 2, the preset semantic segmentation model includes a first semantic segmentation model, and the following steps S201 to S203 are included before the image data of the tomography of the mediastinal organ is input into the preset semantic segmentation model for semantic segmentation and a semantic segmentation result is obtained by fusion:
step S201, pre-processing the marked image data to obtain a plurality of slice image data.
In specific implementation, when performing semantic segmentation on mediastinal organ flat-scan CT image data, a 2.5D network model may be used as a first semantic segmentation model to perform semantic segmentation of an image, and when performing model training on the 2.5D network model, firstly, labeled flat-scan CT image data needs to be acquired as training data, and further, 3D image data segmentation is performed on the labeled flat-scan CT image data to obtain a plurality of slice image data.
Step S202, grouping the plurality of sliced image data to obtain a plurality of sets of training data images.
In a specific implementation, after the plurality of slice image data are obtained, 7 slice image data may be used as a group to form a plurality of training data images as training samples. Of course, the specific grouping method of slice data can be flexibly adjusted by those skilled in the art according to the actual situation, and is not limited specifically herein.
Step S203, training the plurality of sets of training data images respectively to obtain the first semantic segmentation model.
In specific implementation, the obtained multiple groups of training data images are respectively input into a 2.5D network model for training to obtain a final first semantic segmentation model.
As a preferred implementation manner of the embodiment of the present application, as shown in fig. 3, the preset semantic segmentation model includes a plurality of sub-network models; the steps S301 to S303 are that the image data of the tomography of the mediastinal organ is input into a preset semantic segmentation model for semantic segmentation, and a semantic segmentation result is obtained by fusion:
step S301, performing data segmentation on the image data according to a first preset rule, and then training the image data respectively to obtain a plurality of sub-network models.
In specific implementation, the preset semantic segmentation model may include a plurality of 2.5D segmentation network models, and before training a training sample to obtain a plurality of 2.5D segmentation network models, the marked flat-scan CT image data needs to be segmented into 3D data in x, y, and z directions, and then the 3 sub-network models are obtained by training respectively.
Step S302 is to input the image data to the plurality of sub-network models, respectively, to obtain a segmentation result of the plurality of sub-network models.
In specific implementation, when performing image segmentation on unlabeled flat-scan CT image data, it is also necessary to perform segmentation on the unlabeled flat-scan CT image data in three directions, namely x, y, and z, to perform 3D data segmentation, and then input the segmented image data into the trained 3 sub-network models respectively for image segmentation, so as to obtain image segmentation results of the 3 sub-network models.
Step S303, performing average processing on the segmentation results of the plurality of sub-network models to obtain the semantic segmentation result.
In specific implementation, the slice segmentation results output by the 3 2.5D segmentation networks need to be spliced to obtain a complete 3D flat-scan CT segmentation mask. The specific splicing method may be to average the slice segmentation results output by the 3 2.5D segmentation networks, and take the average as the final segmentation result of the 2.5D semantic segmentation model.
Preferably, the preset semantic segmentation model may further include a second semantic segmentation model such as a 3D segmentation network model, and when performing model training, the original 3D flat-scan CT image data is converted into image blocks of 128 × 128 pixels and input into the network, and is trained by using a dice loss function. The 3D segmentation network model can use an improved V-Net network architecture, and then segmentation tests of the image of the mediastinal organ are carried out on the flat-scan CT image without the label, so that the segmentation labels of all important organs are obtained.
And finally, performing weighted fusion processing on the segmentation results of the 2.5D segmentation network model and the 3D segmentation network model to obtain a final image segmentation result of the mediastinal organ.
As shown in fig. 4, a schematic flow chart of image semantic segmentation of total mediastinal diseases is provided, and the specific process is described above in detail and is not repeated here.
As a preferred implementation manner of the embodiment of the present application, as shown in fig. 5, the performing, based on the semantic segmentation result, an example segmentation on the image data of the tomography of the mediastinal organ by using a preset example segmentation model to obtain an example segmentation result includes steps S401 to S402 as follows:
step S401, obtaining the segmentation label information of the semantic segmentation result.
In specific implementation, when performing example segmentation of images of mediastinal organs such as thyroid nodules and lymph nodules, firstly, semantic segmentation results corresponding to CT image data of the thyroid nodules and the lymph nodules need to be obtained, and segmentation labels corresponding to the semantic segmentation results are determined.
Step S402, concatenating the segmentation label information in the convolution layer of the preset instance segmentation model to obtain the instance segmentation result.
In specific implementation, the preset instance segmentation model may adopt a Mask R-CNN network architecture, and segment label information obtained from the semantic segmentation result is concatenated at an rpn (region pro-possible networks) network portion and a convolution layer of a last layer, so as to improve the performance of instance segmentation.
As a preferred implementation manner of the embodiment of the present application, as shown in fig. 6, the performing, based on the semantic segmentation result, an example segmentation on the image data of the tomography of the mediastinal organ by using a preset example segmentation model to obtain an example segmentation result includes steps S501 to S503 as follows:
step S501, performing data segmentation on the marked image data according to a second preset rule to obtain multiple groups of training data images.
In specific implementation, when a preset instance segmentation model is constructed, firstly, data of a 3D flat scan CT image, which is manually labeled, for example, obtained in a contour drawing manner, is segmented in the z-axis direction, wherein 3 slices are taken as a group to form a plurality of groups of training data images, which are taken as an input part of the model.
Step S502, the multiple groups of training data images are respectively trained to obtain the preset example segmentation model.
In specific implementation, the multiple groups of training data are input into a network architecture of Mask R-CNN for training to obtain the preset example segmentation model.
Step S503, performing data segmentation on the image data that is not labeled according to a second preset rule, and inputting the image data into the preset instance segmentation model to obtain the instance segmentation result.
In specific implementation, when the example segmentation is performed on images of mediastinal organs such as thyroid nodules and lymph nodules, the image data of the mediastinal organs such as the thyroid nodules and the lymph nodules which are not marked are subjected to data segmentation in the z-axis direction, and the segmented image data are input into the trained example segmentation model for example segmentation, so that an example segmentation result of each piece of slice data is obtained.
As a preferred implementation manner of the embodiment of the present application, as shown in fig. 7, after the outputting of the image segmentation result of the image data of the tomography of the mediastinal organ according to the semantic segmentation result and the example segmentation result, steps S601 to S602 include as follows:
step S601, measuring the image of the mediastinal organ in the image segmentation result to obtain a measurement result of the image of the mediastinal organ.
In specific implementation, after the image segmentation result of each mediastinal organ is obtained, it is further required to determine whether the mediastinal organ has disease symptoms by measuring or the like according to the image segmentation result. For example, on the basis of the aorta segmentation result output by the semantic segmentation network, the aorta inside diameter value in each slice database image in the z-axis direction is measured, and on the basis of the heart segmentation and the lung segmentation output by the semantic segmentation network, the heart diameter value and the thoracic inside diameter value in each slice data image in the z-axis direction are measured, so as to obtain the heart major diameter and the corresponding thoracic inside diameter.
Step S602, comparing the measurement result of the image of the mediastinal organ with a preset threshold of the image of the mediastinal organ to obtain a detection result of the image of the mediastinal organ.
In specific implementation, the measurement result of each image of the mediastinal organ needs to be compared with a preset threshold of the image of the mediastinal organ to determine whether the mediastinal organ has disease symptoms. For example, on the basis of the measured aorta inner diameter, the relative sizes of the ascending aorta inner diameter length and the descending aorta inner diameter length in each slice data image are compared, and if the difference is more than 1.5 times, or the ascending aorta inner diameter length is more than 4.5cm, or the descending aorta inner diameter length is more than 4cm, then the aorta expansion or aortic aneurysm symptom is considered to exist. On the basis of the measurement results of the heart diameter value and the thoracic diameter value, if the ratio of the maximum heart diameter to the corresponding thoracic diameter is 0.5 or more, it is considered that there is a sign of cardiac enlargement.
Preferably, on the basis of the aorta segmentation result output by the semantic segmentation network, comparing the CT data value of the corresponding aorta in the original CT image with the calcified lesion CT data value, and if the CT data value of the aorta is greater than the calcified lesion threshold, determining that the aorta calcification is present there.
Preferably, if a thyroid nodule is detected by the example segmentation network, then the thyroid disease symptom is considered to be present there. In addition, on the basis of the lymph node segmentation output by the example segmentation network, the internal diameter of each segmented lymph node is measured, and if the long diameter value of the lymph node is more than 1.5cm or the short diameter value of the lymph node is more than 1cm, the lymph node is considered to be in a swollen state.
As shown in fig. 8, a schematic diagram of a whole flow of image detection and measurement of a whole disease species of mediastinum is provided, and the specific process is described above in detail and is not repeated herein.
From the above description, it can be seen that the present invention achieves the following technical effects: according to the method, on a flat-scan CT image, 2.5D and 3D network models are used for segmenting images of organs such as aorta, pulmonary artery, trachea, esophagus and heart, and fusion operation is carried out on the output of the 2.5D and 3D network models to obtain a final segmentation result. On the basis of the segmentation of the aorta and the heart, whether the symptoms such as aortic calcification and the like exist is searched according to a threshold value; according to the measurement and comparison of the ascending aorta inner diameter and the descending aorta inner diameter, whether the aorta is dilated or the aortic aneurysm is searched; on the flat-scan CT image, normal and diseased data are used to segment the heart and chest CT images, and the presence or absence of heart augmentation is determined by comparing the maximum diameter of the heart with the corresponding inside diameter of the thorax. Marking thyroid and lymphatic diseases by adopting a contour drawing mode, and detecting the thyroid and lymphatic diseases by adopting an example segmentation method. And finally, judging whether the disease of lymph node enlargement exists or not by measuring the inner diameter of the lymph node obtained by dividing the example. Through the process, the CT images of the mediastinum total disease species are quickly and accurately segmented and detected, and the efficiency of detecting the CT images of the mediastinum total disease species is improved.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
According to an embodiment of the present invention, there is also provided an apparatus for implementing the above image processing method for mediastinal organ segmentation, as shown in fig. 9, the apparatus including: a first segmentation module 1, a second segmentation module 2 and an output module 3. The first segmentation module 1 of the embodiment of the application is configured to input image data of tomography of a mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtain a semantic segmentation result through fusion. The second segmentation module 2 of the embodiment of the application is configured to perform instance segmentation on the image data of the tomography of the mediastinal organ through a preset instance segmentation model based on the semantic segmentation result to obtain an instance segmentation result. The output module 3 of the embodiment of the application is configured to output an image segmentation result of image data of tomography of the mediastinal organ according to the semantic segmentation result and the example segmentation result.
As a preferred implementation manner of the embodiment of the present application, the preset semantic segmentation model includes a first semantic segmentation model, and the apparatus further includes: the segmentation module is used for preprocessing the marked image data to obtain a plurality of slice image data; the grouping module is used for grouping the plurality of image data of the slices to obtain a plurality of groups of training data images; and the training module is used for respectively training the multiple groups of training data images to obtain the first semantic segmentation model.
As a preferred implementation manner of the embodiment of the present application, the preset semantic segmentation model includes a plurality of sub-network models; the first segmentation module comprises: the first training unit is used for performing data segmentation on the image data according to a first preset rule and then respectively performing training to obtain a plurality of sub-network models; an input unit configured to input the image data to the plurality of sub-network models, respectively, to obtain a segmentation result of the plurality of sub-network models; and the first segmentation unit is used for carrying out average value processing on the segmentation results of the plurality of sub-network models to obtain the semantic segmentation result.
As a preferred implementation of the embodiment of the present application, the second segmentation module includes: an obtaining unit, configured to obtain segmentation label information of the semantic segmentation result; and the cascading unit is used for cascading the segmentation label information in the convolution layer of the preset instance segmentation model to obtain the instance segmentation result.
As a preferred implementation manner of the embodiment of the present application, the second segmentation module further includes: the segmentation unit is used for carrying out data segmentation on the marked image data according to a second preset rule so as to obtain a plurality of groups of training data images; the second training unit is used for respectively training the multiple groups of training data images to obtain the preset example segmentation model; and the second segmentation unit is used for performing data segmentation on the image data which is not marked according to a second preset rule and inputting the image data into the preset example segmentation model so as to obtain the example segmentation result.
As a preferred implementation of the embodiment of the present application, the apparatus further includes: the measurement module is used for measuring the image of the mediastinum organ in the image segmentation result so as to obtain a measurement result of the image of the mediastinum organ; and the comparison module is used for comparing the measurement result of the image of the mediastinal organ with a preset threshold value of the image of the mediastinal organ so as to obtain the detection result of the image of the mediastinal organ.
For the concrete connection relationship and the functions of the modules and the units, please refer to the detailed description of the method, which is not repeated herein.
According to an embodiment of the present invention, there is also provided a computer apparatus including: one or more processors; storage means for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as previously described.
There is also provided, in accordance with an embodiment of the present invention, a computer-readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the steps of the method as previously described.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. An image processing method for mediastinal organ segmentation, comprising:
inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion;
performing example segmentation on image data of tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result to obtain an example segmentation result;
and outputting an image segmentation result of the image data of the tomography of the mediastinum organ according to the semantic segmentation result and the example segmentation result.
2. The image processing method for mediastinal organ segmentation according to claim 1, wherein the preset semantic segmentation model comprises a first semantic segmentation model, and before the image data of the tomography of the mediastinal organ is input into the preset semantic segmentation model for semantic segmentation and a semantic segmentation result is obtained through fusion, the method comprises:
preprocessing the marked image data to obtain a plurality of slice image data;
grouping a plurality of image data of the slices to obtain a plurality of groups of training data images;
and respectively training the multiple groups of training data images to obtain the first semantic segmentation model.
3. The image processing method for mediastinal organ segmentation according to claim 1, wherein the preset semantic segmentation model includes a plurality of sub-network models; the method comprises the following steps of inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation, and obtaining a semantic segmentation result through fusion, wherein the semantic segmentation result comprises the following steps:
training the image data after data segmentation according to a first preset rule to obtain a plurality of sub-network models;
inputting the image data into a plurality of sub-network models respectively to obtain segmentation results of the plurality of sub-network models;
and carrying out average processing on the segmentation results of the plurality of sub-network models to obtain the semantic segmentation result.
4. The image processing method for mediastinal organ segmentation according to claim 1, wherein performing instance segmentation on image data of tomography of the mediastinal organ through a preset instance segmentation model based on the semantic segmentation result to obtain an instance segmentation result comprises:
acquiring segmentation label information of the semantic segmentation result;
and concatenating the segmentation label information in the convolution layer of the preset instance segmentation model to obtain the instance segmentation result.
5. The image processing method for mediastinal organ segmentation according to claim 1, wherein performing instance segmentation on image data of tomography of the mediastinal organ through a preset instance segmentation model based on the semantic segmentation result to obtain an instance segmentation result comprises:
performing data segmentation on the marked image data according to a second preset rule to obtain a plurality of groups of training data images;
respectively training the multiple groups of training data images to obtain the preset example segmentation model;
and performing data segmentation on the image data which is not marked according to a second preset rule and inputting the image data into the preset example segmentation model to obtain the example segmentation result.
6. The image processing method for mediastinal organ segmentation according to claim 1, wherein outputting an image segmentation result of image data of the tomographic scan of the mediastinal organ according to the semantic segmentation result and the instance segmentation result comprises:
measuring a mediastinum organ image in the image segmentation result to obtain a measurement result of the mediastinum organ image;
and comparing the measurement result of the image of the mediastinum organ with a preset threshold value of the image of the mediastinum organ to obtain a detection result of the image of the mediastinum organ.
7. An image processing apparatus for mediastinal organ segmentation, comprising:
the first segmentation module is used for inputting image data of tomography of the mediastinum organ into a preset semantic segmentation model for semantic segmentation and obtaining a semantic segmentation result through fusion;
the second segmentation module is used for carrying out example segmentation on the image data of the tomography of the mediastinum organ through a preset example segmentation model based on the semantic segmentation result so as to obtain an example segmentation result;
and the output module is used for outputting an image segmentation result of the image data of the tomography of the mediastinum organ according to the semantic segmentation result and the example segmentation result.
8. An image processing apparatus for mediastinal organ segmentation according to claim 7, wherein the preset semantic segmentation model includes a first semantic segmentation model, the apparatus further comprising:
the segmentation module is used for preprocessing the marked image data to obtain a plurality of slice image data;
the grouping module is used for grouping the plurality of image data of the slices to obtain a plurality of groups of training data images;
and the training module is used for respectively training the multiple groups of training data images to obtain the first semantic segmentation model.
9. A computer device, comprising:
one or more processors;
storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.
10. A computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, carry out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911111737.XA CN112802028A (en) | 2019-11-13 | 2019-11-13 | Image processing method and device for mediastinal organ segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911111737.XA CN112802028A (en) | 2019-11-13 | 2019-11-13 | Image processing method and device for mediastinal organ segmentation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112802028A true CN112802028A (en) | 2021-05-14 |
Family
ID=75803577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911111737.XA Pending CN112802028A (en) | 2019-11-13 | 2019-11-13 | Image processing method and device for mediastinal organ segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112802028A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115187582A (en) * | 2022-08-17 | 2022-10-14 | 北京医准智能科技有限公司 | Lymph node segmentation method and device, electronic equipment and readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180315188A1 (en) * | 2017-04-21 | 2018-11-01 | General Electric Company | Automated organ risk segmentation machine learning methods and systems |
| CN109544564A (en) * | 2018-11-23 | 2019-03-29 | 清华大学深圳研究生院 | A kind of medical image segmentation method |
| CN109658419A (en) * | 2018-11-15 | 2019-04-19 | 浙江大学 | The dividing method of organella in a kind of medical image |
| CN109801307A (en) * | 2018-12-17 | 2019-05-24 | 中国科学院深圳先进技术研究院 | A kind of panorama dividing method, device and equipment |
| US20190251694A1 (en) * | 2018-02-14 | 2019-08-15 | Elekta, Inc. | Atlas-based segmentation using deep-learning |
| US10467500B1 (en) * | 2018-12-31 | 2019-11-05 | Didi Research America, Llc | Method and system for semantic segmentation involving multi-task convolutional neural network |
-
2019
- 2019-11-13 CN CN201911111737.XA patent/CN112802028A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180315188A1 (en) * | 2017-04-21 | 2018-11-01 | General Electric Company | Automated organ risk segmentation machine learning methods and systems |
| US20190251694A1 (en) * | 2018-02-14 | 2019-08-15 | Elekta, Inc. | Atlas-based segmentation using deep-learning |
| CN109658419A (en) * | 2018-11-15 | 2019-04-19 | 浙江大学 | The dividing method of organella in a kind of medical image |
| CN109544564A (en) * | 2018-11-23 | 2019-03-29 | 清华大学深圳研究生院 | A kind of medical image segmentation method |
| CN109801307A (en) * | 2018-12-17 | 2019-05-24 | 中国科学院深圳先进技术研究院 | A kind of panorama dividing method, device and equipment |
| US10467500B1 (en) * | 2018-12-31 | 2019-11-05 | Didi Research America, Llc | Method and system for semantic segmentation involving multi-task convolutional neural network |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115187582A (en) * | 2022-08-17 | 2022-10-14 | 北京医准智能科技有限公司 | Lymph node segmentation method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10970842B2 (en) | Method and device for identifying pathological picture | |
| CN110232383A (en) | A kind of lesion image recognition methods and lesion image identifying system based on deep learning model | |
| CN100528075C (en) | System and method for segmenting the left ventricle in a cardiac MR image | |
| CN110059697B (en) | Automatic lung nodule segmentation method based on deep learning | |
| CN110298844B (en) | X-ray radiography image blood vessel segmentation and identification method and device | |
| CN105096310B (en) | Divide the method and system of liver in magnetic resonance image using multi-channel feature | |
| CN110956635A (en) | Lung segment segmentation method, device, equipment and storage medium | |
| CN111161275A (en) | Method and device for segmenting target object in medical image and electronic equipment | |
| JP2008529641A (en) | Method for automatic extraction of pulmonary artery tree from 3D medical images | |
| CN107730507A (en) | A kind of lesion region automatic division method based on deep learning | |
| CN111488921A (en) | Panoramic digital pathological image intelligent analysis system and method | |
| CN111667459B (en) | Medical sign detection method, system, terminal and storage medium based on 3D variable convolution and time sequence feature fusion | |
| CN108052909B (en) | Thin fiber cap plaque automatic detection method and device based on cardiovascular OCT image | |
| CN112541893B (en) | Method for detecting tree-structure bifurcation key points in three-dimensional tomography image | |
| CN111080596A (en) | Auxiliary screening method and system for pneumoconiosis fusing local shadows and global features | |
| CN111340756A (en) | Medical image lesion detection and combination method, system, terminal and storage medium | |
| WO2023005634A1 (en) | Method and apparatus for diagnosing benign and malignant pulmonary nodules based on ct images | |
| Priya et al. | Adaptive fruitfly based modified region growing algorithm for cardiac fat segmentation using optimal neural network | |
| CN110738702B (en) | Three-dimensional ultrasonic image processing method, device, equipment and storage medium | |
| CN114155193B (en) | Blood vessel segmentation method and device based on feature enhancement | |
| Li et al. | Left ventricle segmentation by combining convolution neural network with active contour model and tensor voting in short-axis MRI | |
| CN112802028A (en) | Image processing method and device for mediastinal organ segmentation | |
| CN111461065B (en) | Tubular structure identification method, tubular structure identification device, computer equipment and readable storage medium | |
| CN112750110A (en) | Evaluation system for evaluating lung lesion based on neural network and related products | |
| CN115439453A (en) | Vertebral body positioning method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |