CN110956635B

CN110956635B - Lung segment segmentation method, device, equipment and storage medium

Info

Publication number: CN110956635B
Application number: CN201911119958.1A
Authority: CN
Inventors: 韩妙飞; 隗英; 高耀宗
Original assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Current assignee: Shanghai United Imaging Intelligent Healthcare Co Ltd
Priority date: 2019-11-15
Filing date: 2019-11-15
Publication date: 2023-12-01
Anticipated expiration: 2039-11-15
Also published as: CN110956635A

Abstract

The application discloses a lung segment segmentation method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring an image to be identified and a corresponding lung lobe segmentation result; performing lung segment rough segmentation on the image to be identified based on the lung segment rough segmentation model to obtain a lung region segmentation result; determining a first sub-image corresponding to the lung region segmentation result in the image to be identified; determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result; and taking the first sub-image and the second sub-image as input of a two-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the two-channel lung segment subdivision model to obtain a first lung segment subdivision result. The technical scheme provided by the application can be used for quickly carrying out rough positioning on the lung segments, so that the data acquisition speed is improved, the lung segment subdivision is only carried out on the lung region subdivision result obtained by rough subdivision, and the lung segment subdivision is assisted by the lung lobe subdivision result, so that the lung segment subdivision is more accurate and efficient.

Description

Lung segment segmentation method, device, equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for segmenting a lung segment.

Background

Medical research shows that the symptoms of lung cancer are closely related to the occurrence type of lung cancer (including the growth part of the lesion, the size of the lesion, the growth speed and the like), wherein the growth part of the lesion is a very important index, the bronchus of each lung segment and the lung tissues distributed by the branches are called as bronchopulmonary segments, and by dividing lung lobes and lung segments in CT images, accurate anatomical structures can be provided for the disease diagnosis of lung tumors and the lung segment excision surgery, and the method has very important significance for the disease diagnosis and the surgery treatment of lung cancer.

In the prior art, the key point positions of lung lobes are detected firstly, and based on the detected key point positions, the coordinates of lung lesions are mapped into a prefabricated lung three-dimensional model, so that specific lung lobes and lung segments of the lesions in the lung are determined. The method does not fully consider the lung structure difference among different patients, only depends on the detected key points to map the focus to a pre-manufactured model, can not visually display the result on an original image aiming at the patient, and is difficult to evaluate the quality of the focus position mapping result, so that a more reliable and efficient scheme needs to be provided.

Disclosure of Invention

In order to solve the problems in the prior art, the application provides a lung segment segmentation method, a device, equipment and a storage medium. The technical scheme is as follows:

in one aspect, the application provides a method for segmenting a lung segment, the method comprising:

acquiring an image to be identified and a corresponding lung lobe segmentation result;

performing lung segment rough segmentation on the image to be identified based on a lung segment rough segmentation model to obtain a lung region segmentation result;

determining a first sub-image corresponding to the lung region segmentation result in the image to be identified;

determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result;

and taking the first sub-image and the second sub-image as input of a two-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the two-channel lung segment subdivision model to obtain a first lung segment subdivision result.

Another aspect of the application provides another method of lung segment segmentation, the method comprising:

acquiring an image to be identified and a corresponding lung lobe segmentation result and bronchus segmentation result;

determining a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result;

and taking the first sub-image, the second sub-image and the third sub-image as input of a three-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model to obtain a second lung segment subdivision result.

In another aspect, the application provides a lung segment segmentation apparatus, the apparatus comprising:

the first image acquisition module is used for acquiring an image to be identified and a corresponding lung lobe segmentation result;

the first lung segment rough segmentation module is used for carrying out lung segment rough segmentation on the image to be identified based on the lung segment rough segmentation model to obtain a lung region segmentation result;

a first sub-image determining module, configured to determine a first sub-image corresponding to the lung region segmentation result in the image to be identified;

a second sub-image determining module, configured to determine a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result;

And the first lung segment subdivision module is used for taking the first sub-image and the second sub-image as the input of a dual-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the dual-channel lung segment subdivision model to obtain a first lung segment subdivision result.

the second image acquisition module is used for acquiring an image to be identified and a lung lobe segmentation result and a bronchus segmentation result which correspond to the image to be identified;

the second lung segment rough segmentation module is used for carrying out lung segment rough segmentation on the image to be identified based on the lung segment rough segmentation model to obtain a lung region segmentation result;

a third sub-image determining module, configured to determine a first sub-image corresponding to the lung region segmentation result in the image to be identified;

a fourth sub-image determining module, configured to determine a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result;

a fifth sub-image determining module, configured to determine a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result;

and the second lung segment subdivision module is used for taking the first sub-image, the second sub-image and the third sub-image as the input of a three-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model to obtain a second lung segment subdivision result.

In another aspect, the present application provides an apparatus, including a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the method of lung segment segmentation.

In another aspect, the present application provides another apparatus, including a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement the another lung segment segmentation method.

In another aspect, the present application provides a computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement the above-described lung segment segmentation method.

The lung segment segmentation method, the device, the equipment and the storage medium provided by the embodiment of the application have the following technical effects:

According to the embodiment of the application, the image to be identified and the corresponding lung lobe segmentation result are obtained, the lung segment rough segmentation is carried out on the image to be identified based on the lung segment rough segmentation model, so that the lung region segmentation result is obtained, the rough positioning of the lung segment can be rapidly carried out, the speed of data testing and obtaining is improved, and the follow-up lung segment fine segmentation is facilitated; and determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result by determining a first sub-image corresponding to the lung region segmentation result in the image to be identified, inputting the first sub-image and the second sub-image as a dual-channel lung segment segmentation model, and carrying out lung segment fine segmentation on the first sub-image based on the dual-channel lung segment segmentation model to obtain a first lung segment segmentation result, wherein the lung segment fine segmentation is only carried out on the lung region segmentation result obtained by rough segmentation, and lung segment segmentation is carried out by acquiring the lung lobe segmentation result in an auxiliary manner, so that the lung segment segmentation is more accurate and efficient. According to the other lung segment segmentation method, the lung segment rough segmentation is carried out on the image to be identified based on the lung segment rough segmentation model by acquiring the image to be identified and the corresponding lung lobe segmentation result and bronchus segmentation result, so that the lung region segmentation result is obtained, the rough positioning of the lung segment can be rapidly carried out, the data testing and acquiring speed is improved, and the follow-up lung segment fine segmentation is facilitated; determining a first sub-image corresponding to the lung region segmentation result in the image to be identified, determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result, determining a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result, and performing lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model by taking the first sub-image, the second sub-image and the third sub-image as inputs of a three-channel lung segment subdivision model to obtain a second lung segment segmentation result and taking the bronchus segmentation result as inputs of a lung segment subdivision model, so that a more accurate lung segment segmentation result can be obtained.

Additional aspects and advantages of embodiments of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the application, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for segmenting lung segments according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an image to be identified according to an embodiment of the present application;

FIG. 3 is a schematic diagram of left and right lobe segmentation results provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of left lobe segmentation results provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a right lobe segmentation result provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a segmentation result of left and right lung segments according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a left lung segment segmentation result provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a right lung segment segmentation result provided by an embodiment of the present application;

FIG. 9 is a flowchart of obtaining a lung segment annotation result provided by an embodiment of the present application;

FIG. 10 is a schematic view of a bronchus segmentation result provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of left and right pulmonary bronchus branch marker results provided by an embodiment of the present application;

FIG. 12 is a schematic illustration of left pulmonary bronchus branch identification results provided by an embodiment of the present application;

FIG. 13 is a schematic illustration of right pulmonary bronchus branch marker results provided by an embodiment of the present application;

FIG. 14 is a flowchart of a lung segment segmentation of the lung lobe segmentation result based on the bronchus branch identification result to obtain a lung segment annotation result provided by an embodiment of the present application;

FIG. 15 is a flow chart of another lung segment segmentation method provided by an embodiment of the present application;

FIG. 16 is a schematic view of a lung segment segmentation apparatus according to an embodiment of the present application;

FIG. 17 is a schematic view of another lung segment segmentation apparatus according to an embodiment of the present application;

fig. 18 is a hardware block diagram of a server of a lung segment segmentation method according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application. Examples of the embodiments are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements throughout or elements having like or similar functionality.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

In recent years, with research and progress of artificial intelligence technology, the artificial intelligence technology is widely applied in a plurality of fields, and the scheme provided by the embodiment of the application relates to the technology of machine learning/deep learning and the like of artificial intelligence, and is specifically described by the following embodiments:

fig. 1 is a flowchart of a lung segment segmentation method according to an embodiment of the present application, referring to fig. 1, the lung segment segmentation method according to the embodiment includes the following steps:

s101, acquiring an image to be identified and a corresponding lung lobe segmentation result.

Specifically, the lobe segmentation result may include any one of left and right lobe segmentation results, left lobe segmentation results, and right lobe segmentation results.

In practical application, the image to be identified can comprise a computer tomography image (CT image), and the computer tomography image has the characteristics of quick scanning time, clear image and the like, and can be used for checking various diseases; the image to be identified can also include, but is not limited to, a magnetic resonance image, an X-ray image, and a B-ultrasound image. Fig. 2 is a schematic diagram of an image to be identified according to an embodiment of the present application, and referring to fig. 2, in an embodiment of the present application, the image to be identified may include a chest CT image.

In particular, obtaining the lung lobe segmentation result may include:

carrying out lung lobe segmentation processing on the image to be identified based on a lung lobe segmentation model to obtain a lung lobe segmentation result;

specifically, the lung lobe segmentation model is obtained by performing lung lobe segmentation training based on a sample image and a corresponding lung lobe labeling result.

Specifically, the lobe labeling result may include any one of a left and right lobe labeling result, a left lobe labeling result, and a right lobe labeling result. When the lung lobe segmentation result to be obtained is a left lung lobe segmentation result and a right lung lobe segmentation result, the corresponding lung lobe labeling result is a left lung lobe labeling result and a right lung lobe labeling result; when the lung lobe segmentation result to be obtained is a left lung lobe segmentation result, the corresponding lung lobe labeling result is a left lung lobe labeling result; when the lung lobe segmentation result to be obtained is the right lung lobe segmentation result, the corresponding lung lobe labeling result is the right lung lobe labeling result.

Specifically, the training method of the lung lobe segmentation model may include:

1) Acquiring a sample image to be identified and a corresponding lung lobe labeling result;

2) Performing lung lobe segmentation training on a fourth neural network model based on the sample image to be identified;

3) Calculating an error value between a lung lobe segmentation result output by the fourth neural network model and a lung lobe labeling result corresponding to the sample image to be identified based on a first loss function;

4) Judging whether the error value meets a first preset condition or not;

5) When the judgment result is negative, adjusting model parameters in the fourth neural network model, and repeating the steps of segmenting the lung lobes to the judgment;

6) And when the judgment result is yes, taking the current fourth neural network model as the lung lobe segmentation model.

Specifically, the first loss function may include, but is not limited to, a Dice loss function and a Focal loss function. In the embodiment of the application, the neural network used by the lung lobe segmentation model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

Fig. 3 is a schematic diagram of a left and right lobe segmentation result provided by an embodiment of the present application, fig. 4 is a schematic diagram of a left lobe segmentation result provided by an embodiment of the present application, and fig. 5 is a schematic diagram of a right lobe segmentation result provided by an embodiment of the present application; the graph shows left and right lung lobe segmentation results obtained after the lung lobe segmentation processing is performed on the image to be identified, specifically, the left lung is divided into an upper lobe and a lower lobe, the right lung is divided into an upper lobe, a middle lobe and a lower lobe, a clear lung lobe segmentation result can be obtained through lung lobe segmentation, and subsequent lung segment segmentation is assisted through lung lobe segmentation results.

And carrying out the lung lobe segmentation processing on the image to be identified based on the lung lobe segmentation model to obtain a lung lobe segmentation result, wherein the lung lobe segmentation result of the image to be identified can be conveniently and rapidly obtained through the lung lobe segmentation model, and the lung segment segmentation can be carried out with the aid of the lung lobe segmentation result, so that the method is more flexible and efficient.

S103, performing lung segment rough segmentation on the image to be identified based on a lung segment rough segmentation model to obtain a lung region segmentation result.

Specifically, the lung region segmentation result may include any one of a left and right lung region segmentation result, a left lung region segmentation result, and a right lung region segmentation result.

Specifically, the lung segment rough segmentation model is obtained by performing lung segment rough segmentation training based on a sample image and a corresponding lung region labeling result.

Specifically, the lung region labeling result may include any one of a left and right lung region labeling result, a left lung region labeling result, and a right lung region labeling result. When the lung region segmentation result to be obtained is a left and right lung region segmentation result, the corresponding lung region labeling result is a left and right lung region labeling result; when the lung region segmentation result to be obtained is a left lung region segmentation result, the corresponding lung region labeling result is a left lung region labeling result; when the lung region segmentation result to be obtained is a right lung region segmentation result, the corresponding lung region labeling result is a right lung region labeling result.

Specifically, the training method of the lung segment rough segmentation model can comprise the following steps:

1) Acquiring a sample image to be identified and a corresponding lung region labeling result;

2) Performing first preprocessing on the sample image to be identified;

in an embodiment of the present application, the performing a first preprocessing on the sample image to be identified may include:

(1) Resampling the sample image to be identified to an image of a first resolution.

In practical application, the first resolution is obtained by summarizing the results of the lung segment rough segmentation training test based on a large number of sample images to be identified with different resolutions, specifically, the first resolution can be [6mm,6mm ], namely, the resolutions of an X axis, a Y axis and a Z axis are all 6mm.

(2) And carrying out normalization processing on the sample image to be identified.

Specifically, the normalizing the sample image to be identified may include:

and selecting a lung window, and setting the window width and the window level to preset values.

In practical applications, the preset value can be integrated and determined by combining a large number of sample images to be identified and information of the corresponding lung region segmentation result. Specifically, the preset value may be-400, the window width is 1500, and the pixel value of the sample image to be identified may be normalized to between-1 and 1 by setting the window width and the window level to the preset values.

(3) Randomly acquiring a sub-image with a preset size from a sample image to be identified.

In practical application, the preset size can be combined with a large number of sub-images with different sizes in the sample image to be identified for test induction determination. Specifically, the preset size can be [96, 96, 96], and the sub-images with the preset size, namely the image blocks, are adopted for model training, so that less memory is occupied, the model training efficiency can be improved, and the model performance is better.

3) Training the lung segment rough segmentation of the first neural network model based on the preprocessed sample image to be identified;

4) Calculating an error value between a lung region segmentation result output by the first neural network model and a lung region labeling result corresponding to the sample image to be identified based on a second loss function;

5) Judging whether the error value meets a second preset condition or not;

6) When the judgment result is negative, adjusting model parameters in the first neural network model, and repeating the steps of roughly dividing the lung segment until judgment;

7) And when the judgment result is yes, taking the current first neural network model as the lung segment rough segmentation model.

In particular, the second loss function may include, but is not limited to, a Dice loss function and a Focal loss function.

In the embodiment of the application, the neural network used by the lung segment rough segmentation model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

In the embodiment of the present application, before the lung segment rough segmentation is performed on the image to be identified based on the lung segment rough segmentation model, the method further includes:

performing second preprocessing on the image to be identified, wherein the second preprocessing on the image to be identified comprises resampling the image to be identified into an image with a first resolution;

the performing the lung segment rough segmentation on the image to be identified based on the lung segment rough segmentation model may include performing the lung segment rough segmentation on the preprocessed image to be identified based on the lung segment rough segmentation model.

In practical applications, the first resolution is obtained by summarizing the results of the lung segment rough segmentation training test based on a large number of sample images to be identified with different resolutions, and in general, the smaller the resolution value is, the higher the resolution of the image is, specifically, the first resolution value can be [6mm,6mm ], that is, the resolution values of the X-axis, Y-axis and Z-axis are all 6mm. Because the lung segment rough segmentation only needs to obtain a lung region segmentation result, the image to be identified is resampled to be of lower resolution, the image is smaller, only forward network calculation is needed, the operation speed is higher, the lung segment rough segmentation can be efficiently and quickly realized, the lung region segmentation result is obtained, and the occupied memory is less.

The lung region segmentation result of the image to be identified can be conveniently and rapidly obtained through the lung segment rough segmentation model, and the image to be identified is resampled into the image with the first resolution, so that the lung segment rough segmentation is only needed to be carried out on the image with the lower resolution, the occupied memory is less, and the segmentation processing speed and efficiency are improved.

S105, determining a first sub-image corresponding to the lung region segmentation result in the image to be identified.

Specifically, the determining the first sub-image corresponding to the lung region segmentation result in the image to be identified may include:

1) And determining a boundary box corresponding to the lung region segmentation result based on the lung region segmentation result.

In an embodiment of the present application, the bounding box may specifically be a cuboid, and determining, based on the lung region segmentation result, a bounding box corresponding to the lung region segmentation result may include: determining the length, width and height information of the boundary frame based on the coordinate range information of each pixel point in the lung region segmentation result; and determining world coordinates of a central point of the bounding box based on the length, width and height information of the bounding box, and obtaining the bounding box corresponding to the lung region segmentation result.

2) And amplifying the bounding box corresponding to the lung region segmentation result.

Specifically, the amplifying the bounding box corresponding to the lung region segmentation result may include:

amplifying at least one of the length, width and height of the bounding box by a preset size;

or;

amplifying the whole boundary frame in a preset proportion based on world coordinates of a center point of the boundary frame;

specifically, the preset size and the preset proportion can be set according to actual application requirements. In practical applications, the enlarging of the at least one of the length, the width, and the height of the bounding box by a preset size may include, for example, enlarging only the length of the bounding box; or; the length and width of the bounding box are respectively enlarged, wherein different preset dimensions can be taken when the length and width of the bounding box are respectively enlarged.

3) And taking the bounding box after the enlargement processing as a cutting frame.

4) And determining a first sub-image corresponding to the lung region segmentation result in the image to be identified based on the attribute information of the cutting frame and the image to be identified.

Specifically, the attribute information of the crop box may include world coordinates of a center point of the crop box and information of length, width and height.

By determining the first sub-image corresponding to the lung region segmentation result in the image to be identified, namely determining the region of interest corresponding to the lung region segmentation result in the image to be identified, the subsequent lung segment segmentation is only required to be performed on the region of interest (left and right lung regions, left lung region or right lung region), and the lung segment segmentation efficiency is improved. The contour of the resampled lung region segmentation result is amplified by a preset proportion, so that the region of interest can be completely contained, and the accuracy of lung segment segmentation is improved.

And S107, determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result.

Specifically, if the lung region segmentation result at this time is a left and right lung region segmentation result, the lung lobe segmentation result at this time is a left and right lung lobe segmentation result; if the lung region segmentation result at the moment is a left lung region segmentation result, the lung lobe segmentation result at the moment is a left lung lobe segmentation result; if the lung region segmentation result at this time is the right lung region segmentation result, the lung lobe segmentation result at this time is the right lung lobe segmentation result.

Specifically, the specific process of S107 is similar to S105, but in this case, the second sub-image corresponding to the lung region segmentation result in the lung region segmentation result is determined based on the attribute information of the crop frame and the lung region segmentation result, and specific steps may be referred to the related description in step S105, and will not be repeated here.

By determining the second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result, namely determining the image to be identified and the region of interest corresponding to the lung region segmentation result in the lung lobe segmentation result, the subsequent lung segment segmentation is only required to be performed on the region of interest (left and right lung regions, left lung region or right lung region), and the lung segment segmentation efficiency is improved.

By amplifying the bounding box corresponding to the lung region segmentation result, the left and right lung regions or the right lung region can be ensured to be completely contained, the subsequent segmentation of the lung segments is facilitated, and the reliability and accuracy of the subsequently obtained lung segment segmentation result are improved.

And S109, carrying out lung segment subdivision on the first sub-image and the second sub-image based on a two-channel lung segment subdivision model to obtain a first lung segment subdivision result.

Specifically, the lung segment segmentation result may include any one of a first left and right lung segment segmentation result, a first left lung segment segmentation result, and a first right lung segment segmentation result.

Specifically, the two-channel lung segment subdivision model is obtained by performing lung segment subdivision training based on a sample image, a corresponding lung lobe segmentation result and a lung segment labeling result.

Specifically, when the first lung segment segmentation result to be acquired is a first left and right lung segment segmentation result, the corresponding lung lobe segmentation result in the two-channel lung segment subdivision model training is a left and right lung lobe segmentation result, and the lung segment marking result is a left and right lung segment marking result; when the first lung segment segmentation result to be acquired is a first left lung segment segmentation result, the corresponding lung lobe segmentation result in the two-channel lung segment subdivision model training is a left lung lobe segmentation result, and the lung segment marking result is a left lung segment marking result; when the first lung segment segmentation result required to be acquired is a first right lung segment segmentation result, the corresponding lung lobe segmentation result in the two-channel lung segment subdivision model training is a right lung lobe segmentation result, and the lung segment marking result is a right lung segment marking result.

Specifically, the method may include:

1) Acquiring a sample image to be identified and a corresponding lung lobe segmentation result and a lung segment labeling result;

2) Performing fourth preprocessing on the sample image to be identified and the corresponding lung lobe segmentation result;

in the embodiment of the present application, the fourth preprocessing of the sample image to be identified and the corresponding lung lobe segmentation result may further include:

(1) And resampling the sample image to be identified and the corresponding lung lobe segmentation result into an image with a second resolution.

In practical applications, the second resolution is determined by summarizing the results of the lung segment subdivision training test based on a plurality of sample images to be identified with different resolutions, and specifically, the second resolution may be 1mm, that is, the X-axis, Y-axis, and Z-axis resolution values are all 1mm.

(2) And carrying out normalization processing on the sample image to be identified and the corresponding lung lobe segmentation result.

Specifically, the normalizing the sample image to be identified and the corresponding lung lobe segmentation result may include:

selecting a lung window for the sample image to be identified, and setting the window width and the window level as a first preset value;

and setting the window width and the window level of the lung lobe segmentation result corresponding to the sample image to be identified as a second preset value.

In practical application, the first preset value and the second preset value can be combined with a large number of sample images to be identified and corresponding lung lobe segmentation result information to carry out test induction determination. Specifically, the first preset value may be a window level of-400, and a window width of 1500; when the lung lobe segmentation result is the left lung lobe segmentation result, the second preset value can take a window level of 1 and a window width of 2; when the lobe segmentation result is the right lobe segmentation result, the second preset value may take a window level of 1.5 and a window width of 3. The lung window is selected for the sample image to be identified and the lung lobe segmentation result corresponding to the sample image to be identified, and the window width and the window level are set to be a first preset value and a second preset value respectively, so that the pixel values of the image to be identified and the lung lobe segmentation result corresponding to the sample image to be identified can be normalized to be between-1 and-1 respectively.

(3) Randomly acquiring a sub-image with a preset size from a sample image to be identified and a corresponding lung lobe segmentation result.

In practical application, the preset size can be combined with a large number of sample images to be identified and sub-images with different sizes in the corresponding lung lobe segmentation result to carry out test induction determination. Specifically, the preset size can be [96, 96, 96], and the sub-images with the preset size, namely the image blocks, are adopted for model training, so that less memory is occupied, the model training efficiency can be improved, and the model performance is better.

3) Training the lung segment fine segmentation of the second neural network model based on the preprocessed sample image to be identified and the corresponding lung lobe segmentation result;

4) Calculating an error value between a lung segment segmentation result output by the second neural network model and a lung segment labeling result corresponding to the sample image to be identified based on a third loss function;

5) Judging whether the error value meets a third preset condition or not;

6) When the judgment result is negative, adjusting model parameters in the second neural network model, and repeating the step of finely dividing the lung segment until judgment;

7) And when the judgment result is yes, taking the current second neural network model as the two-channel lung segment subdivision model.

In the embodiment of the application, the neural network used by the two-channel lung segment subdivision model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

In an embodiment of the present application, before the lung segment subdivision is performed on the first sub-image and the second sub-image based on the dual-channel lung segment subdivision model, the method further includes:

preprocessing the first sub-image and the second sub-image.

Specifically, the preprocessing the first sub-image and the second sub-image includes resampling the first sub-image and the second sub-image to images of a second resolution;

wherein the second resolution is higher than the first resolution.

By resampling the first sub-image and the second sub-image to higher resolution images, the accuracy and reliability of the lung segment segmentation result can be improved, so that the lung segment segmentation is more accurate.

The lung segment subdivision of the first sub-image and the second sub-image based on the dual-channel lung segment subdivision model may include lung segment subdivision of the preprocessed first sub-image and second sub-image based on the dual-channel lung segment subdivision model.

In practical applications, the second resolution is determined by summarizing the results of the lung segment subdivision training test based on a plurality of sample images to be identified with different resolutions, and the second resolution value may be 1mm, that is, the X-axis, Y-axis, and Z-axis resolutions are all 1mm.

Fig. 6 is a schematic diagram of a left and right lung segment segmentation result provided by the embodiment of the present application, fig. 7 is a schematic diagram of a left lung segment segmentation result provided by the embodiment of the present application, and fig. 8 is a schematic diagram of a right lung segment segmentation result provided by the embodiment of the present application, as shown in the above-mentioned legend, in which the left lung is divided into 8 lung segments and the right lung is divided into 10 lung segments.

By resampling the first sub-image and the second sub-image into images with higher resolution, the accuracy and reliability of the lung segment segmentation result can be improved, and the lung segment segmentation is more accurate; by determining a first sub-image corresponding to the lung region segmentation result in the image to be identified and a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation image, the lung segment subdivision is only required to be performed on the region of interest (left and right lung regions, left lung region or right lung region), so that the memory consumption can be reduced, and the accuracy of the lung segment subdivision is improved by using the second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation image to assist in the lung segment subdivision, so that the lung segment subdivision is more rapid and efficient, and the left and right lung segment subdivision can be performed as required; or; only left lung segment segmentation or right lung segment segmentation is carried out, so that the method is more flexible, convenient and quick.

As shown in fig. 9, in the embodiment of the present application, the obtaining the lung segment annotation result includes:

s901, an image with a resolution higher than a preset threshold is acquired.

In practical applications, the preset threshold is determined by conducting bronchial segmentation test based on a large number of sample images with different resolutions, and generally, the lower the Z-axis resolution value is, the thinner the image is, the higher the image resolution is, and the preset threshold can be used for obtaining a CT image with the Z-axis resolution value of 1mm, that is, the Z-axis resolution value is less than 1 mm.

S903, performing lung lobe segmentation processing on the image with the resolution higher than a preset threshold value to obtain a lung lobe segmentation result.

Specifically, the lobe segmentation result may include any one of left and right lobe segmentation results, left lobe segmentation result, and right lobe segmentation result.

Specifically, the step of obtaining the lung lobe segmentation result is similar to the step of obtaining the lung lobe segmentation result in S101, but the step of obtaining the lung lobe segmentation result in S101 may refer to the relevant description of obtaining the lung lobe segmentation result in S101.

Clear lung lobe segmentation results can be obtained through lung lobe segmentation, and subsequent lung segment segmentation is assisted through the lung lobe segmentation results; the lung lobe segmentation result of the image with the resolution higher than the preset threshold can be conveniently and rapidly obtained through the lung lobe segmentation model, and lung segment segmentation is carried out by using the lung lobe segmentation result in an auxiliary mode, so that the lung segment segmentation is more efficient.

S905, performing bronchus branch identification on the image with the resolution higher than a preset threshold value to obtain a bronchus branch identification result.

Specifically, the bronchus branch identification result may include any one of a left and right lung bronchus branch identification result, a left lung bronchus branch identification result, and a right lung bronchus branch identification result.

Specifically, the performing the bronchial branch identification on the image with the resolution higher than the preset threshold value, and obtaining the bronchial branch identification result includes:

s9051, performing bronchus segmentation on the image with the resolution higher than a preset threshold value based on a bronchus segmentation model to obtain a bronchus segmentation result.

Specifically, the bronchus segmentation result may include any one of left and right lung bronchus segmentation results, left lung bronchus segmentation results, and right lung bronchus segmentation results.

Specifically, the bronchus segmentation model is obtained by carrying out bronchus segmentation training based on a sample image with resolution higher than a preset threshold and a corresponding bronchus labeling result.

Specifically, the bronchus labeling result may include any one of a left and right lung bronchus labeling result, a left lung bronchus labeling result, and a right lung bronchus labeling result.

Specifically, when the bronchus segmentation result to be obtained is a left and right lung bronchus segmentation result, the corresponding bronchus labeling result is a left and right lung bronchus labeling result; when the bronchus segmentation result to be obtained is a left lung bronchus segmentation result, the corresponding bronchus labeling result is a left lung bronchus labeling result; when the bronchus segmentation result to be obtained is the right lung bronchus segmentation result, the corresponding bronchus labeling result is the right lung bronchus labeling result.

Specifically, the method may include:

1) Acquiring a sample image with resolution higher than a preset threshold to be identified and a corresponding bronchus labeling result;

2) Training the bronchus segmentation of a fifth neural network model based on the sample image with the resolution to be identified higher than a preset threshold;

3) Calculating an error value between a bronchus segmentation result output by the fifth neural network model and a bronchus labeling result corresponding to the sample image with the resolution to be identified higher than a preset threshold value based on a fourth loss function;

4) Judging whether the error value meets a fourth preset condition;

5) When the judgment result is negative, adjusting model parameters in the fifth neural network model, and repeating the steps of segmenting the bronchus to the judgment;

6) And when the judgment result is yes, taking the current fifth neural network model as the bronchus segmentation model.

In particular, the fourth loss function may include, but is not limited to, a Dice loss function and a Focal loss function.

In the embodiment of the application, the neural network used by the bronchus segmentation model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

Fig. 10 is a schematic diagram of a bronchus segmentation result according to an embodiment of the present application, as shown in fig. 10, in which a white portion, i.e., a bronchus structure, and a black portion, are background areas.

S9053, performing bronchus branch identification on the bronchus segmentation result based on a bronchus branch identification model to obtain a bronchus branch identification result.

Specifically, the bronchus branch identification model is obtained by training bronchus branch identification based on a bronchus segmentation sample image and a corresponding bronchus branch identification labeling result.

Specifically, the bronchus branch identification labeling result may include any one of a left and right lung bronchus branch identification labeling result, a left lung bronchus branch identification labeling result and a right lung bronchus branch identification labeling result.

Specifically, when the branch identification result of the bronchus to be acquired is the branch identification result of the left and right lung bronchus, the corresponding branch identification marking result of the bronchus is the branch identification marking result of the left and right lung bronchus; when the branch identification result of the bronchus to be acquired is the branch identification result of the left lung bronchus, the corresponding branch identification marking result of the bronchus is the branch identification marking result of the left lung bronchus; when the branch identification result of the bronchus to be acquired is the branch identification result of the right lung bronchus, the corresponding branch identification label result of the bronchus is the branch identification label result of the right lung bronchus.

Specifically, the method may include:

1) Acquiring a bronchus segmentation sample image and a corresponding bronchus branch identification marking result;

2) Training a bronchus branch identifier of a sixth neural network model based on the bronchus segmentation sample image;

3) Calculating an error value between a bronchus branch identification result output by the sixth neural network model and a bronchus branch identification labeling result corresponding to the bronchus segmentation sample image based on a fifth loss function;

4) Judging whether the error value meets a fifth preset condition or not;

5) When the judging result is no, adjusting model parameters in the sixth neural network model, and repeating the step of judging the bronchus branch mark;

6) And when the judgment result is yes, taking the current sixth neural network model as the bronchus branch identification model.

In particular, the fifth loss function may include, but is not limited to, a Dice loss function and a Focal loss function.

In the embodiment of the application, the neural network used by the bronchus branch identification model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

Fig. 11 is a schematic diagram of a left and right bronchus branch identifier result provided by an embodiment of the present application, fig. 12 is a schematic diagram of a left bronchus branch identifier result provided by an embodiment of the present application, and fig. 13 is a schematic diagram of a right bronchus branch identifier result provided by an embodiment of the present application; as shown in fig. 11, 12 and 13, the grey parts of the figures are identified bronchial branches.

Because the continuity of the bronchus branches in the low-resolution thick-layer CT image is poor, the method and the device can ensure that the obtained bronchus branch result is more accurate and reliable by obtaining the sample image with the resolution higher than the preset threshold, namely performing bronchus division on the high-resolution thin-layer CT image so as to further perform bronchus branch identification, and are beneficial to performing lung segment division based on the bronchus branch result; the bronchus segmentation result and the bronchus branch identification result can be obtained rapidly and efficiently by training the bronchus segmentation model and the bronchus branch identification model, the universality is strong, the bronchus branch identification can be confirmed according to the bronchus walking shape of each patient, and the method is suitable for different individual differences and is accurate and efficient.

S907, performing lung segment segmentation on the lung lobe segmentation result based on the bronchus branch identification result to obtain a lung segment labeling result.

Specifically, the lung segment annotation result may include any one of a left and right lung segment annotation result, a left lung segment annotation result, and a right lung segment annotation result. When the lung segment marking result to be obtained is a left and right lung segment marking result, the corresponding bronchus branch marking result is a left and right lung bronchus branch marking result, and the lung lobe segmentation result is a left and right lung lobe segmentation result; when the lung segment marking result to be obtained is a left lung segment marking result, the corresponding bronchus branch marking result is a left lung bronchus branch marking result, and the lung lobe segmentation result is a left lung lobe segmentation result; when the lung segment marking result to be obtained is the right lung segment marking result, the corresponding bronchus branch marking result is the right lung bronchus branch marking result, and the lung lobe segmentation result is the right lung lobe segmentation result.

Specifically, as shown in fig. 14, the performing the lung segment segmentation on the lung lobe segmentation result based on the bronchus branch identification result, to obtain a lung segment labeling result includes:

s1401, determining a distance from each pixel point in the lobe segmentation result to each bronchus branch in the corresponding lobe based on the bronchus branch identification result.

In the embodiment of the present disclosure, in the process of generating the bronchial branch identifier, in order to distinguish different bronchial branches, unique identifier information may be generated for each bronchial branch, so that the bronchial branches can be distinguished from each other. In practical applications, for example, the bronchi branches may be encoded sequentially, specifically, for example, for the left and right lung bronchi branches, the encoding may be performed sequentially with numbers 1 to 18, and the lung segment identifiers are in one-to-one correspondence with the bronchi branch identifiers.

Specifically, based on the obtained bronchus branch identification result, for a pixel point in the lobe segmentation result, the distance from the pixel point to each bronchus branch in the corresponding lobe can be obtained, for example, since the upper left lobe lung segment can only be 1, 2, 3 or 4 # lung segment, and for the upper left lobe pixel point, the distances from the pixel point to the 1, 2, 3 and 4 # bronchus branches are only required to be determined respectively.

S1403, determining the identification of the bronchus branch corresponding to the minimum distance based on the distance from each pixel point in the lobe segmentation result to each bronchus branch in the corresponding lobe.

Specifically, after determining the distance from each pixel point in the lung lobe segmentation result to each bronchus branch in the corresponding lung lobe, comparing the distance from each pixel point to each bronchus branch in the corresponding lung lobe, the identification of the bronchus branch corresponding to the minimum distance can be determined. Taking the above pixel point of the upper left lung as an example, after determining the distances from the pixel point to the 1, 2, 3 and 4 # bronchial branches, comparing the distances from the pixel point to the 1, 2, 3 and 4 # bronchial branches, for example, if the distance from the pixel point to the 1 # bronchial branch is the smallest, the identification of the bronchial branch corresponding to the pixel point is 1.

S1405, determining the lung segment identification corresponding to each pixel point based on the identification of the bronchus branch corresponding to the minimum distance.

Specifically, after the identification of the bronchus branch corresponding to the minimum distance is determined, the lung segment corresponding to each pixel point can be obtained according to the identification of the bronchus branch corresponding to the minimum distance because the lung segment identification corresponds to the bronchus branch identification one to one. Taking the pixel point of the upper left lung as an example, the mark of the bronchus branch corresponding to the minimum distance from the pixel point to each bronchus branch in the corresponding lung lobe is 1, it may be determined that the lung segment corresponding to the pixel point is 1, that is, the lung segment No. 1, where in the embodiment of the present application, the lung segment No. 1 may specifically represent the upper left lung lobe/tip rear segment, and may be displayed as the upper left lung lobe/tip rear segment when output display is performed.

And S1407, obtaining a lung segment marking result based on the lung segment corresponding to each pixel point.

Specifically, traversing all pixel points, and integrating the lung segments corresponding to all pixel points to obtain a lung segment marking result.

When the lung segment marking result is obtained, as the continuity of the bronchus branch in the low-resolution thick-layer CT image is poor, the application further carries out bronchus branch identification by obtaining the sample image with the resolution higher than the preset threshold, namely the high-resolution thin-layer CT image, so that the obtained bronchus branch result can be ensured to be more accurate and reliable, and the lung segment segmentation is facilitated based on the bronchus branch result; the bronchus segmentation result and the bronchus branch identification result can be quickly and efficiently obtained by training the bronchus segmentation model and the bronchus branch identification model, the universality is strong, the bronchus branch identification can be confirmed according to the bronchus shape of each patient, the difference of different individuals is adapted, the accuracy and the efficiency are improved, the lung segment segmentation can be carried out on the left lung and the right lung (the whole lung) or the lung segment segmentation can be carried out on the left lung or the right lung independently according to the needs, and the left lung segment marking result and the right lung segment marking result can be obtained; or; only the left lung segment marking result or the right lung segment marking result is obtained, so that the method is more flexible.

As shown in fig. 15, in other embodiments, the lung segment segmentation method may include:

s1501, an image to be identified, a corresponding lung lobe segmentation result and a corresponding bronchus segmentation result are obtained.

Specifically, the specific step of acquiring the image to be identified and the corresponding lung lobe segmentation result is similar to the step of acquiring the image to be identified and the corresponding lung lobe segmentation result in S101, and the specific step may refer to the related description of acquiring the image to be identified and the corresponding lung lobe segmentation result in S101, which is not described herein.

In particular, obtaining the bronchial segmentation result may include:

performing bronchus segmentation processing on the image to be identified based on a bronchus segmentation model to obtain a bronchus segmentation result;

specifically, the bronchus segmentation model is obtained by carrying out bronchus segmentation training based on a sample image and a corresponding bronchus labeling result.

Specifically, the training method is similar to the method in S9051, and specific steps may be referred to the related description in step S9051, and will not be described herein.

S1503, performing lung segment rough segmentation on the image to be identified based on a lung segment rough segmentation model to obtain a lung region segmentation result.

S1505, determining a first sub-image corresponding to the lung region segmentation result in the image to be identified.

S1507, determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result.

S1509, determining a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result.

Specifically, the specific step of determining the third sub-image corresponding to the lung region segmentation result in S1509 is similar to the step of determining the second sub-image corresponding to the lung region segmentation result in S107 in the lobe segmentation result, and the bronchus segmentation result may include any one of a left and right lung bronchus segmentation result, a left lung bronchus segmentation result, and a right lung bronchus segmentation result. Specifically, if the lung region segmentation result at this time is a left and right lung region segmentation result, the bronchus segmentation result at this time is a left and right lung bronchus segmentation result; if the lung region segmentation result at the moment is a left lung region segmentation result, the bronchus segmentation result at the moment is a left lung bronchus segmentation result; if the lung region segmentation result at this time is the right lung region segmentation result, the bronchus segmentation result at this time is the right lung bronchus segmentation result. Specific steps may be referred to the relevant description in S107, and will not be described herein.

Specifically, the specific procedures of S1503 to S1507 are similar to those of S103 to S107, and the specific procedures can be referred to the relevant descriptions in S103 to S107, and are not repeated here.

S1511, taking the first sub-image, the second sub-image and the third sub-image as input of the three-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model to obtain a second lung segment subdivision result.

Specifically, the three-channel lung segment subdivision model is obtained by performing lung segment subdivision training based on a sample image and a corresponding lung lobe segmentation result, bronchus segmentation result and lung segment labeling result.

Specifically, the method may include:

1) Acquiring a sample image to be identified, a corresponding lung lobe segmentation result, a bronchus segmentation result and a corresponding lung segment labeling result;

2) Preprocessing the sample image to be identified and the corresponding lung lobe segmentation result;

specifically, the specific steps of preprocessing the sample image to be identified and the corresponding lung lobe segmentation result are similar to the steps of S109, and the specific steps may be referred to the related description in S109, which is not repeated herein.

3) Training the lung segment fine segmentation of the third neural network model based on the preprocessed sample image to be identified and the corresponding lung lobe segmentation result and bronchus segmentation result;

4) Calculating an error value between a second lung segment segmentation result output by the third neural network model and a lung segment labeling result corresponding to the sample image to be identified based on a sixth loss function;

5) Judging whether the error value meets a sixth preset condition;

6) When the judging result is no, adjusting the model parameters in the third neural network model, and repeating the step of finely dividing the lung segment until the judgment;

7) And when the judgment result is yes, taking the current third neural network model as the three-channel lung segment subdivision model.

Specifically, the sixth loss function may include, but is not limited to, a Dice loss function and a Focal loss function.

In the embodiment of the application, the neural network used by the three-channel lung segment subdivision model can comprise, but is not limited to, a two-dimensional convolutional neural network, a V-Net and a U-Net in a three-dimensional convolutional neural network.

Determining a first sub-image corresponding to the lung region segmentation result in the image to be identified; determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result; determining a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result; and taking the first sub-image, the second sub-image and the third sub-image as inputs of a three-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model to obtain a second lung segment subdivision result, wherein the lung segment subdivision can be assisted by utilizing the second sub-image corresponding to the lung region subdivision result in the lung lobe subdivision image and the third sub-image corresponding to the lung region subdivision result in the bronchus subdivision image, so that the obtained lung segment subdivision result has higher accuracy and higher reliability.

The embodiment of the application also provides a lung segment segmentation device, as shown in fig. 16, which comprises:

a first image acquisition module 1610, configured to acquire an image to be identified and a corresponding lung lobe segmentation result;

the first lung segment rough segmentation module 1620 is configured to perform lung segment rough segmentation on the image to be identified based on a lung segment rough segmentation model, so as to obtain a lung region segmentation result;

a first sub-image determining module 1630, configured to determine a first sub-image corresponding to the lung region segmentation result in the image to be identified;

a second sub-image determining module 1640, configured to determine a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result;

and the first lung segment subdivision module 1650 is configured to use the first sub-image and the second sub-image as input of a dual-channel lung segment subdivision model, and perform lung segment subdivision on the first sub-image based on the dual-channel lung segment subdivision model, so as to obtain a first lung segment subdivision result.

Another aspect of the application provides another pulmonary segment segmentation apparatus, as shown in fig. 17, the apparatus comprising:

a second image obtaining module 1710, configured to obtain an image to be identified and a lung lobe segmentation result and a bronchus segmentation result corresponding to the image;

A second lung segment rough segmentation module 1720, configured to perform lung segment rough segmentation on the image to be identified based on the lung segment rough segmentation model, to obtain a lung region segmentation result;

a third sub-image determining module 1730, configured to determine a first sub-image corresponding to the lung region segmentation result in the image to be identified;

a fourth sub-image determining module 1740 for determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result;

a fifth sub-image determining module 1750 configured to determine a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result;

and a second lung segment subdivision module 1760, configured to use the first sub-image, the second sub-image, and the third sub-image as input of a three-channel lung segment subdivision model, and perform lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model, so as to obtain a second lung segment subdivision result.

In some embodiments, the lung segment segmentation apparatus further comprises:

the first sample image acquisition module is used for acquiring a sample image and a corresponding lung region labeling result;

the first training module is used for training the lung segment rough segmentation of the first neural network model based on the sample image, and adjusting model parameters of the first neural network model in the training of the lung segment rough segmentation until a lung region segmentation result output by the first neural network model is matched with a lung region labeling result corresponding to the sample image;

And the lung segment rough segmentation model determining module is used for taking the current first neural network model as the lung segment rough segmentation model when the lung region segmentation result output by the first neural network model is matched with the lung region labeling result corresponding to the sample image.

In some embodiments, the lung segment segmentation apparatus further comprises:

the second sample image acquisition module is used for acquiring a sample image, a corresponding lung lobe segmentation result and a corresponding lung segment marking result;

the second training module is used for training the lung segment fine segmentation of the second neural network model based on the sample image and the corresponding lung lobe segmentation result, and adjusting model parameters of the second neural network model in the training of the lung segment fine segmentation until the lung segment segmentation result output by the second neural network model is matched with the lung segment labeling result corresponding to the sample image;

and the two-channel lung segment subdivision model determining module is used for taking the current second neural network model as the two-channel lung segment subdivision model when the first lung segment subdivision result output by the second neural network model is matched with the lung segment marking result corresponding to the sample image.

In some embodiments, the lung segment segmentation apparatus further comprises:

The third sample image acquisition module is used for acquiring a sample image to be identified, a corresponding lung lobe segmentation result, a bronchus segmentation result and a corresponding lung segment marking result;

the third training module is used for training the lung segment fine segmentation of the third neural network model based on the sample image, the lung lobe segmentation result corresponding to the sample image and the bronchus segmentation result corresponding to the sample image, and adjusting model parameters of the third neural network model in the training of the lung segment fine segmentation until the lung segment segmentation result output by the third neural network model is matched with the lung segment labeling result corresponding to the sample image;

and the three-channel lung segment subdivision model determining module is used for taking the current third neural network model as the three-channel lung segment subdivision model when the second lung segment subdivision result output by the third neural network model is matched with the lung segment marking result corresponding to the sample image.

The device and method embodiments in the device embodiments described are based on the same application concept.

An embodiment of the present application provides a lung segment segmentation apparatus, which includes a processor and a memory, where at least one instruction, at least one program, a code set, or an instruction set is stored in the memory, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a lung segment segmentation method as provided in the above method embodiment.

An embodiment of the present application provides another lung segment segmentation apparatus, including a processor and a memory, where the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by the processor to implement another lung segment segmentation method as provided by the method embodiment described above.

The memory may be used to store software programs and modules that the processor executes to perform various functional applications and data processing by executing the software programs and modules stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for functions, and the like; the storage data area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide access to the memory by the processor.

The method embodiments provided by the embodiments of the present application may be performed in a mobile terminal, a computer terminal, a server, or similar computing device. Taking the operation on the server as an example, fig. 18 is a block diagram of the hardware structure of the server of the lung segment segmentation method according to the embodiment of the present application. As shown in fig. 18, the server 1800 may vary considerably in configuration or performance and may include one or more central processing units (Central Processing Units, CPU) 1810 (the processor 1810 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA), memory 1830 for storing data, one or more storage mediums 1820 (e.g., one or more mass storage devices) for storing applications 1823 or data 1822. Wherein the memory 1830 and storage medium 1820 may be transitory or persistent. The program stored on the storage medium 1820 may include one or more modules, each of which may include a series of instruction operations in a server. Further, the central processor 1810 may be configured to communicate with a storage medium 1820 to execute a series of instruction operations on the storage medium 1820 on the server 1800. The server 1800 may also include one or more power supplies 1860, one or more wired or wireless network interfaces 1850, one or more input/output interfaces 1840, and/or one or more operating systems 1821, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.

The input-output interface 1840 may be used to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the server 1800. In one example, the input/output interface 1840 includes a network adapter (Network Interface Controller, NIC) that may be connected to other network devices through a base station to communicate with the internet. In one example, the input/output interface 1840 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.

It will be appreciated by those skilled in the art that the configuration shown in fig. 18 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, server 1800 may also include more or fewer components than shown in fig. 18, or have a different configuration than shown in fig. 18.

Embodiments of the present application also provide a storage medium that may be disposed in a server to store at least one instruction, at least one program, a code set, or a set of instructions related to implementing a lung segment segmentation method in a method embodiment, where the at least one instruction, the at least one program, the code set, or the set of instructions are loaded and executed by the processor to implement the lung segment segmentation method provided in the method embodiment.

Alternatively, in this embodiment, the storage medium may be located in at least one network server among a plurality of network servers of the computer network. Alternatively, in the present embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

As can be seen from the embodiments of the method, the device, the server or the storage medium for segmenting lung segments provided by the present application, the method for segmenting lung segments in the present application can obtain an image to be identified and a corresponding lung lobe segmentation result; performing lung segment rough segmentation on the image to be identified based on a lung segment rough segmentation model to obtain a lung region segmentation result; the lung segment segmentation can be assisted by the lung lobe segmentation result, the lung segment segmentation method is more flexible and efficient, the lung region segmentation result of the image to be identified can be conveniently and rapidly obtained through the lung segment rough segmentation model, and the image to be identified is resampled into the image with the first resolution, so that the lung segment rough segmentation is only needed to be carried out on the image with the lower resolution, the occupied video memory is less, and the segmentation is rapid and efficient. Determining a first sub-image corresponding to the lung region segmentation result in the image to be identified; determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result; carrying out lung segment fine segmentation on the first sub-image and the second sub-image based on a double-channel lung segment fine segmentation model to obtain a first lung segment segmentation result; by resampling the first sub-image and the second sub-image into images with higher resolution, the accuracy and reliability of the lung segment segmentation result can be improved, and the lung segment segmentation is more accurate; the lung segment subdivision is only needed to be carried out on the region of interest (left and right lung regions, left lung region or right lung region) by determining the first sub-image corresponding to the lung region subdivision result in the image to be identified and the second sub-image corresponding to the lung region subdivision result in the lung lobe subdivision image, and the lung segment subdivision is assisted by utilizing the second sub-image corresponding to the lung region subdivision result in the lung lobe subdivision image, so that the lung segment subdivision efficiency is improved, and the lung segment subdivision is more rapid and efficient; and the lung segment segmentation can be carried out on the left lung and the right lung (the whole lung) according to the needs or the left lung or the right lung is independently carried out, so that the lung segment segmentation is more flexible. Another lung segment segmentation method comprises the steps of determining a first sub-image corresponding to a lung region segmentation result in the image to be identified; determining a second sub-image corresponding to the lung region segmentation result in the lung lobe segmentation result; determining a third sub-image corresponding to the lung region segmentation result in the bronchus segmentation result; and taking the first sub-image, the second sub-image and the third sub-image as inputs of a three-channel lung segment subdivision model, carrying out lung segment subdivision on the first sub-image based on the three-channel lung segment subdivision model to obtain a second lung segment subdivision result, and assisting in lung segment subdivision by utilizing the second sub-image corresponding to the lung region subdivision result in the lung lobe subdivision image and the third sub-image corresponding to the lung region subdivision result in the bronchus subdivision image, so that the obtained lung segment subdivision result is higher in accuracy and higher in reliability.

When a lung segment marking result is obtained, an image with resolution higher than a preset threshold value is obtained; carrying out lung lobe segmentation processing on the image with the resolution higher than a preset threshold value to obtain a lung lobe segmentation result; performing bronchial branch identification on the image with the resolution higher than a preset threshold value to obtain a bronchial branch identification result; and performing lung segment segmentation on the lung lobe segmentation result based on the bronchus branch identification result to obtain a lung segment marking result. Because the continuity of the bronchus branches in the low-resolution thick-layer CT image is poor, the method and the device can ensure that the obtained bronchus branch result is more accurate and reliable by obtaining the sample image with the resolution higher than the preset threshold, namely performing bronchus division on the high-resolution thin-layer CT image so as to further perform bronchus branch identification, and are beneficial to performing lung segment division based on the bronchus branch result; the bronchus segmentation result and the bronchus branch identification result can be quickly and efficiently obtained by training the bronchus segmentation model and the bronchus branch identification model, the universality is strong, the bronchus branch identification can be confirmed according to the bronchus shape of each patient, the difference of different individuals is adapted, the accuracy and the efficiency are improved, the lung segment segmentation can be carried out on the left lung and the right lung (the whole lung) or the lung segment segmentation can be carried out on the left lung or the right lung independently according to the needs, and the left lung segment marking result and the right lung segment marking result can be obtained; or; only the left lung segment marking result or the right lung segment marking result is obtained, so that the method is more flexible.

It should be noted that: the sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this specification. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices and storage medium embodiments, the description is relatively simple as it is substantially similar to method embodiments, with reference to the description of method embodiments in part.

Those of ordinary skill in the art will appreciate that all or a portion of the steps implementing the above embodiments may be implemented by hardware, or may be implemented by a program indicating that the relevant hardware is implemented, where the program may be stored on a computer readable storage medium, where the storage medium may be a read only memory, a magnetic disk or optical disk, etc.

The foregoing description of the preferred embodiments of the application is not intended to limit the application to the precise form disclosed, and any such modifications, equivalents, and alternatives falling within the spirit and scope of the application are intended to be included within the scope of the application.

Claims

1. A method of lung segment segmentation, the method comprising:

acquiring a chest image to be identified and a corresponding lung lobe segmentation result;

resampling the chest image to be identified to an image of a first resolution;

performing lung segment rough segmentation on the image with the first resolution based on a lung segment rough segmentation model to obtain a lung region segmentation result;

determining a first sub-image corresponding to the lung region segmentation result in the chest image to be identified;

resampling the first sub-image and the second sub-image to a first sub-image of a second resolution and a second sub-image of the second resolution, respectively, the second resolution being higher than the first resolution;

and taking the first sub-image with the second resolution and the second sub-image with the second resolution as the input of a two-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image with the second resolution according to the second sub-image with the second resolution to obtain a first lung segment subdivision result.

2. The method according to claim 1, wherein the method further comprises:

acquiring a sample image and a corresponding lung region labeling result;

training the first neural network model for lung segment rough segmentation based on the sample image, and adjusting model parameters of the first neural network model in the training of lung segment rough segmentation until a lung region segmentation result output by the first neural network model is matched with a lung region labeling result corresponding to the sample image;

and taking the current first neural network model as the lung segment rough segmentation model.

3. The method according to claim 1, wherein the method further comprises:

acquiring a sample image and a corresponding lung lobe segmentation result and a lung segment labeling result;

training a second neural network model for lung segment fine segmentation based on the sample image and a corresponding lung lobe segmentation result, and adjusting model parameters of the second neural network model in the training for lung segment fine segmentation until a first lung segment segmentation result output by the second neural network model is matched with a lung segment labeling result corresponding to the sample image;

and taking the current second neural network model as the two-channel lung segment subdivision model.

4. The method of claim 3, wherein obtaining a lung segment annotation result comprises:

acquiring an image with resolution higher than a preset threshold;

carrying out lung lobe segmentation processing on the image with the resolution higher than a preset threshold value to obtain a lung lobe segmentation result;

performing bronchial branch identification on the image with the resolution higher than a preset threshold value to obtain a bronchial branch identification result;

and performing lung segment segmentation on the lung lobe segmentation result based on the bronchus branch identification result to obtain a lung segment marking result.

5. The method of claim 4, wherein performing bronchial branch identification on the image with the resolution higher than a preset threshold value, and obtaining a bronchial branch identification result comprises:

performing bronchus segmentation on the image with the resolution higher than a preset threshold value based on a bronchus segmentation model to obtain a bronchus segmentation result;

and carrying out bronchus branch identification on the bronchus segmentation result based on a bronchus branch identification model to obtain a bronchus branch identification result.

6. The method of claim 1, wherein the obtaining a lung lobe segmentation result comprises:

carrying out lung lobe segmentation processing on the chest image to be identified based on a lung lobe segmentation model to obtain a lung lobe segmentation result;

The lung lobe segmentation model is obtained by carrying out lung lobe segmentation training based on a sample image and a corresponding lung lobe labeling result.

7. A method of lung segment segmentation, the method comprising:

acquiring chest images to be identified and corresponding lung lobe segmentation results and bronchus segmentation results;

resampling the chest image to be identified to an image of a first resolution;

resampling the first sub-image, the second sub-image, and the third sub-image to a first sub-image of a second resolution, a second sub-image of the second resolution, and a third sub-image of the second resolution, respectively, the second resolution being higher than the first resolution;

and taking the first sub-image with the second resolution, the second sub-image with the second resolution and the third sub-image with the second resolution as inputs of a three-channel lung segment subdivision model, and carrying out lung segment subdivision on the first sub-image with the second resolution according to the second sub-image with the second resolution and the third sub-image with the second resolution to obtain a second lung segment subdivision result.

8. The method of claim 7, wherein the method further comprises:

acquiring a sample image, a lung lobe segmentation result corresponding to the sample image, a bronchus segmentation result corresponding to the sample image and a lung segment labeling result;

training the lung segment fine segmentation of a third neural network model based on the sample image, a lung lobe segmentation result corresponding to the sample image and a bronchus segmentation result corresponding to the sample image, and adjusting model parameters of the third neural network model in the training of the lung segment fine segmentation until a second lung segment segmentation result output by the third neural network model is matched with a lung segment marking result corresponding to the sample image;

and taking the current third neural network model as the three-channel lung segment subdivision model.

9. The method of claim 7, wherein the obtaining a bronchial segmentation result comprises:

performing bronchus segmentation processing on the chest image to be identified based on a bronchus segmentation model to obtain a bronchus segmentation result;

the bronchus segmentation model is obtained by carrying out bronchus segmentation training based on a sample image and a corresponding bronchus labeling result.

10. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the lung segment segmentation method of any of claims 1-6 and 7-9.