CN115797376B

CN115797376B - Lung segment segmentation method and device based on tracheal tree search and nearest neighbor distribution

Info

Publication number: CN115797376B
Application number: CN202310079399.6A
Authority: CN
Inventors: 吕行; 叶启志
Original assignee: Zhuhai Hengqin Shengao Yunzhi Technology Co ltd
Current assignee: Zhuhai Hengqin Shengao Yunzhi Technology Co ltd
Priority date: 2023-02-08
Filing date: 2023-02-08
Publication date: 2023-04-28
Anticipated expiration: 2043-02-08
Also published as: CN115797376A

Abstract

The invention provides a lung segment segmentation method and device based on tracheal tree search and nearest neighbor distribution, wherein the method comprises the following steps: obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented; performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result; dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree based on the lung lobe segmentation result, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes; and distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain the lung segment segmentation result corresponding to the any one of the lung lobes. The invention improves the precision and efficiency of lung segment segmentation and has stronger interpretability.

Description

Lung segment segmentation method and device based on tracheal tree search and nearest neighbor distribution

Technical Field

The invention relates to the technical field of image segmentation, in particular to a lung segment segmentation method and device based on tracheal tree search and nearest neighbor distribution.

Background

The lung segment segmentation is a relatively critical task in a nodule positioning module of the lung nodule auxiliary diagnosis system, but because the lung segment has no clear boundary, the task of the lung segment segmentation is difficult, the current academic community has relatively few researches on the lung segment segmentation, and the existing lung segment segmentation work generally uses a deep learning method for the lung segment segmentation. However, the application of deep learning in lung segment segmentation is relatively limited, mainly because of two points: first, deep learning needs to be based on a large number of samples, and the sample labeling process of the lung segments is very difficult, and the time and labor cost for obtaining a large number of samples are very high; second, since there is no clear boundary between the lung segments, and the deep learning network has proven to perform poorly in many tasks when segmenting objects with unclear boundaries, the learning effect of the deep learning network on lung segment segmentation is difficult to guarantee. Therefore, a lung segment segmentation method with higher accuracy and efficiency is needed to meet the industrialization requirements of lung segment segmentation.

Disclosure of Invention

The invention provides a lung segment segmentation method and device based on tracheal tree search and nearest neighbor distribution, which are used for solving the defect of poor efficiency and accuracy in the prior art.

The invention provides a lung segment segmentation method based on tracheal tree search and nearest neighbor distribution, which comprises the following steps:

obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented;

performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result;

dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree based on the lung lobe segmentation result, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree;

and distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain the lung segment segmentation result corresponding to the any one of the lung lobes.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor allocation provided by the invention, based on the lung lobe segmentation result, the tracheal subtrees corresponding to all lung lobes are partitioned from the refined tracheal tree, branch nodes in the tracheal subtrees corresponding to all lung lobes are searched, and segment bronchi in the tracheal subtrees corresponding to all lung lobes are determined, and the method specifically comprises the following steps:

performing depth-first search on the refined tracheal tree to obtain each branch node in the refined tracheal tree and the corresponding branch level and maximum depth; wherein the branch level is determined based on a number of branch nodes between the corresponding branch node and the root node, and the maximum depth is determined based on a maximum number of branch nodes between the corresponding branch node and each leaf node;

after the tracheal subtrees corresponding to the lung lobes are divided from the refined tracheal tree based on the lung lobe segmentation result, determining target branch nodes corresponding to the lung lobes based on branch nodes in the tracheal subtrees corresponding to the lung lobes and the corresponding branch levels and the maximum depths of the branch nodes; the target branch node corresponding to any lung lobe is a branch node connected with different sections of bronchi in the tracheal subtree corresponding to any lung lobe;

And determining segment bronchi in the tracheal subtrees corresponding to the lung lobes based on the target branch nodes corresponding to the lung lobes.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor allocation provided by the invention, the target branch node corresponding to each lung lobe is determined based on the branch nodes in the tracheal subtrees corresponding to each lung lobe and the corresponding branch levels and maximum depths thereof, and the method concretely comprises the following steps:

classifying branch nodes in the tracheal subtrees corresponding to any lung lobe according to branch levels to obtain branch node groups corresponding to the branch levels;

selecting target branch nodes corresponding to any lung lobes from branch node groups corresponding to all branch levels according to the order of the branch levels from small to large based on the structure priori information of the bronchi in any lung lobe and branch nodes in a tracheal subtree corresponding to any lung lobe and the corresponding maximum depth of the branch nodes, until the selected target branch nodes reach the number of target nodes corresponding to any lung lobe; wherein the number of target nodes corresponding to the any one lobe is determined based on the number of bronchi in the any one lobe.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution provided by the invention, the target branch nodes corresponding to any lung lobes are selected from branch node groups corresponding to all the branch levels according to the order of the branch levels from small to large based on the structure priori information of the bronchus in any lung lobe middle segment and the branch nodes in the tracheal subtree corresponding to any lung lobe and the maximum depth corresponding to the branch nodes, and the selected target branch nodes reach the target node number corresponding to any lung lobe, and the method specifically comprises the following steps:

if any lung lobe is the upper left lung lobe, selecting a branch node with the highest maximum depth from the branch node group corresponding to the lowest branch level, and selecting the branch nodes with the highest maximum depth and the highest maximum depth from the branch node group corresponding to the next lowest branch level as the target branch node corresponding to any lung lobe;

and if any lung lobe is other lung lobes except the upper left lung lobe, selecting a branch node with the highest maximum depth from the branch node groups corresponding to each branch level according to the order of the branch levels from small to large as a target branch node corresponding to the any lung lobe until the selected target branch node reaches the number of target nodes corresponding to the any lung lobe.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution provided by the invention, the segment bronchi in the tracheal subtree corresponding to each lung lobe are determined based on the target branch nodes corresponding to each lung lobe, and the method specifically comprises the following steps:

removing a target branch node corresponding to any lung lobe from a tracheal subtree corresponding to any lung lobe to obtain a plurality of tracheal segments corresponding to any lung lobe;

and screening out the tracheal tube sections containing the leaf nodes from the tracheal tube sections corresponding to any one of the lung lobes as the segment bronchi in the tracheal subtree corresponding to any one of the lung lobes.

According to the lung segment segmentation method based on tracheal tree search and nearest neighbor distribution provided by the invention, the depth-first search is carried out on the refined tracheal tree to obtain each branch node in the refined tracheal tree and the corresponding branch level and maximum depth thereof, and the method concretely comprises the following steps:

performing depth-first search on the refined tracheal tree to obtain each branch node in the refined tracheal tree and corresponding branch level, maximum depth and minimum depth; wherein the minimum depth is determined based on a minimum number of branch nodes between the corresponding branch node and each leaf node;

And after screening out the branch level, the maximum depth and the minimum depth corresponding to the branch node with the minimum depth smaller than the preset threshold value, updating the branch level corresponding to each branch node in the sub-branches where the branch node with the minimum depth smaller than the preset threshold value is located.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution, the lung tracheal segmentation result is obtained by performing tracheal segmentation on the lung image to be segmented based on a tracheal segmentation network, obtaining an initial tracheal segmentation result and then performing downsampling on the initial tracheal segmentation result; the resolution of the initial trachea segmentation result is consistent with that of the lung lobe segmentation result;

the distance between a pixel in any one of the lung lobes and a segment bronchus in a tracheal subtree corresponding to the any one of the lung lobes is determined based on the following steps:

performing segment bronchus marking on the lung-trachea segmentation result based on segment bronchi in the tracheal subtree corresponding to each lung lobe, and then upsampling the marked lung-trachea segmentation result to obtain a segment bronchus segmentation result with the same resolution as the initial trachea segmentation result;

and calculating the distance between the pixels in any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the position of the segment bronchi in the segmentation result of the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes.

According to the lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution provided by the invention, the pixels in any lung lobe are distributed to the segment bronchi closest to the lung lobe, and a lung segment segmentation result corresponding to the lung lobe is obtained, and the lung segment segmentation method specifically comprises the following steps:

distributing pixels in any lung lobe to the segment bronchus closest to the lung lobe, and marking corresponding pixels based on the type of the bronchus of the segment bronchus closest to the lung lobe to obtain a lung segment segmentation result corresponding to the lung lobe;

the trachea type of any section of bronchus in the tracheal subtree corresponding to any lung lobe is determined based on the position priori information of the bronchus in the middle section of any lung lobe.

The invention also provides a lung segment segmentation device based on tracheal tree search and nearest neighbor distribution, which comprises the following steps:

the segmentation unit is used for acquiring lung lobe segmentation results and lung trachea segmentation results of the lung images to be segmented;

the refinement unit is used for performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result;

a segment bronchus obtaining unit, configured to divide, based on the lobe segmentation result, a subtree corresponding to each lobe from the refined subtree, search branch nodes in the subtree corresponding to each lobe, and determine segment bronchus in the subtree corresponding to each lobe; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree;

And the lung segment segmentation unit is used for distributing the pixels in any lung lobe to the segment bronchi closest to the segment bronchus in the tracheal subtree corresponding to the any lung lobe based on the distance between the pixels in any lung lobe and the segment bronchus in the tracheal subtree corresponding to the any lung lobe in the lung lobe segmentation result to obtain the lung segment segmentation result corresponding to the any lung lobe.

The invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution when executing the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a lung segment segmentation method based on tracheal tree search and nearest neighbor allocation as described in any of the above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements a lung segment segmentation method based on a tracheal tree search and nearest neighbor allocation as described in any of the above.

According to the lung segment segmentation method and device based on the tracheal tree search and nearest neighbor distribution, the lung images to be segmented are segmented through the deep learning algorithm, the accurate lung lobe segmentation result and the lung tracheal segmentation result are obtained, the refinement algorithm is utilized to obtain the refined tracheal tree, branch nodes of tracheal subtrees corresponding to all lung lobes in the refined tracheal tree are searched, so that segment bronchi corresponding to each lung segment in all lung lobes are found, and finally, pixels in the lung lobes are distributed to the segment bronchi closest to the lung segment by the nearest neighbor method to achieve lung segment segmentation, therefore, the whole lung segment method does not need lung segment data labeling for training of a deep learning module, a large amount of time cost and labor cost are saved, more importantly, the pixels in all lung lobes are segmented into different lung segments rapidly and accurately by means of Duan Zhi tracheal segmentation, the accuracy and efficiency of lung segment segmentation are improved, and the lung segment segmentation method is more interpretable according to the lung segment bronchi mode than the deep learning mode.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow diagram of a lung segment segmentation method based on tracheal tree search and nearest neighbor assignment provided by the invention;

FIG. 2 is a flow chart of a segment bronchus identification method provided by the invention;

FIG. 3 is a schematic diagram of a branching level of a branching node provided by the present invention;

FIG. 4 is a flow chart of a method for calculating the distance between a lung lobe pixel and a Duan Zhi trachea;

FIG. 5 is a schematic diagram of a lung segment segmentation apparatus based on tracheal tree search and nearest neighbor assignment according to the present invention;

fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Fig. 1 is a flow chart of a lung segment segmentation method based on tracheal tree search and nearest neighbor distribution, provided by the invention, as shown in fig. 1, the method comprises:

step 110, obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented;

step 120, performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result;

step 130, dividing the subtrees corresponding to each lung lobe from the refined subtree based on the lung lobe segmentation result, searching branch nodes in the subtrees corresponding to each lung lobe, and determining segment bronchi in the subtrees corresponding to each lung lobe; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree;

and 140, distributing the pixels in any lung lobe to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any lung lobe based on the distance between the pixels in any lung lobe and the segment bronchi in the tracheal subtree corresponding to the any lung lobe in the lung lobe segmentation result, and obtaining the lung segment segmentation result corresponding to the any lung lobe.

Specifically, a lung lobe segmentation result and a lung trachea segmentation result of the lung image to be segmented can be obtained by using a deep learning algorithm. Because the boundary of the lung lobe and the lung trachea is clearer and the labeling difficulty is lower compared with the lung segment, the accurate lung lobe segmentation result and lung trachea segmentation result can be obtained more quickly by utilizing a deep learning algorithm. The trained lung lobe segmentation model can be selected to carry out lung lobe segmentation on the lung image to be segmented to obtain a lung lobe segmentation result of the lung image to be segmented, and the lung lobe segmentation result can be a mask graph containing each lung lobe in the lung image to be segmented, so that each lung lobe region can be rapidly distinguished. The lung lobe segmentation model can be trained based on the sample lung images and the corresponding lung lobe labeling results. In addition, the trained air pipe segmentation model can be selected to carry out air pipe segmentation on the lung image to be segmented, so that a lung air pipe segmentation result of the lung image to be segmented is obtained, and the lung air pipe segmentation result can be a mask image containing the lung air pipe in the lung image to be segmented, so that an air pipe region in the image can be rapidly positioned. It should be noted that the lung lobe segmentation model and the air duct segmentation model may be constructed based on an existing target segmentation network, for example, may be constructed based on unet, which is not particularly limited in the embodiment of the present invention.

Then, the lung and trachea segmentation result is subjected to trachea refinement based on a refinement algorithm (such as a refinement function provided in a skimage tool library), and lung and trachea with a certain area are refined into skeleton lines of single voxels, so that a refined trachea tree corresponding to the lung and trachea segmentation result can be obtained. After extracting the tracheal skeleton of the lung and tracheal segmentation result through the refinement algorithm, root nodes, bifurcation points (hereinafter referred to as branch nodes) and leaf nodes in the tracheal skeleton can be marked by means of characteristic point recognition and the like, so that a formal tree structure is generated, and a refined tracheal tree is obtained.

Based on the respective lobe areas indicated in the lobe segmentation result, the corresponding tracheal subtrees of the respective lobes may be partitioned from the refined tracheal tree to process the corresponding tracheal subtrees of the respective lobes separately. The branch nodes in the subtrees of the respective lobes are searched to determine segment bronchi in the subtrees of the respective lobes. Wherein, the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree. By searching for branch nodes in the subtree corresponding to a certain lobe and based on the spatial structure relationship between the branch nodes in the subtree corresponding to the lobe, the segment bronchi in the subtree corresponding to the lobe can be located. Here, any segment of bronchus corresponds to one lung segment in the lung lobes, and the Duan Zhi bronchus and the lung segments have a close association relationship in space position, so that the corresponding lung lobes can be rapidly segmented into different lung segments by the positioning mode of the segment of bronchus.

Specifically, after obtaining the segment bronchi in the subtree corresponding to each lobe, considering that any segment bronchi corresponds to one of the lobes, the distance between the pixel in any one of the segments and the segment bronchi in the lobe is closer to that of the other segment bronchi, so that each pixel in the lobe can be allocated to the segment bronchi closest to the corresponding pixel by calculating the distance between the pixel in the lobe and each segment bronchi in the subtree corresponding to the lobe, and the pixels in the lobe can be separated according to the segment bronchi, thereby obtaining the segmented result of the lobe corresponding to the lobe. The pixels allocated to the same segment of bronchus belong to the same lung segment, so that a lung segment label can be added to each pixel in the lung lobe, and a lung segment segmentation result corresponding to the lung lobe is obtained. And combining the lung segment segmentation results corresponding to the lung lobes to obtain lung segment segmentation results corresponding to the lung images to be segmented. In addition, when calculating the distance between the pixels in any lung lobe and each segment of bronchus in the tracheal subtree corresponding to the lung lobe, the positions of the pixels in the lung lobe and each segment of bronchus in the lung image to be segmented can be first positioned, so that the pixels in the lung lobe and each segment of bronchus are placed in the same coordinate system to accurately calculate the distance between the pixels in the lung lobe and each segment of bronchus.

According to the method provided by the embodiment of the invention, the lung image to be segmented is segmented through the deep learning algorithm, a relatively accurate lung lobe segmentation result and a lung trachea segmentation result are obtained, a refinement algorithm is utilized to obtain a refinement trachea tree, branch nodes of a trachea subtree corresponding to each lung lobe in the refinement trachea tree are searched, so that a segment bronchus corresponding to each lung segment in each lung lobe is found, and finally, the pixels in the lung lobe are distributed to the nearest segment bronchus by using the nearest neighbor method to realize lung segment segmentation, therefore, the whole lung segment method does not need lung segment data labeling for training of a deep learning module, a great amount of time cost and labor cost are saved, more importantly, the corresponding lung lobe can be rapidly and accurately segmented into different lung segments by using segment bronchus positioning and a manner of segmenting each lung lobe according to Duan Zhi trachea, the precision and the efficiency of lung segment segmentation are improved, and the interpretation according to the lung segment segmentation manner is also stronger than the deep learning manner.

Based on the above embodiment, as shown in fig. 2, the steps of dividing the subtrees corresponding to each lung lobe from the refined subtree based on the lung lobe segmentation result, searching the branch nodes in the subtrees corresponding to each lung lobe, and determining the segment bronchi in the subtrees corresponding to each lung lobe specifically include:

Step 210, performing depth-first search on the refined tracheal tree to obtain each branch node in the refined tracheal tree and the corresponding branch level and maximum depth; wherein the branch level is determined based on a number of branch nodes between the corresponding branch node and the root node, and the maximum depth is determined based on a maximum number of branch nodes between the corresponding branch node and each leaf node;

step 220, after the tracheal subtrees corresponding to each lung lobe are divided from the refined tracheal tree based on the lung lobe segmentation result, determining the target branching nodes corresponding to each lung lobe based on the branching nodes in the tracheal subtrees corresponding to each lung lobe and the branching levels and the maximum depths corresponding to the branching nodes; the target branch node corresponding to any lung lobe is a branch node connected with different sections of bronchi in the tracheal subtree corresponding to any lung lobe;

step 230, determining segment bronchi in the tracheal subtree corresponding to each lung lobe based on the target branch node corresponding to each lung lobe.

Specifically, the depth-first search is performed on the nodes of the refined tracheal tree from the root node (i.e., the node with the largest coordinate value in the z-axis direction) of the refined tracheal tree, so as to obtain each branch node in the refined tracheal tree, and the branch level and the maximum depth corresponding to each branch node. The branch level corresponding to any branch node is determined based on the number of branch nodes between the branch node and the root node, as shown in fig. 3, where the branch level may be 1 added to the number of branch nodes between the branch node and the root node; the maximum depth corresponding to any branch node is determined based on the maximum number of branch nodes between the corresponding branch node and each leaf node, and can be obtained by calculating the maximum value of the distances (such as the number of branch nodes) between the branch node and the leaf nodes belonging to the same traversal path.

Based on the lung lobe segmentation result, after the tracheal subtrees corresponding to the lung lobes are partitioned from the refined tracheal tree, the target branch nodes corresponding to the lung lobes can be determined based on the branch nodes in the tracheal subtrees corresponding to the lung lobes and the corresponding branch levels and the maximum depths of the branch nodes. The target branch node corresponding to any lung lobe is a branch node which is connected with different sections of bronchi in the tracheal subtree corresponding to the lung lobe. By finding out the target branch node corresponding to each lung lobe, the corresponding segment bronchus can be accurately divided from the tracheal subtree corresponding to each lung lobe. Here, since the target branch node corresponding to any lobe is a branch node dividing different segments of bronchi, the target branch node is usually located on a main branch in a subtree of the bronchus corresponding to the lobe, the maximum depth is usually higher, and a certain regularity exists in the branch level between the target branch nodes. Thus, the target branch node corresponding to each lobe may be determined based on the branch node in the corresponding tracheal subtree of each lobe and its corresponding branch level and maximum depth, in combination with the structural information of the trachea within each lobe Duan Zhi. Then, based on the target branch node for each lobe, a segment bronchus in the tracheal subtree for each lobe may be determined.

Based on any one of the foregoing embodiments, the determining, based on the branch nodes in the tracheal subtrees corresponding to the respective lung lobes and the corresponding branch levels and maximum depths thereof, the target branch node corresponding to the respective lung lobes specifically includes:

Specifically, the branch nodes in the tracheal subtrees corresponding to any lung lobe are classified according to the branch levels, so that branch node groups corresponding to all branch levels can be obtained, wherein the branch levels of the branch nodes in the branch node groups corresponding to any branch level are the same. Because of the different distributions of bronchi in different lobes, the strategies for selecting the target branch nodes are correspondingly different. For this, based on the structure priori information of the bronchus in the middle lobe of the lung and the branch nodes and the corresponding maximum depths in the subtrees of the bronchus corresponding to the lung, the target branch nodes corresponding to the lung are selected from the branch node groups corresponding to the branch levels according to the order of the branch levels from small to large until the selected target branch nodes reach the number of target nodes corresponding to the lung. The number of the target nodes corresponding to the lung lobes is determined based on the number of the bronchi in the lung lobes, and the number of the target nodes can be the number-1 of the bronchi in the lung lobes.

Here, the structural prior information of the bronchi in any one lobe includes spatial structural information of bronchi in each segment in the lobe in the subtree of the trachea. Taking the upper left lobe as an example, the prior information of the structure of the middle bronchus in the lobe may include two branches (an upper branch and a lingual branch) divided from the main branch of the subtree corresponding to the lobe, the upper branch is divided into two segments of bronchi (a tip posterior segment bronchus and an anterior segment Zhi Qiguan), and the lingual branch is also divided into two segments of bronchi (an upper lingual segment bronchus and a lower lingual segment bronchus). According to the prior information of the structure of the bronchus in the middle part of the lung lobes, the spatial structure information of the target branch node corresponding to the lung lobes can be obtained. In addition, the branch level and the maximum depth of the branch node also reflect the spatial position of the branch node in the tracheal subtree, so that the branch level and the maximum depth information of the branch node are combined with the structure priori information of the bronchus in the middle part of the lung lobe, and the node selection is performed from the branch node groups corresponding to the branch levels according to the order of the branch levels from small to large, so that the target branch node can be accurately selected.

Based on any one of the above embodiments, the selecting, according to the order of the branch levels from the small to the large, the target branch node corresponding to the any one lung lobe from the branch node group corresponding to each branch level based on the structure prior information of the middle bronchus of the any one lung lobe and the branch nodes in the tracheal subtree corresponding to the any one lung lobe and the corresponding maximum depth thereof, until the selected target branch node reaches the target node number corresponding to the any one lung lobe, specifically includes:

Specifically, if the lung lobe is an upper left lung lobe, a target branch node is known to exist on a main branch of the lung lobe according to the structure priori information of the lung lobe, and then two target branch nodes are also respectively arranged on the two branches, so that for the upper left lung lobe, a branch node with the highest maximum depth can be selected from a branch node group corresponding to the lowest branch level, and the branch node with the highest maximum depth and the highest sub-level can be selected from a branch node group corresponding to the sub-low branch level to serve as the target branch node corresponding to the lung lobe.

If the lung lobe is other lung lobes (such as an upper right lung lobe, a lower left lung lobe and the like) except the upper left lung lobe, selecting a branch node with the highest maximum depth from the branch node groups corresponding to the branch levels according to the order of the branch levels from small to large as a target branch node corresponding to the lung lobe until the number of the selected target branch nodes reaches the number of target nodes corresponding to the lung lobe.

Based on any one of the foregoing embodiments, the determining, based on the target branch node corresponding to each of the lung lobes, a segment bronchus in a tracheal subtree corresponding to each of the lung lobes specifically includes:

In particular, since there are several other tracheal segments in the subtree of the lobe of the lung, in addition to the segment bronchi, for example the initial segment of the main branch of the subtree, the tracheal segment before the division into the upper lingual segment bronchi and the lower lingual segment bronchi on the lingual branch, etc. Therefore, in order to accurately locate the segment bronchi therein from the tracheal subtree corresponding to each lung lobe, the target branching node corresponding to the lung lobe may be deleted from the tracheal subtree corresponding to the lung lobe based on the target branching node obtained above, and the tracheal subtree from which the target branching node is deleted may be divided into a plurality of tracheal segments. And then, screening out the tracheal segments containing the leaf nodes from the tracheal segments corresponding to the lung lobes, wherein the tracheal segments containing the leaf nodes are segment bronchi in the tracheal subtree corresponding to the lung lobes.

Based on any one of the foregoing embodiments, the performing depth-first search on the refined tracheal tree to obtain each branch node in the refined tracheal tree and a corresponding branch level and a maximum depth thereof specifically includes:

Specifically, in order to further improve the recognition accuracy of the bronchi of the subsequent segment, burrs in the refined tracheal tree can be eliminated, so that the burrs in the refined tracheal tree are prevented from affecting the recognition of the bronchi of the segment. In this regard, each branching node and its corresponding branching level, maximum depth, and minimum depth may be obtained simultaneously when performing a depth-first search of the refined tracheal tree. Wherein the minimum depth corresponding to any branch node is determined based on the minimum number of branch nodes between the branch node and each leaf node, and can be obtained by calculating the minimum value of the distance (such as the number of branch nodes) between the branch node and the leaf nodes belonging to the same traversal path, and the minimum depth is used for removing burrs in the refined tracheal tree. Specifically, a branch node with a minimum depth less than a preset threshold may be considered as a burr node, and information such as a corresponding branch level, a maximum depth, a minimum depth, and the like thereof may be screened out, so that the burr node is ignored when the branch node is searched subsequently to determine a segment bronchus. And then, updating the branch level corresponding to each branch node in the sub-branches where the branch node (i.e. the burr node) with the minimum depth smaller than the preset threshold value is located. Here, all the sub-branches issued by the above-mentioned burr node may be acquired, and the branch level-1 of all the branch nodes in the sub-branch may be set.

Based on any one of the above embodiments, the lung-air tube segmentation result is obtained by performing air tube segmentation on the lung image to be segmented based on an air tube segmentation network, obtaining an initial air tube segmentation result, and then performing downsampling on the initial air tube segmentation result; the resolution of the initial trachea segmentation result is consistent with that of the lung lobe segmentation result;

as shown in fig. 4, the distance between the pixels in any one lobe and the segment bronchi in the tracheal subtree corresponding to the any one lobe in the lobe segmentation result is determined based on the following steps:

step 410, performing segment bronchus marking on the lung-trachea segmentation result based on segment bronchi in the tracheal subtree corresponding to each lung lobe, and up-sampling the marked lung-trachea segmentation result to obtain a segment bronchus segmentation result consistent with the resolution of the initial trachea segmentation result;

step 420, calculating the distance between the pixel in any one of the lung lobes and the segment bronchi in the corresponding tracheal subtree of the lung lobe based on the position of the segment bronchi in the segment bronchi segmentation result.

Specifically, in order to further improve the efficiency of lung segment segmentation and enable the lung segment segmentation to meet the industrialization requirement, the lung image to be segmented can be subjected to trachea segmentation based on a trachea segmentation network to obtain an initial trachea segmentation result, and then the initial trachea segmentation result is subjected to downsampling to obtain a lung trachea segmentation result. Wherein the initial tracheal segmentation result is consistent with the resolution of the lobe segmentation result. Since the lung and trachea segmentation result is an image with lower resolution after downsampling, the speed of the lung and trachea segmentation can be effectively improved when the lung and trachea segmentation is refined. When the tracheal subtrees corresponding to the respective lung lobes are divided from the refined tracheal tree based on the lung lobe division result, the lung lobe division result needs to be downsampled so that the resolution thereof is identical to that of the refined tracheal tree, but when the distance between the pixels in any lung lobe in the lung lobe division result and the segment bronchi in the tracheal subtree corresponding to the lung lobe is calculated subsequently, the original, unresampled lung lobe division result is adopted.

Specifically, when calculating the distance between a pixel in any one lung lobe of the lung lobe segmentation results and a segment bronchus in a tracheal subtree corresponding to the lung lobe, marking the segment bronchus of the lung and trachea segmentation results based on the segment bronchus in the tracheal subtree corresponding to each lung lobe, and then upsampling the marked lung and trachea segmentation results to obtain a segment bronchus segmentation result consistent with the resolution of the initial trachea segmentation result. It can be seen that the resolution of the segmented bronchi segmentation results also corresponds to the resolution of the lung lobe segmentation results. Then, based on the position of the segment bronchi in the trachesub-tree corresponding to the lobe in the segment bronchi segmentation result and the position of each pixel in the lobe segmentation result, the distance between the pixel in the lobe and the segment bronchi in the trachesub-tree corresponding to the lobe is calculated.

Based on any one of the foregoing embodiments, the assigning the pixels in any one of the lung lobes to the segment bronchus closest to the lung lobe to obtain a lung segment segmentation result corresponding to the any one of the lung lobes specifically includes:

In particular, after assigning the pixels within the lobe to the segment bronchi closest to the corresponding pixels, the corresponding pixels may be labeled based on the type of trachea of the segment bronchi closest to the corresponding pixels, such that pixels of different lung segments within the same lobe may be distinguished from each other. The tracheal type of any section of bronchus in the tracheal subtree corresponding to any lung lobe is determined based on the position priori information of bronchus in the lung lobe. For example, the upper right lung divides three segments of bronchi, and the upper part is a tip segment bronchi according to the prior information of the position of the lung lobe, so that the segment bronchi with the largest center of gravity in the z direction in the three segments of bronchi is taken as the segment bronchi corresponding to the tip segment; the anterior segment bronchus is located near the anterior chest of the remaining two segments, so the tracheal segment with the smallest center of gravity in the y direction of the remaining two tracheal segments is taken as the segment bronchus corresponding to the anterior segment.

The lung segment segmentation device based on the tracheal tree search and the nearest neighbor distribution, which is provided by the invention, is described below, and the lung segment segmentation device based on the tracheal tree search and the nearest neighbor distribution, which are described below, and the lung segment segmentation method based on the tracheal tree search and the nearest neighbor distribution, which are described above, can be correspondingly referred to each other.

Based on any of the above embodiments, fig. 5 is a schematic structural diagram of a lung segment segmentation apparatus based on tracheal tree searching and nearest neighbor distribution according to the present invention, as shown in fig. 5, the apparatus includes: a segmentation unit 510, a refinement unit 520, a Duan Zhi tracheal acquisition unit 530, and a lung segment segmentation unit 540.

The segmentation unit 510 is used for obtaining a lung lobe segmentation result and a lung trachea segmentation result of the lung image to be segmented;

the refinement unit 520 is configured to perform tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm, so as to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result;

the segment bronchus obtaining unit 530 is configured to divide, based on the lobe segmentation result, a subtree corresponding to each lobe from the refined subtree, search for branch nodes in the subtree corresponding to each lobe, and determine segment bronchi in the subtree corresponding to each lobe; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree;

the lung segment segmentation unit 540 is configured to assign, based on a distance between a pixel in any one of the lung lobes and a segment bronchus in a tracheal subtree corresponding to the any one of the lung lobes, the pixel in the any one of the lung lobes to the segment bronchus closest to the any one of the lung lobes, and obtain a lung segment segmentation result corresponding to the any one of the lung lobes.

According to the device provided by the embodiment of the invention, the lung image to be segmented is segmented through the deep learning algorithm, a relatively accurate lung lobe segmentation result and a lung trachea segmentation result are obtained, a refinement algorithm is utilized to obtain a refinement trachea tree, branch nodes of a trachea subtree corresponding to each lung lobe in the refinement trachea tree are searched, so that a segment bronchus corresponding to each lung segment in each lung lobe is found, and finally, the pixels in the lung lobe are distributed to the nearest segment bronchus by using the nearest neighbor method to realize lung segment segmentation, therefore, the whole lung segment method does not need lung segment data labeling for training of a deep learning module, a great amount of time cost and labor cost are saved, more importantly, the corresponding lung lobe can be rapidly and accurately segmented into different lung segments by using segment bronchus positioning and a manner of segmenting each lung lobe according to Duan Zhi trachea, the precision and the efficiency of lung segment segmentation are improved, and the interpretation according to the lung segment segmentation method is also stronger than the deep learning manner.

Based on any one of the above embodiments, the determining, based on the lobe segmentation result, a segment bronchus in the tracheal subtree corresponding to each lobe from the refined tracheal subtree, and searching for a branch node in the tracheal subtree corresponding to each lobe, specifically includes:

Fig. 6 is a schematic structural diagram of an electronic device according to the present invention, and as shown in fig. 6, the electronic device may include: a processor (processor) 610, a memory (memory) 620, a communication interface (communication interface) 630, and a communication bus 640, wherein the processor 610, the memory 620, and the communication interface 630 communicate with each other through the communication bus 640. Processor 610 may invoke logic instructions in memory 620 to perform a method of lung segment segmentation based on a tracheal tree search and nearest neighbor assignments, the method comprising: obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented; performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result; dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree based on the lung lobe segmentation result, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree; and distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain the lung segment segmentation result corresponding to the any one of the lung lobes.

Further, the logic instructions in the memory 620 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-only memory (ROM), a random access memory (RAM, randomAccessMemory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the method for lung segment segmentation based on tracheal tree search and nearest neighbor allocation provided by the methods described above, the method comprising: obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented; performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result; dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree based on the lung lobe segmentation result, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree; and distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain the lung segment segmentation result corresponding to the any one of the lung lobes.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the above provided method for lung segment segmentation based on tracheal tree search and nearest neighbor allocation, the method comprising: obtaining a lung lobe segmentation result and a lung trachea segmentation result of a lung image to be segmented; performing tracheal refinement on the lung tracheal segmentation result based on a refinement algorithm to obtain a refined tracheal tree corresponding to the lung tracheal segmentation result; dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree based on the lung lobe segmentation result, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes; wherein the branch nodes are nodes except root nodes and leaf nodes in the refined tracheal tree; and distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain the lung segment segmentation result corresponding to the any one of the lung lobes.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lung segment segmentation method based on tracheal tree search and nearest neighbor assignment, comprising:

Distributing the pixels in any one of the lung lobes to the segment bronchi closest to the segment bronchus in the tracheal subtree corresponding to the any one of the lung lobes based on the distance between the pixels in the any one of the lung lobes and the segment bronchi in the tracheal subtree corresponding to the any one of the lung lobes, so as to obtain a lung segment segmentation result corresponding to the any one of the lung lobes;

based on the lung lobe segmentation result, dividing the tracheal subtrees corresponding to the lung lobes from the refined tracheal tree, searching branch nodes in the tracheal subtrees corresponding to the lung lobes, and determining segment bronchi in the tracheal subtrees corresponding to the lung lobes, wherein the method specifically comprises the following steps:

2. The method for segmenting lung segments based on tracheal tree search and nearest neighbor assignment according to claim 1, wherein the determining the target branch node corresponding to each lung lobe based on the branch nodes in the tracheal subtrees corresponding to each lung lobe and the corresponding branch levels and maximum depths thereof specifically comprises:

3. The method for segmenting lung segments based on tracheal tree search and nearest neighbor assignment according to claim 2, wherein the selecting the target branch node corresponding to any lung lobe from the branch node groups corresponding to each branch level according to the order of the branch level from small to large based on the prior information of the structure of the bronchus in any lung lobe and the branch nodes in the tracheal subtree corresponding to any lung lobe and the maximum depth thereof, until the selected target branch node reaches the number of target nodes corresponding to any lung lobe specifically comprises:

4. The method for segmenting lung segments based on tracheal tree search and nearest neighbor assignment according to claim 1, wherein the determining segment bronchi in the tracheal subtree corresponding to each lung lobe based on the target branch node corresponding to each lung lobe specifically comprises:

5. The method for segmenting lung segments based on tracheal tree search and nearest neighbor distribution according to claim 1, wherein the performing depth-first search on the refined tracheal tree to obtain each branch node in the refined tracheal tree and the corresponding branch level and maximum depth thereof specifically comprises:

6. The lung segment segmentation method based on the tracheal tree search and nearest neighbor distribution according to claim 1, wherein the lung tracheal segmentation result is obtained by performing tracheal segmentation on the lung image to be segmented based on a tracheal segmentation network, obtaining an initial tracheal segmentation result, and then performing downsampling on the initial tracheal segmentation result; the resolution of the initial trachea segmentation result is consistent with that of the lung lobe segmentation result;

7. The method for segmenting lung segments based on tracheal tree search and nearest neighbor assignment according to claim 1, wherein the assigning pixels in any one of the lung lobes to the segment bronchi closest to the lung lobe to obtain a lung segment segmentation result corresponding to the any one of the lung lobes specifically comprises:

8. A lung segment segmentation apparatus based on tracheal tree search and nearest neighbor assignment, comprising:

a lung segment segmentation unit, configured to allocate, based on a distance between a pixel in any one lung lobe in the lung lobe segmentation result and a segment bronchus in a tracheal subtree corresponding to the any one lung lobe, the pixel in the any one lung lobe to a segment bronchus closest to the any one lung lobe, so as to obtain a lung segment segmentation result corresponding to the any one lung lobe;

9. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements a method of lung segment segmentation based on tracheal tree search and nearest neighbor allocation according to any of claims 1 to 7.