WO2021109988A1

WO2021109988A1 - Ct image-based method and device for realizing lung nodule adaptive matching by bone registration

Info

Publication number: WO2021109988A1
Application number: PCT/CN2020/133091
Authority: WO
Inventors: 蔡飞跃; 赖耀明; 罗召洋; 钱东东; 秦积涛; 魏军
Original assignee: 广州柏视医疗科技有限公司; 广州柏视数据科技有限公司
Priority date: 2019-12-06
Filing date: 2020-12-01
Publication date: 2021-06-10
Also published as: CN112288786A; WO2021109987A1; CN111127527B; CN112288786B; CN111127527A

Abstract

A CT image-based method for realizing lung nodule adaptive matching by bone registration, comprising the following steps: data preparation, extraction of lung and bone point cloud data, three-dimensional point cloud data registration, and lung nodule adaptive matching. In said method, firstly, three-dimensional point cloud rigid transformation registration is performed on lung image data and lung nodule data on the basis of the characteristic of small changes in human bone, so as to achieve the alignment of the lung and the lung nodule data before and after follow-up; secondly, an FGR algorithm is used, and in terms of operation speed and registration accuracy, the FGR algorithm is obviously superior to local refinement algorithms such as ICP; thirdly, the RMSE is used as a lung point cloud registration error, achieving lung nodule adaptive matching, so that the lung nodule registration requires less manual intervention, has a high degree of automation, and achieves an accurate registration result; and fourthly, normalization processing is performed on CT image data, so that the robustness of the algorithm can be improved. The method can be widely applied to CT devices of different models and DICOM data of different pixel spacing values.

Description

Method and device for realizing adaptive matching of lung nodules based on CT image bone registration

Technical field

The invention relates to the technical field of medical equipment, and in particular to a method and device for realizing adaptive matching of lung nodules based on CT image bone registration.

Background technique

Lung cancer is one of the most harmful malignant tumors to human health and life. Malignant pulmonary nodules are an important manifestation of early lung cancer. The growth characteristics of nodules reflect the relationship between the increase in the number or volume of cells in the nodules and time. With the rapid development of medical imaging and computer technology, computer-aided detection of lung nodules based on CT images has become a research hotspot in the early diagnosis of lung cancer. Follow-up observation of CT images can effectively evaluate the growth characteristics of lung nodules over a period of time. , So as to provide a basis for the early detection and accurate diagnosis of lung cancer.

Computer-aided detection of lung nodules requires fast and accurate matching and analysis of lung nodule images before and after follow-up. The existing matching methods mainly include global-based matching methods and local-based matching methods. In actual testing, affected by factors such as differences in patient position and respiratory function, the location and state of lung tissue before and after follow-up are often inconsistent, resulting in large differences between CT images before and after follow-up. Due to the inconsistency of lung CT images and the unpredictability of nodule growth, the matching accuracy of the existing two types of matching methods are relatively low, and matching errors may occur in severe cases.

At the same time, the existing matching algorithms have relatively high computational complexity. With the continuous improvement of CT image quality, mainstream hardware platform configurations can no longer meet actual application requirements, and there is an urgent need for a matching algorithm with better performance.

Summary of the invention

In view of the problems in the prior art, embodiments of the present invention provide a method and device for adaptive matching of lung nodules based on CT image skeletal registration.

In the first aspect, an embodiment of the present invention provides a method for adaptive matching of lung nodules based on CT image skeletal registration, which is characterized in that it includes the following steps:

(1) Data preparation: prepare lung CT images and prepare lung nodules data;

(2) Extract 3D point cloud data of lungs and bones: segment lung and bone regions on CT images, extract 3D point cloud data of lungs and bone contours, and perform sparse sampling processing;

(3) Three-dimensional point cloud data registration: two sets of bone three-dimensional point cloud data are registered, and the conversion matrix obtained by bone registration is used; the registration error of the two sets of lung point cloud data is evaluated;

(4) Adaptive matching of lung nodules: Based on the registration error, a distance-based method is used to match lung nodules.

Further, the method for data preparation in the step (1) is:

(1.1) CT images of the two groups before and after preparation for follow-up;

(1.2) The nodule data of the two groups of CT images before and after the preparation follow-up, including nodule coordinates, length and short diameter, volume and attributes.

Further, the method for extracting three-dimensional point cloud data of lungs and bones in the step (2) is:

(2.1) Convert the DICOM raw data of the two sets of CT images into CT value data;

(2.2) Interpolate the transformed CT value data into the normalized space;

(2.3) Data resampling;

(2.4) Extract the lung area;

(2.5) According to the bone CT value range, extract the bone area to generate the bone 3D point cloud data;

(2.6) Intercept data based on the connected area of the lungs and bones, and remove the areas other than the lungs and bones;

(2.7) Extract the boundary contours of the connected area between the lungs and the bones;

(2.8) Convert the boundary contour data into a three-dimensional point cloud data format.

Further, the method for extracting the lung area in the step (2.4) is:

(2.4.1) Preliminarily extract connected areas according to the threshold range of lung tissue;

(2.4.2) Remove the boundary area of lung tissue and fill the connected area with holes;

(2.4.3) Extract the left and right lungs according to the area and location;

(2.4.4) Combine the left and right lungs, and exclude the connected areas that do not belong to the lungs based on the location of the lungs.

Further, the method for three-dimensional point cloud data registration in the step (3) is:

(3.1) Preprocessing of bone 3D point cloud data, including: extracting FPFH features of bone 3D point cloud data, and sparsely sampling the bone 3D point cloud data;

(3.2) For the extracted FPFH features and the sparsely sampled bone 3D point cloud data, the FGR algorithm is used for point cloud registration to obtain the transformation matrix;

(3.3) According to the transformation matrix, transform the mobile lung point cloud data; calculate the RMSE of the two sets of lung point cloud data after transformation as the registration error.

Further, the method for adaptive matching of lung nodules in the step (4) is:

(4.1) According to the transformation matrix, transform the coordinates of the moving lung nodules;

(4.2) Traverse to find and determine whether the lung nodules match;

(4.3) Generate matching results.

Further, the method for adaptive matching of lung nodules in the step (4) further includes:

According to the registration error of the two sets of lung point cloud data after transformation, adaptively set the threshold of matching nodules;

For the lung nodule searched and determined by traversal, if the coordinate of the lung nodule after transformation is less than the threshold, it is determined that the lung nodule is matched successfully; otherwise, it is determined that the lung nodule is matched unsuccessfully.

Further, the method for realizing adaptive matching of lung nodules based on CT image skeletal registration further includes the following steps:

(5) Analysis of lung nodule growth characteristics: For lung nodules that are successfully matched, calculate the length and short diameter changes, volume changes, and attribute changes of the lung nodules; for lung nodules that are unsuccessfully matched, determine whether the lung nodules are disappeared or Added.

In the second aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor executes the program as described in the first aspect. Steps of the provided method.

In a third aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method provided in the first aspect are implemented.

The embodiment of the present invention provides a method and device for adaptive matching of lung nodules based on CT image bone registration. One is based on the characteristics of small changes in human bones and is obtained by using bones in CT images before and after follow-up for registration. The transformation matrix is used to perform three-dimensional point cloud rigid transformation registration on lung image data and lung nodule data, so as to realize the alignment of lung and lung nodule data before and after follow-up, which can effectively overcome the errors caused by lung deformation; the second is to adopt FGR algorithm, this algorithm does not involve iterative sampling, model fitting or local refinement. In terms of running speed and registration accuracy, it is significantly better than local refinement algorithms such as ICP; the third is to use RMSE as the lung point cloud registration error , Realizes the adaptive matching of lung nodules, the registration of lung nodules requires less manual intervention, high degree of automation, and accurate registration results; fourth, through the normalization of CT image data, the robustness of the algorithm can be improved It can be widely used in different types of CT equipment and DICOM data with different pixel spacing values.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a schematic diagram of the overall flow of a method for adaptive matching of lung nodules based on CT image bone registration according to an embodiment of the present invention;

2 is a schematic diagram of comparison before and after registration of a CT image bone point cloud according to an embodiment of the present invention;

3 is a schematic diagram of comparison before and after registration of a CT image lung point cloud provided by an embodiment of the present invention;

4 is a schematic diagram of a CT image lung nodule matching result provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

A method for adaptive matching of lung nodules based on CT image bone registration, as shown in Figure 1, includes the following steps:

(1) Data preparation: prepare lung CT images and prepare lung nodules data. Specifically, the CT images are two sets of CT images before and after follow-up; the lung nodule data is the lung nodule data reflected by the CT images.

(2) Extract 3D point cloud data of lungs and bones: Using medical image processing methods, segment lung and bone regions on CT images, extract 3D point cloud data of lungs and bone contours, and perform sparse sampling processing.

(3) Three-dimensional point cloud data registration: two sets of bone three-dimensional point cloud data are registered, and the conversion matrix obtained by bone registration is used; the registration error of the two sets of lung point cloud data is evaluated. Preferably, the registration of the two sets of bone 3D point cloud data may use FGR (Fast Global Registration, Fast Global Registration Partially Overlapping 3D Surface Algorithm) or ICP (Iterative Closest Point Iterative Closest Point Algorithm) and other algorithms to improve the conversion matrix. Accuracy, and reduce the amount of calculation.

Specifically, the method for data preparation in the step (1) is:

(1.1) CT images of the two groups before and after the preparation follow-up. In this method, the CT image adopts the DICOM standard data format, that is, the international standard data format conforming to ISO 12052, and can be sorted in ascending order according to the ImagePosition in the DICOM tag.

(1.2) The nodule data of the two groups of CT images before and after the preparation follow-up, including nodule coordinates, length and short diameter, volume and attributes. In this patent, the attributes include but are not limited to calcification, fat, necrosis, density uniformity, CT value, etc.

Specifically, the method for data preparation in the step (2) is:

(2.1) Convert the DICOM raw data of the two sets of CT images into CT value data.

(2.2) Interpolate the transformed CT value data into the normalized space. As a specific implementation manner: the normalized space is [2.0, 2.0, 2.0], and the size spaces in the three directions are kept consistent, thereby improving the generalization ability of the algorithm.

(2.3) Data resampling; as a specific implementation, the resampling interval is [0,255].

(2.4) Using morphology and other methods to extract the lung area, the specific methods are:

(2.4.1) According to the threshold range of lung tissue, the connected area is preliminarily extracted using the binarization method.

(2.4.2) Remove the boundary area of lung tissue and fill the connected area with holes. There may be some unclean areas around the lung tissue in the CT image, which are usually generated during CT imaging, so they need to be removed. In addition, the use of binarization to extract connected regions will cause some mutated lung tissues to fail to extract and produce holes, so they need to be filled.

(2.4.3) According to the area and location of lung tissue, extract the left and right lungs.

(2.4.4) Combine the left and right lungs, and exclude the connected areas that do not belong to the lungs based on their location. In actual work, some data will be extracted through the above method, so there is a need for further optimization processing here.

(2.5) According to the bone CT value range, the bone region is extracted to generate bone 3D point cloud data. The method for extracting the bone region may be based on the specific CT value range of the bone in the CT image, because the CT value range of the bone is different from other organs and tissues.

(2.6) The data is intercepted according to the connecting area between the lungs and the bones, and the area outside the lungs and bones is removed, so that only the connecting area between the lungs and the bones is retained.

(2.7) Extract the boundary contours of the connected area between the lungs and the bones.

Specifically, the method for three-dimensional point cloud data registration in the step (3) is:

(3.1) Preprocessing of skeleton 3D point cloud data: Extract FPFH (Fast Point Feature Histograms) features of skeleton 3D point cloud data; perform sparse sampling of skeleton 3D point cloud data.

(3.2) For the extracted FPFH features and the sparsely sampled bone 3D point cloud data, the FGR algorithm is used for point cloud registration, and the transformation matrix is obtained. As shown in Figure 2, this step is a schematic diagram of the comparison before and after the registration of the CT image bone point cloud. It can be seen from the figure: the left image is the morphology of the bone point cloud before registration, the images of the two bones before registration are inconsistent and cannot be overlapped; the right image is the morphology of the bone point cloud after registration, the two bones after registration The images are the same, basically completely overlapped.

(3.3) According to the transformation matrix, transform the mobile lung point cloud data; calculate the RMSE (Root Mean Square Error) of the two sets of lung point cloud data after transformation as the registration error. The method for transforming and moving lung point cloud data is to multiply the transformation matrix and the point cloud data matrix. As shown in Figure 3, this step is a schematic diagram of the comparison before and after the registration of the lung point cloud of the CT image. It can be seen from the figure: the left picture shows the shape of the lung point cloud before registration, the images of the two lungs before registration are inconsistent and cannot be overlapped; the right picture shows the shape of the lung point cloud after registration, after registration The images of the two lungs are the same, almost completely overlapped.

Specifically, the method for adaptive matching of lung nodules in the step (4) is:

(4.1) According to the transformation matrix, transform the coordinates of the moving lung nodules.

(4.2) Traverse to find and determine whether the lung nodules match.

(4.3) Generate matching results.

Specifically, the method for adaptive matching of lung nodules in the step (4) further includes:

The threshold can be obtained by automatically calculating and matching according to the registration error according to a set formula or a set rule; usually, the threshold is positively correlated with the registration error, so that compared to a fixed threshold setting method, it can be used for different registrations. The result is better adaptability.

As shown in Figure 4, it is a schematic diagram of the matching results of a CT image of lung nodules. It can be seen from the figure: the left picture is the position and coordinates of a nodule in CT before follow-up, and the right picture is the position and coordinates of a nodule in follow-up CT. The two images on the left and right are matched nodules determined by the algorithm. It can be seen from the figure that the coordinates of the nodules are different due to the error in the two shots before and after the follow-up, but the position and surrounding tissues of the nodules can be determined. The two nodules are the same nodules in different periods, which shows that this method has higher matching accuracy.

Preferably, the method for realizing adaptive matching of lung nodules based on CT image skeletal registration further includes the following steps: (5) Analysis of lung nodule growth characteristics: calculating the length of lung nodules for successfully matched lung nodules Diameter change, volume change, and attribute change; for lung nodules that are unsuccessful in matching, determine whether the lung nodule is missing or new. Through this step, the analysis results of the growth characteristics of the lung nodules can be automatically generated, so as to provide support for the doctor to make the diagnosis results quickly and accurately.

In an actual implementation process, 115 test samples (including 601 nodules, 378 with matching and 223 without matching) randomly selected by this method were used for actual testing. The matching time of a single case of this method was <1s , The overall matching accuracy rate is 98.5%.

An embodiment of the present invention also provides an electronic device. As shown in FIG. 3, the electronic device may include: a processor 301, a communications interface 302, a memory 303, and a communications bus 304, where: The processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. The processor 301 can call a computer program stored in the memory 303 and run on the processor 301 to execute the method provided in the above embodiment, for example, including: (1) Data preparation: preparing lung CT images, preparing lung nodules Data; (2) Extracting lung and bone point cloud data: segmenting lung and bone regions on CT images, extracting 3D point cloud data of lung and bone contours, and performing sparse sampling processing; (3) 3D point cloud data matching Accurate: register two sets of bone 3D point cloud data, use the conversion matrix obtained by bone registration; evaluate the registration error of the two sets of lung point cloud data; (4) adaptive matching of lung nodules: based on the registration error, use Distance-based method to match lung nodules; (5) Analysis of lung nodule growth characteristics: calculate the length and short diameter changes, volume changes and attribute changes of lung nodules for successfully matched lung nodules; for unsuccessful lung nodules , It is judged that the lung nodule is disappeared or newly added.

In addition, the above-mentioned logical instructions in the memory 303 can be implemented in the form of a software functional unit and when sold or used as an independent product, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention can be embodied in the form of software products in essence or parts that contribute to the prior art or parts of the technical solutions, and the computer software products are stored in a storage medium. , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

The embodiment of the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. The computer program is implemented when executed by a processor to perform the methods provided in the foregoing embodiments, for example, including: (1) Data Preparation: prepare lung CT images, prepare lung nodules data; (2) extract lung and bone point cloud data: segment lung and bone regions on CT images, extract lung and bone contour 3D point cloud data, and proceed Sparse sampling processing; (3) Three-dimensional point cloud data registration: two sets of bone three-dimensional point cloud data are registered, and the conversion matrix obtained by bone registration is used; the registration error of the two sets of lung point cloud data is evaluated; (4) Lung Nodule adaptive matching: Based on the registration error, a distance-based method is used to match lung nodules; (5) Analysis of lung nodule growth characteristics: For lung nodules that are successfully matched, calculate the length and short diameter changes and volume changes of the lung nodules And attribute changes; for lung nodules that are unsuccessful in matching, judge the lung nodules as disappeared or newly added.

The device embodiments described above are merely illustrative, where the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.

Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM/RAM, magnetic A disc, an optical disc, etc., include several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiment.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

A method for realizing adaptive matching of lung nodules based on CT image bone registration, which is characterized in that it includes the following steps:

(1) Data preparation: prepare lung CT images and prepare lung nodules data;

(2) Extracting lung and bone point cloud data: segmenting lung and bone regions on CT images, extracting three-dimensional point cloud data of lung and bone contours, and performing sparse sampling processing;

(3) Three-dimensional point cloud data registration: two sets of bone three-dimensional point cloud data are registered, and the conversion matrix obtained by bone registration is used; the registration error of the two sets of lung point cloud data is evaluated;

(4) Adaptive matching of lung nodules: Based on the registration error, a distance-based method is used to match lung nodules;

The method for data preparation in the step (1) is:

(1.1) CT images of the two groups before and after preparation for follow-up;

(1.2) Preparation of the nodule data of the two groups of CT images before and after follow-up, including nodule coordinates, length and diameter, volume and attributes; the method of extracting lung and bone point cloud data in step (2) is:

(2.1) Convert the DICOM raw data of the two sets of CT images into CT value data;

(2.2) Interpolate the transformed CT value data into the normalized space;

(2.3) Data resampling;

(2.4) Extract the lung area;

(2.5) Extract the bone region according to the bone CT value range, and generate the bone 3D point cloud data;

(2.6) Intercept data based on the connected area of the lungs and bones, and remove the areas other than the lungs and bones;

(2.7) Extract the boundary contours of the connected area between the lungs and the bones;

(2.8) Convert the boundary contour data into a three-dimensional point cloud data format; the method of extracting the lung area in the step (2.4) is:

(2.4.1) Preliminarily extract connected areas according to the threshold range of lung tissue;

(2.4.2) Remove the boundary area of lung tissue and fill the connected area with holes;

(2.4.3) Extract the left and right lungs according to the area and location;

(2.4.4) Combine the left and right lungs, and exclude the connected areas that do not belong to the lungs based on the location of the lungs; the three-dimensional point cloud data registration method in step (3) is:

(3.1) Preprocessing of bone 3D point cloud data, including: extracting FPFH features of bone 3D point cloud data, and sparsely sampling the bone 3D point cloud data;

(3.2) For the extracted FPFH features and the sparsely sampled bone 3D point cloud data, the FGR algorithm is used for point cloud registration to obtain the transformation matrix;

(3.3) According to the transformation matrix, transform the mobile lung point cloud data; calculate the RMSE of the two sets of lung point cloud data after transformation as the registration error;

The method for adaptive matching of lung nodules in the step (4) is:

(4.1) According to the transformation matrix, transform the coordinates of the moving lung nodules;

(4.2) Traverse to find and determine whether the lung nodules match;

(4.3) Generate matching results.
The method for realizing adaptive matching of lung nodules based on CT image bone registration according to claim 1, wherein the method for adaptive matching of lung nodules in the step (4) further comprises:

According to the registration error of the two sets of lung point cloud data after transformation, adaptively set the threshold of matching nodules;

For the lung nodule searched and determined by traversal, if the coordinate of the lung nodule after transformation is less than the threshold, it is determined that the lung nodule is matched successfully; otherwise, it is determined that the lung nodule is matched unsuccessfully.
The method for realizing adaptive matching of lung nodules based on CT image bone registration according to claim 2, characterized in that it further comprises the following steps:

(5) Analysis of lung nodule growth characteristics: For lung nodules that are successfully matched, calculate the length and short diameter changes, volume changes, and attribute changes of the lung nodules; for lung nodules that are unsuccessfully matched, determine whether the lung nodules are disappeared or Added.