CN113516784B - Tooth segmentation modeling method and device - Google Patents
Tooth segmentation modeling method and device Download PDFInfo
- Publication number
- CN113516784B CN113516784B CN202110855137.5A CN202110855137A CN113516784B CN 113516784 B CN113516784 B CN 113516784B CN 202110855137 A CN202110855137 A CN 202110855137A CN 113516784 B CN113516784 B CN 113516784B
- Authority
- CN
- China
- Prior art keywords
- tooth
- dimensional
- segmentation
- processing
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30036—Dental; Teeth
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Computer Graphics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Architecture (AREA)
- Geometry (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a tooth segmentation modeling method and device, belongs to the technical field of medical image processing, and solves the problems of poor robustness, sensitivity to noise and the like of the existing method. The method comprises the following steps: performing data cleaning, labeling and preprocessing on the CBCT image sequence to establish a CBCT three-dimensional tooth data set; establishing a three-dimensional convolutional neural network model, wherein the three-dimensional convolutional neural network model comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module; training a three-dimensional convolutional neural network model by using a CBCT three-dimensional tooth data set to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map; post-processing the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model. The tooth region can be accurately predicted.
Description
Technical Field
The invention relates to the technical field of medical image processing, in particular to a tooth segmentation modeling method and device.
Background
Cone beam projection computer reconstructed tomographic imaging apparatus (Cone Beam Computed Tomography, CBCT) is a high-end inspection apparatus necessary for the oral field. The radiation dose of CBCT is basically the same as that of the traditional two-dimensional image, and more perfect soft and hard tissue information can be provided from the 3D layer. The height and thickness of each part of the oral cavity can be measured by using CBCT, so that the relationship between the teeth and the bones can be more intuitively estimated, and the method has important guiding significance and reference value for orthodontic risk assessment, scheme selection and implantation treatment and is widely applied to clinical oral disease examination and treatment. Therefore, how to obtain tooth structure information by using CBCT images has important significance in the field of stomatology.
Image segmentation is an indispensable means for extracting quantitative information of special tissues in medical images, and is also a preprocessing step and a precondition for visual realization. The segmented image can be widely applied to various occasions such as quantitative analysis of tissue volume, diagnosis, positioning of pathological tissues, computer-guided surgery and the like. However, the medical image is often characterized by low contrast, uneven gray level and strong noise, and the problems of complex and various tissues of different individuals, fuzzy boundaries between different soft tissues or between the soft tissues and the focus and the like exist, so the technology still belongs to the difficulty of the current research.
Currently, the methods for segmenting CBCT medical images are mainly divided into two categories: conventional methods that require manual design of features and deep learning methods that typically require a large number of data samples. The traditional methods comprise a region-based segmentation method, a boundary-based segmentation method, a random walk method, a level set method and the like, which generally require more manual interaction, have poor robustness of an algorithm, are sensitive to noise and cannot cope with various clinical situations; at present, the application of deep learning in the CBCT medical image segmentation direction is just started, some researchers adopt 2D CNNs to segment in a two-dimensional layer, and the three-dimensional information of teeth cannot be fully utilized by the method, and the efficiency is low. Therefore, a CBCT tooth segmentation method capable of achieving both segmentation accuracy and efficiency is urgently needed, so that the CBCT image can be applied to the field of oral cavities more conveniently and widely.
Disclosure of Invention
In view of the above analysis, the embodiments of the present invention aim to provide a tooth segmentation modeling method and apparatus, so as to solve the problems that the existing tooth segmentation method is poor in robustness, sensitive to noise, unable to fully utilize three-dimensional information of teeth, and low in efficiency.
In one aspect, an embodiment of the present invention provides a tooth segmentation modeling method, including: collecting CBCT image sequences containing various tooth clinical conditions, and performing data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set; establishing a three-dimensional convolutional neural network model, which comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module, wherein the coding branch is used for extracting characteristics, the first decoding branch is used for predicting a tooth area to obtain a tooth prediction probability map, the second decoding branch is used for carrying out secondary prediction on a tooth root area to obtain a tooth root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the tooth root prediction probability map into a tooth segmentation probability map, wherein the tooth area comprises a crown area and the tooth root area; training the three-dimensional convolutional neural network model by utilizing the CBCT three-dimensional tooth data set to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map; performing post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
The beneficial effects of the technical scheme are as follows: and predicting the tooth by using the first decoding branch to predict the tooth root for the first time by using the coding branch extraction characteristic, and secondarily predicting the tooth root region by using the second decoding branch, and obtaining a weighted average value of the primarily predicted tooth root region and the secondarily predicted tooth root region by using the fusion module so as to accurately predict the tooth root region with a small occupied ratio. And then the false positive segmentation result can be removed by post-processing, so that the three-dimensional tooth segmentation point cloud is obtained. The data cleaning can remove samples with poor imaging quality and serious artifacts caused by metal scattering.
Based on a further improvement of the above method, the center point clustering method includes: binarization processing, connected region instance segmentation processing, minimum region rejection processing, connected region centroid solving and clustering processing and invalid class rejection; the binarization processing is used for carrying out binarization processing on the tooth segmentation probability map to obtain an initial segmentation result; the connected region instance segmentation processing is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of connected regions; the minimum region eliminating process is used for eliminating the connected regions with voxels smaller than a preset value to obtain the residual connected regions; the centroid solving and clustering processing of the connected areas is used for solving the centroids of the residual connected areas and clustering the centroids into two types; and the invalid class eliminating process is used for eliminating the false positive connected region according to the clustering result so as to obtain the three-dimensional tooth segmentation point cloud.
The beneficial effects of the technical scheme are as follows: the binarization processing, connected region instance segmentation processing and minimum region elimination processing can eliminate the minimum connected region, and the connected region with voxels larger than 20000 is reserved. The connected region centroid solving and clustering process and the ineffective class eliminating process can be used for gathering the tooth regions contained in the oral cavity into one to two classes, and false positive segmentation results are further eliminated according to the number of the contained body in each class and the distance between the two classes.
Based on a further improvement of the method, the step of cleaning and labeling the CBCT image sequence further comprises the steps of: the data cleaning is used for removing artifact samples from the CBCT image sequence, and then screening samples containing healthy teeth, metal/resin filling, implants, tooth decay, tooth permanent alternation, tooth sockets and high/low precision; and the labeling is used for labeling tooth areas and tooth root areas in the cleaned CBCT image sequence by using a medical image labeling tool.
The beneficial effects of the technical scheme are as follows: samples containing healthy teeth, metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy can be screened out by data cleaning, and the number of healthy teeth is set to be the same as the sum of the numbers of metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy to be able to balance the distribution of the data set.
Based on a further improvement of the above method, the tooth region in the CBCT image sequence to be segmented is marked as 1 and the region outside the tooth is marked as 0; and labeling the root region in the CBCT image sequence to be segmented as 2 and labeling the outer root region as 0.
The beneficial effects of the technical scheme are as follows: tooth areas in the CBCT image sequence to be segmented are marked as 1, areas outside the teeth are marked as 0, the tooth areas can be predicted by taking the input of the first decoding branch, tooth root areas in the CBCT image sequence to be segmented are marked as 2, areas outside the tooth roots are marked as 0, and the tooth root areas can be secondarily predicted by taking the input of the second decoding branch, so that the defect that primary prediction is inaccurate due to the fact that the tooth root areas occupy smaller areas is overcome.
Based on a further improvement of the method, preprocessing the marked CBCT image sequence further includes: file analysis processing, global intensity standardization processing, image clipping processing, three-dimensional data enhancement processing and category balancing processing, wherein the file analysis processing comprises the following steps: reading a marked CBCT image sequence in a DICOM format from a CBCT image file, and analyzing a three-dimensional image matrix Imagedata, a spatial resolution Voxel Spacing and Origin information Origin according to the CBCT image sequence; the global intensity normalization process: setting the gray value outside the scanning boundary of the analyzed three-dimensional image matrix as 0, and carrying out standardization processing on the image intensity of all the image matrixes; the image cropping process comprises the following steps: sequentially cutting the standardized three-dimensional image matrix into three-dimensional image blocks with pixels of n multiplied by c, wherein n represents the preset height and width of the pixels and c represents the preset depth of the pixels; the three-dimensional data enhancement processing comprises the following steps: carrying out three-dimensional data enhancement on the cut three-dimensional image block, wherein the three-dimensional data enhancement comprises one or more of random three-dimensional overturn, random three-dimensional rotation, random contrast adjustment, random three-dimensional elastic deformation, random Gaussian noise and random poisson noise; the category balancing process: and screening the enhanced three-dimensional image blocks, and filtering the image blocks with the foreground occupation ratio smaller than the set threshold value.
The beneficial effects of the technical scheme are as follows: the more information is included in the image, the more accurate the prediction, the larger the image, and the larger the occupied computer memory. The image cropping process sequentially crops the three-dimensional image matrix into three-dimensional image blocks of pixels n x c to balance the amount of computer memory space occupied and the amount of information included in the image blocks, i.e., to balance the prediction speed and the prediction accuracy.
Based on a further improvement of the above method, the coding branch comprises a plurality of coding layers, and the first decoding branch and the second decoding branch each comprise the same number of decoding layers, wherein each coding layer comprises: the first three-dimensional residual block is used for characteristic extraction analysis; the first attention mechanism module is used for strengthening the characteristic information; and a pooling layer for performing downsampling; each decoding layer includes: the three-dimensional up-sampling layer is used for restoring the segmentation position; the second three-dimensional residual block is used for synthesizing the segmentation information; and a second attention mechanism module for smoothing the segmentation boundary.
Based on a further improvement of the method, the first or the second three-dimensional residual block comprises a first convolution block, a second convolution block, a third convolution block and a nonlinear layer which are sequentially connected, wherein the output of the first convolution block is overlapped with the output of the third convolution block and then sent to the nonlinear layer, so that the output of the three-dimensional residual block is obtained, and the first convolution block to the third convolution block comprise a standardized layer, a three-dimensional convolution layer and the nonlinear layer which are sequentially connected.
Based on a further improvement of the above method, the first or second attention mechanism module comprises a spatial enhancement module and a channel enhancement module in parallel, wherein the spatial enhancement module introduces an attention mechanism from a spatial perspective, comprising a first three-dimensional convolution layer and a nonlinear layer; the channel enhancement module refines feature information from a channel perspective, including a global average pooling layer, a second three-dimensional convolution layer, a third three-dimensional convolution layer, and a non-linear layer.
In another aspect, an embodiment of the present invention provides a tooth segmentation modeling apparatus, including: the tooth data set establishing module is used for collecting CBCT image sequences containing various tooth clinical conditions, and carrying out data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set; the neural network module is used for establishing a three-dimensional convolutional neural network model and comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module, wherein the coding branch is used for extracting characteristics, the first decoding branch is used for predicting a tooth area to obtain a tooth prediction probability map, the second decoding branch is used for carrying out secondary prediction on a tooth root area to obtain a tooth root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the tooth root prediction probability map into a tooth segmentation probability map, and the tooth area comprises a tooth crown area and the tooth root area; a prediction model for training the three-dimensional convolutional neural network model with the CBCT three-dimensional tooth dataset to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map; the post-processing module is used for carrying out post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
Based on a further improvement of the above apparatus, the post-processing module comprises: the tooth segmentation probability map comprises a binarization processing sub-module, a communication region instance segmentation processing sub-module, a minimum region rejection processing sub-module, a communication region centroid solving and clustering processing sub-module and an invalid class rejection sub-module, wherein the binarization processing sub-module is used for carrying out binarization processing on the tooth segmentation probability map to obtain an initial segmentation result; the communication region instance segmentation processing submodule is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of communication regions; the minimum region rejection processing submodule is used for rejecting the connected regions with voxels smaller than a preset value to obtain the residual connected regions; the communication region centroid solving and clustering processing submodule is used for solving the centroid of the residual communication region and clustering the centroid into two types; and the invalid class eliminating processing submodule is used for eliminating the false positive connected region according to the clustering result so as to obtain the three-dimensional tooth segmentation point cloud.
Compared with the prior art, the invention has at least one of the following beneficial effects:
1. samples containing healthy teeth, metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy can be screened out by data cleaning, and the number of healthy teeth is set to be the same as the sum of the numbers of metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy to be able to balance the distribution of the data set.
2. Tooth areas in the CBCT image sequence to be segmented are marked as 1, areas outside the teeth are marked as 0, the tooth areas can be predicted by taking the input of the first decoding branch, tooth root areas in the CBCT image sequence to be segmented are marked as 2, areas outside the tooth roots are marked as 0, and the tooth root areas can be secondarily predicted by taking the input of the second decoding branch, so that the defect that primary prediction is inaccurate due to the fact that the tooth root areas occupy smaller areas is overcome.
3. And predicting the tooth by using the first decoding branch to predict the tooth root for the first time by using the coding branch extraction characteristic, and secondarily predicting the tooth root region by using the second decoding branch, and obtaining a weighted average value of the primarily predicted tooth root region and the secondarily predicted tooth root region by using the fusion module so as to accurately predict the tooth root region with a small occupied ratio. And then the false positive segmentation structure can be removed by post-treatment to obtain a three-dimensional tooth segmentation point cloud.
4. The more information is included in the image, the more accurate the prediction, and the larger the image, the larger the occupied computer memory. The image cropping process sequentially crops the three-dimensional image matrix into three-dimensional image blocks of pixels n x c to balance the amount of computer memory space occupied and the amount of information included in the image blocks, i.e., to balance the prediction speed and the prediction accuracy.
In the invention, the technical schemes can be mutually combined to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, like reference numerals being used to refer to like parts throughout the several views.
Fig. 1 is a flowchart of a tooth segmentation method according to an embodiment of the present invention.
FIG. 2 is a flow chart of a method for tooth segmentation and modeling of CBCT images based on a three-dimensional convolutional neural network according to an embodiment of the present invention;
FIG. 3 is a flowchart of a CBCT image tooth segmentation preprocessing process according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a three-dimensional convolutional neural network model structure according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the structure of an encoding layer and decoding layer in a three-dimensional convolutional neural network according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a three-dimensional residual block in a three-dimensional convolutional neural network according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of the structure of an attention mechanism module in a three-dimensional convolutional neural network according to an embodiment of the present invention;
FIG. 8 is a flowchart of a CBCT image tooth segmentation post-processing according to an embodiment of the present invention;
FIG. 9 is a verification accuracy graph of a three-dimensional convolutional neural network model in accordance with an embodiment of the present invention;
FIG. 10 is a graph of segmentation results of a three-dimensional convolutional neural network model in accordance with an embodiment of the present invention;
fig. 11 is a schematic diagram of a result of a CBCT image tooth segmentation and modeling method based on a three-dimensional convolutional neural network according to an embodiment of the present invention.
Fig. 12 is a block diagram of a tooth segmentation apparatus according to an embodiment of the invention.
Detailed Description
Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and together with the description serve to explain the principles of the invention, and are not intended to limit the scope of the invention.
In one embodiment of the present invention, a tooth segmentation modeling method is disclosed, as shown in FIG. 1. Referring to fig. 1, the tooth segmentation modeling method includes: step S102, collecting CBCT image sequences containing various tooth clinical conditions, and performing data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set; step S104, a three-dimensional convolutional neural network model is established, wherein the three-dimensional convolutional neural network model comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module, the coding branch is used for extracting characteristics, the first decoding branch is used for predicting a tooth area to obtain a tooth prediction probability map, the second decoding branch is used for carrying out secondary prediction on a tooth root area to obtain a tooth root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the tooth root prediction probability map into a tooth segmentation probability map, wherein the tooth area comprises a crown area and a tooth root area; step S106, training a three-dimensional convolutional neural network model by using a CBCT three-dimensional tooth data set to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map; step S108, performing post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and step S110, carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
Compared with the prior art, in the tooth segmentation modeling method provided by the embodiment, the coding branch is utilized to extract the characteristics, the first decoding branch is utilized to predict the tooth so as to predict the tooth root for the first time, the second decoding branch is utilized to predict the tooth root area for the second time, and the fusion module is utilized to calculate the weighted average value of the first predicted tooth root area and the second predicted tooth root area so as to accurately predict the tooth root area with small occupied ratio. And then the false positive segmentation result can be removed by post-processing, and the mass centers of the class with more voxels are reserved, so that the three-dimensional tooth segmentation point cloud can be obtained.
Hereinafter, referring to fig. 1 to 8, steps S102 to S110 of the tooth segmentation modeling method are described in detail.
Referring to fig. 1, step S102, a CBCT image sequence containing various dental clinical situations is collected, and the CBCT image sequence is subjected to data cleaning, labeling and preprocessing to create a CBCT three-dimensional dental dataset. Referring to fig. 2, data cleaning and labeling (i.e., preparing a CBCT three-dimensional tooth dataset) of the CBCT image sequence further includes: the data cleaning is used for removing artifact samples from the CBCT image sequence, and then screening samples containing healthy teeth, metal/resin filling, implants, decayed teeth, tooth-filling alternation, tooth sleeves and high/low precision; and labeling is used to label tooth areas and root areas in the cleaned CBCT image sequence using a medical image labeling tool. In the embodiment, the tooth area in the CBCT image sequence to be segmented is marked as 1, and the area outside the tooth is marked as 0; and labeling the tooth root region in the CBCT image sequence to be segmented as 2 and labeling the region outside the tooth root as 0.
Referring to fig. 2 and 3, preprocessing the annotated CBCT image sequence further includes: file parsing processing, global intensity standardization processing, image cropping processing, three-dimensional data enhancement processing and category balancing processing. File analysis processing: reading a marked CBCT image sequence in a DICOM format from a CBCT image file, and analyzing a three-dimensional image matrix Imagedata, a spatial resolution Voxel Spacing and Origin information Origin according to the CBCT image sequence; global intensity normalization processing: and setting the gray value outside the scanning boundary of the analyzed three-dimensional image matrix as 0, and normalizing the image intensity of all the image matrices. Specifically, the mean and variance of all the marked three-dimensional image matrixes are counted, and the image intensity of all the image matrixes is standardized by using the mean and variance, for example, the standardized data are in Gaussian distribution with mean value of 0 and variance of 1; image clipping: sequentially cutting the standardized three-dimensional image matrix into three-dimensional image blocks with pixels of n multiplied by c, wherein n represents the preset height and width of the pixels and c represents the preset depth of the pixels; three-dimensional data enhancement processing: carrying out three-dimensional data enhancement on the cut three-dimensional image block, wherein the three-dimensional data enhancement comprises one or more of random three-dimensional overturn, random three-dimensional rotation, random contrast adjustment, random three-dimensional elastic deformation, random Gaussian noise and random poisson noise; class balancing: and screening the enhanced three-dimensional image blocks, and filtering the image blocks with the foreground occupation ratio smaller than the set threshold value.
Referring to fig. 1 and 4, in step S104, a three-dimensional convolutional neural network model is established, including a coding branch, a first decoding branch, a second decoding branch and a fusion module, wherein the coding branch is used for extracting features, the first decoding branch is used for predicting a tooth region to obtain a tooth prediction probability map, the second decoding branch is used for performing secondary prediction on a root region to obtain a root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the root prediction probability map into a tooth segmentation probability map, wherein the tooth region comprises a crown region and a root region.
Referring to fig. 4, the coding branch includes a plurality of coding layers, e.g., 5 coding layers, and in alternative embodiments, may include more or fewer coding layers. Referring to fig. 5, each coding layer includes: a first three-dimensional residual block, a first attention mechanism module, and a pooling layer. The first three-dimensional residual block is used for feature extraction analysis, the first three-dimensional residual block comprises a first convolution block, a second convolution block, a third convolution block and a nonlinear layer which are sequentially connected, the output of the first convolution block and the output of the third convolution block are overlapped and then fed into the nonlinear layer, so that the output of the three-dimensional residual block is obtained, and the first convolution block to the third convolution block comprise a standardized layer, a three-dimensional convolution layer and the nonlinear layer which are sequentially connected. The first attention mechanism module is used for strengthening characteristic information and comprises a space enhancement module and a channel enhancement module which are parallel. The spatial enhancement module introduces an attention mechanism from a spatial angle and comprises a first three-dimensional convolution layer and a nonlinear layer; the channel enhancement module refines the feature information from a channel perspective, including a global average pooling layer, a second three-dimensional convolution layer, a third three-dimensional convolution layer, and a non-linear layer. The pooling layer is used for downsampling.
Referring to fig. 4, the first decoding branch and the second decoding branch each include the same number of decoding layers. Referring to fig. 5, each decoding layer includes: a three-dimensional upsampling layer, a second three-dimensional residual block, and a second attention mechanism module. The three-dimensional upsampling layer is used to restore the segmentation locations. The second three-dimensional residual block is used for synthesizing the segmentation information and comprises a first convolution block, a second convolution block, a third convolution block and a nonlinear layer which are sequentially connected, the output of the first convolution block and the output of the third convolution block are superposed and then fed into the nonlinear layer to obtain the output of the three-dimensional residual block, and the first convolution block to the third convolution block comprise a standardized layer, a three-dimensional convolution layer and the nonlinear layer which are sequentially connected. A second attention mechanism module for smoothing the segmentation boundary. The second attention mechanism module includes a spatial enhancement module and a channel enhancement module in parallel. The spatial enhancement module introduces an attention mechanism from a spatial angle and comprises a first three-dimensional convolution layer and a nonlinear layer; the channel enhancement module refines the feature information from a channel perspective, including a global average pooling layer, a second three-dimensional convolution layer, a third three-dimensional convolution layer, and a non-linear layer.
And S106, training the three-dimensional convolutional neural network model by using the CBCT three-dimensional tooth data set to obtain a prediction model, and inputting the CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map.
And S108, performing post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud. Specifically, the center point clustering method includes: binarization processing, connected region instance segmentation processing, minimum region rejection processing, connected region centroid solving and clustering processing and invalid class rejection; binarization processing is used for carrying out binarization processing on the tooth segmentation probability map to obtain an initial segmentation result; the communication region instance segmentation process is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of communication regions; minimum region rejection processing, which is used for rejecting the connected regions with voxels smaller than a preset value to obtain the remaining connected regions; the centroid solving and clustering processing of the connected areas are used for solving the centroids of the rest connected areas and clustering the centroids into two types; and invalid class elimination processing, which is used for eliminating the false positive connected region according to the clustering result to obtain a three-dimensional tooth segmentation point cloud, and specifically, reserving one class of centroid with more voxels in the two classes to obtain the three-dimensional tooth segmentation point cloud.
And S110, carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
Specifically, a moving cube algorithm (Marching cubes: A high resolution: 3D surface construction algorithm) is adopted, 1 is used as a threshold parameter, the obtained tooth segmentation point cloud is subjected to three-dimensional reconstruction, and a tooth three-dimensional model is obtained through calculation.
In another embodiment of the present invention, a tooth segmentation modeling apparatus is disclosed, comprising: the tooth data set establishing module 1202 is used for collecting CBCT image sequences containing various tooth clinical situations, and performing data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set; the neural network module 1204 is configured to establish a three-dimensional convolutional neural network model, and includes a coding branch, a first decoding branch, a second decoding branch and a fusion module, where the coding branch is configured to extract features, the first decoding branch is configured to predict a tooth region to obtain a tooth prediction probability map, the second decoding branch is configured to secondarily predict a root region to obtain a root prediction probability map, and the fusion module is configured to fuse the tooth prediction probability map and the root prediction probability map into a tooth segmentation probability map, where the tooth region includes a crown region and a root region; the prediction model 1206 is used for training the three-dimensional convolutional neural network model by using the CBCT three-dimensional tooth data set to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map; the post-processing module 1208 is configured to post-process the tooth segmentation probability map by using a central point clustering method, and reject the false positive segmentation result to obtain a three-dimensional tooth segmentation point cloud; and the three-dimensional reconstruction module 1210 is configured to perform three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by using a moving cube algorithm, so as to obtain a tooth three-dimensional model.
The post-processing module includes: the system comprises a binarization processing sub-module, a connected region instance segmentation processing sub-module, a minimum region elimination processing sub-module, a connected region centroid solving and clustering processing sub-module and an invalid class elimination sub-module, wherein the binarization processing sub-module is used for carrying out binarization processing on a tooth segmentation probability map to obtain an initial segmentation result; the communication region instance segmentation processing submodule is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of communication regions; the minimum region rejection processing submodule is used for rejecting the connected regions with voxels smaller than a preset value to obtain the residual connected regions; the communication region centroid solving and clustering processing submodule is used for solving the centroids of the rest communication regions and clustering the centroids into two types; and the invalid class eliminating processing submodule is used for eliminating the false positive connected region according to the clustering result to obtain a three-dimensional tooth segmentation point cloud, and particularly, one class of centroid with more voxels in the two classes is reserved to obtain the three-dimensional tooth segmentation point cloud.
Hereinafter, a tooth segmentation method according to an embodiment of the present invention will be described in detail by way of specific example with reference to fig. 2 to 11.
Referring to fig. 2, the CBCT image tooth segmentation and modeling method based on the three-dimensional convolutional neural network according to the embodiment of the present invention includes the following steps:
S1, collecting CBCT image sequences containing clinical situations of various teeth, marking the CBCT images by using a medical image marking tool after cleaning the data, and preprocessing the marked data to establish a CBCT three-dimensional tooth data set.
In the embodiment, on one hand, the data cleaning eliminates samples with poor imaging quality and serious artifacts caused by metal scattering, and on the other hand, the samples containing 7 conditions of healthy teeth, metal/resin filling, implants, tooth decay, tooth stuffiness alternation, tooth sockets and high/low precision are screened out, and the distribution of the data set is balanced according to clinical conditions. For example, the sum of metal/resin filling, implant, tooth decay, tooth replacement, shell, and high/low precision samples is the same as the number of samples of healthy teeth to balance the distribution of the data set.
Referring to fig. 3, in the present embodiment, the preprocessing includes one or a combination of several of file parsing processing, global intensity normalization processing, image cropping processing, three-dimensional data enhancement processing, and category balancing processing.
File analysis processing: and reading the marked CBCT image sequence in the DICOM (Digital Imaging and Communications in Medicine) format, and analyzing the three-dimensional image matrix Imagedata, the spatial resolution Voxel Spacing and the Origin information Origin.
Global intensity normalization processing: setting the gray value outside the scanning boundary of the analyzed three-dimensional image matrix to 0 so as to enhance the contrast ratio of the foreground and the background; performing global intensity standardization on all image matrixes to solve intensity differences caused by radiation doses of different CBCT devices; for example, the global image intensities of all image matrices are normalized to a gaussian distribution.
The image cropping process comprises the following steps: sequentially cutting the standardized three-dimensional image matrix into three-dimensional image blocks with pixels of n multiplied by c, wherein n represents the preset height and width of the pixels, c represents a preset depth of the pixel, which, in this embodiment, cutting into 256×256×256 size 128×128×128 sizes, 64×64×64 sizes, and the like. For example, the larger the crop size, the more spatial information is included, but the more memory space is occupied. Preferably, cut to a size of 128 x 128, the memory space of the computer can be saved, by this clipping approach, the computer processing speed and the amount of spatial information contained can be balanced.
The three-dimensional data enhancement processing comprises the following steps: the three-dimensional data enhancement is performed on the cut three-dimensional image block, and in the embodiment, the three-dimensional data enhancement comprises one or a combination of a plurality of random three-dimensional overturn, random three-dimensional rotation, random contrast adjustment, random three-dimensional elastic deformation, random Gaussian noise and random poisson noise.
The category balancing process: the enhanced three-dimensional image block is screened, and the image block with the foreground ratio smaller than a certain threshold value is filtered, so that the problem of unbalanced front/background category is solved, and in the embodiment, the threshold value is preferably set to be 0.2 because the effective area of the teeth accounts for about 9% of the whole area.
S2, constructing a three-dimensional convolutional neural network model, referring to FIG. 4, wherein the three-dimensional convolutional neural network model consists of one path of coding branch and two paths of decoding branches, one path of decoding branches is used for predicting tooth areas, the other path of decoding branches is used for predicting tooth root areas, and tooth root prediction probability graphs generated by the two paths of branches are fused through feature fusion, so that a more accurate tooth segmentation probability graph is obtained.
The coding branches are formed by sequentially connecting 5 coding layers, referring to fig. 5, each coding layer is formed by a three-dimensional residual block for feature extraction analysis, an attention mechanism module for strengthening feature information and a pooling layer for downsampling; for example, the output feature map of the first encoding layer has a size of 128 x 128, the output feature map of the second encoding layer has a size of 64 x 64, the size of the output feature map of the third encoding layer is 32 x 32, the size of the output characteristic diagram of the fourth coding layer is 16×16×16 and the size of the output feature map of the fifth coding layer is 8×8×8.
The decoding branches are formed by connecting two paths of branches in parallel, each path of branch is formed by sequentially connecting 5 decoding layers, and each decoding layer is formed by a three-dimensional upsampling layer for restoring the segmentation position, a three-dimensional residual block for synthesizing segmentation information and an attention mechanism module for smoothing the segmentation boundary. For example, the size of the output feature map of the first decoding layer is 8 x 8, the output feature map of the second decoding layer has a size of 16 x 16, the size of the output feature map of the third decoding layer is 32 x 32, the size of the output feature map of the fourth decoding layer is 64×64×64 the size of the output feature map of the fifth decoding layer is 128×128×128.
Referring to fig. 6, in this embodiment, the three-dimensional residual block is composed of three convolution blocks and a nonlinear layer which are sequentially connected, where the output of the convolution block 1 and the output of the convolution block 3 are superimposed and then sent to the nonlinear layer, so as to obtain the output of the residual block.
Referring to fig. 7, the attention mechanism module is formed by a space enhancement module and a channel enhancement module in parallel, and the output of the space enhancement module is added with the output of the channel enhancement module to obtain the output of the attention mechanism module. The space enhancement module introduces an attention mechanism from a space angle and consists of a three-dimensional convolution layer and a nonlinear layer. Firstly, compressing the channel information of the input feature map by using 1X 1 three-dimensional convolution, changing the dimension of the feature map from [ C, D, H, W ] to [1, D, H, W ], then activating by using a nonlinear layer to obtain a space attention map, and multiplying the space attention map by the input feature map to obtain the output of the space enhancement module. The channel enhancement module refines the feature information from the channel angle, and in this embodiment, is composed of a global average pooling layer, a three-dimensional convolution layer 1, a three-dimensional convolution layer 2 and a nonlinear layer. The method comprises the steps of firstly compressing spatial information of an input feature map by a global averaging pooling layer, changing the dimension of the feature map from [ C, D, H, W ] to [ C, 1], refining channel features by a full-connection layer and a nonlinear layer which are sequentially connected, obtaining a channel attention map, and multiplying the channel attention map with the input feature map to obtain the output of a channel enhancement module.
In this embodiment, the three-dimensional upsampling layer uses three-dimensional transpose convolution.
S3, in the embodiment, training 1000 epochs on the three-dimensional convolutional neural network model constructed in the step S2 by using the CBCT three-dimensional tooth data set in the step S1, setting the learning rate to be 0.0001, setting the optimizer to be Adam, setting the loss function to be DiceLoss, and storing the whole model in a pt format after the training is completed. The training results are shown in fig. 9. And inputting the CBCT image sequence to be segmented into a trained three-dimensional convolutional neural network model to obtain a tooth segmentation probability map.
The model training Loss function is calculated by the Loss function Loss of the teeth area prediction task tooth Loss function Loss with root region prediction task root And substituting the tooth segmentation probability map or the tooth root prediction probability map obtained by the prediction task and labels obtained by labeling into the composition to finish calculation. Since the prediction of the root region only assists in the learning of the tooth region, a weighted summation approach is used to obtain the final loss function:
Loss＝Loss tooth +λLoss root
preferably, the Loss tooth And Loss of root Including but not limited to cross-entropy Loss (CE), weighted cross-entropy Loss (WCE), focal Loss, dice Loss, BCEDiceLoss, L Loss, etc.
And S4, performing post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud, as shown in FIG. 10.
Referring to fig. 8, in the present embodiment, the center point clustering method includes binarization processing, connected region instance segmentation processing, minimum region rejection processing, connected region centroid solution and clustering processing, and invalid class rejection processing.
The binarization processing: binarizing the tooth segmentation probability map obtained by model prediction to obtain an initial segmentation result, wherein in the embodiment, the threshold value is set to be 0.7; for example, "1" is set when the probability is equal to or greater than 0.7, and "0" is set when the probability is less than 0.7.
The connected region instance segmentation process: and carrying out example segmentation on the initial segmentation result to obtain a plurality of connected areas.
The minimum region elimination processing: performing minimum connected region elimination on the example segmentation result, wherein in the embodiment, connected regions with voxels smaller than 3000 are eliminated; for example, the voxels of the teeth are larger than 20000, preferably the size of the voxels of the teeth is 20000 to 30000.
And solving and clustering the mass centers of the connected areas: the centroids of the remaining connected regions are solved and clustered into two classes.
The invalid class elimination processing: and preserving one class of centroid with more voxels to obtain the three-dimensional tooth segmentation point cloud.
In an alternative embodiment, a non-maximum connected component removal method may be used to post-process the tooth segmentation probability map, and the false positive segmentation result is removed to obtain a three-dimensional tooth segmentation point cloud. The non-maximum connected component removal method comprises one or a combination of a plurality of binarization processing, morphological expansion processing, connected region instance segmentation processing and non-maximum connected region removal processing. Specifically, binarization processing: and binarizing the tooth segmentation probability map obtained by model prediction to obtain an initial segmentation result. Morphological expansion treatment: the initial segmentation results are morphologically expanded to allow closer teeth and dentition to communicate into one or two regions. The connected region instance segmentation process: and carrying out example segmentation on the segmentation result after expansion to obtain a plurality of connected areas. Non-maximum connected region rejection processing: and counting voxels of each connected region after the instance segmentation, only preserving one to two regions with more voxels according to the actual value of each region voxel, and carrying out intersection solution on the preserved regions and the initial segmentation result to obtain the three-dimensional tooth segmentation point cloud.
S5, performing three-dimensional reconstruction on the obtained tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model, as shown in fig. 11. Specifically, a moving cube algorithm (Marching cubes: A high resolution: 3D surface construction algorithm) is adopted, 1 is used as a threshold parameter, the obtained tooth segmentation point cloud is subjected to three-dimensional reconstruction, and a tooth three-dimensional model is obtained through calculation.
Compared with the prior art, the invention has at least one of the following beneficial effects:
1. samples containing healthy teeth, metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy can be screened out by data cleaning, and the number of healthy teeth is set to be the same as the sum of the numbers of metal/resin filling, implants, tooth decay, tooth set alternation, and high/low accuracy to be able to balance the distribution of the data set.
2. Tooth areas in the CBCT image sequence to be segmented are marked as 1, areas outside the teeth are marked as 0, the tooth areas can be predicted by taking the input of the first decoding branch, tooth root areas in the CBCT image sequence to be segmented are marked as 2, areas outside the tooth roots are marked as 0, and the tooth root areas can be secondarily predicted by taking the input of the second decoding branch, so that the defect that primary prediction is inaccurate due to the fact that the tooth root areas occupy smaller areas is overcome.
3. And predicting the tooth by using the first decoding branch to predict the tooth root for the first time by using the coding branch extraction characteristic, and secondarily predicting the tooth root region by using the second decoding branch, and obtaining a weighted average value of the primarily predicted tooth root region and the secondarily predicted tooth root region by using the fusion module so as to accurately predict the tooth root region with a small occupied ratio. And then the false positive segmentation result can be removed by post-processing, and the three-dimensional tooth segmentation point cloud can be obtained.
4. The more information is included in the image, the more accurate the prediction, and the larger the image, the larger the occupied computer memory. The image cropping process sequentially crops the three-dimensional image matrix into three-dimensional image blocks of pixels n x c to balance the amount of computer memory space occupied and the amount of information included in the image blocks, i.e., to balance the prediction speed and the prediction accuracy.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Those skilled in the art will appreciate that all or part of the flow of the methods of the embodiments described above may be accomplished by way of a computer program to instruct associated hardware, where the program may be stored on a computer readable storage medium. Wherein the computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention.
Claims (10)
1. A method of modeling tooth segmentation, comprising:
collecting CBCT image sequences containing various tooth clinical conditions, and performing data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set;
establishing a three-dimensional convolutional neural network model, which comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module, wherein the coding branch is used for extracting characteristics, the first decoding branch is used for predicting a tooth area to obtain a tooth prediction probability map, the second decoding branch is used for carrying out secondary prediction on a tooth root area to obtain a tooth root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the tooth root prediction probability map into a tooth segmentation probability map, wherein the tooth area comprises a crown area and the tooth root area;
Training the three-dimensional convolutional neural network model by utilizing the CBCT three-dimensional tooth data set to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map;
performing post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and
and carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
2. The tooth segmentation modeling method as defined in claim 1, wherein the center point clustering method comprises: binarization processing, connected region instance segmentation processing, minimum region rejection processing, connected region centroid solving and clustering processing and invalid class rejection;
the binarization processing is used for carrying out binarization processing on the tooth segmentation probability map to obtain an initial segmentation result;
the connected region instance segmentation processing is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of connected regions;
the minimum region eliminating process is used for eliminating the connected regions with voxels smaller than a preset value to obtain the residual connected regions;
The centroid solving and clustering processing of the connected areas is used for solving the centroids of the residual connected areas and clustering the centroids into two types; and
and the invalid class eliminating process is used for eliminating the false positive connected region according to the clustering result to obtain the three-dimensional tooth segmentation point cloud.
3. The tooth segmentation modeling method as defined in claim 1, wherein data cleaning and labeling the CBCT image sequence further comprises:
the data cleaning is used for removing artifact samples from the CBCT image sequence, and then screening samples containing healthy teeth, metal/resin filling, implants, tooth decay, tooth permanent alternation, tooth sockets and high/low precision; and
the labeling is used to label tooth areas and root areas in the cleaned CBCT image sequence using a medical image labeling tool.
4. The tooth segmentation modeling method as set forth in claim 3, characterized in that,
marking the tooth area in the CBCT image sequence to be segmented as 1 and marking the area outside the tooth as 0; and
and marking the root area in the CBCT image sequence to be segmented as 2 and marking the area outside the root as 0.
5. The tooth segmentation modeling method as defined in claim 3 or 4, wherein preprocessing the annotated CBCT image sequence further comprises: file parsing processing, global intensity normalization processing, image cropping processing, three-dimensional data enhancement processing, and category balancing processing, wherein,
the file analysis processing: reading a marked CBCT image sequence in a DICOM format from a CBCT image file, and analyzing a three-dimensional image matrix Imagedata, a spatial resolution Voxel Spacing and Origin information Origin according to the CBCT image sequence;
the global intensity normalization process: setting the gray value outside the scanning boundary of the analyzed three-dimensional image matrix as 0, and standardizing the image intensity of all the image matrixes to be within the range of 0 and 1;
the image cropping process comprises the following steps: sequentially cutting the standardized three-dimensional image matrix into three-dimensional image blocks with pixels of n multiplied by c, wherein n represents the preset height and width of the pixels and c represents the preset depth of the pixels;
the three-dimensional data enhancement processing comprises the following steps: carrying out three-dimensional data enhancement on the cut three-dimensional image block, wherein the three-dimensional data enhancement comprises one or more of random three-dimensional overturn, random three-dimensional rotation, random contrast adjustment, random three-dimensional elastic deformation, random Gaussian noise and random poisson noise; and
The category balancing process: and screening the enhanced three-dimensional image blocks, and filtering the image blocks with the foreground occupation ratio smaller than the set threshold value.
6. The tooth segmentation modeling method as defined in claim 1, wherein the encoding branch includes a plurality of encoding layers, and the first decoding branch and the second decoding branch each include a same number of decoding layers, wherein,
each coding layer includes: the first three-dimensional residual block is used for characteristic extraction analysis; the first attention mechanism module is used for strengthening the characteristic information; and a pooling layer for performing downsampling; and
each decoding layer includes: the three-dimensional up-sampling layer is used for restoring the segmentation position; the second three-dimensional residual block is used for synthesizing the segmentation information; and a second attention mechanism module for smoothing the segmentation boundary.
7. The tooth segmentation modeling method according to claim 6, wherein the first or second three-dimensional residual block includes a first convolution block, a second convolution block, a third convolution block, and a nonlinear layer connected in sequence, wherein an output of the first convolution block is superimposed with an output of the third convolution block and then fed into the nonlinear layer to obtain an output of the three-dimensional residual block, and the first to third convolution blocks include a normalized layer, a three-dimensional convolution layer, and a nonlinear layer connected in sequence.
8. The tooth segmentation modeling method as defined in claim 6, wherein the first or second attention mechanism module comprises a spatial enhancement module and a channel enhancement module in parallel, wherein,
the spatial enhancement module introduces an attention mechanism from a spatial angle and comprises a first three-dimensional convolution layer and a nonlinear layer;
the channel enhancement module refines feature information from a channel perspective, including a global average pooling layer, a second three-dimensional convolution layer, a third three-dimensional convolution layer, and a non-linear layer.
9. A tooth segmentation modeling apparatus, comprising:
the tooth data set establishing module is used for collecting CBCT image sequences containing various tooth clinical conditions, and carrying out data cleaning, labeling and preprocessing on the CBCT image sequences to establish a CBCT three-dimensional tooth data set;
the neural network module is used for establishing a three-dimensional convolutional neural network model and comprises a coding branch, a first decoding branch, a second decoding branch and a fusion module, wherein the coding branch is used for extracting characteristics, the first decoding branch is used for predicting a tooth area to obtain a tooth prediction probability map, the second decoding branch is used for carrying out secondary prediction on a tooth root area to obtain a tooth root prediction probability map, and the fusion module is used for fusing the tooth prediction probability map and the tooth root prediction probability map into a tooth segmentation probability map, and the tooth area comprises a tooth crown area and the tooth root area;
A prediction model for training the three-dimensional convolutional neural network model with the CBCT three-dimensional tooth dataset to obtain a prediction model, and inputting a CBCT image sequence to be segmented into the prediction model to obtain a tooth segmentation probability map;
the post-processing module is used for carrying out post-processing on the tooth segmentation probability map by adopting a central point clustering method, and removing false positive segmentation results to obtain a three-dimensional tooth segmentation point cloud; and
and the three-dimensional reconstruction module is used for carrying out three-dimensional reconstruction on the three-dimensional tooth segmentation point cloud by adopting a moving cube algorithm to obtain a tooth three-dimensional model.
10. The tooth segmentation modeling apparatus as defined in claim 9, wherein the post-processing module comprises: the system comprises a binarization processing sub-module, a connected region instance segmentation processing sub-module, a minimum region rejection processing sub-module, a connected region centroid solving and clustering processing sub-module and an invalid class rejection sub-module, wherein,
the binarization processing sub-module is used for performing binarization processing on the tooth segmentation probability map to obtain an initial segmentation result;
the communication region instance segmentation processing submodule is used for carrying out instance segmentation on the initial segmentation result so as to obtain a plurality of communication regions;
The minimum region rejection processing submodule is used for rejecting the connected regions with voxels smaller than a preset value to obtain the residual connected regions;
the communication region centroid solving and clustering processing submodule is used for solving the centroid of the residual communication region and clustering the centroid into two types; and
and the invalid class eliminating processing submodule is used for eliminating the false positive connected region according to the clustering result to obtain the three-dimensional tooth segmentation point cloud.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110855137.5A CN113516784B (en) | 2021-07-27 | 2021-07-27 | Tooth segmentation modeling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110855137.5A CN113516784B (en) | 2021-07-27 | 2021-07-27 | Tooth segmentation modeling method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113516784A CN113516784A (en) | 2021-10-19 |
| CN113516784B true CN113516784B (en) | 2023-05-23 |
Family
ID=78068704
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110855137.5A Active CN113516784B (en) | 2021-07-27 | 2021-07-27 | Tooth segmentation modeling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113516784B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114549559A (en) * | 2022-03-01 | 2022-05-27 | 上海博恩登特科技有限公司 | Post-processing method and system for segmenting tooth result based on CBCT (Cone Beam computed tomography) data AI (Artificial Intelligence) |
| CN114757960B (en) * | 2022-06-15 | 2022-09-09 | 汉斯夫(杭州)医学科技有限公司 | Tooth segmentation and reconstruction method based on CBCT image and storage medium |
| CN116485809B (en) * | 2022-07-01 | 2023-12-15 | 山东财经大学 | Tooth example segmentation method and system based on self-attention and receptive field adjustment |
| CN115359257B (en) * | 2022-08-25 | 2023-05-16 | 华科精准（北京）医疗科技有限公司 | Spine image segmentation method and operation navigation positioning system based on deep learning |
| WO2024108341A1 (en) * | 2022-11-21 | 2024-05-30 | 深圳先进技术研究院 | Method and apparatus for automatic tooth alignment based on point cloud understanding, device, and storage medium |
| CN115661141B (en) * | 2022-12-14 | 2023-03-31 | 上海牙典医疗器械有限公司 | Tooth and alveolar bone segmentation method and system based on CBCT image |
| KR102587782B1 (en) * | 2023-02-03 | 2023-10-11 | 고려대학교 산학협력단 | Apparatus and method for generating 3d teeth model |
| CN116523938A (en) * | 2023-05-16 | 2023-08-01 | 北京长木谷医疗科技股份有限公司 | Method, device, equipment and readable storage medium for processing data after bone segmentation |
| CN117830317B (en) * | 2024-03-05 | 2024-06-18 | 天津医科大学口腔医院 | Automatic orthodontic detection method and system based on image processing |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106327535A (en) * | 2016-08-16 | 2017-01-11 | 苏州迪凯尔医疗科技有限公司 | CBCT tooth root and intraoral scanning dental crown fusion method |
| CN108335318A (en) * | 2018-03-26 | 2018-07-27 | 北京正齐口腔医疗技术有限公司 | The method and device of corona and fusion of roots |
| CN108665533A (en) * | 2018-05-09 | 2018-10-16 | 西安增材制造国家研究院有限公司 | A method of denture is rebuild by tooth CT images and 3 d scan data |
| CN110473243A (en) * | 2019-08-09 | 2019-11-19 | 重庆邮电大学 | Tooth dividing method, device and computer equipment based on depth profile perception |
| CN110930421A (en) * | 2019-11-22 | 2020-03-27 | 电子科技大学 | Segmentation method for CBCT (Cone Beam computed tomography) tooth image |
| CN111833353A (en) * | 2020-07-16 | 2020-10-27 | 四川九洲电器集团有限责任公司 | Hyperspectral target detection method based on image segmentation |
| CN112200843A (en) * | 2020-10-09 | 2021-01-08 | 福州大学 | CBCT and laser scanning point cloud data tooth registration method based on hyper-voxels |
| CN112614127A (en) * | 2020-12-31 | 2021-04-06 | 北京朗视仪器有限公司 | Interactive three-dimensional CBCT tooth image segmentation algorithm based on end-to-end |
| CN112991273A (en) * | 2021-02-18 | 2021-06-18 | 山东大学 | Automatic detection method and system for orthodontic characteristics of three-dimensional tooth model |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11645746B2 (en) * | 2018-11-28 | 2023-05-09 | Orca Dental AI Ltd. | Dental image segmentation and registration with machine learning |
-
2021
- 2021-07-27 CN CN202110855137.5A patent/CN113516784B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106327535A (en) * | 2016-08-16 | 2017-01-11 | 苏州迪凯尔医疗科技有限公司 | CBCT tooth root and intraoral scanning dental crown fusion method |
| CN108335318A (en) * | 2018-03-26 | 2018-07-27 | 北京正齐口腔医疗技术有限公司 | The method and device of corona and fusion of roots |
| CN108665533A (en) * | 2018-05-09 | 2018-10-16 | 西安增材制造国家研究院有限公司 | A method of denture is rebuild by tooth CT images and 3 d scan data |
| CN110473243A (en) * | 2019-08-09 | 2019-11-19 | 重庆邮电大学 | Tooth dividing method, device and computer equipment based on depth profile perception |
| CN110930421A (en) * | 2019-11-22 | 2020-03-27 | 电子科技大学 | Segmentation method for CBCT (Cone Beam computed tomography) tooth image |
| CN111833353A (en) * | 2020-07-16 | 2020-10-27 | 四川九洲电器集团有限责任公司 | Hyperspectral target detection method based on image segmentation |
| CN112200843A (en) * | 2020-10-09 | 2021-01-08 | 福州大学 | CBCT and laser scanning point cloud data tooth registration method based on hyper-voxels |
| CN112614127A (en) * | 2020-12-31 | 2021-04-06 | 北京朗视仪器有限公司 | Interactive three-dimensional CBCT tooth image segmentation algorithm based on end-to-end |
| CN112991273A (en) * | 2021-02-18 | 2021-06-18 | 山东大学 | Automatic detection method and system for orthodontic characteristics of three-dimensional tooth model |
Non-Patent Citations (3)
| Title |
|---|
| Automatic Classification and Segmentation of Teeth on 3D Dental Model Using Hierarchical Deep Learning Networks;SUKUN TIAN 等;IEEE Access;第07卷;第84817-84828页 * |
| 口腔CBCT图像牙齿自动分割方法研究;陈燕琳;中国优秀硕士学位论文全文数据库 医药卫生科技辑(第01期);第E074-6页 * |
| 多级层次三维卷积神经网络的牙颌模型分割与识别技术;田素坤 等;计算机辅助设计与图形学学报;第32卷(第08期);第1218-1227页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113516784A (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113516784B (en) | Tooth segmentation modeling method and device | |
| US11494957B2 (en) | Automated correction of metal affected voxel representations of x-ray data using deep learning techniques | |
| US10991091B2 (en) | System and method for an automated parsing pipeline for anatomical localization and condition classification | |
| EP1649810A1 (en) | Image processing method and computer-readable recording medium containing image processing program | |
| CN110473243A (en) | Tooth dividing method, device and computer equipment based on depth profile perception | |
| Loncaric et al. | 3-D image analysis of intra-cerebral brain hemorrhage from digitized CT films | |
| Avuçlu et al. | Novel approaches to determine age and gender from dental x-ray images by using multiplayer perceptron neural networks and image processing techniques | |
| Bielecki et al. | Hand radiographs preprocessing, image representation in the finger regions and joint space width measurements for image interpretation | |
| Benyó | Identification of dental root canals and their medial line from micro-CT and cone-beam CT records | |
| Lin et al. | Tooth numbering and condition recognition on dental panoramic radiograph images using CNNs | |
| CN114638852A (en) | Jaw bone and soft tissue identification and reconstruction method, device and medium based on CBCT image | |
| CN114004970A (en) | Tooth area detection method, device, equipment and storage medium | |
| Moris et al. | Automated tracking of the mandibular canal in CBCT images using matching and multiple hypotheses methods | |
| EP3270355A1 (en) | Determining a complexity value of a stenosis or a section of a vessel | |
| Datta et al. | Neutrosophic set-based caries lesion detection method to avoid perception error | |
| Anitha et al. | A segmentation technique to detect the Alzheimer's disease using image processing | |
| Engelkes | Accuracy of bone segmentation and surface generation strategies analyzed by using synthetic CT volumes | |
| US20220122261A1 (en) | Probabilistic Segmentation of Volumetric Images | |
| CN115410032A (en) | OCTA image classification structure training method based on self-supervision learning | |
| CN117615733A (en) | Three-dimensional dental image display apparatus and method, and method for training three-dimensional dental image display apparatus | |
| Sayin et al. | Optimization of CBCT data with image processing methods and production with fused deposition modeling 3D printing | |
| Kaarthik et al. | Detection and Classification of Dental Defect using CNN | |
| Ghafoor et al. | Multiclass Segmentation using Teeth Attention Modules for Dental X-ray Images | |
| Carson | Automatic Bone Structure Segmentation of Under-sampled CT/FLT-PET Volumes for HSCT Patients | |
| Misra et al. | Image Processing on IOPA Radiographs: A comprehensive case study on Apical Periodontitis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |