CN117876578A

CN117876578A - Orthodontic tooth arrangement method based on crown root fusion

Info

Publication number: CN117876578A
Application number: CN202311730952.4A
Authority: CN
Inventors: 陈欢欢; 宋广瀛; 崔智铭; 孟琛达; 李威; 范祎; 韩冰; 许天民
Original assignee: Peking University School of Stomatology
Current assignee: Peking University School of Stomatology
Priority date: 2023-12-15
Filing date: 2023-12-15
Publication date: 2024-04-12
Anticipated expiration: 2043-12-15

Abstract

The invention provides an orthodontic tooth arrangement method based on crown root fusion. The invention combines the advantages of two modal data of the CBCT image and the IOS image, and utilizes the deep learning network to segment the CBCT image and the IOS image and fuse the multi-modal data, thereby avoiding time and labor consumption of a large amount of manual operation compared with the tooth arrangement test of the traditional gypsum model; the three-dimensional information of the tooth root and the alveolar bone is increased compared with the pure digital model tooth arrangement, the precision of the crown surface information is also improved compared with the pure CBCT image segmentation, and the three-dimensional crown root fusion model with high-precision crown information and complete tooth root information is constructed. The present embodiment also defines boundary information of an alveolar bone, and reduces defects such as bone fenestration, bone fracture, etc., caused by improper movement of orthodontic teeth. In addition, the invention also increases the gap analysis based on the correction target, and performs virtual tooth arrangement operation based on the target arch and incisor position of clinical orthodontic, thereby meeting the clinical actual demands.

Description

Orthodontic tooth arrangement method based on crown root fusion

Technical Field

The invention relates to the technical field of tooth correction, in particular to an orthodontic tooth arrangement method based on crown root fusion.

Background

Orthodontic treatment has potential side effects including tooth root absorption, bone windowing, bone cracking and the like while aligning teeth, wherein alveolar bone defects possibly cause recurrence of staggered tooth deformity, secondary gingival retraction and other problems, and related researches indicate that the alveolar bone absorption lmm is equivalent to tooth root tip absorption of 3mm, so that reduction or avoidance of the alveolar bone defects caused by orthodontic treatment is particularly important, and the main reasons for the defects such as bone windowing, bone cracking and the like are that the tooth roots move beyond the limit of the thickness of the alveolar bone. Therefore, the gap analysis is carried out based on the correction target before the orthodontic treatment, the safe position of the tooth root in the alveolar bone is determined, and virtual tooth arrangement is carried out, so that the defects of bone windowing, bone cracking and the like can be greatly reduced.

The orthodontic tooth arrangement test commonly used at present is roughly divided into two types, one is a tooth arrangement test established on a traditional gypsum model, and the other is a computer simulation tooth arrangement test established on a digital model technology. For the tooth arrangement test of the traditional plaster model, the teeth on the plaster model are divided one by one to carry out measurement and analysis of crowding degree, dental arch shape, occlusion curve and the like, and for realizing healthy, balanced, stable and beautiful tooth states, a diagnostic tooth rearrangement test is carried out. The orthodontic diagnostic tooth arrangement test based on the digital technology is divided into two types, wherein one type is to obtain a three-dimensional digital model through three-dimensional scanning equipment and perform the simulated tooth arrangement test by using a computer aided design technology; the other is to obtain three-dimensional information of the tooth root, the alveolar bone and the relation between the tooth root and the craniofacial bone based on the CBCT three-dimensional image segmentation technology, and then to carry out a simulated tooth arrangement test by utilizing the computer aided design technology.

Most of the tooth arranging technologies follow the Andrews six-standard tooth arranging rule (Andrews tooth arranging rule), but the tooth arranging technologies can only simulate the arrangement position and the occlusion relation of a dental crown no matter a gypsum model or a mouth sweeping model, and the three-dimensional direction of a tooth root and the position relation of the tooth root and an alveolar bone are not considered; however, although three-dimensional information of the tooth root and the alveolar bone is obtained in the tooth arrangement test based on the CBCT image, the accuracy of the information of the tooth crown surface is poor due to the limitations of shooting accuracy, image noise and the like. In addition, the existing tooth arranging technology is still generally limited in the dislocation state before orthodontic treatment, the tooth positioning consideration based on the orthodontic target is lacked, and the gap analysis aiming at the relative relation between the tooth quantity and the bone quantity is an important aspect of orthodontic diagnosis design, and has important guiding significance for target bow and incisor target positioning. Therefore, there is a great need for exploring an orthodontic tooth arrangement technique for correcting target positioning based on high-precision crown information, taking into account the three-dimensional direction of the tooth root and the positional relationship with the alveolar bone.

Disclosure of Invention

In view of the above, the present invention aims to provide an orthodontic tooth arrangement method based on crown root fusion, which combines the advantages of two modal data of CBCT image and three-dimensional digital model, and constructs a three-dimensional crown root fusion model with high-precision crown information and complete root information, so as to obtain the desired tooth target arrangement position.

The invention provides an orthodontic tooth arrangement method based on crown root fusion, which comprises the following steps:

performing image segmentation on Cone Beam Computed Tomography (CBCT) images of teeth to obtain CBCT segmentation results of each tooth;

image segmentation is carried out on the optical scanning image of the teeth to obtain an IOS segmentation result of each tooth;

performing format matching and model registration on the CBCT segmentation result and the IOS segmentation result to obtain a crown root fusion model, wherein the crown root fusion model comprises crown information and tooth root information;

performing gap analysis on the crown root fusion model to obtain tooth crowding degree, spee curve depth and dental arch prominence degree;

and predicting the target arrangement position of the teeth after orthodontic treatment according to the obtained tooth crowding degree, spee curve depth and dental arch prominence degree and combining with the Anderlu six-standard tooth arrangement rule.

In some embodiments, image segmentation of Cone Beam Computed Tomography (CBCT) images of teeth to obtain CBCT segmentation results for each tooth includes:

identifying foreground regions including the upper jaw and the lower jaw in the CBCT image as regions of interest;

dividing a single tooth from the region of interest by adopting a hierarchical form guide network to obtain a tooth quality heart chart and a tooth skeleton chart, wherein each tooth in the tooth quality heart chart is represented by a group of centroid points, and each tooth in the tooth skeleton chart is represented by skeleton lines representing the axial directions of the teeth;

For each tooth, taking the plaque corresponding to the tooth in the region of interest, the tooth quality heart chart and the tooth skeleton chart as the input of three channels of a multi-task tooth segmentation network, wherein the multi-task tooth segmentation network predicts the volume mask of the tooth by simultaneously regressing the tooth cusps and the boundaries;

and integrating the volume mask of each tooth to obtain the CBCT segmentation result.

In some embodiments, the hierarchical morphology guidance network comprises a centroid extraction network and a skeleton extraction network, both employing a V-Net network architecture having two output branches, wherein:

a first output branch of the centroid extraction network outputs a three-dimensional offset map pointing to a tooth centroid point, and a second output branch of the centroid extraction network outputs a binary tooth segmentation mask;

the first output branch of the skeleton extraction network outputs a three-dimensional offset map pointing to a dental skeleton line, and the second output branch of the skeleton extraction network outputs a binary dental segmentation mask.

In some embodiments, the method further comprises:

and setting a maximum pooling layer and three full-connection layers behind the encoder in the multi-task tooth segmentation network so as to identify the standard tooth code corresponding to the tooth.

In some embodiments, image segmentation is performed on the optically scanned image of the tooth to obtain IOS segmentation results for each tooth, including:

and adopting a two-stage neural network based on end-to-end learning, wherein the first-stage neural network predicts the mass center of each tooth, and the second-stage neural network performs single tooth segmentation based on the predicted mass center of each tooth so as to obtain the IOS segmentation result.

In some embodiments, the first-stage neural network adopts a PointNet++ network architecture, and the second-stage neural network adopts two cascaded sub-networks based on the PointNet++ network architecture, and a confidence map mechanism is introduced into both sub-networks.

In some embodiments, performing format matching and model registration on the CBCT segmentation result and the IOS segmentation result to obtain a coronal root fusion model, comprising:

converting the discrete CBCT segmentation result into a continuous three-dimensional grid representation by adopting a marching cube algorithm so as to perform format matching;

after format matching, extracting the directions of principal components of the CBCT segmentation result and the IOS segmentation result respectively by utilizing principal component analysis to calculate rotation and translation parameters, and calculating transformation parameters between centroids of the CBCT segmentation result and the IOS segmentation result by utilizing singular value decomposition to realize coarse registration of the CBCT segmentation result and the IOS segmentation result;

And on the basis of coarse registration, performing point-to-face registration on the CBCT segmentation result and the IOS segmentation result by adopting an iterative closest point matching registration (ICP) algorithm aiming at each tooth so as to obtain the crown root fusion model.

In some embodiments, performing a gap analysis on the coronal root fusion model to obtain a crowding degree, a Spee curve depth, an arching degree, comprising:

calculating the mass center of each tooth based on a CBCT segmentation result, and performing polynomial fitting on a three-dimensional plane by using a least square method to obtain a dental arch curve;

detecting a geodesic distance field on the surface of the dental crown in the IOS segmentation result, and detecting the peak value of the geodesic distance field to obtain the anatomical feature point of each tooth;

and obtaining the crowding degree, the Spee curve depth and the arch protrusion degree based on the obtained arch curve and the anatomical feature points.

In some embodiments, deriving the crowding degree, spee curve depth, arching degree, based on the derived arching curves and anatomical feature points, comprises:

comparing the length of the arch curve with the sum of all tooth widths to obtain the crowding degree of the teeth;

based on the detected anatomical feature points, a Spee curve is obtained, a desired dental arch curve plane is fitted based on the dental arch curve, and the Spee curve depth is calculated according to the Spee curve and the dental arch curve plane;

Based on the detected anatomical feature points, the distance of the maxillary anterior teeth protruding from the mandibular anterior teeth is calculated in three-dimensional space to obtain the degree of arch.

In some embodiments, predicting a target arrangement position of the orthodontic teeth based on the resulting crowding of teeth, spee curve depth, arch convexity, in combination with the andelu six standard tooth arrangement rules, comprises:

determining an arch expansion operation based on the degree of crowding of teeth;

determining an initial tooth arrangement result based on the Spee curve depth and the arch convexity;

and adjusting the initial tooth arrangement result based on the Andrusen six-standard tooth arrangement rule so as to predict the target arrangement position of the teeth after orthodontic treatment.

The invention has at least the following advantages.

1. The invention fuses the CBCT image and the optical scanning image to realize complementary advantages, the CBCT image provides information of tooth root and bone tissue, and the optical scanning image provides high-precision dental crown information. The invention combines two images to obtain a fusion model which simultaneously has complete three-dimensional data of the dental crowns and the dental roots.

2. And an accurate crown root fusion model is constructed, so that complete and high-precision tooth three-dimensional information is provided, and orthodontic design and analysis are facilitated.

3. Automatic segmentation is realized based on a deep learning algorithm, manual operation is not needed, and efficiency is improved.

4. And a confidence sensing mechanism is introduced, so that the segmentation quality of the complex tooth shape is improved.

5. And based on clinical targets, gap analysis is carried out, so that the orthodontic scheme meets the actual requirements better.

6. Avoid excessive tooth migration to cause bone defect risk, improve orthodontic stability.

7. Computer aided design and simulation analysis, and orthodontic scheme can be optimized without actual operation.

8. The virtual design evaluates a plurality of treatment plans and selects an optimal solution.

9. Objective quantitative indicators are provided, reducing the subjectivity of design and diagnosis.

In general, the invention combines the advantages of two modal data of a CBCT image and a three-dimensional digital IOS image, and utilizes an artificial intelligence algorithm based on deep learning to segment the CBCT image and the IOS image and combine the multi-modal data, thereby avoiding time and labor consumption of a large amount of manual operations compared with the tooth arrangement test of the traditional gypsum model; the three-dimensional information of the tooth root and the alveolar bone is increased compared with the pure digital model tooth arrangement, the precision of the crown surface information is also improved compared with the pure CBCT image segmentation data, and the three-dimensional crown root fusion model with high-precision crown information and complete tooth root information is constructed. The invention also limits the boundary information of the alveolar bone, and reduces defects such as bone fenestration, bone cracking and the like caused by improper movement of orthodontic teeth.

In addition, compared with the prior art, the invention also adds the gap analysis based on the correction target, and performs virtual tooth arrangement operation based on the target arch and incisor position of the clinical orthodontic to obtain the target tooth arrangement position after the orthodontic, so that the invention can be ensured to be in compliance, reasonably and circularly in accordance with the clinical actual requirements.

The method and apparatus of the present invention have other features and advantages which will be apparent from or are set forth in detail in the accompanying drawings and the following detailed description, which are incorporated herein, and which together serve to explain certain principles of the present invention.

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular descriptions of exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the invention.

Fig. 1 illustrates a flow chart of a method of orthodontic tooth alignment based on crown root fusion according to one embodiment of the invention.

Fig. 2 shows a schematic diagram of a CBCT image segmentation module according to an exemplary embodiment of the present invention.

FIG. 3 illustrates a schematic diagram of an IOS image segmentation module, according to an example embodiment of the invention.

FIG. 4 illustrates patient image data collected in accordance with an exemplary embodiment of the present invention.

FIG. 5 illustrates a CBCT image segmentation schematic according to an exemplary embodiment of the present invention.

FIG. 6 illustrates an IOS partitioning result diagram in accordance with an exemplary embodiment of the present invention.

FIG. 7 illustrates a crown root fusion model schematic diagram in accordance with an exemplary embodiment of the present invention.

Fig. 8 illustrates obtaining a target alignment position of orthodontic teeth according to an exemplary embodiment of the present invention.

Detailed Description

Preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Here, the concept of the present invention is generally described in general.

Tooth arrangement tests based on traditional gypsum models require a great deal of manual operation of orthodontists, are time-consuming and labor-consuming, the measurement accuracy is limited by the manufacturing quality of the models and the operation level of doctors, and the tooth arrangement process is destructive operation on the original models and has irreversibility. With the advent of the digital orthodontic age, the advantages of using a computer digital technology to assist in simulating tooth arrangement become more obvious, and the digital simulation tooth arrangement technology has very important research significance and application value for tooth correction whether based on a three-dimensional digital model (usually obtained through an intraoral scanner, also called an IOS model or an IOS image) or based on a CBCT three-dimensional image. However, optical data (IOS images) and imaging data (CBCT images) are each characterized in terms of data acquisition range and data accuracy, and the expression of dental and jaw information by a single data source is not sufficiently accurate or complete.

In view of the fact that various three-dimensional scanning technologies at the present stage still cannot acquire crown root integration models with clinical required precision and integrity requirements through a single channel, the inventor has developed the concept of integrating crown root integration treatment of data from the two sources, and considers the application of a specific method or algorithm to construct a three-dimensional crown root integration model with high-precision crown information and complete tooth root information, so that the requirements of oral clinic on orthodontic functional diagnosis and tooth arrangement are effectively met. The invention explores and establishes a fusion registration method of a digital tooth model and a CBCT three-dimensional image, and the process can be summarized into two key steps of overlapping and integrating three-dimensional data. Overlapping of three-dimensional data refers to the realization of coincidence of relative position relationships of two or more three-dimensional models based on certain feature matching, and is also called coarse registration; the integration of the three-dimensional data means that the redundancy and integration of the data layer are further realized on the basis of overlapping of two or more three-dimensional models, and finally, the three-dimensional crown root fusion model with high-precision optical scanning crown data and CBCT root data is obtained, so that support is provided for the subsequent prediction of the target arrangement position of the teeth after orthodontic treatment.

Furthermore, the research direction of the present invention is also based on clinical practice. The existing digital tooth arrangement technology mostly encodes tooth arrangement rules into constraint items which can be defined mathematically, and then virtual tooth arrangement is realized according to the constraint items which can be defined mathematically, but the final result is limited by the encoding mode of the tooth arrangement rules, and the consideration of positioning of correction targets is lacking, so that the method is separated from clinical practice to a certain extent. The invention performs gap analysis based on the correction target, and determines the target arch and incisor positions according to the analysis result, thereby performing virtual tooth arrangement. Wherein gap analysis mainly includes three aspects: crowding of teeth, spee curve depth, and arching. According to the average value of normal values measured by head shadows of specific people, the lower middle incisor-AP distance is 4.9mm, and the actual measured sample is multiplied by 2 after subtracting the corresponding average value of normal values from the lower middle incisor-AP distance before treatment, so that the gap amount required for correcting the incisor process is obtained; and adding the crowding degree and Spee curve depth measured by the model before treatment to obtain the total amount of the needed dental arch clearance. In addition, the arch target state means that the teeth should be arranged on a regular arch form determined by the form of the abutment bone, and the front boundary position of the arch should be determined by the possible correction target of the incisor process, which depends on the side view process, the incisor lip inclination, the form of the tooth Zhou Jigu, the relationship between the upper and lower jaws of the patient, etc., and the head shadow measurement value can be referred to and the above situation can be combined to determine that the teeth, especially the incisors, are arranged in a position satisfying the beauty and stability as much as possible. Because the arch state has certain race and individual difference, and the original arch before treatment is maintained as much as possible to be helpful for long-term stability, the average arch based on specific crowd data can be used for combining the width of the front cuspids and intermolar base bone to determine the target arch so as to obtain the optimal target tooth arrangement position matched with the patient.

Fig. 1 illustrates a flow chart of a method of orthodontic tooth alignment based on crown root fusion according to one embodiment of the invention. As shown in the figure, the method comprises the steps 1 to 5.

And step 1, performing image segmentation on a (CBCT) image of the teeth to obtain a CBCT segmentation result of each tooth.

CBCT (Cone Beam Computed Tomography ) is a type of tomographic modality that uses cone-beam X-ray bundles to obtain three-dimensional images of tooth and jaw areas, which are imaged as radiographic images, unlike optical imaging of IOS.

In some embodiments, the segmentation process may specifically include:

In this section, the inventors developed an artificial intelligence CBCT image segmentation technique based on deep learning that can automatically segment teeth from dental CBCT images clinically stably and accurately. According to the present embodiment, a foreground region including the upper jaw and the lower jaw may be first identified using the ROI extraction network to reduce the computational cost of performing segmentation on the high resolution 3D CBCT image. ROI refers to "Region of Interest", which can be generally translated into a "region of interest". Specifically, the ROI extraction network may use an encoder-decoder network architecture for binary segmentation, automatically segment out the tooth regions of the foreground to locate tooth ROIs, and then perform further single tooth segmentation based on the segmented tooth ROIs. Because the tooth segmentation calculation is very large directly on the whole jaw CBCT three-dimensional image. Thus by first identifying the region of interest including the upper and lower jaws, extraneous regions can be excluded, leaving only the dental region of interest to reduce the computational effort of subsequent segmentation.

Then, the identified region of interest is used as input to a hierarchical guidance network to predict a dentin core map and a tooth skeleton map to segment individual teeth.

a first output branch of the centroid extraction network outputs a three-dimensional offset map (i.e., a three-dimensional vector) pointing to a tooth centroid point, and a second output branch of the centroid extraction network outputs a binary tooth segmentation mask;

the first output branch of the skeleton extraction network outputs a three-dimensional offset map (i.e., a three-dimensional vector) directed to a dental skeleton line, and the second output branch of the skeleton extraction network outputs a binary dental segmentation mask.

Background voxels in the three-dimensional offset map of the first output branch output may be filtered using a segmentation mask of the second output branch output. For the filtered tooth centroid points and the tooth skeleton lines, a rapid clustering algorithm can be adopted for clustering, each tooth is distinguished according to the space centroid position after clustering, the number of the teeth is identified, and each detected tooth can be represented by a skeleton of the tooth.

On this basis, a multi-task tooth segmentation network can be introduced that enables single tooth segmentation, predicting the volume mask for each tooth by simultaneously regressing the corresponding cusps and boundaries. The multi-task tooth segmentation network has a plurality of predictive goals that predict the cusps, contours and masks of the teeth by simultaneously regressing the corresponding cusps and boundaries, thereby providing an overall structure and fine boundaries of the teeth, and properly representing and segmenting each tooth, and particularly the root region, from background tissue, so as to ensure that the root does not penetrate the surrounding bone while the tooth is in motion.

In some embodiments, a max pooling layer and three fully connected layers may be provided after the encoder in the multi-tasked tooth segmentation network to identify the standard tooth code corresponding to the tooth, such as a FDI World Dental Federation marker system based code. The FDI marking system is an internationally used tooth coding system that gives a unique double bit number code for each tooth in humans. The output of the multi-tasking tooth segmentation network may also include the classification result to determine to which class of coded teeth the input teeth belong.

Fig. 2 shows a schematic diagram of a CBCT image segmentation process according to an exemplary embodiment of the present invention. In this example, all CBCT images were normalized to 0.4X0.4X0.4 mm ³ For adjusting the intensity of each CBCT scan to 0, 2500]Normalizing the voxel intensity to 0,1]To pre-process the CBCT image. The preprocessed image is then input into the ROI extraction network, which is an encoder-decoder network, employing the V-Net network architecture. The ROI extraction network performs a binary segmentation task that automatically segments the foreground teeth for tooth region localization, i.e., identifies a region of interest including the upper and lower jaws.

The extracted tooth ROI may be input into a hierarchical morphology guidance network. The hierarchical morphology guidance network comprises a centroid extraction network and a skeleton extraction network, and a tooth centroid diagram and a tooth skeleton diagram are respectively obtained.

Followed by a multi-tasking tooth segmentation network. Three channels of the multi-task tooth segmentation network respectively input plaques cut out of the tooth mass heart map, the tooth skeleton map and the tooth ROI image, and the size of each channel is 96 multiplied by 96. The multi-tasked tooth segmentation network segments each tooth using a V-Net network architecture with multiple task-specific outputs, outputting a corresponding tooth root tip, tooth profile and tooth mask. Combining the tooth root tip, tooth profile and tooth mask can yield a volumetric mask for each tooth as a result of CBCT segmentation.

The encoder section of the multi-tasked tooth segmentation network is followed by a max pooling layer and three fully connected layers to identify the class of each incoming tooth patch, i.e., its FDI code.

Returning to fig. 1, step 2, performing image segmentation on the optical scan image of the teeth to obtain IOS segmentation results of each tooth.

IOS (Intraoral Scanner) is an abbreviation of an intraoral hand-held optical scanning device, which can scan teeth in the oral cavity with high precision by using a laser or structured light mode to obtain a three-dimensional digital model based on optical imaging.

In some implementations, IOS image segmentation may include: and adopting a two-stage neural network based on end-to-end learning, wherein the first-stage neural network predicts the mass center of each tooth, and the second-stage neural network performs single tooth segmentation based on the predicted mass center of each tooth so as to obtain the IOS segmentation result.

In some embodiments, the first-stage network employs a PointNet++ network architecture, and the second-stage network employs two cascaded subnetworks based on the PointNet++ network architecture, and incorporates a confidence mechanism in both subnetworks.

PointNet++ is a classical network architecture for point cloud segmentation. The PointNet++ has strong learning ability, can efficiently process irregular point cloud data, and is excellent in various three-dimensional point cloud analysis tasks. Therefore, in the present embodiment, the IOS image segmentation is implemented using a deep neural network based on the pointet++ architecture.

The core of this embodiment is a two-stage neural network, the first stage first detects all teeth and the second stage precisely segments each tooth detected.

In the first stage of tooth detection, unlike the conventional method of cutting out detection objects using a bounding box, the present embodiment uses the centroid of a tooth to identify each tooth object. Based on the inventor's observations, the centroid point is a stable feature point inside the tooth, regardless of the tooth shape, position and orientation, and thus can be considered to translate the tooth detection problem into the tooth centroid prediction problem. To predict the tooth centroid, a distance-aware voting scheme may be employed to generate the tooth centroid from the downsampled points by learning local context information.

In the second stage of segmenting individual teeth, corresponding points and features may be tailored based on the center of mass of the teeth that has been predicted in the first stage and combined into a suitable segmentation scheme. Thereafter, the segmentation schemes for all teeth are input to a segmentation module to generate individual tooth labels. In addition, in order to improve segmentation accuracy, especially for fuzzy tooth boundary information, a point-by-point confidence map based on a cascade network is introduced in the embodiment, and tag learning is enhanced through a attention mechanism.

FIG. 3 illustrates an IOS image segmentation schematic according to an example embodiment of the invention. The centroid of each tooth is predicted by a first stage neural network (Centroid Prediction) to correctly identify each tooth. Vertex sampling (vertices sampling) may be performed on three-dimensional tooth surface mesh data (input mesh) to obtain an input point cloud P (Sampling points) with dimensions n×6, where n=16000 is the number of sampled input points, each point being represented by a six-dimensional (3+3) vector, including three-dimensional coordinates and normal vectors, which are used as features to provide auxiliary information. The input point cloud P is normalized within a single sphere and geometric features are extracted using a pointnet++ as the backbone Encoder (Encoder), the output of which is a set of sub-sampled points F (Subsampled Points), the dimension of which is mx (3+256), where m=256 is the number of sub-sampled points. For each point, there is another 256-dimensional feature code that provides its surrounding context information in addition to the 3D coordinates. For dental models of the upper or lower jaw, there is a set of artificially labeled real tooth centroids (GT centroids) that hope to predict the centroids of all teeth from sub-sampling point F using learned local features. Thus, according to this embodiment, a displacement function is designed to learn the offset (Offsets) of each sub-sample point from its corresponding centroid, and if one sub-sample point appears around a tooth, the feature encoding capturing the tooth shape can predict the centroid of the nearby tooth. Finally, a regressive centroid set (Predicted centroids) is generated to approximate a real centroid set (GT centroids). Since there are sub-sampling points far from all teeth, which encode little tooth information and cannot predict reliable centroids, a Distance estimation branch including Distance map is used in this embodiment to regress the Distance value of each sub-sampling point, and measure the proximity to its nearest real centroid to automatically filter the points.

Next, in the second stage neural network (Tooth Segmentation), a task of segmenting individual teeth is performed using the predicted tooth centroid as guide information. Each tooth has at least one predicted centroid (Predicted centroids), a proposed segmentation scheme (Propos al) may first be generated based on the predicted centroids, the nearest 4096 points in the (Cropping) input point cloud data, approximately one-fourth of the points of the input tooth model (16,000), are clipped (cropped) based on Euclidean distances to the predicted tooth centroids, and inclusion of the complete tooth in the proposed segmentation scheme (Propos al) is ensured. The proposed segmentation scheme (Propos al) of teeth comprises three components, the first is the coordinates of the cutting point (3-dims), the second is the propagation characteristics of the cutting point (32-dims), the last component is the dense distance field df (i) (1-dims) of the i-th proposed scheme, the distance field proposed here is used for predicting that the foreground tooth corresponding to the centroid has a higher value relative to the other teeth in the cutting point, as a guide for the segmentation sub-network. Three separate features are spliced into the segmentation network for foreground tooth segmentation. The second stage network added with the confidence mechanism, also called a confidence perception cascade segmentation network, is a segmentation network constructed based on PointNet++, takes cascade features of n× (3+16+1) dimensions as input, and outputs a binary label (Predicted mask) of each point belonging to teeth or a background.

Although PointNet++ is excellent in point cloud segmentation, it is difficult to clearly separate the tooth shape from surrounding gums due to the geometrical signal blurring near the tooth boundary and the large change in the tooth shape. Therefore, the second stage neural network in this embodiment uses a cascade split scheme, including two split sub-networks S1 and S2. The cascading scheme of S2 takes as input both the proposed feature (Proposal) and the one-dimensional segmentation result (Predicted mask) of S1. In addition, in order to further improve the segmentation accuracy near the boundary of the complex tooth profile, the present embodiment also adopts a new confidence-aware tooth segmentation attention mechanism, which is specifically as follows. In the first segmentation sub-network S1, in addition to the segmentation result (Predicted mask) of the prediction Proposal (Proposal), another branch (Weighted map) is introduced to estimate the point confidence value λ to measure the accuracy of the segmentation. Lambda is trained in an unsupervised manner, as the boundary region of the geometric signal blur tends to have a lower confidence value, the higher lambda the more accurate the prediction result. In the second segmentation sub-network S2, the confidence map is converted into a normalized weight map, emphasizing the regions of the segmentation S2 where λ is lower, such as the boundary regions. In addition, to identify the foreground tooth ID in each Propos al, the global features extracted in S2 can be used to classify and calculate the cross entropy loss LID to supervise the task.

Returning to fig. 1, step 3, performing format matching and model registration on the CBCT segmentation result and the IOS segmentation result to obtain a coronal root fusion model, wherein the coronal root fusion model comprises dental coronal information and dental root information.

In some embodiments, the coronal root fusion model may be obtained by:

The IOS segmentation result and the CBCT segmentation result are different in data format, and the shooting positions of patients are obviously different, so that the tooth and crown segmentation results obtained from the CBCT segmentation and the IOS segmentation respectively have the problem of difference in format and spatial position, and the IOS segmentation result can be subjected to multi-mode fusion to obtain a crown root fusion model according to the invention.

The multi-mode fusion provided by the invention comprises two parts of data format matching and model registration.

Regarding format matching, according to the present embodiment, a discrete voxel-based CBCT segmentation structure may be converted into a continuous 3D grid representation using a marching cube algorithm, thereby bridging the format gap of the two data sources and obtaining a unified grid model containing CBCT image structure information.

Regarding model registration, a two-stage registration scheme is proposed according to the present embodiment: patient-level coarse registration and tooth-level fine registration. The coarse registration may align the overall position and orientation of the CBCT segmentation results and IOS segmentation results, adjusting the reference coordinate system of the two models to ensure consistency. Specifically, directions of a plurality of (e.g., 3) principal components may be first extracted using Principal Component Analysis (PCA), thereby calculating corresponding rotation and translation parameters, and the principal direction obtained by PCA extraction may be reduced in robustness due to lack of information about the coronal root, so that transformation parameters between two data centroids may be calculated using Singular Value Decomposition (SVD) in combination with information of the centroids, resulting in a coarse registered coronal model. Although rough registration at the patient level may enable preliminary alignment of the entire dentition, insufficient registration may occur, particularly in cases where the patient has a dentition misalignment, and therefore point-to-face ICP registration may be performed at the tooth level according to this embodiment, i.e., on teeth that are not satisfactory in registration results, leaving patient-level registration results as the final results.

And 4, performing gap analysis on the crown root fusion model to obtain the crowding degree, the Spee curve depth and the arch protrusion degree of the teeth.

In some embodiments, performing gap analysis on the crown root fusion model to obtain tooth crowding, spee curve depth, arch prominence, may include:

calculating the mass center of each tooth based on the CBCT segmentation result, and performing polynomial fitting on a three-dimensional plane by using a least square method to obtain a dental arch curve;

Dental arch curve fitting the centroid of each tooth can be calculated based on the tooth results of CBCT segmentation, and polynomial fitting is performed on a three-dimensional plane using a least squares method to obtain a dental arch curve.

The detection of the anatomical feature points (also called as tooth key points) of the teeth can be converted into the detection of the geodesic distance field of the surface of the crown model obtained by IOS segmentation, and the anatomical feature point information is obtained through the detection of the peak value of the distance field. Considering that tooth anatomy feature point detection relies on the local and global geometry of the tooth, multi-scale mechanisms can be utilized to learn potential feature representations, extracting layered potential features by setting different scale radii on the input point cloud and aggregating them. In addition, an enhancement module may be used to link the spatial coordinates, the normals of the input points, and the extracted potential features to further improve the prediction results.

After the fitted dental arch curve is obtained, the crowding degree of the teeth can be judged by comparing the length of the dental arch curve and the sum of all the tooth widths. If the sum of all tooth widths is greater than the tooth bow curve length, the situation that the tooth arrangement is crowded or the tooth arrangement is uneven is indicated, otherwise, the tooth arrangement is sparse.

By anatomical feature point detection, the Spee curve can be formed by connecting all posterior (molar) distal cusps of the upper and lower jaws. At the same time, the ideal plane of the dental arch curve can be fitted. The Spee curve depth, i.e. the perpendicular distance between the Spee curve and the dental arch curve plane, may be calculated from the Spee curve and the dental arch curve plane. The resulting Spee curve depth can then be further evaluated and adjusted to obtain information of proper occlusion and tooth alignment.

The distance of the maxillary anterior teeth protruding from the mandibular anterior teeth can be calculated in three-dimensional space using the results of the anatomical feature point detection, thereby calculating the degree of arch, i.e., the horizontal protrusion of the maxillary anterior teeth relative to the mandibular anterior teeth.

And 5, predicting the target arrangement position of the teeth after orthodontic treatment according to the obtained tooth crowding degree, spee curve depth and dental arch prominence degree and combining with an Anderlu six-standard tooth arrangement rule.

In some embodiments, it may comprise:

Target bit prediction may be performed based on the gap analysis results and Andrews tooth alignment rules (also known as Andrews six standard tooth alignment rules). Specifically, whether an arch expansion operation is needed or not and the degree of the expansion can be determined according to the tooth crowding degree result; then determining the overall arrangement of tooth movement according to the Spee curve depth analysis result and the arch crown analysis result to obtain a rough tooth arrangement result; and finally, performing secondary adjustment on the rough result according to the factors such as the plane relation, the median relation, the plane relation and the like of the Andrews tooth arrangement rule to obtain a final target arrangement position.

The invention combines the advantages of two modal data of the CBCT image and the three-dimensional digital IOS image, and utilizes the artificial intelligence algorithm based on the deep learning to segment the CBCT image and the IOS image and fuse the multi-modal data, compared with the tooth arrangement test of the traditional gypsum model, the invention avoids a great deal of time and labor consumption of manual operation; the three-dimensional information of the tooth root and the alveolar bone is increased compared with the pure digital model tooth arrangement, the precision of the crown surface information is also improved compared with the pure CBCT image segmentation data, and the three-dimensional crown root fusion model with high-precision crown information and complete tooth root information is constructed. The invention also limits the boundary information of the alveolar bone, and reduces defects such as bone fenestration, bone cracking and the like caused by improper movement of orthodontic teeth.

Application instance

The technical solution and the operation method of the present invention will be described below with reference to a certain application example.

A patient is treated with a hospital visit, complaint of malteeth, and mouth, and figure 4 shows patient image data collected in accordance with an exemplary embodiment of the present invention. According to the present invention, based on the collected clinical data, the CBCT image is automatically segmented based on the artificial intelligence algorithm of the deep learning first, specifically, the foreground regions of the upper jaw and the lower jaw are located by using the ROI generating network first, then the complete morphology of the individual teeth is precisely depicted by using the centroid and skeleton information in the tooth layering morphology components by using a specific two-stage depth network (hierarchical morphology guidance network and multi-task tooth segmentation network), and fig. 5 shows a CBCT image segmentation schematic according to an exemplary embodiment of the present invention.

The present invention also performs automatic dental segmentation on the IOS image based on the two-stage neural network of end-to-end learning, and further improves the segmentation accuracy near the complex tooth shape boundary by using the cascade segmentation network scheme and the new confidence-aware tooth segmentation attention mechanism, and fig. 6 shows a schematic diagram of the IOS segmentation result according to an exemplary embodiment of the present invention.

The discrete voxel-based CBCT segmentation results may then be converted to a continuous 3D mesh representation using a marching cubes algorithm, bridging format gaps, and acquiring a unified mesh model containing CBCT image structure information, followed by multi-modal fusion by patient-level coarse-to-fine registration and tooth-level registration, fig. 7 illustrates a schematic diagram of a coronal root fusion model according to an exemplary embodiment of the present invention.

According to the head shadow measurement result, the distance between the middle incisor and the AP is 5.2mm, and the gap amount required for correcting the incisor process is 0.6mm after subtracting the normal value of 4.9mm from the lower middle incisor and the AP and multiplying the normal value by 2 times. Then, according to the automatic detection of the near-far contact points of the tooth surfaces, calculating the sum of the near-far widths of all teeth (from the middle incisors to the first molars) in the dental arch, wherein the upper jaw is 77.14mm, and the lower jaw is 66.26mm; the arch curve was fitted based on the average arch of the particular population in combination with the anterior cuspids and molar interosseous widths, and the existing length of the arch was determined to be 74.08mm for the upper jaw and 61.70mm for the lower jaw, so that the required gap for the upper jaw was 3.06mm and 4.56mm for the lower jaw. Then, the Spee curve was fitted by keypoint detection, and the depth average of the double-sided Spee curve was calculated to be 2.17mm, and the amount of clearance required to correct the Spee curve was calculated to be 2.17mm.

Combining the above information, the orthodontic treatment scheme is designed to extract the upper four and the lower five, determine the overall arrangement of tooth movement according to the target position prediction of the upper and the lower incisors, obtain a rough tooth arrangement result, and then perform secondary adjustment on the rough result according to the factors such as the plane relation, the median relation, the plane relation and the like of the Andrews tooth arrangement rule, so as to obtain the target arrangement position of the teeth after orthodontic treatment, as shown in fig. 8.

It will be appreciated that the foregoing embodiments and implementations of the present disclosure may be combined with each other to form combined embodiments and implementations without departing from the principle logic, and are not repeated herein for the sake of brevity. It will be appreciated by those skilled in the art that in the above-described methods of the embodiments, the particular order of execution of the steps should be determined by their function and possible inherent logic.

Note that all features disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic set of equivalent or similar features. Where used, further, preferably, still further and preferably, the brief description of the other embodiment is provided on the basis of the foregoing embodiment, and further, preferably, further or more preferably, the combination of the contents of the rear band with the foregoing embodiment is provided as a complete construct of the other embodiment. A further embodiment is composed of several further, preferably, still further or preferably arrangements of the strips after the same embodiment, which may be combined arbitrarily.

It will be appreciated by persons skilled in the art that the embodiments of the invention described above and shown in the drawings are by way of example only and are not limiting. The objects of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been shown and described in the examples and embodiments of the invention may be modified or practiced without departing from the principles described.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present disclosure, and not for limiting the same; although the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method of orthodontic tooth alignment based on crown root fusion, the method comprising:

2. The method of claim 1, wherein image segmentation of Cone Beam Computed Tomography (CBCT) images of teeth to obtain CBCT segmentation results for each tooth comprises:

3. The method according to claim 2, characterized in that:

the hierarchical morphology guidance network comprises a centroid extraction network and a skeleton extraction network, wherein the centroid extraction network and the skeleton extraction network both adopt a V-Net network architecture with two output branches, and the hierarchical morphology guidance network comprises a centroid extraction network and a skeleton extraction network, wherein:

4. The method according to claim 2, wherein the method further comprises:

5. The method of claim 1, wherein image segmentation of the optically scanned image of the tooth to obtain IOS segmentation results for each tooth comprises:

6. The method of claim 5, wherein the first-stage neural network employs a PointNet++ network architecture, and the second-stage neural network employs two cascaded subnetworks based on the PointNet++ network architecture, and introduces a confidence mechanism in both subnetworks.

7. The method of claim 1, wherein performing format matching and model registration on the CBCT segmentation result and the IOS segmentation result to obtain a coronal root fusion model comprises:

8. The method of claim 1, wherein performing a gap analysis on the crown root fusion model to obtain a crowding degree of teeth, a Spee curve depth, an arching degree of teeth, comprises:

9. The method of claim 8, wherein deriving a crowding degree of teeth, a Spee curve depth, a arching degree of teeth based on the derived archwire and anatomical feature points comprises:

10. The method of claim 1, wherein predicting the target arrangement position of the orthodontic teeth based on the resulting crowding of teeth, spee curve depth, arch crown, combined with the anderua six standard tooth arrangement rule, comprises: