KR20240105113A - Apparatus for segmenting teeth from three-dimensionally scanned teeth data using deep learning and method therefor - Google Patents

Abstract

The method for segmenting teeth includes the steps of a data processing unit scanning a plurality of teeth in three dimensions and receiving scan data including a plurality of cells, and the segmentation unit weighting the scan data according to the weights learned for the scan data through a segmentation model. It includes performing an operation to derive a prediction vector indicating a probability that each of the plurality of cells belongs to a learning class, and dividing the teeth on the scan data by the division unit according to the probability of the prediction vector.

Description

Apparatus for segmenting teeth from three-dimensionally scanned teeth data using deep learning and method therefor}

The present invention relates to a technology for segmenting teeth, and more specifically, to a device and method for segmenting teeth from three-dimensional teeth scanned data using deep learning.

Technology for segmenting teeth from scan data is required for accurate diagnosis, correction, and implant design in dentistry. Splitting teeth manually requires a high level of skill and time from the medical staff. Therefore, automated tooth segmentation technology is required.

Korean Patent Publication No. 2022-0069655 (published on May 27, 2022)

The purpose of the present invention is to provide a device and method for segmenting teeth from 3D teeth scanned data using deep learning.

A method for segmenting teeth according to a preferred embodiment of the present invention to achieve the above-described object includes the steps of: a data processing unit scanning a plurality of teeth in three dimensions and receiving scan data including a plurality of cells; A division unit performing a weight calculation according to a weight learned for the scan data through a division model to derive a prediction vector indicating a probability that each of the plurality of cells belongs to a learning class, and the division unit calculating the probability of the prediction vector. It includes dividing teeth on the scan data according to.

The method further includes the step of dividing pairs of teeth belonging to the same class according to the extended class by arranging the dividing portions left and right symmetrically except for the central incisors that are adjacent to each other in the learning class.

The method further includes the step of down-sampling the scan data by the data processing unit before deriving the prediction vector. In addition, the method further includes a step of up-sampling the scan data to the original size by the restoration unit after the step of classifying according to the extension class.

The up-sampling step is performed by the restoration unit using the equation

It is characterized by deriving the class of each cell of the up-sampled scan data according to .

Here, f is a KNN (k-Nearest Neighbor) model, x, y, and z are cell coordinates of up-sampled scan data, i is a cell index of up-sampled scan data, and N is It is the number of cells of scanned data of the original size, and C is an extended class.

The up-sampling step is performed by the restoration unit using the equation

It is characterized by deriving the class of each cell of the up-sampled scan data according to .

Here, g is a KNN model where k is M, h is a KNN model where k is N, M and N are preset constants, M<N, and x, y, and z are It is a cell coordinate of sampled scan data, i is a cell index of up-sampled scan data, and C is an extended class.

The method further includes a step where, when the number of cells of a specific extended class is less than a preset threshold, the restoration unit converts the cells belonging to the extended class to the background class and erases them.

The method includes, after receiving the scan data and before deriving the prediction vector, the data processing unit reducing the three-dimensional coordinates of the scan data to two dimensions through principal component analysis, and the data processing unit reducing the scan data to two dimensions. calculating the length of the minor axis and the length of the major axis, and the data processing unit uses the learned machine learning model to calculate the scan data from the length of the minor axis and the ratio of the length of the minor axis and the length of the major axis to total scan data. It further includes a step of checking whether the scan data is scan data or partial scan data, and the data processing unit deleting the scan data if the scan data is partial scan data.

The method includes, after receiving the scan data and before deriving the prediction vector, the data processing unit deriving an eigenvector through principal component analysis on the xy plane of the scan data, and the data processing unit deriving the eigenvector. selecting the smaller angle between the angle formed by the eigenvector and a baseline passing horizontally through the origin of the vector as the rotation angle, and the data processing unit rotating the scan data with the origin as the rotation axis so that the rotation angle is 0, It further includes a reorienting step.

The method includes, before receiving the scan data, the data processing unit preparing training data including scan data for training and a label indicating a learning class corresponding to each of a plurality of cells of the scan data for training, and a segmentation model. A step of performing a weighting operation in which weights for which training has not been completed are applied to the training scan data to derive a training prediction vector indicating the probability that a plurality of cells in the training scan data belong to a learning class, and the learning unit performs segmentation loss and classification loss. It further includes a step of calculating a total loss including, and a step of the learning unit performing optimization to modify the weights of the segmentation model so that the total loss is minimized.

The step of calculating the total loss is performed by the learning unit using the equation

It is characterized in that the division loss is calculated according to.

Here, L is the number of learning classes, l is the learning class index, N is the number of cells, i is the cell index, p is the prediction vector for training, y is the label, and ε is It is characterized by being a preset constant.

The step of calculating the total loss is performed by the learning unit using the equation

It is characterized by calculating classification loss according to .

은 데이터 단위의 확률이고, Here, the above

is the probability of the data unit,

은 수학식 remind

is the math equation

It is characterized by being derived according to .

은 학습클래스 인덱스에 대응하는 셀 개수인 것을 특징으로 한다. Here, L is the number of learning classes, l is the index of the learning class, y is the label, N is the number of cells in the scan data, and

is characterized by being the number of cells corresponding to the learning class index.

A device for segmenting teeth according to a preferred embodiment of the present invention for achieving the above-described object includes a data processing unit that scans a plurality of teeth in three dimensions and receives scan data including a plurality of cells, and a segmentation model. By performing a weight operation according to the learned weight on the scan data, a prediction vector representing the probability that each of the plurality of cells belongs to the learning class is derived, and the teeth are divided on the scan data according to the probability of the prediction vector. Includes a division part.

The dividing portion is arranged symmetrically left and right, excluding the central incisors that are adjacent to each other in the learning class, and is characterized in that it distinguishes pairs of teeth belonging to the same class according to the extended class.

The data processing unit down-samples the scan data before deriving the prediction vector, and the restoration unit up-samples the scan data to the original size after classifying it according to the extension class.

The restoration unit uses the equation

It is characterized by deriving the class of each cell of the up-sampled scan data according to .

Here, f is a KNN (k-Nearest Neighbor) model, x, y, and z are cell coordinates of up-sampled scan data, i is a cell index of up-sampled scan data, and N is It is the number of cells of scanned data of the original size, and C is an extended class.

The restoration unit uses the equation

It is characterized by deriving the class of each cell of the up-sampled scan data according to .

The g is a KNN model where k is M, the h is a KNN model where k is N, the M and the N are preset constants, M<N, and the x, the y, and the z are up-sampled It is a cell coordinate of scan data, i is a cell index of up-sampled scan data, and C is an extended class.

If the number of cells of a specific extended class is less than a preset threshold, the restoration unit converts the cells belonging to the extended class to the background class and erases them.

The data processing unit reduces the three-dimensional coordinates of the scan data to two dimensions through principal component analysis, calculates the length of the minor axis and the major axis of the scan data, and calculates the length of the minor axis and the minor axis using the learned machine learning model. It is characterized in that it is determined whether the scan data is full scan data or partial scan data from the ratio of the length of and the length of the long axis, and if the scan data is partial scan data, the scan data is erased.

The data processing unit derives an eigenvector through principal component analysis of the xy plane of the scan data, selects the smaller angle between the angle formed by the eigenvector and a reference line horizontally passing through the origin of the eigenvector as the rotation angle, and rotates the eigenvector. It is characterized by reorienting the scan data by rotating the scan data around the origin so that the angle becomes 0.

The data processing unit is characterized in that it prepares training data including training scan data and a label indicating a learning class corresponding to each of a plurality of cells of the training scan data. At this time, if the segmentation model performs a weighting operation in which a weight that has not yet been learned is applied to the training scan data to derive a learning prediction vector indicating the probability that a plurality of cells of the training scan data belong to the learning class, the division is performed. It further includes a learning unit that calculates a total loss including loss and classification loss, and performs optimization to modify the weights of the segmentation model so that the total loss is minimized.

The learning unit uses mathematical formulas

It is characterized in that the division loss is calculated according to.

Here, L is the number of learning classes, l is the learning class index, N is the number of cells, i is the cell index, p is the prediction vector for training, y is the label, and ε is It is characterized by being a preset constant.

The learning unit uses mathematical formulas

It is characterized by calculating classification loss according to .

은 데이터 단위의 확률이고, Here, the above

is the probability of the data unit,

은 수학식 remind

is the math equation

It is characterized by being derived according to .

은 학습클래스 인덱스에 대응하는 셀 개수인 것을 특징으로 한다. Here, L is the number of learning classes, l is the index of the learning class, y is the label, N is the number of cells in the scan data, and

is characterized by being the number of cells corresponding to the learning class index.

The present invention can efficiently learn a segmentation model (SM) by using a learning class that assigns mutually symmetrical teeth to the same class. Furthermore, the present invention performs down-sampling and up-sampling using a KNN model, and eliminates noise through a threshold, thereby reducing computing load and improving tooth segmentation performance. According to the present invention, the workload of medical staff can be significantly reduced through automated scan data filtering and tooth segmentation. The present invention is versatile and can be used in a variety of ways because it can obtain arch lines using only scan data. Moreover, the present invention can obtain the coordinates of not only the arch lines but also all existing teeth and missing teeth using scan data. In addition, the present invention can obtain crown angles of existing teeth and missing teeth using scan data.

1 is a diagram for explaining the configuration of a device for processing tooth scan data according to an embodiment of the present invention.
Figures 2 and 3 are diagrams for explaining the configuration of a segmentation model according to an embodiment of the present invention.
Figure 4 is a flowchart illustrating a method of learning a segmentation model (SM) according to an embodiment of the present invention.
Figures 5 and 6 are diagrams for explaining a learning class for learning a segmentation model according to an embodiment of the present invention.
Figure 7 is a flowchart illustrating a method for segmenting teeth from 3D teeth scanned data using deep learning according to an embodiment of the present invention.
8 to 15 are example screens for explaining a method for segmenting teeth from 3D teeth scanned data using deep learning according to an embodiment of the present invention.
Figure 16 is a flowchart illustrating a method for placing a dental crown for a missing tooth in three-dimensional tooth scan data according to an embodiment of the present invention.
Figures 17 to 27 are screen examples for explaining a method for placing a dental crown for a missing tooth in 3D tooth scanned data according to an embodiment of the present invention.
Figure 28 is a diagram showing a computing device according to an embodiment of the present invention.

Prior to the detailed description of the present invention, the embodiments described in this specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention and do not represent the entire technical idea of the present invention, so at the time of filing this application, It should be understood that there may be various equivalents and variations that may replace them.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. At this time, it should be noted that in the attached drawings, identical components are indicated by identical symbols whenever possible. Additionally, detailed descriptions of well-known functions and configurations that may obscure the gist of the present invention will be omitted. For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or schematically shown, and the size of each component does not entirely reflect the actual size.

In particular, terms or words used in the specification and claims described below should not be construed as limited to their usual or dictionary meanings, and the inventor should use the concept of terms to explain his or her invention in the best way. It must be interpreted as meaning and concept consistent with the technical idea of the present invention based on the principle that it can be appropriately defined.

First, a device for processing tooth scan data according to an embodiment of the present invention will be described. 1 is a diagram for explaining the configuration of a device for processing tooth scan data according to an embodiment of the present invention. Referring to Figure 1, the dental simulation device 10 according to an embodiment of the present invention includes a data processing unit 100, a learning unit 200, a dividing unit 300, a restoration unit 400, a reference processing unit 500, It includes a missing tooth processing unit 600 and a placement unit 700.

The data processing unit 100 is used to receive scan data obtained by scanning teeth in three dimensions from a scanning device (not shown) and to perform preprocessing on the received scan data. This preprocessing includes filtering partial scan data, downsampling, or reorienting scan data. Additionally, the data processing unit 100 may generate learning data based on scan data.

The learning unit 200 is for learning a segmentation model (SM), which is a deep learning model based on an artificial neural network. The learning unit 200 provides the learned segmentation model (SM) to the segmentation unit 300. The configuration of this learning method and segmentation model (SM) will be explained in more detail below.

The segmentation unit 300 analyzes the scan data using the learned segmentation model (SM) and divides the teeth on the scan data according to the learning class. In an embodiment of the present invention, the term 'segmentation' will be used to mean specifying a target area by calculating the weight of a segmentation model (SM). For example, dividing teeth is used to mean distinguishing teeth from other areas. Anyone with ordinary knowledge in this technical field will be able to understand the meaning of 'segmentation' performed by the calculation of an artificial neural network.

The restoration unit 400 is for post-processing scan data including segmented teeth. The restoration unit 400 may up-sample the scanned data to the original size or remove noise.

Meanwhile, the reference processing unit 500, the missing tooth processing unit 600, and the placement unit 700 detect missing teeth on the scan data into which the teeth are segmented, and place the tooth crown corresponding to the detected missing tooth at the position of the detected missing tooth. It is for this purpose. To this end, the reference processing unit 500 derives existing tooth information, which is information about existing teeth, from scan data, and the missing tooth processing unit 600 derives missing tooth information, which is information about missing teeth, based on the existing tooth information. . Then, the placement unit 700 places a tooth crown corresponding to the missing tooth based on the missing tooth information.

Tooth simulation device (10) including a data processing unit (100), a learning unit (200), a dividing unit (300), a restoration unit (400), a reference processing unit (500), a missing tooth processing unit (600), and a placement unit (700). )'s specific operation will be explained in more detail below.

Next, the configuration of a segmentation model according to an embodiment of the present invention will be described. Figures 2 and 3 are diagrams for explaining the configuration of a segmentation model according to an embodiment of the present invention.

Referring to FIG. 2, a segmentation model (SM) according to an embodiment of the present invention includes a plurality of layers or modules, the plurality of layers or modules are connected by weights, and each of the plurality of layers or modules has a plurality of layers or modules. Includes the computation of In other words, the segmentation model (SM) performs a plurality of operations to which weights between multiple layers are applied, and this will be referred to as a weight operation. This segmentation model (SM) is based on Dynamic Graph (Convolutional Neural Networks) CNN (DGCNN).

The segmentation model (SM) includes a spatial transform module (ST), a plurality of edge convolution modules (EC), each including an attention module (SE: squeeze and excitations), and a plurality of multi-layer perceptron modules (MLP: multi-layer perceptron). In Figure 2, n is the number of cells (eg, n=15,000), k is the size of the edge feature set, and c is the number of learning classes (eg, c=10). an is the number of neurons.

The spatial conversion module (ST) receives scan data including a plurality of cells. Each cell of scan data may include cell coordinates (3-dimensional coordinates) and optionally color information of the cell. When scan data is input, the spatial transformation module (ST) performs canonical transformation by applying a transformation matrix (for example, a 3 × 3 matrix) to align it in a canonical space. Specifically, the spatial transformation module (ST) detects a tensor (knn) connecting the coordinate differences between each cell of the scan data and the k cells neighboring each cell, and the matrix components for the detected tensor are learned weights. By performing canonical transformation through matrix multiplication for a transformation matrix (e.g., a 3 × 3 matrix), a feature map in which the scan data is aligned in a canonical space is generated.

)를 생성한다. The edge convolution module (EC) performs an edge convolution operation on the input feature map to generate a feature map with feature values corresponding to each cell. For this purpose, the edge convolution module (EC) detects, for each cell of the feature map, a tensor connecting the coordinate differences between k cells neighboring each cell in the feature space (Knn graph), and n for the detected tensor. Edge features are derived by performing a weighting operation using a multi-layer perceptron module (MLP) with nodes, and performing a weighting operation by an attention module (SE), then a feature map (e.g., max pooling) is performed through a pooling operation (e.g., max pooling). n ×

) is created.

Subsequently, the feature map according to the results of the plurality of edge convolution calculations is concatenated with the feature map according to the final edge convolution calculation result, and weight calculation by a plurality of multi-layer perceptron modules (MLP) is performed again to produce a final feature map ( Derive n × c). Each component of the finally output feature map corresponds to each cell of the scan data and represents the probability that each cell belongs to each of a plurality of learning classes.

Meanwhile, with reference to FIG. 3, the attention module (SE) will be described as follows. The attention module (SE) divides the input feature map into a plurality of channels and calculates attention for each of the plurality of channels according to the inter-dependency between the channels. And apply the calculated attention to the corresponding channel. For this purpose, the attention module (SE) includes a converted input layer (CI), a compression layer (SQ), an active layer (EOC), and an attention output layer (CO).

의 규격을 가지면, 변환입력층(CI)은 이를 H×W×C의 규격을 가지는 특징 지도(feature map)로 변환할 수 있다.The conversion input layer (CI) converts the input feature map into a feature map having a plurality of channels (C) with a predetermined height (H) and width (W). For example, the input feature map is n×k×

If it has the specifications, the conversion input layer (CI) can convert it into a feature map with the specifications of H×W×C.

The compression layer (SQ) generates a channel descriptor, which is a scalar value representing each of the plurality of channels (C) of the feature map. At this time, the compression layer (SQ) can generate a channel descriptor by generating a feature map with specifications of 1 × 1 × C through pooling of a feature map with specifications of H × W × C. This channel descriptor can be said to be a vector value in which all information about the corresponding channel (C) is embedded.

The active layer (EOC) calculates the attention of each of the plurality of channels through a plurality of operations on the channel descriptor of each of the plurality of channels. To this end, the active layer (EOC) includes a first fully connected layer (FC1: Fully-connected layer 1) and a second fully connected layer (FC2: Fully-connected layer 2).

The first fully connected layer (FC1) linearly converts the channel descriptor to the standard of 1 × 1 × (C/r) (FC: Fully-connected) and performs a rectification linear (ReLU) operation for each of the plurality of channels. do. Here, r is a hyperparameter to reduce the amount of computation. That is, r is a preset number.

The second fully connected layer (FC2) linearly converts the operation result of the first fully connected layer (FC1) on the channel descriptor back to the standard of 1 × 1 × C (FC: Fully-connected), and Correspondingly, a sigmoid operation is performed. These calculation results represent attention for each channel. In this way, the active layer (EOC) can calculate the attention of each of the plurality of channels (C) through a plurality of operations on the channel descriptor of each of the plurality of channels (C).

The attention output layer (CO) applies the attention calculated for each channel to each of a plurality of channels and outputs the attention by applying the attention to each of the plurality of channels (C) of the feature map input from the conversion input layer (CI). That is, the attention output layer (CO) receives a feature map with a plurality of channels from the conversion input layer (CI), receives attention for each channel from the second fully connected layer (FC2), and receives features with a plurality of input channels. Apply the calculated attention to the corresponding channel on the map.

Next, a method for learning a segmentation model (SM) according to an embodiment of the present invention will be described. Figure 4 is a flowchart illustrating a method of learning a segmentation model (SM) according to an embodiment of the present invention. Figures 5 and 6 are diagrams for explaining a learning class for learning a segmentation model according to an embodiment of the present invention.

, l은 학습클래스 인덱스). 학습클래스는 잇몸 및 복수의 치아 각각을 구분하기 위한 것이다. Referring to FIG. 4, the data processing unit 100 prepares learning data in step S110. The training data includes training scan data and a label indicating a learning class corresponding to each of a plurality of cells in the training scan data. The data processing unit 100 may down-sample scan data to increase network operation speed. For example, the data processing unit 100 may down-sample the original scan data into training scan data having 15,000 cells. Data augmentation can be performed to increase the amount of learning data. Data augmentation can use random scaling, random rotation, random translation, random horizontal flipping, etc. The training scan data is a three-dimensional scan of a portion including a plurality of teeth and at least a portion of the gums connected to the teeth. Cells of scan data have three-dimensional coordinates (x, y, z). Scan data is a 3D scan of a person's teeth and their surroundings. Therefore, in an embodiment of the present invention, a cell may be a voxel in a grid unit, a point in a point cloud, or a triangle in STL (Stereo Lithography) format. Accordingly, the coordinates of a cell in scan data should be understood as the three-dimensional coordinates of the center (or center of gravity) of the cell. The label indicates whether each cell belongs to each learning class (

, l is the learning class index). The learning class is for distinguishing between the gums and multiple teeth.

The learning class of the present invention sets the upper and lower teeth, which are arranged symmetrically up and down across all teeth, to the same class, and sets the teeth arranged symmetrically left and right, excluding the central incisors that are adjacent to each other, to the same class. These learning classes are explained in more detail as follows. According to one embodiment, in order to reduce the number of learning classes in the segmentation model, 32 teeth according to FDI notation are reduced to 9 teeth and the learning classes are classified into 10 classes (background (gum) 1, tooth 9). . First, as shown in Figure 5 (A), the 32 types of teeth are reduced to 16 types by eliminating the distinction between the upper and lower jaws, as shown in Figure 5 (B). That is, the upper and lower teeth, which are arranged symmetrically up and down, are set to the same class. For example, tooth No. 17 of the upper jaw and tooth No. 47 of the lower jaw in Figure 5 (A) are set as tooth No. 2, as shown in Figure 5 (B). As another example, tooth No. 27 of the upper jaw and tooth No. 37 of the lower jaw in FIG. 5 (A) are set as tooth No. 15, as shown in (B) of FIG. 5. Second, as shown in Figure 5 (C), teeth arranged symmetrically on the left and right, excluding central incisors that are adjacent to each other, are set to the same class. Accordingly, as shown in Figure 5 (C), pairs of 1st to 3rd molars, 1st to 2nd premolars, canines, and lateral incisors are classified as teeth 1 to 7 in that order, and the central incisors adjacent to each other are classified as teeth 8 ( They are classified as teeth 11 and 41 according to FDI) and teeth 9 (tooth 21 and 32 according to FDI). Accordingly, as shown in Figure 6, the learning class of the present invention consists of a total of 10 classes, including a background class representing gums and a class representing 9 types of teeth. Here, the background class is set to 0.

The learning unit 200 inputs training scan data into the segmentation model (SM) for which learning has not been completed in step S120.

Then, the segmentation model (SM) performs a weighting operation in which unlearned weights are applied to the training scan data in step S130 to derive a learning feature map including a learning prediction vector for each of a plurality of cells. The prediction vector for learning represents the probability that each cell belongs to each of a plurality of learning classes. As described above, the learning class includes the background (gum) and teeth 1 to 9, and the prediction vector for learning will be a score indicating the probability that the corresponding cell belongs to the background (gum) and teeth 1 to 9, respectively. You can. The weight calculation of the segmentation model (SM) is the same as previously described with reference to FIGS. 2 and 3.

Then, the learning unit 200 calculates the total loss including the segmentation loss and classification loss in step S140.

Segmentation loss is derived through the Dice loss function according to Equation 1 below.

. ε은 기 설정된 상수(constant)이다. 여기서, p와 y의 크기는 N×L이다. Here, L is the number of learning classes, l is the learning class index, N is the number of cells, i is the cell index, p is the prediction vector for learning, y is the label (ground truth label),

. ε is a preset constant. Here, the sizes of p and y are N×L.

Additionally, the classification loss is derived through the MLS loss (multi-label-softmargin-loss) function according to Equation 2 below.

.

은 데이터 단위의 확률(data sample-wise probability)을 나타내며, 0 혹은 1의 값을 가진다(

). 이러한

은 다음의 수학식 3과 같이 정의된다. Here, L is the number of learning classes, l is the index of the learning class, and y is the label.

.

represents the probability of a data unit (data sample-wise probability) and has a value of 0 or 1 (

). Such

is defined as in Equation 3 below.

은 레이블(학습클래스) 인덱스(l)에 대응하는 셀 개수(

<N)를 의미한다. 임계치(threshold) 값에 따라서

의 값이 변동된다. 예를 들면, 임계치가 0인 경우, 하나의 학습용 특징지도에서 특정 학습클래스로 도출되는 한 개의 셀이라도 존재한다면,

은 1의 값을 가진다. 다른 예로, 임계치가 50인 경우, 특정 학습클래스로 도출된 셀의 개수가 51 이상이면,

은 1의 값을 가진다. 반대로, 임계치가 50일 때, 특정 학습클래스로 도출된 셀의 개수가 50 이하이면,

은 0의 값을 가진다. Here, N is the number of cells in the scan data, l is the label (learning class) index,

is the number of cells corresponding to the label (learning class) index (l) (

means <N). Depending on the threshold value

The value of changes. For example, when the threshold is 0, if there is even one cell derived from a specific learning class in one learning feature map,

has a value of 1. As another example, if the threshold is 50 and the number of cells derived from a specific learning class is 51 or more,

has a value of 1. Conversely, when the threshold is 50 and the number of cells derived from a specific learning class is 50 or less,

has the value of 0.

Conventional technologies utilized cross entropy loss for cell-wise classification terms. However, in clinical practice, determination of the presence or absence of missing teeth is required for data samples rather than cell units. Therefore, the present invention uses an MLS loss function that operates on a data sample-wise basis.

Additionally, the total loss is derived according to Equation 4 below.

Here, λ is a constant to balance the classification loss and is a preset value.

As described above, when the total loss is calculated, the learning unit 200 performs optimization to modify the weights of the segmentation model (SM) so that the total loss, including the segmentation loss and classification loss, is minimized in step S150.

Steps S120 to S150 described above are repeatedly performed using different learning data until a predetermined learning completion condition is satisfied. These learning completion conditions are when the overall loss is less than the preset threshold or converges, when the segmentation loss and classification loss are each less than the preset threshold and the overall loss converges, and when the number of iterations of steps S120 to S150 reaches the target value. , etc. can be exemplified. Accordingly, the learning unit 200 determines the learning completion condition in step S160, and if the learning completion condition is satisfied, it completes learning in step S170. When learning is completed, the segmentation model (SM) is provided to the segmentation unit 300.

Next, a method for segmenting teeth from 3D teeth scanned data using deep learning according to an embodiment of the present invention will be described. Figure 7 is a flowchart illustrating a method for segmenting teeth from 3D teeth scanned data using deep learning according to an embodiment of the present invention. 8 to 15 are example screens for explaining a method for segmenting teeth from 3D teeth scanned data using deep learning according to an embodiment of the present invention.

Referring to FIG. 7, the data processing unit 100 receives scan data obtained by scanning human teeth in three dimensions in step S210. Here, the scan data is a three-dimensional scan of a person's teeth and their surroundings. Therefore, in an embodiment of the present invention, a cell may be a voxel in a grid unit, a point in a point cloud, or a triangle in STL (Stereo Lithography) format. Accordingly, the coordinates of a cell in scan data should be understood as the three-dimensional coordinates of the center (or center of gravity) of the cell.

In particular, the scan data may be full scan data in which an area including all of the person's teeth is scanned, or it may be partial scan data in which only a portion of the person's teeth are scanned. Accordingly, the data processing unit 100 filters the partial scan data in step S220. That is, the data processing unit 100 determines whether the scan data is full scan data or partial scan data, and erases the scan data if it is partial scan data. This S220 step is described in more detail as follows.

To filter partial scan data, a machine learning model based on unsupervised learning is used. Here, the machine learning model can be K-means with two clusters. Figure 8 (A) is an example of “full scan data” in which all teeth are photographed in 3D. And Figure 8 (B) is an example of “partial scan data” in which a part of a tooth is photographed in three dimensions. As shown, there is a clear difference in the plane, that is, the x and y dimensions, between the full scan data and the partial scan data. Therefore, the data processing unit 100 uses the x and y dimensions in the scan data, excluding the z dimension. To this end, the data processing unit 100 analyzes cells with (p, q) coordinates through principal component analysis (PCA) on a plurality of cells with (x, y, z) coordinates. Convert to That is, the data processing unit 100 reduces the scan data to two-dimensional coordinates (p, q) by excluding the z dimension from the three-dimensional coordinates (x, y, z) of the scan data through principal component analysis.

As shown in FIG. 8, the data processing unit 100 uses two-dimensional coordinates (p, q) to calculate the length of scan data on the short axis (PC1) and the length of scan data on the long axis (PC2) in Equation 5 below. It can be calculated accordingly.

및

은 각각 스캔데이터의 p 좌표의 최대값 및 최소값이고,

및

은 각각 q 좌표에서 스캔데이터의 q좌표의 최대값 및 최소값이다. Here, PC1 and PC2 represent the short axis length and long axis length of the scan data, respectively. also,

and

are the maximum and minimum values of the p coordinate of the scan data, respectively,

and

are the maximum and minimum values of the q-coordinate of the scan data at the q-coordinate, respectively.

The data processing unit 100 uses the short axis length (PC1) and the short axis length ratio (PC1/PC2) as K-means learning data. Here, dimensionless variables PC1/PC2 are used to control variables due to gender and age. The data processing unit 100 can use a learned machine learning model (eg, K-means) to check whether the scan data is full scan data or partial scan data according to Equation 6 below. Here, the machine learning model (e.g., K-means) is a model learned to classify (e.g., clustering) scan data into full scan data and partial scan data according to the length of the minor axis and the ratio of the length of the minor axis to the length of the major axis. .

Here, f represents the learned machine learning model (e.g., K-means). r is a value of 0 or 1. If the variable r is 1, it represents partial scan data, and if it is 0, it represents full scan data. Accordingly, the data processing unit 100 erases the scan data when the variable r is 1, that is, when it is partial scan data.

Next, the data processing unit 100 redirects the scan data in step S230. This maintains a constant axis direction of the scan data. Specifically, referring to FIG. 9, the data processing unit 100 generates an eigenvector through principal component analysis (PCA) on the xy plane of the scan data, as shown in (A) of FIG. 9. eigen vector, V1, V2) is derived. Then, the data processing unit 100 converts the smaller angle between the angles formed by the eigenvectors (V1, V2) and the reference line (L) horizontally passing through the origin (Z) of the eigenvectors (V1, V2) into the rotation angle (θ). Choose. Next, as shown in (B) of FIG. 9, the data processing unit 100 sets the origin (Z) so that the eigenvector (V1) forming the reference line (L) and the rotation angle (θ) coincides with the reference line (L). The scan data is reoriented by rotating the scan data around the rotation axis. In other words, the data processing unit 100 reorients the scan data by rotating the scan data around the origin (Z) as the rotation axis so that the rotation angle (θ) becomes 0. Meanwhile, in the present invention, the upper jaw scan data is reoriented based on the lower jaw scan data in order to match the directions of the upper jaw scan data and the lower jaw scan data. As shown in Figures 10 (A1) and (A2), the upper jaw scan data and the lower jaw scan data have axial directions that are symmetrical to each other with respect to the xy plane. Therefore, as shown in (B1) and (B2) of Figure 10, the scan data of the upper jaw is flipped based on the xy plane and reoriented in the same direction as the scan data of the lower jaw.

Next, the data processing unit 100 down-samples the scan data to a predetermined size in step S240. For example, you can downsample to 15,000 cells.

Then, the segmentation unit 300 performs a weighting operation according to the weights learned for the scan data through the segmentation model (SM) in step S250 to generate a feature map including prediction vectors corresponding to a plurality of cells of the scan data. Derive. Here, the prediction vector represents the probability that each cell constituting the scan data belongs to each learning class. As explained with reference to Figures 5 and 6, the learning class includes 10 classes, that is, classes 0 to 9, where classes 0 to 9 are background, third molar, second molar, first molar, and first molar. 2 Represents the premolars, first premolars, canines, lateral incisors, left central incisors and right central incisors. Accordingly, the prediction vector for one cell represents the probability that the corresponding cell belongs to each of classes 0 to 9. For example, the prediction vector for one cell is "[background (class 0), third molar (class 1), second molar (class 2), ..., right central incisor ( Class 9)]=[0.008, 0.003, 0.012, ..., 0.809]". This represents "(background, third molar, second molar, ..., right central incisor)=[1%, 0%, 1%, ..., 81%]".

Next, the segmentation unit 300 divides the teeth on the scan data according to the prediction vector in step S260. That is, the segmentation unit 300 divides the teeth on the scan data according to the probability for each learning class indicated by the prediction vector of each cell. That is, the segmentation unit 300 determines that the corresponding cell belongs to the learning class with the highest probability, and divides the corresponding tooth according to the determined class. In this way, teeth can be segmented by classifying all cells according to the probability of the prediction vector. For example, the segmentation unit 300 determines that the prediction vector of one voxel is "(background, left central incisor, lateral incisor, canine, ..., right central incisor) = [0.008, 0.041, 0.752, ..., 0.016] If ", then the voxel can be determined as "(background, left central incisor, lateral incisor, canine, ..., right central incisor) = [0, 0, 1, ..., 0]". This indicates that the lateral incisor (class 2), which is the class with the highest probability, is selected and the tooth mask of the corresponding voxel, that is, the maxillary tooth mask, constitutes the lateral incisor.

Next, the division unit 300 subdivides the teeth divided according to the learning class in step S270 according to the expansion class. That is, the dividing unit 300 is arranged symmetrically left and right, distinguishes classes of tooth pairs belonging to the same class, and expands 8 tooth classes according to the learning class to 16 tooth classes according to the expansion class. This converts it into 17 extended classes, including 16 tooth classes and 1 background class. At this time, the division unit 300 can convert 8 tooth classes according to 3 cases into 16 tooth classes according to extended classes. This S260 step is described in more detail as follows. That is, the dividing unit 300 is arranged symmetrically left and right, excluding the central incisors that are adjacent to each other in the learning class, and distinguishes pairs of teeth belonging to the same class according to the extended class.

In the first case, referring to Figure 11, when there are central incisors (class 8 and class 9: FDI notation standards 11, 21, 31, 41), the dividing unit 300 derives a reference point (P) from the central incisors, and the reference point Find the center line (L) that passes vertically through (P), distinguish the left and right sides based on the center line (L), and expand the 9 tooth classes into 16 tooth classes according to the expansion class. Here, when both the left central incisor and the right central incisor exist, the reference point (P) can select the coordinates of any one of the cells closest to each other between the left central incisor and the right central incisor on horizontal coordinates. Additionally, as the reference point (P), if only the left central incisor exists, the coordinates of the rightmost cell on the horizontal coordinates may be selected, and if only the right central incisor exists, the coordinates of the leftmost cell on the horizontal coordinates may be selected.

In the second case, referring to FIG. 12, when there is no central incisor, the segmentation unit 300 detects symmetrical teeth that are symmetrical to each other using an unsupervised clustering model. This unsupervised clustering model may exemplify DBSCAN (Density-based spatial clustering of applications with noise). Then, the dividing unit 300 derives the center line L using teeth that are symmetrical to each other. After dividing the left and right sides based on the center line (L), 9 tooth classes are expanded into 16 tooth classes according to the expansion class.

In the third case, referring to Figure 13, when there are no central incisors and no mutually symmetrical teeth, a center point representing the center of all cells is derived, a center line (L) passing vertically through the center point is obtained, and the center line (L) After dividing the left and right based on , and then dividing the left and right based on the center line (L), the 9 tooth classes are expanded into 16 tooth classes according to the expansion class.

The restoration unit 400 performs up-sampling of the scanned data to the original size in step S280. Here, the original size refers to the size before downsampling in step S240. For example, scan data that has been down-sampled to 15,000 can be up-sampled to the original size. At this time, the restoration unit 400 may predict the class of each cell of the up-sampled scan data using a k-Nearest Neighbor (KNN) model. According to this upsampling, the prediction results of the segmentation model (SM) can be supplemented.

The restoration unit 400 uses the learned KNN model to derive the class of each cell of the up-sampled scan data according to Equation 7 below.

Here, f represents the KNN model. In addition, x, y, z are the cell coordinates of the up-sampled scan data, i is the cell index of the up-sampled scan data, Z is a set of integers, N is the number of cells in the original size scan data, C represents the extended class as the output of the KNN model. Here, C is 0,1,… , is an integer of 16 (C∈{0,1,…,16}). The KNN model according to this embodiment of the present invention can be learned using the output value of the segmentation model (SM). At this time, the size of the input data (i.e., cell coordinates) is, for example, 15,000 × 3, and the size of the label indicating the extended class is 15,000 × 1.

In the KNN model, the up-sampled tooth segmentation area varies depending on the constant k value. When the k value is small (e.g., k=3), as shown in (A) of FIG. 14, the tooth division boundary is clear (a2), but incorrectly predicted areas may not be removed (a1). Additionally, when the k value is large (e.g., k=100), as shown in (B) of FIG. 14, incorrectly predicted areas can be removed (b1), but tooth boundaries may not be clear (b2). Accordingly, according to an additional embodiment of the present invention, a KNN model (g) where k is M (e.g., 3) and a KNN model (h) where k is N (e.g., 100) are generated, and the following Equation 8 Accordingly, the class of each cell of the upsampled scan data can be predicted.

Here, g represents a KNN model where k is M (constant), and h represents a KNN model where k is N (constant). Here, M<N. For example, M may be 3 and N may be 100. Additionally, x, y, and z are the cell coordinates of the up-sampled scan data, i is the cell index of the up-sampled scan data, and C represents the extended class. That is, if C = 0, it means a background class, and if C is 1 to 16, it means an extended class representing a tooth. In this way, by performing up-sampling using the KNN model, as shown in Figure 14 (C), the tooth division boundary line is clear (c2) and incorrectly predicted areas can be removed (c1).

Next, if the number of cells of a specific extended class is less than a preset threshold in step S290, the restoration unit 400 converts the cells belonging to the extended class to the background class and erases them. Specifically, for example, as shown in (A) of FIG. 15, when the number of cells of a specific class is less than the threshold, cells belonging to that class are converted to the background class according to the following equation 8 as shown in (B) of FIG. 15. and erase it.

Here, j is the index of the extended class, i is the cell index, and n is the number of cells in the extended class.

Next, a method for placing a dental crown for a missing tooth in three-dimensional tooth scan data according to an embodiment of the present invention will be described. Figure 16 is a flowchart illustrating a method for placing a dental crown for a missing tooth in three-dimensional tooth scan data according to an embodiment of the present invention. Figures 17 to 27 are screen examples for explaining a method for placing a dental crown for a missing tooth in 3D tooth scanned data according to an embodiment of the present invention.

The reference processing unit 500 receives three-dimensional scan data in step S310. An example of such scan data is shown in (A) of FIG. 17. As shown, the scan data includes a plurality of segmented teeth 17a. In the embodiment of the present invention, the plurality of divided teeth 17a will be referred to as “ready-made teeth.” Additionally, the present invention assumes a situation in which there is at least one missing tooth 17b as the opposite concept of “existing tooth.” In the case of Figure 17 (A), there are two missing teeth 17b.

When scan data is input, the reference processing unit 500 detects information about existing teeth in step S320. Information on the existing tooth includes the coordinates and rotation angle (rotation angle based on the horizontal axis) of the existing tooth.

To this end, the reference processing unit 500 first detects the coordinates (x, y, z) of the existing tooth from the coordinates (x, y, z) of a plurality of cells constituting the existing tooth in step S320. Scan data is a 3D scan of a person's teeth and their surroundings. Therefore, in an embodiment of the present invention, a cell may be a voxel in a grid unit, a point in a point cloud, or a triangle in STL (Stereo Lithography) format. Accordingly, the coordinates of a cell should be understood as the three-dimensional coordinates of the center (or center of gravity) of the cell. Additionally, in an embodiment of the present invention, the coordinates of a ready-made tooth refer to the three-dimensional coordinates of the center of the ready-made tooth. For example, as shown in FIG. 18, the reference processing unit 500 calculates the coordinates (x, y, z) of a plurality of cells (indicated by circles) constituting the tooth 18a according to the following equation 10: The coordinates (x, y, z) of the ready-made tooth 18a can be obtained.

및

은 각각 기성 치아를 구성하는 복수의 셀의 x 좌표 중 최대값과 최소값이다.

및

은 각각 기성 치아를 구성하는 복수의 셀의 y 좌표 중 최대값과 최소값이다.

및

은 각각 기성 치아를 구성하는 복수의 셀의 z 좌표 중 최대값과 최소값이다. Here, x, y, z are the coordinates of the existing teeth. also,

and

are the maximum and minimum values among the x-coordinates of a plurality of cells constituting each ready-made tooth.

and

are the maximum and minimum values among the y coordinates of the plurality of cells that make up the existing teeth, respectively.

and

are the maximum and minimum values among the z-coordinates of the plurality of cells constituting the ready-made teeth, respectively.

Next, the reference processing unit 500 detects the horizontal axis reference rotation angles (β, γ) among the rotation angles (Euler angles: α, β, γ) of the existing teeth in step S330. To this end, the reference processing unit 500 first sets the local rotation axes (x-axis, y-axis, z-axis) based on the coordinates (x, y, z) of the existing teeth. That is, with the coordinates (x, y, z) of the prepared tooth as the origin, and the coordinates (x, y, z) of the prepared tooth as the origin, three lines that are orthogonal to each other and pass horizontally and vertically are defined as the local rotation axis ( x-axis, y-axis, z-axis). Then, as shown in FIG. 18, the reference processing unit 500 first calculates an eigen vector (PC1, PC2, PC3) are derived. Then, the reference processing unit 500 detects the angle between the local rotation axis (x-axis, y-axis, z-axis) and the eigenvectors (PC1, PC2, PC3) to determine the rotation angle (Euler angle: α, β, γ) of the existing tooth. ) Find the rotation angle (β, γ) based on the horizontal axis. More specifically, the reference processing unit 500 uses the coordinates (x, y, z) of the existing tooth as the origin, and three points that pass horizontally and vertically while being orthogonal to each other with the coordinates (x, y, z) of the existing tooth being the origin. Set the line as the local rotation axis (x-axis, y-axis, z-axis, local axis). Referring to Figures 19 (A) and 20, the reference processing unit 500 derives eigenvectors (PC1, PC2) through two-dimensional principal component analysis of the x and z coordinates of a plurality of cells constituting the ready-made teeth, , the angle between the z-axis, which is the local rotation axis, and the eigenvector closest to the z-axis is derived as the rotation angle β based on the horizontal axis. In addition, referring to Figure 19 (B) and Figure 21, the reference processing unit 500 derives eigenvectors (PC1, PC2) through two-dimensional principal component analysis of the y and z coordinates of a plurality of cells constituting the tooth. And, the angle between the z-axis, which is the local rotation axis, and the eigenvector closest to the z-axis is derived as the rotation angle γ based on the horizontal axis.

As described above, after deriving the existing tooth information, that is, the coordinates and rotation angle of the existing tooth, referring to (B) and (C) of FIG. 17, the missing tooth processing unit 600 determines the existing tooth information from the existing tooth information. Missing tooth information is derived using the arch line (17c). Here, the missing value information includes the coordinates and rotation angle (17d) of the missing value.

Meanwhile, in an embodiment of the present invention, missing teeth can be classified into symmetrical missing teeth, inner missing teeth, and outer missing teeth. Symmetrical missing teeth are missing teeth that are symmetrical based on the line of symmetry that divides a plurality of existing teeth into left and right sides. Internal missing teeth are missing teeth that exist between existing teeth on the arch line, that is, on the inside. External missing teeth are missing teeth that exist outside the existing teeth on the arch line, that is, on the outside. The missing tooth processing unit 600 derives the coordinates of the missing tooth and the rotation angle relative to the horizontal axis according to the classification of the missing tooth in steps S340, S350, and S360, which will be described below. Although steps S340, S350, and S360 will be described sequentially, steps S340, S350, and S360 may be executed in a different order, or at least one step may be repeated.

First, if the missing tooth is a symmetrical missing tooth that has a symmetrical relationship with the existing missing tooth, the missing tooth processing unit 600 detects the missing value information of the symmetrical missing tooth using the symmetry line that divides the plurality of existing teeth into left and right in step S340. . For example, as shown in Figure 22, based on the FDI notation, assume that teeth 11 to 17, 21, 22, 24, and 25 are existing teeth, and teeth 23, 26, and 27 are missing teeth. As shown, human teeth are arranged along the arch line 22b, and are symmetrical left and right, so two or more pairs of ready-made teeth [(11, 21), (12, 22), (14, 24) , (15, 25)] may exist. The missing tooth processing unit 600 derives intermediate coordinates (e.g., coordinates of 1m, 2m, 3m, and 4m) using plane coordinates (x, y) among the coordinates of two or more pairs of ready-made teeth, and linear coordinates using the intermediate coordinates. Through linear regression analysis, the equation of a straight line, that is, the symmetry line 22a, is derived as shown in Equation 11 below.

Here, a is the slope and b is the y-intercept.

When the symmetry line 22a is derived, the missing value processing unit 600 uses the symmetry line 22a to specify the symmetrical missing value and detects the missing value information of the specified symmetrical missing value. That is, the missing tooth processing unit 600 specifies teeth 23, 26, and 27, which are symmetrical missing teeth, based on the coordinates (x, y) of teeth 13, 16, and 17 and the line of symmetry (22a, Equation 11). , the plane coordinates (x*, y*) of the specified symmetrical vanishing value are derived according to Equation 12 below.

x, y are the coordinates of the symmetrical missing tooth and the existing tooth that is symmetrical based on the symmetry line. x*, y* are the x and y coordinates of the symmetric vanishing value, respectively. Additionally, a is the slope of the symmetry line (22a, Equation 11), and b is the y-intercept of the symmetry line (Equation 11).

Additionally, the missing tooth processing unit 600 derives the coordinates z* and the rotation angles β* and γ* of the missing tooth from z, β, and γ of the symmetrical teeth according to Equation 13 below.

및

는 대칭 소실치와 대칭선을 기준으로 대칭되는 기성 치아의 수평축 기준 회전각이며,

및

는 대칭 소실치의 수평축 기준 회전각이다. z* is the coordinate of the symmetric missing tooth, and z is the z coordinate of the existing tooth that is symmetrical with respect to the symmetric missing tooth and the symmetry line. Also, a is the slope of the line of symmetry, and b is the y-intercept of the line of symmetry. and

and

is the rotation angle based on the horizontal axis of the symmetrical missing tooth and the existing tooth that is symmetrical with respect to the symmetry line,

and

is the rotation angle based on the horizontal axis of the symmetric missing tooth.

Meanwhile, if the inner missing tooth is located between the existing teeth on the arch line, the missing tooth processing unit 600 derives the arch line in step S350 and detects the inner missing tooth disposed at equal intervals between the existing teeth on the arch line, Derive the missing value information of the detected inner missing tooth. For example, as shown in Figure 23, based on the FDI notation, teeth 11, 12, 15, 16, 21, 22, 25, and 26 are existing teeth, and teeth 13, 14, 23, and 24 are internal missing teeth. Assume. In this case, the missing tooth processing unit 600 uses a spline interpolation technique based on the plane coordinates (x, y) of the existing teeth, that is, teeth Nos. 11, 12, 15, 16, 21, 22, 25, and 26. A spline, that is, a dental arch line 23b, is created through the process. At this time, the interpolation technique can use quadratic spline interpolation or cubic spline interpolation to give curvature like an actual arch line. The missing tooth processing unit 600 arranges missing teeth at equal intervals on the generated arch line 23b and detects missing tooth information therefrom. For example, the positions of teeth 13 and 14, which are missing teeth, are placed the same as the spacing between teeth 12 and 13, the spacing between teeth 13 and 14, and the spacing between teeth 14 and 15. This is explained in more detail as follows. First, the missing tooth processing unit 600 derives the curved length of the arch line between two adjacent existing teeth of the inner missing tooth according to the following equation 14.

는 악궁 라인의 시작점, 즉, 내측 소실치 중 첫 번째 내측 소실치의 이웃 기성 치아(예컨대, 15번 치아)의 x 좌표이다.

는 악궁 라인의 끝점, 즉, 내측 소실치 중 마지막 내측 소실치의 이웃 기성 치아(예컨대, 12번 치아)의 x 좌표이다. f는 보간(spline interpolation) 함수이다. Here, l is the curved length of the arch line between two neighboring existing teeth of the inner missing tooth.

is the starting point of the arch line, that is, the x-coordinate of the existing tooth (eg, tooth 15) next to the first inner missing tooth among the inner missing teeth.

is the end point of the arch line, that is, the x-coordinate of the existing tooth (eg, tooth 12) next to the last inner missing tooth among the inner missing teeth. f is an interpolation (spline interpolation) function.

In this way, after obtaining the curve length (l) of the arch line, the missing tooth processing unit 600 calculates the x* coordinate of the missing tooth according to the following equation 15.

는 악궁 라인의 시작점, 즉, 내측 소실치 중 첫 번째 내측 소실치의 이웃 기성 치아(예컨대, 15번 치아)의 x 좌표이다. f는 스플라인 보간(spline interpolation) 함수이다. 또한, i는 내측 소실치의 인덱스이다. 여기서, 14번 치아는 i=1이 되고, 13번 치아는 i=2가 되며,

이다. 여기서, N은 자연수(natural numbers)의 집합을 나타내며, n은 내측 소실치의 개수이고, n=2이다. Here, x* is the x coordinate of the missing value. l is the curved length of the arch line, that is, the curved length of the arch line between two neighboring existing teeth of the inner missing tooth.

is the starting point of the arch line, that is, the x-coordinate of the existing tooth (eg, tooth 15) next to the first inner missing tooth among the inner missing teeth. f is a spline interpolation function. Additionally, i is the index of the inner missing tooth. Here, tooth 14 becomes i=1, tooth 13 becomes i=2,

am. Here, N represents a set of natural numbers, n is the number of inner missing values, and n=2.

Then, the missing value processing unit 600 calculates the y* and z* coordinates according to the following equation 16 based on the missing value x* coordinates.

는 악궁 라인의 시작점, 즉, 내측 소실치 중 첫 번째 내측 소실치의 이웃 기성 치아(예컨대, 15번 치아)의 z 좌표이다.

는 악궁 라인의 끝점, 즉, 내측 소실치 중 마지막 내측 소실치의 이웃 기성 치아(예컨대, 12번 치아)의 z 좌표이다. i는 내측 소실치의 인덱스이다. Here, x*, y*, z* are the x, y, z coordinates of the missing value. f is the interpolation function,

is the starting point of the arch line, that is, the z coordinate of the existing tooth (eg, tooth 15) next to the first inner missing tooth among the inner missing teeth.

is the end point of the arch line, that is, the z-coordinate of the existing tooth (eg, tooth 12) next to the last inner missing tooth among the inner missing teeth. i is the index of the inner missing tooth.

Next, the missing value processing unit 600 calculates the rotation angles β* and γ* of the missing value according to Equation 17 below.

및

는 소실치의 회전각(로컬 회전각)이며,

,

은 글로벌 회전각이다. 이러한 글로벌 회전각이 도 24 및 도 25에 도시되었다. 글로벌 회전각은 스캔 데이터 전체의 셀 좌표에 대해 2차원 주성분(PCA) 분석을 통해 얻어질 수 있다. 2차원 주성분(PCA) 분석의 방법은 앞서 도 18 내지 도 21을 참조로 설명된 바와 같다. 즉, 소실치처리부(600)는 스캔 데이터에 포함된 전체 셀의 좌표의 평균값을 전체 스캔 데이터의 중심 좌표(x, y, z)로 도출하고, 도출된 스캔 데이터의 중심 좌표에 따라 글로벌 회전축(golbal axis, x축, y축, z축)을 설정한다. 즉, 소실치처리부(600)는 스캔 데이터의 중심 좌표(x, y, z)를 원점으로 하고, 원점인 스캔 데이터의 중심 좌표(x, y, z)를 상호 직교(orthogonal)하면서 수평 및 수직으로 지나는 3개의 선을 글로벌 회전축(x축, y축, z축)으로 설정한다. 그런 다음, 소실치처리부(600)는 스캔 데이터를 구성하는 복수의 셀의 좌표에 대한 주성분 분석(Principal Component Analysis, PCA)을 통해 고유벡터(eigen vector: PC1, PC2, PC3)를 도출한다. 이어서, 소실치처리부(600)는 글로벌 회전축(x축, y축, z축) 및 고유벡터(PC1, PC2, PC3) 간 각도를 검출하여 스캔 데이터의 글로벌 회전각 중 수평축 기준 글로벌 회전각(

,

)을 구한다. 즉, 소실치처리부(600)는 도 24와 같이, 스캔 데이터를 구성하는 복수의 셀의 x, z 좌표에 대한 2차원 주성분 분석을 통해 고유벡터(PC1, PC2)를 도출하고, z축과 z축에 가장 근접한 고유벡터와의 각을 글로벌 회전각

로 도출한다. 또한, 소실치처리부(600)는 도 25와 같이, 스캔 데이터를 구성하는 복수의 셀의 y, z 좌표에 대한 2차원 주성분 분석을 통해 고유벡터(PC1, PC2)를 도출하고, z축과 z축에 가장 근접한 고유벡터와의 각을 글로벌 회전각

로 도출한다. 이어서, 소실치처리부(600)는 수학식 17과 같이, 수평축 기준 글로벌 회전각을 내측 소실치의 수평축 기준 회전각으로 치환한다. Here, i is the index of the missing value.

and

is the rotation angle (local rotation angle) of the missing tooth,

,

is the global rotation angle. This global rotation angle is shown in Figures 24 and 25. The global rotation angle can be obtained through two-dimensional principal component (PCA) analysis of the cell coordinates of the entire scan data. The method of two-dimensional principal component (PCA) analysis is the same as previously described with reference to FIGS. 18 to 21. That is, the missing value processing unit 600 derives the average value of the coordinates of all cells included in the scan data as the center coordinates (x, y, z) of the entire scan data, and the global rotation axis ( Set golbal axis, x-axis, y-axis, z-axis). That is, the missing value processing unit 600 uses the center coordinates (x, y, z) of the scan data as the origin, and the center coordinates (x, y, z) of the scan data, which are the origin, are orthogonal to each other horizontally and vertically. Set the three lines passing through as the global rotation axes (x-axis, y-axis, z-axis). Then, the loss processing unit 600 derives eigen vectors (PC1, PC2, PC3) through principal component analysis (PCA) on the coordinates of a plurality of cells constituting the scan data. Next, the missing value processing unit 600 detects the angle between the global rotation axes (x-axis, y-axis, z-axis) and the eigenvectors (PC1, PC2, PC3) and detects the global rotation angle relative to the horizontal axis among the global rotation angles of the scan data (

,

) is obtained. That is, as shown in FIG. 24, the missing value processing unit 600 derives eigenvectors (PC1, PC2) through two-dimensional principal component analysis of the x and z coordinates of a plurality of cells constituting the scan data, and the z-axis and z-axis The global rotation angle is the angle with the eigenvector closest to the axis.

Derived as In addition, as shown in FIG. 25, the vanishing value processing unit 600 derives eigenvectors (PC1, PC2) through two-dimensional principal component analysis of the y and z coordinates of a plurality of cells constituting the scan data, and The global rotation angle is the angle with the eigenvector closest to the axis.

Derived as Next, the missing tooth processing unit 600 replaces the global rotation angle based on the horizontal axis with the rotation angle based on the horizontal axis of the inner missing tooth, as shown in Equation 17.

Meanwhile, if the missing tooth is an outer missing tooth located outside the existing missing tooth on the arch line, the missing tooth processing unit 600 derives the arch line in step S360, and determines the outer missing tooth according to the distance from the existing tooth on the derived arch line. Detection is performed and vanishing value information of the detected outer missing value is derived. For example, as shown in Figure 26, based on the FDI notation, assume that teeth 11 to 15 and 21 to 25 are existing teeth, and teeth 16, 17, 26, and 27 are external missing teeth. In this case, the missing tooth processing unit 600 derives the arch line 26a through a spline interpolation technique, and uses the preset distance between teeth on the arch line 26a to provide missing tooth information of the outer missing tooth. Derive. At this time, linear spline interpolation can be used to prevent the creation of inflection points. This is explained in more detail as follows. The missing value processing unit 600 derives the coordinates x*, y*, and z* of the outer missing value according to Equation 18 below.

이다. 여기서, N은 자연수(natural numbers)의 집합을 나타내며, n은 외측 소실치의 개수이며, n=2이다. Here, x*, y* are the x, y coordinates of the outer missing value. d represents the distance between teeth (distance between coordinates of teeth, for example, set to 10 mm). i is the index of the outer missing tooth. Here, tooth number 16 becomes i=1 and tooth number 17 becomes i=2.

am. Here, N represents a set of natural numbers, n is the number of outer missing values, and n=2.

는 악궁 라인의 시작점, 즉, 외측 소실치의 이웃 기성 치아(예컨대, 15번 치아)의 x 좌표이고, g는 보간 함수를 나타낸다. 여기서, 보간 함수는 선형 스플라인 보간(linear spline interpolation) 함수가 될 수 있다. 또한,

는 악궁 라인의 시작점, 즉, 외측 소실치의 이웃 기성 치아(예컨대, 15번 치아)의 z 좌표이다.

is the starting point of the arch line, that is, the x-coordinate of the existing tooth (eg, tooth 15) neighboring the outer missing tooth, and g represents the interpolation function. Here, the interpolation function may be a linear spline interpolation function. also,

is the starting point of the arch line, that is, the z coordinate of the existing tooth (eg, tooth 15) neighboring the outer missing tooth.

Next, the missing tooth processing unit 600 derives the rotation angles (β and γ) of the outer missing tooth relative to the horizontal axis according to Equation 19 below.

및

는 외측 소실치의 회전각(로컬 회전각)이다.

,

은 글로벌 회전각이며, 앞서, 도 24 및 도 25를 참조로 설명된 바와 같다. 즉, 소실치처리부(600)는 수학식 19와 같이, 수평축 기준 글로벌 회전각(

,

)을 외측 소실치의 수평축 기준 회전각으로 치환한다. Here, i is the index of the outer missing value.

and

is the rotation angle (local rotation angle) of the outer missing tooth.

,

is the global rotation angle, as previously described with reference to FIGS. 24 and 25. That is, the missing tooth processing unit 600 calculates the global rotation angle relative to the horizontal axis, as shown in Equation 19 (

,

) is replaced by the rotation angle based on the horizontal axis of the outer missing tooth.

를 도출한다. Next, in step S370, the missing tooth processing unit 600 generates a dental arch line through spline interpolation using the coordinates of all teeth including existing teeth and missing teeth (symmetrical, inner and outer missing teeth), and creates the dental arch line through spline interpolation. Rotation angle using the slope of the tangent on the line

Derive .

를 도출한다. Specifically, referring to FIG. 27, the missing tooth processing unit 600 derives the arch line 27a through spline interpolation, and derives the slope 27b of the tangent line on the arch line according to the following equation 20, based on the vertical axis. rotation angle

Derive .

는 회전각이고, f는 보간 함수이다. 보간 함수는 quadratic spline interpolation 또는 cubic spline interpolation 함수가 될 수 있다. here,

is the rotation angle, and f is the interpolation function. The interpolation function can be a quadratic spline interpolation or cubic spline interpolation function.

As described above, after deriving the missing tooth information, that is, the coordinates and rotation angle of the missing tooth, the placement unit 700 places the tooth crown corresponding to the missing tooth on the scan data according to the missing tooth information in step S380. Referring to FIG. 17, a tooth crown 17e is placed as shown in FIG. 17(D) in response to the missing tooth 17b in the scan data shown in FIG. 17(A).

Figure 28 is a diagram showing a computing device according to an embodiment of the present invention. Computing device TN100 of FIG. 28 may be a device described herein, such as a dental simulation device 10.

In the embodiment of FIG. 28, the computing device TN100 may include at least one processor TN110, a transceiver device TN120, and a memory TN130. Additionally, the computing device TN100 may further include a storage device TN140, an input interface device TN150, an output interface device TN160, etc. Components included in the computing device TN100 may be connected by a bus TN170 and communicate with each other.

The processor TN110 may execute a program command stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Processor TN110 may be configured to implement procedures, functions, and methods described in connection with embodiments of the present invention. The processor TN110 may control each component of the computing device TN100.

Each of the memory TN130 and the storage device TN140 can store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may be comprised of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 may be comprised of at least one of read only memory (ROM) and random access memory (RAM).

The transceiving device TN120 can transmit or receive wired signals or wireless signals. The transmitting and receiving device (TN120) can be connected to a network and perform communication.

Meanwhile, the method according to the embodiment of the present invention described above can be implemented in the form of a program readable through various computer means and recorded on a computer-readable recording medium. Here, the recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the recording medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the art of computer software. For example, recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. It includes specially configured hardware devices to store and execute program instructions, such as magneto-optical media and ROM, RAM, and flash memory. Examples of program instructions may include machine language wires, such as those produced by a compiler, as well as high-level language wires that can be executed by a computer using an interpreter, etc. These hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the present invention has been described above using several preferred examples, these examples are illustrative and not limiting. As such, those of ordinary skill in the technical field to which the present invention pertains will understand that various changes and modifications can be made according to the theory of equivalents without departing from the spirit of the present invention and the scope of rights set forth in the appended claims.

10: Tooth simulation device
100: Data processing unit
200: Learning Department
300: division part
400: restoration unit
500: Standard processing unit
600: Missing tooth processing unit
700: Placement unit

Claims (22)

In the method for dividing teeth,
A data processing unit scanning a plurality of teeth in three dimensions and receiving scan data including a plurality of cells;
A division unit performing a weight calculation according to a weight learned for the scan data through a division model to derive a prediction vector indicating a probability that each of the plurality of cells belongs to a learning class; and
The segmentation unit dividing teeth on scan data according to the probability of the prediction vector;
Characterized by including
Method for dividing teeth. According to paragraph 1,
A step of dividing pairs of teeth belonging to the same class according to the extended class by arranging the dividing portion symmetrically left and right except for the central incisors that are adjacent to each other in the learning class;
Characterized by further comprising
Method for dividing teeth. According to paragraph 2,
Before deriving the prediction vector,
Down-sampling the scan data by the data processing unit;
It further includes,
After the step of classifying according to the above extended class,
Up-sampling the scanned data to the original size by the restoration unit;
Characterized by further comprising
Method for dividing teeth. According to paragraph 3,
The upsampling step is
The restoration department
math equation

Accordingly, the class of each cell of the upsampled scan data is derived,
The f is a KNN (k-Nearest Neighbor) model,
The x, y, and z are cell coordinates of the up-sampled scan data,
The i is the cell index of the up-sampled scan data,
N is the number of cells in the scanned data of the original size,
The C is characterized in that it is an extended class.
Method for dividing teeth. According to paragraph 3,
The upsampling step is
The restoration department
math equation

Accordingly, the class of each cell of the upsampled scan data is derived,
The g is a KNN model where k is M,
The h is a KNN model where k is N,
The M and the N are preset constants,
M<N,
The x, y, and z are cell coordinates of the up-sampled scan data,
The i is the cell index of the up-sampled scan data,
The C is characterized in that it is an extended class.
Method for dividing teeth. According to paragraph 3,
When the restoration unit determines that the number of cells of a specific extended class is less than a preset threshold, converting the cells belonging to the extended class to a background class and erasing them;
Characterized by further comprising
Method for dividing teeth. According to paragraph 1,
After receiving the scan data,
Before deriving the prediction vector,
Reducing the three-dimensional coordinates of the scan data to two dimensions by the data processing unit through principal component analysis;
The data processing unit calculating the short axis length and the long axis length of the scan data;
The data processing unit confirming whether the scan data is full scan data or partial scan data based on the length of the minor axis and the ratio of the length of the minor axis and the length of the major axis using the learned machine learning model;
If the data processing unit determines that the scan data is partially scan data, deleting the scan data;
Characterized by further comprising
Method for dividing teeth. According to paragraph 1,
After receiving the scan data,
Before deriving the prediction vector,
The data processing unit deriving an eigenvector through principal component analysis of the xy plane of the scan data;
The data processing unit selecting as a rotation angle the smaller angle between the angle formed by the eigenvector and a reference line horizontally passing through the origin of the eigenvector;
reorienting the scan data by rotating the scan data around the origin so that the rotation angle becomes 0;
Characterized by further comprising
Method for dividing teeth. According to paragraph 1,
Before the step of receiving the scan data,
Preparing, by the data processing unit, training data including training scan data and a label indicating a learning class corresponding to each of a plurality of cells of the training scan data;
A step where the segmentation model performs a weighting operation in which a weight that has not yet been learned is applied to the training scan data to derive a learning prediction vector indicating the probability that a plurality of cells of the training scan data belong to a learning class;
A step where the learning unit calculates an overall loss including segmentation loss and classification loss;
the learning unit performing optimization to modify the weights of the segmentation model so that the total loss is minimized;
Characterized by further comprising
Method for dividing teeth. According to clause 9,
The step of calculating the total loss is
The above learning department
math equation

The split loss is calculated according to
Where L is the number of learning classes,
Where l is the learning class index,
N is the number of cells,
Where i is the cell index,
The p is a prediction vector for learning,
where y is a label,
Wherein ε is a preset constant.
Method for dividing teeth. According to clause 9,
The step of calculating the total loss is
The above learning department
math equation

Calculate classification loss according to
remind

is the probability of the data unit,
remind

silver
math equation

It is derived according to
Where L is the number of learning classes,
Where l is the index of the learning class,
where y is a label,
N is the number of cells of scan data,
remind

Characterized by being the number of cells corresponding to the learning class index.
Method for dividing teeth. In a device for dividing teeth,
a data processing unit that scans a plurality of teeth in three dimensions and receives scan data including a plurality of cells; and
By performing a weight calculation according to the weight learned for the scan data through a partition model, a prediction vector representing the probability that each of the plurality of cells belongs to a learning class is derived,
a division unit that divides teeth on scan data according to the probability of the prediction vector;
Characterized by including
A device for dividing teeth. According to clause 12,
The division part is
In the learning class, teeth pairs belonging to the same class are arranged symmetrically left and right, excluding the central incisors that are adjacent to each other, and are characterized by distinguishing them according to the extended class.
A device for dividing teeth. According to clause 13,
The data processing unit
Before deriving the prediction vector, down-sample the scan data,
The restoration unit classifies according to the extension class and then up-samples the scanned data to the original size.
A device for dividing teeth. According to clause 14,
The restoration department
math equation

Accordingly, the class of each cell of the upsampled scan data is derived,
The f is a KNN (k-Nearest Neighbor) model,
The x, y, and z are cell coordinates of the up-sampled scan data,
The i is the cell index of the up-sampled scan data,
N is the number of cells in the scanned data of the original size,
The C is characterized in that it is an extended class.
A device for dividing teeth. According to clause 14,
The restoration department
math equation

Accordingly, the class of each cell of the upsampled scan data is derived,
The g is a KNN model where k is M,
The h is a KNN model where k is N,
The M and the N are preset constants,
M<N,
The x, y, and z are cell coordinates of the up-sampled scan data,
The i is the cell index of the up-sampled scan data,
The C is characterized in that it is an extended class.
A device for dividing teeth. According to clause 14,
The restoration department
If the number of cells of a specific extended class is less than a preset threshold, cells belonging to the extended class are converted to the background class and erased.
A device for dividing teeth. According to clause 12,
The data processing unit
Through principal component analysis, the three-dimensional coordinates of the scan data are reduced to two dimensions,
Calculate the short axis length and long axis length of the scan data,
Using the learned machine learning model, determine whether the scan data is full scan data or partial scan data from the length of the minor axis and the ratio of the length of the minor axis and the length of the major axis,
If the scan data is partial scan data, the scan data is erased.
A device for dividing teeth. According to clause 12,
The data processing unit derives an eigenvector through principal component analysis of the xy plane of the scan data,
Selecting the smaller angle between the angle formed by the eigenvector and a reference line horizontally passing through the origin of the eigenvector as the rotation angle,
Characterized in that the scan data is reoriented by rotating the scan data with the origin as the rotation axis so that the rotation angle is 0.
A device for dividing teeth. According to clause 12,
The data processing unit
Prepare training data including training scan data and a label indicating a learning class corresponding to each of a plurality of cells in the training scan data,
The device is
When the segmentation model performs a weighting operation in which weights that have not yet been learned are applied to the training scan data to derive a learning prediction vector indicating the probability that a plurality of cells in the training scan data belong to the learning class,
Calculate the overall loss including segmentation loss and classification loss,
a learning unit that performs optimization by modifying the weights of the segmentation model to minimize the overall loss;
Characterized by further comprising
A device for dividing teeth. According to clause 20,
The learning department
math equation

The split loss is calculated according to
Where L is the number of learning classes,
Where l is the learning class index,
N is the number of cells,
Where i is the cell index,
The p is a prediction vector for learning,
where y is a label,
Wherein ε is a preset constant.
A device for dividing teeth. According to clause 20,
The learning department
math equation

Calculate classification loss according to
remind

is the probability of the data unit,
remind

silver
math equation

It is derived according to
Where L is the number of learning classes,
Where l is the index of the learning class,
where y is a label,
N is the number of cells of scan data,
remind

Characterized by being the number of cells corresponding to the learning class index.
A device for dividing teeth.