KR102334519B1 - Automated and simplified method for extracting landmarks of three dimensional dental scan data and computer readable medium having program for performing the method - Google Patents

Abstract

An automatic landmark detection method of the three-dimensional dental scan data comprises: a step of generating a two-dimensional depth image by projecting the three-dimensional scan data; a step of detecting a two-dimensional landmark in the two-dimensional depth image using a fully-connected convolutional neural network model; and a step of detecting the three-dimensional landmark of the three-dimensional scan data by back-projecting the two-dimensional landmark onto the three-dimensional scan data. Therefore, the present invention is capable of increasing an accuracy of the landmark of the three-dimensional scan data.

Description

A simplified automatic landmark detection method for dental 3D scan data and a computer-readable recording medium in which a program for executing the same is recorded on a computer HAVING PROGRAM FOR PERFORMING THE METHOD}

The present invention relates to a method for automatically detecting a landmark of dental 3D scan data and a computer-readable recording medium in which a program for executing the same on a computer is recorded, and more particularly, to a dental CT image and digital It relates to a method for automatically detecting landmarks in dental three-dimensional scan data that can reduce time and effort for matching impression models, and to a computer-readable recording medium in which a program for executing the same in a computer is recorded.

CT (Computed Tomography) or CBCT (Cone Beam Computed Tomography) (collectively referred to as CT) data, which are 3D volume data required when diagnosing oral and maxillofacial conditions or establishing surgery and treatment plans in dentistry and plastic surgery It includes not only hard tissue corresponding to bones or teeth, but also various information such as soft tissue such as the tongue or lips, and the location and shape of neural tube existing inside the bone. However, due to the presence of metallic materials in the oral cavity, such as implants, orthodontic devices, and dental crowns, that the patient has previously treated, metal artifacts occur in CT, an X-ray-based image, and the teeth and surrounding areas are significantly distorted and identified. and difficulties in diagnosis. In addition, it is difficult to specify the shape of the gum or the interface with the teeth. To supplement this limited dental and oral information, a three-dimensional digital scan model is acquired and used. This is obtained by directly scanning the patient's oral cavity, or data is obtained by scanning the patient's plaster impression model. .

When using scan data together with CT data, a matching process of overlapping data of different modalities is performed. In general, the same location on the CT scan data is matched by the user manually setting each landmark. In addition, scan data of the same patient acquired at different times for treatment progress or before and after comparison are matched in the same way. Since registration results are important basic data for treatment and surgery, it is very important to increase the accuracy of registration. In particular, in the case of implants, the location of the landmark, which is the matching criterion, requires high accuracy because it is the basis for planning work to place the implant in the optimal position by grasping the location of the neural tube, tissue, etc. However, manually marking a three-dimensional landmark in two different types of data on a consistent basis or at a fixed location is difficult and takes a lot of time, and there is a deviation for each user.

In order to obtain a landmark, when a marker is directly attached to the oral cavity to generate scan data, it may cause inconvenience to the patient, and since the inside of the oral cavity is a soft tissue, it is difficult to fix the marker, so it is difficult to be an appropriate method.

Republic of Korea Patent Publication No. 10-2020-0023703 Republic of Korea Patent Publication No. 10-2020-0083822

An object of the present invention is to automatically detect landmarks in 3D scan data for dental use in order to reduce time and effort for registration of dental CT images and 3D scan data. To provide a detection method.

Another object of the present invention is to provide a computer-readable recording medium in which a program for executing the automatic landmark detection method of the dental three-dimensional scan data is recorded on a computer.

A method for automatically detecting a landmark of a dental 3D scan data according to an embodiment for realizing the object of the present invention includes generating a 2D depth image by projecting the 3D scan data, a fully convolutional neural network model ( detecting a two-dimensional landmark in the two-dimensional depth image using a fully-connected convolutional neural network model) detecting the landmark.

In an embodiment of the present invention, generating the 2D depth image may include determining the projection direction vector through principal component analysis of the 3D scan data.

집합을 행렬로 나타낸

를

의 평균값

를 중심으로 이동시키는 단계, 상기 3차원 n개의 점 좌표에 대한 공분산

을 계산하는 단계, 상기 공분산 Σ를 고유분해(

)하는 단계(

이고

이며,) 및 w₁={w_1p,w_1q,w_1r},w₂={w_2p,w_2q,w_2r},w₃={w_3p,w_3q,w_3r} 중 고유값 λ가 가장 작은 방향 벡터 w₃을 이용하여 상기 투영 방향 벡터를 결정하는 단계를 포함할 수 있다.In an embodiment of the present invention, the determining of the projection direction vector comprises three-dimensional n point coordinates of the three-dimensional scan data.

set as a matrix

cast

mean value of

moving to the center, covariance for the three-dimensional n point coordinates

calculating the eigen decomposition (

) step (

ego

, and w ₁ ={w _1p ,w _1q ,w _1r },w ₂ ={w _2p ,w _2q ,w _2r },w ₃ ={w _3p ,w _3q ,w _3r } and determining the projection direction vector using the smallest direction vector w _{3 .}

일 때,

이면 w₃을 상기 투영 방향 벡터로 결정하고,

이면 -w₃을 상기 투영 방향 벡터로 결정할 수 있다.In one embodiment of the present invention, in the determining of the projection direction vector, the average of the normal vector of the 3D scan data is

when,

If w ₃ is determined as the projection direction vector,

Then, -w ₃ may be determined as the projection direction vector.

In an embodiment of the present invention, the 2D depth image may be formed on a projection plane defined by the projection direction vector as a normal vector and a first distance away from the 3D scan data.

In an embodiment of the present invention, the detecting of the 3D landmark may include inversely projecting the 2D landmark onto the 3D scan data in a direction opposite to the projection direction vector by using the projection direction vector. .

In an embodiment of the present invention, the fully convolutional neural network model includes a convolution process for detecting landmark features in the two-dimensional depth image, and a deconvolution process for adding landmark location information to the detected landmark features. can be performed.

In an embodiment of the present invention, the convolution process and the deconvolution process may be repeatedly performed in the fully convolutional neural network model.

In an embodiment of the present invention, the result of the deconvolution process may be in the form of a heat map corresponding to the number of the two-dimensional landmarks.

In one embodiment of the present invention, the pixel coordinates having the largest value in the heat map may indicate the position of the two-dimensional landmark.

In an embodiment of the present invention, the detecting of the two-dimensional landmark may further include learning the convolutional neural network model. In the learning of the convolutional neural network model, a learning 2D depth image and user-defined landmark information may be input. The user-defined landmark information may use the type of the learning landmark and information on the correct answer position in the learning 2D depth image of the learning landmark.

In an embodiment of the present invention, a program for executing the method of automatically detecting a landmark of the dental 3D scan data in a computer may be recorded in a computer-readable recording medium.

According to the method for automatically detecting landmarks of 3D scan data for dentistry according to the present invention, since the landmarks of 3D scan data are automatically detected using deep learning, the user's Effort and time can be reduced, and the accuracy of landmarks in 3D scan data can be increased.

In addition, since the landmark of the 3D scan data is automatically detected using deep learning, the accuracy of the registration of the dental CT image and the 3D scan data is improved, and the user's efforts for the registration of the dental CT image and the 3D scan data are reduced. time can be reduced.

1 is a flowchart illustrating a method for automatically detecting a landmark of dental 3D scan data according to the present embodiment.
2 is a perspective view illustrating an example of a landmark of 3D scan data.
3 is a conceptual diagram illustrating a method of generating a 2D depth image by projecting 3D scan data.
4 is a perspective view illustrating an example of a projection direction when generating a two-dimensional depth image.
5 is a perspective view illustrating an example of a projection direction when generating a two-dimensional depth image.
6 is a plan view illustrating an example of a two-dimensional depth image.
7 is a plan view illustrating an example of a two-dimensional depth image.
8 is a conceptual diagram illustrating an example of training data of a fully convolutional neural network for detecting two-dimensional landmarks.
9 is a conceptual diagram illustrating a fully convolutional neural network for detecting a two-dimensional landmark.
10 is a conceptual diagram illustrating a landmark detection unit.
11 is a plan view illustrating an example of a two-dimensional landmark.
12 is a conceptual diagram illustrating a method of detecting a 3D landmark by back-projecting a 2D landmark onto 3D scan data.

With respect to the embodiments of the present invention disclosed in the text, specific structural or functional descriptions are only exemplified for the purpose of describing the embodiments of the present invention, and the embodiments of the present invention may be embodied in various forms and the text It should not be construed as being limited to the embodiments described in .

Since the present invention can have various changes and can have various forms, specific embodiments are illustrated in the drawings and described in detail in the text. However, this is not intended to limit the present invention to the specific disclosed form, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

When an element is referred to as being “connected” or “connected” to another element, it is understood that it may be directly connected or connected to the other element, but other elements may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Other expressions describing the relationship between elements, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as meanings consistent with the context of the related art, and unless explicitly defined in the present application, they are not to be interpreted in an ideal or excessively formal meaning. .

On the other hand, when a certain embodiment can be implemented differently, functions or operations specified in a specific block may occur in a different order from that specified in the flowchart. For example, two consecutive blocks may be performed substantially simultaneously, or the blocks may be performed in reverse according to a related function or operation.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

1 is a flowchart illustrating a simplified automatic landmark detection method of dental 3D scan data according to the present embodiment. 2 is a perspective view illustrating an example of a landmark of 3D scan data.

1 and 2, the simplified automatic landmark detection method of the 3D scan data for dentistry generates a 2D depth image by projecting the 3D scan data (S100), the 2D depth image A step of detecting a two-dimensional landmark by applying to a fully-connected convolutional neural network (S200) and back-projecting the two-dimensional landmark onto the three-dimensional scan data 3D of the three-dimensional scan data It may include detecting the landmark (S300).

The generating of the 2D depth image ( S100 ) may be a process of imaging the depth of 3D scan data of the virtual camera. The automatic two-dimensional landmark detection step (S200) is a step of detecting a landmark in a two-dimensional image using a fully convolutional neural network deep learning model. In the landmark three-dimensional projection step (S300), the two-dimensional landmark detected in the previous two-dimensional landmark automatic detection step (S200) may be three-dimensionalized and reflected in the scan data.

Figure 2 shows three landmarks (LM1, LM2, LM3) of the three-dimensional scan data. In this embodiment, the landmark is located at a predetermined interval or at the top of a specific tooth (incisor, canine, molar, etc.) to estimate the shape of a dental arch. The landmark can be automatically detected at once by applying the same method to all landmarks without additional processing according to the location or characteristics of the landmark.

The landmarks of the 3D scan data may be points indicating specific positions of teeth. For example, the landmark of the 3D scan data may include three points LM1, LM2, and LM3. Here, the 3D scan data may be data representing the patient's upper jaw or data representing the patient's mandible. For example, the first landmark LM1 and the third landmark LM3 of the 3D scan data may represent the outermost point of the tooth of the 3D scan data in a lateral direction, respectively. The second landmark LM2 of the three-dimensional scan data is the first landmark LM1 and the third landmark LM3 in the arch including the first landmark LM1 and the third landmark LM3 ) can be a point between For example, the second landmark LM2 of the 3D scan data may indicate between two central incisors of a patient.

3 is a conceptual diagram illustrating a method of generating a 2D depth image by projecting 3D scan data. 4 is a perspective view illustrating an example of a projection direction when generating a two-dimensional depth image. 5 is a perspective view illustrating an example of a projection direction when generating a two-dimensional depth image.

1 to 5 , the depth image performs principal component analysis of each 3D point p(x, y, z) of the scan data and the scan data when projecting the 3D scan data onto a 2D plane. It is an image showing the vertical distance information between the plane UVs defined through The pixel value of the 2D image represents the distance d(u,v) from the 2D plane defined above to the surface of the scan data.

집합을 행렬로 나타낸

의 평균값

를 중심으로 데이터를 이동시킨다(

).In this case, principal component analysis (PCA) may be performed to determine the projection direction and plane. First, the three-dimensional coordinates of n points in the scan data

set as a matrix

mean value of

Move the data around (

).

을 구한다. 상기 공분산은 3차원 n개의 점 좌표들이 x, y, z 축으로 어떻게 분산되어 있는지를 나타낼 수 있다. 상기 공분산 Σ를 고유분해한 결과는

로 나타낼 수 있다. 행렬

는 열 벡터가 Σ의 고유 벡터 w(p,q,r)로 구성된다. 대각행렬

는 대각원소가 Σ의 고유값 λ이다. w={w₁,w₂,w₃} 중 고유값 λ가 가장 작은 방향 벡터 w₃은 치아의 뿌리에서 교합면 방향과 같거나(도 4) 그 반대 방향(도 5)일 수 있다. 예를 들어, 도 3에서 고유값 λ이 가장 큰 w₁은 횡 방향으로 치아의 양쪽 끝을 연결하는 방향일 수 있고, 고유값 λ이 두번째로 큰 w₂는 환자의 정면 방향 또는 환자의 후면 방향일 수 있으며, 고유값 λ이 가장 작은 w₃은 치아 뿌리에서 교합면을 향하는 방향 또는 그 반대 방향일 수 있다. 상기 방향 벡터 w₃은 w₃={w_3p,w_3q,w_3r}로 표현할 수 있다.Then, the covariance for the three-dimensional n point coordinates

save The covariance may indicate how the three-dimensional n point coordinates are distributed along the x, y, and z axes. The result of eigendecomposition of the covariance Σ is

can be expressed as procession

The column vector is composed of the eigenvector w(p,q,r) of Σ. diagonal matrix

is the eigenvalue λ of the diagonal element Σ. _{The direction vector w 3} having the smallest eigenvalue λ among w={w ₁ ,w ₂ ,w ₃ } may be the same as the direction of the occlusal surface at the root of the tooth ( FIG. 4 ) or the opposite direction ( FIG. 5 ). _{For example, w 1} having the largest eigenvalue λ in FIG. 3 may be a direction connecting both ends of the teeth in the lateral direction, and w ₂ having the second largest eigenvalue λ may be in the front direction of the patient or the posterior direction of the patient _{, and w 3} having the smallest eigenvalue λ may be in a direction from the tooth root to the occlusal surface or vice versa. The direction vector w ₃ may be expressed as w ₃ ={w _3p ,w _3q ,w _{3r }.}

을 이용한다.

이면 w₃을 투영 방향으로 결정하고,

이면 -w₃을 깊이 영상 생성 시 투영 방향으로 결정한다. 투영 평면은 투영 방향 벡터를 법선 벡터로 하고 3차원 스캔 데이터로부터 일정 거리만큼 떨어진 곳에 정의하고 깊이 영상을 생성한다. _{To find w 3} in the same direction from the tooth root to the occlusal surface, the average of the normal vector of the triangle set in the 3D scan data

use the

then _{determine w 3} as the projection direction,

If -w ₃ is determined as the projection direction when generating a depth image. The projection plane uses a projection direction vector as a normal vector, defines a predetermined distance from the 3D scan data, and generates a depth image.

는 치아가 상부를 향해 정출되어 있으면, 상부 방향으로 형성되고, 치아가 하부를 향해 정출되어 있으면, 하부 방향으로 형성될 수 있다. 도 4에서는 w₃이 치아의 정출 방향과 대체로 일치하는 방향이므로,

으로 계산되며, w₃ 벡터를 투영 방향 벡터로 이용하는 경우를 예시한다.In FIG. 4 , the three axial directions of the three-dimensional scan data obtained through the principal component analysis are w ₁ , w ₂ , and w ₃ , among which _{the eigenvalue λ of w 1} is the largest, and the eigenvalue λ of _{w 3 is} smallest Here, the projection direction is determined using the _{direction vector w 3} with the smallest eigenvalue λ. Normal vector average of the triangle set of 3D scan data

If the tooth is elongated toward the top, it may be formed in the upper direction, and if the tooth is elongated toward the lower side, it may be formed in the lower direction. In Fig. 4, w ₃ is a direction that generally coincides with the direction of tooth extraction,

, and the _{case of using the w 3} vector as the projection direction vector is exemplified.

으로 계산되며, -w₃ 벡터를 투영 방향 벡터로 이용하는 경우를 예시한다.In FIG. 5 , the three axial directions of the three-dimensional scan data obtained through principal component analysis are w ₁ , w ₂ , and w ₃ , among which _{the eigenvalue λ of w 1} is the largest, and the eigenvalue λ of _{w 3 is} smallest Here, the projection direction is determined using the _{direction vector w 3} with the smallest eigenvalue λ. In FIG. 5, w ₃ is in the direction substantially opposite to the direction of tooth extraction,

, and exemplifies the case of using the _{-w 3 vector as the projection direction vector.}

_{As such, since the projection direction is determined using the direction vector w 3} having the smallest eigenvalue λ in the principal component analysis, the teeth may be well formed so that the teeth do not overlap in the two-dimensional depth image.

6 is a plan view illustrating an example of a two-dimensional depth image. 7 is a plan view illustrating an example of a two-dimensional depth image.

6 and 7 are examples of 2D depth images obtained through the 2D depth image generating step S100. A bright portion in the image means a point having a large distance from the projection plane, and a dark portion in the image means a point having a short distance from the projection plane. That is, the two-dimensional depth image is an image having a depth value (d) with respect to two-dimensional coordinates {u, v}, and when the two-dimensional depth image is back-projected in a direction opposite to the projection direction, the three-dimensional scan data can be restored.

8 is a conceptual diagram illustrating an example of training data of a fully convolutional neural network for detecting two-dimensional landmarks. 9 is a conceptual diagram illustrating a fully convolutional neural network for detecting a two-dimensional landmark. 10 is a conceptual diagram illustrating a landmark detection unit. 11 is a plan view illustrating an example of a two-dimensional landmark.

1 to 11 , a landmark deep learning model using a fully convolutional neural network is trained using the depth image and user-defined landmark information as inputs. As shown in FIG. 10 , the user-defined landmark information used during learning includes 1) the type of landmark to be found (eg, divided by indices 0, 1, 2) and 2) the two-dimensional depth image of the landmark. It may be the correct position coordinates (u _i ,v _{i ).}

The landmark detector for automatic landmark detection may include a fully convolutional neural network. The fully convolutional neural network may be a neural network deep learning model composed of convolutional layers.

The fully convolutional neural network largely includes two processes as shown in FIG. 9 . In the convolution process, a feature of each landmark is detected and classified in a depth image through a plurality of pre-trained convolution layers. By combining this with the entire image information through the deconvolution process, location information is added to the feature, and the location of the landmark on the image is output as a heatmap. In this case, each heat map image may be output as many as the number of user-defined landmarks used when learning the deep learning model. For example, if the number of the user-defined landmarks is three, three heat map images corresponding to the three landmarks may be output.

That is, the convolution process can be said to be a process of extracting only features from the 2D depth image instead of losing location information. The landmark feature may be extracted through the convolution process. On the other hand, the deconvolution process can be referred to as a process of reviving the lost location information for the landmarks extracted in the convolution process.

In this embodiment, a deep learning neural network model in which a fully convolutional neural network is repeatedly superposed may be used for more precise detection.

In the fully convolutional neural network model, the convolution process and the deconvolution process may be repeatedly performed. In the fully convolutional neural network model, the number of times the convolution process and the deconvolution process are repeatedly performed may be determined in consideration of the accuracy of the landmark detection result.

As shown in FIG. 10 , for example, the landmark detector may include four overlapping neural networks (four convolution processes and four deconvolution processes).

The landmark detection unit may construct a system in which the two-dimensional depth image is input and a heat map indicating the location of a desired target landmark is output for each channel according to the learning model user-defined landmark index. The final result heat map can be obtained by summing the output heat map data of each step of the nested neural network for each channel. The pixel coordinates having the largest value in the result heat map data indicate the location of the detected landmark. Since the heat map is output for each channel in the order of the user-defined landmark index used during training, location information of the desired landmark can be obtained.

11 shows a result of automatically detecting a landmark of the 2D depth image using a fully convolutional neural network model. The 2D landmarks in the 2D depth image are expressed as L1, L2, and L3.

12 is a conceptual diagram illustrating a method of detecting a 3D landmark by back-projecting a 2D landmark onto 3D scan data.

1 to 12, the two-dimensional coordinates of the landmarks (L1, L2, L3) obtained in the landmark automatic detection step (S200) are the coordinates of the landmarks (LM1, LM2, LM3) of the three-dimensional scan data convert to The coordinates of the final 3D landmark may be calculated using the projection information used in generating the depth image ( S100 ). The two-dimensional landmarks L1, L2, and L3 are back-projected onto the three-dimensional scan data by using the projection information used in generating the depth image (S100), and the three-dimensional landmark LM1 of the three-dimensional scan data , LM2, LM3) can be obtained.

According to this embodiment, since the landmarks (LM1, LM2, LM3) of the three-dimensional scan data are automatically detected using deep learning, the user for extracting the landmarks (LM1, LM2, LM3) of the three-dimensional scan data It is possible to reduce the effort and time of the 3D scan data and to increase the accuracy of the landmarks (LM1, LM2, LM3) of the 3D scan data.

In addition, since landmarks (LM1, LM2, LM3) of 3D scan data are automatically detected using deep learning, the accuracy of registration of dental CT images and 3D scan data is improved, and dental CT images and 3D scan data are automatically detected. The user's effort and time for registration can be reduced.

According to an embodiment of the present invention, a computer-readable recording medium in which a program for executing the above-described simplified automatic landmark detection method of dental 3D scan data is recorded on a computer may be provided. The above-described method can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above-described method may be recorded in a computer-readable medium through various means. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and ROMs, RAMs, flash memories, etc. Hardware devices specially configured to store and execute program instructions are included. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention.

In addition, the above-described simplified automatic landmark detection method of 3D scan data for dentistry may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The present invention relates to a simplified automatic landmark detection method of dental three-dimensional scan data and a computer-readable recording medium in which a program for executing the same on a computer is recorded, for extracting landmarks of three-dimensional scan data The user's effort and time can be reduced, and the effort and time for matching the dental CT image and the digital impression model can be reduced.

Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. you will understand that you can

Claims (12)

generating a two-dimensional depth image by projecting the three-dimensional scan data;
detecting a two-dimensional landmark in the two-dimensional depth image using a fully-connected convolutional neural network model; and
and detecting the three-dimensional landmark of the three-dimensional scan data by back-projecting the two-dimensional landmark onto the three-dimensional scan data. The method of claim 1 , wherein the generating of the two-dimensional depth image comprises:
and determining a projection direction vector through principal component analysis of the three-dimensional scan data. 3. The method of claim 2, wherein determining the projection direction vector comprises:
3D coordinates of n points of the 3D scan data

set as a matrix

cast

mean value of

moving to the center;
Covariance for the three-dimensional n point coordinates

calculating ;
The covariance Σ is an eigendecomposition (

) step (

ego

is,); and
w ₁ ={w _1p ,w _1q ,w _1r },w ₂ ={w _2p ,w _2q ,w _2r },w ₃ ={w _3p ,w _3q ,w _3r } is the direction vector with the smallest eigenvalue λ and determining the projection direction vector by using w _{3 .} 4. The method of claim 3, wherein determining the projection direction vector comprises:
The normal vector average of the three-dimensional scan data is

when,

If w ₃ is determined as the projection direction vector,

The automatic landmark detection method of 3D scan data for dentistry, characterized in that the back surface _{-w 3 is determined as the projection direction vector.} The method of claim 2, wherein the two-dimensional depth image is
Using the projection direction vector as a normal vector, the method for automatically detecting a landmark of 3D scan data for dentistry, characterized in that it is formed on a defined projection plane at a distance from the 3D scan data by a first distance. The method of claim 2, wherein the detecting of the three-dimensional landmark comprises:
Using the projection direction vector, the automatic landmark detection method of 3D scan data for dentistry, characterized in that the reverse projection of the 2D landmark onto the 3D scan data in the reverse direction of the projection direction vector. The method of claim 1, wherein the fully convolutional neural network model is
a convolution process for detecting landmark features in the two-dimensional depth image; and
A method of automatically detecting landmarks in dental 3D scan data, characterized in that a deconvolution process is performed for adding landmark location information to the detected landmark features. The method of claim 7, wherein the convolution process and the deconvolution process are repeatedly performed in the fully convolutional neural network model. The method of claim 7, wherein the result of the deconvolution process is in the form of a heat map corresponding to the number of the two-dimensional landmarks. The method of claim 9, wherein the pixel coordinates having the largest value in the heat map indicate the position of the two-dimensional landmark. The method of claim 1, wherein detecting the two-dimensional landmark further comprises learning the convolutional neural network model;
In the step of learning the convolutional neural network model, input a learning two-dimensional depth image and user-defined landmark information,
The user-defined landmark information is a type of learning landmark and an automatic landmark detection method of 3D scan data for dental use, characterized in that the correct position information in the learning 2D depth image of the learning landmark is used. A computer-readable recording medium in which a program for executing the method of any one of claims 1 to 11 in a computer is recorded.

Images