KR20210098683A - Method for providing information about orthodontics and device for providing information about orthodontics using deep learning ai algorithm - Google Patents

Abstract

The present invention provides an information providing method for orthodontics and a device using the same. The information providing method comprises the following steps of: receiving a side head medical image or an intraoral image of a subject; predicting a side head face region or a dental region including a side head face portion within the side head medical image or the intraoral image; and classifying surgical or non-surgical treatment needed for the subject based on the side head face region or the dental region.

Description

A method for providing information on orthodontics using a deep learning artificial intelligence algorithm and a device using the same

The present invention relates to a method for providing information on orthodontics and a device for providing information on orthodontics using the same, and more particularly, to a method for providing information on orthodontics based on a medical image used for orthodontic analysis and orthodontics using the same. It relates to a device for providing information about.

In general, a state in which the dentition is not aligned and the upper and lower teeth occlusion is abnormal is called malocclusion. Such malocclusion may cause functional problems such as mastication and pronunciation problems and aesthetic problems on the face, as well as health problems such as tooth decay and gum disease.

Orthodontic treatment may be performed as a method for making this malocclusion into a normal bite.

On the other hand, in orthodontic treatment, an important part may be to determine a treatment plan. In particular, with respect to the prognosis of orthodontic treatment, the determination of whether to extract, furthermore, the treatment plan establishment step of determining the tooth to be extracted may be very important. For example, a poor treatment plan may cause failure of anchorage control, abnormal inclination of anterior teeth, improper occlusion, inadequate horizontal overjet and overbite of teeth, and difficulty in occlusion of the extraction space. It can cause problems.

Based on their experience and knowledge, most orthodontists use data from clinical evaluations, dental photographs, dental models, and radiographs to determine treatment plans. In this case, such decision-making may often cause errors in the treatment planning process. Furthermore, differences in treatment plan may occur between experienced and inexperienced medical staff.

Therefore, as the accuracy of establishing a treatment plan before orthodontic treatment is more required to improve the quality of medical services, the development of a method for providing information on orthodontic treatment by accurately determining the treatment required for orthodontic treatment is continuously required. there is a situation.

The description underlying the invention has been prepared to facilitate understanding of the invention. It should not be construed as an admission that the matters described in the background technology of the invention exist as prior art.

On the other hand, in order to solve the above-mentioned problems, orthodontic analysis programs configured to perform analysis before orthodontic treatment programmatically have appeared. However, such an orthodontic analysis program may cause inconvenience to medical staff as anatomical landmarks must be input one by one. Furthermore, in this conventional technique, in that the medical staff must directly find the anatomical landmark based on the lateral head medical image, there may still be a limitation in that the accuracy of the analysis may vary depending on the skill of the medical staff.

The inventors of the present invention paid attention to artificial intelligence technology that is being used in various fields such as classification, face recognition, and character recognition as a way to solve the above problems.

More specifically, the inventors of the present invention recognize that, when artificial intelligence technology is applied to the diagnostic field of orthodontics, it is possible to provide information about orthodontics by making an appropriate decision for diagnosis based on clinical data. could

The inventors of the present invention could expect that, by the system for providing information on orthodontics, an orthodontist can make an accurate diagnosis and establish a treatment plan by referring to the diagnosis probability determined by the predictive model regardless of skill level.

As a result, the inventors of the present invention provide an information providing system for orthodontics based on a predictive model that determines necessary treatments such as non-surgical correction and orthodontic surgery based on clinical data, in particular, medical images such as dental images and radiographic images. came to development.

At this time, the inventors of the present invention built a predictive model of artificial intelligence that probabilistically provides information on orthodontics based on a plurality of medical images through a supervised learning method. Furthermore, the inventors of the present invention configured an orthodontic information providing system to determine the probability of an appropriate diagnosis when a new medical image is input using this.

In addition, the inventors of the present invention provide a system for providing information on orthodontics so that a predictive model classifies and evaluates necessary treatment only with medical images without a process of extracting measurement values from clinical data, which is an essential step in a conventional orthodontic analysis program. was configured.

Accordingly, the inventors of the present invention derive measurement values by taking specific landmarks from the standard radiographic image of the lateral head, and derive the measurement values by performing model analysis from the oral model. could be expected to do.

Furthermore, the inventors of the present invention can implement a system for providing information about orthodontics only with lateral head medical images and/or intraoral images, compared to conventional techniques using all of radiographic images, intraoral images, extraoral images and oral models. could be configured to do so.

The inventors of the present invention, in particular, applied a deep learning technique (eg, two-stream CNN) based on a convolutional neural network (CNN), and orthodontized to perform a diagnosis covering not only non-surgical treatment but also surgical treatment. It was possible to increase the performance of diagnosis by configuring an information providing system for

Accordingly, the problem to be solved by the present invention is to predict the lateral facial region or tooth region in each of the received lateral head medical images and/or intraoral images, and to classify the surgical or non-surgical treatment necessary for the subject based on this. It is to provide a method for providing information on orthodontics to provide and a device using the same.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the problems as described above, there is provided a method for providing information about orthodontics according to an embodiment of the present invention. The method includes the steps of receiving a paratrophic medical image or an intraoral image of a subject, predicting a parafacial region or a dental region comprising a parafacial region in the paratrophic medical image or intraoral image, and and classifying the surgical or non-surgical treatment required for the subject based on the dental area.

According to a feature of the present invention, the predicting of the lateral facial region or the dental region includes predicting the lateral facial region using a lateral facial region prediction model configured to predict the lateral facial region within the paratrophic medical image. can do. In addition, the step of classifying the surgical treatment or the non-surgical treatment may include classifying the necessary treatment for the subject as a surgical treatment or a non-surgical treatment based on the parafacial region.

According to another feature of the present invention, predicting the lateral facial region may include predicting the coordinates of the lateral facial region in the lateral head medical image by using the lateral facial region prediction model.

According to another feature of the present invention, predicting the parafacial area or the tooth area may include predicting the tooth area using a tooth area prediction model configured to predict the tooth area in the intraoral image. . In addition, the step of classifying the surgical treatment or the non-surgical treatment may include classifying the necessary treatment for the subject as a surgical treatment or a non-surgical treatment based on the tooth area.

According to another feature of the present invention, predicting the tooth region may include predicting the coordinates of the tooth region in the intraoral image by using the tooth region prediction model.

According to another feature of the present invention, the step of classifying the surgical treatment or the non-surgical treatment includes an orthodontic classification model configured to classify the necessary treatment as a surgical treatment or a non-surgical treatment based on the parafacial region or the tooth region. It may include the step of classifying the necessary treatment for the subject by using the.

According to another feature of the present invention, the orthodontic classification model may be further configured to classify a necessary treatment into a surgical treatment or a non-surgical treatment based on the parafacial region and the tooth region.

According to another feature of the present invention, the orthodontic classification model includes two independent feature extraction layers, configured to extract features, and a fusion layer that integrates two features extracted from the two feature extraction layers. fusion layer).

According to another feature of the present invention, the orthodontic classification model may be a Two-Stream Convolutional Neural Network (CNN).

According to another feature of the present invention, the method may further include, after the step of predicting the lateral facial region or the tooth region, generating a box including the lateral facial region or the tooth region.

According to another feature of the present invention, the surgical treatment may be at least one of orthodontic class 2 extraction surgery, orthodontic class 2 non-extraction surgery, orthodontic class 3 extraction surgery, and orthodontic class 3 non-extraction surgery.

According to another feature of the present invention, the non-surgical treatment includes non-extraction correction, upper and lower first premolar extraction correction, upper and lower second premolar extraction correction, maxilla first premolar extraction correction, and maxilla first premolar mandibular second premolar extraction correction. may be at least one of

According to another feature of the present invention, the step of classifying the surgical treatment or the non-surgical treatment may include probabilistically predicting the surgical treatment or the non-surgical treatment. Furthermore, after the classifying step, the method may further include providing a surgical treatment diagnosis probability or a non-surgical treatment diagnosis probability.

According to another feature of the present invention, after predicting the paratrophic facial region or tooth region, the steps of converting the paratrophic facial region or the tooth region to black and white, and vectorizing the black-and-white converted paratrophic facial region or the black-and-white converted tooth region It may further include the step of

According to another feature of the present invention, prior to the receiving step, receiving a sample collateral head medical image or a sample intraoral image of a sample subject, selecting a sample paratrophic facial region in the sample collateral head medical image or sample intraoral The method may further include receiving a selection input for a sample tooth region within the image, and generating a paratrophic medical image for training based on the coordinates of the sample parafacial region or the coordinates of the sample tooth region and the sample paracephalometric medical image. can

According to another feature of the present invention, the generating of the lateral head medical image for learning includes the step of generating a json file or xml file including information on the sample lateral head medical image and coordinates for the sample lateral head facial region. may include

According to another feature of the present invention, after the classifying step, based on the classification result, the method may further include determining a treatment plan for the subject.

In order to solve the problems as described above, there is provided a device for providing information about orthodontics according to another embodiment of the present invention. The device includes a receiving unit configured to receive a paracephalometric medical image or an intraoral image of a subject, and a processor connected to the receiving unit. At this time, the processor predicts a parafacial region or a dental region including a paratrophic region in the paratrophic medical image or intraoral image, and based on the paratrophic facial region or tooth region, a surgical treatment or non-surgical treatment required for the subject It is structured to classify treatments.

According to a feature of the present invention, the processor predicts the lateral facial region using a lateral facial region prediction model configured to predict the lateral facial region within the lateral head medical image, and a necessary treatment for the subject based on the lateral facial region may be further configured to classify as surgical treatment or non-surgical treatment.

According to another feature of the present invention, the processor predicts a tooth area using a tooth area prediction model configured to predict a tooth area in an intraoral image, and provides a surgical treatment or a necessary treatment for the subject based on the tooth area. It can be configured to classify as a non-surgical treatment.

According to another feature of the present invention, the processor uses an orthodontic classification model configured to classify a necessary treatment as a surgical treatment or a non-surgical treatment, based on the parafacial region or the dental region, to determine the necessary treatment for the subject. It may be further configured to classify.

The present invention provides an information providing system for orthodontics using a predictive model configured to predict a parafacial region and a dental region for each of the paracephalic medical images and/or intraoral images, and furthermore, a classification model configured to classify necessary treatments. There is an effect that can provide information for the orthodontic analysis of the subject.

More specifically, the present invention provides information on orthodontic treatment by using a predictive model to classify and evaluate necessary treatments only with medical images, without the process of extracting measurement values from clinical data, which is an essential step in the conventional orthodontic analysis program. can do.

Accordingly, the present invention is to overcome the limitations of the conventional orthodontic system, which derives measurement values by taking specific landmarks from the standard radiographic image of the lateral head, and derives the measurement values by performing model analysis from the oral model, and performs diagnosis based on this. can

In addition, the present invention can implement a system that provides information on orthodontics only with a lateral head medical image and/or an intraoral image, compared to the prior art using both a radiographic image, an intraoral image, an extraoral image, and an oral model. can

Furthermore, the present invention has an effect of overcoming the limitations of the conventional orthodontic information providing system that is dependent on the knowledge and experience of the medical staff based on clinical data.

More specifically, the present invention provides a system for providing information on orthodontics based on various predictive models, thereby overcoming errors in diagnosis according to the proficiency of medical staff and low reliability of diagnosis results, and correct correction for the subject It may be possible to establish a treatment plan before treatment.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.

1A illustrates an orthodontic analysis system using a device for providing information on orthodontics according to an embodiment of the present invention.
1B exemplarily shows the configuration of a device for providing information on orthodontics according to an embodiment of the present invention.
Figure 2 shows a procedure of a method for providing information about orthodontics according to an embodiment of the present invention.
3A exemplarily illustrates a procedure of a lateral facial region based on a lateral facial region prediction model in a method for providing information on orthodontics according to an embodiment of the present invention.
3B exemplarily illustrates a procedure of a tooth area based on a tooth area prediction model in a method for providing information on orthodontics according to an embodiment of the present invention.
4 exemplarily illustrates a procedure of medical treatment classification based on an orthodontic classification model in a method for providing information on orthodontics according to an embodiment of the present invention.
5 exemplarily illustrates a pre-processing procedure of a lateral facial region and a tooth region in a method for providing orthodontic information according to an embodiment of the present invention.
6A to 6D are diagrams illustrating a process of generating image data for training of predictive models used in a method for providing orthodontic information and a device for providing orthodontic information using the same according to an embodiment of the present invention.

Advantages of the invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims.

The shapes, sizes, proportions, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are exemplary, and thus the present invention is not limited to the illustrated matters. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When 'including', 'having', 'consisting', etc. mentioned in this specification are used, other parts may be added unless 'only' is used. When a component is expressed in a singular, the case in which the plural is included is included unless otherwise explicitly stated.

In interpreting the components, it is interpreted as including an error range even if there is no separate explicit description.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and technically various interlocking and driving are possible, as will be fully understood by those skilled in the art, and each embodiment may be independently implemented with respect to each other, It may be possible to implement together in a related relationship.

For clarity of interpretation of the present specification, terms used herein will be defined below.

As used herein, the term “lateral head medical image” may refer to all images including a side view of a subject received from a medical imaging apparatus. Preferably, the lateral head medical image disclosed herein may be a lateral head standard radiographic image, but is not limited thereto. Meanwhile, the lateral head medical image may be a two-dimensional image, a three-dimensional image, a still image of one cut, or a moving image composed of a plurality of cuts. For example, if the sidebar head medical image is a moving picture composed of a plurality of cuts, the sideface facial region for each of the plurality of sidehead head medical images may be predicted according to the orthodontic information providing method according to an embodiment of the present invention. can

Meanwhile, the lateral head medical image may include a lateral facial region including a lateral facial region.

As used herein, the term “intraoral image” is an image representing the structure of a subject's teeth, and may preferably be an image including both upper and lower teeth of the subject.

In this case, the intraoral image may include a tooth area for the incisors, canines, premolars, and molars of the upper jaw and/or a tooth area for at least one of the incisors, canines, premolars, and molars of the lower teeth. there is.

Preferably, the tooth region may be a tooth region for the incisors, canines, premolars, and molars of the upper teeth and/or regions for the plurality of teeth of the incisors, canines, premolars, and molars of the lower teeth. . Accordingly, the arrangement of the tooth region may reflect the characteristics of the examinee's dentition.

As used herein, the term "paralateral facial region prediction model" may be a model configured to predict a paratrophic facial region including a lateral facial region, which is a target region to be measured for orthodontic treatment with respect to a lateral head medical image. For example, the lateral facial region prediction model may be a faster R-CNN trained to predict the lateral facial region within the lateral head medical image. More specifically, the lateral facial region prediction model receives a lateral head medical image for training in which the coordinates of the lateral facial region are predetermined, and sets the lateral facial region as a region of interest in the lateral head medical image for training based on the coordinates of the lateral facial region. It can be a model trained to predict. However, the lateral facial region prediction model is not limited thereto and may be based on more various image segmentation algorithms.

In this case, the lateral facial region may have different pixel values and textures from other regions, for example, the background region. Accordingly, the lateral facial region prediction model may predict the lateral facial region based on a pixel value or a texture.

As used herein, the term "tooth region prediction model" may be a model configured to predict a plurality of tooth regions of a subject corresponding to upper and lower teeth, which is a target region to be measured for orthodontic treatment with respect to a paracephalometric medical image. . For example, the tooth area prediction model may be a faster R-CNN trained to predict a tooth area within an intraoral image. More specifically, the tooth area prediction model receives an intraoral image for learning in which the coordinates of these areas are predetermined for each of the incisors, canines, premolars, and molars of the upper and lower teeth, and the coordinates of the predetermined tooth area are received. It may be a model trained to predict a tooth region as a region of interest in an intraoral image for learning based on it. However, the tooth region prediction model is not limited thereto and may be based on more various image segmentation algorithms.

In this case, the tooth region may have different pixel values and textures from other regions, for example, background regions such as lips and tongue. Accordingly, the tooth region prediction model may predict the tooth region based on a pixel value or texture.

As used herein, the term “surgical treatment” refers to surgical orthodontic treatment, and may refer to orthognathic surgery. In this case, the surgical treatment may be determined based on the arrangement of the lateral facial region and/or the tooth region of the subject.

According to a feature of the present invention, the surgical treatment may be at least one of orthodontic class 2 extraction surgery, orthodontic class 2 non-extraction surgery, orthodontic class 3 extraction surgery, and orthodontic class 3 non-extraction surgery.

As used herein, the term “non-surgical treatment” may refer to non-surgical orthodontic correction, for example, correction using a bracket. In this case, the non-surgical treatment may be determined based on the arrangement of the lateral facial region and/or the tooth region of the subject.

According to a feature of the present invention, the non-surgical treatment includes at least one of non-extraction correction, upper and lower first premolar extraction correction, upper and lower second premolar extraction correction, maxilla first premolar extraction correction, and maxilla first premolar mandibular second premolar extraction correction. can be one

As used herein, the term “orthodontic classification model” may be a model configured to classify surgical or non-surgical treatment required for a subject based on a parafacial region and/or a tooth region. For example, the orthodontic classification model may include at least one of orthodontic class 2 extraction surgery, orthodontic class 2 non-extraction surgery, orthodontic class 3 extraction surgery, and orthodontic class 3 non-extraction surgery required for the subject, from the images of the lateral facial region and the tooth region. It may be a model trained to probabilistically provide a diagnostic result of a surgical procedure. In addition, the orthodontic classification model is, from the images of the lateral facial region and the tooth region, non-extraction correction, upper and lower first premolar extraction correction, upper and lower second premolar extraction correction, upper and lower first premolar extraction correction, and maxilla first premolar mandible. It may be a model trained to probabilistically provide a diagnostic result of at least one non-surgical treatment of 2 premolar extraction correction.

In this case, the orthodontic classification model is a fusion that integrates two features extracted from two feature extraction layers and two feature extraction layers configured to extract features for each of the input lateral facial region and tooth region. It may be formed of a fusion layer. Preferably, the orthodontic classification model may be a Two-Stream Convolutional Neural Network (CNN), but is not limited thereto. For example, the orthodontic classification model may include a first orthodontic classification model configured to classify a surgical or non-surgical treatment based on a parafacial region and a first orthodontic classification model configured to classify a surgical or non-surgical treatment based on a tooth region. It may be an ensemble model in which two independent models of the second orthodontic classification model are combined.

Meanwhile, according to another feature of the present invention, the orthodontic classification model has an initial learning rate of 0.01, a momentum of 0.9, a weight decay of 0.0005, and a dropout ( dropout), a learning factor value of 0.5 may be set. However, learning factor values of parameters input for learning are not limited thereto.

Hereinafter, an orthodontic analysis system using a device for providing information about orthodontics and a device for providing information about orthodontics according to an embodiment of the present invention will be described with reference to FIGS. 1A and 1B .

1A illustrates an orthodontic analysis system using a device for providing information on orthodontics according to an embodiment of the present invention. 1B exemplarily shows the configuration of a device for providing information on orthodontics according to an embodiment of the present invention.

First, referring to FIG. 1A , a lateral head medical image 210 may be acquired from a lateral head measurement radiography apparatus, and an intraoral image 220 of a subject may be acquired from the intraoral imaging apparatus. In this case, the acquired medical image 200 of the paratrophic medical image 210 and/or the intraoral image 220 is received by the device 100 for providing orthodontic information according to an embodiment of the present invention.

The device 100 for providing information on orthodontics predicts a parafacial region and/or a tooth region in the received paratrophic medical image 210 and/or intraoral image 220 , and based on the predicted region The necessary surgical or non-surgical treatment is determined and provided for the subject.

More specifically, referring to FIG. 1B , the device 100 for providing information on orthodontics includes a receiving unit 110 , an input unit 120 , an output unit 130 , a storage unit 140 , and a processor 150 . do.

In more detail, the receiver 110 may be configured to receive a lateral head medical image of the subject from the paracephalometric radiography apparatus or to receive an intraoral image of the subject from the intraoral imaging apparatus. In this case, the lateral head medical image 210 obtained by the receiver 110 may be a lateral head standard radiographic image, and the intraoral image 220 may be an RGB color image, but is not limited thereto. Meanwhile, the receiving unit 110 may be further configured to transmit the acquired paracephalometric medical image 210 and/or intraoral image 220 to the processor 150 to be described later. Meanwhile, the lateral head medical image 210 obtained through the receiver 110 may include a lateral facial region, and the intraoral image 220 may include a plurality of tooth regions.

The input unit 120 may set the device 100 for providing information on orthodontics. Furthermore, the user may directly select a parafacial region and a tooth region with respect to each of the lateral head medical image 210 and the intraoral image 220 . In this case, the input unit 120 may be a keyboard, a mouse, or a touch screen panel, but is not limited thereto.

Meanwhile, the output unit 130 may visually display the lateral head medical image 210 and/or the intraoral image 220 obtained from the receiving unit 110 . Furthermore, the output unit 130 may be configured to display a parafacial region determined by the processor 150 in the paratrophic medical image 210 , and/or a plurality of tooth regions determined in the intraoral image 210 . . Furthermore, the output unit 130 may be configured to display information about a diagnosis necessary for the subject determined by the processor 150 . However, the present invention is not limited thereto, and the output unit 130 may be configured to display more various information determined by the processor 150 for orthodontic treatment of the examinee.

The storage unit 140 stores the medical image 200 of the subject acquired through the receiving unit 110, and stores the instruction of the device 100 for providing orthodontic information set through the input unit 120. can be Furthermore, the storage unit 140 is configured to store results predicted by the processor 150, which will be described later. However, it is not limited to the above, and the storage unit 140 may store more various pieces of information determined by the processor 150 for orthodontic treatment of the examinee.

Meanwhile, the processor 150 may be a component for providing an accurate prediction result to the device 100 for providing orthodontic information. In this case, the processor 150 predicts the parafacial region with respect to the paratrophic medical image 210 or predicts a plurality of tooth regions within the intraoral image 220, and predicts the predicted facial region and/or the plurality of tooth regions. It may be configured to classify and provide diagnostic results according to the dental condition of the examinee based on the classification.

For example, the processor 150 obtains from the receiving unit 110 a prediction model trained to predict a parafacial region in the paratrophic medical image 210 of the subject, and a tooth region in the intraoral image 220 . It may be configured to use a predictive model that has been trained to predict. Furthermore, the processor 150 may be configured to use a classification model that classifies and provides a diagnosis result according to a dental condition of a subject based on a parafacial region and/or a tooth region predicted by the predictive model. In this case, the model trained to predict each of the parafacial region and the tooth region may be based on a faster R-CNN, and the classification model of orthodontic diagnosis may be based on a Two-Stream CNN, but is not limited thereto. For example, the prediction models and classification models used in various embodiments of the present invention, for example, are a Deep Neural Network (DNN), a Deep Convolution Neural Network (DCNN), a Recurrent Neural Network (RNN), Predict the region of interest based on RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), SSD (Single Shot Detector) model, SVM (Support Vector Machine) or U-net, or provide appropriate medical treatment for the subject can be configured to classify.

Meanwhile, according to various embodiments of the present disclosure, the device for providing information on orthodontics converts the medical image to a black-and-white image when the lateral head medical image and/or the intraoral image received by the receiving unit 110 is an RGB color image. It may further include a data pre-processing unit configured to convert to and vectorize a black-and-white image.

Hereinafter, a method of providing information on orthodontics according to an embodiment of the present invention will be described in detail with reference to FIGS. 2, 3A, 3B, 4 and 5 . Figure 2 shows a procedure of a method for providing information about orthodontics according to an embodiment of the present invention. 3A exemplarily illustrates a procedure of a lateral facial region based on a lateral facial region prediction model in a method for providing information on orthodontics according to an embodiment of the present invention. 3B exemplarily illustrates a procedure of a tooth area based on a tooth area prediction model in a method for providing information on orthodontics according to an embodiment of the present invention. 4 exemplarily illustrates a procedure of medical treatment classification based on an orthodontic classification model in a method for providing information on orthodontics according to an embodiment of the present invention. 5 exemplarily illustrates a pre-processing procedure of a lateral facial region and a tooth region in a method for providing orthodontic information according to an embodiment of the present invention.

First, referring to FIG. 2 , a medical treatment decision procedure according to an embodiment of the present invention is as follows. First, a lateral head medical image or an intraoral image of the subject is received (S210). Thereafter, the lateral facial region is predicted with respect to the lateral head medical image, or the tooth region is predicted within the intraoral image ( S220 ). Next, a surgical treatment or a non-surgical treatment necessary for the subject is determined based on the parafacial area and/or the tooth area ( S230 ). Finally, a predicted result is provided (S240).

More specifically, in the step S210 of receiving the lateral head medical image or the intraoral image, a lateral head medical image of the subject is received from the lateral head measurement radiography apparatus, or an intraoral image of the subject is received from the intracranial imaging apparatus. can be

Preferably, in the step of receiving the lateral head medical image or the intraoral image ( S210 ), the lateral head medical image and the intraoral image may be received together. In this case, the lateral head medical image may be a lateral head standard radiographic image, and the intraoral image may be an RGB color image, but is not limited thereto.

According to an embodiment of the present invention, in the step of receiving the medical image of the temporal head or the oral cavity (S210), the medical image of the temporal head is pre-processed to have constant pixels so that the medical image and the intraoral image can be quickly analyzed. Images and intraoral images may be received.

Next, in step S220 of predicting the parafacial region or predicting the tooth region in the intraoral image with respect to the paracephalic medical image, the region of interest for each medical image may be predicted.

According to a feature of the present invention, in the step (S220) of predicting the sidemofacial region with respect to the paratrophic medical image or predicting the tooth region within the intraoral image (S220), the sidemolar facial region prediction model and/or the tooth region prediction model A facial region prediction model and/or a tooth region may be determined.

For example, referring together with FIG. 3A , in the step of predicting the paratrophic facial region with respect to the paratrophic medical image or predicting the tooth region within the intraoral image (S220), the paratrophic facial region prediction model 310, the above-mentioned The lateral head medical image 210 obtained in the step S210 of receiving the unilateral head medical image or the intraoral image is input. Then, the paratrophic region 312 , which is a region of interest corresponding to the lateral facial region in the lateral head medical image 210 , is determined by the lateral facial region prediction model 310 . In this case, the lateral facial region prediction model 310 may be further configured to form a box surrounding the lateral facial region 312 . Meanwhile, the lateral facial region prediction model 310 may be configured to predict the lateral facial region 312 based on the coordinates for the lateral facial region 312 (or box) in the lateral head medical image 210 , but It is not limited.

On the other hand, the lateral facial region 312 predicted by the result of predicting the lateral facial region with respect to the lateral head medical image or predicting the tooth region within the intraoral image ( S220 ) may be cropped to include only the corresponding region. there is.

Referring further to FIG. 3B , in the step S220 of predicting the parafacial region with respect to the paratrophic medical image or predicting the dental region within the intraoral image ( S220 ), the dental region prediction model 320 is applied to the above-described paratrophic medical image. Alternatively, the intraoral image 220 obtained in step S210 of receiving the intraoral image is input. Next, a plurality of tooth regions 322 that are regions of interest corresponding to each tooth in the intraoral image 220 are determined by the tooth region prediction model 320 . Preferably, the tooth region prediction model 320 may be configured to predict the region for all teeth appearing in the oral image 220 rather than predicting the region of a single tooth specified in the oral image 220 . Accordingly, the arrangement of the plurality of tooth regions 322 may represent the characteristics of the examinee's teeth. Meanwhile, the tooth region prediction model 320 may be further configured to form a box surrounding the tooth region 322 . At this time, the tooth region prediction model 320 calculates a plurality of tooth regions 322 based on the coordinates for the regions (or boxes) of the upper and lower incisors, canines, premolars, and molars in the intraoral image 220 . It may be configured to predict, but is not limited thereto.

On the other hand, a plurality of tooth regions 322 predicted by the result of predicting the parafacial region with respect to the paratrophic medical image or predicting the dental region within the intraoral image ( S220 ) are to be cropped to include only the corresponding region. may be

Referring again to FIG. 2 , in the step of determining a surgical treatment or a non-surgical treatment ( S230 ), based on the parafacial region and/or the plurality of tooth regions, surgical treatment or non-surgical treatment according to the dental status of the subject A diagnostic outcome of the treatment may be determined.

According to a feature of the present invention, in the step of determining the surgical treatment or the non-surgical treatment (S230), the surgical treatment diagnosis probability or the non-surgical treatment diagnosis probability may be determined.

For example, in the step of determining the surgical treatment or the non-surgical treatment (S230), for the subject, the residual probability of orthodontic class 2 extraction surgery, the diagnosis probability of the orthodontic class 2 non-extraction surgery, and the diagnosis probability of the orthodontic class 3 extraction surgery , or the diagnosis probability of orthognathic grade 3 non-extraction surgery may be determined. In addition, in the step of determining the surgical treatment or the non-surgical treatment (S230), the non-extraction orthodontic diagnosis probability, the upper and lower first premolar extraction correction diagnosis probability, the upper and lower second premolar extraction correction diagnosis probability, the maxilla first premolar extraction correction diagnosis probability A probability, or a diagnosis probability of maxillary first premolar and mandibular second premolar extraction correction may be determined. In this case, the higher the diagnosis probability of the medical treatment, the higher the success rate of orthodontic treatment when applied to the subject.

According to another feature of the present invention, in the step of determining the surgical treatment or the non-surgical treatment (S230), the treatment required for the subject may be determined by the orthodontic classification model.

For example, referring together with FIG. 4 , the paratrophic facial region 312 predicted by the result of the step S220 of predicting the lateral facial region with respect to the above-described lateral head medical image or predicting the tooth region within the intraoral image. ) and a plurality of tooth regions 322 may be input to the orthodontic classification model 330 .

At this time, the orthodontic classification model 330 integrates two independent feature extraction layers 332 and their features configured to extract features for each of the parafacial region 312 and the plurality of tooth regions 322, and finally may consist of a fusion layer 334 configured to determine the necessary medical treatment. More specifically, the feature extraction layer 332 may correspond to a Two-Stream CNN, and the fusion layer 334 may correspond to a fusion and a fully connected (FC) layer, but is not limited thereto. For example, the orthodontic classification model 330 is based on the plurality of tooth regions 322 and a first orthodontic classification model configured to classify surgical or non-surgical procedures based on the parafacial region 312 . It may be an ensemble model in which two independent models of the second orthodontic classification model configured to classify surgical treatment or non-surgical treatment are combined.

Each of the input lateral facial region 312 and the plurality of dental regions 322 is subjected to a feature extraction layer 332, features are extracted, and features are integrated via a fusion layer 334, and finally an appropriate surgical treatment; or a diagnostic result 342 of the non-surgical treatment is determined. At this time, the diagnosis result 342 is, for the subject, the residual probability of orthodontic class 2 extraction surgery, the diagnosis probability of orthodontic class 2 non-extraction surgery, the diagnosis probability of orthodontic class 3 extraction surgery, and the diagnosis probability of orthodontic class 3 non-extraction surgery. It may include at least one. In addition, the diagnosis result 342 is the non-extraction orthodontic diagnosis probability, the maxillary and mandibular first premolar extraction correction diagnosis probability, the upper and lower second premolar extraction correction diagnosis probability, the maxilla first premolar extraction correction diagnosis probability, and the maxilla first premolar mandible second It may include at least one of premolar extraction orthodontic diagnosis probability.

At this time, according to the diagnosis result 342, the diagnosis probability for the second grade extraction surgery orthodontics appears as 95%. This may mean that the success rate of orthodontic treatment may be higher than that of other surgical treatments when grade 2 orthodontic extraction is performed on the base of the test.

That is, as a result of the step (S230) of determining the surgical treatment or the non-surgical treatment, the medical treatment according to the dental condition of the examinee can be determined probabilistically.

On the other hand, according to another feature of the present invention, before the step of determining the surgical treatment or the non-surgical treatment (S230), when each medical image including the lateral facial region and/or the tooth region is an RGB color image, A step of preprocessing these medical images may be further performed.

More specifically, referring together with FIG. 5 , in the pre-processing step, the RGB color lateral facial region 312 and the tooth region 322 are converted to black and white, and the black and white converted lateral facial region or black and white converted tooth region is vectorized can be In this case, each of the lateral facial region 312 and the tooth region 322 converted into a black-and-white image may be vectorized in the direction of the pixel having the largest brightness difference value with respect to a plurality of pixels.

As a result of the pre-processing step, when the parafacial region 312 and the dental region 322 are input to the orthodontic classification model, the processing speed can be improved.

Finally, in the step (S240) of providing the predicted result, information on the surgical treatment or the non-surgical treatment determined as a result of the step (S230) of determining the surgical treatment or non-surgical treatment may be provided.

For example, referring back to FIG. 4 , in the step of providing the predicted result ( S240 ), the 'second-class extraction surgery' determined according to the tooth condition based on the subject's lateral facial region 312 and the tooth region 322 The surgical treatment of 'correction (95% diagnosis probability)' can be printed out and provided to the medical staff.

Accordingly, the medical staff, according to the information providing method according to various embodiments of the present invention, can be provided with information about the subject's orthodontics, so that they can make a decision and establish a treatment plan with a high probability of success.

Hereinafter, a learning method of the lateral facial region prediction model and the tooth region prediction model used in various embodiments of the present invention will be described with reference to FIGS. 6A to 6D . 6A to 6D are diagrams illustrating a process of generating image data for training of predictive models used in a method for providing orthodontic information and a device for providing orthodontic information using the same according to an embodiment of the present invention.

First, referring to FIGS. 6A and 6B , first, for learning the lateral facial region prediction model, a plurality of sample lateral head standard radiographic images are prepared. Then, in the plurality of lateral head standard radiographic images, the lateral facial region is indicated as a rectangular area, and then its coordinates are designated. Then, a json file including location information on the sample lateral facial region (box) formed for each of the plurality of lateral head standard radiographic images is prepared. More specifically, the json file is a name for each of the plurality of sample lateral head standard radiographic image files, and the position values (top, left, width and height) of the box formed for the sample lateral head area within the plurality of lateral head standard radiographic images. may include. As a result, a lateral head medical image for learning about the lateral facial region prediction model containing location information on the sample lateral facial region may be generated. On the other hand, medical imaging of the lateral head for learning

The lateral lateral facial region prediction model may be trained to predict a lateral lateral facial region, ie, a sample lateral facial region, in which position information is predetermined in the lateral cephalometric medical image for training.

Meanwhile, the lateral facial region prediction model may be a faster R-CNN-based AI model capable of detecting a region of interest in a lateral head medical image for training. However, it is not limited thereto.

Next, referring to FIGS. 6C and 6D , first, a plurality of sample intraoral images are prepared for learning the tooth region prediction model. Then, in the plurality of sample intraoral images, a box surrounding each area of the plurality of teeth (incisors, canines, premolars, molars) is formed, and thereafter, coordinates for each box are designated. Then, a json file including location information for a plurality of sample tooth regions (boxes) formed for each of a plurality of intraoral images is prepared. More specifically, the json file may include a name for each of a plurality of intraoral image files, and position values (top, left, width and height) of a box formed for each of a plurality of sampled tooth regions within a plurality of intraoral images. can As a result, an intraoral image for learning about the tooth area prediction model containing position information on the sample tooth area may be generated.

The tooth region prediction model may be trained to predict each of a plurality of tooth regions, that is, a sample tooth region, for which position information is predetermined in an intraoral image for training.

Meanwhile, the tooth region prediction model may be a faster R-CNN-based AI model capable of detecting a region of interest in an intraoral image for training. However, it is not limited thereto.

Predictive models used in a method for providing orthodontic information providing method and a device for providing orthodontic information using the same according to an embodiment of the present invention, by adopting the above learning algorithm, a parafacial region or tooth in a medical image The region of interest of the region can be predicted with high accuracy. However, the lateral facial region prediction model and the tooth region prediction model are not limited to the above-described ones and may be learned in more various ways.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: device for providing information about orthodontics
110: receiver
120: input unit
130: output unit
140: storage
150: processor
200: medical image
210: lateral head medical image
220: video in premises
310: lateral facial region prediction model
312: lateral facial region
320: tooth area prediction model
322: tooth area
330: orthodontic classification model
332: feature extraction layer
334: fusion layer
342: diagnosis result

Claims (21)

receiving a lateral head medical image or an intraoral image of the subject;
Predicting a paratrophic region or a tooth region including a paratrophic region in the paratrophic medical image or the intraoral image, and
A method for providing information on orthodontics, comprising the step of classifying a surgical or non-surgical treatment required for the subject based on the parafacial region or the tooth region. According to claim 1,
Predicting the parafacial area or the tooth area comprises:
Using a paratrophic facial region prediction model configured to predict a paratrophic facial region in a paratrophic medical image, comprising the step of predicting the paratrophic facial region,
The step of classifying surgical treatment or non-surgical treatment is,
A method of providing information on orthodontic treatment, comprising the step of classifying the necessary treatment for the subject into surgical treatment or non-surgical treatment based on the parafacial region. 3. The method of claim 2,
Predicting the lateral facial region comprises:
and estimating coordinates of the paratrophic facial region in the lateral head medical image by using the lateral facial region prediction model. According to claim 1,
Predicting the parafacial area or the tooth area comprises:
Using a tooth area prediction model configured to predict a tooth area within an intraoral image, predicting the tooth area,
The step of classifying surgical treatment or non-surgical treatment is,
A method of providing information on orthodontic treatment, comprising the step of classifying a treatment necessary for the subject based on the tooth area into a surgical treatment or a non-surgical treatment. 5. The method of claim 4,
Predicting the tooth area comprises:
Using the tooth region prediction model, the method of providing information on orthodontics comprising the step of predicting the coordinates of the tooth region in the intraoral image. According to claim 1,
The step of classifying the surgical treatment or non-surgical treatment is,
Using an orthodontic classification model configured to classify a necessary treatment as a surgical or non-surgical treatment based on a parafacial area or a dental area, classifying the necessary treatment for the subject. HOW TO PROVIDE INFORMATION. 7. The method of claim 6,
The orthodontic classification model is,
The method for providing information on orthodontics, further configured to classify a necessary treatment into a surgical treatment or a non-surgical treatment based on the parafacial area and the dental area. 7. The method of claim 6,
The orthodontic classification model is,
A method for providing information on orthodontics, comprising two independent feature extraction layers configured to extract features, and a fusion layer integrating two features extracted from the two feature extraction layers . 7. The method of claim 6,
The orthodontic classification model is,
The Two-Stream Convolutional Neural Network (CNN), a method of providing information about orthodontics. According to claim 1,
After predicting the paratrophic facial region or the tooth region, the method further comprising generating a box including the paratrophic facial region or the tooth region. According to claim 1,
The surgical treatment is
A method of providing information on orthodontics, which is at least one of orthodontic class 2 extraction surgery, orthodontic class 2 non-extraction surgery, orthodontic class 3 extraction surgery, and orthodontic class 3 non-extraction surgery. According to claim 1,
The non-surgical treatment is
At least one of non-extraction correction, upper and lower first premolar extraction correction, upper and lower second premolar extraction correction, maxillary first premolar extraction correction, and maxilla first premolar mandibular second premolar extraction correction, orthodontic information providing method. According to claim 1,
The step of classifying the surgical treatment or non-surgical treatment is,
Probabilistically predicting the surgical or non-surgical treatment,
After the classifying step,
The method of providing information about orthodontics, further comprising the step of providing a surgical treatment diagnosis probability or a non-surgical treatment diagnosis probability. According to claim 1,
After predicting the parafacial area or tooth area,
converting the lateral facial region or the tooth region to black and white, and
The method of providing information on orthodontics, further comprising the step of vectorizing the black-and-white converted side facial area or the black-and-white converted tooth area. According to claim 1,
Prior to the receiving step,
receiving a sample lateral head medical image or a sample intraoral image of a sample subject;
receiving an input for selection of a sample side facial region in the sample lateral head medical image or a selection for a specimen tooth region in the sample intraoral image;
The method of providing information on orthodontics, further comprising the step of generating a lateral head medical image for learning based on the coordinates of the sample lateral facial region or the specimen tooth region and the specimen lateral head medical image. 16. The method of claim 15,
The step of generating the lateral head medical image for learning comprises:
A method for providing information on orthodontics, comprising generating a json file or an xml file including information on the sample lateral head medical image and coordinates for the sample lateral facial region. According to claim 1,
After the classifying step,
Based on the classification result, further comprising the step of determining a treatment plan for the subject, orthodontic information providing method. a receiving unit configured to receive a lateral head medical image or an intraoral image of the subject; and
and a processor connected to the receiver;
The processor is
Predicting a paratrophic facial region or a dental region including a paratrophic facial region in the paratrophic medical image or the intraoral image, and based on the paratrophic facial region or the dental region, a surgical treatment or non-surgical treatment required for the subject A device for providing information about orthodontics, configured to classify treatments. 19. The method of claim 18,
The processor is
Using a paratrophic facial region prediction model configured to predict a paratrophic facial region within a paratrophic medical image, predicts the paratrophic facial region, and based on the paratrophic facial region, surgical or non-surgical treatment is required for the subject. A device for providing information about orthodontics, further configured to classify as a treatment. 19. The method of claim 18,
The processor is
Using a tooth area prediction model configured to predict a tooth area within an intraoral image, predicting the tooth area, and classifying a necessary treatment for the subject based on the tooth area as a surgical treatment or a non-surgical treatment, Device for providing information about orthodontics. 19. The method of claim 18,
The processor is
Using an orthodontic classification model configured to classify a necessary treatment as a surgical treatment or a non-surgical treatment based on the parafacial region or the dental region, further configured to classify the necessary treatment for the subject, providing information on orthodontics for device.

Images