KR20200012707A - Method for predicting anatomical landmarks and device for predicting anatomical landmarks using the same - Google Patents

Abstract

Provided are an anatomical landmark prediction method and a device using the same. The anatomical landmark prediction method comprises the steps of: receiving a facial head medical image of a subject; predicting a landmark in the facial head medical image by using a landmark prediction model configured to predict an anatomical landmark in the facial head medical image; and providing an image on which the predicted landmark is displayed.

Description

METHOD FOR PREDICTING ANATOMICAL LANDMARKS AND DEVICE FOR PREDICTING ANATOMICAL LANDMARKS USING THE SAME}

The present invention relates to a method for predicting anatomical landmarks and a device using the same, and more particularly, to a method for predicting anatomical landmarks and a device using the same, configured to predict anatomical measurement points used for orthodontic analysis.

In general, the state in which the teeth are not correct and the upper and lower teeth occlusion is called malocclusion. Such malocclusion not only causes functional problems such as chewing and pronunciation problems and aesthetic problems on the face but also health problems such as tooth decay and gum disease.

Orthodontic treatment can be performed as a method for making such malocclusion normal.

On the other hand, in orthodontic treatment, orthodontic analysis such as measuring the size of maxillofacial skeleton, measuring the growth direction of maxillofacial skeleton, and measuring the degree of protruding teeth for the subject performed in the treatment planning stage is closely related to the prognosis of orthodontic treatment. Can be associated with the In particular, in establishing an accurate treatment plan for orthodontic treatment, finding anatomically predetermined anatomical landmarks for metrology may be very important. To this end, medical practitioners have visually confirmed anatomical landmarks on the side head medical images obtained from the subject, and have performed the above-described orthodontic treatments. However, such a method may involve a problem that the location of anatomical landmarks may vary according to the skill of the medical practitioner, thereby reducing the accuracy of treatment planning.

As the accuracy of the treatment planning before orthodontic treatment is more demanded to improve the quality of medical services, the development of a method for accurately detecting anatomical landmarks is continuously required.

The background art of the invention has been written in order to facilitate understanding of the present invention. It should not be understood that the matters described in the background of the invention are present as prior art.

On the other hand, in order to solve the above-mentioned problems, dental orthodontic analysis programs have been introduced, which are configured to programmatically perform pre-orthodontic analysis. However, such an orthodontic analysis program may be cumbersome for the medical person as the anatomical landmarks must be input manually. Furthermore, such a prior art, in that the medical personnel still need to directly find anatomical landmarks based on the side head medical image, the accuracy of the analysis may vary depending on the skill of the medical personnel.

The inventors of the present invention, in order to solve the problems of the conventional orthodontic analysis program and to improve the accuracy of the pre-orthodontic analysis, the prediction learned to predict the landmark for correction by the data of the head head medical image It was recognized that the model could be used.

As a result, the inventor of the present invention, by using a predictive model to predict the position of the anatomical landmark for correction in the side head medical image for the subject, can provide information for orthodontic analysis A calibration analysis system has been developed.

More specifically, the inventors of the present invention, when using the two predictive models trained to predict the lateral facial region in the lateral head medical image, and predict anatomical landmarks for measurement within the predicted lateral facial region It is noted that the accuracy of the prediction can be increased.

As a result, a dental orthodontic analysis system using a predictive model configured to predict the predicted lateral facial region in the lateral head medical image and a predictive model configured to predict anatomical landmarks in the lateral facial region could be developed.

Accordingly, an object of the present invention is to predict a landmark in a medical image by using a landmark prediction model configured to predict an anatomical landmark for measurement based on the received side head medical image. To provide a method of predicting anatomical landmarks, configured to provide.

Another problem to be solved by the present invention is to predict the coordinates for the lateral facial region using a lateral facial region prediction model configured to predict the lateral facial region in the lateral head medical image, and use the landmark prediction model It provides a method for predicting anatomical landmarks, configured to predict landmarks within the facial area.

Another problem to be solved by the present invention is an anatomical landmark, configured to measure at least one selected from the size of the maxillofacial skeleton, the direction of growth of the maxillofacial skeleton and the degree of protruding teeth, based on the predicted landmark. To provide a prediction method.

Another problem to be solved by the present invention is a prediction unit for predicting anatomical landmarks, including a receiver configured to receive a side head medical image for a subject and a processor operatively connected to the receiver and configured to use various prediction models To provide a device.

The objects of the present invention are not limited to the above-mentioned objects, and other objects that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, there is provided a method for predicting anatomical landmarks according to an embodiment of the present invention. The method includes receiving a side head medical image for a subject and predicting a landmark in the medical image using a landmark prediction model configured to predict an anatomical landmark in the side head medical image. And providing an image on which the predicted landmark is displayed.

According to a feature of the present invention, the prediction method of the present invention comprises a lateral face in the medical image using a lateral face region prediction model configured to predict the side face region in the medical image, which is performed before the step of predicting the landmark. The method may further include predicting coordinates for the region. Furthermore, the step of predicting the landmark may further include predicting the landmark in the side face region using the landmark prediction model.

According to another aspect of the invention, the step of predicting the landmark, the x-axis and y for the landmark in the side of the facial region based on the medical image and the coordinates for the side-facial region predicted using the landmark prediction model Predicting the axis coordinates, and generating an xml, json, or csv file that includes the predicted landmark, the x and y axis coordinates. In addition, the providing may further include providing an image in which a landmark position is indicated in the lateral head medical image based on the lateral head medical image and the xml, json, or csv file.

According to another feature of the present invention, the landmark prediction model is a model trained to predict a landmark in a side head medical image based on a gradient boosting algorithm, and the side face region prediction model is a support. It may be a model trained to predict the side face region in the side head medical image based on a support vector machine algorithm (SVM).

According to another feature of the present invention, the side face region prediction model may include receiving a training side head medical image in which coordinates of the side face region are predetermined, and a pattern of the side face region and the side head medical image for learning. It may be a model learned through the step of predicting the side of the facial region in the training side-head medical image based on.

According to still another aspect of the present invention, the learning side head medical image comprises the steps of transforming a sample side head medical image for a sample subject to a histogram of oriented gradient (HOG), and coordinates of the side face region in the sample side head medical image. Based on the path of the sample side head medical image, it may be an image obtained by displaying a side face region on the HOG transformed sample side head medical image.

According to another feature of the invention, the prediction method of the present invention, based on the predicted landmarks, measuring at least one selected from the size of the maxillofacial skeleton, the growth direction of the maxillofacial skeleton and the degree of protruding teeth to the subject It may further include.

According to another feature of the present invention, when the received medical image is an RGB color image, the method may further include converting the medical image into a black and white image, and vectorizing the black and white image.

According to another aspect of the invention, the vectorizing step is to calculate the brightness difference value for each of a plurality of pixels adjacent to a certain pixel and any one selected from a plurality of pixels of the black and white image, and adjacent The method may include vectorizing the black-and-white image in the direction of the pixel having the greatest brightness difference value among the plurality of pixels.

According to another feature of the invention, the landmark is an A-point, a B-point, an anterior nasal spine that is predetermined as a cephalometric landmark for calibration. ), AN (Antegonial notch), Articulare, Artione, Basion, C (Cervical point), Condylion, Columella, Columella, CL (Corpus Left), Cosum (Dorsum) of Nose, Glabella, Gnathion, Gionion, Infradentale, Lower incisor crown tip, LIRT (Lower incisor root tip), LMDP (Lower molar distal) point), LMMP (Lower molar mesial point), Li (Labrale inferius), Ls (Labrale superius), LE (Lower embrasure), Lower lip, Menton, Nasion, Nasal, Nasal bridge), Orbitale, PM point, PNS (Posterior nasal spine), Poion, Pogonion, Pn (Pronasale), Pt (Pterygoid point), R1 point, R3 point, RD ( Ramus down), Sella, Sd (Supradentale), Soft tissue A point, Soft tissue B point, Gn '(Soft tissue Gnathion), Me' (Soft tissue Menton), N '(Soft tissue Nasion), Pg' ( Soft tissue Pogonion, Stmi (Stomion inferius), Stms (Stomion superius), SM point (Submandibular point), Sn (Subnasale), UICT (Upper incisor crown tip), UIRT (Upper incisor root tip), UMDP (Upper molar distal) point, UMMP (Upper molar mesial point), UE (Upper embrasure), Upper lip, Mendibular outline 1, Mendibular outline 2, Mendibular outline 3, Mendibular outline 4, mendibular outline 5, mendibular outline 6, maxilla outline 1, maxillary outline 2, maxillary outline 3, maxillary outline 4, maxillary outline 5, maxillary outline Outline 6, maxillary outline 7, maxillary outline 8, maxillary outline 9, maxillary outline 10, maxillary outline 11, Pisiseu it may be the outline (outline Symphysis) 1, the center pisiseu outline 2, 3 and the center core pisiseu outline pisiseu outline 4 selected from the group consisting of at least one.

According to another aspect of the present invention, the landmark prediction model, receiving a training side head medical image predetermined coordinates of a plurality of landmarks with respect to the side face region, and a plurality of lands in the training side head medical image The model may be trained by predicting coordinates of a plurality of landmarks in the training-side head medical image based on a shape formed by the coordinates of the mark.

In order to solve the above problems, there is provided a device for predicting anatomical landmarks according to an embodiment of the present invention. The device includes a receiver configured to receive side head medical images of a subject, and a processor operatively coupled to the receiver. In this case, the processor is configured to predict the landmark in the medical image using a landmark prediction model configured to predict the anatomical landmark in the side head medical image.

According to a feature of the present invention, the processor predicts coordinates for the lateral face region in the medical image using a lateral face region prediction model configured to predict the side face region in the medical image, and uses the landmark prediction model. To predict landmarks within the side face area.

According to another feature of the invention, the processor predicts the x-axis and y-axis coordinates for the landmark in the side-face region based on the predicted side-face region and the medical image using the landmark prediction model. Generating an xml, json or csv file including the predicted landmark, the x-axis and y-axis coordinates, the landmark in the lateral head medical image based on the lateral head medical image and the xml, json or csv file The location of may be further configured to provide an image, indicated.

According to another feature of the invention, the predictive device of the present invention is configured to measure at least one selected from the size of the maxillofacial skeleton, the growth direction of the maxillofacial skeleton and the degree of protruding teeth on the subject, based on the predicted landmark. The measurement unit may further include.

 According to another feature of the invention, the prediction device of the present invention, if the received medical image is an RGB color image, further comprises a data preprocessor, configured to convert the medical image to a black and white image, and to vectorize the black and white image Can be.

According to another feature of the present invention, the data preprocessor calculates brightness difference values of a selected pixel of a plurality of pixels included in a black and white image and a plurality of pixels adjacent to the predetermined pixel, respectively, and the plurality of adjacent pixels Among them, it may be configured to vectorize the black-and-white image in the direction of the pixel having the greatest brightness difference value with any pixel.

The present invention provides a method for predicting anatomical landmarks using a lateral facial region prediction model configured to predict lateral facial regions with respect to lateral head medical images and a landmark prediction model configured to predict anatomical landmarks within the lateral facial region; By providing a device using the same, there is an effect that can provide information for orthodontic analysis for the subject.

More specifically, the present invention may predict and provide anatomical landmarks, which are measurement points for orthodontic analysis, to enable accurate treatment planning before orthodontic treatment for a subject. Accordingly, the present invention can contribute to providing an accurate and effective calibration method tailored to the condition of the individual.

For example, by providing anatomical landmarks, the medical practitioner can more accurately and easily measure the size of the maxillofacial skeleton, measure the growth direction of the maxillofacial skeleton, and analyze the extent of dentition.

In addition, the present invention has the effect of predicting and providing a highly accurate measurement point of the anatomical landmarks in the received side head medical image regardless of the skill of the medical personnel.

The present invention has the effect of providing highly accurate prediction information by processing the training head-side head medical image to more accurately predict the head-side facial region with respect to the head-side head medical image and learning the prediction model.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1A illustrates a dental orthodontic analysis system using a device for predicting anatomical landmarks according to an embodiment of the present invention. It is shown.
1B exemplarily illustrates a configuration of a device for predicting anatomical landmarks according to an embodiment of the present invention.
2A illustrates a procedure of a method for predicting anatomical landmarks according to an embodiment of the present invention.
2B exemplarily illustrates a procedure of landmark prediction for a side head medical image by a method of predicting anatomical landmarks according to an embodiment of the present invention.
3A to 3C are diagrams illustrating methods of generating anatomical landmarks according to an embodiment of the present invention, and for generating training head-to-head medical image data of a side-facial region prediction model used in a device for predicting anatomical landmarks using the same. It is shown.
4A to 4D illustrate a method of generating anatomical landmarks according to an embodiment of the present invention and a procedure for generating side head medical image data for learning a landmark prediction model, which is used in a device for predicting anatomical landmarks using the same. It is shown.
FIG. 5 illustrates a landmark predicted by an anatomical landmark prediction method and a device for predicting anatomical landmark using the same, according to an embodiment of the present invention.

Advantages of the invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and only the present embodiments make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims.

Shapes, sizes, ratios, angles, numbers, and the like disclosed in the drawings for describing the embodiments of the present invention are exemplary, and thus, the present invention is not limited thereto. In addition, in describing the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When 'comprises', 'haves', 'consists of' and the like mentioned in the present specification are used, other parts may be added unless 'only' is used. In the singular form, the plural includes the plural unless specifically stated otherwise.

In interpreting a component, it is interpreted to include an error range even if there is no separate description.

Each of the features of the various embodiments of the present invention may be combined or combined with each other, partly or wholly, and various technically interlocking and driving are possible as one skilled in the art can fully understand, and each of the embodiments may be independently implemented with respect to each other. It may be possible to carry out together in an association.

For clarity of interpretation of the present specification, the terms used herein will be defined below.

As used herein, the term "lateral head medical image" may refer to any image including a profile of a subject received from a medical image diagnosis apparatus. Preferably, the side head medical image disclosed herein may be, but is not limited to, a side head standard radiographic image. Meanwhile, the lateral head medical image may be a 2D image, a 3D image, a still image of one cut, or a video composed of a plurality of cuts. For example, when the lateral head medical image is a video composed of a plurality of cuts, landmarks for each of the lateral head medical images may be predicted according to an anatomical landmark prediction method according to an embodiment of the present invention. have. As a result, the present invention may provide information predicted in real time by performing prediction of anatomical landmarks simultaneously with the reception of the side head medical image from the imaging apparatus.

Meanwhile, the term "anatomical landmark" as used herein may mean an anatomically predetermined landmark as a measurement point for orthodontics. In this case, the anatomical landmark may be, but is not limited to, a cephalometric landmark.

On the other hand, in establishing an accurate treatment plan for orthodontic treatment, finding anatomical landmarks can be very important. At this time, the position of the anatomical landmarks determined according to the proficiency of the medical practitioner may vary, which may involve a problem of inaccurate treatment planning.

To solve this problem, a predictive model trained to predict anatomical landmarks within the lateral head medical image may be used.

 As used herein, the term "landmark prediction model" may be a model configured to predict a landmark point for measurement with respect to a side head medical image. For example, the landmark prediction model may be a model trained to predict a landmark in the received head-to-head medical image based on a gradient boosting algorithm. More specifically, the landmark prediction model is trained by predicting coordinates of a plurality of landmarks in the training head to head medical image based on the shape formed by the coordinates of the plurality of landmarks in the training head to head medical image. It may be a model.

At this time, A-point, B-point, ANS (Anterior nasal spine), AN (Antegonial notch), Articura, which are previously determined as a cephalometric landmark for correction Articulare, Basion, C (Cervical point), Condylion, Columella, Columella, CL (Corpus Left), Dorsum of Nose, Glabella, Gna Gnathion, Gion, Infradentale, LICT (Lower incisor crown tip), LIRT (Lower incisor root tip), LMDP (Lower molar distal point), LMMP (Lower molar mesial point), Li (Labrale inferius), Ls (Labrale superius), LE (Lower embrasure), Lower lip, Menton, Nasion, Nasal bridge, Orbitale, PM branch , PNS (Posterior nasal spine), Poion, Pogonion, Pn (Pronasale), Pt (Pterygoid point), R1 point, R3 point, RD (Ramus down), Sella, Sd ( Supradentale), Soft tissue A point, Soft tissue B point, Gn '(Soft tissue Gnathion), Me' (Soft tissue Menton), N '(Soft tissue Nasion), Pg' (Soft tissue Pogonion), Stmi (Stomion inferius), Stms (Stomion superius ), SM points (Submandibular point), Sn (Subnasale), UICT (Upper incisor crown tip), UIRT (Upper incisor root tip), UMDP (Upper molar distal point), UMMP (Upper molar mesial point), UE (Upper embrasure) ), Upper lip, Mendibular outline 1, Mendibular outline 2, Mendibular outline 3, Mendibular outline 4, Mendibular outline 5, Mendibular outline 6, Maxil Maxilla outline 1, maxillary outline 2, maxillary outline 3, maxillary outline 4, maxillary outline 5, maxillary outline 6, maxillary outline 7, maxillary outline 8 , Maxillary outline 9, maxillary outline 10, maxillary outline 11, simphysis outline 1, simpy Scan-out line 2, line 3 and the center core pisiseu out pisiseu may be an outline 4 selected from the group consisting of at least one, without being limited thereto.

Meanwhile, the landmark prediction model may be a model trained to predict a location of a landmark based on a medical image cropped to include only a side-facial face region to be measured in the side-head medical image. . For example, the landmark prediction model may be a model trained to predict the x-axis and y-axis coordinates for the landmark in the parietal face area based on the coordinates for the parietal face area and the side head medical image. By this learning method, the landmark prediction model can predict the landmark with higher accuracy than when the landmark prediction model is configured to predict the landmark in the whole side head medical image.

As used herein, the term "lateral face region prediction model" may be a model configured to predict a side face region including a side face region, which is a target region to be measured for orthodontic treatment on the side head medical image. For example, the side face facial region prediction model may be a model trained to predict the side face facial region in the side head medical image based on a support vector machine algorithm. More specifically, the side face region prediction model receives a training side head medical image whose coordinates of the side face region are predetermined, and based on the pattern of the side face region and the training side head medical image, the training side head medical image is obtained. It may be a model trained to predict the lateral facial region within.

At this time, the "flag facial area" may have a different pixel value, texture than another area, for example, a background area. Thus, the side face facial region prediction model may predict the side face facial region based on the pixel value or texture.

Meanwhile, the lateral head medical image converts a sample lateral head medical image of a sample subject to a histogram of oriented gradient (HOG), and based on the coordinates of the lateral face region in the sample side head medical image and the path of the sample side head medical image. It may be an image obtained by displaying a side face region on a HOG transformed sample side head head medical image. However, the learning side head medical image is not limited thereto.

As described above, the predictive models used in various embodiments of the present invention may be used singly or in combination in the prediction of anatomical landmarks.

Hereinafter, a dental orthodontic analysis system using a device for predicting anatomical landmarks and a device for predicting anatomical landmarks will be described with reference to FIGS. 1A and 1B.

1A illustrates a dental orthodontic analysis system using a device for predicting anatomical landmarks according to an embodiment of the present invention. 1B exemplarily illustrates a configuration of a device for predicting anatomical landmarks according to an embodiment of the present invention.

First, referring to FIG. 1A, a side head medical image of a subject may be obtained by the side head measurement radiographic imaging apparatus 300. In this case, the acquired side head medical image may be a side head standard radiographic image. The side head medical image obtained by photographing the profile of the subject is received by the anatomical landmark predicting device 100 according to an embodiment of the present invention.

The anatomical landmark prediction device 100 predicts the lateral facial region based on the received lateral head medical image and predicts a landmark within the predicted lateral facial region.

More specifically, referring to FIG. 1B, the device 100 for predicting anatomical landmarks includes a receiver 110, an input 120, an output 130, a storage 140, and a processor 150. do.

In detail, the receiver 110 may be configured to receive a side head medical image of a subject from the side head measurement radiographic apparatus 300. As described above, the side head medical image obtained by the receiver 110 may be a side head standard radiographic image, but is not limited thereto. Meanwhile, the receiver 110 may be further configured to transmit the acquired side head medical image to the processor 150 to be described later. Furthermore, the receiver 110 may receive a side head medical image from the outside. In this case, the side head medical image obtained through the receiver 110 may include a side face region.

The input unit 120 may set the device 100 for predicting anatomical landmarks, and may direct the operation of the receiver 110 described above. The input unit 120 may be a keyboard, a mouse, or a touch screen panel, but is not limited thereto.

On the other hand, the output unit 130 may visually display the side head medical image obtained from the receiver 110. In addition, the output unit 130 may be configured to display the location information of the side face region or the landmark determined in the side head medical image by the processor 150. However, the present invention is not limited thereto, and the output unit 130 may be configured to display more various information determined by the processor 150 for prediction of anatomical landmarks.

The storage unit 140 may be configured to store side head medical images of a subject acquired through the receiver 110, and store instructions of the device 100 for predicting anatomical landmarks set through the input unit 120. Can be. Further, the storage 140 is configured to store the results predicted by the processor 150 to be described later. However, without being limited to the foregoing, the storage 140 may store more various information determined by the processor 150 for prediction of anatomical landmarks.

Meanwhile, the processor 150 may be a component for providing accurate prediction results for the device 100 for predicting anatomical landmarks. At this time, in order to predict the anatomical landmark, the processor 150 may be configured to predict the lateral facial region with respect to the lateral head medical image, and predict the anatomical landmark within the predicted facial region. For example, the processor 150 is trained to predict the lateral facial region within the lateral head medical image for the subject obtained from the receiver 110, and learn to predict anatomical landmarks within the lateral facial region. Can be configured to use the predictive model. In this case, the model trained to predict the side face region may be based on a support vector machine algorithm, and the model trained to predict a landmark may be based on a gradient boosting algorithm, but is not limited thereto. For example, in various examples of the present invention, prediction models used in various embodiments of the present invention may include a deep neural network (DNN), a convolutional neural network (CNN), a deep convolution neural network (DCNN), and a recurrent neural (RNN). Networked facial regions and landmarks within medical images based on Network (RBM), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Single Shot Detector (SSD), You Only Look Once (YOLO) models, or U-net It may be a model trained to predict.

On the other hand, according to various embodiments of the present invention, the device for predicting anatomical landmarks is derived from the size of the maxillofacial skeleton, the growth direction of the maxillofacial skeleton and dentition for the subject based on the landmark predicted by the processor 150 The measurement unit may further include a measurement unit configured to measure at least one selected from the degree.

Furthermore, according to various embodiments of the present disclosure, when the side head medical image received by the receiver 110 is an RGB color image, the device for predicting anatomical landmarks may convert the side head medical image into a black and white image. The apparatus may further include a data preprocessor configured to vectorize the black and white image.

Hereinafter, a method of predicting anatomical landmarks according to an embodiment of the present invention will be described in detail with reference to FIGS. 2A to 2B. 2A illustrates a procedure of a method for predicting anatomical landmarks according to an embodiment of the present invention. FIG. 2B exemplarily illustrates a procedure of predicting anatomical landmarks on a side head medical image by a method of predicting anatomical landmarks according to an embodiment of the present invention.

Referring to FIG. 2A, a procedure for predicting anatomical landmarks according to an embodiment of the present invention is as follows. First, a side head medical image of a subject is received (S210). Next, the lateral facial region is predicted using a lateral facial region prediction model configured to predict the lateral facial region with respect to the lateral head medical image (S220). Next, the landmark, which is an anatomical measurement point, is predicted using the landmark prediction model configured to predict the landmark (S230). Finally, it provides a predicted result (S240).

For example, referring to FIG. 2B, step S210 of receiving a side head medical image 2 may receive a side head medical image 212 of a subject. In this case, the side head medical image 212 may be a side head standard radiographic image, but is not limited thereto.

According to an embodiment of the present invention, in step S210 of receiving the side head medical image 2, the pre-processed side head medical image 212 has a predetermined pixel to enable fast analysis of the side head medical image 212. Can be further received. If not, after the step S210 of receiving the side head medical image (S210), to adjust the size or to adjust the contrast, resolution, contrast, or left-right symmetry with respect to the received side head medical image 212 The preprocessing step for the side head medical image 212 may be further performed. For example, if the side head medical image 212 is an RGB color image, in the preprocessing step the side head medical image 212 is converted to a black and white image, and vectorization may be performed. More specifically, the side head medical image 212 converted to the black and white image may be vectorized in the direction of the pixel having the largest brightness difference value for the plurality of pixels.

As a result of the preprocessing step, the parenchymal head medical image 212 has the resolution or size required by the predictive model, which will be described later, and may be smaller in resolution or size than the original parenchymal head medical image, so that the processing speed in the predictive model Can be improved.

Next, referring to FIG. 2B, in step S220 of predicting the side face region, the side head medical image obtained in step S210 of receiving the side head medical image to the side face region prediction model 222. 212) is input. At this time, the side face facial prediction model 222 may predict the side face region based on the coordinates of the side face region in the side head medical image 212 and the pattern of the side head medical image 212, but is not limited thereto. It is not.

On the other hand, as a result of the step of predicting the side facial region (S220), a side head medical image 224 in which the side head region is predicted is obtained, and the side head medical image 224 in which the side head region is predicted is obtained from the side head facial region. It can be cropped to include.

Next, referring to FIG. 2B, the cropped side face region 226 is input to the landmark prediction model 232 in step S230 of predicting the landmark. In this case, the landmark prediction model 232 may predict the location of the landmark, more specifically, the x-axis and y-axis coordinates based on the cropped side face region 226, but is not limited thereto.

According to an embodiment of the present invention, in the step of predicting the landmark (S230), the xml, json or csv file including the landmark predicted by the landmark prediction model 232, the x-axis and the y-axis coordinate is stored. Can be generated.

As a result of the step of predicting the landmark (S230), the side head medical image 234 with the landmark predicted may be obtained. At this time, the landmark head predicted side head medical image 234 is the head head medical image 212 and the lateral head medical image based on the xml, json or csv file generated in the step (S230) of predicting the landmark described above The location of the landmark may be displayed in 212, but is not limited thereto.

Referring back to FIG. 2B, in step S240 of providing a prediction result, a side head medical image 234 in which the landmark acquired in step S230 is predicted may be provided.

Meanwhile, in the method for predicting anatomical landmarks according to an embodiment of the present invention, the size of the maxillofacial skeleton, the growth direction of the maxillofacial skeleton, and the degree of protruding teeth in the subject are selected based on the measured points of the predicted landmark. Measuring at least one may be further performed.

By the method of predicting anatomical landmarks according to an embodiment of the present invention, the anatomical landmarks can be predicted with high accuracy in the lateral head medical images obtained from the subject, and accurate measurement for orthodontic treatment is possible. Can be. Thus, the present invention can be applied to the orthodontic analysis system based on the side head medical image.

Hereinafter, a method of learning a side face facial region prediction model used in various embodiments of the present invention will be described with reference to FIGS. 3A to 3C. 3A to 3C are diagrams illustrating methods of generating anatomical landmarks according to an embodiment of the present invention, and for generating training head-to-head medical image data of a side-facial region prediction model used in a device for predicting anatomical landmarks using the same. It is shown.

Referring to FIG. 3A, first, a plurality of side head standard radiographic images are prepared for learning a side face region prediction model. Then, in the plural lateral head standard radiographic images, the lateral face region including the landmark of the measurement point is coordinated with a rectangular area. Next, an xml file is prepared that includes a location (image path) in which each of the plurality of side head standard radiographic images is stored and coordinates for the side face region (box) formed in each image. More specifically, the xml file includes names for each of the plurality of side head standard radiographic images, and position values (top, left, width, and height) of boxes formed with respect to the side face region in the plurality of side head standard radiographic images. can do.

Meanwhile, referring to (a) and (b) of FIG. 3B, pre-processing, such as a histogram of oriented gradient (HOG) transformation, may be performed on the prepared plurality of head-to-head radiographic images. In the pre-processing step, the side head standard radiographic images may have a boundary of an object of the side face region as the contrast difference value between adjacent pixels in the plurality of pixels of the image is calculated and converted. More specifically, in the preprocessing step, the side head standard radiographic images are converted into black and white images, and for a specific pixel among the pixels constituting each image, a difference in brightness value with eight adjacent pixels is calculated, and the brightness value difference is Each image may be vectorized in the largest direction (for example, directions 2 to 8 in (a) of FIG. 3B). As a result, it is possible to obtain a radiographic image of the side head to the head with a clear boundary.

Referring to FIG. 3C, the position of the side face region is based on the HOG transformed side head specification radiographic image described above with reference to FIG. 3B based on an xml file including coordinates for the image path and the side face region described above with reference to FIG. 3A. Is displayed. As a result, the training-side side head medical image for the side-face region prediction model, which contains the position information on the side-face region, may be generated.

Meanwhile, the lateral face region prediction model may be based on an SVM algorithm configured to classify a category, ie, lateral face region, by recognizing a pattern with respect to the training side head medical image.

For example, the prediction of the side face region by the side face region prediction model may be performed based on the SVM machine learning algorithm of Equation 1 below.

[Equation 1]

[Equation 1]

가 최소값을 갖게 하도록 구성될 수 있다. This algorithm is expressed in Equation 1

Can be configured to have a minimum value.

에서 'C-value'는 1로 설정될 수 있다. 나아가, [수학식 1]에서

의 코스트 함수에서, 'Cost'를 부과하지 않는 폭을 의미하는 파라미터인 'epsion' (ξ) 값은 0.01일 수 있다. In this case, the parameter 'C-vlaue' ( C ), which determines the 'Cost' of the degree that allows the misclassification during machine learning, may be 1. That is, the cost function in [Equation 1]

'C-value' can be set to '1'. Furthermore, in [Equation 1]

In the cost function of, 'epsion' (ξ), a parameter representing a width that does not impose 'Cost', may be 0.01.

However, the learning factor values of the parameters input for learning are not limited thereto.

In the method of predicting anatomical landmarks according to an embodiment of the present invention and a device for predicting anatomical landmarks using the same, the side-facial region prediction model, which adopts the above algorithm, has a side-head in the side-head medical image. Facial area can be predicted with high accuracy. However, the lateral face region prediction model is not limited to the above and can be learned in more various ways.

Hereinafter, a learning method of a landmark prediction model used in various embodiments of the present invention will be described with reference to FIGS. 4A to 4D. 4A to 4D illustrate a method of generating anatomical landmarks according to an embodiment of the present invention and a procedure for generating side head medical image data for learning a landmark prediction model, which is used in a device for predicting anatomical landmarks using the same. It is shown.

Referring to FIG. 4A, landmarks preset for learning a landmark prediction model are A-point, B-point, ANS anatomically predetermined as measurement points for orthodontic treatment. (Anterior nasal spine), AN (Antegonial notch), Articuree, Basion, C (Cervical point), Condylion, Columella, CL (Corpus Left) , Dorsum of Nose, Glabella, Gnathion, Gionion, Infradentale, Lower incisor crown tip, LIRT (Lower incisor root tip), LMDP (Lower molar distal point), LMMP (Lower molar mesial point), Li (Labrale inferius), Ls (Labrale superius), LE (Lower embrasure), Lower lip, Menton, Nasion , Nasal bridge, Orbitale, PM branch, PNS (Posterior nasal spine), Poion, Pogonion, Pn (Pronasale), Pt (Pterygoid point), R1 branch, R3Point, RD (Ramus down), Sella, Sd (Supradentale), Soft tissue A point, Soft tissue B point, Gn '(Soft tissue Gnathion), Me' (Soft tissue Menton), N '(Soft tissue Nasion) ), Pg '(Soft tissue Pogonion), Stmi (Stomion inferius), Stms (Stomion superius), SM point (Submandibular point), Sn (Subnasale), UICT (Upper incisor crown tip), UIRT (Upper incisor root tip), Upper molar distal point (UMDP), upper molar mesial point (UMMP), upper embrasure (UE), upper lip, mandibular outline 1, mendibular outline 2, mendibular outline 3 , Mendibular Outline 4, Mendibulous Outline 5, Mendibulous Outline 6, Maxillary Outline 1, Maxillary Outline 2, Maxillary Outline 3, Maxillary Outline 4, Maxillary Outline Line 5, maxillary outline 6, maxillary outline 7, maxillary outline 8, maxillary outline 9, maxillary outline 10, maxillary Smiling line 11, the center pisiseu outline (outline Symphysis) 1, the center pisiseu outline 2, the center pisiseu may include the outline 3 and the center line 75 pisiseu out temporal measurement radiation landmarks 4. However, the anatomical landmarks are not limited thereto, and may be easily changed according to the facial anatomical structure of the examinee, the type of the cephalal head medical image obtained, and the like.

Referring to FIG. 4B, first, an image in which a side face region is cropped is prepared for a plurality of side head standard radiographic images for learning a landmark prediction model. Then, in the cropped plurality of side face images, coordinates for each of the plurality of landmarks are specified. Next, an xml file is prepared that includes a location (image path) in which each of the plurality of side face images are stored and coordinates for a plurality of landmarks existing in each image. More specifically, the formed xml file includes a name for each of a plurality of side face image files, a plurality of side face images, that is, position values (top, left, width and height) of a box formed with respect to the side face region, It may include a name for each of the landmarks and the x and y axis coordinates.

Referring to FIG. 4C, the plurality of landmarks for each of the plurality of landmarks are based on an xml file including x-axis and y-axis coordinates of the image path and each of the plurality of landmarks in the plurality of side-facial facial images described above in FIG. 4B. The location is displayed. As a result, a training-side head medical image for a landmark prediction model including location information of a plurality of landmarks may be generated.

On the other hand, the learning side head medical image used for learning the landmark prediction model is not limited thereto.

For example, referring to (a) of FIG. 4D, the learning side head medical image may be an image having a randomized size of an image including a side face region including a landmark. More specifically, the random box area (yellow box) between the area created by the leftmost point, the rightmost point, the highest point, and the lowest point (yellow box) and the outermost area (red box) of the original image is a random function. It can be set and used as a learning video.

Referring to (b) of FIG. 4D, the training head-side head medical image for the landmark prediction model may be rotated within an angle, for example, within 10 degrees with respect to the same head-side head medical image. As a result, a plurality of learning side head medical images may be acquired according to angle diversification. More specifically, the coordinate values of the xml file may also be rotated according to the angle diversification, so that a plurality of learning side head medical images including various location information with respect to the landmark may be obtained. Thus, the landmark prediction model may improve the prediction level as the learning amount increases.

Referring to (c) of FIG. 4D, brightness and contrast may be randomized for the training-side head medical images for the landmark prediction model. Accordingly, a plurality of learning side head medical images may be obtained according to the brightness and contrast randomization. More specifically, the brightness, contrast, and gamma random values (using a random function) of the image are adjusted while the coordinate values for each landmark are fixed with respect to the learning side head medical image. Thus, the landmark prediction model can be trained to predict landmarks at any aspect ratio. As a result, the landmark prediction model is improved in performance to predict landmarks with high accuracy for various side head medical images.

Meanwhile, the landmark prediction model may be based on a gradient boosting algorithm configured to predict the position of the landmark based on a form formed by the plurality of landmarks.

For example, the position prediction of the landmark in the landmark prediction model may be performed based on a machine learning algorithm of gradient boosting of Equation 2 below.

[Equation 2]

는 복수회의 예측 단계 중 특정한 예측 단계에서 예측된 형태를 의미한다. 이러한 알고리즘을 통해 특정한 예측 단계 이전에 형성된 예측 형태에서 업데이트된 벡터를 더함으로써 다음 단계의 예측 형태를 얻게 되면서, 최종적으로 복수개의 랜드마크들이 이루는 형상이 예측될 수 있다.Here, S means a shape formed by a position value of each of the plurality of landmarks, I means a training side head medical image (Image),

Denotes a shape predicted in a specific prediction step among a plurality of prediction steps. Through this algorithm, the shape of the plurality of landmarks can be predicted while finally obtaining the prediction form of the next step by adding the updated vector from the prediction form formed before the specific prediction step.

At this time, the 'Cascade depth' which means the number of 'Cascade' which means the process of continuous prediction during machine learning may be 10. That is, t may be 10 in [Equation 2]. Furthermore, the oversampling amount can be set to 500.

Meanwhile, the gradient boosting algorithm is composed of k weak learners connected in series and may be represented by Equation 3 below.

[Equation 3]

는 각각의 약한 학습기를 의미할 수 있다. 이때, 약한 학습기에서 학습률을 의미하는 'v-value' (v) 은 0.1일 수 있고, 약한 학습기의 개수를 의미하는 'Number of trees per casade level' (k) 은 500일 수 있다. here,

May mean each weak learner. In this case, the 'v-value' ( v ), which means the learning rate in the weak learner, may be 0.1, and the 'Number of trees per casade level' ( k ), which means the number of the weak learners, may be 500.

의 약한 학습기는, 중요도에 따라 비교 특성의 우선 순위를 정하고, 기준에 따라 분기해 나가는 과정을 반복하여 낮은 특징까지 내려가는 결정 트리의 회귀 트리 (regression tree) 일 수 있다. Meanwhile,

The weak learner may be a regression tree of a decision tree that prioritizes comparative features according to their importance and branches to the criteria and then descends to lower features.

At this time, in the prediction of the position of the landmark, the comparison characteristic of the branch at each node of the regression tree is the difference in brightness between two adjacent pixels, which means the number of longest nodes from the top node to the lowest node of the regression tree. 'Tree depth' can be set to 5.

However, the learning factor values of the parameters input for learning are not limited thereto.

The landmark prediction model, which is used in the method for predicting anatomical landmarks and the device for predicting anatomical landmarks using the same, according to an embodiment of the present invention, is a landmark in the side head medical image according to the above algorithm. Can be predicted with high accuracy. However, the landmark prediction model is not limited to the above, but can be learned in more various ways.

Example 1 Prediction of Landmark Position Using Side-face Facial Region Prediction Model and Landmark Prediction Model

Hereinafter, with reference to FIG. 5, the prediction result of the landmark position using the side-face facial region prediction model and the landmark prediction model will be described. FIG. 5 illustrates a landmark predicted by an anatomical landmark prediction method and a device for predicting anatomical landmark using the same, according to an embodiment of the present invention.

Referring to FIG. 5, a lateral face region prediction model predicts a lateral face region and a plurality of landmarks within the lateral face region where the landmark prediction model is predicted with respect to the received side head medical image. As a result, as the medical image predicting the anatomical landmark is provided, the medical person may acquire information for orthodontic analysis of the subject. However, the effects of the present invention are not limited thereto.

For example, the present invention may enable accurate treatment planning before orthodontic treatment for a subject by providing a predictive anatomical landmark, which is a measurement point for orthodontic analysis. Accordingly, the present invention can contribute to providing an accurate and effective calibration method tailored to the condition of the individual. For example, by providing anatomical landmarks, the medical practitioner can more accurately and easily measure the size of the maxillofacial skeleton, measure the growth direction of the maxillofacial skeleton and analyze the extent of dentition.

In addition, the present invention has the effect of predicting and providing a highly accurate measurement point of the anatomical landmarks in the received side head medical image regardless of the skill of the medical personnel.

Thus, the present invention can be applied to the orthodontic analysis system based on the side head medical image.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: device for predicting anatomical landmarks
110: receiver
120: input unit
130: output unit
140: storage unit
150: processor
212: lateral tofu medical image
222: side face prediction model
224: Lateral head medical image with predicted side facial area
226: cropped side hair facial area
232: landmark prediction model
234: landmark head tofu medical image predicted
300: side head measurement radiographic apparatus

Claims (18)

The present invention relates to a method for predicting anatomical landmarks performed by a device for predicting anatomical landmarks including a processor.
Receiving a side head medical image for the subject;
Anatomical landmarks in the lateral head medical image
Predicting the landmark in the side head medical image using a landmark prediction model configured to predict
And providing an image displaying the predicted landmark. The method of claim 1,
The method,
Performed before the step of predicting the landmark,
Predicting coordinates for the side face region in the side head medical image using a side face region prediction model configured to predict a side face region including the side face region in the side head medical image; and,
Predicting the landmark,
Predicting the landmark in the side face region using the landmark prediction model. The method of claim 2,
Predicting the landmark,
Predicting x-axis and y-axis coordinates for the landmark in the parietal face region based on the predicted coordinates for the parietal face region and the parietal head medical image using the landmark prediction model, and
Generating at least one file of xml, json and csv including the predicted landmark, the x and y axis coordinates,
The providing step,
And providing an image in which the location of the landmark is displayed within the lateral head medical image based on the lateral head medical image and at least one file of the xml, json, and csv. Forecast method. The method of claim 2,
The landmark prediction model,
A model trained to predict the landmark within the lateral head medical image based on a gradient boosting algorithm,
The side face facial region prediction model,
Based on the Support Vector Machine algorithm
And a model trained to predict the parietal facial region within the parietal head medical image. The method of claim 2,
The side face facial region prediction model,
Receiving a training side head medical image whose coordinates of the side face region are predetermined; and
And a model trained through the step of predicting the lateral face region in the training lateral head medical image based on the coordinates of the lateral face region and the pattern of the training side head medical image. The method of claim 5,
The learning side head tofu medical image,
Specimen side head tofu medical image of HOG (Histogram of
Converting Oriented Gradient, and
An image obtained by displaying the side face region on the HOG-converted sample side head medical image based on the coordinates of the side face region in the sample side head medical image and the path of the sample side head medical image; Method of predicting anatomical landmarks. The method of claim 1,
The method,
The size of the maxillofacial skeleton for the subject, based on the predicted landmarks,
And measuring at least one selected from a growth direction of the maxillofacial skeleton and a degree of protrusion of the dental fissure. The method of claim 1,
The method,
If the received side head medical image is an RGB color image,
Converting the side head medical image into a black and white image, and
Vectorizing the black and white image, the method of predicting anatomical landmarks. The method of claim 8,
The vectorizing step,
Calculating brightness difference values of a selected one of a plurality of pixels of the black and white image and a plurality of pixels adjacent to the arbitrary pixel, and
Vectorizing the black-and-white image in a direction of a pixel having the greatest brightness difference value with the arbitrary pixel among the plurality of adjacent pixels. The method of claim 1,
The landmark is a side head measurement radiographic landmark for correction
A-point, B-point, ANS (Anterior nasal spine), AN (Antegonial notch), Articulare, Basion, C (Cervical point), Condylion, Columella, Colpus Left, CL (Dorsum of Nose), Glabella, Gnathion, Gionion, Infradentale (Lower incisor crown tip), LICT (Lower incisor root tip), LIRT (Lower molar distal point), LMMP (Lower molar mesial point), LMMP (Labrale inferius), Ls (Labrale superius) , LE (Lower embrasure), Lower lip, Menton, Nasion, Nasal bridge, Orbitale, PM branch, PNS (Posterior nasal spine), Porion ( Porion, Pogonion, Pn (Pronasale), Pt (Pterygoid point), R1 point, R3 point, RD (Ramus down), Sella, Sd (Supradentale), Soft tissue A point, Soft tissue B point, Gn '(Soft tissue Gnathion), Me' (Soft ti ssue Menton), N '(Soft tissue Nasion), Pg' (Soft tissue Pogonion), Stmi (Stomion inferius), Stms (Stomion superius), SM point (Submandibular point), Sn (Subnasale), UICT (Upper incisor crown tip UIRT (Upper incisor root tip), UMDP (Upper molar distal point), UMMP (Upper molar mesial point), UE (Upper embrasure), Upper lip, Mendibular outline 1, Mehndi Bullor Outline 2, Mendibuler Outline 3, Mendibuler Outline 4, Mendibuler Outline 5, Mendibuler Outline 6, Maxillary Outline 1, Maxillary Outline 2, Maxillary Outline 3 , Maxilary Outline 4, Maxilary Outline 5, Maxilary Outline 6, Maxilary Outline 7, Maxilary Outline 8, Maxilary Outline 9, Maxilary Outline 10, Maxilary Outline 11, Sim Symphysis outline 1, simphysis outline 2, simphysis outline 3 and simphysis eye A method of predicting anatomical landmarks, wherein the method is at least one selected from the group consisting of upper lines 4. The method of claim 1,
The landmark prediction model,
Receiving a learning side head medical image for which the coordinates of a plurality of landmarks with respect to the side face region is predetermined;
A model trained by predicting coordinates of the plurality of landmarks in the training side head medical image based on a shape formed by the coordinates of the plurality of landmarks in the training side head medical image. , Method of prediction of anatomical landmarks. A receiver configured to receive side head medical images of the subject, and
A processor coupled to communicate with the receiver;
The processor,
And predict the landmark in the lateral head medical image using a landmark prediction model configured to predict the landmark in the lateral head medical image. The method of claim 12,
The processor,
Predict coordinates for the parietal face region in the parietal head medical image by using a parietal face region prediction model configured to predict the parietal face region in the parietal head medical image, and use the landmark prediction model to predict Device for predicting anatomical landmarks, further configured to predict the landmarks within the facial area. The method of claim 13,
The processor,
The landmark prediction model is used to predict the x-axis and y-axis coordinates for the landmark in the parietal face region based on the predicted coordinates for the parietal face region and the parietal head medical image. Generating at least one file of xml, json, and csv including the landmark, the x-axis and y-axis coordinates, and generating the at least one file based on the lateral head medical image and at least one file of the xml, json, and csv. A device for predicting anatomical landmarks, further configured to provide an image in which the location of the landmark is indicated within a head medical image. Further comprising a measurement unit configured to measure at least one selected from the size of the maxillofacial skeleton, the growth direction of the maxillofacial skeleton and the degree of protrusion of the dentition for the subject, based on the predicted landmark. Device. The method of claim 12,
The device,
If the received side head medical image is an RGB color image,
And a data preprocessor configured to convert the side head medical image into a black and white image and vectorize the black and white image. The method of claim 16,
The data preprocessing unit,
Calculating brightness difference values of a selected one of a plurality of pixels of the black and white image and a plurality of pixels adjacent to the predetermined pixel, respectively,
Device for predicting anatomical landmarks, configured to vectorize the black and white image in a direction of a pixel having the greatest brightness difference value with the arbitrary pixel among the plurality of adjacent pixels. The method of claim 12,
The landmark prediction model,
A model trained to predict the landmark within the lateral head medical image based on a gradient boosting algorithm,
The side face facial region prediction model,
A device for predicting anatomical landmarks, the model being trained to predict the lateral face region within the lateral head medical image based on a Support Vector Machine algorithm.

Images