WO2023113272A1 - Method for providing information on endodontic treatment and device for providing information on endodontic treatment using same - Google Patents

Abstract

The present invention provides a method, implemented by a processor, for providing information on endodontic treatment, and a device using same, wherein the method comprises the steps of: receiving a dental medical image of a subject; generating a heat map image on the basis of the dental medical image by using a heat map generation model configured to generate an explainable heat map image based on clinical features of the dental medical image; and determining whether or not the endodontic treatment in the subject is successful on the basis of the dental medical image and the heat map image by using a predictive model configured to prognose endodontic treatment, with the dental medical image and the heat map image being inputted.

Description

Method for providing information on endodontic treatment and device for providing information on endodontic treatment using the same

The present invention relates to a method for providing information on root canal treatment and a device for providing information on root canal treatment using the same.

In general, if the dental pulp inside the tooth is infected by bacteria or inflamed by harmful stimuli, completely remove the contaminated tissue and pulp in the root canal, wash it, and fill it with a filling material to prevent bacteria from entering the root canal, thereby preventing the bacteria from entering the root canal. to keep teeth healthy. This treatment method is referred to as root canal treatment or root canal treatment.

Most dental specialists use clinical evaluation results and radiograph data based on experience and knowledge to determine the treatment plan for endodontic treatment. These decisions can often lead to errors in the treatment planning process. Furthermore, differences in treatment plans may occur between more experienced medical staff and less experienced medical staff. This can also lead to failure of root canal treatment.

Therefore, there is a continuous demand for the development of a method for accurately providing information related to the prognosis of root canal treatment in order to improve the quality of medical services.

The background description of the invention has been prepared to facilitate understanding of the present invention. It should not be construed as an admission that matters described in the background art of the invention exist as prior art.

Meanwhile, in order to solve the above problems, an information providing system based on artificial intelligence technology such as U-net has appeared. More specifically, the information providing system predicts (segments) a tooth area within an input dental medical image by utilizing an artificial neural network-based network, or a treatment area within a tooth (eg, gutta percha, prosthesis, etc.), furthermore, a tooth My disease can be predicted.

In particular, the researchers further applied image preprocessing procedures to the system so that the artificial neural network-based network can predict a specific area (eg, tooth area, treatment area, or lesion area) with higher accuracy. could design

Such an image segmentation-based information providing system may have excellent segmentation performance for a specific region within an image as preprocessed dental medical images are used for region segmentation.

However, it is difficult to predict the success or failure of endodontic treatment, so there may still be limitations in applying the information providing system for predicting the prognosis of endodontic treatment.

Furthermore, in the artificial neural network-based information providing system, a plan to increase the number of learning data has been proposed to improve the accuracy of prediction of treatment prognosis, but there may be limitations in securing medical data.

As a method for overcoming the above-mentioned limitations, the inventors of the present invention use the original dental medical image together with the preprocessed dental medical image to improve not only the accuracy of prediction of a specific region but also the accuracy of prediction of the prognosis of endodontic treatment. I was aware that it could be.

Furthermore, the inventors of the present invention further apply CAM (Class-activation map) images based on clinical features of teeth associated with prognosis of endodontic treatment, which can be confirmed by dental medical images in clinical practice, to predict the prognosis of endodontic treatment did

In particular, the inventors of the present invention apply CAM images to predict the prognosis of root canal treatment, just as specialists consider clinical features such as temporary restoration materials, absence of adjacent teeth, and fistulas in the treatment planning stage for actual root canal treatment. It could be recognized that clinical features can reinforce the predictions of artificial neural networks.

Furthermore, the inventors of the present invention could expect that the prognosis prediction performance of endodontic treatment can be improved even when learning data is insufficient in learning an artificial neural network by applying CAM images.

Furthermore, the inventors of the present invention could expect that the new endodontic treatment information provision system would enable specialists to establish accurate diagnosis and treatment plans by referring to the diagnosis probability determined by the predictive model regardless of skill level.

Accordingly, an object to be solved by the present invention is to provide a method for providing information on root canal treatment configured to obtain a CAM image from a received dental medical image and determine whether or not the root canal treatment is successful for an individual based on the obtained CAM image, and a device using the same. .

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a method for providing information on root canal treatment according to an embodiment of the present invention is provided. The method of providing information is a method of providing information about endodontic treatment implemented by a processor, comprising the steps of receiving a dental medical image of an object, class-activation map (CAM) image based on clinical features of the dental medical image Generating a CAM image based on a dental medical image by using a CAM generation model configured to generate a dental medical image, using a predictive model configured to predict a prognosis of endodontic treatment using the dental medical image and the CAM image as inputs, and determining whether endodontic treatment of the subject is successful based on the image and the CAM image.

According to a feature of the present invention, the information providing method may further include, after the receiving, pre-processing of the dental medical image to obtain a pre-processed image. At this time, the step of generating the CAM image includes the step of generating the CAM image based on the preprocessed image and the dental medical image using a CAM generation model, and the step of determining whether root canal treatment is successful uses a predictive model. Thus, determining whether or not root canal treatment of the object was successful based on the CAM image, the preprocessing image, and the dental medical image may be included.

According to another feature of the present invention, performing pre-processing may include performing intensity-based clustering on dental medical images to obtain a pre-processed image.

According to another feature of the present invention, the performing of preprocessing may include converting a grayscale value of a dental medical image to a predetermined level so as to obtain a preprocessed image.

According to another feature of the present invention, performing preprocessing may include performing CLAHE (Contrast Limited Adaptive Histogram Equalization) on a dental medical image to obtain a preprocessed image.

According to another feature of the present invention, the CAM generation model is composed of at least a part of the predictive model, and the acquiring of the CAM image includes processing for predicting the prognosis of endodontic treatment for the dental medical image and the preprocessed image using the predictive model Among them, a step of acquiring a CAM image as a feature based on a clinical feature may be included.

According to another feature of the present invention, clinical features include full coverage restoration (FV), absence of proximal teeth (PrX), coronal defect (CoD), and residual root (Root). Rest; RR), Temporary restoration (TM), Canal visibility (CV), Previous filling (PF), Periapical radiolucency (PAR), Sinus tract (ST) and a root post (PS).

According to another feature of the present invention, the clinical features may include a combination of at least two of a full length prosthesis, proximal absence of adjacent teeth, root canal visibility of a crown defect, an existing filling, an apical radiographic image, and a fistula.

According to another feature of the present invention, the predictive model includes a self-attention layer and a plurality of feature extraction layers, and the self-attention layer may exist between the plurality of feature extraction layers.

According to another feature of the present invention, the information providing method includes, after the step of generating the CAM image, generating a feature emphasis image including a background region and a feature region based on the activity of the CAM image, The determining whether treatment is successful may further include determining whether root canal treatment is successful for the object based on the dental medical image and the feature-enhanced image by using a predictive model.

According to another feature of the present invention, the dental medical image includes a target tooth region, and the information providing method crops the target tooth region in the dental medical image to obtain the target tooth image after the receiving step. The step of generating the CAM image may include generating the CAM image based on the image of the target tooth by using the CAM generation model.

According to another feature of the present invention, the predictive model may be configured to output a success or failure of endodontic treatment within a predetermined period by taking dental medical images, preprocessing images, and CAM images as inputs.

In order to solve the above problems, a device for providing information on root canal treatment according to another embodiment of the present invention is provided. The device includes a communication unit configured to receive a dental medical image of an object, and a processor connected to communicate with the communication unit. At this time, the processor generates a CAM image based on the dental medical image using a CAM generation model configured to generate a class-activation map (CAM) image based on clinical features of the dental medical image, and A predictive model configured to predict a prognosis of endodontic treatment using the medical image and the CAM image as inputs is configured to determine whether or not the object's endodontic treatment is successful based on the dental medical image and the CAM image.

According to a feature of the present invention, a processor performs pre-processing on a dental medical image to obtain a pre-processed image, generates a CAM image based on the pre-processed image and the dental medical image using a CAM generation model, and generates a predictive model Using , it may be configured to determine whether the root canal treatment is successful for the object based on the CAM image, the preprocessing image, and the dental medical image.

According to another feature of the present invention, the processor may be further configured to perform intensity-based clustering on dental medical images to obtain a pre-processed image.

According to another feature of the present invention, the processor may be further configured to convert a grayscale value for a dental medical image to a predetermined level, so as to obtain a preprocessed image.

According to another feature of the present invention, the processor may be further configured to perform Contrast Limited Adaptive Histogram Equalization (CLAHE) on a dental medical image, so as to obtain a preprocessed image.

According to another feature of the present invention, the CAM generation model is composed of at least a part of the predictive model, and the processor, using the predictive model, during processing for predicting the endodontic treatment prognosis for the dental medical image and the preprocessed image, clinical features It may be further configured to acquire a CAM image as a feature on the basis of.

According to a feature of the present invention, the processor generates a feature-enhanced image including a background region and a feature region based on the activity of the CAM image, and uses a predictive model to determine the root canal of the object based on the dental medical image and the feature-enhanced image. It may be further configured to determine the success of the treatment.

According to another feature of the present invention, the processor crops a target tooth region in a dental medical image to obtain a target tooth image, and generates a CAM image based on the target tooth image using a CAM generation model. More can be configured.

Other embodiment specifics are included in the detailed description and drawings.

The present invention provides a system for providing information on root canal treatment using a predictive model configured to predict the success of root canal treatment using dental medical images, CAM images, and furthermore, preprocessed images, thereby providing information for analyzing root canal treatment of a subject. There is an effect that can provide.

In particular, the present invention can overcome the limitations of the conventional information providing system capable of only predicting a specific region of a tooth by using only preprocessed medical images, and can determine the success of root canal treatment with higher reliability. can provide information.

More specifically, as the present invention applies CAM images based on clinical characteristics of teeth, in the treatment planning stage for actual root canal treatment, just as a specialist considers clinical characteristics such as temporary restoration materials, absence of adjacent teeth, and fistula, root canal treatment Clinical characteristics can be considered in predicting the prognosis of

Accordingly, the present invention can provide a predictive model with excellent performance in predicting the prognosis of endodontic treatment even when learning data is insufficient in learning an artificial neural network.

That is, the present invention provides a system for providing information on endodontic treatment based on a predictive model, so that a specialist can provide information to establish an accurate diagnosis and treatment plan by referring to the diagnosis probability determined by the predictive model regardless of skill level.

More specifically, the present invention provides a system for providing information on root canal treatment based on various predictive models, thereby overcoming errors in diagnosis according to the proficiency of medical staff and low reliability of diagnosis results, and providing an accurate root canal treatment for a subject. It may be possible to establish a treatment plan prior to treatment.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 illustrates a system for providing information on root canal treatment using a device for providing information on root canal treatment according to an embodiment of the present invention.

FIG. 2A illustrates a configuration of a device for providing information on root canal treatment according to an embodiment of the present invention.

FIG. 2B illustratively illustrates the configuration of a medical staff device receiving information on root canal treatment according to an embodiment of the present invention.

3 illustrates a procedure of a method for providing information on root canal treatment according to an embodiment of the present invention.

4A and 4B exemplarily illustrate procedures of a method for providing information on root canal treatment according to an embodiment of the present invention.

FIG. 5 illustratively illustrates the structure of a predictive model used in a method for providing information on root canal treatment according to an embodiment of the present invention.

6A to 6C illustrate CAM images of clinical features used to predict success or failure of root canal treatment in the method for providing information on root canal treatment according to an embodiment of the present invention.

7A to 7D illustrate evaluation results for prediction of endodontic treatment according to a combination of clinical characteristics in a method for providing information on endodontic treatment according to an embodiment of the present invention.

8, 9a and 9b illustrate evaluation results of a predictive model including a self-attention layer in a method for providing information on root canal treatment according to an embodiment of the present invention.

The advantages of the invention, and how to achieve them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are illustrative, so the present invention is not limited to the details shown. In addition, in describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. When 'includes', 'has', 'consists of', etc. mentioned in this specification is used, other parts may be added unless 'only' is used. In the case where a component is expressed in the singular, the case including the plural is included unless otherwise explicitly stated.

In interpreting the components, even if there is no separate explicit description, it is interpreted as including the error range.

Each feature of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as those skilled in the art can fully understand, various interlocking and driving operations are possible, and each embodiment can be implemented independently of each other. It may be possible to implement together in an association relationship.

For clarity of interpretation of this specification, terms used in this specification will be defined below.

As used herein, the term "dental medical image" may refer to an oral cavity image of a subject received from a medical imaging diagnosis apparatus. Preferably, the dental medical image disclosed herein may be a dental radiograph image, but is not limited thereto. Meanwhile, the dental medical image may be a 2D image, a 3D image, a still image of one cut, or a video composed of a plurality of cuts. For example, when a dental medical image is a video composed of a plurality of cuts, the success or failure of root canal treatment for each of the plurality of dental medical images can be predicted according to the method for providing information on root canal treatment according to an embodiment of the present invention. can

In this case, the dental medical image includes a tooth region of the upper jaw teeth for incisors, canines, premolars, and molars, and/or a target tooth region for at least one of the incisors, canines, molars, and molars of the lower jaw teeth. can do.

As used herein, the term "preprocessed image" refers to image parameters such as background noise deletion, grayscale conversion, and image contrast control for a dental medical image, particularly a target tooth image in which a target tooth region is cropped. It may mean an image on which transformation has been performed.

As used herein, the term “explainable heatmap image” may refer to an image in which activity is visually distinguishable. According to a feature of the present invention, the heat map image may be a class-activation map (CAM) image.

At this time, the "CAM (Class-activation map) image" is based on the weight determined in the calculation process of convolution and global average pooling in the classification process of the CNN (Convolutional Neural Network). It may be an image generated by At this time, it may be possible to check which part the predictive model focuses on checking through the CAM image.

According to a feature of the present invention, a CAM image may be an image generated based on at least one clinical feature.

As used herein, the term "clinical characteristics" may refer to dental clinical characteristics associated with prediction of endodontic treatment.

At this time, the clinical characteristics are full coverage restoration (FV), absence of proximal teeth (PrX), coronal defect (CoD), root rest (RR), temporary restoration ( Temporary restoration (TM), canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), sinus tract (ST) and root post (Post; PS) It may include at least one or a combination of two or more selected from among them.

As used herein, the term "feature-enhanced image" may refer to an image in which a CAM image is converted to have a background region or a feature region based on the activity of the CAM image.

For example, a feature-enhanced image is an image converted to be distinguished as 1 (feature region) when the activity is 100 or more and 0 (background region) when the activity is 100 or less based on a predetermined threshold value of 100 for a CAM image. can be

Here, the feature-enhanced image may be a masking image of a clinical feature region in a dental medical image.

As used herein, the term “heat map generation model” may be a model learned to generate a heat map image based on clinical features of a dental medical image.

In this case, the “CAM image generation model” may be a model trained to generate a CAM image based on clinical features of dental medical images.

More specifically, the CAM image generation model may be a model learned to determine (presence or absence) by learning clinical features associated with prediction of endodontic treatment with respect to dental medical images (and/or pre-processed images).

For example, the CAM image generation model may include two input channels that take a dental medical image and a preprocessed image as inputs, and two output channels that output whether or not a clinical feature exists.

In this case, the CAM image may be acquired in the process of predicting the presence or absence of clinical features in the dental medical image by the CAM image generation model.

At this time, the CAM image generation model is full coverage restoration (FV), absence of proximal teeth (PrX), coronal defect (CoD), root rest (RR), temporary Temporary restoration (TM), canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), sinus tract (ST) and root post (Post; PS) It may be a plurality of models learned to output whether or not each clinical feature is present.

However, it is not limited thereto, and the CAM image generation model may be a model obtained by learning a combination of two or more clinical features as one mask.

Optionally, the CAM image generation model may consist of a partial network structure of a predictive model to be described later.

As used herein, the term “prediction model” may be a model learned to output success or failure of endodontic treatment by taking a dental medical image, a CAM image, and optionally a preprocessing image as inputs.

For example, the prediction model includes three input channels of an original dental medical image (or a cropped target tooth image), a preprocessing image, and a CAM image (particularly, a feature-enhanced image in which clinical features are masked) and a predetermined period of time. It can be configured to have two output channels of endodontic treatment success within or root canal treatment failure within a predetermined period of time.

Here, the "predetermined period" may be 1 month to 10 years after root canal treatment, but is not limited thereto.

However, it is not limited thereto, and the predictive model may be configured to have two input channels of a dental medical image and a CAM image, and two output channels of success or failure of endodontic treatment.

According to the features of the present invention, the predictive model may be a network equipped with a self-attention layer.

Accordingly, in the present specification, “prediction model” may be used interchangeably with “network according to an embodiment of the present invention”.

As used herein, the term "self attention layer" may be a layer designed to re-receive an original image, that is, a dental medical image, in a prediction process. The self-attention layer may be present between a plurality of feature extraction layers in which calculations for feature extraction are performed, and thus, prediction based on a main region may be performed in a process of predicting success or failure of root canal treatment.

Hereinafter, with reference to FIGS. 1, 2a and 2b, a system for providing information on endodontic treatment using a device for providing information on endodontic treatment according to an embodiment of the present invention and a device for providing information on endodontic treatment will be described.

1 illustrates a system for providing information on root canal treatment using a device for providing information on root canal treatment according to an embodiment of the present invention. FIG. 2A illustrates a configuration of a device for providing information on root canal treatment according to an embodiment of the present invention. FIG. 2B illustratively illustrates the configuration of a medical staff device receiving information on root canal treatment according to an embodiment of the present invention.

First, referring to FIG. 1 , an information providing system 1000 may be a system configured to provide information related to root canal treatment based on a dental medical image of an object. At this time, the information providing system 1000 generates a CAM image based on the received dental medical image, and determines the success or failure of root canal treatment within a predetermined period therefrom. It can be composed of a medical device 200 that receives information related to the success of treatment and a dental medical imaging device 300 that provides dental medical images.

First, the information providing device 100 performs various calculations for determining success or failure of root canal treatment within a predetermined period based on a dental medical image provided from a dental medical imaging device 300 such as a radiographic device. general purpose computers, laptops, and/or data servers, and the like. In this case, the medical staff device 200 may be a device for accessing a web server providing a web page or a mobile web server providing a mobile web site, but is not limited thereto.

More specifically, the device for providing information 100 receives a dental medical image from the dental medical imaging device 300, obtains a preprocessed image and/or a CAM image from the received dental medical image, and obtains a preprocessed image and/or a CAM image therefrom within a predetermined period of time. It can be provided by determining the success of root canal treatment. At this time, it is possible to obtain a CAM projection from a dental medical image using the device 100 for providing information and a predictive model, and predict whether root canal treatment is successful or not.

The device 100 for providing information may provide a result of predicting success or failure of endodontic treatment to the medical device 200 .

The information provided from the device 100 for providing information in this way may be provided as a web page through a web browser installed in the medical device 200, or may be provided in the form of an application or program. In various embodiments, this data may be provided in a form incorporated into the platform in a client-server environment.

Next, the medical device 200 is an electronic device that provides a user interface for indicating success or failure of root canal treatment within a predetermined period of time, and includes at least one of a smart phone, a tablet PC (Personal Computer), a laptop computer, and/or a PC. may contain one.

The medical staff device 200 may receive a predictive result of success or failure of endodontic treatment for an object within a predetermined period from the information providing device 100 and display the received result through a display unit to be described later.

Next, with reference to FIG. 2A, the components of the information providing device 100 of the present invention will be described in detail.

Referring to FIG. 2A , the information providing device 100 includes a storage unit 110, a communication unit 120 and a processor 130.

First, the storage unit 110 may store various data generated in the process of determining success or failure of root canal treatment within a predetermined period from dental medical images and CAM images. For example, the storage unit 110 stores dental medical images received from the dental medical imaging device 300 through the communication unit 120 to be described later, further along with CAM images and pre-processed images together with various data in the prediction process of the predictive model. It may be configured to store products. In various embodiments, the storage unit 110 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory , a magnetic disk, and an optical disk may include at least one type of storage medium.

The communication unit 120 connects the information providing device 100 to enable communication with an external device. The communication unit 120 is connected to the medical staff device 200 and further to the dental medical imaging device 300 using wired/wireless communication to transmit/receive various data. Specifically, the communication unit 120 may receive a dental medical image of an object from the dental medical imaging device 300 . Furthermore, the communication unit 120 may transmit a prediction result and a CAM image based on clinical features to the medical device 200 .

The processor 130 is operatively connected to the storage unit 110 and the communication unit 120, and can perform various commands for analyzing a dental medical image of an object.

Specifically, the processor 130 may be configured to acquire a CAM image and/or a pre-processed image from a dental medical image received through the communication unit 120, and finally determine whether root canal treatment is successful within a predetermined period of time. .

At this time, the processor 130 may be based on a CAM generation model for generating a CAM image based on a dental medical image, and a predictive model for determining success or failure of root canal treatment within a predetermined period from a plurality of images.

In particular, processor 130 may be based on a self-attention layer-based network according to an embodiment of the present invention. However, it is not limited thereto, and the processor 130 includes SegNet, VGG-16, DCNN (Deep Convolutional Neural Network) and ResNet DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), SSD (Single Shot Detector) model, or U-net based prediction model.

Meanwhile, the device 100 for providing information is not limited to being designed in terms of hardware. For example, the processor 130 of the device 100 for providing information may be implemented as software. Accordingly, a display unit (not shown) connected to the dental medical imaging device 300 to which software is applied may be able to provide information related to root canal treatment for an object.

Meanwhile, referring to FIG. 2B , the medical device 200 includes a communication unit 210 , a display unit 220 , a storage unit 230 and a processor 240 .

The communication unit 210 may be configured to allow the medical device 200 to communicate with an external device. The communication unit 210 may be connected to the information providing device 100 using wired/wireless communication to transmit whether root canal treatment has been successful within a predetermined period.

The display unit 220 may display various interface screens to indicate success or failure of root canal treatment within a predetermined period of time.

In various embodiments, the display unit 220 may include a touch screen, and for example, a touch using an electronic pen or a part of the user's body, a gesture, a proximity, a drag, or a swipe A swipe or hovering input may be received.

The storage unit 230 may store various data used to provide a user interface for displaying result data. In various embodiments, the storage unit 230 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (for example, SD or XD memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium.

The processor 240 is operatively connected to the communication unit 210, the display unit 220, and the storage unit 230, and executes various commands for providing a user interface for displaying success or failure of root canal treatment within a predetermined period of time. can be done

Hereinafter, a method of providing information on root canal treatment according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 and 4A and 4B. 3 illustrates a procedure of a method for providing information on root canal treatment according to an embodiment of the present invention. 4A and 4B exemplarily illustrate procedures of a method for providing information on root canal treatment according to an embodiment of the present invention.

First, referring to FIG. 3, an information providing procedure according to an embodiment of the present invention is as follows. First, a dental medical image of an object is received (S310). Then, preprocessing is performed on the dental medical image (S320). Then, a CAM image is generated by the CAM generation model (S330). Finally, the success of root canal treatment is determined using the predictive model (S340).

More specifically, in the step of receiving a dental medical image (S310), a dental radiographic image including a target tooth may be received.

According to a feature of the present invention, a tooth radiographic image in DICOM form may be received in step S310 of receiving a dental medical image.

Next, a preprocessing step (S320) may be optionally performed.

More specifically, referring to FIG. 4A , various image preprocessing techniques are performed so that a preprocessed image 314 is obtained with respect to the dental medical image 312 received in the preprocessing step (S320).

According to a feature of the present invention, intensity-based clustering may be performed on the dental medical image in the pre-processing step (S320). For example, in the preprocessing step (S320), K-means clustering is performed twice to obtain a preprocessed image in which background noise is removed and a soft tissue region corresponding to the gum is maintained.

According to another feature of the present invention, in the preprocessing step (S320), the grayscale value of the dental medical image may be converted to a predetermined level.

For example, in the preprocessing step ( S320 ), a tooth radiographic image in DICOM format having a gray scale of 12 to 16 bits may be replaced with 8 bits. Due to such preprocessing, computing power for predicting the success of endodontic treatment may be reduced.

According to another feature of the present invention, CLAHE (Contrast Limited Adaptive Histogram Equalization) may be performed on the dental medical image in the preprocessing step (S320).

As a result, it may be possible to acquire a clear pre-processed image with improved image contrast.

According to another feature of the present invention, the target tooth region may be cropped in the dental medical image so that the target tooth image is obtained before the preprocessing step (S320) is performed.

That is, in the preprocessing step (S320), preprocessing may be performed on the target tooth image.

Referring back to FIG. 3 , next, in the step of generating a CAM image (S330), a CAM image may be generated by a CAM generation model based on clinical features of a dental medical image.

For example, referring to FIG. 4B together, in the step of generating a CAM image (S330), a dental medical image 312 and/or a preprocessed image 314 may be input to the CAM image generation model 340. Then, during the processing of the CAM image generation model 340, a CAM image 342 with high activity for a region of high interest, particularly a clinical feature region, may be obtained as a feature.

In this case, the CAM image generation model 340 may be a model learned to determine whether (existence or absence) exists by learning clinical features associated with prediction of endodontic treatment with respect to dental medical images (and/or pre-processed images).

According to a feature of the present invention, in the step of generating a CAM image (S330), the CAM image generation model 340 is used to generate a full coverage restoration (FV) and the absence of adjacent teeth (absence of proximal teeth) in a dental medical image. teeth; PrX), Coronal Defect (CoD), Root Rest (RR), Temporary restoration (TM), Canal visibility (CV), Previous filling (PF), A CAM image may be acquired in the process of predicting whether or not at least two clinical features selected from among a periapical radiolucency (PAR), a sinus tract (ST), and a root post (PS) are present.

According to another feature of the present invention, a step of generating a feature-enhanced image including a background region and a feature region based on the activity of the CAM image may be further performed after the step of generating the CAM image (S330).

For example, referring back to FIG. 4B , in the step of generating a feature-enhanced image, the CAM image 342 has an activity of 100 or more based on a predetermined threshold value of 100, and the activity is 1 (feature region), and the activity is 100. If it is less than or equal to 0 (background area), it can be converted to be distinguished.

That is, as a result of this, a feature-enhanced image 316 in which clinical features are masked may be obtained.

According to another feature of the present invention, in the step of generating a CAM image (S330), the CAM image is characterized based on clinical characteristics during processing for predicting the endodontic treatment prognosis for the dental medical image and the preprocessed image using the predictive model can be obtained as

That is, the CAM generation model may be composed of at least a part of the predictive model.

Referring back to FIG. 3 , the success of root canal treatment is finally determined based on a plurality of images (S340).

According to a feature of the present invention, in the step of determining whether root canal treatment is successful (S340), a predictive model learned to output success or failure of root canal treatment using a dental medical image, a CAM image, and optionally a preprocessed image as inputs is used. can

For example, referring to FIGS. 4A and 4B, in the step of determining whether root canal treatment is successful (S340), a dental medical image 312, a preprocessing image 314, and a feature-enhanced image 316 form a predictive model 314. is entered into As a result, the success or failure of root canal treatment 332 for a predetermined period (eg, within 3 years) is predicted and output. At this time, success or failure 332 may be probabilistically output.

Accordingly, the medical staff may be provided with information on whether or not the subject's root canal treatment was successful according to the information providing method according to various embodiments of the present invention, and thus may make a decision with a high probability of success and establish a treatment plan.

Hereinafter, with reference to FIG. 5 , structural characteristics of the root canal treatment success prediction model used in various embodiments of the present invention will be described.

FIG. 5 illustratively illustrates the structure of a predictive model used in a method for providing information on root canal treatment according to an embodiment of the present invention.

First, the predictive model 320 largely includes an 'input layer' receiving dental medical images, CAM images, and/or preprocessing images, a plurality of 'feature extraction layers 3202' for feature extraction, and an original image in the prediction process. It can be composed of a 'self-attention layer 3204' designed to receive re-input and an 'output layer' that finally outputs the success or failure of root canal treatment.

More specifically, the feature extraction layer 3202 may include a convolution layer for extracting features from an input image, a batch normalization layer and a max pooling layer, and a ReLu layer of an activation function. Various operations for feature extraction can be performed in the plurality of feature extraction layers 3202 .

In this case, a self-attention layer 3204 may be provided between the plurality of feature extraction layers 3202 . That is, as the image is re-input through the self-attention layer, prediction centered on the main region may be performed in the process of predicting success or failure of root canal treatment.

Prediction models used in various embodiments of the present invention can predict the success of root canal treatment with high accuracy in a dental medical image according to the above structural characteristics.

Meanwhile, the predictive model 320 may have a network structure of an embodiment of the present invention equipped with a self-attention layer, but is not limited thereto, and SegNet, VGG-16, DCNN (Deep Convolutional Neural Network), and ResNet DNN (Deep Neural Network) Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), SSD (Single Shot Detector) model, or U-net based network. .

Hereinafter, clinical features considered in CAM images used in various embodiments of the present invention will be described with reference to FIGS. 6A to 6C.

6A to 6C illustrate CAM images of clinical features used to predict success or failure of root canal treatment in the method for providing information on root canal treatment according to an embodiment of the present invention.

Referring to (a), (b) and (c) of FIG. 6a, full coverage restoration (FV), absence of proximal teeth (PrX), and coronal defect (CoD) CAM images according to whether or not (absent or present) are shown.

More specifically, the CAM image generation model appears to have different activities in dental medical images depending on the presence of a full-length prosthesis, the absence of adjacent teeth, and crown defects.

Full length prostheses, absence of adjacent teeth, and crown defects can be clinical features associated with root canal treatment. Therefore, CAM images according to the presence of full length prostheses, absence of adjacent teeth, and crown defects can be used to predict the success of root canal treatment using a predictive model. can be used

Referring to (a), (b) and (c) of FIG. 6B, whether or not Root Rest (RR), Temporary filled (TM), and Canal visibility (CV) are present (absent or present) Alternatively, CAM images according to visibility are shown. More specifically, the CAM image generation model appears to have different activities in dental medical images depending on the presence of residual root, temporary restorative material, and root canal visibility.

At this time, residual root, temporary restoration material, and root canal visibility may be clinical features related to root canal treatment, so CAM images according to residual root, temporary restorative material, and root canal visibility can be used to predict the success of root canal treatment using a predictive model. .

Referring to (a), (b) and (c) of FIG. 6c, previous filling (PF), periapical radiolucency (PAR), and sinus tract (ST) presence (absent or present) ) The CAM image according to is shown.

More specifically, the CAM image generation model appears to have different activities within dental medical images depending on the presence of existing fillings, apical radiographic images, and fistulas.

At this time, existing filling, apical radiographic image and fistula may be clinical features related to root canal treatment, so CAM images according to existing filling, apical radiographic image and fistula can be used to predict the success of root canal treatment using a predictive model. there is.

Assessment 1: Assessment of clinical characteristics

Hereinafter, evaluation results for predicting success of endodontic treatment according to various combinations of clinical characteristics will be described with reference to FIGS. 7A to 7D .

Referring to FIG. 7A, full coverage restoration (FV), absence of proximal teeth (PrX), coronal defect (CoD), root rest (RR), temporary restoration ( Temporary restoration (TM), canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), sinus tract (ST) and root post (Post; PS) Results of predicting success of endodontic treatment within 3 years excluding a single selected feature among the 10 clinical features of are shown.

First, in FIG. 7A , when the clinical characteristics of canal visibility (CV) are excluded, the average F1 score is the highest at 40.74 and the accuracy is also the highest at 68%.

These results may mean that the importance of the remaining 8 clinical characteristics excluding canal visibility (CV) is high in predicting the success of endodontic treatment within 3 years.

Subsequently, referring to FIG. 7B , when the clinical features of the temporary restorative material (Temporary filled; TM) were excluded, the average F1 score was 43.33, which is higher than when all nine features were used. Furthermore, when the clinical features of periapical radiolucency (PAR) were excluded, the average F1 score was 44.19, which was higher than when all nine features were used.

Furthermore, when the clinical features of the provisional restoration were excluded, the prediction accuracy was 66.00%, similar to the accuracy (68.00%) when all nine features were used.

Summarizing the evaluation results in 7b, when using 8 features excluding the clinical features of the temporary restoration, the predictive model can have improved performance while maintaining the accuracy of predicting the success of endodontic treatment.

Optionally, full veneer (FV), absence of proximal teeth (PrX), coronal defect (CoD), root post (PS), canal visibility (CV) ), previous filling (PF), periapical radiolucency (PAR), and fistula (sinus tract; ST) eight clinical features, and furthermore, CAM images considering the corresponding features predict endodontic treatment with the predictive model can be applied to learning for Next, referring to FIG. 7c, full veneer (FV), absence of proximal teeth (PrX), coronal defect (CoD), root post (Post; PS), root canal visibility Canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), and fistula (Sinus tract; ST), based on seven features excluding a single feature selected from the eight clinical features. A predictive result of success or failure of endodontic treatment within 3 years is shown.

More specifically, when the clinical characteristics of the full veneer (FV) together with temporary filled (TM) and root rest (RR) were excluded, that is, 7 clinical characteristics were used. The average F1 score is 49.23, and the accuracy is 67%. These results were higher than when all 10 clinical features were used (see positive control in FIG. 7a) or when all 10 clinical features were excluded (see negative control in FIG. 7a), and multiple clinical features were used as auxiliary. It can be emphasized as , which means that the prediction accuracy of the model can be increased.

According to the above results, when using 7 features excluding the clinical features of the full length prosthesis, the predictive model can have more improved performance by improving the predictive accuracy of the success of endodontic treatment.

Optionally Absence of proximal teeth (PrX), Coronal Defect (CoD), Root post (PS), Canal visibility (CV), Previous filling (PF) , CAM images considering at least one of 7 clinical features of periapical radiolucency (PAR) and sinus tract (ST), or a combination thereof, can be applied to learning of a predictive model to predict endodontic treatment. .

In particular, referring to FIG. 7D , absence of proximal teeth (PrX), coronal defect (CoD), root post (PS), canal visibility (CV), existing filling ( Predictive results of endodontic treatment success within 3 years excluding single features selected from among seven clinical features of previous filling (PF), periapical radiolucency (PAR), and sinus tract (ST) are shown.

More specifically, except for the clinical features of the full veneer (FV) together with the temporary filled (TM) and root rest (RR) (7 Features + 3 Blank (w/o TM RR) When using the 7 features of FV)), the F1 score and accuracy appear to be the highest.

That is, it appears that the F1 score and prediction accuracy decrease when any one of the seven features is excluded.

These results, in predicting the success of root canal treatment, include absence of proximal teeth (PrX), coronal defect (CoD), root post (PS), and canal visibility (CV). ), previous filling (PF), periapical radiolucency (PAR), and sinus tract (ST).

Evaluation 2: Performance evaluation of predictive model including self-attention layer

Hereinafter, performance evaluation results of the predictive model based on the self-attention layer used in various embodiments of the present invention will be described with reference to FIGS. 8, 9a and 9b.

8, 9a and 9b illustrate evaluation results of a predictive model including a self-attention layer in a method for providing information on root canal treatment according to an embodiment of the present invention.

More specifically, referring to FIG. 8 , the prediction model based on the network (AtteNet17') according to various embodiments of the present invention has an accuracy of 71.00%, a sensitivity of 60.0%, a specificity of 75.7%, and a precision of endodontic treatment prognosis prediction. has a 51.4% F1 score of 55.4, showing better prediction performance than a model based on a network (ResNET-18') that does not include a self-attention layer.

Referring to FIG. 9A , AUC of the prediction model based on the network (AtteNet17) according to various embodiments of the present invention is 0.667 and p is 0.008.

Referring to FIG. 9B, the AUC of the network (AtteNet17)-based prediction model according to various embodiments of the present invention is higher than that of the conventional network (ResNET)-based model having a similar number of layers without the self-attention layer. appears as

This may mean that it is a better diagnostic model than the conventional ResNET-based model. That is, the evaluation result is centered on the main area in the process of predicting the success of endodontic treatment by the self-attention layer designed to re-input dental medical images in the prediction process. This may mean that the prediction model according to various embodiments of the present invention may have excellent prediction performance because prediction of .

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be variously modified and implemented without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims (24)

1. As a method for providing information on endodontic treatment implemented by a processor,

   Receiving a dental medical image of an object;

   generating a heat map image based on the dental medical image using a heat map generation model configured to generate an explainable heat map image based on clinical features of the dental medical image;

   Using a predictive model configured to predict a prognosis of root canal treatment using a dental medical image and a heat map image as inputs, and determining whether or not the root canal treatment is successful for the object based on the dental medical image and the heat map image. How to provide information about root canal treatment.
2. According to claim 1,

   After the receiving step,

   Further comprising performing pre-processing on the dental medical image to obtain a pre-processed image,

   Generating the heat map image,

   Generating a heat map image based on the preprocessed image and the dental medical image using the heat map generation model;

   The step of determining whether the root canal treatment is successful,

   and determining whether the root canal treatment of the object is successful based on the heat map image, the preprocessing image, and the dental medical image, using the predictive model.
3. According to claim 2,

   Performing the preprocessing step,

   and performing intensity-based clustering on the dental medical image to obtain the preprocessed image.
4. According to claim 2,

   Performing the preprocessing step,

   and converting a grayscale value of the dental medical image to a predetermined level so as to obtain the preprocessed image.
5. According to claim 2,

   Performing the preprocessing step,

   and performing CLAHE (Contrast Limited Adaptive Histogram Equalization) on the dental medical image to obtain the preprocessed image.
6. According to claim 1,

   The heat map generation model is composed of at least a part of the prediction model,

   Acquiring the heat map image,

   obtaining the heat map image as a feature based on the clinical feature during processing for predicting the prognosis of endodontic treatment for the dental medical image and the preprocessed image using the predictive model; How to Provide Information.
7. According to claim 1,

   The clinical features are,

   Full veneer (FV), Absence of proximal teeth (PrX), Coronal defect (CoD), Root Rest (RR), Temporary filled (TM), A method for providing information on root canal treatment, which is at least one of canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), and sinus tract (ST).
8. According to claim 7,

   The clinical features are,

   The method of providing information on root canal treatment, comprising a combination of at least two of the absence of adjacent teeth, the crown defect, the root canal visibility, the existing filling material, the apical radiographic image, and the fistula.
9. According to claim 1,

   The predictive model,

   Including a self attention layer and a plurality of feature extraction layers,

   The method of providing information on root canal treatment, wherein the self-attention layer exists between the plurality of feature extraction layers.
10. According to claim 1,

    After generating the heat map image,

    Generating a feature emphasis image including a background region and a feature region based on the activity of the heat map image;

    The step of determining whether the root canal treatment is successful,

    The method of providing information on root canal treatment, further comprising determining whether or not the root canal treatment of the individual has succeeded, based on the dental medical image and the feature-enhanced image, using the predictive model.
11. According to claim 1,

    The dental medical image,

    Including the target tooth area,

    After the receiving step,

    Cropping the target tooth region in the dental medical image to obtain a target tooth image;

    Generating the heat map image,

    and generating a heat map image based on the target tooth image using the heat map generation model.
12. According to claim 1,

    The predictive model,

    A method for providing information about endodontic treatment, configured to output a success or failure of endodontic treatment within a predetermined period by taking a dental medical image, a preprocessing image, and a heat map image as inputs.
13. A communication unit configured to receive a dental medical image of an object, and

    A processor coupled to communicate with the communication unit;

    the processor,

    Generating a heat map image based on the dental medical image using a heat map generation model configured to generate a class-activation map (CAM) image based on clinical features of the dental medical image;

    Using a predictive model configured to predict the prognosis of endodontic treatment by taking a dental medical image and a heat map image as inputs, determining whether or not the object's endodontic treatment is successful based on the dental medical image and the heat map image Root canal, configured to determine A device for providing information about treatment.
14. According to claim 13,

    the processor,

    Performing pre-processing on the dental medical image to obtain a pre-processed image;

    generating a heat map image based on the preprocessed image and the dental medical image using the heat map generation model;

    A device for providing information about root canal treatment, configured to determine whether the root canal treatment of the object is successful based on the heat map image, the preprocessing image, and the dental medical image by using the predictive model.
15. According to claim 14,

    the processor,

    The device for providing information on root canal treatment, further configured to perform intensity based clustering on the dental medical image to obtain the preprocessed image.
16. According to claim 14,

    the processor,

    The device for providing information about endodontic treatment, further configured to convert a grayscale value of the dental medical image to a predetermined level so as to obtain the preprocessed image.
17. According to claim 14,

    the processor,

    The device for providing information about endodontic treatment, further configured to perform CLAHE (Contrast Limited Adaptive Histogram Equalization) on the dental medical image, so as to obtain the preprocessed image.
18. According to claim 13,

    The heat map generation model is composed of at least a part of the prediction model,

    the processor,

    Further configured to acquire the heat map image as a feature based on the clinical feature during processing for predicting the prognosis of endodontic treatment for the dental medical image and the preprocessed image using the prediction model, providing information on endodontic treatment device for.
19. According to claim 13,

    The clinical features are,

    Full veneer (FV), absence of proximal teeth (PrX), coronal defect (CoD), root rest (RR), temporary filled (TM), A device for providing information on root canal treatment, which is at least one of canal visibility (CV), previous filling (PF), periapical radiolucency (PAR), and sinus tract (ST).
20. According to claim 19,

    The clinical features are,

    A device for providing information about root canal treatment, comprising a combination of at least two of the absence of adjacent teeth, the crown defect, the root canal visibility, the existing filling, the apical radiographic image, and the fistula.
21. According to claim 13,

    The predictive model,

    Including a self attention layer and a plurality of feature extraction layers,

    The device for providing information on root canal treatment, wherein the self-attention layer exists between the plurality of feature extraction layers.
22. According to claim 13,

    the processor,

    Based on the activity of the heat map image, a feature-enhanced image including a background region and a feature region is generated;

    The device for providing information about root canal treatment, further configured to determine whether the root canal treatment of the object is successful based on the dental medical image and the feature-enhanced image by using the predictive model.
23. According to claim 13,

    The dental medical image,

    Including the target tooth area,

    the processor,

    Cropping the target tooth region in the dental medical image to obtain a target tooth image;

    A device for providing information on root canal treatment, further configured to generate a heat map image based on the target tooth image using the heat map generation model.
24. According to claim 13,

    The predictive model,

    A device for providing information about endodontic treatment, configured to output a success or failure of endodontic treatment within a predetermined period by taking a dental medical image, a preprocessing image, and a heat map image as inputs.

Images