KR20210033440A - Method for providing tooth leison information and apparatus using the same - Google Patents

Abstract

The present disclosure relates to a tooth lesion information providing method performed by a computing device. The tooth lesion information providing method performed by a computing device comprises the steps of: (a) receiving a tooth image of a subject; (b) detecting a lesion region included in the tooth image through a pre-learned reading model or supporting another device linked to the computing device to detect the lesion region; (c) generating tooth lesion information for the lesion region or supporting another device linked to the computing device to generate the tooth lesion information; and (d) providing lesion information to an external entity.

Description

Dental lesion information providing method and device using the same {METHOD FOR PROVIDING TOOTH LEISON INFORMATION AND APPARATUS USING THE SAME}

The present invention relates to a method for providing tooth lesion information and an apparatus using the same.

Various X-ray images taken in various ways are widely used for dental diagnosis and treatment. Panorama X-ray images are widely used in the establishment of treatment plans in that the oral structure can be confirmed as a whole and the anatomical structure of the oral cavity is properly revealed. Usually, a method of predicting a lesion area by identifying an X-ray panoramic image with the naked eye is widely used in dental treatment, but techniques for automatically identifying a lesion have been developed to overcome the limitation of identification by the naked eye. Research to establish a lesion candidate region in an X-ray image remains at the level of using the contrast change in an X-ray image.

An object of the present invention is to provide a means for effectively determining a lesion area in a tooth image by providing information on whether or not a lesion of each tooth occurs together with identification information of the corresponding tooth.

In addition, the present invention is to provide the visualized lesion information through a map that visualizes the possibility that each region corresponds to the lesion region in the input tooth image.

In addition, an object of the present invention is to provide a readout model capable of more accurately detecting a lesion area by using a loss function in consideration of a correlation between teeth in a process of learning a readout model for detecting a lesion area in a tooth image.

A characteristic configuration of the present invention for achieving the object of the present invention as described above and realizing the characteristic effects of the present invention described later is as follows.

According to an aspect of the present invention, a method for providing tooth lesion information performed by a computing device includes the steps of: (a) receiving a tooth image of a subject; (b) detecting a lesion area included in the tooth image through a pre-learned read model or supporting another device linked to the computing device to detect the lesion area; (c) generating tooth lesion information on the lesion area or supporting another device linked to the computing device to generate the tooth lesion information; And (d) providing lesion information to an external entity.

According to an aspect, the step (c) includes the steps of (c1) generating a map for visualizing a lesion area in the tooth image; And (c2) generating the tooth lesion information through visualization information in which the lesion area is visualized in the tooth image based on the map.

According to another aspect, the step (c) includes the steps of (c3) generating identification information for identifying the position or order of individual teeth included in the tooth image; (c4) generating matching information by matching the identification information with information on whether a lesion has occurred in an individual tooth corresponding to the identification information; And (c5) generating the tooth lesion information based on the matching information.

According to an aspect, the readout model is pre-trained in a direction that minimizes the result of the loss function, and the loss function may be determined based on correlation information between individual teeth.

A computing device that provides tooth lesion information according to an aspect includes: a communication unit that receives a tooth image of a subject; And a processor for generating tooth lesion information for the tooth image, wherein the processor generates tooth lesion information on a lesion region included in the tooth image through a pre-learned read model, or other information linked to the computing device. It is possible to support the device to generate the lesion information and provide the tooth lesion information to an external entity.

According to an embodiment of the present invention, by learning a readout model based on a loss function in consideration of a correlation between teeth, it is possible to dramatically improve the accuracy of detecting a lesion area for a tooth image.

In addition, it provides a means to effectively visualize the lesion area in the tooth image through a map that visualizes the lesion detection result of the tooth image, and through an interface that provides information on whether a lesion has occurred in the tooth and a tooth identification number, the reader can A single tooth image may provide a means for reading whether or not lesions have occurred in individual teeth.

According to the present invention, the potential effect of being able to innovate the workflow in the medical field by ultimately improving the accuracy of diagnosis by the medical staff and allowing the entire tooth to be able to read whether or not lesions have occurred through a single image. have.

In addition, in the present invention, a medical image used in a conventional hospital can be used as it is, so it goes without saying that the method of the present invention is not dependent on a specific type of image or platform.

The accompanying drawings, which are attached to be used in the description of the embodiments of the present invention, are only some of the embodiments of the present invention, and those of ordinary skill in the technical field to which the present invention belongs Other drawings can be obtained on the basis of these drawings without efforts to reach the invention.
1 is a conceptual diagram schematically showing an exemplary configuration of a computing device that performs a method for providing lesion information according to the present invention.
2 is an exemplary block diagram showing hardware or software components of a computing device that performs a method for providing lesion information according to the present invention.
3 is a flowchart illustrating a method of providing tooth lesion information according to an exemplary embodiment.
4 is a diagram illustrating an example in which a method of providing lesion information according to an exemplary embodiment is performed.
5 is a diagram illustrating an example in which a Pearson correlation matrix used in a process of determining a loss function is visualized.

Detailed description of the present invention to be described later, in order to clarify the objects, technical solutions and advantages of the present invention, reference is made to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced as an example. These embodiments are described in detail sufficient to enable a person skilled in the art to practice the present invention.

The term "image" or "image data" used throughout the detailed description and claims of the present invention refers to multidimensional data composed of discrete image elements (eg, pixels in a 2D image and voxels in a 3D image). Refers to. For example, "image" refers to an X-ray image, (cone-beam) computed tomography, magnetic resonance imaging (MRI), ultrasound, or any other medical image known in the art. It may be a medical image of a subject, that is, a subject collected by the system. Also, an image may be provided in a non-medical context, such as a remote sensing system, an electron microscopy, and the like.

Throughout the detailed description and claims of the present invention,'image' refers to an image that can be seen by the eye (e.g., displayed on a video screen) or of an image (e.g., a file corresponding to a pixel output of a CT, MRI detector, etc.) It is a term referring to digital representations.

Throughout the detailed description and claims of the present invention, the'DICOM (Digital Imaging and Communications in Medicine; Medical Digital Imaging and Communications)' standard is a generic term for various standards used for digital image expression and communication in medical devices. Standards are published by a coalition committee formed by the American Radiological Society (ACR) and the American Electrical Industry Association (NEMA).

In addition, throughout the detailed description and claims of the present invention,'medical image storage and transmission system (PACS; picture archiving and communication system)' is a term that refers to a system that stores, processes, and transmits according to the DICOM standard, and X-ray, CT , Medical image images acquired using digital medical imaging equipment such as MRI are stored in DICOM format and can be transmitted to terminals inside and outside the hospital through a network, and reading results and medical records can be added to this.

And throughout the detailed description and claims of the present invention,'learning' or'learning' is a term that refers to performing machine learning through computing according to procedures. It will be appreciated by those of skill in the art that it is not intended to be referred to.

And throughout the detailed description and claims of the present invention, the word'comprise' and its variations are not intended to exclude other technical features, additions, components or steps. In addition,'one' or'one' is used in one or more meanings, and'another' is limited to at least a second or more.

Other objects, advantages, and features of the present invention to those skilled in the art will appear, in part, from this description, and in part from the practice of the present invention. The examples and drawings below are provided by way of example and are not intended to limit the invention. Therefore, the details disclosed in this specification with respect to a specific structure or function are not to be construed in a limiting sense, but a representative of providing guidance for a person skilled in the art to variously implement the present invention with any detailed structures that are substantially suitable. It should be interpreted as basic data.

Moreover, the present invention covers all possible combinations of the embodiments indicated herein. It should be understood that the various embodiments of the present invention are different from each other, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. In addition, it should be understood that the location or arrangement of individual components in each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description to be described below is not intended to be taken in a limiting sense, and the scope of the present invention, if appropriately described, is limited only by the appended claims, along with all ranges equivalent to those claimed by the claims. Like reference numerals in the drawings refer to the same or similar functions over several aspects.

Unless otherwise indicated in this specification or clearly contradicting the context, items referred to in the singular encompass the plural unless otherwise required by that context. In addition, in describing the present invention, when it is determined that a detailed description of a related known configuration or function may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to enable those skilled in the art to easily implement the present invention.

1 is a conceptual diagram schematically showing an exemplary configuration of a computing device that performs a method for providing lesion information according to the present invention.

Referring to FIG. 1, a computing device 100 according to an embodiment of the present invention includes a communication unit 110 and a processor 120, and is directly or indirectly connected to an external computing device (not shown) through the communication unit 110. Can communicate with enemies.

Specifically, the computing device 100 is a device that may include typical computer hardware (eg, computer processor, memory, storage, input devices and output devices, and other components of existing computing devices; routers, switches, etc.). Electronic communication devices; electronic information storage systems such as network-attached storage (NAS) and storage area networks (SANs) and computer software (i.e., allowing the computing device to function in a specific way). Instructions) may be used to achieve the desired system performance.

The communication unit 110 of such a computing device can transmit and receive requests and responses to and from other computing devices to which it is linked. As an example, such requests and responses may be made by the same transmission control protocol (TCP) session. However, the present invention is not limited thereto, and may be transmitted/received as, for example, a user datagram protocol (UDP) datagram. In addition, in a broad sense, the communication unit 110 may include a keyboard, a mouse, other external input devices, printers, displays, and other external output devices for receiving commands or instructions.

In addition, the processor 120 of the computing device may include a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU) or a tensile processing unit (TPU), a cache memory, and a data bus. ), and the like. In addition, it may further include an operating system and a software configuration of an application that performs a specific purpose.

In the following description, the image of the present invention targets a tooth image, and the tooth image targets a tooth X-ray panoramic image as an example, but the scope of the present invention is not limited thereto, and all of the general form tooth image It will be readily understood by those of ordinary skill in the art that it can be applied and that it can be applied in the process of detecting an arbitrary lesion area included in a target image.

2 is an exemplary block diagram showing hardware or software components of a computing device that performs a method for providing lesion information according to the present invention.

The individual modules shown in FIG. 2 may be implemented by interlocking the communication unit 110 or the processor 120 included in the computing device 100, or the communication unit 110 and the processor 120, for example. Technicians will be able to understand.

Referring to FIG. 2 to briefly overview the configuration of the method and apparatus according to the present invention, the computing device 100 may include an image acquisition module 210 as its constituent element. The image acquisition module 210 may acquire a tooth image of a subject, which is previously stored in a database or obtained from a device dedicated to photographing an image. The tooth image may be an X-ray panoramic image of a subject's teeth captured through a dedicated photographing device built into the computing device 100 or an external photographing dedicated device.

The tooth image acquired through the image acquisition module 210 may be transmitted to the tooth lesion region detection module 220. The tooth lesion region detection module 220 may detect a lesion region included in the tooth image through a read model. The readout model may be an artificial neural network that has been trained in advance to detect a lesion area in a tooth image. The readout model included in the tooth lesion region detection module 220 is a deep learning-based artificial neural network that has been trained to output at least one of the location of the lesion region, the shape of the lesion region, and the probability corresponding to the lesion region from the input tooth image. I can. The artificial neural network included in the tooth lesion region detection module 220 determines whether the region included in the input tooth image matches the detection target region (eg, bone loss region) based on the similarity, and outputs the determination result. For example, DNN (Deep Neural Network), CNN (Convolutional Neural Network), DCNN (Deep Convolution Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network) ), SSD (Single Shot Detector), YOLO (You Only Look Once), etc. can be used, it will be understood by those of ordinary skill in the art. The type of artificial neural network that can be used in the tooth lesion region detection module 220 is not limited to the presented example, and may include any artificial neural network that can be learned to recognize the lesion region in the tooth image based on the labeled learning data. It will be understood by those of ordinary skill in the art that it can be.

The detection result of the lesion region performed through the tooth lesion region detection module 220 may be transmitted to the visualization module 230, and the visualization module 230 may generate a map for visualizing the lesion region in the tooth image. . For example, the visualization module 230 may generate a heat map in which the color of a corresponding region is changed according to a probability that each region of a tooth corresponds to a lesion region (eg, a bone loss region). . The method of generating the map by the visualization module 230 is not limited to the method described above, and an arbitrary method capable of visually expressing the probability that each region of the tooth image corresponds to the lesion region (e.g., It will be apparent to those of ordinary skill in the art that the corresponding probability may be expressed by quantifying it.

The detection result performed through the tooth lesion region detection module 220 may be transmitted to the classification module 240, and the classification module 240 may determine identification information of a tooth corresponding to the detected lesion region. The classification module 240 may generate identification information for identifying the position or order of individual teeth included in the tooth image, and may determine identification information of the tooth in which the lesion area is detected based on the generated identification information and the detection result. . The identification information for individual teeth may be determined based on the FDI World Dental Federation notation, which is a dental identification number most used by dentists around the world, but is not limited thereto, and is based on an arbitrary method by which individual teeth can be identified. Can be determined. For example, when a tooth identification number is used as the identification information, the classification module 240 determines the identification number of the tooth from which the lesion is detected among identification numbers generated for individual teeth, and based on the determination result, the identification number and It is possible to generate matching information by matching information on whether a lesion has occurred in an individual tooth. Matching information is a method of providing a list of identification information of teeth determined to have lesions according to implementation, a method of matching identification information of all teeth and whether lesions have occurred, etc. It can be created in the way of

The tooth lesion information generation and transmission module 250 may generate tooth lesion information based on a map generated by the visualization module 230 or matching information generated through the classification module 240. The tooth lesion information may be determined based on at least one of visualization information generated by applying the map to a tooth image or information on a tooth list in which a lesion is detected based on the matching information.

The tooth lesion information generated through the tooth lesion information generation and transmission module 250 may be stored in a database or may be provided to an external entity. When the tooth lesion information is provided to an external entity, the tooth lesion information generation and transmission module 250 may provide an output image reflecting the tooth lesion information to the external entity using a predetermined display device or the like, or through an provided communication unit. . Here, the external entity includes a user of the computing device 100, an administrator, a medical professional in charge of the subject, etc., but in addition to this, a subject who needs tooth lesion information for the input tooth image If so, it should be understood that any subject is included. For example, the external entity may be an external AI device including a separate artificial intelligence (AI) hardware and/or software module that utilizes the tooth lesion information. In addition,'external' in an external entity is not intended to exclude an embodiment in which an AI hardware and/or software module using the tooth lesion information is integrated into the computing device 100, but the method of the present invention. It should be noted that the tooth lesion information, which is the result of the hardware and/or software module performing the procedure, is used to suggest that it can be used as input data for other methods. That is, the external entity may be the computing device 100 itself.

Although the components shown in FIG. 2 are illustrated as being realized in one computing device for convenience of description, it will be understood that a plurality of computing devices 100 performing the method of the present invention may be configured to interlock with each other.

Now, an embodiment of an image providing method according to the present invention will be described in more detail with reference to FIGS. 3 to 5.

3 is a flowchart illustrating a method of providing tooth lesion information according to an exemplary embodiment.

Referring to FIG. 3, the computing device may receive a tooth image of a subject through a communication unit in step S100. As described above, the tooth image may be a tooth X-ray panoramic image, but the present invention is not limited thereto, and may include an image of an arbitrary shape photographing the subject's teeth.

The computing device may detect a lesion area included in the tooth image through the reading model in step S200. The readout model may include a pre-trained artificial neural network to detect a lesion area in a tooth image.

The loss function for training the readout model may be determined based on a maxillary loss function for a maxillary tooth and a mandibular loss function for a mandibular tooth in addition to a cross entropy function commonly used as a loss function. The upper jaw may mean an upper jaw included in the tooth image, and the mandible may mean a lower jaw included in the tooth image. Accordingly, the maxillary loss function may mean a loss function determined corresponding to the upper jaw, and the mandibular loss function may mean a loss function corresponding to the mandible.

More specifically, individual loss functions for calculating the maxillary loss function and the mandibular loss function may be determined based on Equation 1.

는 개별 손실 함수,

는 치아의 병변 발생 여부를 나타내는 값,

는 i번째 데이터 샘플의 j번째 치아의 병변 발생 여부를 나타내는 값,

는 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 i번째 데이터 샘플의 j번째 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 피어슨 상관 행렬(pearson correlation matrix),

는 피어슨 상관 행렬의 각 요소로써 j번째 치아와 j'번째 치아에 상응하는 상관 계수 값c는 피어슨 상관 행렬에 기초하여 결정되는 상관 파라미터,

는 i번째 데이터 샘플의 j번째 치아에 대해 산출된 상관 파라미터, k는 치아 번호의 개수,

은 데이터 샘플의 개수,

는

를 0 및 1사이로 정규화하기 위한 값을 나타낸다.

Is the individual loss function,

Is a value indicating whether a tooth lesion has occurred,

Is a value indicating whether a lesion of the j-th tooth of the i-th data sample has occurred,

Is the predicted value of the readout model for the occurrence of a tooth lesion,

Is the predicted value of the readout model for the occurrence of lesion of the j-th tooth in the i-th data sample,

Is the pearson correlation matrix,

Is each element of the Pearson correlation matrix, and the correlation coefficient value c corresponding to the j-th tooth and the j'-th tooth is a correlation parameter determined based on the Pearson correlation matrix,

Is the correlation parameter calculated for the j-th tooth of the i-th data sample, k is the number of tooth numbers,

Is the number of data samples,

Is

Represents a value for normalizing between 0 and 1.

는 하기의 수학식 2에 기초하여 결정될 수 있다.The correlation coefficient between the a-th tooth and the b-th tooth included in the Pearson correlation matrix of Equation 1

May be determined based on Equation 2 below.

는 a번째 치아와 b번째 치아의 상관 계수,

는 i번째 학습 데이터에서 a번째 치아의 병변 발생 여부를 나타내는 값,

는i번째 학습 데이터에서 b번째 치아의 병변 발생 여부를 나타내는 값,

는

의 평균,

는

의 평균을 의미할 수 있다.

Is the correlation coefficient between the a-th tooth and the b-th tooth,

Is a value indicating whether a lesion of the a-th tooth has occurred in the i-th training data,

Is a value indicating whether a lesion of the b-th tooth has occurred in the i-th training data,

Is

Mean of,

Is

Can mean the average of.

=1, 병변이 발생하지 않는 경우

로 출력된다.)은 a = {1, 0, 0, 0, 0, 1, 1}이고, b = {1, 1, 0, 0, 0, 1, 1}로 학습 데이터가 7개인 경우,

는 0.428,

는 0.571로 산출되고, 수학식 2에 따라

는 0.75로 산출될 수 있다. For example, the value of whether a lesion has occurred on the a-th tooth (if a lesion occurs

=1, if the lesion does not develop

Is outputted as a = {1, 0, 0, 0, 0, 1, 1}, and b = {1, 1, 0, 0, 0, 1, 1} with 7 training data,

Is 0.428,

Is calculated as 0.571, and according to Equation 2

Can be calculated as 0.75.

An example in which the correlation coefficient calculated based on Equation 2 is visualized is shown in FIG. 5 attached below.

Based on Equation 1, the maxillary loss function and the mandibular loss function may be calculated as shown in Equations 3 and 4.

는 상악 치아의 병변 발생 여부를 나타내는 값,

는 상악 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 상악 치아에 대한 상관 파라미터를 나타낸다.

Is a value indicating whether a lesion of the maxillary tooth has occurred,

Is the predicted value of the readout model for the occurrence of lesions of the maxillary teeth,

Represents the correlation parameter for the maxillary teeth.

는 하악 치아의 병변 발생 여부를 나타내는 값,

는 하악 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 하악 치아에 대한 상관 파라미터를 나타낸다.

Is a value indicating whether a lesion of the mandibular tooth has occurred,

Is the predicted value of the readout model for the occurrence of lesion of the mandibular tooth,

Represents the correlation parameter for the mandibular tooth.

Based on Equation 3 and Equation 4, the final loss function used for learning the readout model may be calculated as in Equation 5.

는 조절 가능한 파라미터를 의미할 수 있다.CrossEntropy is the cross entropy function,

May mean an adjustable parameter.

및

에 기초하여 결정된다. 실제로 치아 병변의 경우 인접한 치아들 사이에 함께 발생되는 경우가 많으므로, 병변 발생과 관련하여 인접한 치아들 사이에 상관 관계는 매우 높을 수 있다. 최종 손실 함수는 상악 및 하악에 포함되는 치아들 사이의 상관 관계가 반영되어 있으므로, 이에 기반하여 학습된 판독 모델의 병변 영역 예측에 대한 정확도는 상관 관계를 고려하지 않는 손실 함수를 이용하는 경우에 비해 향상될 수 있다. 보다 구체적으로, 치아들 사이의 상관 관계를 고려하지 않는 손실 함수에 기초하여 판독 모델이 학습되는 경우, 병변이 48번 치아에 발생하였으나, 47번 치아에 병변이 발생한 것으로 예측한 경우와 41번 치아에 병변이 발생한 것으로 예측하는 경우 오류는 동일하게 측정될 수 있다. 하지만, 수학식 5에 따른 최종 손실 함수에 기반하여 학습이 진행되는 경우, 치아들 사이의 상관 관계가 학습 과정에서 반영되어 있기 때문에, 상기 상황에서 41번 치아로 잘못 예측한 경우에 더 큰 패널티가 부여될 수 있고, 이를 통해 보다 예측 정확도가 높은 판독 모델이 생성될 수 있다.The final loss function is determined based on the maxillary loss function and the mandibular loss function, and the maxillary loss function and the mandibular loss function are determined based on the correlation between the teeth included in each.

And

It is determined on the basis of. In fact, since dental lesions often occur together between adjacent teeth, the correlation between adjacent teeth may be very high in relation to the occurrence of the lesion. Since the final loss function reflects the correlation between the teeth included in the maxilla and mandible, the accuracy of the prediction of the lesion area of the read model learned based on this is improved compared to the case of using the loss function that does not consider the correlation. Can be. More specifically, when the readout model is learned based on the loss function that does not take into account the correlation between teeth, the lesion occurred on tooth 48, but predicted that the lesion occurred on tooth 47 and tooth 41 The error can be measured in the same way when predicting that a lesion has occurred. However, when learning is performed based on the final loss function according to Equation 5, since the correlation between teeth is reflected in the learning process, a larger penalty is incurred in the case of incorrectly predicting tooth 41 in the above situation. May be given, and through this, a readout model with higher prediction accuracy may be generated.

The computing device may generate tooth lesion information on the detected lesion area in step S300 or support another device interlocked with the computing device to generate tooth lesion information. The tooth lesion information may be determined by at least one of visualization information in which a lesion area is visualized in a tooth image, and matching information in which identification information of individual teeth and whether or not a lesion occurs.

According to an embodiment, the computing device may generate a map for visualizing the lesion area detected in the tooth image based on the detection result in step S200. The computing device may determine a weight for each region of the tooth image based on the output of the last layer of the read model, and generate a map representing different visual elements according to the determined weight. For example, the computing device represents an area corresponding to a high weight in the output of the final layer, and an area with a high probability of corresponding to a lesion area is expressed in a color close to red, and an area with a low probability of corresponding to a lesion area due to a low weight is expressed. It is possible to create a heat map that is represented by an area close to green. The computing device applies the generated heat map to the input tooth image and visualizes the region with a high probability of corresponding to the lesion region with a color close to red, and visualizes the region with a low probability of corresponding to the lesion region with a color close to green. Visualization information for a tooth image can be generated. The map for generating the visualization information is not limited to the heat map and color configuration as an example, and it can be easily understood by those of ordinary skill in the art that each region can be implemented in an arbitrary manner to visualize the probability that it corresponds to the lesion region. There will be.

According to another embodiment, the computing device generates identification information that identifies the position or sequence of individual teeth included in the tooth image, and information on whether a lesion has occurred in the individual tooth corresponding to the generated identification information and the identification information. It is possible to generate matching information that matches. For example, the identification information may be an identification number for an individual tooth according to the FDI World Dental Federation notation, and the computing device provides information on whether a lesion has occurred in the tooth based on the detection result in step S200. Matching information matching the identification number of can be generated.

The computing device may provide tooth lesion information to an external entity in step S400.

4 is a diagram illustrating an example in which a method of providing lesion information according to an exemplary embodiment is performed.

The computing device 420 may provide tooth lesion information including visualization information 430 and matching information 440 of the input tooth image 410. To this end, the computing device 420 may detect a lesion area in the tooth image 410 through a pre-learned reading model. The computing device generates the visualization information 430 to which the heat maps 431, 432, 433 that visualize the detected lesion area is applied, or the matching information 440 that matches the identification information of the tooth and the information on whether the lesion has occurred. It is possible to provide tooth lesion information by creating and providing it to an external entity. The heat maps 431, 432, and 433 may be expressed in different colors according to the probability of corresponding to the lesion area. The identification number of the tooth included in the matching information 440 may be determined according to the FDI World Dental Federation notation shown in the identification information 441.

5 is a diagram illustrating an example in which a Pearson correlation matrix is visualized according to an embodiment.

Lesions that occur on teeth are more likely to occur continuously between adjacent teeth. Thus, with regard to the detection of a lesion of a tooth, adjacent teeth can have a very high correlation. The present invention can generate a lesion prediction model with higher accuracy by using the final loss function corresponding to Equation 5 in which the correlation between adjacent teeth is reflected in the process of training the artificial neural network corresponding to the reading model.

Each of the matrices 510 and 520 corresponds to an example of visualizing the Pearson correlation matrix of the maxillary teeth and the Pearson correlation matrix of the mandibular teeth calculated based on Equation (2). Elements of each matrix may correspond to a degree of correlation between teeth. An element visualized with a darker color corresponds to a higher correlation, and an element visualized with a lighter color corresponds to a lower correlation. Referring to the matrices 510 and 520, it can be seen that the correlation between teeth adjacent to each other is calculated high, which corresponds to the above-mentioned.

Based on the description of the above embodiment, a person skilled in the art will know that the method and/or processes of the present invention, and the steps thereof, can be implemented in hardware, software, or any combination of hardware and software suitable for a specific application. Can understand the point clearly. The hardware may include a general-purpose computer and/or a dedicated computing device, or a specific computing device or special features or components of a specific computing device. The processes may be realized by one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable devices, with internal and/or external memory. In addition, or as an alternative, the processes can be configured to process application specific integrated circuits (ASICs), programmable gate arrays, programmable array logic (PAL) or electronic signals. May be implemented with any other device or combination of devices. Furthermore, the objects of the technical solution of the present invention or parts contributing to the prior arts may be implemented in the form of program instructions that can be executed through various computer components and recorded in a machine-readable recording medium. The machine-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the machine-readable recording medium may be specially designed and configured for the present invention, or may be known to and usable by a person skilled in the computer software field. Examples of machine-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM, DVD, Blu-ray, and magnetic-optical media such as floptical disks. (magneto-optical media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include a processor, a processor architecture, or a heterogeneous combination of different hardware and software combinations, as well as any one of the aforementioned devices, or storing and compiling or interpreting to be executed on a machine capable of executing any other program instructions. Can be created using a structured programming language such as C, an object-oriented programming language such as C++, or a high-level or low-level programming language (assembly, hardware description languages and database programming languages and technologies), machine code, This includes not only bytecode but also high-level language code that can be executed by a computer using an interpreter or the like.

Thus, in an aspect according to the present invention, when the above-described method and combinations thereof are performed by one or more computing devices, the method and combinations of methods may be implemented as executable code that performs the respective steps. In another aspect, the method may be implemented as systems that perform the steps, and the methods may be distributed in several ways across devices or all functions may be integrated into one dedicated, standalone device or other hardware. In yet another aspect, the means for performing the steps associated with the processes described above may include any hardware and/or software described above. All such sequential combinations and combinations are intended to be within the scope of this disclosure.

For example, the hardware device may be configured to operate as one or more software modules to perform the processing according to the present invention, and vice versa. The hardware device may include a processor such as an MPU, CPU, GPU, or TPU that is combined with a memory such as ROM/RAM for storing program instructions and configured to execute instructions stored in the memory, and external devices and signals It may include a communication unit that can send and receive. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described by specific matters such as specific components and limited embodiments and drawings, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, Anyone with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all modifications that are equally or equivalent to the claims as well as the claims to be described later fall within the scope of the spirit of the present invention. I would say.

Such equivalently or equivalently modified ones will include, for example, a logically equivalent method capable of producing the same result as that of carrying out the method according to the present invention, the true meaning and scope of the present invention. Is not to be limited by the above-described examples, and should be understood in the broadest possible meaning by law.

Claims (19)

In the method of providing tooth lesion information performed by a computing device,
Inputting a tooth image into a pre-learned readout model to detect a lesion region included in the tooth image; And
Providing tooth lesion information on the lesion area to an external entity
Including,
The reading model,
It is learned in the direction of minimizing the result of the loss function calculated through the correlation parameter for the correlation between individual teeth,
The correlation parameter,
A method of providing tooth lesion information that is calculated based on a correlation between information on whether or not lesions of individual teeth included in the learning data have occurred. The method of claim 1,
The correlation parameter,
A method of providing dental lesion information that is calculated using a correlation matrix calculated based on a correlation between information on whether the lesion has occurred. The method of claim 2,
The correlation parameter is calculated based on Equation 1,
Equation 1 is,

ego,

Is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

Is a value to normalize between 0 and 1,

Is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

Denotes a value indicating whether or not a lesion of the j'th tooth of the i-th learning data has occurred, a method of providing tooth lesion information. The method of claim 1,
The loss function is,
And an individual loss function calculated based on an operation between the correlation parameter and the prediction result of the read model. The method of claim 4,
The individual loss function is calculated based on Equation 2,
Equation 2,

ego,

Is the individual loss function,

Is a value indicating whether or not a lesion of the tooth included in the training data has occurred,

Is the predicted value of the readout model for the occurrence of a tooth lesion,

Is the predicted value of the readout model for the occurrence of a lesion on the j-th tooth of the i-th training data,

Is the number of training data, k is the number of teeth, c is a correlation parameter determined based on the correlation matrix,

Represents the correlation parameter calculated for the j-th tooth of the i-th learning data, tooth lesion information providing method. The method of claim 4,
The individual loss function is,
A method for providing tooth lesion information, including a maxillary individual loss function calculated for the maxillary teeth included in the training data and a mandibular individual loss function calculated for the mandible teeth included in the training data. The method of claim 1,
The tooth lesion information,
According to a probability that the individual regions included in the tooth image correspond to the lesion regions, the method of providing tooth lesion information includes a visualization map in which the individual regions are identified in different forms. A machine-readable non-transitory recording medium containing a computer program including instructions embodied to cause a computing device to perform the method of claim 1. In a learning method for learning a readout model for detecting a lesion area in a tooth image,
Calculating a loss function through the training data; And
Training the readout model in a direction that minimizes the result of the loss function
Including,
The loss function is,
It is calculated using a correlation parameter for a correlation between individual teeth included in the learning data,
The correlation parameter,
A learning method that is calculated based on a correlation between information on whether or not lesions of individual teeth included in the learning data have occurred. The method of claim 9,
The correlation parameter,
A learning method that is calculated using a correlation matrix calculated based on a correlation between information on whether the lesion has occurred. The method of claim 10,
The correlation parameter is calculated based on Equation 1,
Equation 1 is,

ego,

Is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

Is a value to normalize between 0 and 1,

Is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

Denotes a value indicating whether a lesion of the j'th tooth of the ith learning data has occurred. The method of claim 9,
The loss function is,
And an individual loss function calculated based on an operation between the correlation parameter and a prediction result of the readout model. The method of claim 12,
The individual loss function is calculated based on Equation 2,
Equation 2,

ego,

Is the individual loss function,

Is a value indicating whether or not a lesion of the tooth included in the training data has occurred,

Is the predicted value of the readout model for the occurrence of a tooth lesion,

Is the predicted value of the readout model for the occurrence of a lesion on the j-th tooth of the i-th training data,

Is the number of training data, k is the number of teeth, c is a correlation parameter determined based on the correlation matrix,

Denotes a correlation parameter calculated for the j-th tooth of the i-th training data. The method of claim 12,
The individual loss function is,
A learning method comprising a maxillary individual loss function calculated for the maxillary teeth included in the training data and a mandibular individual loss function calculated for the mandible teeth included in the training data. A machine-readable non-transitory recording medium containing a computer program including instructions embodied to cause a computing device to perform the method of claim 9. In a computing device for performing a method for providing tooth lesion information,
A communication unit for receiving an image of a subject's teeth;
Processor for detecting a lesion area of the tooth image
Including,
The processor,
Inputting a tooth image to a pre-learned readout model to detect a lesion area included in the tooth image,
Providing tooth lesion information on the lesion area to an external entity,
The readout model,
It is learned in advance in the direction of minimizing the result of the loss function calculated through the correlation parameter for the correlation between individual teeth,
The correlation parameter,
A computing device that is calculated based on a correlation between information on whether or not lesions of individual teeth included in the training data have occurred. The method of claim 16,
The correlation parameter,
It is calculated using a correlation matrix calculated based on a correlation between information on whether the lesion has occurred,
The correlation parameter is calculated based on Equation 1,
Equation 1 is,

ego,

Is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

Is a value to normalize between 0 and 1,

Is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

Denotes a value indicating whether a lesion of the j'th tooth of the ith learning data has occurred. In a computing device for learning a readout model for detecting a lesion area in a tooth image,
Communication department;
A processor that trains the readout model in a direction that minimizes the result of the loss function calculated through the training data
Including,
The loss function is,
It is calculated using a correlation parameter for a correlation between individual teeth included in the learning data,
The correlation parameter,
A computing device that is calculated based on a correlation between information on whether or not lesions of individual teeth included in the learning data have occurred. The method of claim 18,
The correlation parameter,
It is calculated using a correlation matrix calculated based on a correlation between information on whether the lesion has occurred,
The correlation parameter is calculated based on Equation 1,
Equation 1 is,

ego,

Is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

Is a value to normalize between 0 and 1,

Is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

Denotes a value indicating whether a lesion of the j'th tooth of the ith learning data has occurred.

Images