KR102290799B1 - Method for providing tooth leison information and apparatus using the same - Google Patents

Abstract

The present disclosure relates to a method of providing tooth lesion information performed by a computing device.
The method for providing dental lesion information performed by a computing device includes the steps of (a) receiving a tooth image of a subject, (b) detecting a lesion region included in the tooth image through a pre-learned reading model, or interlocking with the computing device Supporting another device to detect the lesion region, (c) generating dental lesion information for the lesion region or supporting another device interlocked with the computing device to generate the dental lesion information; and (d) providing the lesion information to an external entity.

Description

Dental lesion information providing method and device using the same

The present invention relates to a method for providing dental lesion information and an apparatus using the same.

Various X-ray images taken by various methods are widely used for dental diagnosis and treatment. X-ray panoramic image (panorama image) is widely used to establish a treatment plan in that it can confirm the oral structure as a whole and properly reveal the anatomical structure of the oral cavity. In general, a method of estimating a lesion area by visually identifying an X-ray panoramic image is widely used in dental treatment, but technologies for automatically identifying a lesion have been developed to overcome the limitation of visual identification. Research to establish a lesion candidate region in an X-ray image is limited to using a contrast change in an X-ray image.

Republic of Korea Patent Publication No. 10-1903424

An object of the present invention is to provide a means for effectively determining a lesion area in a tooth image by providing information on whether a lesion has occurred in each tooth together with identification information of the corresponding tooth.

In addition, the present invention intends to provide visualized lesion information through a map that visualizes the possibility that each area corresponds to a lesion area in an input tooth image.

In addition, an object of the present invention is to provide a reading model capable of more accurately detecting a lesion region by using a loss function in consideration of a correlation between teeth in a process of learning a reading model for detecting a lesion region in a tooth image.

The characteristic configuration of the present invention for achieving the object of the present invention as described above and for realizing the characteristic effects of the present invention to be described later is as follows.

According to an aspect of the present invention, a method for providing dental lesion information performed by a computing device includes the steps of: (a) receiving an image of a subject's teeth; (b) detecting a lesion region included in the tooth image through a pre-learned reading model or supporting another device linked to the computing device to detect the lesion region; (c) generating dental lesion information for the lesion region or supporting another device interworking with the computing device to generate the dental lesion information; and (d) providing the lesion information to an external entity.

According to one aspect, the step (c) comprises the steps of (c1) generating a map for visualizing the lesion area in the tooth image; and (c2) generating the dental lesion information through visualization information in which a lesion region is visualized in the dental image based on the map.

According to another aspect, the step (c) may include (c3) generating identification information for identifying the position or order of individual teeth included in the tooth image; (c4) generating matching information in which the identification information is matched with information on whether a lesion has occurred in an individual tooth corresponding to the identification information; and (c5) generating the dental lesion information based on the matching information.

According to an aspect, the reading model is pre-trained in a direction to minimize a result of a loss function, and the loss function may be determined based on correlation information between individual teeth.

A computing device for providing tooth lesion information according to an aspect includes: a communication unit configured to receive an image of a subject's teeth; and a processor for generating tooth lesion information for the tooth image, wherein the processor generates tooth lesion information on a lesion region included in the tooth image through a pre-learned reading model or other devices linked to the computing device. Support the device to generate the lesion information, and provide the dental lesion information to an external entity.

According to an embodiment of the present invention, by learning a reading model based on a loss function in consideration of the correlation between teeth, it is possible to dramatically improve the accuracy of detecting a lesion region for a tooth image.

In addition, it provides a means to effectively visualize the lesion area in the tooth image through a map that visualizes the lesion detection result of the tooth image, and the reader can It is possible to provide a means for determining whether a lesion has occurred in individual teeth with a single tooth image.

According to the present invention, the potential effect of innovating the workflow in the medical field by ultimately improving the diagnosis accuracy of the medical staff and allowing the lesion of the entire tooth to be read through a single image. have.

And, in the present invention, the conventional medical images used in hospitals can be used as they are, so it goes without saying that the method of the present invention is not dependent on a specific type of image or platform.

The accompanying drawings for use in the description of the embodiments of the present invention are only a part of the embodiments of the present invention, and for those of ordinary skill in the art to which the present invention pertains (hereinafter referred to as "a person skilled in the art"), Other drawings may be obtained based on these drawings without an effort leading to the invention.
1 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device for performing a method for providing lesion information according to the present invention.
2 is an exemplary block diagram illustrating hardware or software components of a computing device performing a method for providing lesion information according to the present invention.
3 is a flowchart for explaining a method for providing information on dental lesions according to an embodiment.
4 is a diagram illustrating an example in which a method for providing lesion information according to an embodiment is performed.
5 is a diagram illustrating an example in which a Pearson correlation matrix used in a process of determining a loss function is visualized.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description of the present invention refers to the accompanying drawings, which show by way of illustration a specific embodiment in which the present invention may be practiced, in order to clarify the objects, technical solutions and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention.

The term "image" or "image data" as used throughout the present description and claims refers to multidimensional data composed of discrete image elements (e.g., pixels in a two-dimensional image, voxels in a three-dimensional image). refers to For example, "imaging" means X-ray imaging, (cone-beam) computed tomography, magnetic resonance imaging (MRI), ultrasound or any other medical imaging known in the art. It may be a medical image of a subject, ie, a subject, collected by the system. Also, the image may be provided in a non-medical context, for example, a remote sensing system, an electron microscopy, and the like.

Throughout the description and claims of the present invention, 'image' refers to a viewable image (eg, displayed on a video screen) or an image (eg, a file corresponding to a pixel output of a CT, MRI detector, etc.) A term that refers to digital representations.

Throughout the detailed description and claims of the present invention, the 'DICOM (Digital Imaging and Communications in Medicine)' standard is a generic term for various standards used for digital image representation and communication in medical devices, DICOM Standards are published by a joint committee formed by the American Society of Radiological Medicine (ACR) and the National Electrical Engineers Association (NEMA).

In addition, throughout the detailed description and claims of the present invention, 'picture archiving and communication system (PACS)' is a term that refers to a system that stores, processes, and transmits in accordance with the DICOM standard, X-ray, CT , medical imaging images acquired using digital medical imaging equipment such as MRI are stored in DICOM format and can be transmitted to terminals inside and outside the hospital through a network, to which reading results and medical records can be added.

And throughout the detailed description and claims of the present invention, 'learning' or 'learning' is a term referring to performing machine learning through computing according to a procedure, and a mental action such as human educational activity. It will be understood by those of ordinary skill in the art that this is not intended to be a reference.

And throughout this description and claims, the word 'comprise' and variations thereof are not intended to exclude other technical features, additions, components or steps. In addition, 'one' or 'an' is used to mean more than one, and 'another' is limited to at least a second or more.

Other objects, advantages and characteristics of the present invention will appear to a person skilled in the art, in part from this description, and in part from practice of the present invention. The following illustrations and drawings are provided by way of illustration and are not intended to limit the invention. Therefore, the details disclosed herein with respect to a specific structure or function are not to be construed in a limiting sense, but merely representative should be interpreted as basic data.

Moreover, the invention encompasses all possible combinations of the embodiments indicated herein. It should be understood that various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment. In addition, it should be understood that the position or arrangement of individual components in each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the present invention, if properly described, is limited only by the appended claims, along with all scope equivalents to those claimed. Like reference numerals in the drawings refer to the same or similar functions throughout the various aspects.

Unless otherwise indicated herein or otherwise clearly contradicted by context, items referred to in the singular encompass the plural unless the context requires otherwise. In addition, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, in order to enable those skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an exemplary configuration of a computing device for performing a method for providing lesion information according to the present invention.

Referring to FIG. 1 , a computing device 100 according to an embodiment of the present invention includes a communication unit 110 and a processor 120 , and is directly or indirectly connected to an external computing device (not shown) through the communication unit 110 . can communicate effectively.

Specifically, the computing device 100 includes typical computer hardware (eg, a computer processor, memory, storage, input device and output device, device that may include other components of a conventional computing device; router, switch, etc.) Electronic communication devices; electronic information storage systems, such as network-attached storage (NAS) and storage area networks (SANs)) and computer software (i.e., that enable the computing device to function in a particular way). instructions) to achieve the desired system performance.

The communication unit 110 of such a computing device may transmit and receive a request and a response to and from another computing device that is interlocked. As an example, such a request and a response may be made by the same transmission control protocol (TCP) session. , but is not limited thereto, and may be transmitted and received as, for example, a user datagram protocol (UDP) datagram. In addition, in a broad sense, the communication unit 110 may include a keyboard, a mouse, other external input devices, printers, displays, and other external output devices for receiving commands or instructions.

In addition, the processor 120 of the computing device may include a micro processing unit (MPU), a central processing unit (CPU), a graphics processing unit (GPU) or a tensor processing unit (TPU), a cache memory, and a data bus. ) may include a hardware configuration such as In addition, it may further include an operating system, a software configuration of an application for performing a specific purpose.

In the following description, the image of the present invention is a dental image, and the dental image is a dental X-ray panoramic image as an example. Those skilled in the art will readily understand that it is applicable and can be applied in the process of detecting an arbitrary lesion area included in a target image.

2 is an exemplary block diagram illustrating hardware or software components of a computing device performing a method for providing lesion information according to the present invention.

The individual modules shown in FIG. 2 may be implemented by, for example, the communication unit 110 or the processor 120 included in the computing device 100 or the communication unit 110 and the processor 120 interworking. technicians will be able to understand.

A brief overview of the configuration of the method and apparatus according to the present invention with reference to FIG. 2 , the computing device 100 may include an image acquisition module 210 as a component thereof. The image acquisition module 210 may acquire an image of the subject's teeth stored in advance in a database or acquired from a photographing-only device for photographing an image. The dental image may be an X-ray panoramic image of the subject's teeth photographed through a photographing-only device built in the computing device 100 or an external photographing-only device.

The tooth image acquired through the image acquisition module 210 may be transmitted to the tooth lesion region detection module 220 . The tooth lesion area detection module 220 may detect the lesion area included in the tooth image through the read model. The read model may be an artificial neural network pre-trained to detect a lesion region in a tooth image. The reading model included in the dental lesion region detection module 220 is a deep learning-based artificial neural network trained to output at least one of the position of the lesion region, the shape of the lesion region, and the probability of corresponding to the lesion region in the input dental image. can The artificial neural network included in the tooth lesion area detection module 220 determines whether an area included in the input tooth image and a detection target area (eg, a bone loss area) match based on the similarity, and outputs the determination result. As it is learned to do, for example, DNN (Deep Neural Network), CNN (Convolutional Neural Network), DCNN (Deep Convolution Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network) ), a Single Shot Detector (SSD), You Only Look Once (YOLO), etc. may be used, as will be understood by those skilled in the art. The type of artificial neural network that can be used in the dental lesion region detection module 220 is not limited to the presented example, and may include any artificial neural network that can be trained to recognize a lesion region in a tooth image based on labeled learning data. It will be understood by those of ordinary skill in the art.

The detection result of the lesion area performed through the dental lesion area detection module 220 may be transmitted to the visualization module 230 , and the visualization module 230 may generate a map for visualizing the lesion area in the tooth image. . For example, the visualization module 230 may generate a heat map in which a color of a corresponding area is changed according to a probability that each area of the tooth corresponds to a lesion area (eg, a bone loss area). . The method of generating the map by the visualization module 230 is not limited to the method described above, and any method that can visually express the probability that each area of the tooth image corresponds to the lesion area (for example, in the lesion area) It will be apparent to those skilled in the art that the corresponding probability may be numerically expressed, etc.).

The detection result performed through the tooth lesion area detection module 220 may be transmitted to the classification module 240 , and the classification module 240 may determine identification information of a tooth corresponding to the detected lesion area. The classification module 240 may generate identification information for identifying the position or order of individual teeth included in the tooth image, and may determine identification information of a tooth in which a lesion area is detected based on the generated identification information and the detection result. . Identification information for individual teeth may be determined based on the FDI World Dental Federation notation, which is the tooth identification number most used by dentists around the world, but is not limited thereto, and is based on an arbitrary method that can identify individual teeth. can be decided. For example, when a tooth identification number is used as identification information, the classification module 240 determines an identification number of a tooth in which a lesion is detected from among identification numbers generated for individual teeth, and based on the determination result, the identification number and Matching information by matching information on whether a lesion has occurred in an individual tooth may be generated. Matching information is a method of providing identification information of teeth determined to have a lesion as a list according to implementation, a method of matching identification information of all teeth with whether a lesion has occurred, etc. Any matching of identification information with information on whether a lesion has occurred can be created in the manner of

The tooth lesion information generation and transmission module 250 may generate tooth lesion information based on a map generated by the visualization module 230 or matching information generated through the classification module 240 . The dental lesion information may be determined based on at least one of visualization information generated by applying the map to a tooth image or information about a tooth list in which a lesion is detected based on the matching information.

The dental lesion information generated through the tooth lesion information generation and transmission module 250 may be stored in a database or provided to an external entity. When the dental lesion information is provided to an external entity, the dental lesion information generation and transmission module 250 may use a predetermined display device or the like or provide an output image in which the dental lesion information is reflected to an external entity through a communication unit provided. . Here, the external entity includes a user of the computing device 100 , an administrator, a medical professional in charge of the subject, and the like, but in addition to this, a subject requiring dental lesion information for the input dental image If so, it should be understood that any subject is included. For example, the external entity may be an external AI device including a separate artificial intelligence (AI) hardware and/or software module that utilizes the tooth lesion information. In addition, 'external' in the external entity is not intended to exclude an embodiment in which an AI hardware and/or software module using the tooth lesion information is integrated into the computing device 100, but the method of the present invention It is disclosed that the dental lesion information, which is the result of the hardware and/or software module performing the That is, the external entity may be the computing device 100 itself.

Although the components shown in FIG. 2 are exemplified as being realized in one computing device for convenience of description, it will be understood that a plurality of computing devices 100 performing the method of the present invention may be configured to interwork with each other.

Now, an embodiment of the image providing method according to the present invention will be described in more detail with reference to FIGS. 3 to 5 .

3 is a flowchart for explaining a method for providing information on dental lesions according to an embodiment.

Referring to FIG. 3 , the computing device may receive an image of the subject's teeth through the communication unit in step S100 . The dental image may be a dental X-ray panoramic image as described above, but is not limited thereto, and may include any type of image obtained by photographing the subject's teeth.

The computing device may detect the lesion region included in the tooth image through the read model in step S200 . The reading model may include an artificial neural network pre-trained to detect a lesion region in a tooth image.

The loss function for training the reading model may be determined based on the maxillary loss function for the maxillary teeth and the mandibular loss function for the mandibular teeth, along with a cross entropy function commonly used as a loss function. The upper jaw may mean an upper jaw included in the tooth image, and the mandible may mean the lower jaw included in the tooth image. Accordingly, the maxillary loss function may mean a loss function determined corresponding to the maxilla, and the mandibular loss function may mean a loss function corresponding to the mandible.

More specifically, individual loss functions for calculating the maxillary loss function and the mandibular loss function may be determined based on Equation (1).

는 개별 손실 함수,

는 치아의 병변 발생 여부를 나타내는 값,

는 i번째 데이터 샘플의 j번째 치아의 병변 발생 여부를 나타내는 값,

는 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 i번째 데이터 샘플의 j번째 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 피어슨 상관 행렬(pearson correlation matrix),

는 피어슨 상관 행렬의 각 요소로써 j번째 치아와 j'번째 치아에 상응하는 상관 계수 값c는 피어슨 상관 행렬에 기초하여 결정되는 상관 파라미터,

는 i번째 데이터 샘플의 j번째 치아에 대해 산출된 상관 파라미터, k는 치아 번호의 개수,

은 데이터 샘플의 개수,

는

를 0 및 1사이로 정규화하기 위한 값을 나타낸다.

is the individual loss function,

is a value indicating whether a tooth lesion occurs,

is a value indicating whether a lesion occurs on the j-th tooth of the i-th data sample,

is the predictive value of the readout model for the occurrence of tooth lesions,

is the predicted value of the reading model for the occurrence of a lesion on the j-th tooth of the i-th data sample,

is a Pearson correlation matrix,

is each element of the Pearson correlation matrix, and the correlation coefficient value c corresponding to the j-th tooth and the j'-th tooth is a correlation parameter determined based on the Pearson correlation matrix,

is the correlation parameter calculated for the j-th tooth of the i-th data sample, k is the number of tooth numbers,

is the number of data samples,

Is

represents a value for normalizing between 0 and 1.

는 하기의 수학식 2에 기초하여 결정될 수 있다.Correlation coefficient between the a-th tooth and the b-th tooth included in the Pearson correlation matrix of Equation 1

may be determined based on Equation 2 below.

는 a번째 치아와 b번째 치아의 상관 계수,

는 i번째 학습 데이터에서 a번째 치아의 병변 발생 여부를 나타내는 값,

는i번째 학습 데이터에서 b번째 치아의 병변 발생 여부를 나타내는 값,

는

의 평균,

는

의 평균을 의미할 수 있다.

is the correlation coefficient between the a-th tooth and the b-th tooth,

is a value indicating whether a lesion occurs in the a-th tooth in the i-th training data,

is a value indicating whether a lesion occurs in the b-th tooth in the i-th training data,

Is

mean of,

Is

can mean the average of

=1, 병변이 발생하지 않는 경우

로 출력된다.)은 a = {1, 0, 0, 0, 0, 1, 1}이고, b = {1, 1, 0, 0, 0, 1, 1}로 학습 데이터가 7개인 경우,

는 0.428,

는 0.571로 산출되고, 수학식 2에 따라

는 0.75로 산출될 수 있다. For example, the value of whether a lesion occurs on the ath tooth (if a lesion occurs)

=1, if no lesion occurs

) is a = {1, 0, 0, 0, 0, 1, 1} and b = {1, 1, 0, 0, 0, 1, 1}, where there are 7 training data,

is 0.428,

is calculated as 0.571, and according to Equation 2

can be calculated as 0.75.

An example in which the correlation coefficient calculated based on Equation 2 is visualized is shown in FIG. 5 attached below.

Based on Equation 1, the maxillary loss function and the mandibular loss function may be calculated as in Equations 3 and 4.

는 상악 치아의 병변 발생 여부를 나타내는 값,

는 상악 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 상악 치아에 대한 상관 파라미터를 나타낸다.

is a value indicating whether a lesion occurs in the maxillary teeth,

is the predicted value of the readout model for the occurrence of lesion in the maxillary teeth,

represents the correlation parameters for the maxillary teeth.

는 하악 치아의 병변 발생 여부를 나타내는 값,

는 하악 치아의 병변 발생 여부에 대한 판독 모델의 예측 값,

는 하악 치아에 대한 상관 파라미터를 나타낸다.

is a value indicating whether a lesion of the mandible occurs,

is the predicted value of the readout model for the occurrence of lesion in the mandible,

represents the correlation parameters for the mandibular teeth.

Based on Equations 3 and 4, the final loss function used for learning the read model may be calculated as Equation 5.

는 조절 가능한 파라미터를 의미할 수 있다.CrossEntropy is a cross entropy function,

may mean an adjustable parameter.

및

에 기초하여 결정된다. 실제로 치아 병변의 경우 인접한 치아들 사이에 함께 발생되는 경우가 많으므로, 병변 발생과 관련하여 인접한 치아들 사이에 상관 관계는 매우 높을 수 있다. 최종 손실 함수는 상악 및 하악에 포함되는 치아들 사이의 상관 관계가 반영되어 있으므로, 이에 기반하여 학습된 판독 모델의 병변 영역 예측에 대한 정확도는 상관 관계를 고려하지 않는 손실 함수를 이용하는 경우에 비해 향상될 수 있다. 보다 구체적으로, 치아들 사이의 상관 관계를 고려하지 않는 손실 함수에 기초하여 판독 모델이 학습되는 경우, 병변이 48번 치아에 발생하였으나, 47번 치아에 병변이 발생한 것으로 예측한 경우와 41번 치아에 병변이 발생한 것으로 예측하는 경우 오류는 동일하게 측정될 수 있다. 하지만, 수학식 5에 따른 최종 손실 함수에 기반하여 학습이 진행되는 경우, 치아들 사이의 상관 관계가 학습 과정에서 반영되어 있기 때문에, 상기 상황에서 41번 치아로 잘못 예측한 경우에 더 큰 패널티가 부여될 수 있고, 이를 통해 보다 예측 정확도가 높은 판독 모델이 생성될 수 있다.The final loss function is determined based on the maxillary loss function and the mandibular loss function, and the maxillary loss function and the mandibular loss function are determined based on the correlation between the teeth included in each.

and

is determined based on In fact, since dental lesions often occur together between adjacent teeth, the correlation between adjacent teeth in relation to the occurrence of lesions may be very high. Since the final loss function reflects the correlation between the teeth included in the maxilla and the mandible, the accuracy of the lesion area prediction of the read model learned based on this is improved compared to the case of using a loss function that does not consider the correlation. can be More specifically, when the reading model is trained based on a loss function that does not take into account the correlation between teeth, the lesion occurred in the 48th tooth, but it was predicted that the lesion occurred in the 47th tooth, and the 41st tooth In the case of predicting that a lesion has occurred, the error can be measured equally. However, when learning is carried out based on the final loss function according to Equation 5, since the correlation between teeth is reflected in the learning process, in the above situation, when the 41st tooth is incorrectly predicted, a greater penalty is can be given, and through this, a reading model with higher predictive accuracy can be generated.

The computing device may generate dental lesion information on the detected lesion region in step S300 or support another device linked to the computing device to generate dental lesion information. The dental lesion information may be determined by at least one of visualization information in which a lesion region is visualized in a tooth image, and matching information in which identification information of individual teeth is matched with whether or not a lesion occurs.

According to an embodiment, the computing device may generate a map for visualizing the lesion area detected in the tooth image, based on the detection result of step S200 . The computing device may determine a weight for each area of the tooth image based on the output of the last layer of the read model, and may generate a map expressing different visual elements according to the determined weight. For example, in the output of the final layer, the computing device expresses a region with a high probability of corresponding to a lesion region as a region corresponding to a high weight in a color close to red, and a region with a low probability of corresponding to a lesion region because of a low weight. It is possible to create a heat map expressed as an area close to green. The computing device applies the generated heat map to the input tooth image to visualize an area that is highly likely to correspond to the lesion area in a color close to red, and visualizes an area that is unlikely to correspond to the lesion area in a color close to green. Visualization information for the tooth image may be generated. A map for generating visualization information is not limited to the exemplary heat map and color configuration, and it can be easily understood by those skilled in the art that each region may be implemented in any way to visualize the probability that it corresponds to a lesion region. There will be.

According to another embodiment, the computing device generates identification information for identifying the position or order of individual teeth included in a tooth image, and information on whether a lesion has occurred in the generated identification information and individual teeth corresponding to the identification information It is possible to generate matching information that matches the . For example, the identification information may be an identification number for an individual tooth according to the FDI World Dental Federation notation, and the computing device provides information on whether a lesion has occurred in the tooth based on the detection result of step S200. It is possible to generate matching information matching the identification number of .

The computing device may provide tooth lesion information to an external entity in step S400 .

4 is a diagram illustrating an example in which a method for providing lesion information according to an embodiment is performed.

The computing device 420 may provide dental lesion information including visualization information 430 and matching information 440 of the input dental image 410 . To this end, the computing device 420 may detect a lesion region from the tooth image 410 through a pre-learned reading model. The computing device generates the visualization information 430 to which the heat maps 431, 432, and 433 that visualize the detected lesion area are applied, or the matching information 440 that matches the identification information of the tooth with the information on whether a lesion has occurred. By generating and providing the information to an external entity, it is possible to provide dental lesion information. The heat maps 431 , 432 , and 433 may be expressed in different colors according to the probability of corresponding to the lesion region. The identification number of the tooth included in the matching information 440 may be determined according to the FDI World Dental Federation notation shown in the identification information 441 .

5 is a diagram illustrating an example in which a Pearson correlation matrix is visualized according to an embodiment.

Lesions occurring on teeth are likely to occur continuously between adjacent teeth. Therefore, with respect to the detection of a lesion of a tooth, adjacent teeth may have a very high correlation. In the present invention, a more accurate lesion prediction model can be generated by using the final loss function corresponding to Equation 5 in which the correlation between adjacent teeth is reflected in the process of learning the artificial neural network corresponding to the read model.

Each of the matrices 510 and 520 corresponds to an example of a visualization of the Pearson correlation matrix of the maxillary tooth and the Pearson correlation matrix of the mandibular tooth calculated based on Equation (2). Elements of each matrix may correspond to correlations between teeth. Elements visualized with a darker color correspond to those calculated with high correlation, and elements visualized with a lighter color correspond to those with low correlations. Referring to the matrices 510 and 520 , it can be seen that the correlation between mutually adjacent teeth is calculated to be high, which corresponds to the above description.

Based on the description of the above embodiments, those skilled in the art will recognize that the method and/or processes of the present invention, and the steps thereof, may be implemented in hardware, software, or any combination of hardware and software suitable for a particular application. point can be clearly understood. The hardware may include general purpose computers and/or dedicated computing devices or specific computing devices or special features or components of specific computing devices. The processes may be realized by one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable devices, having internal and/or external memory. Additionally, or alternatively, the processes may be configured to process an application specific integrated circuit (ASIC), programmable gate array, programmable array logic (PAL) or electronic signals. It may be implemented with any other device or combination of devices. Moreover, the objects of the technical solution of the present invention or parts contributing to the prior arts may be implemented in the form of program instructions that can be executed through various computer components and recorded in a machine-readable recording medium. The machine-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the machine-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the machine-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM, DVD, Blu-ray, and a magneto-optical medium such as a floppy disk (magneto-optical media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include any one of the devices described above, as well as heterogeneous combinations of processors, processor architectures or combinations of different hardware and software, or stored and compiled or interpreted for execution on a machine capable of executing any other program instructions. can be created using a structured programming language such as C, an object-oriented programming language such as C++, or This includes not only bytecode, but also high-level language code that can be executed by a computer using an interpreter or the like.

Accordingly, in one aspect according to the present invention, when the method and combinations thereof described above are performed by one or more computing devices, the methods and combinations of methods may be implemented as executable code for performing respective steps. In another aspect, the method may be implemented as systems that perform the steps, the methods may be distributed in various ways across devices or all functions may be integrated into one dedicated, standalone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such sequential combinations and combinations are intended to fall within the scope of this disclosure.

For example, the hardware device may be configured to operate as one or more software modules to perform processing in accordance with the present invention, and vice versa. The hardware device may include a processor such as an MPU, CPU, GPU, TPU coupled with a memory such as ROM/RAM for storing program instructions and configured to execute instructions stored in the memory, and an external device and signal It may include a communication unit that can send and receive. In addition, the hardware device may include a keyboard, a mouse, and other external input devices for receiving commands written by developers.

In the above, the present invention has been described with specific matters such as specific components and limited embodiments and drawings, but these are only provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments, Those of ordinary skill in the art to which the present invention pertains can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and not only the claims described below, but also all modifications equivalently or equivalently to the claims described below belong to the scope of the spirit of the present invention. will do it

Such equivalent or equivalent modifications shall include, for example, logically equivalent methods capable of producing the same results as practiced by the methods according to the present invention, the spirit and scope of the present invention. should not be limited by the above examples, and should be understood in the broadest sense permitted by law.

Claims (19)

In the method for providing dental lesion information performed by a computing device,
detecting a lesion region included in the tooth image by inputting the tooth image into a pre-learned reading model; and
generating dental lesion information for the lesion area;
The loss function used to train the read model is,
A method for providing tooth lesion information, comprising a correlation parameter calculated for each individual tooth based on a lesion occurrence correlation between the individual teeth included in the learning data. According to claim 1,
The correlation parameter is
It is calculated using a correlation matrix related to the occurrence of lesions between teeth, wherein each element of the correlation matrix represents a degree of correlation between the occurrence of lesions between teeth corresponding to a row and column of the corresponding element. 3. The method of claim 2,
The correlation parameter is calculated based on Equation 1 below,
Equation 1

here,

is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

is a value for normalizing between 0 and 1,

is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

is a value indicating whether a lesion occurs in the j'th tooth of the i-th learning data, a method of providing tooth lesion information. 4. The method of claim 3,
The loss function is

including an individual loss function calculated based on

represents the correlation parameter calculated for the j-th tooth of the i-th training data,

is a method of providing tooth lesion information, indicating a prediction value of the reading model for whether a lesion occurs in the j-th tooth of the i-th training data. 5. The method of claim 4,
The individual loss function is calculated based on Equation 2 below,
Equation 2

here,

is the individual loss function,

is a value indicating whether a tooth lesion occurred in the learning data,

is the predictive value of the readout model for the occurrence of tooth lesions,

is the predicted value of the reading model for the occurrence of a lesion on the j-th tooth of the i-th training data,

is the number of training data, k is the number of tooth numbers, c is a correlation parameter determined based on a correlation matrix,

represents a correlation parameter calculated for the j-th tooth of the i-th learning data, a method of providing tooth lesion information. 5. The method of claim 4,
The individual loss function is
A method of providing tooth lesion information, which is calculated for each of the maxillary and mandibular teeth included in the learning data. According to claim 1,
The tooth lesion information is,
The method for providing dental lesion information, comprising a visualization map in which the individual regions are identified in different forms according to a probability that the individual regions included in the dental image correspond to the lesion regions. According to claim 1,
The method of providing dental lesion information, further comprising providing a user interface reflecting the dental lesion information for the lesion region to an external entity. A computer program comprising instructions embodied to cause a computing device to perform the method of any one of claims 1 to 8, stored on a readable medium. A learning method for learning a reading model for detecting a lesion area from a tooth image,
for the read model, calculating a result of a loss function through training data; and
training the read model in a direction that minimizes the result of the loss function;
The loss function is
A learning method for learning a reading model for detecting a lesion region from a tooth image, which includes a correlation parameter calculated for each individual tooth based on the correlation between the occurrence of lesions between the individual teeth included in the training data. 11. The method of claim 10,
The correlation parameter is
It is calculated using a correlation matrix related to the occurrence of lesions between teeth, and each element of the correlation matrix represents a correlation of lesion occurrence between teeth corresponding to the row and column of the corresponding element. How to learn to learn. 12. The method of claim 11,
The correlation parameter is calculated based on Equation 1 below,
Equation 1

here,

is the correlation parameter calculated for the j-th tooth of the i-th training data, k is the number of tooth numbers,

Is

is a value for normalizing between 0 and 1,

is the correlation coefficient corresponding to the j-th tooth and the j'-th tooth as each element of the correlation matrix,

A learning method for learning a reading model for detecting a lesion region from a tooth image, which means a value indicating whether a lesion occurs in the j'th tooth of the i-th training data. 13. The method of claim 12,
The loss function is

including an individual loss function calculated based on

represents the correlation parameter calculated for the j-th tooth of the i-th training data,

is a learning method for learning a reading model for detecting a lesion region from a tooth image, indicating a predicted value of the reading model for whether a lesion occurs in the j-th tooth of the i-th training data. 14. The method of claim 13,
The individual loss function is calculated based on Equation 2 below,
Equation 2

here,

is the individual loss function,

is a value indicating whether a tooth lesion occurred in the learning data,

is the predictive value of the readout model for the occurrence of tooth lesions,

is the predicted value of the reading model for the occurrence of a lesion on the j-th tooth of the i-th training data,

is the number of training data, k is the number of tooth numbers, c is a correlation parameter determined based on a correlation matrix,

is a learning method for learning a reading model for detecting a lesion region from a tooth image, representing a correlation parameter calculated for the j-th tooth of the i-th training data. 14. The method of claim 13,
The individual loss function is
A learning method for learning a reading model for detecting a lesion region from a tooth image, which is calculated for each of the maxillary and mandibular teeth included in the training data. A computer program, stored on a readable medium, comprising instructions implemented to cause a computing device to perform the method of any one of claims 10 to 15. communication department; and a processor, wherein the computing device is implemented to perform the method of any one of claims 1 to 8, for performing a method of providing dental lesion information. communication department; and a processor, wherein the computing device trains a reading model for detecting a lesion region in a tooth image, which is implemented to perform the method of any one of claims 10 to 15. delete

Images