KR20220019860A - Teeth examination system using deep learning algorithms - Google Patents

Abstract

Disclosed is a dental examination method using a deep learning algorithm. The dental examination method using the deep learning algorithm comprises: a dental image acquisition step of photographing the upper jaw teeth image and the lower jaw teeth image with an oral camera; an individual tooth ROI image extraction step of extracting an individual tooth ROI image by learning the upper jaw teeth image and the lower jaw teeth image with a deep learning algorithm; and a dental disease prediction step of learning the individual tooth ROI image with the deep learning algorithm to determine whether or not the corresponding tooth needs a dental treatment. Therefore, the present invention is capable of effectively managing the dental health of a user.

Description

Teeth examination system using deep learning algorithms

The present invention relates to a dental examination method, and more particularly, to a dental examination method using a deep learning algorithm.

Oral diseases, including tooth decay, periodontal disease and tooth loss, are one of the most prevalent diseases worldwide and reduce people's quality of life due to the pain and economic loss caused by the disease. If oral health is managed through continuous monitoring, treatment at an early stage and progression of oral disease can be prevented. With the development of network and sensor technology, various IOT-based health care services are being provided, and people's health management is supported in various ways. However, among these IOT-based healthcare service platforms, there is a lack of platforms and related research that provide services so that individuals can conveniently receive oral care and services.

In recent years, deep learning has shown tremendous potential in computer vision fields such as classification, object detection, instance segmentation, and image captioning. With the advancement of CPU, GPU, and dataset, deep learning-based methodologies have produced promising results in the medical field. It has been applied to most clinical-level dental problems analyzing medical images such as radiographic. Most approaches to the dental industry have been in the form of dental practice assistants using a Convolutional Neural Network (CNN) deep learning methodology based on tomographic and radiographic images. In particular, studies are being actively conducted on the potential risk of nerve damage after removal of a wisdom tooth using radiographic, a method for predicting sensory impairment of the lower lip and jaw, a method for detecting periodontal bone loss, and a method for classifying dental diseases. there is.

The present invention provides a dental examination method using a deep learning algorithm that can effectively manage dental health.

A dental examination method using a deep learning algorithm according to the present invention includes: a tooth image acquisition step of photographing an image of a maxillary tooth and an image of a mandible with an oral camera; an individual tooth ROI image extraction step of extracting an individual tooth ROI image by learning the maxillary tooth image and the mandibular tooth image using a deep learning algorithm; and a dental disease prediction step of learning the individual tooth ROI image with a deep learning algorithm to determine whether the corresponding tooth needs dental treatment.

In addition, the oral camera may include a body having a head and a torso; a camera lens provided on one surface of the head; LED lighting provided on one surface of the head; and a communication unit that provides image information input through the camera lens to a user terminal.

In addition, the tooth image acquisition step extracts a molar tooth image and an incisor tooth image from each of the maxillary tooth image and the mandibular tooth image input from the camera lens, and the front boundary line of the molar tooth image and the incisor tooth image. Create a reference guide line connecting the front boundary line, create an auxiliary guide line behind the reference guide line at an interval corresponding to the thickness of the molar tooth image, and set the reference guide line and the auxiliary guide line to the user terminal can be displayed on the display of

In addition, the tooth image acquisition step extracts a tooth ROI box from each of the maxillary tooth image and the mandibular tooth image input from the camera lens, and connects the outer corners of the tooth ROI boxes to generate a reference guideline, An auxiliary guard line may be generated by connecting the inner corners, and the reference guide line and the auxiliary guide line may be displayed on the display of the user terminal.

In addition, the dental disease prediction step may provide an image of the tooth disease by applying image blending.

According to the present invention, it is easy to verify the user's dental disease, and the need for dental visit treatment can be accurately and quickly provided to the user, so that the user's dental health can be effectively managed.

1 is a diagram illustrating a dental examination system using a deep learning algorithm according to an embodiment of the present invention.
Figure 2 is a view showing an oral camera according to an embodiment of the present invention.
3 is a view showing an image of the maxillary teeth taken with the oral camera of FIG. 2 .
Figure 4 is a view showing an image of the mandible taken with the oral camera of Figure 2;
5 is a diagram illustrating a user terminal on which guide lines are displayed according to an embodiment of the present invention.
6 is a diagram illustrating a process of extracting an individual tooth ROI image by a deep learning analysis engine learning with a deep learning algorithm according to an embodiment of the present invention.
7 is a diagram illustrating an individual tooth ROI image extracted in FIG. 6 .
8 is a diagram illustrating a disease image of a tooth to which Poisson image blending is applied.
9 is a flowchart illustrating a dental examination method using a deep learning algorithm according to an embodiment of the present invention.
10 is a diagram illustrating an individual tooth ROI image and a tooth disease image displayed on a user terminal according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the technical spirit of the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content may be thorough and complete, and the spirit of the present invention may be sufficiently conveyed to those skilled in the art.

In this specification, when a component is referred to as being on another component, it means that it may be directly formed on the other component or a third component may be interposed therebetween. In addition, in the drawings, thicknesses of films and regions are exaggerated for effective description of technical content.

In addition, in various embodiments of the present specification, terms such as first, second, third, etc. are used to describe various components, but these components should not be limited by these terms. These terms are only used to distinguish one component from another. Accordingly, what is referred to as a first component in one embodiment may be referred to as a second component in another embodiment. Each embodiment described and illustrated herein also includes a complementary embodiment thereof. In addition, in this specification, 'and/or' is used in the sense of including at least one of the components listed before and after.

In the specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as "comprise" or "have" are intended to designate that a feature, number, step, element, or a combination thereof described in the specification exists, but one or more other features, number, step, configuration It should not be construed as excluding the possibility of the presence or addition of elements or combinations thereof. In addition, in this specification, "connection" is used in a sense including both indirectly connecting a plurality of components and directly connecting a plurality of components.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a view showing a dental examination system using a deep learning algorithm according to an embodiment of the present invention, FIG. 2 is a view showing an oral camera according to an embodiment of the present invention, and FIG. 3 is taken with the oral camera of FIG. It is a view showing an image of a maxillary tooth, and FIG. 4 is a view showing an image of a mandible taken with the oral camera of FIG. 2 .

1 to 4 , the dental examination system 10 using a deep learning algorithm includes an oral camera 100 , a user terminal 200 , and an image analysis unit 300 .

The oral camera 100 is inserted into the user's mouth or takes images of the upper and lower teeth outside the user's mouth. The intraoral camera 100 includes a body 110 , 120 , a camera lens 130 , a light 140 , a communication unit 150 , and a battery 160 .

The bodies 110 and 120 are provided with a thin thickness that can be inserted into the user's mouth, and include a body part 110 that the user can hold by hand, and a head part 120 provided in front of the body part 110 . do.

The camera lens 130 is provided on one surface of the head 120, and a tooth image is input. The camera lens 130 may have a wide angle of view. According to an embodiment, the camera lens 130 may have an angle of view of 100° to 150°. Preferably, the camera lens 130 may have an angle of view of 120°. The camera lens 130 may have a focal length of 1 to 3 cm in consideration of the distance from the maxillary teeth. Through the camera lens 130 , the entire area of the maxillary tooth and the entire area of the mandible may be captured as individual images.

The illumination 140 is provided on one surface of the head 120 , and irradiates light to the front of the camera lens 130 . The light 140 may be located at a point adjacent to the camera lens 130 . As the lighting 140 , a general LED, an infrared LED, and an LED for Q-ray inspection may be used.

The communication unit 150 is located in the body 110 , and transmits image information input through the camera lens 130 to the user terminal 200 .

The battery 160 is located in the body 110, and provides power to each component of the oral camera 100).

Although not shown in the drawings, the oral camera 100 may further include a gas sensor, a heat wire, and a HW photographing guide.

The gas sensor is provided on the head 120 and measures the user's bad breath when taking a dental image. The measured bad breath is transmitted to the user terminal 200 through the communication unit 150 . The user terminal 200 may provide a change in the user's bad breath and a change in tooth decay to the user.

The heating wire is embedded in the periphery of the camera lens 130 , and prevents fogging by heating the camera lens 130 .

The HW shooting guide is coupled to the other surface of the head 120, and provides a groove that can be hung over the teeth. When the camera lens 130 captures the maxillary teeth, the mandibular teeth are inserted into the grooves of the HW imaging guide, and when the camera lens 130 captures the mandibular teeth, the maxillary teeth are inserted into the grooves of the HW imaging guides. When a tooth is inserted into the groove of the HW shooting guide, shaking and shaking of the oral camera are prevented, so that a stable image can be obtained.

A dedicated application for dental examination is installed in the user terminal 200 , and image information is received from the oral camera 100 through the application, and the received image information is transmitted to the image analysis unit 300 . And the user terminal 200 displays the image transmitted from the oral camera 100 and the image analysis unit 300 on the display.

In addition, the user terminal 200 generates a guide line on the image information transmitted from the oral camera 100, and displays the generated guide line on the display together with the image information.

5 is a diagram illustrating a user terminal on which a guide line is displayed according to an embodiment of the present invention.

Referring to FIG. 5 , the user terminal extracts a molar tooth image T1 and an incisor tooth image T2 from each of the maxillary tooth image and the mandibular tooth image, and the front boundary line of the molar tooth image T1 and the incisor tooth image T2 ), create a reference guide line (L1) connecting the anterior boundary of and the reference guide line L1 and the auxiliary guide line L2 are displayed on the display unit of the user terminal 200 . The user may acquire the maxillary tooth image and the mandibular tooth image, respectively, by aligning his/her teeth indicated in the image information to the guide lines L1 and L2.

According to another embodiment, the user terminal extracts a tooth ROI box (11 in FIG. 6) from each of the maxillary tooth image and the mandibular tooth image input from the camera lens 130, and connects the outer corners of the tooth ROI boxes to guide the reference The auxiliary guard line L2 may be created by creating the line L1 and connecting the inner corners. The user terminal displays the reference guide line L1 and the auxiliary guide line L2 on the display unit of the user terminal 200 . The user may acquire the maxillary tooth image and the mandibular tooth image, respectively, by aligning his/her teeth indicated in the image information to the guide lines L1 and L2.

The image analysis unit 300 receives the maxillary tooth image and the mandibular tooth image from the user terminal 200, learns the image with a deep learning algorithm, extracts an individual tooth ROI (Region Of Interest) image, and extracts an individual tooth ROI image is learned with a deep learning algorithm to determine whether or not dental treatment is required for the corresponding tooth.

And the image analysis unit 300 transmits the disease image of the tooth to the user terminal by applying image blending. For image blending, Poisson image blending, image blending using a Gaussian kernel, and a method of directly creating and blending a 2D kernel may be used. The image analysis unit 300 may be provided as a separate server or may be mounted in the user terminal.

The image analysis unit 300 includes a communication unit 310 , a data storage unit 320 , and a deep learning analysis engine 330 .

The communication unit 310 may communicate with the user terminal 200 by wire or wirelessly.

The data storage unit 320 stores various tooth image information used for learning of a deep learning algorithm. The data storage unit 320 may store the maxillary tooth image and the mandibular tooth image captured by the oral camera 100 . In addition, the data storage unit 320 may store a tooth image taken according to gender, age, and race.

6 is a diagram illustrating a process in which a deep learning analysis engine extracts an individual tooth ROI image by learning with a deep learning algorithm according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating an individual tooth ROI image extracted in FIG. .

6 and 7 , the deep learning analysis engine 330 extracts an individual tooth ROI image 11 by learning with a deep learning algorithm. The deep learning analysis engine 330 extracts the tooth ROI image 11 using a Tooth ROI network, and the Tooth ROI network is composed of a CNN network and a Region Proposal Network (RPN).

And the deep learning analysis engine 330 learns the individual tooth ROI image with a deep learning algorithm to determine whether the corresponding tooth needs dental treatment. The deep learning analysis engine 330 uses a Feature Pyramid Network (FPN) method to effectively deliver high-level features and low-level features of an image to a CNN network such as ResNet for feature extraction from individual tooth ROI images. can The deep learning analysis engine 330 may generate an image of a tooth disease by applying Poisson image blending. 8 is a diagram illustrating a disease image of a tooth to which Poisson image blending is applied.

The deep learning analysis engine 330 may predict various dental diseases including occlusal caries, proximal caries, cavitation, dental fluorosis, periodontitis, cracked tooth, dental calculus, dental plaque, and tooth loss through the disease image of the tooth.

9 is a flowchart illustrating a dental examination method using a deep learning algorithm according to an embodiment of the present invention.

Referring to FIG. 9 , a dental examination method using a deep learning algorithm includes a tooth image acquisition step (S10), an individual tooth ROI image extraction step (S20), and a dental disease prediction step (S30).

In the tooth image acquisition step (S10), the user's upper and lower teeth images are captured while the user inserts the head 120 of the oral camera 100 into the mouth. When the LED lighting 140 is on, the camera lens 130 photographs the maxillary or mandibular teeth, and the captured image is transmitted to the user terminal 200 through the communication unit 150 . The user terminal 200 displays the image transmitted from the camera lens 130 on the display unit in real time by executing the dedicated application, analyzes the teeth displayed in the image, and displays the guide lines L1 and L2 on the display unit. The application extracts the molar teeth (T1) and the incisors (T2) shown in the image, creates a reference guideline (L1) that connects the anterior boundary line of the molar tooth image and the front boundary line of the incisor tooth image, and the thickness of the molar tooth image ( An auxiliary guide line L2 is generated behind the reference guide line L1 at an interval corresponding to d). The user may photograph the maxillary tooth image and the mandibular tooth image by matching the tooth line displayed on the image with the guide lines L1 and L2. The maxillary tooth image and the mandibular tooth image are transmitted to the image analysis unit through the user terminal.

In the individual tooth ROI image extraction step (S20), an individual tooth ROI image is extracted by learning the maxillary tooth image and the mandibular tooth image with a deep learning algorithm. Deep learning algorithm learning for individual tooth ROI image extraction is performed by a professional dentist who annotates individual tooth regions of the maxillary and mandibular teeth images stored in the data storage unit in pixel units, and learns them with an ROI detection network to ROI on individual teeth. You can extract images.

In addition, when learning a deep learning algorithm, a preprocessing process may be performed first for individual images. The preprocessing process includes an image normalization step and a data augmentation step.

In the image normalization step, the input tooth image is normalized to make the distribution of the overall contrast values even, and the contrast is increased to highlight features of dental diseases. The image normalization step optimizes deep learning learning by converting image pixel values between 0 and 255 into 0-1.

The data increase step is performed to obtain more data to solve the problem of imbalance between classes to be classified and overfitting problems during deep learning learning. In the data increase step, an image processing methodology that secures various data while maintaining data characteristics such as random horizontal and vertical shift, color jitter, contrast enhancement, affine transformation, and crop may be applied.

In the tooth disease prediction step (S30), the individual tooth ROI image is learned by a deep learning algorithm to determine whether the corresponding tooth needs dental treatment. In the dental disease prediction step ( S30 ), a region requiring dental treatment may be extracted by extracting key points from the individual tooth ROI image.

According to another embodiment, the dental disease prediction step (S30) learns individual tooth ROI images with a deep learning algorithm to detect normal teeth and teeth requiring dental treatment, and finds a suspicious part in the tooth image requiring dental treatment and highlights it. can be displayed

According to another embodiment, in the tooth disease prediction step (S30), the individual tooth ROI image is learned by a deep learning algorithm and classified into normal teeth, superficially decayed teeth, deep decayed teeth, and simple colored teeth. It is possible to find and highlight the suspicious part in the decayed tooth image.

The dental disease prediction step S30 may provide an image of a tooth disease by applying the above-described image blending.

The individual tooth ROI image and the tooth disease image generated in the tooth disease prediction step S30 are transmitted to the user's terminal 200 .

10 is a diagram illustrating an individual tooth ROI image and a tooth disease image displayed on a user terminal according to an embodiment of the present invention.

Referring to FIG. 10A , a region 15 from which individual tooth ROI images are extracted from the maxillary tooth image and the mandibular tooth image is displayed. The development area 15 is given a sequence number of the individual tooth ROI image.

Referring to FIG. 10B , an ROI image of a tooth predicted to require dental treatment and a disease image 16 of a tooth to which Poisson image blending is applied are displayed. And the disease probability of the tooth is expressed as a numerical value, and the degree of need for dental treatment (17)) is displayed in stages.

Such analysis information may be transmitted to a computerized image analysis unit of a dental hospital designated by a user through a user terminal according to a user's request, and may be utilized for treatment diagnosis by a dentist. In addition, such analysis information is available for telemedicine.

As mentioned above, although the present invention has been described in detail using preferred embodiments, the scope of the present invention is not limited to specific embodiments and should be construed according to the appended claims. In addition, those skilled in the art should understand that many modifications and variations are possible without departing from the scope of the present invention.

10: dental examination system
100: oral camera
200: user terminal
300: image analysis unit

Claims (5)

A dental image acquisition step of taking a maxillary tooth image and a mandibular tooth image with an oral camera;
an individual tooth ROI image extraction step of extracting an individual tooth ROI image by learning the maxillary tooth image and the mandibular tooth image using a deep learning algorithm; and
A dental examination method using a deep learning algorithm, comprising a dental disease prediction step of learning the individual tooth ROI image with a deep learning algorithm to determine whether the corresponding tooth needs dental treatment. The method of claim 1,
the oral camera
a body having a head and a torso;
a camera lens provided on one surface of the head;
LED lighting provided on one surface of the head; and
A dental examination method using a deep learning algorithm comprising a communication unit that provides image information input through the camera lens to a user terminal. 3. The method of claim 2,
The tooth image acquisition step is,
A molar tooth image and an incisor tooth image are extracted from each of the maxillary tooth image and the mandibular tooth image input from the camera lens, and a reference guideline connecting the front boundary line of the molar tooth image and the front boundary line of the incisor tooth image. create,
creating an auxiliary guide line behind the reference guide line at an interval corresponding to the thickness of the molar tooth image,
A dental examination method using a deep learning algorithm for displaying the reference guideline and the auxiliary guideline on a display of the user terminal. 3. The method of claim 2,
The tooth image acquisition step is,
Extracting a tooth ROI box from each of the maxillary tooth image and the mandibular tooth image input from the camera lens,
Connecting the outer corners of the tooth ROI boxes to create a reference guide line, connecting the inner corners to create an auxiliary guard line,
A dental examination method using a deep learning algorithm for displaying the reference guideline and the auxiliary guideline on a display of the user terminal. The method of claim 1,
The dental disease prediction step is a dental examination method using a deep learning algorithm that provides an image of the tooth disease by applying image blending.

Images