KR102421739B1 - System and method for monitoring oral health using camera device - Google Patents

Abstract

The management server analyzes and manages the image obtained as the user's mouth or inside the ear is photographed using the portable camera device, and the mobile terminal receives and displays status information about the conception or the inside of the ear from the management server A method is provided.

Description

SYSTEM AND METHOD FOR MONITORING ORAL HEALTH USING CAMERA DEVICE

The embodiments relate to an oral condition monitoring system and method using a camera device, and more particularly, to a system and method for obtaining information about a user's oral (or ear) condition based on an image captured using a portable camera device. it's about

Recently, interest in a method for self-diagnosing diseases in the oral cavity or diseases in the ear is increasing. In particular, in the case of periodontal disease, it is often associated with other diseases of the body (diabetes, lung disease, heart disease, arteriosclerosis, liver disease, etc.) it is known In the case of severe treatment in the dental field, more than 40% of cases are caused by not detecting teeth or gum problems at an early stage.

Meanwhile, since the outbreak of COVID-19, the demand for and need for non-face-to-face remote diagnosis/treatment has increased. Accordingly, there is a need for a technology that allows a patient to effectively obtain diagnostic information for diseases such as dental caries, cracks, pigmentation, or other ear-related diseases, without requiring a patient to visit a hospital.

Korean Patent Application Laid-Open No. 10-2019-0111909 (published on October 02, 2019) relates to a system for monitoring the use of an oral appliance, wherein the oral appliance includes a body positioned in the user's mouth when in use, , discloses that the system comprises a device monitoring device attached to or embedded within the oral appliance in use.

The information described above is for understanding only, and may include content that does not form a part of the prior art.

In one embodiment, a management server analyzes and manages an image obtained as the user's mouth or inside the ear is photographed using a portable camera device, and the mobile terminal, from the management server, obtains state information about the concept or the inside of the ear A method for receiving and displaying can be provided.

In one aspect, in a method for obtaining state information about a user's mouth or ear, performed by a mobile terminal communicating with a portable camera device and a management server, the user's mouth or inside of the ear is photographed by the portable camera device In response, acquiring an image of the user's mouth or inside the ear from the portable camera device, the mobile terminal, transmitting the acquired image to a management server - The transmitted image is transmitted from the management server to the user is accumulated and managed together with the image inside the user's mouth or ear previously acquired for There is provided a method for obtaining state information about the oral cavity or ear, comprising the steps of receiving, from the management server, the state information about the user's oral cavity or ear, and displaying the received state information.

The method includes the steps of transmitting the obtained image to a consultation server - The consultation server is configured to be able to communicate with a terminal used by an oral or ear specialist, and the image transmitted to the consultation server can be checked by the specialist Provided to the terminal used by the specialist -, from the consultation server, receiving specialist diagnosis information on the user's mouth or ear - The specialist diagnosis information is based on information input from the terminal used by the specialist generated by the consultation server, and displaying the received diagnosis information of the specialist.

The terminal used by the specialist is configured to be accessible to the management server through the consultation server, and the diagnostic information of the specialist is generated based on an image of the user's mouth or the inside of the ear that is accumulated and managed in the management server it may have been

The received state information is diagnostic information about the user's mouth or ear obtained by analyzing the images of the user's mouth or inside the ear obtained for the user for a predetermined period in the management server by the management server may include.

The diagnosis information may be generated as a result of the AI model implemented in the management server analyzing images of the user's mouth or inside the ear.

Each of the images includes dental caries data, and the diagnosis information is based on data output as a result of inputting features extracted from the dental caries data into a class classifier learned in advance based on predetermined test data. can be created by

Each of the images is a photograph of the user's teeth, and the diagnosis information is, among the images, the color of the teeth included in the first image captured at a first point in time during the predetermined period among the images. It may be generated based on a difference between the colors of the teeth included in the second image captured at a second time point during the predetermined period.

The method further includes displaying a warning message when the diagnostic information indicates an abnormality in the user's teeth or ears, wherein the diagnostic information is configured to communicate with a terminal used by an oral or ear specialist may be transmitted to the consultation server.

The obtained image is a two-dimensional image of the user's oral cavity as a plurality of images, and the method, in the mobile terminal or the management server, by synthesizing the two-dimensional images, the three-dimensionality of the user's mouth The method may further include generating an image of

The portable camera device is configured to have a detachable head, and is configured to photograph the inside of the user's ear when the head is separated, and is configured to photograph the user's oral cavity in a state in which the head is not separated. can

By allowing the mobile terminal to display status information (diagnostic information) about the inside of the ear or conception based on the image of the user's mouth or inside the ear taken using a portable camera device, the user does not have to visit the hospital directly. Appropriate diagnostic information can be obtained.

By communicating with a management server that can obtain opinions from specialists, it is possible to remotely obtain expert diagnosis information about the user's mouth and ears for images taken using a portable camera device. have.

By analyzing the images inside the user's mouth or ear accumulated in the management server using artificial intelligence, it is possible to provide time-series tracking diagnostic information for the user's mouth or inside the ear.

1 illustrates a method of acquiring state information about a user's mouth or ears through a system including a portable camera device, a mobile terminal, and a management server, according to an embodiment.
2 illustrates a portable camera device used to photograph the inside of a user's mouth or ear, according to an embodiment.
3 is a flowchart illustrating a method of acquiring state information about a user's mouth or ears, according to an embodiment.
4 is a flowchart illustrating a method of acquiring expert diagnosis information on a user's mouth or ears through communication with a counseling server, according to an example.
5 illustrates a method of acquiring (generating) a 3D image based on a plurality of 2D images captured with respect to the inside of the user's mouth or ear, according to an example.
6 illustrates a method of acquiring diagnostic information by analyzing dental caries data using an artificial intelligence model, according to an example.
7A to 7C show dental caries data and an analysis method thereof, according to an example.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

1 illustrates a method of acquiring state information about a user's mouth or ears through a system including a portable camera device, a mobile terminal, and a management server, according to an embodiment.

In FIG. 1 , an image of the user's mouth or the inside of the ear is obtained by photographing the user's oral cavity (including teeth and gums) or the inside of the ear using a portable camera device 100 that the user can carry; the image acquired by the mobile terminal 110 communicating with the portable camera device 100 is transmitted to the management server 120 and/or the consultation server 130 ; State information (or diagnostic information) about the user's mouth or ears is received from the management server 120 to the mobile terminal 110 , and from the consultation server 130 to the mobile terminal 110 , the user's mouth or ears are diagnosed by a specialist How the information is received is shown.

The portable camera device 100 will be described in more detail with reference to FIG. 2 .

2 illustrates a portable camera device used to photograph the inside of a user's mouth or ear, according to an embodiment.

The portable camera device 100 (hereinafter, the camera device 100 ) may include a head unit 210 , a body unit 240 , and a manipulation unit 230 included in the body unit 230 . The manipulation unit 230 may include at least one button for controlling an operation (eg, power on/off) and photographing of the camera device 100 .

The head portion 210 (head) may be configured to be detachable, and thus the head portion 210 may be detachably cleaned.

The camera device 100 may be configured to photograph the inside of the user's ear while the head 210 is separated. In this case, the camera unit 220 (the lens unit of the camera) may be located in the head unit 210 . By inserting the camera unit 220 into the user's mouth or inside the ear, the inside of the mouth or ear may be photographed.

On the other hand, the camera device 100 may be configured to be able to photograph the user's oral cavity in a state in which the head unit 210 is not separated. In this case, the camera unit 220 (the lens unit of the camera) may be located at the upper end of the body unit 240 from which the head unit 210 is separated.

The camera device 100 may include a memory for temporarily/permanently storing the photographed image, and may include a communication unit for transmitting the photographed image to the mobile terminal 110 . For example, the camera device 100 may be connected to each other through the mobile terminal 110 and Wi-Fi, or may be paired through Bluetooth. Alternatively, the camera device 100 may communicate with the mobile terminal 110 using short-range wireless communication.

The camera device 100 automatically transmits the captured image to the mobile terminal 110 , or for operation in the mobile terminal 110 (eg, operation of an application) or operation of the operation unit 230 of the camera device 100 . Accordingly, the captured image may be transmitted to the mobile terminal 110 .

Next, the mobile terminal 110 will be described in more detail below. The mobile terminal 110 may be a user's terminal for communicating with the camera device 100 , the management server 120 , and the consultation server 130 . An application/program for acquiring an image from the camera device 100 and displaying information received from the management server 120 and the consultation server 130 may be installed in the mobile terminal 110 .

The mobile terminal 110 is a computing device, and may be, for example, a terminal used by a user, such as a smart phone, a tablet, an Internet of Things (IoT) device, or a wearable computer.

Although not shown, the mobile terminal 110 may include a communication unit, a processor, and a display unit.

The communication unit may be configured to receive an image captured by the camera device 100 , and transmit/receive information/data to and from the management server 120 and the consultation server 130 . In other words, the communication unit 210 is a configuration for the mobile terminal 110 to transmit/receive data and/or information wirelessly or wired to other devices such as the camera device 100 and the servers 120 and 130, It may be a hardware module such as a network interface card, a network interface chip, and a networking interface port of the mobile terminal 110 or a software module such as a network device driver or a networking program.

The processor may manage components of the mobile terminal 110 , and may be configured to execute a program or application used by the mobile terminal 110 . The processor may be at least one processor of the mobile terminal 110 or at least one core in the processor. A method of acquiring state information about the user's mouth or ears, which will be described later with reference to FIGS. 3 to 6 , may be performed by the operation by the processor. The processor may be implemented to execute computer-readable instructions, and the above methods may be performed through the execution of such instructions.

The display unit outputs data input by the user of the mobile terminal 110 or outputs (displays) the image(s) captured by the camera device 100 and status information and diagnostic information received from the servers 120 and 130 . It may be a display device for The display unit may include a touch screen.

The management server 120 may be a device that receives and manages an image of the user's mouth or inside the ear received through the mobile terminal 110, and generates state information about the user's mouth or ear based on the image. The management server 120 may be a server or other computing device that is remote from the mobile terminal 110 . The management server 120 may utilize big data in generating state information. Or/additionally, the management server 120 may use an artificial intelligence module (eg, a deep learning module) implemented therein in generating state information.

The management server 120 may include a communication unit and a processor. For the communication unit and the processor of the management server 120, the technical description of the communication unit and the processor of the mobile terminal 110 may be similarly applied, and thus overlapping description will be omitted.

The consultation server 130 may be a device that generates expert diagnosis information about the user's mouth or ear based on the image of the user's mouth or the inside of the ear received through the mobile terminal 110 or the management server 120 . . The counseling server 130 may be a server or other computing device that is remote from the mobile terminal 110 .

The consultation server 130 may be configured to be able to communicate with the terminal(s) 140 used by the oral or ear specialist(s). The image transmitted to the consultation server 130 through the mobile terminal 110 or the management server 120 may be provided by the terminal 140 used by the specialist so that the specialist can check it. Based on the information input from the terminal 140 of the specialist who has confirmed the image, the consultation server 130 may generate the diagnosis information of the specialist about the user's mouth or ears. The terminal 140 is a computing device, and may be, for example, a terminal used by a specialist such as a smart phone, a tablet, an Internet Of Things device, or a wearable computer. The terminal 140 may be a device used in a hospital (eg, a personal computer used by a doctor disposed in the hospital).

The consultation server 130 may include a communication unit and a processor. For the communication unit and the processor of the management server 130, the technical description of the communication unit and the processor of the mobile terminal 110 may be similarly applied, and thus overlapping description will be omitted.

For details of a specific method of the mobile terminal 110, the management server 120, and the consultation server 130, and a method of obtaining state information about the user's mouth or ears, refer to FIGS. 3 to 7 to be described later. Describe in detail.

In the detailed description to be described later, the operation performed by the mobile terminal 110 or components of the processor or the operation performed by the mobile terminal 110 or the application/program executed by the processor is the mobile terminal 110 for convenience of description. It can be described as an operation performed by The same applies to the management server 120 , the consultation server 130 , and the camera device 100 .

3 is a flowchart illustrating a method of acquiring state information about a user's mouth or ears, according to an embodiment.

In step 310 , the mobile terminal 110 may acquire an image of the user's mouth or the inside of the ear from the portable camera device 100 as the user's mouth or the inside of the ear is photographed by the portable camera device 100 . have. The mobile terminal 110 may store the acquired (received) image (in memory or storage).

In step 320 , the mobile terminal 110 may transmit the image obtained in step 310 to the management server 120 . The transmitted image may be accumulated and managed together with the image of the user's mouth or the inside of the ear previously acquired for the user in the management server 120 . In other words, the management server 120 may accumulate and manage the images captured for the user in time series. The management server 120 may manage these images for each user.

In step 330 , the mobile terminal 110 may receive status information about the user's mouth or ears from the management server 120 . As the status information is generated by the management server 120, the management server 120 generates such management information by analyzing the image(s) inside the user's mouth or ears that are accumulated and managed by the management server 120 . can

In step 340, the mobile terminal 110 may display the received status information. The mobile terminal 110 may display such state information through an application installed in the mobile terminal 110 that provides a state of the oral cavity or the ear.

Reception and display of status information may occur periodically (eg, weekly or monthly).

The state information about the user's mouth or ears generated by the management server 120 may include diagnostic information about the user's mouth or ears. For example, such diagnostic information may be obtained as the management server 120 analyzes the images of the user's mouth or the inside of the ears obtained for the user for a predetermined period by the management server 120 .

For example, in the case where each of the images is a photograph of a user's teeth, the above-described diagnostic information includes a color of a tooth included in a first image captured at a first time point during a predetermined period among images and The image may be generated based on a difference between the colors of teeth included in the second image captured at a second time point (a point in time when a predetermined period has elapsed after the first time point) during a predetermined period among the corresponding images. The management server 120 may generate different types of diagnostic information when the color difference is relatively large and small. For example, the management server 100 may compare the color of teeth from images taken 1 month ago and 6 months ago. The determination of the color change of the tooth may be performed through an artificial intelligence model 600 that will be described later with reference to FIG. 6 .

The management server 120 may determine whether there is an abnormality in the user's teeth or ears by analyzing the image(s) inside the user's mouth or ears. For example, the management server 120 may determine that there is an abnormality in the user's teeth when the color difference is greater than or equal to a certain level, and may determine that the user's teeth are normal when the color difference is less than a certain amount. The above-described diagnostic information may include information indicating whether the user's teeth (or inside the ears) are abnormal.

In step 340 , the mobile terminal 110 may output a warning message when the diagnostic information indicates an abnormality in the user's teeth or ears. The warning message may be output from the mobile terminal 110 as a voice message or displayed on the mobile terminal 110 as a visual message.

The user can self-diagnose his or her oral cavity (teeth) or ear by checking the status information (or diagnostic information) received from the management server 120 , and through a warning message output from the mobile terminal 110 , the oral cavity (teeth) ) or to call attention to ear abnormalities.

As described above, the technical features described above with reference to FIGS. 1 and 2 can be applied to FIG. 3 as they are, and thus a redundant description will be omitted.

4 is a flowchart illustrating a method of acquiring expert diagnosis information on a user's mouth or ears through communication with a counseling server, according to an example.

In step 410 , the mobile terminal 110 (or the management server 120 ) may transmit the acquired image (ie, the image captured for the user's mouth or inside the ear) to the counseling server 130 . As described above, the counseling server 130 may be configured to be able to communicate with the terminal 140 used by the oral or ear specialist. In step 410, the image transmitted to the consultation server 130 may be provided to the terminal 140 used by the specialist so that the specialist can check. That is, the specialist may view the captured image of the user's mouth or the inside of the ear through his/her terminal 140 .

In step 420 , the mobile terminal 110 (or the mobile terminal 110 via the management server 120 ) may receive, from the consultation server 130 , the expert diagnosis information for the user's mouth or ear. . The specialist diagnosis information may be generated by the consultation server 130 based on information input (by the specialist) from the terminal 140 used by the specialist. For example, the specialist may write a diagnosis opinion as his/her own specialist on the user's image viewed through the terminal 140 , and the consultation server 130 processes the diagnosis opinion to provide a message to the mobile terminal 110 . diagnostic information can be generated.

In step 410, the mobile terminal 110 may display the received diagnosis information of the specialist.

According to the embodiment, the mobile terminal 110 may obtain the diagnosis information of a specialist through the consultation server 130 (similarly periodically) in addition to periodically receiving the status information through the management server 120 . Accordingly, the user can remotely check diagnostic information indicating a more accurate opinion from a specialist. For example, regular dental health care and counseling for infants and children may be performed non-face-to-face.

The aforementioned status information from the management server 120 and/or the expert diagnosis information from the consultation server 130 may be transmitted to the mobile terminal 110 when there is a request from the mobile terminal 110 . The mobile terminal 110 may transmit such a request to the management server 120 and/or the consultation server 130 through a predetermined application (eg, "consultation" on the screen of the mobile terminal 110 shown in FIG. 1 ). When selected, expert diagnosis information may be requested from the consultation server 130).

Meanwhile, the terminal 140 used by the specialist may be configured to be accessible to the management server 120 through the consultation server 130 . The expert diagnosis information may be generated further based on the image(s) inside the user's mouth or ears, which are accumulated and managed by the management server 120 .

In other words, the specialist can also access the user's image accumulated and managed in the management server 120 using the terminal 140 , and can input a more accurate diagnosis opinion about the user into the terminal 140 with reference to this. have. Such access may be enabled only when permission is granted by the user (mobile terminal 110 ) (ie, when a notification of permission is output to the management server 120 and/or the consultation server 130 ).

According to an embodiment, a user may more accurately obtain remote medical information.

On the other hand, in the case in which the warning message described above with reference to FIG. 3 is output to the mobile terminal 110, diagnostic information related to the warning message is configured to be able to communicate with the terminal 140 used by a specialist for the oral cavity or ear. ) may be transmitted to the consultation server 130 . Accordingly, the specialist can check such diagnostic information through the terminal 140 , and provide feedback on the corresponding diagnostic information to the mobile terminal 110 through the consultation server 130 through the terminal 140 . have.

As described above, the technical features described above with reference to FIGS. 1 to 3 can be applied to FIG. 4 as it is, and thus a redundant description will be omitted.

5 illustrates a method of acquiring (generating) a 3D image based on a plurality of 2D images captured with respect to the inside of the user's mouth or ear, according to an example.

The image obtained in step 310 (or the image accumulated and managed by the management server 120 ) may be two-dimensional images obtained by photographing the user's mouth (or the inside of the ear) as a plurality of images.

In step 510 , the management server 120 (or the mobile terminal 110 ) may acquire a plurality of two-dimensional images of the user's oral cavity (or inside the ear).

In step 520, the management server 120 (or the mobile terminal 110) may generate (obtain) a three-dimensional image of the user's mouth (or inside the ear) by synthesizing these two-dimensional images.

The 3D image may be a panoramic image of the user's mouth (or inside the ear) of the user.

Each of the plurality of two-dimensional images may be images obtained by photographing different portions of the oral cavity (or inside the ear). Each of these images may be images obtained by photographing different portions of the oral cavity (or the inside of the ear) with a predetermined overlap. The management server 120 (or the mobile terminal 110) may use any image synthesis method in generating a 3D image (eg, a panoramic image) by synthesizing the 2D images.

The generated 3D image may be displayed on the mobile terminal 110 .

As described above, the technical features described above with reference to FIGS. 1 to 4 can be applied to FIG. 5 as they are, and thus a redundant description will be omitted.

6 illustrates a method of obtaining diagnostic information by analyzing dental caries data using an artificial intelligence model, according to an example.

The diagnostic information described above with reference to FIG. 3 may be generated as a result of the artificial intelligence (AI) model 600 implemented in the management server analyzing images inside the user's mouth or ears. The AI model 600 may be, for example, a model composed of an artificial neural network, a CNN model, or a deep learning model.

The AI model 600 may include a feature extractor 610 and a class classifier 620 . The AI model 600 (class classification unit 620) may be pre-trained by predetermined test data. The test data may be data for which the correct answer is already known.

The AI model 600 may use, as an input, dental caries data included in an image taken for a user. That is, each of the images analyzed by the management server 120 may include dental caries data, and the AI model 600 may use this as input data. The test data may be training data for training the AI model 600 to output dental state information by inputting dental caries data.

The dental caries data included in the image taken for the user may be feature extracted by the feature extraction unit (610). The dental caries data and the extracted features may be input to the class classification unit 620 , and the result data may be output after being classified by the class classification unit 620 . Based on the result data, the management server 120 may determine the state of the teeth included in the image taken for the user. For example, the management server 120 may determine the presence or absence of a disease of the tooth or predict and evaluate the possibility of a disease based on the result data.

For example, the diagnostic information (generated by the management server 120) described above with reference to FIG. 3 is a feature extracted by the feature extraction unit 610 from the dental caries data included in the image taken for the user, It may be generated based on data output as a result of being input to the class classifier 620 learned in advance based on predetermined test data.

In some embodiments, state information about the user's mouth or ears may be provided according to the user's image analysis based on the AI model.

7A to 7C show dental caries data and an analysis method thereof, according to an example.

7a to 7c show standard ICDAS codes according to the progression of dental caries. The management server 120 may determine which state of the ICDAS codes (0 to 6) shown corresponds to the state of the user's teeth by analyzing the image taken for the user. The aforementioned AI model 600 may also be trained to classify the state of teeth included in the user's image with respect to the inputted user's image into any one of the ICDAS codes (0 to 6). That is, the state information (or diagnostic information) described above with reference to FIGS. 3 to 6 may include information indicating which of the ICDAS codes 0 to 6 the user's teeth correspond.

As described above, the technical features described above with reference to FIGS. 1 to 6 can be applied to FIGS. 7 and 8 as they are, and thus a redundant description will be omitted.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more of these, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Claims (10)

A method of obtaining status information about the user's oral cavity, performed by a mobile terminal communicating with a portable camera device and a management server, the method comprising:
As the inside of the user's mouth is photographed by the portable camera device, obtaining an image of the inside of the user's mouth from the portable camera device;
Transmitting, by the mobile terminal, the acquired image to a management server, the transmitted image is accumulated and managed together with the image of the user's oral cavity previously acquired for the user in the management server;
Receiving, from the management server, state information about the user's oral cavity generated as the image inside the user's oral cavity accumulated and managed by the management server is analyzed by the management server; and
displaying the received status information
including,
The received state information includes diagnostic information about the user's oral cavity obtained by analyzing the images in the user's oral cavity obtained for the user for a predetermined period in the management server by the management server, and ,
Each of the images is a photograph of the user's teeth,
The diagnosis information may include a color of the tooth included in a first image captured at a first time point during the predetermined period among the images and a second image captured at a second time point during the predetermined period among the images. It is generated based on a difference comparing the color of the teeth included in the image, and includes information indicating whether the user's teeth are abnormal,
The diagnosis information includes first information indicating that there is an abnormality in the user's teeth when the difference between the colors of the teeth is greater than or equal to a predetermined value according to a determination in the management server, and the color of the teeth and second information indicating that the user's teeth are normal when the compared difference is less than a predetermined value,
A method of obtaining state information about the oral cavity,
displaying a warning message when the diagnostic information includes the first information
further comprising,
The diagnosis information including the first information is transmitted to a counseling server configured to be able to communicate with a terminal used by an oral specialist,
The consultation server is configured to be able to communicate with a terminal used by a specialist for oral cavity, and the diagnosis information including the first information transmitted to the consultation server is provided to a terminal used by the specialist so that the specialist can confirm become,
A method of obtaining state information about the oral cavity,
receiving, from the counseling server, diagnostic information of a specialist about the user's oral cavity as the diagnosis information including the first information is transmitted to the counseling server; and
Displaying the received expert diagnosis information
further comprising,
Only when there is permission from the mobile terminal, the terminal used by the specialist is configured to be able to access the management server through the consultation server,
The specialist diagnosis information is generated by the consultation server based on information input from the terminal used by the specialist who has confirmed the diagnosis information including the first information, and is accessed by the terminal used by the specialist It is generated based on the image inside the user's oral cavity that is accumulated and managed by the authorized management server,
A method of obtaining state information about the oral cavity,
periodically receiving, through the consultation server, expert diagnosis information on the user's oral cavity generated based on the image inside the user's oral cavity accumulated and managed by the management server; and
When the user interface displayed on the screen of the mobile terminal is selected, receiving expert diagnosis information about the user's oral cavity generated based on the image inside the user's oral cavity accumulated and managed by the management server
Further comprising, a method of obtaining state information about the oral cavity. delete delete delete The method of claim 1,
The diagnostic information is generated as a result of the AI model implemented in the management server analyzing the images inside the user's mouth, how to obtain state information about the oral cavity. 6. The method of claim 5,
Each of the images includes dental caries data,
The diagnosis information is generated based on data output as a result of inputting the features extracted from the dental caries data to a class classifier learned in advance based on predetermined test data, to obtain state information about the oral cavity Way. delete delete The method of claim 1,
The acquired image is a two-dimensional image of the user's oral cavity as a plurality of images,
Generating a three-dimensional image of the user's mouth by synthesizing the two-dimensional images in the mobile terminal or the management server
Further comprising, a method of obtaining state information about the oral cavity. The method of claim 1,
The portable camera device is configured to have a detachable head,
In the state in which the head is separated, it is configured to be able to photograph the inside of the user's ear,
In a state in which the head is not separated, the method for obtaining state information about the oral cavity is configured to be capable of photographing the user's oral cavity.

Images