JP6938410B2 - Oral Disease Diagnosis System and Oral Disease Diagnosis Program - Google Patents

Description

The present invention relates to an oral disease diagnostic system and an oral disease diagnostic program.

Diseases such as dental caries and periodontal disease are diagnosed based on the images in the oral cavity. The diagnosis of the disease is performed, for example, by visual inspection and image analysis of the image by a doctor. Intraoral photographs are used to diagnose the disease. An intraoral photograph consists of a plurality of images showing a predetermined area in the oral cavity. As an example of the intraoral photograph, a predetermined intraoral standard photograph may be used. Each image constituting the intraoral photograph is generated as a so-called panoramic image in which a plurality of images representing a portion of the oral cavity are combined based on a superposed region. Further, there is known a technique for classifying an image of a tooth surface for detecting dental caries based on an intraoral image (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2014-509908

Diagnosis of a disease based on an intraoral photograph is made based on the color and shape of the part to be diagnosed. In order to make a highly accurate diagnosis, it is necessary that the target site is correctly represented in the intraoral photograph. However, in a normally taken intraoral image, saliva may cause reflexes. In such an image, the part where the reflection occurs is represented by a color or the like different from the actual state. Misdiagnosis can occur when a diagnosis of a disease in the oral cavity is made based on an image that includes a reflexed area.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to reduce the possibility of misdiagnosis of a disease in the diagnosis of a disease based on an intraoral photograph.

In order to solve the above problems, the oral disease diagnosis system according to one embodiment of the present invention diagnoses a disease in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. In the oral disease diagnosis system, an acquisition unit that acquires a plurality of partial images of a part of the user's oral cavity, and a partial image that does not include a reflection portion or a reflection portion that is a region where reflection occurs in the partial image. A removal unit that acquires a partial image that does not include the reflective portion, a generation unit that synthesizes a plurality of partial images including the partial image acquired by the removal unit to generate an intraoral photograph, and a generation unit that generates an intraoral photograph. The output unit is provided with an output unit for outputting the intraoral photograph, the acquisition unit acquires a plurality of partial images obtained by capturing the same part in the oral cavity from different directions, and the removal unit is represented by the partial image. Converting multiple partial images so that the projection direction of the parts is aligned with a predetermined direction, aligning the pixels representing the same part in the oral cavity so that they are associated with each other, and adding and averaging the multiple partial images. Acquires a partial image from which the reflective portion has been removed .
Further, the oral disease diagnosis system according to one embodiment of the present invention is an oral disease diagnosis system for diagnosing a user's oral disease based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. By removing a reflective portion, which is a region where reflection occurs in the partial image, or extracting a partial image that does not include the reflective portion, the reflective portion is acquired by acquiring a plurality of partial images obtained by capturing a plurality of partial images of the portion in the oral cavity of the user. A removal unit that acquires a partial image that does not include a An output unit for outputting is provided, and a determination unit for determining the presence or absence and a state of a predetermined disease in the oral cavity based on the characteristics of an image of a predetermined determination target area shown in an intraoral photograph is provided and output. The department outputs the result of the determination regarding a predetermined disease in the oral cavity on the second terminal of a doctor different from the user, and the necessity of consultation regarding the necessity of consultation of the user, which is input to the second terminal. The information is output on the user's first terminal.

In addition, the oral disease diagnosis program according to one embodiment of the present invention uses a computer to diagnose a disease in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. It is an oral disease diagnosis program for functioning as a system, and has an acquisition function of acquiring a plurality of partial images of a part in the user's oral cavity on a computer, and a reflection part which is a region where reflection occurs in the partial image. A removal function that acquires a partial image that does not include a reflective part by removing or extracting a partial image that does not include a reflective part, and a plurality of partial images that include the partial image acquired by the removal function are combined to generate an intraoral photograph. The generation function and the output function that outputs the intraoral photograph generated by the generation function are realized, and the acquisition function acquires and removes a plurality of partial images obtained by capturing the same part in the oral cavity from different directions. The function converts multiple partial images so that the projection direction of the part represented in the partial image is aligned with a predetermined direction, and aligns the pixels representing the same part in the oral cavity so that they are associated with each other. By adding and averaging a plurality of partial images, a partial image from which the reflection portion has been removed is acquired .
In addition, the oral disease diagnosis program according to one embodiment of the present invention uses a computer to diagnose a disease in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity. It is an oral disease diagnosis program for functioning as a system, and it has an acquisition function of acquiring a plurality of partial images of a part in the user's oral cavity on a computer, and a reflection part which is a region where reflection occurs in the partial image. A removal function that acquires a partial image that does not include a reflective part by removing or extracting a partial image that does not include a reflective part, and a plurality of partial images that include the partial image acquired by the removal function are combined to generate an intraoral photograph. Based on the generation function to output, the output function to output the intraoral photograph generated by the generation function, and the characteristics of the image of the predetermined determination target area represented by the intraoral photograph, the presence or absence of a predetermined disease in the oral cavity and The determination function for determining the state and the output function are realized, and the output function outputs the result of determination regarding a predetermined disease in the oral cavity on a second terminal of a doctor different from the user and inputs it to the second terminal. The medical examination necessity information regarding the user's medical examination necessity is output to the user's first terminal.

According to the above embodiment, an intraoral photograph is generated based on a partial image that does not include a reflective portion. By diagnosing an oral disease based on the intraoral photograph generated in this way, the possibility of misdiagnosis of the disease can be reduced.
Further, according to the above embodiment, a plurality of partial images obtained by capturing the same portion from different directions are acquired. In the plurality of partial images acquired in this way, even if reflection occurs at a specific portion in one partial image, there is a high possibility that the reflection portion and the reflection state are different in the other partial images. Since a plurality of partial images converted so that the projection direction of a certain portion is aligned in a predetermined direction are aligned and then added and averaged, it is possible to obtain a partial image in which the reflection portion is removed. Become.
Further, according to the above embodiment, since the disease is determined based on the intraoral photograph not including the reflex portion, it is possible to carry out the diagnosis with a low possibility of misdiagnosis.
Further, according to the above embodiment, since the result of the determination of the disease is output on the terminal of the doctor, the doctor can be easily notified of the state of the disease in the oral cavity of the user. Further, since the information regarding the necessity of the user's consultation, which is input by the doctor according to the result of the notified diagnosis, can be notified to the user, it is possible to have the user receive an appropriate diagnosis.

In diagnosing a disease based on an intraoral photograph, it is possible to reduce the possibility of misdiagnosis of the disease.

It is a figure which shows the apparatus configuration of the oral disease diagnosis system which concerns on this embodiment. It is a block diagram which shows the functional structure of the oral disease diagnostic apparatus of this embodiment. It is a hard block diagram of the oral disease diagnostic apparatus. It is a figure which shows the functional configuration of a user's terminal. It is a figure which shows the functional structure of the terminal of a doctor. It is a figure which shows the example of the panoramic image which was combined based on a partial image and a plurality of partial images. It is a figure which shows typically the example of the plurality of partial images including a reflection part, and the part image which a reflection part was removed. It is a flowchart which shows the processing content of the removal of the reflection part by the removal part. It is a figure which shows the example of the intraoral photograph generated by the generation part. It is a figure which shows the flowchart of the periodontal disease determination process by a determination part. It is a figure which shows the example of the information of the result of the determination of the periodontal disease progression level, and is the figure which shows typically the intraoral photograph in which the part of the disease is marked. It is a figure which shows the flowchart of the caries determination processing by a determination part. It is a figure which shows the example of the information of the result of the determination of the caries progression level, and is the figure which shows typically the intraoral photograph which marked the part of a disease. It is a flowchart which shows the example of the processing content of the oral disease diagnosis of this embodiment. It is a flowchart which shows the example of the processing content in a user's terminal. It is a flowchart which shows the example of the processing content in the terminal of a doctor. It is a figure which shows the structure of the oral disease diagnosis program.

An embodiment of the oral disease diagnosis system according to the present invention will be described with reference to the drawings. If possible, the same parts will be designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 is a diagram showing an apparatus configuration of an oral disease diagnosis system according to the present embodiment. The oral disease diagnosis system 1 is a system for diagnosing a disease in the user's oral cavity based on an intraoral photograph composed of a plurality of images showing a predetermined area in the oral cavity. The oral disease diagnosis system 1 shown in FIG. 1 includes an oral disease diagnosis device 10, a user terminal 20, and a doctor's terminal 30, and the oral disease diagnosis device 10 and terminals 20, 30 can communicate with each other via a network N. be.

The oral disease diagnostic apparatus 10 is composed of, for example, a computer such as a server, but the apparatus constituting the oral disease diagnostic apparatus 10 is not limited. The terminals 20 and 30 are composed of, for example, a stationary or portable personal computer, a high-performance mobile phone (smartphone), etc., but the devices constituting the terminals 20 and 30 are not limited, and for example, the mobile phone and mobile information. It may be a mobile terminal such as a terminal (PDA). In FIG. 1, one terminal 20 and one terminal 30 are shown as an example, but the number thereof is not limited to this.

FIG. 2 is a diagram showing a functional configuration of the oral disease diagnostic apparatus 10 according to the present embodiment. The oral disease diagnosis device 10 of the present embodiment is a device for diagnosing a disease in the user's oral cavity based on an intraoral photograph. The intraoral photograph is a photograph of the oral cavity obtained for diagnosing a disease such as dental caries or periodontal disease in the oral cavity. An intraoral photograph consists of a plurality of images showing a predetermined area in the oral cavity.

As shown in FIG. 2, the oral disease diagnosis device 10 functionally includes an acquisition unit 11, a removal unit 12, a generation unit 13, a determination unit 14, and an output unit 15. In the present embodiment, the oral disease diagnosis device 10 is composed of one device as shown in FIG. 2, but each functional unit included in the oral disease diagnosis device 10 is dispersed in a plurality of devices. May be done. Each functional part of the oral disease diagnostic apparatus 10 will be described in detail later.

The block diagram shown in FIG. 2 shows a block for each function. These functional blocks (components) are realized by any combination of hardware and / or software. Further, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one physically and / or logically coupled device, or directly and / or indirectly by two or more physically and / or logically separated devices. (For example, wired and / or wireless) may be connected and realized by these a plurality of devices.

For example, the oral disease diagnostic apparatus 10 according to the embodiment of the present invention may function as a computer. FIG. 3 is a diagram showing an example of the hardware configuration of the oral disease diagnostic apparatus 10 according to the present embodiment. The oral disease diagnosis device 10 may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the oral disease diagnostic device 10 may be configured to include one or more of the devices shown in FIG. 2, or may be configured not to include some of the devices.

Each function of the oral disease diagnostic apparatus 10 is performed by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs an calculation, and communication by the communication device 1004, the memory 1002, and the memory 1002. It is realized by controlling the reading and / or writing of data in the storage 1003.

Processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be composed of a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic unit, a register, and the like. For example, each of the functional units 11 to 15 shown in FIG. 1 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module, and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, each functional unit 11 to 15 of the oral disease diagnostic apparatus 10 may be realized by a control program stored in the memory 1002 and operated by the processor 1001. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted on one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to implement the shelving allocation information generation method according to the embodiment of the present invention.

The storage 1003 is a computer-readable recording medium, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing memory 1002 and / or storage 1003.

The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. Bus 1007 may be composed of a single bus, or may be composed of different buses between devices.

Further, the oral disease diagnosis device 10 includes hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be configured to include, and a part or all of each functional block may be realized by the hardware. For example, processor 1001 may be implemented on at least one of these hardware.

FIG. 4 is a diagram showing a functional configuration of the user terminal 20 of the present embodiment. As shown in FIG. 4, the terminal 20 includes an image acquisition unit 21, an image transmission unit 22, a determination result receiving unit 23, and a determination result presenting unit 24. The terminal 20 also includes an image sensor 25 and a display 26.

The image acquisition unit 21 acquires a partial image captured by the image sensor 25. The partial image is an image obtained by capturing a portion of the oral cavity. The image sensor 25 is a sensor that optically acquires an image to be imaged. The image acquisition unit 21 acquires a plurality of partial images.

Each of the plurality of partial images may be acquired as a still image, or may be extracted from each frame acquired as a moving image. By acquiring the plurality of partial images in this way, the image acquisition unit 21 can acquire a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions. That is, in the present embodiment, the user can capture an image of his / her oral cavity using the terminal 20.

The image transmission unit 22 transmits a plurality of partial images acquired by the image acquisition unit 21 to the oral disease diagnosis device 10.

The determination result receiving unit 23 receives information indicating the result of determination regarding a predetermined disease in the oral cavity from the oral disease diagnostic apparatus 10. The determination of the disease in the oral cavity is performed by the oral disease diagnostic apparatus 10 based on the plurality of partial images transmitted by the image transmitting unit 22. The result of the determination includes, for example, an intraoral photograph in which the location of the disease is marked, a comment on the disease found, and the like.

The determination result presenting unit 24 presents to the user information indicating the result of the disease determination received by the determination result receiving unit 23. The determination result presentation unit 24 displays, for example, information indicating the result of the determination of the disease on the display 26.

FIG. 5 is a diagram showing a functional configuration of the doctor's terminal 30 of the present embodiment. As shown in FIG. 5, the terminal 30 includes a receiving unit 31, a presenting unit 32, a receiving unit 33, and a transmitting unit 34. The terminal 30 also includes a display 35.

The receiving unit 31 receives information indicating the result of determination regarding a predetermined disease in the oral cavity from the oral disease diagnostic device 10. The determination of the disease in the oral cavity is carried out in the oral disease diagnostic apparatus 10 based on the plurality of partial images transmitted by the image transmission unit 22 of the terminal 20. The result of the determination includes, for example, an intraoral photograph in which the location of the disease is marked, a comment on the disease found, and the like.

The presenting unit 32 presents to the doctor information indicating the result of the determination of the disease, which has been examined by the receiving unit 31. The presentation unit 32 displays, for example, information indicating the result of determining the disease on the display 26.

The reception unit 33 receives the information input by the doctor to the terminal 30 regarding the necessity of the user's consultation. Specifically, the doctor refers to the result of the determination of the disease presented by the presentation unit 32, and examines and determines whether or not the user needs to see the doctor regarding the result of the determination. The doctor inputs information regarding the necessity of consultation to the terminal 30 in order to encourage the user to receive a consultation. The reception unit 33 receives the information regarding the necessity of consultation, which is input in this way.

The transmission unit 34 transmits information regarding the necessity of consultation by the user. Specifically, the transmission unit 34 transmits information regarding the necessity of consultation to the user's terminal 20, and outputs the transmitted information on the terminal 20, for example, by displaying it on a display. In addition, the transmission unit 34 may transmit information regarding the necessity of consultation to the terminal 20 via the oral disease diagnosis device 10. In this case, the output unit 15 of the oral disease diagnosis device 10 transfers the information regarding the necessity of consultation from the transmission unit 34 to the terminal 20, and causes the terminal 20 to output the information regarding the necessity of consultation.

Subsequently, each functional part of the oral disease diagnostic apparatus 10 will be described. The acquisition unit 11 acquires a plurality of partial images obtained by capturing an image of a portion in the oral cavity of the user. Specifically, the acquisition unit 11 acquires a partial image captured by the image sensor 25 of the terminal 20.

FIG. 6 is a diagram showing an example of a partial image and a panoramic image synthesized based on a plurality of partial images. As shown in FIG. 6, the acquisition unit 11 acquires, for example, a partial image A01 on the front right side of the dentition, a partial image A02 on the front center, and a partial image A03 on the front left side. Each of these partial images may be acquired as a still image, or may be extracted from each frame acquired as a moving image.

As will be described later, in the present embodiment, the panoramic image B is generated by synthesizing a plurality of partial images A01 to A03 based on the superposed region in which the same portion in the oral cavity is represented in each of the partial images A01 to A03. Will be done. The panoramic image B shown in FIG. 6 constitutes a frontal image in an intraoral photograph. Various well-known techniques can be applied to the composition of panoramic images based on the superposed region of partial images.

The removal unit 12 acquires a partial image that does not include the reflection portion by removing the reflection portion that is a region where reflection occurs in the partial image or extracting the partial image that does not include the reflection portion. Images taken in the oral cavity may have reflexes due to saliva present in the oral cavity. In order to diagnose the disease with high accuracy, it is necessary to correctly show the condition of the oral cavity in the intraoral photograph, but the color etc. of the image of the part where the reflex occurred is different from the actual condition. expressed. In the present embodiment, a highly accurate diagnosis is realized by acquiring a partial image that does not include the reflection portion.

Specifically, the removal unit 12 can extract a partial image that does not include the reflection portion from the partial image acquired by the acquisition unit 11. The removing unit 12 extracts a partial image that does not include the reflective portion by a well-known image analysis technique. For example, the removing unit 12 extracts a partial image that does not have the peculiar image characteristics of the reflective portion (for example, having a brightness equal to or higher than a predetermined value), assuming that the portion that does not include the reflective portion bears the partial image. In addition, the removing unit 12 can extract a partial image that does not include the reflective portion by various other well-known techniques.

Further, specifically, the removing unit 12 can acquire a partial image that does not include the reflecting portion by removing the reflecting portion that is a region where the reflection occurs in the partial image. More specifically, the removing unit 12 converts a plurality of partial images so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction, and the pixels representing the same portion in the oral cavity are associated with each other. A partial image in which the reflection portion is removed is acquired by aligning the images so as to be able to perform the addition and averaging of a plurality of partial images.

The process of removing the reflected portion by the removing portion 12 will be described in detail with reference to FIGS. 7 and 8. FIG. 7 is a diagram schematically showing an example of a plurality of partial images including a reflective portion and a partial image in which the reflective portion is removed. FIG. 8 is a flowchart showing the processing content of removing the reflected portion by the removing unit 12.

In step S31, the removing unit 12 acquires one partial image. In the example shown in FIG. 7, the removing unit 12 acquires the partial image A11. The partial image A11 includes a reflective portion rf1.

Subsequently, in step S32, the removing unit 12 converts a plurality of partial images so that the projection direction of the portion represented by the partial image is aligned with a predetermined direction. Specifically, the removing unit 12 performs homography conversion on the partial image so as to meet the standard of each image constituting the intraoral photograph, for example. Homography conversion is a well-known image conversion technique, in which a two-dimensional image represented by projecting a three-dimensional object onto one plane is projected onto another plane using a projective transformation. Is a technology to generate. The removal unit 12 performs homography conversion on the partial image A11 so as to meet the standard of the front image in the intraoral photograph, and obtains the converted partial image A21.

In step S33, the removing unit 12 determines whether or not the processing of all the partial images has been completed. If it is determined that the processing of all the partial images is completed, the processing proceeds to step S34. On the other hand, if it is not determined that the processing of all the partial images is completed, the processing proceeds to step S31.

In the example described with reference to FIG. 7, since the processing of the partial images A12 and A13 is not completed, the processing of steps S31 and S32 for these partial images A12 and A13 is repeated. That is, the removing unit 12 performs homography transformation on the partial image A12 including the reflective portion rf2 to obtain the partial image A22. Further, the removing unit 12 performs homography transformation on the partial image A13 including the reflective portion rf3 to obtain the partial image A23.

In step S34, the removing unit 12 aligns the pixels representing the same portion in the oral cavity so that they are associated with each other, and adds and averages a plurality of partial images. In the example described with reference to FIG. 7, the removing unit 12 aligns the converted partial images A21 to A23 so that pixels representing the same portion in the oral cavity are associated with each other, and adds and averages them. Synthesize. The averaging composition is a well-known technique in the field of image processing.

Each of the partial images A11 to A13 is an image obtained by capturing the same portion in the oral cavity from different directions. Therefore, in the partial image A22 and the partial image A23, no reflection occurs at the position of the reflection portion rf1 in the partial image A21. Further, in the partial image A21 and the partial image A23, no reflection occurs at the position of the reflection portion rf2 in the partial image A22. Further, in the partial image A21 and the partial image A22, no reflection occurs at the position of the reflection portion rf3 in the partial image A23.

Even if reflection occurs in a certain part of one partial image, the reflection is removed by adding and averaging a plurality of other partial images in which the reflection does not occur and the one partial image. You can get an image. In the example described with reference to FIG. 7, the removal unit 12 can acquire the partial image C1 from which the reflection portion has been removed by aligning the partial images A21 to A23 and performing addition averaging synthesis.

With reference to FIG. 2 again, the generation unit 13 synthesizes a plurality of partial images including the partial image acquired by the removal unit 12 to generate an intraoral photograph. Specifically, as shown in FIG. 6, the generation unit 13 synthesizes partial images based on the superposed region in which the same part in the oral cavity is represented to generate a panoramic image, thereby generating a panoramic image of the intraoral photograph. Generate multiple images according to the standard. The partial image used for generating the intraoral photograph includes a partial image in which the reflective portion is removed by the removing portion 12 and a partial image in which the reflective portion is not included extracted by the removing portion 12. A well-known method for generating a so-called panoramic image can be applied to the generation of an intraoral photograph by synthesizing a partial image.

Specifically, the generation unit 13 adjusts the orientation and arrangement of the acquired plurality of partial images. Subsequently, the generation unit 13 generates one panoramic image by superimposing and synthesizing partial images adjacent to each other based on the superposed region in which the same portion in the oral cavity is represented. One panoramic image constitutes one of a plurality of images constituting an intraoral photograph.

FIG. 9 is a diagram showing an example of an intraoral photograph generated by the generation unit 13. The intraoral photograph MP consists of a plurality of images showing a predetermined area in the oral cavity. The intraoral photograph of the present embodiment is, for example, a so-called intraoral standard photograph. The method of taking a standard intraoral photograph includes a 5-image method, a 9-image method, a 12-image method, a 14-image method, and the like, depending on the number of images to be composed and the imaged portion of each image. In the present embodiment, an example of a five-photographing method will be described. FIG. 9 is an intraoral photograph MP taken by a five-image method, and includes a palatal side image mp1, a right side image mp2, a front image mp3, a left side image mp4, and a lingual side image mp5.

The determination unit 14 determines the presence / absence and state of a predetermined disease in the oral cavity based on the characteristics of the image of the predetermined determination target region shown in the intraoral photograph. Specifically, the determination unit 14 extracts at least one of the gum region in which the gum is represented and the tooth region in which the tooth is represented from the intraoral photograph, and makes the feature of at least one image of the gum region and the tooth region. Based on this, the presence and condition of periodontal disease are determined. The determination of periodontal disease will be described with reference to FIGS. 10 and 11. FIG. 10 is a diagram showing a flowchart of periodontal disease determination processing by the determination unit 14.

In step S41, the determination unit 14 extracts at least one of the gum region in which the gums are represented and the tooth region in which the teeth are represented from the intraoral photograph generated by the generation unit 13.

In step S42, the determination unit 14 extracts the feature A relating to the determination of periodontal disease from the gum region. In view of the appearance of symptoms such as redness of the gingiva, swelling of the gingiva, and disappearance of the gingival stippling as the periodontal disease progresses, feature A is, for example, the color, shape, and surface condition of the gingiva. Contains information. In addition, the determination unit 14 extracts the feature B related to the determination of periodontal disease from the tooth region. In view of the fact that as the periodontal disease progresses, the length of the tooth becomes longer due to the retraction of the gingiva, the feature B includes, for example, information on the area of the tooth.

Feature A and feature B are information input to the periodontal disease determination model including the neural network. The periodontal disease determination model is a model for determining the progress level of periodontal disease by using the data of the feature A and the feature B for learning as input values, and is constructed in advance by machine learning. The neural network included in the periodontal disease judgment model is generated by performing machine learning including error calculation (error back propagation) based on the judgment value of the periodontal disease progression level obtained by inputting the features A and B. Will be done. Various loss functions can be applied to the error calculation.

In step S43, the determination unit 14 inputs the feature A and feature B extracted in step S42 into the periodontal disease determination model. The periodontal disease progression level is determined based on the periodontal disease determination model by inputting either the feature A or the feature B into the model based on the model constructed based on the one of the feature A and the feature B. It may be done by doing.

In step S44, the determination unit 14 determines the periodontal disease progression level based on the output of the periodontal disease determination model. FIG. 11 is a diagram showing an example of information on the result of determination of the periodontal disease progression level, and is a diagram schematically showing an intraoral photograph in which a diseased portion is marked. As shown in FIG. 11, the intraoral photograph MP1 output by the determination unit 14 includes a plurality of images mp11 to mp15. Image mp12 includes mark D11 at the disease site. Information on the progress level of periodontal disease at the diseased site may be output, for example, by text or the like in association with the mark D11 at the diseased site. Further, the image mp13 includes the mark D12 and the mark D13 of the diseased part. Further, the image mp14 includes the mark D14 of the diseased part.

In addition, the determination unit 14 extracts the tooth region in which the tooth is represented from the intraoral photograph, and determines the presence / absence and state of caries based on the characteristics of the image of the tooth region. The determination of caries will be described with reference to FIGS. 12 and 13. FIG. 12 is a diagram showing a flowchart of caries determination processing by the determination unit 14.

In step S51, the determination unit 14 extracts the tooth region in which the tooth is represented from the intraoral photograph generated by the generation unit 13.

In step S52, the determination unit 14 extracts the characteristic C related to the determination of dental caries from the tooth region. In view of the fact that the tooth color becomes black as the caries progresses, the feature C includes, for example, information on the tooth color.

Feature C is information input to the caries determination model including the neural network. The caries determination model is a model for determining the progress level of caries by using the data of the feature C for learning as an input value, and is constructed in advance by machine learning. The neural network included in the caries determination model is generated by performing machine learning including error calculation (error back propagation) based on the judgment value of the caries progression level obtained by inputting the feature C. Various loss functions can be applied to the error calculation.

In step S53, the determination unit 14 inputs the feature C extracted in step S52 into the caries determination model.

In step S54, the determination unit 14 determines the caries progress level based on the output of the caries determination model. FIG. 13 is a diagram showing an example of information on the result of determination of the caries progression level, and is a diagram schematically showing an intraoral photograph in which a diseased portion is marked. As shown in FIG. 13, the intraoral photograph MP2 output by the determination unit 14 includes a plurality of images mp21 to mp25.

The image mp21 includes marks D21 and D22 of the diseased part. Information on the progress level of caries at the diseased site may be output, for example, by text or the like in association with the marks D21 and D22 at the diseased site. Further, the image mp22 includes the mark D23 of the diseased part. Further, the image mp23 includes the mark D24 of the diseased part. Further, the image mp24 includes the mark D25 of the diseased part. In addition, the image mp25 includes the mark D26 of the diseased part.

With reference to FIG. 2 again, the output unit 15 outputs an intraoral photograph generated by the generation unit 13. Specifically, the output unit 15 displays the generated intraoral photograph on the display of the user's terminal 20. The output unit 15 displays the generated intraoral photograph on the display of the doctor's terminal 30. Further, the output unit 15 stores, for example, the generated intraoral photograph data in a predetermined storage means.

In addition, the output unit 15 outputs the result of determination regarding a predetermined disease in the oral cavity to the user's terminal 20. Specifically, the output unit 15 outputs information on the state of periodontal disease and / or caries determined by the determination unit 14 to the user's terminal 20 in addition to the intraoral photograph generated by the generation unit 13. Let me.

In addition, the output unit 15 outputs the result of determination regarding a predetermined disease in the oral cavity to the doctor's terminal 30. Specifically, the output unit 15 outputs information on the state of periodontal disease and / or caries determined by the determination unit 14 to the doctor's terminal 30 together with the intraoral photograph generated by the generation unit 13. Let me. Further, the output unit 15 may output the consultation necessity information regarding the necessity of consultation of the user, which is input in the terminal 30 of the doctor, in the terminal 20 of the user.

Next, with reference to FIG. 14, the oral disease diagnosis method in the oral disease diagnosis system 1 and the oral disease diagnosis device 10 will be described. FIG. 14 is a flowchart showing an example of the processing content of the oral disease diagnosis of the present embodiment.

In step S1, the acquisition unit 11 acquires a plurality of partial images of the portion in the oral cavity of the user. In step S2, the removing unit 12 performs a removing process (see FIG. 8) for removing the reflected portion, which is a region where saliva reflects in the partial image. Further, the process of step S2 may include a process of extracting a partial image that does not include the reflection portion from the partial image acquired in step S1.

In step S3, the generation unit 13 synthesizes a plurality of partial images including the partial image not including the reflection portion obtained in step S2 to generate an intraoral photograph. In step S4, the determination unit 14 determines the presence / absence and state of a predetermined disease in the oral cavity based on the characteristics of the image of the predetermined determination target area shown in the intraoral photograph (FIGS. 10 and 10). 12) is carried out.

In step S5, the determination unit 14 determines whether or not it is determined that there is a disease in the disease determination process in step S4. If it is determined that there is a disease, the process proceeds to step S6. On the other hand, if it is not determined that there is a disease, the process proceeds to step S7.

In step S6, the determination unit 14 marks the diseased part in the intraoral photograph (see FIGS. 11 and 13). In step S7, the output unit 15 provides information on the periodontal disease and / or caries state determined by the determination unit 14 in step S4 in addition to the intraoral photograph generated by the generation unit 13, to the user's terminal 20. And / or output to the doctor's terminal 30.

Next, the process in the terminal 20 of the user of the oral disease diagnosis system 1 will be described with reference to FIG. FIG. 15 is a flowchart showing an example of processing contents in the user's terminal 20.

In step S11, the image sensor 25 captures a plurality of partial images. Then, the image acquisition unit 21 acquires a plurality of partial images captured by the image sensor 25.

In step S12, the image transmission unit 22 transmits a plurality of partial images acquired by the image acquisition unit 21 in step S11 to the oral disease diagnosis device 10.

In step S13, the determination result receiving unit 23 receives information indicating the determination result regarding a predetermined disease in the oral cavity from the oral disease diagnosis device 10. The information indicating the result of the determination includes, for example, an intraoral photograph in which the location of the disease is marked, a comment regarding the discovered disease, and the like.

In step S14, the determination result presenting unit 24 presents to the user information indicating the result of the disease determination, which was received by the determination result receiving unit 23 in step S13. The determination result presentation unit 24 displays, for example, information indicating the result of the determination of the disease on the display 26.

Next, with reference to FIG. 16, the processing at the terminal 30 of the doctor of the oral disease diagnosis system 1 will be described. FIG. 16 is a flowchart showing an example of processing contents in the terminal 30 of a doctor.

In step S21, the receiving unit 31 receives information indicating the result of determination regarding a predetermined disease in the oral cavity from the oral disease diagnostic device 10. The information indicating the result of the determination may include an intraoral photograph in which the location of the disease is marked, a comment on the discovered disease, and the like.

In step S22, the presenting unit 32 presents to the doctor information indicating the result of the determination of the disease, which was examined by the receiving unit 31 in step S21. The presentation unit 32 displays, for example, information indicating the result of determining the disease on the display 26.

In step S23, the reception unit 33 receives the consultation necessity information regarding the necessity of consultation of the user, which is input to the terminal 30 by the doctor.

In step S24, the transmission unit 34 transmits the user's consultation necessity information received in step S23 to the user's terminal 20. In addition, the transmission unit 34 may transmit the consultation necessity information to the terminal 20 via the oral disease diagnosis device 10.

Next, an oral disease diagnosis program for causing the computer to function as the oral disease diagnosis device 10 of the present embodiment will be described. FIG. 17 is a diagram showing the configuration of the oral disease diagnosis program P1.

The oral disease diagnosis program P1 includes a main module m10, an acquisition module m11, a removal module m12, a generation module m13, a determination module m14, and an output module m15 that collectively control the oral disease diagnosis process in the oral disease diagnosis device 10. Will be done. Then, each module m11 to m15 realizes each function for the acquisition unit 11, the removal unit 12, the generation unit 13, the determination unit 14, and the output unit 15 in the oral disease diagnosis device 10. The oral disease diagnosis program P1 may be transmitted via a transmission medium such as a communication line, or may be stored in the recording medium M1 as shown in FIG. ..

In the oral disease diagnosis system 1, the oral disease diagnosis device 10, the oral disease diagnosis method, and the oral disease diagnosis program P1 of the present embodiment described above, an intraoral photograph is generated based on a partial image in which the reflex portion is removed. By diagnosing an oral disease based on the intraoral photograph generated in this way, the possibility of misdiagnosis of the disease can be reduced.

Further, in the oral disease diagnosis system according to another form, the removal portion may remove the reflection portion in the partial image.

According to the above embodiment, it is possible to obtain a partial image that does not include the reflection portion based on the acquired partial image.

Further, in the oral disease diagnosis system according to another form, the acquisition unit acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions, and the removal unit is the portion represented by the partial image. By converting a plurality of partial images so that the projection direction is aligned with a predetermined direction, aligning the pixels representing the same part in the oral cavity so that they are associated with each other, and averaging the plurality of partial images. A partial image in which the reflective portion is removed may be acquired.

According to the above embodiment, a plurality of partial images obtained by capturing the same portion from different directions are acquired. In the plurality of partial images acquired in this way, even if reflection occurs at a specific portion in one partial image, there is a high possibility that the reflection portion and the reflection state are different in the other partial images. Since a plurality of partial images converted so that the projection direction of a certain portion is aligned in a predetermined direction are aligned and then added and averaged, it is possible to obtain a partial image in which the reflection portion is removed. Become.

Further, the oral disease diagnosis system according to another form further includes a determination unit for determining the presence / absence and state of a predetermined disease in the oral cavity based on the characteristics of the image of the predetermined determination target area shown in the intraoral photograph. It may be prepared.

According to the above embodiment, since the disease is determined based on the intraoral photograph that does not include the reflex portion, it is possible to carry out a diagnosis with a low possibility of misdiagnosis.

Further, in the oral disease diagnosis system according to another form, the determination unit extracts at least one of the gum region in which the gum is represented and the tooth region in which the tooth is represented from the intraoral photograph, and determines the gum region and the tooth region. The presence or absence and condition of periodontal disease may be determined based on the characteristics of at least one image.

According to the above embodiment, since the disease is determined based on a predetermined feature of at least one image of the gum region and the tooth region extracted as the determination target region, the diagnosis regarding periodontal disease is appropriately performed.

Further, in the oral disease diagnosis system according to another form, the determination unit extracts the tooth region in which the tooth is represented from the intraoral photograph and determines the presence / absence and state of caries based on the characteristics of the image of the tooth region. May be.

According to the above embodiment, since the disease is determined based on a predetermined feature of the image of the tooth region extracted as the determination target region, the diagnosis regarding dental caries is appropriately performed.

Further, in the oral disease diagnosis system according to another form, the output unit may output the result of determination regarding a predetermined disease in the oral cavity on the user's first terminal.

According to the above embodiment, since the result of the disease determination is output on the user's terminal, the user can be easily notified of the state of the disease in the oral cavity of the user.

Further, in the oral disease diagnosis system according to another embodiment, the output unit outputs the result of determination regarding a predetermined disease in the oral cavity to a second terminal of a doctor different from the user, and causes the second terminal to output the result. The input consultation necessity information regarding the necessity of consultation of the user may be output to the user's first terminal.

According to the above form, since the result of the determination of the disease is output on the terminal of the doctor, the doctor can be easily notified of the state of the disease in the oral cavity of the user. Further, since the information regarding the necessity of the user's consultation, which is input by the doctor according to the result of the notified diagnosis, can be notified to the user, it is possible to have the user receive an appropriate diagnosis.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in the present specification. This embodiment can be implemented as a modified or modified mode without departing from the spirit and scope of the present invention determined by the description of the claims. Therefore, the description herein is for purposes of illustration only and has no limiting implications for this embodiment.

Each aspect / embodiment described herein includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered Trademarks), GSM (Registered Trademarks), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), It may be applied to systems utilizing Bluetooth®, other suitable systems and / or next-generation systems extended based on them.

The order of the processing procedures, sequences, flowcharts, and the like of each aspect / embodiment described in the present specification may be changed as long as there is no contradiction. For example, the methods described herein present elements of various steps in an exemplary order, and are not limited to the particular order presented.

Information and the like can be output from the upper layer (or lower layer) to the lower layer (or upper layer). Input / output may be performed via a plurality of network nodes.

The input / output information and the like may be stored in a specific location (for example, memory) or may be managed by a management table. Input / output information and the like can be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present specification may be used alone, in combination, or switched with execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether referred to as software, firmware, middleware, microcode, hardware description language, or other names, is an instruction, instruction set, code, code segment, program code, program, subprogram, software module. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc. should be broadly interpreted.

Further, software, instructions, and the like may be transmitted and received via a transmission medium. For example, the software uses wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

The terms described herein and / or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

The terms "system" and "network" as used herein are used interchangeably.

Further, the information, parameters, etc. described in the present specification may be represented by an absolute value, a relative value from a predetermined value, or another corresponding information. ..

The phrase "based on" as used herein does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

When the terms "first", "second", etc. are used herein, any reference to the elements does not generally limit the quantity or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must somehow precede the second element.

As long as "include", "including", and variations thereof are used within the scope of this specification or claims, these terms are similar to the term "comprising". Is intended to be inclusive. Furthermore, the term "or" as used herein or in the claims is intended not to be an exclusive OR.

In the present specification, a plurality of devices shall be included unless the device is clearly present in only one context or technically.

In the whole of the present disclosure, if the context clearly does not indicate the singular, it shall include the plural.

1 ... Oral disease diagnosis system, 10 ... Oral disease diagnosis device, 11 ... Acquisition unit, 12 ... Removal unit, 13 ... Generation unit, 14 ... Judgment unit, 15 ... Output unit, 20, 30 ... Terminal, 21 ... Image acquisition unit , 22 ... image transmission unit, 23 ... judgment result reception unit, 24 ... judgment result presentation unit, 25 ... image sensor, 26 ... display, 31 ... reception unit, 32 ... presentation unit, 33 ... reception unit, 34 ... transmission unit, 35 ... Display, M1 ... Recording medium, m10 ... Main module, m11 ... Acquisition module, m12 ... Removal module, m13 ... Generation module, m14 ... Judgment module, m15 ... Output module, P1 ... Oral disease diagnosis program.

Claims (7)

It is an oral disease diagnosis system that diagnoses a disease in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity.
An acquisition unit that acquires a plurality of partial images of the portion in the oral cavity of the user,
A removal unit that acquires a partial image that does not include the reflection portion by removing a reflection portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflection portion.
A generation unit that synthesizes a plurality of partial images including the partial image acquired by the removal unit to generate the intraoral photograph, and a generation unit.
An output unit that outputs the intraoral photograph generated by the generation unit, and
Equipped with a,
The acquisition unit acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions.
The removing portion transforms a plurality of the partial images so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction so that pixels representing the same portion in the oral cavity are associated with each other. A partial image from which the reflection portion has been removed is acquired by aligning the plurality of partial images and averaging the plurality of partial images.
Oral disease diagnostic system. It is an oral disease diagnosis system that diagnoses a disease in the user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity.
An acquisition unit that acquires a plurality of partial images of the portion in the oral cavity of the user,
A removal unit that acquires a partial image that does not include the reflection portion by removing a reflection portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflection portion.
A generation unit that synthesizes a plurality of partial images including the partial image acquired by the removal unit to generate the intraoral photograph, and a generation unit.
An output unit that outputs the intraoral photograph generated by the generation unit, and
A determination unit for determining the presence or absence and a state of a predetermined disease in the oral cavity based on the characteristics of the image of the predetermined determination target area shown in the intraoral photograph is provided.
The output unit outputs the result of determination regarding a predetermined disease in the oral cavity to a second terminal of a doctor different from the user, and is input to the second terminal. The information on the necessity of consultation regarding the denial is output on the first terminal of the user.
Oral disease diagnostic system. The determination unit extracts at least one of the gum region in which the gum is represented and the tooth region in which the tooth is represented from the intraoral photograph, and is based on the characteristics of at least one image of the gum region and the tooth region. Determine the presence and condition of periodontal disease,
The oral disease diagnosis system according to claim 2. The determination unit extracts the tooth region in which the tooth is represented from the intraoral photograph, and determines the presence / absence and state of caries based on the characteristics of the image of the tooth region.
The oral disease diagnosis system according to claim 2. The output unit outputs the result of determination regarding a predetermined disease in the oral cavity to the user's first terminal.
The oral disease diagnostic system according to any one of claims 2 to 4. An oral disease diagnosis program for making a computer function as an oral disease diagnosis system for diagnosing a disease in a user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity.
On the computer
An acquisition function for acquiring a plurality of partial images of the user's oral cavity, and
A removal function for acquiring a partial image that does not include the reflective portion by removing a reflective portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflective portion.
A generation function for generating the intraoral photograph by synthesizing a plurality of partial images including the partial image acquired by the removal function, and
An output function that outputs the intraoral photograph generated by the generation function, and
Realized,
The acquisition function acquires a plurality of partial images obtained by capturing the same portion in the oral cavity from different directions.
The removal function transforms a plurality of the partial images so that the projection direction of the portion represented in the partial image is aligned with a predetermined direction so that pixels representing the same portion in the oral cavity are associated with each other. A partial image from which the reflection portion has been removed is acquired by aligning the plurality of partial images and averaging the plurality of partial images.
Oral disease diagnosis program. An oral disease diagnosis program for making a computer function as an oral disease diagnosis system for diagnosing a disease in a user's oral cavity based on an intraoral photograph consisting of a plurality of images showing a predetermined area in the oral cavity.
On the computer
An acquisition function for acquiring a plurality of partial images of the user's oral cavity, and
A removal function for acquiring a partial image that does not include the reflective portion by removing a reflective portion that is a region where reflection occurs in the partial image or extracting a partial image that does not include the reflective portion.
A generation function for generating the intraoral photograph by synthesizing a plurality of partial images including the partial image acquired by the removal function, and
An output function that outputs the intraoral photograph generated by the generation function, and
Based on the characteristics of the image of the predetermined determination target area shown in the intraoral photograph, a determination function for determining the presence or absence and state of a predetermined disease in the oral cavity is realized.
The output function outputs the result of determination regarding a predetermined disease in the oral cavity to a second terminal of a doctor different from the user, and is input to the second terminal. The information on the necessity of consultation regarding the denial is output on the first terminal of the user.
Oral disease diagnosis program.

Images