WO2024004100A1 - Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme - Google Patents

Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme Download PDF

Info

Publication number
WO2024004100A1
WO2024004100A1 PCT/JP2022/026058 JP2022026058W WO2024004100A1 WO 2024004100 A1 WO2024004100 A1 WO 2024004100A1 JP 2022026058 W JP2022026058 W JP 2022026058W WO 2024004100 A1 WO2024004100 A1 WO 2024004100A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
intraoral
user
side image
teeth
Prior art date
Application number
PCT/JP2022/026058
Other languages
English (en)
Japanese (ja)
Inventor
孝博 岡本
Original Assignee
株式会社スクリエ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社スクリエ filed Critical 株式会社スクリエ
Priority to PCT/JP2022/026058 priority Critical patent/WO2024004100A1/fr
Publication of WO2024004100A1 publication Critical patent/WO2024004100A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/24—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth

Definitions

  • the present invention relates to a technique that is effective in supporting the photographing of images for examining the inside of a user's oral cavity.
  • Patent Document 1 on the proximal end side of the insertion plate inserted into the oral cavity, there is a smartphone holding plate formed to form an angle of more than 180° and less than 270° with respect to the surface of the insertion plate, and a smartphone holding plate.
  • An intraoral photographing aid is disclosed that includes a smartphone fixing means for fixing a smartphone.
  • a cylindrical main body has a first end having an opening into which an imaging device can be inserted, and a first end located on the opposite side of the first end, from the inside of the main body to the outside of the main body.
  • An intraoral photography aid is disclosed that includes a second end having a window that provides a field of view, and an inner circumferential surface that extends from the first end toward the second end and allows sliding of an imaging device. There is.
  • Patent Documents 1 and 2 when a patient or a photography assistant uses a mobile terminal to photograph intraoral images, there are variations in the photographic results, resulting in images that are not suitable for image analysis. may be photographed. In this case, it is not possible to perform an appropriate examination of the oral cavity remotely. Therefore, the present inventor focused on the need for a system that supports patients and photography assistants in photographing the inside of the oral cavity.
  • the present invention provides an intraoral photography support system, an intraoral photography support method, and a program that support photography in order to easily enable photography of the inside of a patient's oral cavity using a camera app on a mobile terminal.
  • the purpose is to provide
  • the present invention is an intraoral imaging support system that supports imaging of images for examining the inside of a user's oral cavity.
  • a guide unit that guides the user to take a front image, a right side image, and a left side image that show the alignment of the teeth or the condition of the teeth and gums while wearing a predetermined mouth opener; an installation instruction section that instructs the user to attach a predetermined intraoral photography aid to the terminal; a photographing instruction section that instructs a user's terminal to which the intraoral photographing aid is attached to photograph an intraoral image; an acquisition unit that acquires a frontal image, a right side image, a left side image, and an intraoral image taken by the user; a condition determination unit that determines whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions; If it is determined that the predetermined condition is not met, re-photograph the image that is determined to not satisfy the predetermined condition among the frontal image, right side image, left side image, and intraoral image for the user
  • an intraoral photographing support system that supports photographing of images for examining the inside of a user's oral cavity. Guide the user to take a front image, right side image, and left side image that show the state of the gums, instruct the user to attach a prescribed intraoral photography aid to the user's terminal, and attach the intraoral photography aid.
  • Instruct the user to take an intraoral image with the user's terminal acquire the front image, right side image, left side image, and intraoral image taken by the user, and obtain the acquired front image, right side image, left side image, and It is determined whether the intraoral image satisfies a predetermined condition, and if it is determined that the predetermined condition is not met, the user is asked to display the front image, right side image, left side image, and inside of the intraoral image. , instructs the user to retake images that are determined not to meet predetermined conditions.
  • the present invention when photographing the inside of a patient's oral cavity using a camera application on a mobile terminal, it is possible to easily perform this photographing, thereby supporting the photographing.
  • FIG. 1 is a diagram illustrating an overview of an intraoral imaging support system 1.
  • FIG. 1 is a diagram showing the functional configuration of an intraoral imaging support system 1.
  • FIG. 2 is a diagram showing a flowchart of image acquisition processing executed by the intraoral imaging support system 1.
  • FIG. 3 is a diagram schematically showing an example of an instruction to photograph a front image displayed by the user terminal 100.
  • FIG. 3 is a diagram schematically illustrating an example of an instruction to photograph a right image displayed by the user terminal 100.
  • FIG. 7 is a diagram schematically illustrating an example of an instruction to shoot a left image displayed by the user terminal 100.
  • FIG. It is a perspective view of the intraoral photographing aid 6 seen from the rear side.
  • FIG. 6 is a perspective view of the intraoral photographing aid 6 seen from the front side.
  • FIG. 6 is a side view of the intraoral photographing aid 6.
  • FIG. FIG. 6 is a front view of the intraoral photographing aid 6.
  • FIG. FIG. 6 is a side view of the intraoral photographing aid 6.
  • FIG. FIG. 6 is a rear view of the intraoral photographing aid 6.
  • FIG. FIG. 6 is an explanatory diagram showing an intraoral photographing aid 6 separated. 11 is an enlarged cross-sectional view taken along line AA in FIG. 10.
  • FIG. 3 is a diagram schematically illustrating an example of an instruction for photographing an upper jaw image displayed by the user terminal 100.
  • FIG. FIG. 6 is an explanatory diagram of the usage state of the intraoral photographing aid 6.
  • FIG. 3 is a diagram schematically showing an example of an instruction to photograph a lower jaw image displayed by the user terminal 100.
  • FIG. 5 is a diagram showing a flowchart of re-imaging instruction processing executed by the intraoral imaging support system 1.
  • FIG. FIG. 2 is a diagram schematically showing an example of a reshooting instruction displayed by the user terminal 100.
  • FIG. 2 is a diagram showing a flowchart of intraoral determination processing executed by the intraoral imaging support system 1.
  • FIG. 3 is a diagram schematically showing an example of a notification displayed by the user terminal 100.
  • FIG. 2 is a diagram showing a flowchart of dentition determination processing executed by the intraoral imaging support system 1.
  • FIG. 2 is a diagram showing a flowchart of learned model generation processing executed by the intraoral imaging support system 1.
  • FIG. 1 is a schematic diagram for explaining an overview of an intraoral imaging support system 1.
  • a computer 10 having a server function cooperates with a user terminal 100 owned by the user to implement processing for supporting the imaging of images for examining the user's oral cavity.
  • the intraoral imaging support system 1 may be any system as long as it includes a computer 10 having at least a server function.
  • This computer 10 may be realized by, for example, one computer, or may be realized by a plurality of computers such as a cloud computer.
  • a cloud computer refers to one that uses any computer in a scalable manner to perform a certain function, or one that includes multiple functional modules to realize a certain system, and whose functions can be freely combined. It can be anything.
  • the intraoral imaging support system 1 includes a computer 10 and a user terminal 100 owned by a user.
  • the terminal devices constituting the intraoral imaging support system 1 are merely examples, and the number, types, and functions of the terminal devices other than the computer 10 can be changed as appropriate.
  • the computer 10 guides the user to take a front image, a right side image, and a left side image that show the bite of the teeth or the condition of the teeth and gums while wearing a predetermined mouse opener (step S1 ).
  • the computer 10 transmits to the user terminal 100 an instruction to wear a predetermined mouse opener and an instruction on how to wear it.
  • User terminal 100 receives this instruction and displays it on its own display unit. Based on this instruction, the user attaches a predetermined mouth opener to his or her oral cavity.
  • the user terminal 100 notifies the computer 10 that the attachment of the mouse opener has been completed. Based on this notification, the computer 10 sends an instruction to the user terminal 100 to photograph each of the front image, right side image, and left side image.
  • This photographing instruction includes a sample image serving as a sample of each image and an instruction on a method of photographing each image.
  • the user terminal 100 receives this photographing instruction and displays it on its own display unit.
  • the user views the photographing instructions and understands the appropriate image condition and photographing method.
  • the user uses the user terminal 100 to display a front image, a right side image, and a left side image, which show the tooth alignment similar to the sample image or the condition of the teeth and gums. Take one or more photos.
  • the computer 10 acquires a notification of completion of photographing the front image, right side image, and left side image from the user terminal 100, and executes subsequent processing.
  • the computer 10 instructs the user terminal 100 to attach a predetermined intraoral photographing aid 6 (step S2).
  • the computer 10 transmits to the user terminal 100 an instruction to attach a predetermined intraoral photographing aid 6 and an instruction on the attachment method.
  • User terminal 100 receives this instruction and displays it on its own display unit.
  • the user attaches the predetermined intraoral photographing aid 6 to the user terminal 100 based on this instruction.
  • the computer 10 acquires the notification of the completion of attachment of the intraoral photographing aid 6 from the user terminal 100, and executes the subsequent processing.
  • the computer 10 instructs the user terminal 100 to which the intraoral photographing aid 6 is attached to photograph an intraoral image (step S3).
  • the computer 10 transmits an instruction to insert the intraoral photographing aid 6 into the user's oral cavity and an instruction on the insertion method to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit. The user views this instruction and understands how to insert this intraoral photographing aid 6.
  • the user terminal 100 notifies the computer 10 that the insertion of the intraoral photographing aid 6 has been completed. Based on this notification, the computer 10 transmits an instruction to take an intraoral image to the user terminal 100.
  • This photographing instruction includes a sample image serving as a sample of the intraoral image and an instruction on how to photograph the image.
  • the user terminal 100 receives this photographing instruction and displays it on its own display unit.
  • the user views the photographing instructions and understands the appropriate image condition and photographing method.
  • the user uses the user terminal 100 to photograph one or more intraoral images that show the alignment of teeth similar to the sample image or the condition of the teeth and gums.
  • the computer 10 obtains a notification of completion of intraoral image capturing from the user terminal 100, and executes subsequent processing.
  • the computer 10 acquires a front image, a right side image, a left side image, and an intraoral image photographed by the user (step S4).
  • the computer 10 requests the user terminal 100 to transmit the front image, right side image, left side image, and intraoral image photographed by the user. Based on this request, the user terminal 100 transmits the photographed frontal image, right side image, left side image, and intraoral image to the computer 10.
  • the image acquisition module receives the front image, right side image, left side image, and intraoral image, and acquires the front image, right side image, left side image, and intraoral image photographed by the user.
  • the computer 10 determines whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions (step S5).
  • the predetermined condition is, for example, that all teeth are aligned.
  • the present embodiment will be described on the assumption that the predetermined condition is that all of the user's teeth are in alignment. Note that in consideration of people who have lost their teeth, the predetermined condition may be set as whether or not all the places where teeth would have grown are shown.
  • the computer 10 analyzes each of the acquired front image, right side image, left side image, and intraoral image, and detects the teeth and gums shown in each image. Based on the analysis results, the computer 10 separates the teeth shown in each image one by one and recognizes each tooth. The computer 10 determines whether all teeth in each recognized image are aligned. The computer 10 determines whether the user's teeth are all aligned based on each tooth in the recognition result.
  • the computer 10 determines that the predetermined condition is not met, the computer 10 displays the image that is determined not to satisfy the predetermined condition among the frontal image, right side image, left side image, and intraoral image to the user. An instruction is given to take the image again (step S6). If the computer 10 determines that the predetermined condition is not met, the computer 10 determines the predetermined condition in the image that is determined not to satisfy the predetermined condition among the acquired frontal image, right side image, left side image, and intraoral image. Identify a shooting method that satisfies the following. Based on the recognition results, the computer 10 identifies locations where teeth not shown in the image exist, locations where the entire image of the teeth could not be recognized, and the like. The computer 10 specifies how to photograph these locations.
  • the computer 10 notifies the user terminal 100 that it is necessary to retake the image determined not to satisfy the predetermined conditions.
  • the user terminal 100 receives this notification and displays it on its own display unit. The user views this notification and learns which images require re-shooting.
  • the user terminal 100 receives an input from the user to retake an image that has been determined not to satisfy a predetermined condition, and transmits the input content to the computer 10.
  • the re-photographing instruction module acquires this input content and transmits to the user terminal 100 an instruction to re-photograph the image determined not to satisfy a predetermined condition.
  • This re-photographing instruction includes the type of image that needs to be re-photographed and instructions for the specified photographing method.
  • the user terminal 100 receives this reshooting instruction and displays it on its own display unit.
  • the user views this re-photographing instruction and understands the images that have been determined not to satisfy the predetermined conditions and the photographing method for satisfying the predetermined conditions.
  • the user uses the user terminal 100 to replay one or more images that meet the predetermined conditions so that the alignment of the teeth or the condition of the teeth and gums can be seen. Take a photo.
  • the computer 10 acquires the re-photographed image and re-executes the above-described processing from step S5 on this image, so that all images including the front image, right side image, left side image, and intraoral image are Support photography to meet predetermined conditions.
  • the above is an overview of the intraoral imaging support system 1.
  • the present intraoral photography support system when photographing the inside of a patient's oral cavity using a camera application of a mobile terminal, this photography can be easily performed, so that it is possible to support photography.
  • FIG. 2 is a block diagram showing the configuration of the intraoral imaging support system 1.
  • the intraoral imaging support system 1 is a system that supports the imaging of images for examining the user's oral cavity, and includes at least a computer 10 .
  • the intraoral imaging support system 1 further includes a user terminal 100 carried by the user.
  • the intraoral imaging support system 1 is a system in which a computer 10 and a user terminal 100 are connected to enable data communication via a network 5 such as a public line network.
  • a network 5 such as a public line network.
  • the terminal devices constituting the intraoral imaging support system 1 are merely examples, and the number, types, and functions of the terminal devices other than the computer 10 can be changed as appropriate.
  • the computer 10 has a server function, and may be implemented by, for example, one computer or multiple computers such as a cloud computer.
  • the computer 10 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc. as a control unit, and as a communication unit, with other terminals, devices, etc.
  • the apparatus includes a re-photographing instruction section 15 and the like that instructs to re-photograph an image determined not to satisfy a predetermined condition among the images, the left-side image, and the intraoral image.
  • the computer 10 includes, as a processing unit, various devices that execute various processes, a condition determination unit 16 that determines whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions. .
  • the control section by reading a predetermined program, the control section cooperates with the communication section to create a request acquisition module, a guide module, an attachment instruction module, a photographing instruction module, an image acquisition module, a re-photographing instruction module, a notification module, Realize a learning data acquisition module.
  • the control unit reads a predetermined program, thereby collaborating with the processing unit to create an analysis module, a recognition module, a condition determination module, a specific module, an oral condition determination module, a dentition determination module, and the like. Realize modules, learning modules, and generation modules.
  • the user terminal 100 carried by the user is, for example, a mobile terminal such as a mobile phone, a smartphone, or a tablet terminal.
  • the user terminal 100 includes a CPU, GPU, RAM, ROM, etc. as a terminal control unit, a device etc. as a communication unit to enable communication with other terminals, devices, etc., and an input/output unit for processing various information. Equipped with devices, etc. for executing input/output.
  • each process executed by the intraoral imaging support system 1 will be explained together with the processes executed by each of the modules described above.
  • each module may execute its processing content as its own function, or may execute it via a predetermined application.
  • FIG. 3 This figure is a diagram showing a flowchart of image acquisition processing executed by the computer 10.
  • This image acquisition process guides the above-mentioned user to take a front image, right side image, and left side image that show the bite of the teeth or the condition of the teeth and gums while wearing the specified mouse opener.
  • step S1 installation instruction processing
  • step S2 installation instruction processing
  • step S3 Details of the photographing instruction process
  • step S4 for acquiring the front image, right side image, left side image, and intraoral image photographed by the user.
  • the request acquisition module acquires a request to start photographing an image from the user terminal 100 (step S10).
  • the user terminal 100 transmits a request to the computer 10 to start capturing images necessary for online medical treatment, remote medical consultation, and the like.
  • the request acquisition module receives this request and acquires a request to start capturing an image from the user terminal 100.
  • the guide module guides the user to take a front image, a right side image, and a left side image showing the bite of the teeth or the condition of the teeth and gums while the user is wearing a predetermined mouth opener (step S11). ).
  • the front image is an image of the user's oral cavity taken from the front direction.
  • the right side image is an image of the user's oral cavity taken from the right direction.
  • the left side image is an image of the user's oral cavity taken from the left direction.
  • the guide module transmits to the user terminal 100 an instruction to wear a predetermined mouse opener and an instruction on how to wear the mouse opener. User terminal 100 receives this instruction and displays it on its own display unit. Based on this instruction, the user attaches a predetermined mouth opener to his or her oral cavity.
  • the user terminal 100 notifies the computer 10 that the attachment of the mouse opener has been completed. Based on this notification, the guide module transmits to the user terminal 100 an instruction to photograph each of the front image, right side image, and left side image.
  • This photographing instruction includes a sample image serving as a sample of each image and an instruction on a method for photographing each image.
  • the user terminal 100 receives this photographing instruction and displays it on its own display unit. The user views the photographing instructions and understands the appropriate image condition and photographing method.
  • the user uses the user terminal 100 to take one image each of a front image, a right side image, and a left side image that show the tooth alignment similar to the sample image or the condition of the teeth and gums. Or take multiple photos.
  • the user terminal 100 When the front image, the right side image, and the left side image have been photographed, the user terminal 100 notifies the computer 10 that the front image, the right side image, and the left side image have been photographed.
  • the guide module sends instructions to the user terminal 100 to wear a predetermined mouse opener and to take a front image, a right-side image, and a left-side image, thereby displaying a state in which the user is wearing a predetermined mouse opener.
  • the user is guided to take a front image, a right side image, and a left side image, which show the alignment of the teeth or the condition of the teeth and gums.
  • the guide module guides the user to take a front image, a right side image, and a left side image.
  • the guide module first transmits an instruction to take a front image to the user terminal 100.
  • the user terminal 100 receives the instruction to take a frontal image and displays it on its own display unit (see FIG. 4). Based on the displayed content, the user uses the user terminal 100 to take one or more frontal images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the photographing of the frontal image has been completed.
  • FIG. 4 is a diagram schematically showing an example of an instruction to photograph a front image displayed by the user terminal 100.
  • an instruction 50 for photographing is shown.
  • This photographing instruction 50 includes a sample image 51 of a front image and a photographing method 52.
  • the sample image 51 is a sample image of the oral cavity photographed from the front.
  • the photographing method 52 is a character string related to instructions for the photographing method, such as "Please photograph as much of the molars as possible.”
  • the user views the sample image 51 and the photographing method 52, and uses the user terminal 100 to photograph one or more frontal images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the photographing of the frontal image has been completed.
  • the guide module transmits an instruction to photograph the right side image to the user terminal 100.
  • the user terminal 100 receives the instruction to photograph the right side image and displays it on its own display unit (see FIG. 5). Based on the displayed content, the user uses the user terminal 100 to take one or more images of the right side showing the alignment of the teeth or the condition of the teeth and gums. The user terminal 100 notifies the computer 10 that the shooting of the right side image has been completed.
  • FIG. 5 is a diagram schematically showing an example of an instruction to photograph the right image displayed by the user terminal 100.
  • an instruction 53 for photographing is shown.
  • This photographing instruction 53 includes a sample image 54 of the right side image and a photographing method 55.
  • the sample image 54 is a sample image of the oral cavity taken from the right direction.
  • the photographing method 55 indicates a character string related to instructions for the photographing method, such as "Please photograph as far as the molars as possible.”
  • the user views the sample image 54 and the photographing method 55, and uses the user terminal 100 to photograph one or more right side images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the shooting of the right side image has been completed.
  • the guide module transmits an instruction to photograph the left image to the user terminal 100.
  • the user terminal 100 receives the instruction to shoot the left side image and displays it on its own display unit (see FIG. 6). Based on the displayed content, the user uses the user terminal 100 to take one or more images of the left side showing the alignment of the teeth or the condition of the teeth and gums. The user terminal 100 notifies the computer 10 that the shooting of the left side image has been completed.
  • FIG. 6 is a diagram schematically showing an example of an instruction to photograph a left image displayed by the user terminal 100.
  • an instruction 56 for photographing is shown.
  • This photographing instruction 56 includes a sample image 57 of the left image and a photographing method 58.
  • the sample image 57 is a sample image of the oral cavity taken from the left direction.
  • the photographing method 58 is a character string related to instructions for the photographing method, such as "Please photograph as much of the molars as possible.”
  • the user views the sample image 57 and the photographing method 58, and uses the user terminal 100 to photograph one or more left side images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the shooting of the left side image has been completed.
  • the guide module may send together sample images of the front image, right side image, and left side image, and the shooting instruction to the user terminal 100 as the shooting instruction.
  • the user terminal 100 displays these sample images and shooting instructions together. Based on the displayed content, the user uses the user terminal 100 to photograph a front image, a right side image, and a left side image that show the alignment of the teeth or the condition of the teeth and gums. The user terminal 100 notifies the computer 10 that the shooting of the front image, right side image, and left side image has been completed.
  • the user terminal 100 does not notify the computer 10, but instead acquires instructions for shooting the front image, right side image, and left side image all at once, and When photographing of an image is completed, instructions for photographing the next image may be displayed. Furthermore, the order in which the front image, right side image, and left image are photographed is not limited to the example described above, and can be changed as appropriate.
  • step S12 When the guide module completes guidance for photographing a frontal image, a right-side image, and a left-side image in which the alignment of the teeth or the condition of the teeth and gums can be seen (frontal image, right-side image, and left-side image), the computer 10 When it is notified that the photographing of has been completed), the process of step S12 is executed.
  • the attachment instruction module instructs the user terminal 100 to attach a predetermined intraoral photographing aid 6 (step S12). Details of the intraoral photographing aid 6 will be described later.
  • the attachment instruction module transmits instructions for attaching a predetermined intraoral photographing aid 6 (see FIGS. 7 to 14) and instructions for the attachment method to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit.
  • the user attaches this intraoral photographing aid 6 to the user terminal 100 based on this instruction.
  • the user terminal 100 notifies the computer 10 that the installation of the intraoral photographing aid 6 has been completed.
  • the computer 10 executes the process of step S13 when the user terminal 100 completes the installation of the intraoral photography aid 6 (when it is notified that the installation of the intraoral photography aid 6 has been completed). .
  • FIG. 7 is a perspective view of the intraoral photographing aid 6 viewed from the rear side.
  • FIG. 8 is a perspective view of the intraoral photographing aid 6 seen from the front side.
  • FIG. 9 is a side view of the intraoral photographing aid 6.
  • FIG. 10 is a front view of the intraoral photographing aid 6.
  • FIG. 11 is a side view of the intraoral photographing aid 6.
  • FIG. 12 is a rear view of the intraoral photographing aid 6.
  • FIG. FIG. 13 is an exploded explanatory diagram of the intraoral photographing aid 6.
  • FIG. FIG. 14 is an enlarged cross-sectional view taken along line AA in FIG. 10.
  • the intraoral photographing aid 6 includes an insertion plate 2, a holder 3 that holds a user terminal 100, and an attachment/detachment mechanism 4.
  • the direction along the width of the user terminal 100 is the left-right direction X
  • the direction along the thickness direction of the user terminal 100 is the front-back direction Y
  • the vertical direction of the user terminal 100 is defined as the vertical direction Z.
  • the insertion plate 2 is made of a metal plate such as stainless steel, has a relatively thin thickness, is long on one side, and has a predetermined area. As shown in FIG. 11, at least one side of the insertion plate 2 has a mirror finish, and a reflective surface 21 is formed. When the insertion plate 2 is inserted into the user's oral cavity, the interior of the oral cavity will be reflected on the reflective surface 21.
  • the reflective surface 21 of the insertion plate 2 is anti-fogged by, for example, coating with a resin film or a transparent anti-fog agent.
  • the insertion plate 2 has a concave central portion at the tip and is rounded on both sides, so that it is not easily damaged even when inserted into the oral cavity.
  • the insertion plate 2 has a constricted portion formed at an intermediate position in the longitudinal direction. As also shown in FIGS. 8 and 10, the base end portion of the insertion plate 2 is removably attached to a clip 41, which will be described later. Note that the size and shape of the insertion plate 2 may be arbitrary. For example, the insertion plates 2 may be classified into types for children, adults, men, women, the elderly, etc.
  • the holder 3 is attached to the holder 3 with the insertion plate 2 projecting forward at a predetermined angle via an attachment/detachment mechanism 4.
  • the holder 3 allows the user terminal 100 to be mounted with the camera facing the reflective surface 21 of the insertion plate 2 so that the user terminal 100 can accurately photograph the reflective surface 21 that reflects the inside of the oral cavity. and the reflective surface 21 of the insertion plate 2 are held at a predetermined angle ⁇ .
  • the insertion plate 2 is attached to the front surface of the holder 3 at an angle ⁇ .
  • the angle ⁇ between the insertion plate 2 and the user terminal 100 is preferably set to be greater than 180° and less than 270°, for example. More preferably, the angle ⁇ is set within a range of about 210° to 240°.
  • the holder 3 is made of, for example, a hard synthetic resin such as plastic, and is a flat, horizontally elongated, substantially octagonal solid body with some thickness in the front-rear direction Y. As shown in FIGS. Note that the shape of the holder 3 may be arbitrary, such as a rectangular shape, other polygonal shapes, circular shapes, and elliptical shapes.
  • the holder 3 has a mounting portion 31 for the user terminal 100 formed at a substantially central rectangular portion in the left-right direction X.
  • the attachment part 31 has a notch with a width matching the width of the user terminal 100 on the rear side of the holder 3, which is opposite to the side where the insertion plate 2 is attached. It consists of a recess formed.
  • the mounting section 31 mounts and fixes the user terminal 100 in a fitted manner.
  • a holding surface 33 is provided on the concave side of the mounting portion 31 to hold the user terminal 100 in position using adhesive force or suction force.
  • the mounting portion 31 is capable of adjusting the mounting position of the user terminal 100 along the vertical direction Z. That is, the position of the camera of the user terminal 100 is provided so as to be changeable.
  • the attachment part 31 may use any other fixing means, such as a hook, a belt, a magnet, an adhesive part, or the like.
  • Handle portions 32 are integrally formed on both left and right sides of the holder 3 and protrude in the left-right direction so that the user can hold the holder 3 by hand.
  • the handle portion 32 is formed, for example, so that the width in the vertical direction Z gradually becomes narrower from the center of the holder 3 toward both outer sides, and the width in the thickness direction Y also becomes narrower, and the corner portions are rounded. It has a similar shape.
  • the handle portion 32 of the holder 3 is not limited to the above-described structure; for example, it may be partially recessed to match the shape of the fingers or hand, or may be made with rubber or textured to prevent slipping. It may be shaped to fit a finger or have other structures that make it easy to hold. Further, the handle portion 32 may be formed only on either the left side or the right side of the holder 3. Further, a simple structure may be adopted in which the handle portions 32 are not provided on both the left and right sides of the holder 3.
  • the user In the configuration of the holder 3 without the handle part 32, the user has to hold it with one hand, so it may be less safe during use than the handle part 32 described above, but the structure is simplified and Moreover, since it is compact, it is advantageous in manufacturing at low cost and in carrying and storing.
  • the attachment/detachment mechanism 4 includes, for example, a clip 41 for attaching the insertion plate 2, and an attachment/detachment guide 42 that detachably engages with this clip 41 provided on the holder 3 side. There is.
  • the attachment/detachment mechanism 4 will be described as a sliding mechanism that can be attached and detached in a sliding manner. Since the attachment/detachment mechanism 4 is a sliding mechanism, it also serves as a position adjustment means that allows the insertion plate 2 to slide relative to the holder 3 to adjust the attachment position.
  • the attachment/detachment mechanism 4 is another mechanism such as a magnetic mechanism that can be attached and detached using a magnetic type
  • the insertion plate 2 can be attached to the holder 3 at a predetermined angle and in a detachable manner.
  • the attachment/detachment mechanism 4 may have, for example, a structure in which the insertion plate 2 is detachably attached to the holder 3 at a predetermined angle, such as a structure in which the insertion plate 2 is detachably attached at a predetermined angle, such as a structure in which the insertion plate 2 is detachably attached in a fitting type, a structure in which it is detachably attached in a clamp type. Any structure may be used.
  • the clip 41 is made of a hard synthetic resin such as the same plastic as the holder 3, for example.
  • the clip 41 is formed with an insertion hole 411 into which the base end of the insertion plate 2 is inserted in a fitting manner.
  • the insertion hole 411 is recessed in an inclined direction with respect to the holder 3, and is set at the above-mentioned predetermined angle ⁇ .
  • a stealth hole is formed on the upper side of the clip 41.
  • an engaging element 412 which has a substantially L-shaped cross section and is elongated in a straight line in the lateral direction, is protruded.
  • Two engaging members 412 are provided parallel to each other with substantially L-shapes facing in opposite directions.
  • the attachment/detachment guide 42 includes, for example, a plurality of rail portions 421 and 422 that protrude from the holder 3 and are provided so as to be slidably engaged with the engager 412 of the clip 41 in a detachable manner.
  • the engager 412 of the clip 41 is inserted into the gap between the rail parts 421 and 422, and the clip 41 with the insertion plate 2 attached is moved laterally along the rail part (groove) while maintaining a predetermined angle ⁇ . Slide linearly in direction X.
  • the attachment/detachment guide 42 includes, for example, rail portions 421 that protrude from the holder 3 in a substantially L-shape and are elongated in the lateral direction X. There is.
  • the attachment/detachment guide 42 is provided with an intermediate rail portion 422 that is disposed between the opposing engagers 412 of the clip 41 .
  • Guide tapers 423 are formed at both left and right ends of the rail portion 421 for smooth insertion of the engager 412 of the clip 41.
  • the attachment/detachment mechanism 4 is not limited to the structure described above.
  • the attachment/detachment mechanism 4 may have a configuration in which the base end portion of the insertion plate 2 is directly detachably engaged with the holder 3 without using the clip 41.
  • the attachment/detachment mechanism 4 may have a structure in which it slides in the vertical direction Z in addition to sliding in the lateral direction X, for example.
  • the photographing instruction module instructs the user terminal 100 to which the intraoral photographing aid 6 is attached to photograph an intraoral image (step S13).
  • the intraoral images include an upper jaw image that captures the upper jaw side of the user's oral cavity, and a lower jaw image that captures the lower jaw side.
  • the photographing instruction module transmits an instruction to insert the intraoral photographing aid 6 into the user's oral cavity and an instruction on the insertion method to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit. The user views this instruction and understands how to insert the intraoral photographing aid 6.
  • the user terminal 100 notifies the computer 10 that the method for inserting the intraoral photographing aid 6 has been determined.
  • the photographing module transmits an instruction to photograph each of the upper jaw image and the lower jaw image to the user terminal 100.
  • This photographing instruction includes a sample image serving as a sample of each image and an instruction on a method of photographing each image.
  • the user terminal 100 receives this photographing instruction and displays it on its own display unit. The user views the shooting instructions and understands the appropriate image condition.
  • the user uses the user terminal 100 to photograph one or more intraoral images that show the alignment of teeth similar to the sample image or the state of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the intraoral image capturing is completed.
  • the photographing instruction module instructs to photograph an intraoral image. This is a specific example after the user understands how to insert the intraoral photographing aid 6.
  • the photographing instruction module may or may not always display the method of inserting the intraoral photographing aid 6 together with the sample image. In this embodiment, a case will be described in which only a sample image is displayed.
  • the method of inserting the intraoral photographing aid 6 may be displayed in a smaller size than the sample image, which may hinder viewing of the sample image. It may be configured so that it does not appear.
  • the photographing instruction module first transmits an instruction to photograph an upper jaw image to the user terminal 100.
  • the user terminal 100 receives an instruction to capture an upper jaw image and displays it on its own display unit (see FIG. 15). Based on the displayed content, the user uses the user terminal 100 to take one or more upper jaw images that show the alignment of the teeth or the condition of the teeth and gums (see FIG. 16).
  • the user terminal 100 notifies the computer 10 that the capturing of the upper jaw image has been completed.
  • FIG. 15 This figure is a diagram schematically showing an example of an instruction to photograph an upper jaw image displayed by the user terminal 100.
  • an instruction 60 for photographing is shown.
  • This photographing instruction 60 includes a sample image 61 of an upper jaw image and a photographing method 62.
  • the sample image 61 is a sample image of the maxillary side of the oral cavity.
  • the photographing method 62 indicates a character string related to instructions for the photographing method, such as "Please photograph as far as the molars as possible.”
  • the user views the sample image 61 and the photographing method 62, and uses the user terminal 100 to photograph one or more upper jaw images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the capturing of the upper jaw image has been completed.
  • the photography instruction module transmits an instruction to take a lower jaw image to the user terminal 100.
  • the user terminal 100 receives the instruction to take a lower jaw image and displays it on its own display unit (see FIG. 17). Based on the displayed content, the user uses the user terminal 100 to take one or more images of the lower jaw showing the alignment of the teeth or the condition of the teeth and gums (see FIG. 16). The user terminal 100 notifies the computer 10 that the shooting of the lower jaw image has been completed.
  • FIG. 17 This figure is a diagram schematically showing an example of an instruction to photograph a lower jaw image displayed by the user terminal 100.
  • an instruction 63 for photographing is shown.
  • This photographing instruction 63 includes a sample image 64 of a lower jaw image and a photographing method 65.
  • the sample image 64 is a sample image of the lower jaw side inside the oral cavity.
  • the photographing method 65 is a character string related to instructions for the photographing method, such as "Please photograph as much of the molars as possible.”
  • the user views the sample image 64 and the photographing method 65, and uses the user terminal 100 to photograph one or more upper jaw images showing the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the shooting of the lower jaw image has been completed.
  • a method for a user to photograph an intraoral image using the user terminal 100 will be described.
  • the user attaches the user terminal 100 to the attachment part 31 of the holder 3.
  • the user terminal 100 is slid in the vertical direction Z to adjust the position where the camera 101 of the user terminal 100 can appropriately photograph the reflective surface 21 of the insertion plate 2.
  • the user slides the insertion plate 2 and the clip 41 with respect to the holder 3 in the horizontal direction Adjust to a position where you can take pictures appropriately. If the user has removed the predetermined mouth opener, the user attaches the predetermined mouth opener to the user's oral cavity again.
  • the user holds and operates the holder 3, aligns the tip of the insertion plate 2 with the oral cavity, and inserts the insertion plate 2 into the oral cavity.
  • the user starts the camera app on the user terminal 100 and photographs the inside of the oral cavity.
  • the insertion plate 2 is inserted into the oral cavity with the reflective surface 21 facing upward, and one or more images of the state of the upper jaw are taken with the insertion plate 2 pressed against the lower jaw and stabilized.
  • the user may directly check the image displayed on the screen of the user terminal 100, or reflect the screen of the user terminal 100 on a mirror such as a washstand, and check the captured screen while looking at the screen. While doing so, take one or more photos.
  • the user turned the entire intraoral photographing aid 6 upside down along with the user terminal 100, inserted it into the oral cavity with the reflective surface 21 of the insertion plate 2 facing downward, and pressed the insertion plate 2 against the upper jaw to stabilize it. In this state, perform the same operation as for the upper jaw image to take one or more images of the lower jaw state.
  • the photographing instruction module may transmit sample images of the upper jaw image and the lower jaw image and the photographing instruction together to the user terminal 100.
  • the user terminal 100 displays these sample images and shooting instructions together.
  • the user uses the user terminal 100 to photograph an upper jaw image and a lower jaw image that show the alignment of the teeth or the condition of the teeth and gums.
  • the user terminal 100 notifies the computer 10 that the capturing of the upper jaw image and the lower jaw image, which show the alignment of the teeth or the condition of the teeth and gums, has been completed.
  • the shooting of each image is completed, the user terminal 100 does not notify the computer 10, but acquires the sample images of the upper jaw image and the lower jaw image together with the shooting instructions in advance.
  • the next sample image and photographing instructions may be displayed.
  • the order in which the upper jaw images and lower jaw images are taken is not limited to the example described above, and can be changed as appropriate.
  • the photographing instruction module completes the instruction to photograph an intraoral image that shows the alignment of teeth or the condition of teeth and gums (when the computer 10 is notified that the photographing of an intraoral image has been completed)
  • the computer 10 performs step S14. Execute the process.
  • the image acquisition module acquires a front image, a right side image, a left side image, and an intraoral image photographed by the user (step S14).
  • the image acquisition module requests the user terminal 100 to transmit the front image, right side image, left side image, and intraoral image taken by the user. Based on this request, the user terminal 100 transmits the photographed frontal image, right side image, left side image, and intraoral image to the computer 10.
  • the image acquisition module receives the front image, right side image, left side image, and intraoral image, and acquires the front image, right side image, left side image, and intraoral image photographed by the user.
  • the intraoral imaging support system 1 uses the front image, right side image, left side image, and intraoral image acquired by the main image acquisition process to execute the process described below.
  • This figure is a flowchart of the reshooting instruction process executed by the computer 10.
  • This re-imaging instruction process includes a condition determination process (step S5) for determining whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions. If it is determined that the image does not meet the predetermined conditions, the user is instructed to retake the image that is determined not to meet the predetermined conditions, among the frontal image, right side image, left side image, and intraoral image. This is the details of the process (step S6).
  • This reshooting instruction process is a process that uses the front image, right side image, left side image, and intraoral image acquired by the image acquisition process described above.
  • the analysis module analyzes the acquired front image, right side image, left side image, and intraoral image (step S20).
  • the analysis module performs image analysis on each of the acquired front image, right side image, left side image, and intraoral image (upper jaw image and lower jaw image).
  • the analysis module extracts feature points and feature amounts from each image, and detects teeth and gums appearing in each image. At this time, the analysis module detects the tooth and gum regions as well as the teeth and gums themselves.
  • the recognition module recognizes the teeth shown in the acquired frontal image, right side image, left side image, and intraoral image based on the analysis results (step S21).
  • the recognition module separates the teeth in each image one by one based on the analysis results of each image, and recognizes each tooth.
  • the recognition module also recognizes the name of each tooth based on the shape and positional relationship of each tooth.
  • the condition determination module determines whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions (step S22).
  • the predetermined condition is, for example, that all of the user's teeth are in alignment.
  • This predetermined condition may be set in advance, or may be set when each image is acquired or when a request to start photographing an image is acquired.
  • the present embodiment will be described on the assumption that the predetermined condition is that all of the user's teeth are in alignment. Note that in consideration of people who have lost their teeth, the predetermined condition may be set as whether or not all the places where teeth would have grown are shown. In this case, instead of determining whether all teeth are present, it is sufficient to determine whether all the places where teeth would have grown are shown.
  • the condition determination module determines whether all of the user's teeth are aligned in each image based on the user's teeth in the recognition result. Note that the computer 10 uses images for which the condition determination module determines that the predetermined conditions are not satisfied for intraoral determination processing, which will be described later.
  • the computer 10 executes the predetermined process (step S23).
  • the predetermined processing includes, for example, a transmission process in which each acquired image is sent to a medical facility, etc., a medical examination process in which an examination is performed using artificial intelligence, etc. using each acquired image, and this processing is performed based on the user's current location, address, etc. This is an introduction process that introduces medical facilities etc. near the user.
  • the computer 10 ends the main re-imaging instruction process. Note that the computer 10 may terminate the main re-imaging instruction process without executing the predetermined process.
  • the predetermined process is not limited to the example described above, and may be any process that executes necessary processes as appropriate.
  • the condition determination module determines that the acquired front image, right side image, left side image, and intraoral image do not satisfy the predetermined conditions (step S22 NO)
  • the specific module determines that the acquired front image, right side image, left side image, and intraoral image do not satisfy the predetermined conditions.
  • a photographing method that satisfies a predetermined condition is specified for the image that is determined not to exist (step S24).
  • the specifying module specifies a photographing method necessary for satisfying the predetermined condition for an image determined not to satisfy the predetermined condition.
  • the specific module identifies locations that are the basis for determining that predetermined conditions are not met, such as locations where teeth are not visible in the image, locations where teeth are missing, and locations where gums are not visible.
  • the identification module identifies locations where there are teeth that could not be recognized, locations where teeth were recognized but the overall image of the teeth could not be recognized, locations where the entire image of the gums could not be recognized, etc. Identify.
  • the identification module identifies an imaging method for photographing this tooth based on the location of this tooth.
  • the specifying module specifies a photographing method for making an image that does not meet a predetermined condition closer to a sample image. For example, when some teeth are not shown, the identification module identifies a method of photographing an image that shows the teeth.
  • the photographing method is to instruct the photographing location, for example, "Please photograph the upper side,”"Please photograph the lower side,” or "Please photograph the back side.”
  • the re-imaging instruction module instructs the user to re-imaging the image that is determined not to satisfy a predetermined condition among the frontal image, right side image, left side image, and intraoral image (step S25).
  • the re-photographing instruction module notifies the user terminal 100 that it is necessary to re-photograph an image determined not to satisfy a predetermined condition.
  • the user terminal 100 receives this notification and displays it on its own display unit. The user views this notification and learns which images require re-shooting.
  • the user terminal 100 receives an input from the user to retake an image that has been determined not to satisfy a predetermined condition, and transmits the input content to the computer 10.
  • the re-photographing instruction module acquires this input content and transmits to the user terminal 100 an instruction to re-photograph the image determined not to satisfy a predetermined condition. At this time, the reshooting instruction module transmits to the user terminal 100 different instruction contents for the front image, right side image, left side image, and intraoral image.
  • the reason why the instructions are different is that the intraoral imaging aid 6 does not need to be used to capture the front image, right side image, and left side image, whereas the intraoral imaging aid 6 is used to capture the intraoral image. This is because it is necessary.
  • the reshooting instruction module transmits an instruction to wear a predetermined mouse opener and an instruction on how to wear it to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit. Based on this instruction, the user attaches a predetermined mouth opener to his or her oral cavity.
  • the user terminal 100 notifies the computer 10 that the attachment of the mouse opener has been completed. Note that if the user has already attached the mouse opener before the user terminal 100 receives this instruction, the user terminal 100, upon receiving this instruction, notifies the computer 10 that the mouse opener has been attached. All you have to do is notify us.
  • the re-photographing instruction module transmits to the user terminal 100 an instruction to re-photograph the image determined not to satisfy the predetermined conditions.
  • This re-photographing instruction includes an image determined not to satisfy a predetermined condition, and an instruction for the location and photographing method determined to not satisfy the predetermined condition specified by the process of step S24 described above.
  • the user terminal 100 receives this re-photography instruction and displays it on its own display unit (see FIG. 19).
  • the user views this re-photographing instruction and understands the image that has been determined not to meet the predetermined conditions and the appropriate method for photographing the image.
  • the user uses the user terminal 100 to take one image that has been determined not to meet the predetermined conditions so that the alignment of the teeth or the condition of the teeth and gums can be seen. Or retake multiple shots.
  • FIG. 19 This figure is a diagram schematically showing an example of an instruction for reshooting displayed by the user terminal 100.
  • an instruction 70 for re-imaging is shown.
  • This re-photographing instruction 70 includes an image 71 that has been determined not to meet predetermined conditions, a photographing method 72, and an instruction icon 73.
  • the image 71 is an image determined not to satisfy a predetermined condition, and in this embodiment, the right side image is shown.
  • the photographing method 72 is a specified photographing method, and in this embodiment, it indicates "Please photograph the back side a little so that the back teeth can be seen.”
  • the instruction icon 73 indicates a location where it is determined that a predetermined condition is not met.
  • the user views the re-photographing instructions 70 and, based on the photographing method 72 and the instruction icon 73, selects images that are determined by the user terminal 100 to not meet the predetermined conditions by adjusting the teeth alignment or One or more photos will be taken to show the condition of your teeth and gums.
  • the reshooting instruction module transmits an instruction to wear a predetermined mouse opener and an instruction on how to wear it to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit. Based on this instruction, the user attaches a predetermined mouth opener to his or her oral cavity.
  • the user terminal 100 notifies the computer 10 that the attachment of the mouse opener has been completed. Note that if the user has already attached the mouse opener before the user terminal 100 receives this instruction, the user terminal 100, upon receiving this instruction, notifies the computer 10 that the mouse opener has been attached. All you have to do is notify us.
  • the re-imaging instruction module transmits to the user terminal 100 an instruction to attach the intraoral imaging aid 6 to the user terminal 100 and an instruction on the attachment method.
  • User terminal 100 receives this instruction and displays it on its own display unit.
  • the user attaches the intraoral photographing aid 6 to the user terminal 100 based on this instruction.
  • the user terminal 100 notifies the computer 10 that the installation of the intraoral photographing aid 6 has been completed. Note that if the intraoral photography aid 6 has already been attached to the user terminal 100 before the user terminal 100 receives this instruction, the user terminal 100 will install the intraoral photography aid 6 when receiving this instruction. What is necessary is to notify the computer 10 that the installation is completed.
  • the re-imaging instruction module transmits an instruction to insert the intraoral imaging aid 6 into the user's oral cavity and an instruction on the insertion method to the user terminal 100.
  • User terminal 100 receives this instruction and displays it on its own display unit. Based on this instruction, the user understands how to insert the intraoral photographing aid 6.
  • the user terminal 100 notifies the computer 10 that the method for inserting the intraoral photographing aid 6 has been determined. Note that if the user terminal 100 has already understood how to insert the intraoral photographing aid 6 before receiving this instruction, the user terminal 100 will insert the intraoral photographing aid 6 when receiving this instruction. It is sufficient to notify the computer 10 that the understanding of the insertion method has been completed.
  • the re-photographing instruction module transmits to the user terminal 100 an instruction to re-photograph the image determined not to satisfy the predetermined conditions.
  • This re-photographing instruction includes the image determined not to satisfy the predetermined conditions, and instructions for the location and photographing method specified by the process of step S24 described above.
  • the user terminal 100 receives this reshooting instruction and displays it on its own display unit. The user views this re-photographing instruction and understands the image that has been determined not to meet the predetermined conditions and the appropriate method for photographing the image.
  • the user uses the user terminal 100 to take one image that has been determined not to meet the predetermined conditions so that the alignment of the teeth or the condition of the teeth and gums can be seen. Or retake multiple shots.
  • the user terminal 100 completes the re-capturing of the image determined not to satisfy the predetermined conditions, the user terminal 100 notifies the computer 10 that the re-capturing of the image determined not to satisfy the predetermined condition is completed. do.
  • the computer 10 executes the process of step S26 when the re-photographing instruction module completes the instruction to re-photograph the image that has been determined not to satisfy the predetermined conditions (when the computer 10 is notified that the re-photographing of the image has been completed). .
  • the image acquisition module acquires the re-photographed image (step S26).
  • the image acquisition module requests the user terminal 100 to transmit the re-photographed image. Based on this request, the user terminal 100 transmits the re-photographed image to the computer 10.
  • the image acquisition module receives this re-photographed image and acquires the re-photographed image.
  • the computer 10 performs an analysis process on the acquired re-photographed image (processing similar to step S20), a tooth recognition process based on the analysis result (processing similar to step S21), and a process similar to the acquired re-photographing process.
  • a process for determining whether the photographed image satisfies a predetermined condition is executed, and these processes are repeated until all images satisfy the predetermined condition. Note that, in FIG. 18, for the purpose of simplifying the description, it is assumed that the computer 10 finishes the main re-shooting instruction process after executing the process of step S26.
  • the above is the reshooting instruction process.
  • the intraoral photography support system 1 executes the process described below using the image determined to not satisfy the predetermined conditions through this re-photography instruction process.
  • FIG. 20 This figure is a flowchart of the intraoral determination process executed by the computer 10.
  • This intra-oral determination process is a process that uses an image that was determined not to satisfy a predetermined condition in the above-described re-imaging instruction process. Further, the present intraoral determination process may be executed simultaneously with the processes after step S24 described above, or may be executed at a different timing from the processes after step S24.
  • the analysis module analyzes the image determined not to satisfy the predetermined condition (step S30).
  • the analysis module performs image analysis on the image determined to not satisfy the predetermined condition in the process of step S22 described above.
  • the analysis module extracts feature points and feature amounts from this image, and detects teeth, gums, tongue, and lips appearing in this image.
  • the analysis module then detects areas of the teeth, gums, tongue and lips in addition to the teeth, gums, tongue and lips themselves.
  • the recognition module Based on the analysis result, the recognition module recognizes the state of the oral cavity depicted in the image determined not to satisfy a predetermined condition (step S31).
  • the condition in the oral cavity is the condition of teeth, gums, tongue, and lips.
  • the recognition module separates the teeth in the image one by one based on the analysis results, recognizes each tooth, and also recognizes the name of each tooth based on the shape and positional relationship of each tooth. .
  • the recognition module also recognizes the gums, tongue, and lips based on the analysis results.
  • the recognition module recognizes the recognized conditions of teeth, gums, tongue and lips.
  • the recognition module detects the condition of teeth and gums, including the presence of cavities, acid erosion, periodontal disease, unpolished areas, staining, whitening progress, alveolar bone abnormalities, and treatment marks. Recognize the growth status of teeth, the status of implants or bridges, and the presence or absence of stomatitis. Furthermore, the recognition module recognizes the presence or absence of stomatitis and the presence or absence of redness as the condition of the tongue. Furthermore, the recognition module recognizes the presence or absence of stomatitis and the presence or absence of lip abnormality as the condition of the lips.
  • the recognition module can detect tooth decay, acid erosion, periodontal disease, unpolished teeth, discoloration, degree of whitening progress, alveolar bone abnormalities, growth status of treatment scars, implant or bridge status, stomatitis, tongue redness, and lips. It is sufficient if it recognizes at least one of the abnormalities.
  • steps S30 and S31 described above may be executed during the processing in steps S20 and S21 described above.
  • the processes in steps S30 and S31 may be omitted, and the results of the processes in steps S20 and S21 for images that do not meet the predetermined conditions may be used in the process described later.
  • the intraoral condition determination module determines whether there is an abnormality in the intraoral condition (step S32).
  • the oral condition judgment module detects tooth decay, acid erosion, periodontal disease, unpolished teeth, discoloration, degree of whitening progress, alveolar bone abnormality, Determine at least one of the following: growth of treatment scars, condition of implants or bridges, stomatitis, redness of the tongue, and abnormalities of the lips. Based on the recognition results of the condition of teeth, gums, tongue, and lips, the oral condition judgment module determines the presence of cavities, acid erosion, periodontal disease, unpolished areas, staining, and whitening.
  • the intraoral state determination module may be configured to determine a preset condition, or may be configured to determine whether at least one of these conditions exists.
  • the computer 10 ends the intraoral determination process.
  • the notification module notifies the user of the intraoral condition (step S33).
  • the notification module notifies the user terminal 100 of an image determined to have an abnormality (an image determined not to satisfy a predetermined condition) and the details of the abnormality.
  • the user terminal 100 receives this notification and displays it on its own display unit (see FIG. 21).
  • the user views this notification and understands the image determined to have an abnormality and the details of the abnormality.
  • the user understands the location of the abnormality in his or her oral cavity and the details of the abnormality.
  • the notification module notifies the user of the state of the oral cavity by causing the user terminal 100 to display an image determined to have an abnormality and the details of the abnormality.
  • FIG. 21 is a diagram schematically showing an example of a notification displayed by the user terminal 100.
  • a notification 80 is shown.
  • This notification 80 shows an image 81 that has been determined to have an abnormality, details of the abnormality 82, and an instruction icon 83.
  • the image 81 determined to have an abnormality is an image determined to have an abnormality, and in this embodiment, the right side image is shown.
  • the content of the abnormality 82 is the content of the determined abnormality, and in this embodiment, it indicates "There was some unpolished parts.”
  • the instruction icon 83 indicates a location where it is determined that there is an abnormality. The user views this notification and understands the location of the abnormality and the details of the abnormality in the state of the user's oral cavity.
  • the computer 10 can also be configured to specify a countermeasure method for an abnormality in the oral cavity, and to notify the user of the countermeasure method when notifying the user of the intraoral condition.
  • the dentition determination process executed by the computer 10 will be explained based on FIG. 22.
  • This figure is a flowchart of the dentition determination process executed by the computer 10.
  • This dentition determination process is a process that uses the front image, right side image, left side image, and intraoral image acquired in the image acquisition process described above. Furthermore, this dentition judgment process may be executed simultaneously with the above-described re-imaging instruction process, or may be executed at a different timing from the above-described re-imaging instruction process.
  • the analysis module analyzes the acquired frontal image, right side image, left side image, and intraoral image (step S40).
  • the analysis module performs image analysis on the frontal image, right side image, left side image, and intraoral image acquired through the process of step S14 described above.
  • the analysis module extracts feature points and feature amounts from these images, and detects teeth and gums appearing in these images.
  • the analysis module then detects the teeth and gum regions in addition to the teeth and gums themselves.
  • the recognition module recognizes the state of the dentition shown in the acquired frontal image, right side image, left side image, and intraoral image based on the analysis result (step S41).
  • the condition of the dentition is the condition of the teeth and gums.
  • the recognition module separates the teeth in the image one by one based on the analysis results, recognizes each tooth, and also recognizes the name of each tooth based on the shape and positional relationship of each tooth. .
  • the recognition module also recognizes gums based on the analysis results.
  • the recognition module recognizes the recognized tooth and gum conditions.
  • the recognition module recognizes the condition of the teeth and gums, including the size of the dentition and the presence or absence of sufficient alveolar bone space to move the teeth. Note that the recognition module may be one that recognizes at least one of the size of the dentition and the presence or absence of sufficient alveolar bone space for moving the teeth.
  • steps S40 and S41 described above may be executed during the processing in steps S20 and S21 described above.
  • the processes in steps S40 and S41 may be omitted, and the results of the processes in steps S20 and S21 on the acquired frontal image, right side image, left side image, and intraoral image may be used in the process described later.
  • the dentition determination module determines whether there is an abnormality in the state of the dentition (step S42).
  • the dentition determination module determines at least one of the size of the dentition and the presence or absence of sufficient alveolar bone space for moving the teeth, based on the recognition results for the acquired frontal image, right side image, left side image, and intraoral image. to judge.
  • the dentition determination module may be configured to determine a preset condition, or may be configured to determine whether at least one of these conditions exists.
  • step S42 determines that there is no abnormality in the state of the dentition. If the dentition determination module determines that there is no abnormality in the state of the dentition (step S42: NO), the computer 10 ends this dentition determination process.
  • the notification module notifies the user of the state of the dentition (step S43).
  • the notification module notifies the user terminal 100 of the image determined to be abnormal and the details of the abnormality.
  • the user terminal 100 receives this notification and displays it on its own display unit.
  • the user views this notification and understands the image determined to have an abnormality and the details of the abnormality.
  • the notification module notifies the user of the state of the dentition by causing the user terminal 100 to display an image determined to have an abnormality and details of the abnormality.
  • the computer 10 executes the process in this dentition judgment process using the front image, right side image, left side image, and intraoral image acquired in the process of step S14 described above. Similar to the determination process, the process may be performed using an image determined not to satisfy a predetermined condition in the process of step S22 described above. In this case, the computer 10 executes the process using the image determined not to satisfy the predetermined condition in the process of steps S40 and S41 described above. In addition, instead of executing steps S40 and S41, the computer 10 executes processes similar to those of steps S40 and S41 in the processes of steps S30 and S31 described above, and uses the results of these processes, The process of step S42 may also be executed.
  • the trained model generation process executed by the computer 10 will be described based on FIG. 23.
  • This figure is a flowchart of learned model generation processing executed by the computer 10.
  • This trained model generation process is a process that is performed before the above-described re-shooting instruction process, and is a process for generating a trained model to be used in the above-mentioned re-shooting instruction process.
  • the learning data acquisition module acquires learning data (step S50).
  • the learning data includes, for example, images in which all teeth are aligned, such as frontal images, right side images, left images, and intraoral images, and front images in which all teeth are not aligned. These are images such as an image, a right side image, a left image, and an intraoral image in which all teeth are not aligned.
  • the learning data includes, for example, all the places where teeth would have grown, such as frontal images, right side images, left images, and intraoral images that show all the places where teeth would have grown.
  • the learning data acquisition module acquires learning data from a predetermined terminal device or the like. Note that the method by which the learning data acquisition module acquires the learning data is not particularly limited.
  • the creation module creates a learning data set based on the acquired learning data (step S51).
  • the creation module uses each acquired image in which all teeth are aligned and each image in which all teeth are not aligned as a learning data set to determine whether all teeth are aligned.
  • the creation module uses each image that shows all the acquired places where teeth would have grown and the places where teeth would have grown as a learning dataset to determine whether or not teeth would have grown.
  • a learning data set is created by adding training data indicating whether or not teeth should have grown to each image that does not show all of the teeth.
  • These training data may accept input of pass/fail for each image using a predetermined method, may be prepared separately from each image, or may be other data. It may be. Further, these teacher data may be acquired together with the learning data in the process of step S50 described above.
  • the learning module learns the created learning data set (step S52).
  • Learning methods include machine learning using supervised learning, unsupervised learning, reinforcement learning, etc., deep learning using convolutional neural networks, recurrent neural networks, long/short-term memory, etc. In this modification, machine learning using supervised learning will be explained as an example.
  • the learning module performs machine learning by supervised learning using the training data in each created learning data set.
  • the generation module generates a learned model based on the learning results (step S53).
  • the generation module generates a trained model based on the learning results using a predetermined algorithm.
  • the generation module generates a trained model for determining whether all teeth are present and a trained model for determining whether teeth would have erupted. Note that the generation method by which the generation module generates the trained model is not particularly limited.
  • the above is the learned model generation process.
  • the computer 10 executes the process of step S22 described above using the trained model generated by this trained model generation process. That is, the condition determination module executes a process of determining whether a predetermined condition is satisfied using the acquired frontal image, right side image, left side image, and intraoral image, and the generated trained model. do. This case will be explained.
  • the computer 10 calculates the front image, right side image, left image, and performs analysis of intraoral images and recognition of teeth in the images.
  • the front image, right image, and right image are similarly determined based on the acquired front image, right side image, left image, and intraoral image, and the generated trained model.
  • the left side image, and the intraoral image, and the teeth in the images may be recognized.
  • the condition determination module determines whether the acquired frontal image, right side image, left side image, and intraoral image satisfy predetermined conditions based on the analysis results and recognition results. do.
  • the computer 10 can also be configured to update the generated trained model based on the analysis results and recognition results. When the computer 10 executes the processes of steps S20 and S21 described above from next time onwards, it executes the processes using the updated learned model.
  • the computer 10 can also be configured to execute a part or all of the above-mentioned processes in combination. Further, the computer 10 can also be configured to execute each process at a timing other than the timing described above.
  • the means and functions described above are realized by a computer (including a CPU, an information processing device, and various terminals) reading and executing a predetermined program.
  • the program may be provided from a computer via a network (SaaS: Software as a Service) or in a cloud service.
  • the program may be provided in a form recorded on a computer-readable recording medium.
  • the computer reads the program from the recording medium, transfers it to an internal recording device or an external recording device, records it, and executes it.
  • the program may be recorded in advance on a recording device (recording medium) and provided to the computer from the recording device via a communication line.
  • An intraoral photography support system that supports taking images for examining the user's oral cavity, A guide section (for example, a guide section) that guides the user to take a front image, a right side image, and a left side image showing the bite of the teeth or the condition of the teeth and gums while the user is wearing a predetermined mouth opener.
  • a guide section for example, a guide section
  • an attachment instruction unit for example, an attachment instruction unit 12, an attachment instruction module
  • a photography instruction unit for example, photography instruction unit 13, photography instruction module
  • an acquisition unit for example, acquisition unit 14, image acquisition module
  • a condition determining unit e.g., condition determining unit 16, condition determining module
  • the condition determining unit analyzes the acquired front image, right side image, left side image, and intraoral image to determine whether a predetermined condition is satisfied.
  • the intraoral imaging support system according to (1).
  • the condition determination unit determines whether a predetermined condition is satisfied using the acquired frontal image, right side image, left side image, and intraoral image, and the learned model.
  • the intraoral imaging support system according to (1).
  • a creation unit that creates a learning dataset
  • a generation unit that learns the created training data set and generates a trained model
  • Equipped with The condition determination unit determines whether a predetermined condition is satisfied using the acquired frontal image, right side image, left side image, and intraoral image, and the generated trained model.
  • the predetermined condition is that all teeth of the user are in alignment;
  • the predetermined condition is whether all the places where teeth would have grown are shown.
  • an intraoral condition determination unit e.g., an intraoral condition determination module that determines at least one of alveolar bone abnormalities, treatment scar growth conditions, implant or bridge conditions, stomatitis, tongue redness, and lip abnormalities;
  • a dentition determination unit (for example, a dentition determination module) that determines at least one of the size of the user's dentition and the presence or absence of sufficient alveolar bone space for moving the teeth, based on the acquired image. )and,
  • An intraoral photography support method executed by a computer that supports photography of images for examining the user's oral cavity comprising: A step of guiding the user to take a front image, a right side image, and a left side image showing the bite of the teeth or the condition of the teeth and gums while wearing a predetermined mouse opener (for example, step S11 )and, a step of instructing the user to attach a predetermined intraoral photographing aid to the terminal (for example, step S12); a step of instructing a user to photograph an intraoral image with a terminal of the user to which the intraoral photographing aid is attached (for example, step S13); a step of acquiring a front image, a right side image, a left side image, and an intraoral image taken by the user (for example, step S14); a step of determining whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions (for example, step S22); If it is determined that the predetermined condition is not met, re-photograph
  • a computer that supports taking images for examining the user's oral cavity, A step of guiding the user to take a front image, a right side image, and a left side image showing the bite of the teeth or the condition of the teeth and gums while wearing a predetermined mouse opener (for example, step S11 ), a step of instructing the user to attach a predetermined intraoral photographing aid to the terminal (for example, step S12); a step of instructing a user to photograph an intraoral image with a terminal of the user to which the intraoral photographing aid is attached (for example, step S13); acquiring a front image, a right side image, a left side image, and an intraoral image taken by the user (for example, step S14); a step of determining whether the acquired front image, right side image, left side image, and intraoral image satisfy predetermined conditions (for example, step S22); If it is determined that the predetermined condition is not met, re-photograph the image that is determined to not satisfy the predetermined condition among the frontal image, right
  • Intraoral photography support system Insertion plate 21 Reflection surface 3 Holder 32 Handle portion 4 Attachment/detaching mechanism 41 Clip 411 Insertion hole 412 Engagement element 42 Attachment/detaching guide 421, 422 Rail portion 423 Guide taper 5 Network 6 Intraoral photography aid 10 Computer 11 Guide section 12 Installation instruction section 13 Photography instruction section 14 Acquisition section 15 Re-photography instruction section 16 Condition judgment section 100 User terminal 101 Camera 50, 53, 56, 60, 63 Photography instruction 51, 54, 57, 61, 64 Sample Images 52, 55, 58, 62, 65 Photographing method 70 Re-capture instruction 71 Image determined not to meet predetermined conditions 72 Photography method 73 Instruction icon 80 Notification 81 Image determined to have an abnormality 82 Abnormal Content 83 Instruction icon

Abstract

[Problème] Aider à l'imagerie, lors de l'imagerie de l'intérieur de la cavité buccale d'un patient à l'aide d'une application de caméra d'un terminal mobile, afin de permettre à l'imagerie d'être facilement réalisée. [Solution] Système d'aide à l'imagerie intrabuccale destiné à aider à la capture d'une image pour examiner l'intérieur de la cavité buccale d'un utilisateur, le système d'aide à l'imagerie intrabuccale : réalisant un guidage de façon à capturer une image avant, une image côté droit et une image côté gauche indiquant une occlusion dentaire ou l'état de dents et de gencives dans un état dans lequel l'utilisateur porte un ouvre-bouche prescrit; émettant une instruction pour fixer un dispositif d'aide à l'imagerie intrabuccale prescrit à un terminal utilisateur; émettant une instruction pour capturer une image intrabuccale à l'aide du terminal utilisateur auquel le dispositif d'aide à l'imagerie intrabuccale est fixé; acquérant une image avant, une image côté droit, une image côté gauche et une image intrabuccale capturées par l'utilisateur; déterminant si l'image avant, l'image côté droit, l'image côté gauche et l'image intrabuccale acquises satisfont à une condition prescrite; et, lorsqu'il est déterminé que la condition prescrite n'est pas satisfaite, ordonnant à l'utilisateur de recapturer l'image, parmi l'image avant, l'image côté droit, l'image côté gauche et l'image intrabuccale, qui a été déterminée comme ne satisfaisant pas à la condition prescrite.
PCT/JP2022/026058 2022-06-29 2022-06-29 Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme WO2024004100A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/026058 WO2024004100A1 (fr) 2022-06-29 2022-06-29 Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/026058 WO2024004100A1 (fr) 2022-06-29 2022-06-29 Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme

Publications (1)

Publication Number Publication Date
WO2024004100A1 true WO2024004100A1 (fr) 2024-01-04

Family

ID=89382275

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/026058 WO2024004100A1 (fr) 2022-06-29 2022-06-29 Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme

Country Status (1)

Country Link
WO (1) WO2024004100A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6806400B1 (ja) * 2020-10-26 2021-01-06 株式会社スクリエ 口腔内撮影補助具
US20210068745A1 (en) * 2019-09-06 2021-03-11 SmileDirectClub LLC Systems and methods for user monitoring
JP2021053174A (ja) * 2019-09-30 2021-04-08 株式会社Ｎｔｔドコモ 携帯端末および撮影補助プログラム
WO2021064112A1 (fr) * 2019-10-04 2021-04-08 Adent Aps Procédé d'obtention d'images pour évaluation de la santé buccodentaire à l'aide d'un appareil photographique de dispositif mobile et d'un miroir
JP2021053175A (ja) * 2019-09-30 2021-04-08 株式会社Ｎｔｔドコモ 携帯端末、情報処理装置および口腔状態判断プログラム
WO2022085724A1 (fr) * 2020-10-22 2022-04-28 サンスター株式会社 Système d'évaluation d'état buccal, système de recommandation de soins buccaux, et système de notification d'état buccal

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210068745A1 (en) * 2019-09-06 2021-03-11 SmileDirectClub LLC Systems and methods for user monitoring
JP2021053174A (ja) * 2019-09-30 2021-04-08 株式会社Ｎｔｔドコモ 携帯端末および撮影補助プログラム
JP2021053175A (ja) * 2019-09-30 2021-04-08 株式会社Ｎｔｔドコモ 携帯端末、情報処理装置および口腔状態判断プログラム
WO2021064112A1 (fr) * 2019-10-04 2021-04-08 Adent Aps Procédé d'obtention d'images pour évaluation de la santé buccodentaire à l'aide d'un appareil photographique de dispositif mobile et d'un miroir
WO2022085724A1 (fr) * 2020-10-22 2022-04-28 サンスター株式会社 Système d'évaluation d'état buccal, système de recommandation de soins buccaux, et système de notification d'état buccal
JP6806400B1 (ja) * 2020-10-26 2021-01-06 株式会社スクリエ 口腔内撮影補助具

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KATAYAMA YOUHEI: "Proposal for an Oral Cavity Image Capture Support System", IPSJ SYMPOSIUM: PROCEEDINGS OF THE 28TH WORKSHOP ON MULTIMEDIA COMMUNICATION AND DISTRIBUTED PROCESSING, 4 November 2020 (2020-11-04), pages 253 - 256, XP093123941 *

Similar Documents

Publication Publication Date Title
US11737853B2 (en) Methods and apparatuses for dental images
EP3297517B1 (fr) Systèmes et procédés pour imagerie de la gorge
US20060001740A1 (en) Apparatus for dental diagnosis and treatment
JP6806400B1 (ja) 口腔内撮影補助具
CN109829915A (zh) 基于三维空间面型的正畸专用微笑美学评测方法及系统
CN204427992U (zh) 牙医手术用戴式可视结构
JPH07275202A (ja) 歯科治療ユニット
WO2024004100A1 (fr) Système d'aide à l'imagerie intrabuccale, procédé d'aide à l'imagerie intrabuccale et programme
WO2022009285A1 (fr) Dispositif de traitement, programme de traitement, procédé de traitement, et système de traitement
JP7289582B1 (ja) 口腔内撮影補助具
Yılmaz et al. Recent photography trends in orthodontics
JP7365722B2 (ja) 規格撮影支援装置
US20230386682A1 (en) Systems and methods to chronologically image orthodontic treatment progress
JP2024004880A (ja) 撮影支援装置、撮像装置、撮像装置の制御方法
JP2023004133A (ja) 口角鈎器
EP4362787A1 (fr) Procédé et système pour effectuer un balayage dentaire
KR20190027510A (ko) 치과용 서포트 블록 및 이를 포함하는 영상 촬영 장치
JP2002238927A (ja) 口腔内歯列正画像入力装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22948197

Country of ref document: EP

Kind code of ref document: A1