JP2022127947A - Intraoral camera system and tooth column model generating method - Google Patents

Abstract

To improve accuracy of information indicative of the state of a tooth column.SOLUTION: An intraoral camera system comprises: an intraoral camera 10 which generates image data by manually photographing intraoral teeth of a user; and a model generation unit 107 which generates a tooth column model of the user by pasting the image data to a basic model 108 which is a three-dimensional model having a U-shape in a top view. For example, the basic model 108 may include an upper jaw basic model corresponding to the upper jaw and a lower jaw basic model corresponding to the lower jaw.SELECTED DRAWING: Figure 14

Description

The present disclosure relates to an intraoral camera system and a tooth row model generation method.

Patent Document 1 discloses a method for generating a panoramic image of a row of teeth as information indicating the state of the row of teeth in the oral cavity.

WO2016/148297

In such an intraoral camera system or the like, it is desired to be able to improve the accuracy of information indicating the state of the tooth row.

Therefore, an object of the present disclosure is to provide an intraoral camera system and a tooth row model generation method that can improve the accuracy of information indicating the state of the tooth row.

An intraoral camera system according to an aspect of the present disclosure includes an imaging unit that generates image data by manually imaging teeth in the oral cavity of a user, and a basic model that is a three-dimensional model having a U shape in top view and a model generation unit that generates a tooth row model of the user by pasting the image data.

The present disclosure can provide an intraoral camera system and a dentition model generation method that can improve the accuracy of information indicating the state of dentition.

FIG. 1 is a perspective view of an intraoral camera in an intraoral camera system according to an embodiment. FIG. 2 is a schematic configuration diagram of an intraoral camera system according to an embodiment. FIG. 3 is a diagram showing the flow of intraoral imaging operation in the intraoral camera system according to the embodiment. FIG. 4 is a diagram showing teeth in the oral cavity according to the embodiment. FIG. 5 is a functional block diagram of the mobile terminal according to the embodiment. FIG. 6 is a diagram showing an example of an intraoral region according to the embodiment. FIG. 7 is a diagram illustrating an example of classification of reference data according to the embodiment. FIG. 8 is a diagram showing an example of reference data according to the embodiment. FIG. 9 is a flowchart of type identification processing according to the embodiment. FIG. 10 is a diagram showing an example of a tooth image according to the embodiment. FIG. 11 is a flowchart illustrating another example of type identification processing according to the embodiment. FIG. 12 is a diagram showing the intraoral state of a user with missing teeth according to the embodiment. FIG. 13 is a flowchart of tooth row model generation processing according to the embodiment. FIG. 14 is a perspective view showing the configuration of the basic model according to the embodiment. FIG. 15 is a top view of the basic model according to the embodiment. FIG. 16 is a perspective view showing the configuration of blocks according to the embodiment. FIG. 17 is a diagram illustrating an example of stitching processing according to the embodiment. FIG. 18 is a perspective view showing the configuration of the basic model according to the embodiment. 19 is a perspective view of a block according to the embodiment; FIG. FIG. 20 is a diagram showing the attachment positions of the second molar and the first molar on the right side of the lower jaw on the crown surface of the basic model in the stitching process according to the embodiment. FIG. 21A is a diagram showing an example of tooth images used for stitching processing according to the embodiment. FIG. 21B is a diagram showing an example of tooth images used for stitching processing according to the embodiment. FIG. 22A is a diagram showing an example of processing for attaching the second molar on the right side of the mandible to the crown side surface of the basic model in the stitching processing according to the embodiment. FIG. 22B is a diagram showing an example of processing for attaching the second molar on the right side of the mandible to the crown side surface of the basic model in the stitching processing according to the embodiment. FIG. 23A is a diagram showing an example of processing for attaching the first molar on the right side of the mandible to the crown side surface of the basic model in the stitching processing according to the embodiment. FIG. 23B is a diagram showing an example of processing for attaching the first molar on the right side of the mandible to the crown side surface of the basic model in the stitching processing according to the embodiment. FIG. 23C is a diagram showing an example of processing for attaching the first molar on the right side of the mandible to the crown side surface of the basic model in the stitching processing according to the embodiment. FIG. 24A shows an example of processing for pasting the second molar and the first molar on the right side of the mandible to the crown side surface of the basic model using the boundary image between two teeth in the stitching processing according to the embodiment. FIG. 4 is a diagram showing; FIG. 24B shows an example of processing for pasting the second molar and the first molar on the right side of the mandible to the crown side surface of the basic model using the boundary image between two teeth in the stitching processing according to the embodiment. FIG. 4 is a diagram showing; FIG. 24C shows an example of processing for pasting the second molar and the first molar on the right side of the mandible to the crown side surface of the basic model using the boundary image between two teeth in the stitching processing according to the embodiment. FIG. 4 is a diagram showing;

An intraoral camera system according to an aspect of the present disclosure includes an imaging unit that generates image data by manually imaging teeth in the oral cavity of a user, and a basic model that is a three-dimensional model having a U shape in top view and a model generation unit that generates a tooth row model of the user by pasting the image data.

According to this, the intraoral camera system can generate a tooth row model that accurately represents the state of the tooth row.

For example, the image data includes upper jaw image data obtained by photographing the teeth of the upper jaw of the user and lower jaw image data obtained by photographing the teeth of the lower jaw of the user, and the basic model is an upper jaw basic model corresponding to the upper jaw. The model generation unit pastes the upper jaw image data to the upper jaw basic model, and pastes the lower jaw image data to the lower jaw basic model. model may be generated.

For example, the image data includes buccal image data obtained by photographing the tooth from the buccal side, lingual image data obtained by photographing the tooth from the lingual side, and crown side image data obtained by photographing the tooth from the crown side. The model generating unit pastes the buccal image data from the buccal side to the basic model, pastes the lingual image data from the lingual side to the basic model, and attaches the dental crown to the basic model. The tooth row model may be generated by pasting the crown side image data from the side.

For example, the cross-sectional shape of the basic model may differ depending on the position of the tooth.

For example, the intraoral camera system further includes a user information acquisition unit that acquires user information indicating at least one of gender, age group, and race of the user, and the model generation unit generates Alternatively, the tooth row model may be generated by selecting a basic model to be used from a plurality of basic models and pasting the image data onto the selected basic model.

According to this, the intraoral camera system can generate a tooth row model according to the user.

For example, the intraoral camera system further comprises an identification unit that identifies the type and position of the tooth indicated by the image data from the image data, and the basic model includes a plurality of models each corresponding to one tooth. The model generation unit may paste the image data to the corresponding position of the basic model based on the type and position of the tooth identified by the identification unit.

For example, the identification unit may identify the type and position of the tooth using an estimation model that includes a neural network, receives the image data, and outputs the type and position of the tooth.

For example, the identification unit detects interdental positions from the image data, generates tooth images each representing one tooth based on the detected interdental positions, and generates tooth images based on the tooth images. The type and position of the tooth to be extracted may be identified, and based on the identified type and position of the tooth, the tooth image may be pasted onto the corresponding position of the base model.

For example, the model generation unit may generate connected image data in which adjacent teeth are connected from the image data, and paste the connected image data on the basic model.

Further, in the tooth row model generating method according to an aspect of the present disclosure, the image capturing unit manually captures images of the teeth in the oral cavity of the user to generate image data, and a three-dimensional model having a U shape in top view. The user's tooth row model is generated by pasting the image data onto the basic model.

According to this, the tooth row model generating method can generate a tooth row model that accurately represents the state of the tooth row.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art.

It is to be noted that the inventors provide the accompanying drawings and the following description for a full understanding of the present disclosure by those skilled in the art, which are intended to limit the subject matter of the claims. not a thing

(Embodiment)
FIG. 1 is a perspective view of an intraoral camera in an intraoral camera system according to this embodiment. As shown in FIG. 1, the intraoral camera 10 has a toothbrush-like housing that can be handled with one hand. It includes a gripping handle portion 10b and a neck portion 10c connecting the head portion 10a and the handle portion 10b.

The photographing optical system 12 is incorporated in the head portion 10a and the neck portion 10c. The photographing optical system 12 includes an image sensor 14 and a lens arranged on its optical axis LA (not shown in FIG. 1).

The imaging element 14 is a photographing device such as a C-MOS sensor or a CCD element, and an image of the tooth is formed by a lens. The imaging device 14 outputs a signal (image data) corresponding to the formed image to the outside.

In addition, the intraoral camera 10 is equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices for irradiating light onto the tooth to be imaged during imaging. The first to fourth LEDs 26A to 26D are white LEDs, for example.

FIG. 2 is a schematic configuration diagram of an intraoral camera system according to this embodiment. As shown in FIG. 2, the intraoral camera system according to the present embodiment generally captures an image of a row of teeth using an intraoral camera 10, and performs image processing on the captured image. It is configured.

As shown in FIG. 2 , the intraoral camera system includes an intraoral camera 10 , a mobile terminal 70 and a cloud server 80 . The mobile terminal 70 is, for example, a smart phone or a tablet terminal capable of wireless communication. The mobile terminal 70 includes, as an input device and an output device, a touch screen 72 capable of displaying, for example, a row-of-teeth image. The mobile terminal 70 functions as a user interface of the intraoral camera system.

The cloud server 80 is a server that can communicate with the mobile terminal 70 via the Internet or the like, and provides the mobile terminal 70 with an application for using the intraoral camera 10 . For example, a user downloads an application from the cloud server 80 and installs it on the mobile terminal 70 . The cloud server 80 also acquires the row-of-teeth image captured by the intraoral camera 10 via the mobile terminal 70 .

The intraoral camera system includes a central control unit 50 as the main parts that control the system, an image processing unit 52 that processes the row of teeth image from the imaging device 14, and an LED control unit 54 that controls the plurality of LEDs 26A to 26D. , a lens driver 56 for controlling the composition adjustment mechanism actuator 36 and the focus adjustment mechanism actuator 40 , and a position sensor 90 .

The intraoral camera system also includes a wireless communication module 58 that wirelessly communicates with the mobile terminal 70, and a power control unit 60 that supplies power to the central control unit 50 and the like.

The central control unit 50 of the intraoral camera system is, for example, the handle unit 1 of the intraoral camera 10
0b is installed. For example, the central control unit 50 also includes a controller 62 such as a CPU or MPU that executes various processes described later, and a memory 64 such as RAM or ROM that stores programs for causing the controller 62 to execute various processes. contains. In addition to the programs, the memory 64 stores a row-of-teeth image (image data) captured by the imaging device 14, various setting data, and the like.

The image processing unit 52 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and acquires a row of teeth image (image data) captured by the imaging device 14 based on a control signal from the controller 62 of the central control unit 50. Then, image processing is performed on the acquired row-of-teeth image, and the row-of-teeth image after the image processing is output to the central control unit 50 . The image processing unit 52 is configured by, for example, a circuit, and performs image processing such as noise removal and AWB (Automatic White Balance) processing on the row of teeth image. The controller 62 transmits the row of teeth image output from the image processing unit 52 to the mobile terminal 70 via the wireless communication module 58 . The portable terminal 70 displays the transmitted row-of-teeth image on the touch screen 72, thereby presenting the row-of-teeth image to the user.

The LED control unit 54 is mounted on the handle portion 10b of the intraoral camera 10, for example, and turns on and off the first to fourth LEDs 26A to 26D based on control signals from the controller 62. The LED control unit 54 is configured by, for example, a circuit. For example, when the user performs an operation to activate the intraoral camera 10 on the touch screen 72 of the mobile terminal 70 , a corresponding signal is sent from the mobile terminal 70 to the controller 62 via the wireless communication module 58 . Based on the received signal, the controller 62 transmits a control signal to the LED control section 54 to light the first to fourth LEDs 26A to 26D.

The lens driver 56 is mounted, for example, on the handle portion 10b of the intraoral camera 10, and controls the actuator 36 of the composition adjustment mechanism and the actuator 40 of the focus adjustment mechanism based on control signals from the controller 62 of the central control unit 50. . The lens driver 56 is configured by, for example, a circuit. For example, when the user performs an operation related to composition adjustment or focus adjustment on the touch screen 72 of the mobile terminal 70 , a corresponding signal is transmitted from the mobile terminal 70 to the central control unit 50 via the wireless communication module 58 . The controller 62 of the central control unit 50 transmits a control signal to the lens driver 56 so as to execute composition adjustment and focus adjustment based on the received signal. Further, for example, the controller 62 calculates control amounts for the actuators 36 and 40 necessary for composition adjustment and focus adjustment based on the row-of-teeth image from the image processing unit 52, and a control signal corresponding to the calculated control amount is output to the lens. It is sent to driver 56 .

The wireless communication module 58 is mounted, for example, on the handle portion 10 b of the intraoral camera 10 and performs wireless communication with the mobile terminal 70 based on control signals from the controller 62 . The wireless communication module 58 performs wireless communication with the mobile terminal 70 in compliance with existing communication standards such as WiFi (registered trademark) and Bluetooth (registered trademark). Through the wireless communication module 58 , a row-of-teeth image showing the teeth D is transmitted from the intraoral camera 10 to the mobile terminal 70 , and an operation signal is transmitted from the mobile terminal 70 to the intraoral camera 10 .

In the case of the present embodiment, the power control unit 60 is mounted on the handle portion 10b of the intraoral camera 10, and is installed in the central control unit 50, the image processing unit 52, the LED control unit 54, the lens driver 56, and the wireless communication module 58. , distribute the power of the battery 66 . The power control unit 60 is configured by, for example, a circuit. In the case of the present embodiment, the battery 66 is a rechargeable secondary battery, which is wirelessly charged by an external charger 69 connected to a commercial power supply via a coil 68 mounted on the intraoral camera 10 . charged.

The position sensor 90 is a sensor for detecting the orientation and position of the intraoral camera 10,
For example, it is a multi-axis (three axes of x, y, and z in this case) acceleration sensor. For example, as shown in FIG. 1, the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging plane and extends longitudinally of the intraoral camera 10 . Also, the x-axis is parallel to the imaging plane and orthogonal to the y-axis. Output (sensor data) of each axis of the position sensor 90 is transmitted to the mobile terminal 70 via the central control unit 50 and the wireless communication module 58 .

The position sensor 90 may be a piezoresistive, capacitive, or thermal sensing MEMS sensor. Further, although not shown, it is preferable to provide a correction circuit for correcting the sensitivity balance of the sensors on each axis, temperature characteristics of sensitivity, temperature drift, and the like. Also, a band-pass filter (low-pass filter) for removing dynamic acceleration components and noise may be provided. Further, noise may be reduced by smoothing the output waveform of the acceleration sensor.

Next, the intraoral imaging operation in the intraoral camera system will be described. FIG. 3 is a diagram showing the flow of intraoral imaging operations in the intraoral camera system.

The user uses the intraoral camera 10 to photograph the teeth and gums in his/her own oral cavity, thereby generating image data (S101). Next, the intraoral camera 10 transmits the captured image data and the sensor data obtained by the position sensor 90 at the time of capturing to the mobile terminal 70 (S102). Note that the image data may be a moving image or one or a plurality of still images. Moreover, when the image data is a moving image or a plurality of still images, the sensor data is transmitted for each frame of the moving image or for each still image. Note that when the image data is a moving image, the sensor data may be transmitted every multiple frames.

Also, the image data and sensor data may be transmitted in real time, or may be collectively transmitted after a series of images (for example, images of all teeth in the oral cavity) are performed.

The mobile terminal 70 acquires reference data from the cloud server 80 (S103), and uses the received image data and sensor data and the acquired reference data to determine the type and position of each of the plurality of teeth included in the image data. is identified (S104).

FIG. 4 is a diagram showing teeth in the oral cavity. The types of teeth identified by the mobile terminal 70 are, for example, central incisors, lateral incisors, and canines shown in FIG. , left side, etc. In other words, identifying the type and position of the tooth means identifying which of the plurality of teeth shown in FIG. 4 the target tooth is.

Further, the mobile terminal 70 generates a tooth row model, which is a three-dimensional model of a plurality of teeth in the oral cavity, from the photographed image data, for example, using the identified types and positions of the teeth (S105). Also, the mobile terminal 70 may display an image based on the generated tooth row model.

By using such an intraoral camera system, the user can capture an image of the user's own intraoral cavity with the intraoral camera 10 and check the intraoral condition displayed on the mobile terminal 70 . This allows the user to easily check the health condition of his/her teeth.

Here, an example in which the mobile terminal 70 identifies the type of teeth and the like and generates a tooth row model will be described. may

FIG. 5 is a functional block diagram of the mobile terminal 70. As shown in FIG. Portable terminal 70 includes area detection section 101 , user information acquisition section 102 , identification section 103 , and model generation section 107 . The functions of these processing units are implemented by, for example, a program execution unit such as a CPU or processor reading and executing a software program recorded in a recording medium such as a hard disk or semiconductor memory.

The area detection unit 101 uses sensor data to detect an intraoral area corresponding to each image data, and generates area information indicating the detected area.

The user information acquisition unit 102 acquires user information indicating user attributes. For example, the user information acquisition unit 102 acquires user information input by the user via the user interface of the mobile terminal 70 . Alternatively, the user information acquisition unit 102 may acquire user information stored in the mobile terminal 70 or another device (for example, the cloud server 80). Specifically, the user information indicates at least one of the user's gender, age group (or age), and race.

The identification unit 103 identifies the type and position of each tooth included in the image data using the image data, region information, user information, and reference data. Identification unit 103 includes tooth image generation unit 104 and type identification unit 105 . From the image data, tooth images are generated, each representing one tooth. The type identification unit 105 identifies the type and position of the tooth included in the tooth image using the region information, user information, reference data, and estimation model 106 .

The estimation model 106 is a model for estimating the types and positions of teeth included in the tooth image from the tooth image and reference data. For example, estimation model 106 may include a neural network.

The model generator 107 holds a plurality of basic models 108, which are three-dimensional models that form the basis of the shape of the tooth row. The model generation unit 107 generates a tooth row model using one of the plurality of basic models 108 and the tooth image and position information (type and position of tooth) generated by the identification unit 103 .

In the imaging of the dentition, for example, from the upper left region shown in FIG. ), from the left region of the mandible to the pre-mandibular region (left molars to anterior teeth of the mandible), and then from the pre-mandibular region to the right region of the mandible (front teeth to the right molars of the mandible). to generate a tooth row model. Also, since the dentition is complicatedly curved in photographing the dentition, for example, from the left upper jaw region to the pre-maxillary region (from the left molar region to the front tooth region of the upper jaw), from the right region of the upper jaw to the pre-maxillary region (upper jaw) From the right molars to the anterior teeth of the upper jaw), the left mandibular region to the pre-mandibular region (the left molars to the anterior teeth of the mandible), and the right mandibular region to the pre-maxillary region (the right molars to the anterior teeth of the upper jaw). A plurality of tooth row models may be generated.

Next, examples of areas detected by the area detection unit 101 will be described. FIG. 6 is a diagram showing an example of this intraoral region. In the figure, for example, a plurality of teeth in the oral cavity are divided into six regions: maxillary left, maxillary front, maxillary right, mandibular left, mandibular front, and mandibular right. Although an example of dividing into six regions is shown here, the number of regions may be arbitrary. It may also be divided into two regions, an upper jaw and a lower jaw. Also, each region may be further divided according to the imaging direction. For example, as shown in FIG. 6, each region may be divided into two imaging directions, buccal and lingual. Further, although an example in which each tooth does not belong to multiple regions is shown here, some teeth may belong to two or more regions. For example, a tooth near the border of two adjacent regions may belong to both of these regions. For example, the leftmost canine tooth No. 3 in the front of the maxilla in FIG. 6 may belong to both the front of the maxilla and the left of the maxilla.

A specific example of a method for determining this area and shooting direction from sensor data will be described below. First, the region detection unit 101 determines whether it is the upper jaw or the lower jaw based on the output Az of the z-direction acceleration sensor. Here, when photographing the row of teeth in the upper jaw, the imaging plane is more or less upward, and when photographing the row of teeth in the lower jaw, the imaging plane is more or less downward. Therefore, the area detection unit 101 determines that the area corresponding to the image data is the lower jaw when Az>0, and the area corresponding to the image data is the upper jaw when Az≦0.

Next, a method of determining which region of the maxilla is determined when the region is determined to be the maxilla will be described. The region detection unit 101 determines whether or not it is a front tooth based on the output Ay of the acceleration sensor in the y direction. Here, the intraoral camera 10 is relatively horizontal when photographing anterior teeth, but the intraoral camera 10 must be tilted when photographing molars due to interference with lips. Therefore, the region detection unit 101 determines that the region is in front of the maxilla when Ay≦threshold value a.

Furthermore, when the region detection unit 101 determines that it is in front of the maxilla, it determines whether it is on the buccal side or the tongue side based on the output Ax of the acceleration sensor in the x direction. Here, the direction of the imaging surface is reversed between the buccal side and the tongue side. Therefore, the region detection unit 101 determines the “maxillary anterior buccal side” when Ax>0, and determines the “maxillary anterior lingual side” when Ax≦0.

On the other hand, if the area detection unit 101 determines that the area is not in front of the maxilla, the area detection unit 101 determines the orientation of the imaging plane based on the output Ax of the acceleration sensor in the x direction. Specifically, when Ax>0, the region detection unit 101 determines “maxillary right buccal side or maxillary left lingual side”, and when Ax≦0, determines “maxillary left buccal side or maxillary right lingual side”. judge.

moreover. Therefore, the area detection unit 101 narrows down the area based on the area determined in the previous process. Specifically, when determining whether the upper jaw is on the right buccal side or the upper jaw is on the left lingual side, the area detection unit 101 determines that the previous area was "maxillary front buccal side, maxillary right buccal side, maxillary right lingual side, mandibular front buccal side." , lower jaw right buccal side, or lower jaw right lingual side”, the current region is assumed to be “upper jaw right buccal side”, and the previous region was “maxillary anterior lingual side, maxillary left buccal side, maxillary If it is one of "left lingual side, mandibular anterior lingual side, mandibular left buccal side, mandibular left lingual side", the current region is estimated to be "maxillary left lingual side".

Further, when determining whether the upper jaw left buccal side or the upper jaw right lingual side, the region detection unit 101 determines that the previous region was “maxillary front buccal side, maxillary left cheek side, maxillary left tongue side, mandibular front buccal side, mandibular left side”. If it is either buccal, left lingual of the lower jaw, the current area is estimated to be the left buccal of the upper jaw, and the previous area is the anterior lingual of the upper jaw, right buccal of the upper jaw, or right lingual of the upper jaw. , anterior lingual side of the lower jaw, right buccal side of the lower jaw, or right lingual side of the lower jaw", the current region is estimated to be the "right lingual side of the upper jaw". This utilizes the fact that there is a high probability that the imaging plane will be moved so that the amount of movement and direction change of the imaging plane are minimized.

Similar determination is also used for the mandible. Specifically, the area detection unit 101 determines whether or not it is the front tooth based on the output Ay of the acceleration sensor in the y direction. Specifically, the region detection unit 101 determines that the region is in front of the mandible when Ay≦threshold value b.

If it is determined to be in front of the mandible, the region detection unit 101 determines whether it is on the buccal side or the tongue side based on the output Ax of the acceleration sensor in the x direction. Specifically, the region detection unit 101 determines the “mandibular anterior buccal side” when Ax<0, and determines the “mandibular anterior tongue side” when Ax≧0.

On the other hand, if the area detection unit 101 determines that the area is not in front of the mandible, the area detection unit 101 determines the orientation of the imaging plane based on the output Ax of the acceleration sensor in the x direction. Specifically, when Ax>0, the region detection unit 101 determines “the right buccal side of the lower jaw or the left lingual side of the lower jaw”, and when Ax≦0, determines the “left buccal side of the lower jaw or the right lingual side of the lower jaw”. judge.

Further, when determining whether the area detection unit 101 is on the right buccal side of the mandible or the left tongue side of the mandible, the previous area was "mandibular front buccal side, mandibular right buccal side, mandibular right lingual side, mandibular front buccal side, maxillary right side." If it is either the buccal side or the right lingual side of the upper jaw, the current area is assumed to be the 'right buccal side of the lower jaw', and the previous area was 'anterior lingual side of the lower jaw, left buccal side of the lower jaw, left lingual side of the lower jaw'. , maxillary anterior lingual side, maxillary left buccal side, maxillary left lingual side", the current region is estimated to be 'mandibular left lingual side'.

Further, when determining whether the region is on the left buccal side of the mandible or the right lingual side of the mandible, the region detection unit 101 determines that the previous region was “mandibular front buccal side, mandibular left buccal side, mandibular left lingual side, maxillary front buccal side, maxillary left side. In the case of either buccal, maxillary left lingual side”, the current region is estimated to be “mandibular left buccal side”, and the previous region was “mandibular anterior lingual side, mandibular right buccal side, mandibular right lingual side”. , maxillary anterior lingual side, maxillary right buccal side, maxillary right lingual side", the current region is estimated to be 'lower jaw right lingual side'.

By the above processing, the current regions are "maxillary anterior buccal side", "maxillary anterior lingual side", "maxillary right buccal side", "maxillary left buccal side", "maxillary left buccal side", and "maxillary right lingual side". , "mandibular anterior buccal side", "mandibular anterior lingual side", "mandibular right buccal side", "mandibular left lingual side", "mandibular left buccal side", and "mandibular right lingual side". .

The above determination algorithm is merely an example, and any determination algorithm may be used as long as the area can be identified from the outputs Ax, Ay, and Az of the acceleration sensor. For example, instead of using the values of Ax, Ay, and Az as they are as variables for determination, secondary variables obtained by appropriately combining Ax, Ay, and Az may be used for determination. Secondary variables can be arbitrarily set, for example, Ay/Az, Ax·Ax+Ay·Ay, Az−Ax. Alternatively, the area may be determined after converting the acceleration information Ax, Ay, and Az of each axis into angle information (attitude angles) α, β, and γ. For example, the angle of the x-axis with respect to the direction of gravitational acceleration may be defined as roll angle α, the angle of y-axis with respect to the direction of gravitational acceleration as pitch angle β, and the angle of z-axis with respect to the direction of gravitational acceleration as yaw angle γ. Also, the threshold used for each determination can be determined from the results of clinical experiments and the like.

Also, in the above description, two directions, ie, the buccal side and the lingual side, are determined as imaging directions, but three directions including the crown side may be determined. For example, when photographing the crown side, it is possible to determine whether the photographing direction is the crown side by utilizing the fact that the imaging surface is more horizontal than when photographing the buccal and lingual sides.

Next, the details of the operation of the identification unit 103 will be described. Note that processing for one image data (one frame included in a moving image or one still image) will be described below.

First, the tooth image generation unit 104 generates tooth images each representing one tooth from the image data. Specifically, the tooth image generation unit 104 detects interdental positions from image data by image analysis or the like, and extracts a tooth image using the detected interdental positions. For example, the tooth image generating unit 104 generates a tooth image by extracting an image using interdental positions as boundaries.

Next, the type identification unit 105 identifies the type and position of the tooth included in the tooth image using the region information, user information, reference data, and estimation model 106 .

The reference data is data that is referred to when identifying the type and position of the tooth included in the tooth image. For example, the reference data is tooth data whose type and position are known. Specifically, the reference data may be a group of image data of teeth photographed in advance, a group of image data of a row of teeth, or a panoramic image of the row of teeth. Alternatively, the reference data may be information indicating the standard shape or characteristic amount of each tooth.

Note that the reference data may be classified by shooting direction and by user attribute in addition to the type and position. A user attribute is one of a user's sex, age group (or age), and race, or a combination of two or more. That is, user attributes are uniquely determined according to the user's gender, age group, and race.

FIG. 7 is a diagram showing an example of classification of reference data. Note that although reference data is hierarchically classified in the figure, it is not always necessary to classify reference data hierarchically. Reference data used for identification is represented as A(n). Also, n is uniquely associated with a set of tooth type, position, and imaging direction. FIG. 8 is a diagram showing an example of reference data. As an example, buccal, lingual and coronal reference data are shown for maxillary incisors, canines and first molars, respectively.

As shown in FIG. 8, teeth vary in shape and size depending on their type. For example, the maxillary central incisors have the following characteristics: The buccal profile is generally elongated and trapezoidal, with an almost straight incisal edge. The cervical line is convex towards the root, and the mesial and distal margins are slightly curved. The apex of the mesial curve is near the mesial incisal corner. The curved top of the distal edge is 1/3 the height of the incisal side. The lingual profile is triangular, with the mesial and distal marginal ridges and the lingual cervical ridge forming the marginal ridge, forming the lingual fossa.

In addition, the canine teeth of the upper jaw have the following characteristics. The buccal profile is generally pentagonal, with the incisal margin projecting centrally to form a cusp. The cervical line is convex towards the root. The mesial margin is straight or slightly convex and the distal margin is straight or slightly concave. The lingual profile is rhomboid, with the mesial and distal marginal ridges and the lingual cervical ridge forming the marginal ridge.

Also, the maxillary first molar has the following characteristics. The buccal profile is generally trapezoidal and the mesiodistal margin is nearly straight. The cervical line becomes convex at the bifurcation in the horizontal and central parts. The contact point is mesial and 1/3 of the height on the occlusal side, and distal to 1/2 of the height. The lingual profile is trapezoidal, with a lingual groove running almost centrally. The coronal side has a parallelogram shape, and the buccolingual diameter is greater than the mesiodistal diameter.

A tooth image to be processed is represented by B(m). Therefore, the tooth images of the teeth adjacent to the tooth image to be processed are represented by B(m−1) and B(m+1).

Also, the area information corresponding to the tooth image (B(m)) to be processed, which is detected by the area detection unit 101, is represented as C(m). For example, area information is generated for each image data. Therefore, when a plurality of teeth are included in one image data and a plurality of tooth images are generated, the same area information corresponding to the one image data is obtained for the plurality of tooth images. be associated.

FIG. 9 is a flowchart of type identification processing by the type identification unit 105 . First, the type identification unit 105 performs initialization (S111). Specifically, the type identifying unit 105 sets n to 0, Err to Err_Max, and N to 0. Here, Err is an evaluation value, which will be described later, and the smaller the value of Err, the higher the evaluation. Err_Max is the theoretical maximum value that Err can take. Also, N indicates the n value of the minimum Err.

Next, the type identification unit 105 selects reference data to be used based on the user information and the area information (S112). Specifically, the type identifying unit 105 is assigned the user attribute indicated by the user information, and the reference data assigned to the tooth type, position, and imaging direction corresponding to the region indicated by the region information. to select. For example, if the region indicated by the region information is on the left lingual side of the upper jaw, a total of five reference data on the lingual side of five teeth included in the left of the upper jaw are selected as reference data to be used. Also, n_max, which is the maximum value of n, is set according to the number of selected reference data. For example, if the number of reference data is 5, n=0 to 4 are assigned to the 5 reference data, and n_max is set to 4.

Next, the type identification unit 105 calculates Err(n) from the tooth image B(m) and the reference data (A(n)) (S113). For example, the type identification unit 105 calculates Err(n) using Err(n)=f(A(n))−f(B(m)). where f(A(n)) and f(B(m)) are the values of A(n) and B(m) thrown into the function f(). Function f( ) is a function for extracting feature amounts of A(n) and B(m). Note that f() may be represented by a vector instead of a scalar.

As shown in FIG. 8, each tooth has a characteristic shape and size according to its type. The type identification unit 105 extracts these characteristic shapes and sizes as feature amounts using the function f described above.

The feature quantity extracted by the function f will be described with reference to FIG. FIG. 10 is a diagram showing an image of the first molar in the maxillary right region taken from the crown side. The occlusal surface of the first molar has a shape close to a parallelogram, and the straight line AB and the straight line DC connecting the mesiodistal buccal cusp and the lingual cusp are in a nearly parallel relationship. AD and straight line BC are almost parallel to each other. Also, the distances between the cusp vertices are substantially equal (AB=DC, AD=BC). As an example. The above-described two cusp-to-cusp distances can be used as feature amounts.

Err(n) is a value indicating the difference (distance in the case of a vector) between f(A(n)) and f(B(m)). In other words, the closer B(m) is to A(n), the smaller "f1(A(n)) - f1(B(m))" is, and when n=m, Err(n) has a local minimum value. take.

When the calculated Err(n) is smaller than Err, the type identification unit 105 sets Err to Err(n) and sets N to n (S114).

If n is not n_max (No in S115), the type identification unit 105 increments n by 1 (S116), and performs the processing from step S113 onward again. That is, steps S113 and S114 are performed for all reference data to be used.

If n=n_max (Yes in S115), the type identification unit 105 outputs the type, position and imaging direction corresponding to N as the type, position and imaging direction of the tooth included in the tooth image (S117).

Through the above processing, the type identification unit 105 can identify the type, position, and imaging direction of the tooth image. Further, in step S112, it is possible to reduce candidates for tooth types, positions, and imaging directions based on user information and area information. Therefore, the amount of processing can be reduced and the accuracy of identification can be improved.

FIG. 11 is a flow chart showing another example of type identification processing by the type identification unit 105 . The process shown in FIG. 11 is different from the process shown in FIG. 9 in that step S112 is changed to step S112A and step S118 is added.

At step S112A, the type identification unit 105 selects reference data to be used based on the user information. Specifically, the type identification unit 105 selects reference data to which the user attribute indicated by the user information is assigned.

In step S118, the type identifying unit 105 weights the Err(n) calculated in step S113 based on the area information. Specifically, the type identification unit 105 multiplies Err(n) by w according to the area information. For example, if the area indicated by the area information includes the tooth corresponding to n, Err(n) is multiplied by w0. If the tooth corresponding to n is not included in the area indicated by the area information, Err(n) is multiplied by w1, which is greater than w0. As a result, the Err of the tooth included in the area indicated by the area information becomes smaller, so that the tooth included in the tooth image is more likely to be determined as the tooth included in the area.

Also, the weighting does not have to be in two stages, i.e. whether it is included in the area or not. For example, the weight may be set according to the distance from the area indicated by the area information. For example, the weight of teeth located close to the area indicated by the area information may be set smaller than the weight of teeth located far from the area.

Also, the user information may be used for weighting Err(n) in the same manner as the region information, rather than for selecting reference data.

Further, selection of reference data based on area information as described with reference to FIG. 9 and weighting described with reference to FIG. 11 may be combined. For example, teeth located far from the area indicated by the area information may be excluded from the target, and weights may be used for teeth located close to the area.

In addition, in the case where a tooth image of the user has been obtained in the past, such as when the user performs intraoral imaging on a regular basis, the tooth image may be used as reference data. In this case, selection of reference data based on the user information is not performed, and only processing based on area information is performed.

In the above description, the tooth image to be processed is compared with the reference data. good too.

For example, the type identification unit 105 is Err(n)=f(A(n))-f(B(m))+f'(A(n-1))-f'(B(m-1))+f Err(n) may be calculated by '(A(n+1))-f'(B(m+1)). where A(n−1) and A(n+1) are the reference data of teeth adjacent to the tooth of A(n), and B(m−1) and B(m+1) are the reference data of B(m). 4 is a tooth image of a tooth adjacent to the tooth; Also, f'( ) is a function for extracting a feature amount for evaluating teeth on both sides of the tooth of interest. In this way, identification accuracy can be improved by also using information on adjacent teeth.

Also, in the above, an example in which a tooth image is used as reference data has been described, but a feature amount (that is, the value of f(A(n)) described above) may be used as reference data.

In addition, the estimation model 106 used for identification by the type identification unit 105 may include a trained model such as a neural network. For example, the function f or f' mentioned above may be a trained model. Moreover, the technique using a neural network is not limited to this, and for example, the entire exterior of the estimation model 106 that performs type estimation may be configured with a neural network. In this case, for example, an estimation model 106 may be provided for each user attribute. Each estimation model 106 is trained by machine learning using a plurality of pairs of tooth images, region information, tooth types, positions, and imaging directions of corresponding user attributes as teacher data (learning data). is a model. The estimation model 106 receives the tooth image and the area information as input,
Tooth type, position and imaging direction are output. In this case, the type identification unit 105 selects the corresponding estimation model 106 using the user information, and acquires the type, position, and imaging direction of the tooth by inputting the tooth image and region information to the selected estimation model 106. do.

Alternatively, the estimation model 106 may be provided for each set of user attribute and area information. In this case, each estimated model 106 is a learned model generated by machine learning using a plurality of pairs of corresponding user attributes and region information, tooth images, tooth types, positions, and imaging directions as teacher data. be. Also, the estimation model 106 receives tooth images as input and outputs tooth types, positions, and imaging directions. In this case, the type identification unit 105 selects the corresponding estimation model 106 using the user information and the region information, and inputs the tooth image to the selected estimation model 106 to acquire the type, position, and imaging direction of the tooth. .

In the above description, an example in which both user information and area information are used has been described, but only one of them may be used.

Also, in the above description, the example where the area information indicates the area and the shooting direction was described, but only one of them may be indicated.

Next, the details of the operation performed by the identification unit 103 in the intraoral camera system when the user has lost some teeth (for example, the “second premolar on the left side of the upper jaw”) due to dental caries treatment or the like will be described. Note that processing for one image data (one frame included in a moving image or one still image) will be described below. FIG. 12 is a diagram showing the intraoral state of a user whose upper left second premolar is missing.

The region detection unit 101 identifies that the intraoral camera 10 is capturing an image of the “maxillary left side” region including the second premolar. Then, the intraoral camera 10 captures the image B(m') of the region corresponding to the second premolar, and the absence of the tooth is detected by image analysis or the like.

Next, the type identification unit 105 calculates Err(n) from the tooth images B(m′−1), B(m′+1) and the reference data (A(n)), and calculates the tooth image B(m '-1) and B(m'+1), identifying the type and position of the tooth, and confirming that B(m') is an image of the area sandwiched between the first premolar and first molar on the left side of the upper jaw. It is determined that the second premolar is missing, and the determination result is output.

When the third molar is missing, it can be determined that the third molar is missing by specifying the adjacent second molar.

In addition, in the case where a tooth image of the user has been obtained in the past, such as when the user performs intraoral imaging on a regular basis, the tooth image may be used as reference data. In this case, it is possible to obtain information about the missing tooth of the user from the results of past intraoral imaging.

Next, the details of the operation of the model generation unit 107 will be described. FIG. 13 is a flowchart of tooth row model generation processing by the model generation unit 107 . First, the model generation unit 107 selects a basic model 108 to be used from among a plurality of basic models 108 based on user information (S121). Specifically, the basic model 108 is provided for each user attribute (one of the user's sex, age group (or age), and race, or a combination of two or more). The model generation unit 107 selects the basic model 108 associated with the user attribute indicated by the user information as the basic model 108 to be used.

FIG. 14 is a perspective view showing the configuration of the basic model 108. As shown in FIG. The basic model 108 is a three-dimensional model that schematically represents the shape of a row of teeth, and has a U shape (arch shape or horseshoe shape) in top view (as seen from the crown side). Here, the basic model of the lower jaw dentition (lower jaw basic model) will be described as an example, but the basic model of the upper jaw dentition (upper jaw basic model) is also used in the same manner, and the same processing is performed. The upper jaw basic model is, for example, a shape obtained by vertically inverting the lower jaw basic model.

FIG. 15 is a top view of the basic model 108 of the lower jaw, and is a plan view of the basic model 108 viewed from the crown side. Here, the basic model 108 is composed of a plurality of blocks 108A. FIG. 16 is a perspective view showing the configuration of block 108A. Here, one block 108A corresponds to one tooth in the oral cavity. That is, the lower jaw basic model is composed of a plurality of blocks 108A that are in one-to-one correspondence with a plurality of teeth included in the lower jaw. The cross-sectional shape of the basic model 108 shown in FIGS. 14 and 16 is rectangular.

Note that the shape of the basic model 108 is not limited to the above. FIG. 18 is a perspective view showing the configuration of a modification of the basic model 108. As shown in FIG. FIG. 19 is a perspective view of block 108A in this case. For example, as shown in FIGS. 18 and 19, the cross-sectional shape of the basic model 108 does not have to consist entirely of straight lines, and may include curved lines. Also in this case, the cross-sectional shape may differ depending on the position. Moreover, the cross-sectional shape may be square, rectangular, or trapezoidal. Moreover, the cross-sectional shape is not limited to a quadrangle, and may be an arbitrary polygon. Moreover, the cross-sectional shape does not have to be composed entirely of straight lines, and may include curved lines. Also in this case, the cross-sectional shape may differ depending on the position.

Also, the cross-sectional shape of the basic model 108 may vary depending on the position. For example, as shown in FIG. 15, the width w1 of the back teeth (the width between the buccal side and the tongue side) may be wider than the width w2 of the front teeth. In addition, the length (the length in the direction perpendicular to the width in plan view) or height may be different, and the cross-sectional shape (for example, the curvature of the arc on the interdental side) may be different.

Next, the model generation unit 107 performs a stitching process of pasting a plurality of tooth images onto the selected basic model 108 (S122). The model generator 107 uses images of boundaries between adjacent teeth in the stitching process. Therefore, it is desirable that the tooth image used in the stitching process includes an image of the boundary with at least one of the adjacent teeth.

Therefore, when generating a tooth image by extracting an image using interdental positions, the tooth image generating unit 104 generates a tooth image so as to include an image of the boundary between the photographed tooth and at least one adjacent tooth. It is desirable to generate Alternatively, the tooth image generation unit 104 generates a tooth image by extracting an image using the interdental positions, and also generates an image of the boundary between the tooth for which the tooth image was generated and the adjacent tooth. can be It should be noted that, for example, when teeth on both the left and right sides of the target tooth are missing, there are no teeth adjacent to the target tooth. In this case, for example, the tooth image preferably includes a boundary portion between the target tooth and the area where the tooth is missing (the space where the adjacent tooth originally exists).

It should be noted that the tooth image used for identifying the tooth type and position described above does not necessarily include the boundary portion described above. That is, the tooth image used for identifying the tooth type and position may be different from the tooth image used for the stitching process. For example, the tooth image used for identifying the tooth type and position may not include the boundary portion, and the tooth image used for the stitching process may be included.

FIG. 17 is a diagram showing an example of this stitching process. For example, the model generator 107
pastes a tooth image having the same location information (type and location) as block 108A to block 108A. Specifically, the model generator 107 pastes the buccal tooth image to the buccal side of the block 108A, pastes the lingual tooth image to the lingual side of the block 108A, and pastes the crown side tooth image to the crown of the block 108A. stick to the side. Here, the buccal tooth image, the lingual tooth image, and the crown side tooth image are tooth images photographed in the buccal side, the lingual side, and the crown side, respectively. The model generator 107 generates a tooth row model in which the tooth images of all the teeth in the oral cavity are attached to the basic model 108 by performing this process on all blocks 108A.

Here, an example in which tooth images in each imaging direction are individually pasted is shown, but an image (panoramic image) is generated by connecting tooth images in a plurality of imaging directions, and the generated image is applied to the surface shape of block 108A. You can paste it along the

Also, although an example of pasting a tooth image for each block 108A is shown here, a tooth row image (connected image data) representing a tooth row including a plurality of teeth is collectively pasted to the corresponding multiple blocks 108A. may The row of teeth included in the row of teeth image may include all or part of the teeth of the upper jaw or the lower jaw.

For example, the model generation unit 107 may generate a tooth row image using a plurality of tooth images, or may use the tooth row image included in the image data obtained by the intraoral camera 10 as the tooth row image. may For example, the model generation unit 107 generates a tooth row image by connecting a plurality of tooth images or image data. For example, images can be connected using a known panorama image generation method such as a block matching method, an optical flow method, or an intermediate luminance method.

For example, the model generation unit 107 detects corresponding points, which are feature points corresponding to each other, by performing feature point matching of the two images, and connects the two images so that the corresponding points overlap. At this time, the model generation unit 107 may perform transformation processing such as enlargement, reduction, rotation, etc., by affine transformation or the like on at least one of the two images. In addition, the model generation unit 107 may use the pixel values of one of the images as the pixel values of the overlapping area between the two images, or calculate the pixel values of the two images (for example, average or weighted addition). By doing so, the pixel value of the area may be calculated. In addition, the model generation unit 107 may use inter-tooth positions as feature points. Also, the model generation unit 107 may use the position information of each tooth image at the time of connection. The model generation unit 107 uses the image of the boundary between adjacent teeth in each image to connect the two images, and performs feature point matching of the two images to determine corresponding points, which are feature points that correspond to each other. The two images are connected so that the corresponding points overlap. For this reason, it is desirable that the tooth image includes an image of the boundary with at least one of the adjacent teeth. Alternatively, the model generation unit 107 uses the image of the boundary between the tooth for which the tooth image is generated and the adjacent tooth, and creates three images: two tooth images to be connected and synthesized, and an image of the boundary between them. Corresponding points, which are feature points corresponding to each other, may be detected by feature point matching, and three images may be connected so that the corresponding points overlap. In other words, when connecting two tooth images, the model generation unit 107 may use three images, that is, the two tooth images and the image of the boundary between the two teeth. In this way, by additionally using the image of the boundary, at least one of the two tooth images to be connected does not include the boundary, only a part of the boundary is included, and the image of the tooth image Images can be connected with high accuracy even when the edge and the boundary part match or are close to each other.

It is desirable that the tooth image includes an image of the boundary between at least one of the adjacent teeth. dimension), or an image of adjacent teeth within a range of at least about 10% of the array dimension (for example, the interval between adjacent teeth) is included in the tooth image.

In the above, an example of using the position information of the tooth image obtained by the identification unit 103 was described, but the method of generating the position information is not limited to this. For example, the model generator 107 may acquire tooth images and position information from an external device.

In this manner, the model generating unit 107 may generate connected image data by connecting adjacent teeth from image data and paste the connected image data on the basic model 108 .

Even when tooth images are pasted individually, image linking processing or complementing processing similar to that described above may be performed in an area where the pasted images overlap (an area near the boundaries of blocks).

FIG. 20 is a top view of the basic model 108 of the lower jaw, and is a plan view of the basic model 108 viewed from the crown side. Below, as shown in FIG. 20, the coronal images of the right mandibular second molar, first molar, and second premolar are stitched to the coronal sides of the corresponding blocks 108A1 and 108A2 of the basic model 108. The operation of ringing will be explained.

21A and 21B are diagrams showing examples of tooth images used for stitching processing. FIG. 21A is a diagram showing an image 121 in which a part of the upper central incisor and lateral incisor is captured. FIG. 21B is a diagram showing an image 122 in which a portion of the second and first molars of the mandible is captured. The image 121 shown in FIG. 21A includes the boundary between the central and lateral incisors. The image 122 shown in FIG. 21B includes the boundary between the second molar and the first molar.

Next, as an example of the stitching process, the operation of pasting the crown images of the second molar and the first molar on the crown side of the basic model 108 will be described. FIGS. 22A and 22B are diagrams for explaining the process of attaching the second molar on the right side of the mandible to the crown side surface of the basic model 108 in the stitching process. FIG. 22A is an example of an image 122 of a second molar. This image 122 includes the second molar and part of the first molar. A buccal groove runs along the buccal contour of the crown of the first molar. For example, as shown in FIG. 22B, the model generation unit 107 generates an image 122 such that the buccal contour 110 in which the buccal groove is formed is in contact with the buccal edge of the crown surface of the block 108A1 of the basic model 108. is attached to the crown surface of block 108A1. In this way, the orientation of pasting the image 122 is determined. At this time, the model generation unit 107 performs transformation processing such as enlargement, reduction, and rotation on the image 122 by, for example, affine transformation in order to match the image 122 of the second molar to the shape of the crown surface of the block 108A1. is preferred.

23A, 23B, and 23C are diagrams for explaining the operation of connecting the image of the first molar to the image of the second molar attached to the crown surface of block 108A1. FIG. 23A shows a state in which the contour 112 of the second molar facing the boundary with the first molar is extracted as a singular point from the image 122 of the second molar pasted on the crown surface of the block 108A1. showing. FIG. 23B is a diagram showing an image 123 of the first molar. This image 123 includes a first molar and a portion of a second molar located above the first molar. FIG. 23B shows a state in which the outline 114 facing the boundary between the second molar and the first molar is extracted as a singular point in the image 123 of the first molar. FIG. 23C shows a state in which the image 122 of the second molar and the image 123 of the first molar are connected by overlapping the contour 112 and the contour 114 extracted in FIGS. 23A and 23B. At this time, when superimposing the contour 112 extracted from the image 122 of the second molar and the contour 114 extracted from the image 123 of the first molar, the model generation unit 107 generates It is preferable to perform transformation processing such as enlargement, reduction, rotation, etc., for example, by affine transformation. In addition to the process of superimposing the contour 112 and the contour 114, pasting the image 123 so that the buccal contour of the crown of the first molar is in contact with the buccal side of the crown surface of the block 108A2 is the actual alignment of the teeth. and is desirable.

In the above description, contours are used for pasting, but a plurality of singular points may be set on the tooth and images may be connected by singular point matching.

Also, the tooth images may be connected in the direction from the central incisor to the lateral incisor to the canine. In this case, the images can be connected by the operation described for the second molar. Also, when pasting the images of the crowns of the central incisors, the lateral incisors, and the canines on the crown surface of the basic model 108, the images of the central incisors, the lateral incisors, and the canines are attached to the buccal side (labial side) of the block 108A. It is preferable that the image is attached to the block 108A so that the buccal (lipal) contour is in contact with the actual tooth alignment.

By the above operation, a partial panorama image of the left region (left mandible and left front of mandible) and right region (right mandible and right front of mandible) of the mandible can be generated. Next, the model generation unit 107 generates a block 108A4 where the crown surface of the block 108A3 to which the left central incisor in front of the mandible is pasted and the block 108A4 to which the right central incisor in front of the mandible is pasted in the basic model 108 shown in FIG. , to generate a panoramic dentition image of the whole.

When the tooth crown surface of block 108A3 and the tooth crown surface of block 108A4 are connected, the model generation unit 107 performs stitching processing using images of the boundary between the left central incisor in front of the mandible and the right central incisor in front of the mandible. can make the interdental area clearer.

24A, 24B, and 24C are diagrams showing examples of tooth images used to connect the coronal surfaces of blocks 108A3 and 108A3. FIG. 24A shows an image 124 of the left central incisor, part of the right central incisor, and part of the left lateral incisor of the mandible. FIG. 24B shows an image 125 of the right central incisor, part of the left central incisor, and part of the right lateral incisor of the mandible. Images 124 and 125 of FIGS. 24A and 24B, respectively, include the boundary between the left central incisor and the right central incisor. Also, the image 124 can be attached to the block 108A3 so that the buccal side (labial side) of the crown surface of the block 108A3 is in contact with the buccal side (labial side) of the crown of the left central incisor. and is desirable. Similarly, it is practical to paste the image 125 to the block 108A4 so that the buccal (labial) contour of the crown of the right central incisor is in contact with the buccal (labial) side of the crown surface of the block 108A4. It conforms to the tooth alignment and is desirable.

FIG. 24A extracts the outline 116 of the left central incisor facing the boundary with the right central incisor as a singular point from the image 124 of the left central incisor pasted on the coronal surface of block 108A3. It shows the state of FIG. 24B shows the portion of the left central incisor included in the image 125 of the right central incisor pasted on the coronal surface of block 108A4, versus the left side facing the boundary with the right central incisor. shows a state in which the contour 118 of the central incisor is extracted as a singular point. FIG. 24C shows a state in which the image 124 of the left central incisor in block 108A3 and the image 125 of the right central incisor in block 108A4 are connected by overlapping the contours 116 and 118 extracted in FIGS. 24A and 24B. showing. At this time, when the contour 116 and the contour 118 are superimposed, the model generating unit 107 preferably performs transformation processing such as enlargement, reduction, rotation, etc., by affine transformation, for example, on the image 125 of the right central incisor. .

In the above description, contours are used for pasting, but a plurality of singular points may be set on the tooth and images may be connected by singular point matching.

In the above, the tooth row model generation processing has been described with a focus on teeth, but the image data, the tooth image, the tooth row image, and the tooth row model may include an image of the gingiva (gums) in addition to the teeth. Further, as shown in FIG. 12, when some teeth are missing, the model generation unit 107 places a mark indicating that there are no teeth on the surface to which the image of the block 108A without teeth is pasted, or You can make a notation.

As described above, the intraoral camera system includes a photographing unit (for example, the intraoral camera 10) that generates image data by manually photographing the teeth in the user's oral cavity, and a three-dimensional A model generation unit 107 is provided for generating a user's tooth row model by pasting image data onto a basic model 108 which is a model.

According to this, the intraoral camera system can generate a tooth row model that accurately represents the state of the tooth row.

Here, regarding the image stitching process of the tooth row and the gingiva, when the tooth row and the gingiva are photographed and the local photographed images are stitched together to model the overall visual information of the tooth row and the gingiva, In some cases, general stitching used in panorama photos cannot be modeled well. This is because the tooth row and gingiva of the upper jaw and the tooth row and gingiva of the lower jaw are not two-dimensional but three-dimensional. end up

On the other hand, in the present embodiment, the stitching process is performed on the surface of the basic model 108 representing the tooth row in a U-shape. As a result, the occurrence of distortion or the like can be suppressed, so the accuracy of the tooth row model can be improved. In addition, for example, since an image of the tooth row model viewed from any direction can be displayed to the user, user convenience can be improved.

For example, the image data includes upper jaw image data obtained by photographing the teeth of the user's upper jaw and lower jaw image data obtained by photographing the teeth of the user's lower jaw. The basic model 108 includes an upper jaw basic model corresponding to the upper jaw and a lower jaw basic model corresponding to the lower jaw. The model generation unit 107 generates a tooth row model by pasting the upper jaw image data to the upper jaw basic model and pasting the lower jaw image data to the lower jaw basic model.

For example, as shown in FIG. 17 and the like, the image data includes buccal image data in which the teeth are photographed from the buccal side, lingual image data in which the teeth are photographed from the lingual side, and image data in which the teeth are photographed from the crown side. The model generation unit 107 pastes the buccal-to-buccal image data to the basic model 108, pastes the lingual-to-lingual image data to the basic model 108, and pastes the lingual image data to the basic model 108. A tooth row model is generated by pasting the crown side image data from the crown side.

For example, as shown in FIG. 15, etc., the cross-sectional shape of the basic model 108 differs depending on the position of the tooth.

For example, the intraoral camera system further includes a user information acquisition unit 102 that acquires user information indicating at least one of the user's gender, age group, and race, and the model generation unit 107 generates a plurality of A basic model 108 to be used is selected from the basic models 108 of , and image data is pasted on the selected basic model 108 to generate a tooth row model. According to this, the intraoral camera system can generate a tooth row model according to the user.

For example, the intraoral camera system further includes an identification unit 103 that identifies the type and position of the tooth indicated by the image data from the image data. As shown in FIG. 15, etc., the basic model 108 is composed of a plurality of blocks 108A each corresponding to one tooth. Based on this, the image data is pasted on the corresponding position of the basic model 108 .

According to this, the model generation unit 107 can easily determine the image to be attached to each position of the basic model 108 based on the information obtained by the identification unit 103 . Further, since the tooth row model can be managed in units of blocks 108A, that is, in units of teeth, it is possible to easily perform processing such as displaying a specific tooth model.

For example, the identification unit 103 identifies the type and position of the tooth using an estimation model 106 that includes a neural network, receives image data, and outputs the type and position of the tooth.

For example, the identification unit 103 detects interdental positions from the image data, generates tooth images each representing one tooth based on the detected interdental positions, and generates tooth images each representing one tooth based on the tooth images. The type and position are identified, and based on the identified tooth type and position, the tooth image is pasted onto the corresponding position of the base model 108 .

For example, the model generation unit 107 generates connected image data by connecting adjacent teeth from the image data, and pastes the connected image data on the basic model 108 .

Although the intraoral camera system according to the embodiment of the present disclosure has been described above, the present disclosure is not limited to this embodiment.

Each processing unit included in the intraoral camera system according to the above embodiment is typically implemented as an LSI, which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them.

Further, circuit integration is not limited to LSIs, and may be realized by dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure connections and settings of circuit cells inside the LSI may be used.

Further, in each of the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

The present disclosure may also be implemented as a tooth row model generation method or the like executed by an intraoral camera system. Also, the present disclosure may be implemented as an intraoral camera, a mobile terminal, or a cloud server included in an intraoral camera system.

Also, the division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be moved to other functional blocks. may Moreover, single hardware or software may process the functions of a plurality of functional blocks having similar functions in parallel or in a time-sharing manner.

Also, the order in which each step in the flowchart is executed is for illustrative purposes in order to specifically describe the present disclosure, and orders other than the above may be used. Also, some of the above steps may be executed concurrently (in parallel) with other steps.

Although the intraoral camera system and the like according to one or more aspects have been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art can think of are applied to this embodiment, and a form constructed by combining the components of different embodiments is also within the scope of one or more aspects may be included within

The present disclosure is applicable to intraoral camera systems.

REFERENCE SIGNS LIST 10 intraoral camera 10a head portion 10b handle portion 10c neck portion 12 photographing optical system 14 imaging element 26A first LED
26B second LED
26C Third LED
26D 4th LED
36, 40 actuator 50 central control unit 52 image processing unit 54 LED control unit 56 lens driver 58 wireless communication module 60 power control unit 62 controller 64 memory 66 battery 68 coil 69 charger 70 mobile terminal 72 touch screen 80 cloud server 90 position sensor 101 region detection unit 102 user information acquisition unit 103 identification unit 104 tooth image generation unit 105 type identification unit 106 estimation model 107 model generation unit 108 basic model 108A block

Claims (10)

an imaging unit that generates image data by manually imaging the teeth in the user's oral cavity;
an intraoral camera system, comprising: a model generation unit that generates a tooth row model of the user by pasting the image data onto a base model that is a three-dimensional model having a U-shape in top view. The image data includes upper jaw image data obtained by photographing the teeth of the upper jaw of the user and lower jaw image data obtained by photographing the teeth of the lower jaw of the user,
The basic model includes an upper jaw basic model corresponding to the upper jaw and a lower jaw basic model corresponding to the lower jaw,
The intraoral camera system according to claim 1, wherein the model generating unit generates the tooth row model by pasting the upper jaw image data to the upper jaw basic model and pasting the lower jaw image data to the lower jaw basic model. The image data includes buccal image data obtained by photographing the tooth from the buccal side, lingual image data obtained by photographing the tooth from the lingual side, and coronal image data obtained by photographing the tooth from the crown side. and
The model generation unit pastes the buccal image data from the buccal side to the basic model, pastes the lingual image data from the lingual side to the basic model, and pastes the basic model from the dental crown side to the dental crown side. The intraoral camera system according to claim 1 or 2, wherein the tooth row model is generated by pasting image data. The intraoral camera system according to any one of claims 1 to 3, wherein the cross-sectional shape of the basic model differs according to the position of the tooth. The intraoral camera system further includes a user information acquisition unit that acquires user information indicating at least one of gender, age group, and race of the user,
The model generating unit selects a basic model to be used from a plurality of basic models based on the user information, and pastes the image data to the selected basic model to create the tooth row model. 5. The intraoral camera system according to any one of 4. The intraoral camera system further comprises:
An identification unit that identifies the type and position of the tooth indicated by the image data from the image data,
The basic model is composed of a plurality of blocks each corresponding to one tooth,
6. The model generation unit according to any one of claims 1 to 5, wherein the image data is pasted on the corresponding position of the basic model based on the type and position of the tooth identified by the identification unit. Intraoral camera system. 7. The intraoral camera system according to claim 6, wherein the identification unit identifies the type and position of the tooth using an estimation model that includes a neural network, receives the image data, and outputs the type and position of the tooth. . The identification unit
Detecting interdental positions from the image data,
generating tooth images, each representing one tooth, based on the detected interdental positions;
identifying the type and position of the tooth shown in the tooth image based on the tooth image;
8. The intraoral camera system according to claim 6 or 7, wherein the tooth image is pasted on the corresponding position of the base model based on the identified tooth type and position. 6. The model generation unit according to any one of claims 1 to 5, wherein connected image data is generated by connecting mutually adjacent teeth from the image data, and the connected image data is pasted on the basic model. Intraoral camera system. an image capturing unit manually capturing an image of teeth in the user's oral cavity to generate image data;
A tooth row model generation method for generating a tooth row model of the user by pasting the image data onto a basic model, which is a three-dimensional model having a U-shape in top view.

Images