JP2020096800A - Identification device, teeth type identification system, identification method, and identification program - Google Patents

Abstract

To provide an identification device capable of identifying a type of teeth precisely, to provide a scanner system having the identification device, and to provide an identification method and an identification program.SOLUTION: An identification device 100 includes: an input unit 1102 to which three-dimensional data including the teeth are inputted; an identification unit 1130 for identifying types of teeth on the basis of the three-dimensional data including features of the teeth inputted from the input unit 1102, and an estimation model 114 including a neural network 1142; and an output unit 1103 for outputting an identification result by the identification unit 1130. The estimation model 114 is learned based on information on teeth corresponding to types of teeth related to the three-dimensional data, and an identification result of types of the teeth using the three-dimensional data.SELECTED DRAWING: Figure 5

Description

The present invention relates to an identification device, a scanner system including the identification device, an identification method, and an identification program.

Conventionally, in the field of dentistry, a three-dimensional scanner incorporating a three-dimensional camera for acquiring a three-dimensional shape of a tooth is known for digitally designing a prosthesis or the like on a computer. For example, Patent Document 1 discloses a technique of recording the shape of a tooth by imaging the tooth using a three-dimensional camera. An operator such as a dentist can record the three-dimensional shape of the tooth to be imaged by using the three-dimensional camera disclosed in Patent Document 1, and the recorded three-dimensional shape of the tooth While confirming the three-dimensional image shown, it is possible to identify the type of the tooth by its own knowledge.

JP 2000-74635 A

As described above, conventionally, the operator has identified the type of the tooth by his/her own knowledge based on the three-dimensional image including the tooth acquired by the three-dimensional camera, but the level of knowledge is different for each operator. Since they are different, there is a problem that the accuracy of the identification result varies depending on the level of knowledge of the operator. For example, the shapes of central incisor and lateral incisor, canine and first premolar, first premolar and second premolar, second premolar and first molar, first molar and second molar, etc. The identification accuracy is not particularly high between similar teeth.

The present invention has been made to solve such a problem, and provides an identification device capable of accurately identifying the type of tooth, a scanner system including the identification device, an identification method, and an identification program. The purpose is to do.

According to the present invention, an identification device for identifying a tooth type is provided. The identification device identifies the type of the tooth based on an input unit to which the three-dimensional data including the tooth is input, and an estimated model including the three-dimensional data including the feature of the tooth input from the input unit and a neural network. And an output unit that outputs the identification result by the identification unit, the estimation model, the tooth information corresponding to the tooth type associated with the three-dimensional data, and the type of the tooth using the three-dimensional data. Learning is performed based on the identification result.

According to the present invention, there is provided a scanner system for acquiring tooth shape information. The scanner system is a three-dimensional scanner that acquires three-dimensional data including teeth using a three-dimensional camera, and an identification that identifies the type of the tooth based on the three-dimensional data including the tooth characteristics acquired by the three-dimensional scanner. The identification device includes an input unit to which the three-dimensional data is input, and the type of the tooth is identified based on the estimation model including the neural network and the three-dimensional data including the features of the tooth input from the input unit. The estimation model includes an identification unit that outputs the identification result of the identification unit, and the estimation model includes tooth information corresponding to the type of tooth associated with the three-dimensional data, and the tooth information using the three-dimensional data. Learning is performed based on the type identification result.

According to the present invention, an identification method for identifying a tooth type is provided. The identification method includes a step of inputting three-dimensional data including a tooth, a step of identifying a type of the tooth based on an estimation model including a three-dimensional data including tooth characteristics and a neural network, and an identification by the identifying step. And outputting the result, the estimation model is learned based on the tooth information corresponding to the tooth type associated with the three-dimensional data and the identification result of the tooth type using the three-dimensional data. ..

According to the present invention, an identification program for identifying the type of tooth is provided. The identification program includes a step of inputting three-dimensional data including a tooth to a computer, a step of identifying the type of the tooth based on an estimated model including the three-dimensional data including a tooth characteristic and a neural network, and an identification step. The step of outputting the identification result by the step of performing, the estimation model, the tooth information corresponding to the tooth type associated with the three-dimensional data, and the identification result of the tooth type using the three-dimensional data and Learned based on.

According to the present invention, it is possible to accurately identify the type of tooth based on the three-dimensional data including the tooth.

It is a schematic diagram which shows the application example of the identification device which concerns on this Embodiment. It is a schematic diagram which shows the whole structure of the system which concerns on this Embodiment. It is a schematic diagram which shows the hardware constitutions of the identification device which concerns on this Embodiment. It is a schematic diagram which shows the hardware constitutions of the server apparatus which concerns on this Embodiment. It is a schematic diagram which shows the functional structure of the identification device which concerns on this Embodiment. It is a schematic diagram for explaining the identification processing by the identification device according to the present embodiment. It is a schematic diagram which shows an example of the tooth used as the identification object in the identification process which concerns on this Embodiment. It is a schematic diagram for explaining the generation of the learning data according to the present embodiment. It is a schematic diagram for demonstrating an example of the learning data set which concerns on this Embodiment. It is a schematic diagram for explaining the generation of a learned model based on the learning data set according to the present embodiment. 5 is a flowchart for explaining an example of a learning process executed by the identification device according to the present embodiment. 7 is a flowchart for explaining an example of a learning process executed by the server device according to the present embodiment. 7 is a flowchart for explaining an example of a service providing process executed by the identification device according to the present embodiment. 9 is a flowchart for explaining an example of a service providing process executed by an identification device according to a modification. It is a schematic diagram for demonstrating generation of the learned model based on the learning data set which concerns on a modification. 9 is a flowchart for explaining an example of a service providing process executed by an identification device according to a modification. It is a schematic diagram for demonstrating an example of the learning data set which concerns on a modification. 9 is a flowchart for explaining an example of a service providing process executed by an identification device according to a modification.

Embodiments of the present invention will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals and the description thereof will not be repeated.

[Application example]
An application example of the identification device 100 according to the present embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic diagram showing an application example of the identification device 100 according to the present embodiment. FIG. 2 is a schematic diagram showing the overall configuration of the system according to the present embodiment.

As shown in FIG. 1, by using the scanner system 10, the user 1 can acquire data of a three-dimensional shape including teeth of the subject 2 (hereinafter, also referred to as “three-dimensional data”). It should be noted that the “user” is any person who uses the scanner system 10, such as an operator such as a dentist, a dental assistant, a teacher or student at a dental university, a dental technician, a technician at a manufacturer, or a worker at a manufacturing factory. May be The “target person” may be any person who is a target of the scanner system 10, such as a patient in a dental clinic or a subject in a dental university.

The scanner system 10 according to the present embodiment includes a three-dimensional scanner 200, an identification device 100, a display 300, and a speaker 400. The three-dimensional scanner 200 acquires three-dimensional data to be scanned by a built-in three-dimensional camera. Specifically, the three-dimensional scanner 200 scans the inside of the oral cavity to obtain, as three-dimensional data, position information (vertical direction, horizontal direction, and height direction) of each of a plurality of points forming a tooth to be scanned. The coordinate of each axis) is acquired using an optical sensor or the like. The identification device 100 generates a three-dimensional image based on the three-dimensional data acquired by the three-dimensional scanner 200, and displays the generated three-dimensional image on the display 300.

For example, the user 1 images the inside of the oral cavity of the target person 2 by the three-dimensional scanner 200 in order to digitally design a prosthesis or the like for compensating for the missing part of the tooth of the target person 2 on the computer, and thus the oral cavity including the tooth. Acquire the three-dimensional data in. Each time the user 1 images the inside of the oral cavity, three-dimensional data is sequentially acquired, and the three-dimensional image of the oral cavity is displayed on the display 300. The user 1 focuses on the insufficient portion of the 3D data while checking the 3D image displayed on the display 300. At this time, the user 1 identifies the type of tooth being scanned or completed scanning based on his/her own knowledge based on the three-dimensional image in which the three-dimensional data including the tooth acquired by the three-dimensional scanner 200 is visualized. However, since the level of knowledge differs for each user 1, the accuracy of the identification result may vary depending on the knowledge of the user 1.

Therefore, the scanner system 10 according to the present embodiment automatically uses AI (Artificial Intelligence) of the identification device 100 to automatically determine the tooth type based on the three-dimensional data acquired by the three-dimensional scanner 200. Is configured to perform the identifying process. The process of identifying the tooth type by the identification device 100 is also referred to as “identification process”.

In addition, "dental type" means the central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar, upper left side of upper jaw Central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar, mandibular right central incisor, lateral incisor, canine, 1st premolar, 2nd premolar, 1st premolar, 2nd premolar, and 3rd premolar, lower left central incisor, lateral incisor, canine, 1st premolar, 2nd premolar, It means the type of each tooth, such as the first molar, the second molar, and the third molar.

Specifically, when the user 1 scans the teeth in the oral cavity of the subject 2 using the three-dimensional scanner 200, the three-dimensional data including the teeth is input to the identification device 100. The identification device 100 executes identification processing for identifying the type of the tooth based on the input three-dimensional data including the tooth feature and the estimation model including the neural network.

The “estimation model” includes a neural network and parameters used by the neural network, and tooth information corresponding to the tooth type associated with the three-dimensional data, and identification of the tooth type using the three-dimensional data. It is optimized (adjusted) by learning based on the result. Specifically, when the three-dimensional data including the tooth is input, the estimation model extracts the tooth feature by a neural network based on the three-dimensional data and estimates the tooth type based on the extracted tooth feature. .. Then, the estimation model does not update the parameters if both match based on the tooth type estimated by itself and the tooth type (tooth information) associated with the input three-dimensional data, while both are If they do not match, the parameters are optimized by updating the parameters so that they match. As described above, the estimation model is learned by optimizing the parameters by using the teacher data including the three-dimensional data that is the input data and the tooth type (teeth information) that is the correct answer data.

The process of learning such an estimation model is also referred to as “learning process”. The estimation model optimized by the learning process is also referred to as a “learned model”. That is, in the present embodiment, the pre-learning estimation model and the learned estimation model are collectively referred to as an “estimation model”, and particularly, the learned estimation model is also referred to as a “learned model”.

"Tooth information" is the upper central right incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar, upper left central incision. Tooth, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar, lower middle central incisor, lateral incisor, canine, first minor Molars, 2nd premolars, 1st molars, 2nd molars, and 3rd molars, mandibular left central incisors, lateral incisors, canines, 1st premolars, 2nd premolars, 1st molars , Second molar, and third molar, etc. In addition, “tooth information” includes 1st assigned to the central incisor, 2nd assigned to the lateral incisor, 3rd assigned to the canine, 4th assigned to the 1st premolar and 2nd small. Numbers assigned to each tooth, such as number 5 assigned to molars, number 6 assigned to first molars, number 7 assigned to second molars, number 8 assigned to third molars, and so on. (For example, the number of teeth commonly used in the dental field). In addition, the “teeth information” may include information on the color assigned to each tooth, or may include information on the symbol assigned to each tooth.

When the identification device 100 performs the identification process using the learned model, the identification result is output to the display 300 and the speaker 400.

The display 300 displays at least one of an image, letters, numbers, icons, and symbols corresponding to the identification result. For example, the display 300 uses the identification result of the tooth by the identification device 100 after the completion of scanning the second molar corresponding to No. 7 on the right side of the lower jaw by the three-dimensional scanner 200, and displays “No. The scan is completed.”, and an image indicating that the scan of the second molar corresponding to No. 7 on the right side of the lower jaw is completed is displayed.

The speaker 400 outputs a sound corresponding to the identification result. For example, the speaker 400 uses the result of tooth identification by the identification device 100 after completing the scan of the second molar corresponding to No. 7 on the right side of the lower jaw by the three-dimensional scanner 200, and says “Completing lower right No. 7”. ", the voice indicating that the scan of the second molar corresponding to No. 7 on the right side of the lower jaw is completed is output.

Further, the identification result by the identification device 100 is output as scan information to the server device 500 arranged in the dental laboratory and the management center together with the three-dimensional data used in the identification process.

For example, as shown in FIG. 2, the scanner system 10 is arranged in each of the locals A to C. For example, the local A and the local B are dental clinics, and an operator or a dental assistant who is the user 1 in the clinic of the dental clinic uses the scanner system 10 to generate a three-dimensional image including the teeth of the patient who is the subject 2. Get the data. Further, the local C is a dental university, and in the dental university, the teacher or the student who is the user 1 acquires the three-dimensional data in the oral cavity of the subject who is the target person 2. The scan information (three-dimensional data, identification result) acquired by each of the locals A to C is output via the network 5 to the server apparatus 500 located at the local D dental laboratory and the management center.

In the dental laboratory, a dental technician or the like creates a prosthesis or the like for compensating for the missing part of the tooth of the subject 2 based on the scan information acquired from each of the local areas A to C. In the management center, the server device 500 accumulates and stores the scan information acquired from each of the local A to C, and holds it as big data.

It should be noted that the server device 500 is not limited to being arranged in a dental clinic in a management center different from the local, and may be arranged in the local. For example, the server device 500 may be arranged in any one of the locals A to C. Further, a plurality of identification devices 100 may be arranged in one local, and further, a server device 500 capable of communicating with the plurality of identification devices 100 may be arranged in the one local. Also, the server device 500 may be realized in the form of a cloud service.

In the dental laboratory, scan information is collected from various places such as local A to C. Therefore, the scan information held in the dental laboratory may be transmitted to the management center via the network 5, or the removable disk 550 such as a CD (Compact Disc) and a USB (Universal Serial Bus) memory. May be sent to the management center via.

It should be noted that the scan information may be sent to the management center via the removable disk 550 from each of the local A to C instead of via the network 5. Further, the scan information may be sent to each other via the network 5 or the removable disk 550 between the locals A to C.

The identification devices 100 of the respective localities A to C hold their own estimation models, and identify the types of teeth using the estimation models held by themselves during the identification processing. The identification device 100 of each of the local A to C learns its own estimation model by its own learning process to generate a learned model. Further, in the present embodiment, the server device 500 also holds the estimation model. The server device 500 learns the estimation model by a learning process using the identification device 100 of each local A to C and the scan information acquired from the dental laboratory, thereby generating a learned model, and generating a learned model of each local A to C. The learned model is distributed to the identification device 100. In the present embodiment, both the identification devices 100 of the local A to C and the server device 500 execute the learning process, but only the identification device 100 of the local A to C executes the learning process. Alternatively, only the server device 500 may execute the learning process. In the case where only the server device 500 executes the learning process, the estimation model (learned model) held by the identification device 100 of each local A to C is common to the identification devices 100 of each local A to C. Be converted.

Further, the server device 500 may have a function of identification processing in the identification device 100. For example, each local A to C transmits the acquired three-dimensional data to the server apparatus 500, and the server measure 500 identifies the tooth type in each based on the respective three-dimensional data received from each local A to C. The result may be calculated. Then, the server device 500 may transmit the respective identification results to the respective local ACs, and the respective local ACs may output the identification result received from the server device 500 to a display or the like. In this way, each local A to C and the server device 500 may be configured in the form of a cloud service. In this way, as long as the server device 500 holds the estimation model (learned model), each local AC can obtain the identification result without holding the estimation model (learned model). it can.

As described above, according to the scanner system 10 of the present embodiment, the type of tooth is automatically identified based on the three-dimensional data acquired by the three-dimensional scanner 200 using the AI of the identification device 100. It By using the AI, it is possible to find the tooth characteristics obtained from the knowledge of the user 1. Further, it is also possible to find out the tooth features that cannot be extracted by the user 1, so that the user 1 can accurately identify the tooth type without relying on his or her own knowledge.

[Hardware configuration of identification device]
An example of the hardware configuration of the identification device 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a schematic diagram showing a hardware configuration of the identification device 100 according to the present embodiment. The identification device 100 may be realized by, for example, a general-purpose computer or a computer dedicated to the scanner system 10.

As shown in FIG. 3, the identification device 100 includes a scanner interface 102, a display interface 103, a speaker interface 104, a peripheral device interface 105, a network controller 106, and a media reading device 107 as main hardware elements. , A PC display 108, a memory 109, a storage 110, and an arithmetic unit 130.

The scanner interface 102 is an interface for connecting the three-dimensional scanner 200, and realizes data input/output between the identification device 100 and the three-dimensional scanner 200.

The display interface 103 is an interface for connecting the display 300, and realizes input/output of data between the identification device 100 and the display 300. The display 300 is composed of, for example, an LCD (Liquid Crystal Display) or an organic ELD (Electroluminescence) display.

The speaker interface 104 is an interface for connecting the speaker 400, and realizes data input/output between the identification device 100 and the speaker 400.

The peripheral device interface 105 is an interface for connecting peripheral devices such as a keyboard 601 and a mouse 602, and realizes data input/output between the identification device 100 and the peripheral devices.

The network controller 106 exchanges data with each of the devices arranged in the dental laboratory, the server device 500 arranged in the management center, and the other identification devices 100 arranged locally through the network 5. Send and receive. The network controller 106 corresponds to any communication method such as Ethernet (registered trademark), wireless LAN (Local Area Network), and Bluetooth (registered trademark).

The media reading device 107 reads various data such as scan information stored in the removable disk 550.

The PC display 108 is a display dedicated to the identification device 100. The PC display 108 is composed of, for example, an LCD or an organic EL display. Although the PC display 108 is separate from the display 300 in the present embodiment, it may be shared with the display 300.

The memory 109 provides a storage area for temporarily storing a program code, a work memory, etc. when the arithmetic unit 130 executes an arbitrary program. The memory 109 is composed of, for example, a volatile memory device such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory).

The storage 110 provides a storage area for storing various data necessary for identification processing, learning processing, and the like. The storage 110 is configured by a non-volatile memory device such as a hard disk or SSD (Solid State Drive).

The storage 110 includes scan information 112, an estimation model 114 (learned model 114a), a learning data set 116, color classification data 118, profile data 119, an identification program 120, a learning program 121, and An OS (Operating System) 127 is stored.

The scan information 112 includes the three-dimensional data 122 acquired by the three-dimensional scanner 200 and the identification result 124 by the identification processing executed based on the three-dimensional data 122. The identification result 124 is stored in the storage 110 in association with the three-dimensional data 122 used in the identification process. The learning data set 116 is a group of learning data used for the learning process of the estimation model 114. The color classification data 118 is data used for generating the learning data set 116 and learning processing. The profile data 119 is attribute information related to the target person 2 and is data (for example, medical record information) in which profiles of the target person 2 such as age, sex, race, height, weight, and place of residence are summarized. is there. The identification program 120 is a program for executing identification processing. The learning program 121 is a program for executing the learning process of the estimation model 114, and a part thereof includes a program for executing the identification process.

The arithmetic unit 130 is an arithmetic main body that executes various programs such as identification processing and learning processing by executing various programs, and is an example of a computer. The arithmetic unit 130 is composed of, for example, a CPU (Central Processing Unit) 132, an FPGA (Field-Programmable Gate Array) 134, and a GPU (Graphics Processing Unit) 136.

[Hardware configuration of server device]
An example of the hardware configuration of the server device 500 according to the present embodiment will be described with reference to FIG. FIG. 4 is a schematic diagram showing the hardware configuration of the server device 500 according to this embodiment. The server device 500 may be realized by, for example, a general-purpose computer or a computer dedicated to the scanner system 10.

As shown in FIG. 4, the server device 500 has, as main hardware elements, a display interface 503, a peripheral device interface 505, a network controller 506, a media reading device 507, a memory 509, a storage 510, and an operation. And a device 530.

The display interface 503 is an interface for connecting the display 350, and realizes input/output of data between the server device 500 and the display 350. The display 350 is composed of, for example, an LCD or an organic ELD display.

The peripheral device interface 505 is an interface for connecting peripheral devices such as a keyboard 651 and a mouse 652, and realizes data input/output between the server device 500 and the peripheral devices.

The network controller 506 transmits/receives data to/from each of the identification device 100 arranged locally and the device arranged in the dental laboratory via the network 5. The network controller 506 may support any communication method such as Ethernet (registered trademark), wireless LAN, and Bluetooth (registered trademark).

The media reading device 507 reads various data such as scan information stored in the removable disk 550.

The memory 509 provides a storage area for temporarily storing a program code, a work memory, etc. when the arithmetic unit 530 executes an arbitrary program. The memory 509 is composed of, for example, a volatile memory device such as DRAM or SRAM.

The storage 510 provides a storage area for storing various data necessary for learning processing and the like. The storage 510 is composed of, for example, a non-volatile memory device such as a hard disk or SSD.

The storage 510 stores scan information 512, an estimation model 514 (learned model 514a), a learning data set 516, color classification data 518, profile data 519, a learning program 521, and an OS 527.

The scan information 512 includes the three-dimensional data 522 acquired from the identification device 100 and the dental laboratory which are locally arranged via the network 5, and the identification result 524 by the identification process executed based on the three-dimensional data 522. .. The identification result 524 is stored in the storage 510 in association with the three-dimensional data 522 used in the identification process. The learning data set 516 is a group of learning data used for the learning process of the estimation model 514. The color classification data 518 is data used for the generation and learning processing of the learning data set 516. The profile data 519 is attribute information about the target person 2 and is data (for example, medical record information) in which profiles of the target person 2 such as age, sex, race, height, weight, and place of residence are summarized. .. The learning program 521 is a program for executing the learning process of the estimation model 514, and a part thereof includes a program for executing the identification process.

It should be noted that the estimation model 514 (learned model 514a) is transmitted to the local identification device 100, so that the identification device 100 holds the estimation model 114 (learned model 114a).

The arithmetic unit 530 is an arithmetic entity that executes various processes such as a learning process by executing various programs, and is an example of a computer. The arithmetic unit 530 includes, for example, a CPU 532, an FPGA 534, a GPU 536, and the like.

[Identification processing by the identification device]
An example of the identification processing by the identification device 100 according to the present embodiment will be described with reference to FIGS. 5 to 7. FIG. 5 is a schematic diagram showing the functional configuration of the identification device 100 according to the present embodiment. FIG. 6 is a schematic diagram for explaining the identification processing by the identification device 100 according to the present embodiment. FIG. 7 is a schematic diagram showing an example of a tooth to be identified in the identification processing according to the present embodiment. In FIG. 7, the tooth to be scanned by the three-dimensional scanner 200 is represented by a diagram.

As illustrated in FIG. 5, the identification device 100 includes an input unit 1102, a profile acquisition unit 1119, an identification unit 1130, and an output unit 1103 as functional units related to identification processing. Each of these functions is realized by the arithmetic unit 130 of the identification device 100 executing the OS 127 and the identification program 120.

The three-dimensional data acquired by the three-dimensional scanner 200 is input to the input unit 1102. The profile acquisition unit 1119 acquires the profile data 119 of the target person 2. The identification unit 1130 uses the estimation model 114 (learned model 114a) based on the three-dimensional data input to the input unit 1102 and the profile data 119 of the target person 2 acquired by the profile acquisition unit 1119. Perform identification processing to identify the type.

The estimation model 114 includes a neural network 1142 and parameters 1144 used by the neural network 1142. The parameter 1144 includes a weighting coefficient used for calculation by the neural network 1142 and a judgment value used for judgment of discrimination. The output unit 1103 outputs the identification result of the identification unit 1130 to the display 300, the speaker 400, and the server device 500.

Here, as shown in FIG. 6, the three-dimensional data input to the input unit 1102 includes three-dimensional position information at each tooth point and color information at each tooth point. Position information is used in the identification process. The position information includes the coordinates of the absolute position in three dimensions based on a predetermined position. For example, the position information includes an X-axis (for example, a lateral axis of the tooth), a Y-axis (for example, a longitudinal axis of the tooth), and a Z-axis (for example, the tooth) with the center position at each point of the tooth as an origin. The coordinate of the absolute position in each axis is included. The position information is not limited to the coordinates of the absolute position in three dimensions based on the predetermined position, and may include the coordinates of the relative position in three dimensions indicating the distance from the adjacent point, for example.

Here, as shown in FIG. 7, when the tooth to be scanned by the three-dimensional scanner 200 is the upper incisor, the obtained three-dimensional image shows the upper lip side portion, the palate side portion, and the incisal edge side. The user 1 scans the inside of the oral cavity of the subject 2 so as to include at least the image of the region. When the teeth to be scanned by the three-dimensional scanner 200 are the maxillary canine and molars, the three-dimensional image obtained should include at least images of the buccal region, the palate region, and the occlusal region. The user 1 scans the inside of the oral cavity of the target person 2. When the tooth to be scanned by the three-dimensional scanner 200 is the lower jaw incisor, the obtained three-dimensional image includes at least images of the lower lip side portion, the lingual side portion, and the incisal side portion. The user 1 scans the inside of the oral cavity of the subject 2. When the teeth to be scanned by the three-dimensional scanner 200 are the lower canine and the molars, the user can make the obtained three-dimensional image include at least images of the buccal region, the lingual region, and the occlusal region. 1 scans the inside of the oral cavity of the subject 2.

In general, the tooth of the subject 2 has different shapes and sizes depending on its type. For example, in the case of an upper incisor, the upper labial surface is generally U-shaped, whereas in the upper canine, the buccal surface is generally pentagonal. Each tooth is characteristic in shape and size according to its type, and the identifying unit 1130 uses the estimation model 114 based on the three-dimensional data in which these characteristic shapes and sizes are digitized. The type of tooth corresponding to the three-dimensional data is identified.

As shown in FIG. 6, the estimation model 114 includes a neural network 1142. In the neural network 1142, the position information value included in the three-dimensional data input to the input unit 1102 is input to the input layer. Then, in the neural network 1142, for example, the value of the input position information is multiplied by a weighting coefficient or a predetermined bias is added by the intermediate layer, and a calculation by a predetermined function is performed. The result is compared with the judgment value. Then, in the neural network 1142, the result of the calculation and the determination is output from the output layer as the identification result. In addition, for the calculation and determination by the neural network 1142, any method may be used as long as it can identify the tooth based on the three-dimensional data.

In the neural network 1142 of the estimation model 114, the intermediate layer has a multi-layer structure, so that processing by deep learning is performed. In the present embodiment, for example, VoxNet, 3D is used as the identification program 120 that performs identification processing specialized for three-dimensional images.
ShapeNets, Multi-View CNN, RotationNet, OctNet, FusionNet, PointNet, PointNet++, SSCNet, and MarrNet are used, but other programs may be used. Moreover, an existing one may be applied to the mechanism of the neural network 1142.

With such a configuration, when the three-dimensional data corresponding to the three-dimensional image including a plurality of teeth is input, the identification device 100 estimates a model of each of the plurality of teeth based on the three-dimensional data. It is possible to identify each type of the plurality of teeth based on the respective characteristics of the plurality of extracted teeth by extracting using the neural network 1142 of 114. Further, as shown in FIG. 6, not only the tooth to be identified, but also three-dimensional data including adjacent teeth is input to the identification device 100, so that the neural network 1142 of the estimation model 114 causes the adjacent tooth. The feature of the tooth can be extracted in consideration of the relationship with the shape of the tooth. The identification device 100 can extract not only generally recognized tooth characteristics but also generally unrecognized tooth characteristics, and thereby can accurately identify the tooth type. ..

The neural network included in the estimation model 514 held by the server device 500 has the same configuration as the neural network 1142 included in the estimation model 114 shown in FIG.

[Learning data generation]
An example of generation of the learning data set 116 will be described with reference to FIGS. 8 and 9. FIG. 8 is a schematic diagram for explaining the generation of learning data according to this embodiment. FIG. 9 is a schematic diagram for explaining an example of the learning data set 116 according to the present embodiment.

As shown in FIG. 8, first, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). The three-dimensional data acquired by the three-dimensional scanner 200 includes three-dimensional position information at each tooth point corresponding to the three-dimensional data and color information (RGB values) at each tooth point. When a three-dimensional image is generated based on the three-dimensional data acquired by the three-dimensional scanner 200, a three-dimensional image including teeth with actual colors is generated as shown in FIG. 8A.

Next, a noise removal process is performed in preparation for the color coding process of each tooth, which will be described later. For example, in the present embodiment, the three-dimensional image corresponding to the three-dimensional data is grayscaled (STEP 2). The grayscale conversion of the three-dimensional image is performed by the user 1 (in this case, a technician of a maker who generates learning data or an operator of a manufacturing factory). When the three-dimensional image is grayscaled, a three-dimensional image including grayscaled teeth is generated as shown in FIG. 8B. Further, in accordance with the gray scale of the three-dimensional image, the color information (RGB value) at each point of the tooth corresponding to the three-dimensional data is changed to the value corresponding to the gray scale.

Next, each tooth is color-coded by applying a predetermined color to each tooth included in the three-dimensional image corresponding to the three-dimensional data (STEP 3). For example, as shown in FIG. 9, the color classification data 118 held by the identification device 100 is provided for each site in the oral cavity such as the lower jaw left side, the lower jaw right side, the upper jaw left side, and the upper jaw right side. In FIG. 9, the color classification data 118 corresponding to the left side of the lower jaw is shown. In each color classification data 118, a tooth number generally used in the dental field and predetermined color information are assigned to each tooth type.

For example, the second molar is assigned tooth number 7 and red color information. The first molar is assigned with tooth number 6 and green as color information. For the second premolar, tooth number 5 is assigned and blue is assigned as color information. As described above, in each color classification data 118, the tooth number and the color information are assigned in advance to each tooth type.

The application of the color to each tooth is performed by the user 1 (a technician of the manufacturer, an operator of the manufacturing factory, or the like). Specifically, the user 1 identifies the type of each tooth included in the three-dimensional image based on his/her own knowledge, and identifies the color corresponding to the identified type of the tooth with reference to the color classification data 118, Apply the identified color to the image of the tooth.

For example, when the user 1 identifies that the tooth included in the three-dimensional image is the second molar, the user 1 applies red to the image of the tooth. When the tooth included in the three-dimensional image is identified as the first molar, green is applied to the image of the tooth. When the predetermined color is applied to each tooth included in the three-dimensional image, the predetermined color is applied to each tooth as shown in FIGS. 8(c) and 9(d). A three-dimensional image is generated. For easy understanding, each color is hatched in the drawings.

Further, the color information (RGB value) at each point of the tooth corresponding to the three-dimensional data is changed to a value corresponding to the color applied to each tooth according to the color classification of each tooth. For example, for each position coordinate of the second molar applied to red, the color information (RGB value) is "2550,000", and for each position coordinate of the first molar applied to green, The color information (RGB value) is "000255000", and the color information (RGB value) is "00000255" for each position coordinate of the second premolar coated in blue. That is, predetermined color information (RGB value) is associated with each tooth point corresponding to the three-dimensional data.

When the predetermined color information is associated with each tooth, the three-dimensional data comes to include position information and color information corresponding to the applied color. Used as learning data. That is, in the learning data according to the present embodiment, the color information corresponding to the tooth type is associated (labeled) with the position information referred to in the identification process. Furthermore, color information is associated with the three-dimensional data so that the ranges of the plurality of teeth corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each position information corresponding to each tooth. A group of such learning data is held in the identification device 100 as a learning data set 116.

As described above, when the learning data is generated, the user 1 applies many colors to each tooth included in the three-dimensional image and labels the correct answer data, which has many advantages. For example, when labeling is performed using only letters or symbols, the user 1 has difficulty in recognizing the range of each tooth, but when labeling is performed by color coding, the user 1 identifies the tooth to be labeled and the tooth adjacent to the tooth. It is possible to easily recognize the boundary between the teeth and the boundary between the tooth and the gingiva to be labeled by applying the color. Further, the user 1 applies the color while checking the three-dimensional image from various angles at the time of labeling. Even when the angle of the viewpoint is changed, to what extent the application is completed for the tooth during the labeling work. Will be easier to recognize.

In the present embodiment, the user 1 manually applies a color to each tooth included in the three-dimensional image based on his/her own knowledge, but it is also possible to supplement a part of the operation with software. .. For example, the boundary between the tooth to be labeled and the tooth adjacent to the tooth, and the boundary between the tooth to be labeled and the gingiva may be specified by edge detection. It is possible to extract only the tooth to be labeled.

Note that the generation of the learning data set 116 shown in FIGS. 8 and 9 is also applicable to the generation of the learning data set 516 held by the server device 500. For example, the learning data set 116 illustrated in FIG. 9 may be applied to the learning data set 516 held by the server device 500, or the color classification data 118 illustrated in FIG. 9 may be color classified by the server device 500. It may be applied to the data 518.

[Trained model generation]
An example of generation of the learned model 114a will be described with reference to FIG. FIG. 10 is a schematic diagram for explaining generation of the learned model 114a based on the learning data set 116 according to the present embodiment.

As shown in FIG. 10, the learning data set 116 can be classified into each category based on the profile of the target person 2 who is the scan target when the learning data set 116 is generated. For example, age (minors, active generation, elderly), gender (male, female), race (Asian, Western, African), height (less than 150 cm, 150 or more), weight (less than 50 kg, 50 kg) It is possible to assign the learning data set generated from the three-dimensional data including the teeth of the relevant subject 2 to each of the above) and the place of residence (resident in Japan, living outside Japan). The stratification of each category can be set as appropriate. For example, regarding the age, for each predetermined age difference (every 3 years in this case), specifically, 0 to 3 years old, 4 years to 6 years old, 7 years to 9 years old, and so on. Can be stratified.

The identification device 100 generates a learned model 114a by learning the estimation model 114 using a plurality of learning data sets 116a to 116o that can be classified into each category. Although the learning data may overlap depending on how the categories are classified, if the learning data overlaps, the estimation model 114 may be trained using only one of the learning data. ..

Generally, the shape of a tooth has different characteristics depending on its genetics or living environment, such as age, sex, race, height, weight, and place of residence. For example, an adult permanent tooth is generally larger than a child's milk tooth, and their shapes are different from each other. Further, in general, male teeth are larger than female teeth, and their shapes are different. In addition, in general, European teeth tend to have sharp tips so that hard meat or bread can be easily bitten, whereas Japanese teeth have sharp tips so that soft rice or vegetables can be easily ground. Tends to be smooth. Therefore, as in the present embodiment, if the learning process is executed based on the profile data, it is possible to generate a learned model that can identify the type of tooth in consideration of genetics or living environment.

The generation of the learned model 114a shown in FIG. 10 is also applicable to the generation of the learned model 514a held by the server device 500. For example, the learning data sets 116a to 116o shown in FIG. 10 may be applied to the learning data set 516 held by the server apparatus 500, or the estimation model 114 shown in FIG. 10 may be estimated by the server apparatus 500. It may be applied to the model 514.

[Learning process of identification device]
The learning process performed by the identification device 100 will be described with reference to FIG. 11. FIG. 11 is a flowchart for explaining an example of the learning process executed by the identification device 100 according to the present embodiment. Each step shown in FIG. 11 is realized by the arithmetic unit 130 of the identification apparatus 100 executing the OS 127 and the learning program 121.

As shown in FIG. 11, the identification device 100 selects the learning data used for learning from the learning data set 116 (S2). Specifically, the identification device 100 selects one or a plurality of pieces of learning data from the learning data set 116 included in the learning data set group shown in FIG. 10. The identification device 100 is not limited to automatically selecting the learning data, and may use the learning data selected by the user 1 for the learning process.

The identification device 100 inputs the position information of the three-dimensional data included in the selected learning data and the profile data of the target person 2 who is the scan target when generating the learning data to the estimation model 114 (S4). ). At this time, the correct answer data labeled as the three-dimensional data is not input to the identification device 100. The identification device 100 executes an identification process of identifying the type of the tooth using the estimation model 114 based on the tooth feature corresponding to the three-dimensional data (S6). In the identification processing, the identification device 100 identifies the type of the tooth using the estimation model 114 based on the profile data in addition to the three-dimensional data.

The identification device 100 updates the parameter 1144 of the estimation model 114 based on the error between the identification result of the tooth type identified by the identification process and the correct answer data corresponding to the learning data used in the learning process (S8).

For example, the identification device 100 estimates the color information corresponding to the specific tooth as a result of the identification based on the position information of the specific tooth. The identification device 100 compares the color information (correct answer data) corresponding to the specific tooth included in the learning data with the color information estimated by itself, and if they match, maintains the parameter 1144 of the estimation model 114. If the answer is incorrect, the parameter 1144 of the estimation model 114 is updated so that the two match.

Alternatively, the identification device 100 estimates the color information corresponding to the specific tooth as a result of the identification based on the position information of the specific tooth, and based on the color classification data 118, the type of the tooth and the tooth corresponding to the color information. Specify the number (correct answer data). The identification device 100 determines the tooth type and tooth number (correct answer data) assigned to the color information corresponding to the specific tooth included in the learning data, and the tooth type and tooth number estimated by itself. The parameters 1144 of the estimation model 114 are maintained if they are compared with each other, and the parameters 1144 of the estimation model 114 are updated so that they match each other if they are incorrect.

Next, the identification device 100 determines whether or not learning is performed based on all learning data (S10). If the identification device 100 has not learned based on all the learning data (NO in S10), the identification device 100 returns to the process of S2.

On the other hand, when the identification device 100 learns based on all the learning data (YES in S10), the learned estimation model 114 is stored as the learned model 114a (S12), and this process ends.

In this way, the identification device 100 identifies tooth information (color information, tooth name, tooth number, etc.) corresponding to the tooth type associated with the three-dimensional data included in the learning data as correct data. The learned model 114a can be generated by learning the estimation model 114 based on the identification result of the tooth type using the three-dimensional data by the processing.

Further, in the learning process, the identification device 100 learns the estimation model 114 by considering profile data in addition to the learning data, and thus can generate the learned model 114a in consideration of the profile of the subject 2.

[Learning process of server device]
The learning process executed by the server device 500 will be described with reference to FIG. FIG. 12 is a flowchart for explaining an example of the learning process executed by server device 500 according to the present embodiment. Each step shown in FIG. 12 is realized by the arithmetic unit 530 of the server apparatus 500 executing the OS 527 and the learning program 521.

As shown in FIG. 12, the server apparatus 500 selects the learning data used for learning from the learning data set (S502). Here, the learning data may be generated using big data accumulated and stored by the server device 500. For example, the server device 500 uses the three-dimensional data included in the scan information acquired from the identification devices 100 and the dental laboratory of each of the local A to C to generate learning data, and the generated learning data. May be used to execute the learning process. The server device 500 is not limited to automatically selecting the learning data, and may use the learning data selected by the user 1 for the learning process.

The server device 500 inputs the three-dimensional data (positional information) included in the selected learning data and the profile data of the target person 2 who is the scan target when generating the learning data to the estimation model 514 ( S504). At this time, the correct answer data labeled as the three-dimensional data is not input to the server device 500. The server device 500 executes the identification process for identifying the type of the tooth using the estimation model 514 based on the tooth feature corresponding to the three-dimensional data (S506). In the identification process, the server device 500 identifies the type of the tooth using the estimation model 514 based on the profile data in addition to the three-dimensional data.

The server device 500 updates the parameters of the estimation model 514 based on the error between the identification result of the tooth type identified by the identification process and the correct answer data corresponding to the learning data used for learning (S508).

For example, the server device 500 estimates the color information corresponding to the specific tooth as a result of the identification based on the position information of the specific tooth. The server device 500 compares the color information (correct answer data) corresponding to the specific tooth included in the learning data set with the color information estimated by itself, and maintains the parameters of the estimation model 514 if they match. If the answer is incorrect, the parameters of the estimation model 514 are updated so that the two match.

Alternatively, the server device 500 estimates the color information corresponding to the specific tooth as a result of the identification based on the position information of the specific tooth, and based on the color classification data 518, the type or tooth of the tooth corresponding to the color information. Specify the number (correct answer data). The server device 500 stores the tooth type and tooth number (correct answer data) assigned to the color information corresponding to the specific tooth included in the learning data set, and the tooth type and tooth number estimated by itself. And the parameters of the estimation model 514 are maintained if they match, while the parameters of the estimation model 514 are updated so that they match if they are incorrect.

Next, the server device 500 determines whether or not learning has been performed based on all learning data (S510). If the server device 500 has not learned all the learning data (NO in S510), the process returns to S502.

On the other hand, when learning is performed based on all the learning data (YES in S510), the server device 500 stores the learned estimation model 514 as a learned model 514a (S512). After that, the server device 500 transmits the generated learned model 514a to each local identification device 100 (S514), and ends this processing.

As described above, the server device 500 uses the tooth information (color information, tooth name, tooth number) corresponding to the tooth type associated with the three-dimensional data included in the learning data as correct data, and performs the identification process. The learned model 514a can be generated by learning the estimation model 514 based on the identification result of the tooth type using the three-dimensional data.

Further, in the learning process, the server device 500 learns the estimation model 514 in consideration of the profile data in addition to the learning data, and thus can generate the learned model 514a in consideration of the profile of the subject 2.

Further, since the server device 500 uses the three-dimensional data included in the scan information acquired from the identification devices 100 and the dental laboratory as the learning data used for the learning process, the identification device 100. The learning process can be executed based on a larger amount of learning data than the learning process executed for each time, and the learned model 514a that can identify the tooth type with higher accuracy can be generated. ..

[Identification device service provision processing]
The service providing process performed by the identification device 100 will be described with reference to FIG. 13. FIG. 13 is a flowchart for explaining an example of a service providing process executed by identification device 100 according to the present embodiment. Each step illustrated in FIG. 13 is realized by the arithmetic unit 130 of the identification device 100 executing the OS 127 and the identification program 120.

As shown in FIG. 13, the identification device 100 determines whether or not the start condition of the service providing process is satisfied (S42). The start condition may be satisfied, for example, when the power of the three-dimensional scanner 200 is turned on, or when the mode is switched to a mode corresponding to the service providing process after the power of the three-dimensional scanner 200 is turned on. You may. Alternatively, the start condition may be satisfied when the start switch is operated after an icon (for example, AI assist icon) corresponding to the service providing process is operated and the icon is in a blinking state. Further, the start condition may be satisfied when a certain amount of three-dimensional data is acquired. The starting condition may be satisfied if any action is performed on the three-dimensional scanner 200.

If the start condition is not satisfied (NO in S42), the identification device 100 ends this process. On the other hand, if the start condition is satisfied (YES in S42), the identification device 100 determines whether three-dimensional data has been input (S44). For example, the identification device 100 determines whether or not a sufficient amount of three-dimensional data has been input to perform the identification process. If a sufficient amount of three-dimensional data has not been input (NO in S44), the identification device 100 repeats the process of S44.

On the other hand, when a sufficient amount of three-dimensional data is input (YES in S44), the identification device 100 determines whether or not the user 1 has input profile data of the target person 2 (S46). When the profile data is not input (NO in S46), the identification device 100 inputs the three-dimensional data (positional information) into the learned model 114a (S48). On the other hand, when the profile data is input (YES in S46), the identification device 100 inputs the three-dimensional data (positional information) and the profile data into the learned model 114a (S50). The learned model used at this time is not limited to the learned model 114a generated by the identification device 100 in the learning process shown in FIG. 11, and the learned model generated by the server device 500 in the learning process shown in FIG. It may be 514a.

After S48 and S50, the identification device 100 executes an identification process for identifying the type of the tooth using the learned model 114a based on the tooth feature corresponding to the three-dimensional data (S52). At this time, when the profile data is input to the learned model 114a in S50, the identification device 100 identifies the tooth type using the learned model 114a based on the profile data in addition to the three-dimensional data. In this case, it is possible to identify the tooth type more accurately than to identify the tooth type using the learned model 114a based on only the three-dimensional data.

After that, the identification device 100 outputs the identification result obtained by the identification process to the display 300, the speaker 400, the server device 500, and the like (S54), and ends this process.

As described above, the identification device 100 identifies the type of the tooth using the learned model 114a based on the tooth feature corresponding to the input three-dimensional data, and therefore the type of the tooth depends on the knowledge of the user himself. It is possible to identify the type of tooth more accurately than to identify.

Furthermore, since the identification device 100 identifies the tooth type in consideration of the profile data in addition to the input three-dimensional data in the identification processing, the tooth type can be identified more accurately.

[Main configuration]
As described above, the present embodiment includes the following disclosures.

The identification device 100 includes an input unit 1102 to which three-dimensional data including teeth is input, and an estimation model 114 (learned model 114a) including three-dimensional data including tooth features input from the input unit 1102 and a neural network 1142. The estimation model 114 includes an identification unit 1130 that identifies the tooth type based on the above, and an output unit 1103 that outputs the identification result by the identification unit 1130, and the estimation model 114 includes the tooth corresponding to the tooth type associated with the three-dimensional data. Learning is performed based on the information and the identification result of the tooth type using the three-dimensional data.

This allows the user 1 to convert the three-dimensional data including the teeth into the neural network 114.
By inputting to the estimation model 114 (learned model 114a) including 2, it is possible to identify the tooth type, and therefore it is possible to accurately determine the tooth type rather than relying on the knowledge of the user to identify the tooth type. The type can be identified.

The learning of the estimation model 114 may be realized by learning the estimation model 514 executed by the server device 500.

At least three-dimensional data corresponding to a plurality of teeth and gums adjacent to each other in the oral cavity are input to the input unit 1102, and the identification unit 1130 calculates the plurality of data based on the three-dimensional data including the respective features of the plurality of teeth. Identify each type of tooth.

Accordingly, the user 1 inputs the three-dimensional data corresponding to a plurality of adjacent teeth and gums in the oral cavity into the estimation model 114 (learned model 114a) including the neural network 1142, thereby Since each type can be identified, the type of tooth can be identified more accurately and smoothly than when each type of tooth is identified by relying on the knowledge of the user himself. Further, the user 1 can extract the tooth feature by the estimation model 114 including the neural network 1142 in consideration of the relationship with the shape of the adjacent tooth, and thus can accurately identify the tooth type.

As shown in FIG. 6, the three-dimensional data includes three-dimensional position information on each of a plurality of points forming a tooth corresponding to the three-dimensional data.

Thereby, the user 1 inputs the three-dimensional position information at each of the plurality of points forming the tooth corresponding to the three-dimensional data into the estimation model 114 (learned model 114a) including the neural network 1142, It is possible to identify the type of the tooth.

As shown in FIG. 6, the position information includes absolute position coordinates with reference to a predetermined position.

As a result, the user 1 uses the estimation model 114 (learned already) including the neural network 1142 to calculate the coordinates of the absolute position based on the predetermined position as the three-dimensional position information at each tooth point corresponding to the three-dimensional data. By inputting into the model 114a), the type of the tooth can be identified.

As shown in FIG. 7, when the tooth corresponding to the three-dimensional data input to the input unit 1102 is the upper jaw incisor, the three-dimensional image corresponding to the three-dimensional data is a region on the upper lip side and a palate side. When the teeth corresponding to the three-dimensional data input to the input unit 1102 include the maxillary canine and molars, the three-dimensional image corresponding to the three-dimensional image is a cheek. If the tooth corresponding to the three-dimensional data input to the input unit 1102 is an incisor of the lower jaw including at least images of the side portion, the palate side portion, and the occlusal portion, the tertiary corresponding to the three-dimensional data The original image includes at least images of the lower lip side portion, the lingual side portion, and the incisal side portion, and when the teeth corresponding to the three-dimensional data input to the input unit 1102 are the lower canine and molar teeth, The three-dimensional image corresponding to the three-dimensional data includes at least images of the buccal region, the lingual region, and the occlusal region.

Thus, the user 1 uses the estimation model 114 (learned model 114a) including the neural network 1142 for each of the upper incisor, the upper canine and molars, the lower incisor, the lower canine and molars. The type of can be identified.

As shown in FIGS. 1 and 5, the output unit 1103 outputs the identification result by the identification unit 1130 to the display 300, and the display 300 displays the image, the character, the number, the icon, and the image corresponding to the identification result by the identification unit 1130. Display at least one of the symbols.

As a result, the image corresponding to the identification result by the estimation model 114 (learned model 114a) including the neural network 1142 is displayed on the display 300, so that the user 1 can intuitively recognize the identification result, which is convenient. The property is improved.

As shown in FIGS. 1 and 5, the output unit 1103 outputs the identification result by the identification unit 1130 to the speaker 400, and the speaker 400 outputs the sound corresponding to the identification result by the identification unit 1130.

Thereby, the voice corresponding to the identification result by the estimation model 114 (learned model 114a) including the neural network 1142 is output from the speaker 400, so that the user 1 can intuitively recognize the identification result, which is convenient. The property is improved.

As shown in FIGS. 1 and 5, the output unit 1103 outputs the identification result by the identification unit 1130 to the server device 500, and the server device 500 accumulates and stores the identification result by the identification unit 1130.

Thereby, since the server device 500 accumulates and stores the identification result to form big data, the user 1 causes the server device 500 to perform the learning process using such big data, for example. Thus, it is possible to generate a learned model that can more accurately identify the tooth type.

As illustrated in FIG. 5, the estimation model 114 includes at least one of a weighting coefficient and a determination value as the parameter 1144 used by the neural network 1142. The estimation model 114 includes the tooth information and the identification result by the identification unit 1130. Learning is performed by updating the parameter 1144 based on

Accordingly, the user 1 can generate the learned model 114a that can more accurately identify the tooth type by updating the parameter 1144 of the estimation model 114.

As shown in FIG. 9, the tooth information includes at least one of color, letters, numbers, and symbols associated with the type of tooth corresponding to the three-dimensional data.

Accordingly, the user 1 can generate the learned model 114a that can more accurately identify the tooth type based on the color, the letters, the numbers, the symbols, and the like associated with the tooth type. ..

As shown in FIG. 9, the tooth information is associated with the three-dimensional data so that each range of a plurality of teeth corresponding to the three-dimensional data can be specified.

This allows the user 1 to specify the range of each of the plurality of teeth based on the tooth information, which improves convenience during labeling.

As shown in FIG. 9, the tooth information is associated with each of a plurality of points forming the tooth corresponding to the three-dimensional data.

As a result, the tooth information is associated with each of the plurality of points forming the tooth corresponding to the three-dimensional data, and thus the user 1 can associate the tooth information with the tooth finely, which is convenient at the time of labeling. Is improved.

The estimation model is learned based on the tooth information and the identification result by the identification unit 1130, as well as the attribute information regarding the target person 2 having the tooth.

Accordingly, the user 1 can learn the estimation model 114 based on the attribute information about the target person 2 in addition to the learning data, and thus can generate the learned model in consideration of the target person 2's attribute information. it can.

As shown in FIG. 10, the attribute information includes at least one of information about the age, sex, race, height, weight, and place of residence of the target person.

This allows the user 1 to learn the estimation model 114 based on at least one of the age, sex, race, height, weight, and place of residence of the subject in addition to the learning data. Therefore, the learned model 114a in which the profile of the target person 2 is taken into consideration can be generated.

The scanner system 10 uses a three-dimensional camera to acquire three-dimensional data including teeth and a three-dimensional scanner 200, and based on the three-dimensional data including the tooth characteristics acquired by the three-dimensional scanner 200, determines the type of the tooth. The identification device 100 includes an identification unit 100 for identification, and the identification device 100 includes an input unit 1102 to which three-dimensional data is input, an estimation model 114 including three-dimensional data including tooth features input from the input unit 1102 and a neural network 1142. The estimation model 114 includes an identification unit 1130 for identifying the type of the tooth based on the (learned model 114a) and an output unit 1103 for outputting the identification result by the identification unit 1130, and the estimation model 114 includes the teeth associated with the three-dimensional data. Is learned based on the tooth information corresponding to the tooth type and the identification result of the tooth type using the three-dimensional data.

Accordingly, the user 1 can identify the type of the tooth by inputting the three-dimensional data including the tooth into the estimation model 114 (learned model 114a) including the neural network 1142, and thus the user's own It is possible to identify the tooth type with higher accuracy than to rely on the knowledge to identify the tooth type.

The identification method is a step of inputting three-dimensional data including teeth (S48, S50) and a step of identifying the tooth type based on the estimation model 114 including the three-dimensional data including tooth characteristics and the neural network 1142. The estimation model 114 includes (S52) and a step (S54) of outputting the identification result of the identification step, and uses the tooth information corresponding to the tooth type associated with the three-dimensional data and the three-dimensional data. The learning is performed based on the identification result of the type of the tooth in question.

Accordingly, the user 1 can identify the type of the tooth by inputting the three-dimensional data including the tooth into the estimation model 114 (learned model 114a) including the neural network 1142, and thus the user's own It is possible to identify the tooth type with higher accuracy than to rely on the knowledge to identify the tooth type.

The identification program 120 is based on the step (S48, S50) of inputting the three-dimensional data including the tooth to the arithmetic device 130 and the estimation model 114 including the three-dimensional data including the tooth characteristic and the neural network 1142. The step of identifying the tooth type (S52) and the step of outputting the identification result of the identifying step (S54) are executed, and the estimation model 114 sets the tooth information corresponding to the tooth type associated with the three-dimensional data. And the result of identifying the type of tooth using the three-dimensional data.

Accordingly, the user 1 can identify the type of the tooth by inputting the three-dimensional data including the tooth into the estimation model 114 (learned model 114a) including the neural network 1142, and thus the user's own It is possible to identify the tooth type with higher accuracy than to rely on the knowledge to identify the tooth type.

[Modification]
The present invention is not limited to the above-described embodiments, and various modifications and applications are possible. Hereinafter, modified examples applicable to the present invention will be described.

(Learning process during service provision process)
As shown in FIG. 13, the identification device 100 according to the present embodiment does not perform the learning process in the service providing process. However, as shown in FIG. 14, the identification device 100a according to the modification provides the service providing process. The learning process may be executed in the process. FIG. 14 is a flowchart for explaining an example of a service providing process executed by the identification device 100a according to the modification. Since the processing of S42 to S54 shown in FIG. 14 is the same as the processing of S42 to S54 shown in FIG. 13, only the processing after S56 will be described in FIG.

As shown in FIG. 14, the identification device 100a executes the service provision learning process after outputting the identification result by the processes of S42 to S54. Specifically, the identification device 100a determines whether or not the correct answer data for error correction is input after S54 (S56). For example, when the type of tooth that is the identification result output in S54 is different from the type of tooth that was actually the scan target, the identification device 100a allows the user 1 to determine the type of tooth that was actually the scan target. By inputting, it is determined whether or not the error is corrected.

When the correct answer data for error correction has not been input (NO in S56), the identification device 100a ends this process. On the other hand, when the correct answer data for error correction is input (YES in S56), the identification device 100a gives a reward based on the identification result and the correct answer data (S58).

For example, the smaller the degree of dissociation between the identification result and the correct answer data, the smaller the negative point as the reward to be given, and the larger the degree of dissociation between the two, the larger the negative point as the reward to be given. Good. Specifically, the identification device 100a gives a minus point having a small value if the tooth output as the identification result and the tooth input as the correct answer data are adjacent to each other, and if both are separated, the Give big negative points. In this way, the identification device 100a gives rewards of different values according to the dissociation degree between the identification result and the correct answer data. The reward is not limited to minus points, and may be plus points.

The identification device 100a updates the parameter 1144 of the learned model 114a based on the given reward (S60). For example, the identification device 100a updates the parameter 1144 of the learned model 114a so that the negative point given as the reward approaches 0. Then, the identification device 100a ends the present process.

As described above, since the identification device 100a according to the modification executes the learning process also in the service providing process, the accuracy of the identification process improves as the user 1 uses it, and the tooth type is identified more accurately. be able to.

(Generation of trained models for each category)
As shown in FIG. 10, the identification device 100 according to the present embodiment trains the estimation model 114 using a learning data set group including a plurality of learning data sets 116a to 116o classified into each category. Thus, although one learned model 114a is generated, as illustrated in FIG. 15, the identification device 100b according to the modified example uses each of the plurality of learning data sets classified into each category as a category. A learned model for each category may be generated by learning the estimation model 114 by using each model. FIG. 15 is a schematic diagram for explaining generation of a learned model based on a learning data set according to the modification.

As shown in FIG. 15, the learning data set 116 is classified and held in each category based on the profile of the target person 2 who is the scan target when the learning data set 116 is generated. For example, learning data sets are assigned to six categories based on age (minors, active generations, seniors) and gender (male, female).

The identification device 100b generates the learned models 114p to 114u for each category by learning the estimation model 114 by using each of the plurality of learning data sets 116p to 116u classified for each category for each category.

In this way, the identification device 100b according to the modified example can generate a plurality of learned models 114p to 114u that are classified into each category, and therefore, a more detailed analysis according to the profile of the target person 2 makes it more accurate. The type of tooth can be identified well.

The generation of the learned models 114p to 114u shown in FIG. 15 can be applied to the generation of the learned model 514a held by the server device 500. For example, the learning data sets 116p to 116u shown in FIG. 15 may be applied to the learning data set 516 held by the server apparatus 500, or the learned models 114p to 114u shown in FIG. You may apply to the learned model 514a hold|maintained.

(Service provision processing using a trained model for each category)
The service providing process executed by the identification device 100 using the learned models 114p to 114u for each category will be described with reference to FIG. FIG. 16 is a flowchart for explaining an example of a service providing process executed by the identification device 100b according to the modification. Each step shown in FIG. 16 is realized by the arithmetic unit 130 of the identification device 100b executing the OS 127 and the identification program 120.

As shown in FIG. 16, the identification device 100b determines whether or not the start condition of the service providing process is satisfied (S142). Since the start condition is the same as the start condition shown in FIG. 13, the description thereof will be omitted.

If the start condition is not satisfied (NO in S142), the identification device 100b ends this process. On the other hand, when the start condition is satisfied (YES in S142), the identification device 100b determines whether three-dimensional data has been acquired (S144). For example, the identification device 100b determines whether or not a sufficient amount of three-dimensional data has been acquired to execute the identification process. If the identification device 100b has not acquired a sufficient amount of three-dimensional data (NO in S144), the processing of S144 is repeated.

On the other hand, when the identification device 100b has acquired a sufficient amount of three-dimensional data (YES in S144), the identification device 100b acquires the profile data of the target person 2 input by the user 1 (S146). After that, the identification device 100b selects the learned model corresponding to the profile data from the learned model group shown in FIG. 15 (S148). For example, if the target person 2 is an elderly woman, the identification device 100b selects the learned model 114u.

Then, the identification device 100b inputs the three-dimensional data (positional information) into the learned model (S150). The identification device 100b performs an identification process for identifying the type of the tooth using the learned model based on the feature of the tooth corresponding to the three-dimensional data (S152).

After that, the identification device 100b outputs the identification result obtained by the identification process to the display 300, the speaker 400, the server device 500, and the like (S154), and ends this process.

As described above, the identification device 100b according to the modified example can perform the identification process by using the learned model that is most suitable for the profile of the target person 2, and thus a more detailed analysis according to the profile of the target person 2 can be performed. The tooth type can be identified more accurately.

(Profile output)
The identification device 100 according to the present embodiment identifies the type of tooth by the identification process. However, considering that the learned model is generated based on the learning data set in consideration of the profile of the subject 2 as shown in FIGS. 10 and 15, three-dimensional data is input to the learned model in the identification processing. By doing so, the profile of the owner of the tooth may be output as the identification result based on the characteristics of the tooth corresponding to the three-dimensional data. By doing so, it is possible to specify the profile of the target person 2 whose identity is unknown discovered in a disaster or a case from the three-dimensional data including the tooth.

(Learning process)
The identification device 100 according to the present embodiment updates the parameters 1144 of the estimation model 114 by the learning process, but the invention is not limited to updating the parameters 1144, and the neural network 1142 is updated by the learning process ( For example, the algorithm of the neural network 1142 is updated). Although the server device 500 according to the present embodiment updates the parameters of the estimation model 514 by the learning process, the invention is not limited to updating the parameters, and the neural network is updated by the learning process (for example, , The neural network algorithm is updated).

(Identification using normal and/or color information)
FIG. 17 is a schematic diagram for explaining an example of a learning data set according to the modification. The identification device 100c according to the modification learns the estimation model 114 by inputting the actual color information of the tooth to the estimation model 114 in addition to the position information included in the three-dimensional data acquired by the three-dimensional scanner 200. You may.

For example, as described with reference to FIG. 9, the learning data set includes position information that is input data to the estimation model 114 and color information that is correct answer data and that has been color-coded and associated with the tooth type. Although it is included, in addition to this, as shown in FIG. 17, color information of teeth before color division may be included. In the learning process, the learned model 114a is generated in consideration of the actual color information of the tooth by inputting the color information of the tooth before color division into the estimation model 114 in addition to the position information. May be.

Further, in the learning data set, in addition to the position information that is the input data to the estimation model 114 and the color information after the color classification associated with the tooth type that is the correct answer data, as shown in FIG. Normal line information that can be calculated based on the position information may be included. Then, in the learning process, the learned model 114a may be generated in consideration of the normal information by inputting the normal information in addition to the position information to the estimation model 114.

The normal line information can be calculated as follows, for example. For example, one of the plurality of points forming the tooth is focused on, and the normal line of the point of interest is generated based on the plurality of points belonging to a predetermined range in the vicinity of the point of interest. Specifically, the normal line of the attention point can be generated by using the principal component analysis for a plurality of points belonging to a predetermined range near the attention point. Principal component analysis can generally be performed by calculating the variance-covariance matrix. In this calculation of the variance-covariance matrix, the eigenvector may be calculated and the principal component direction may be generated as the normal line of the target point. Since a method of generating a normal line to a certain point in the point group is known, other generally known techniques may be used.

In this way, by adding the normal information to the learning data set, the identification device can also learn which side of the tooth formed by each point is the surface at each of the plurality of points forming the tooth. .. In addition, the identification device can learn the features of the shape such as the depression based on only a small number of point groups that belong to a predetermined range near the target point.

The learning data set may include both the color information and the normal information of the tooth before being color-coded, or may include only one of them.

Next, a service providing process for executing an identification process using a learned model learned based on a learning data set including tooth color information and normal line information before color classification will be described with reference to FIG. FIG. 18 is a flowchart for explaining an example of a service providing process executed by the identification device 100c according to the modification.

As illustrated in FIG. 18, the identification device 100c according to the modification additionally executes the process of S245, unlike the service providing process executed by the identification device 100 illustrated in FIG. That is, after the three-dimensional data is input (YES in S44), the identification device 100c generates normal lines at a plurality of points forming the tooth based on the position information included in the input three-dimensional data (S245). ). The input three-dimensional data includes the color information of the tooth before being color-coded in addition to the position information.

Then, when the identification device 100c determines that the profile data is not input (NO in S46), the normal information is input to the learned model 114a in addition to the three-dimensional data (position information, color information) (S248). ). On the other hand, when the identification device 100c determines that the profile data is input (YES in S46), the normal information is input to the learned model 114a in addition to the three-dimensional data (position information, color information) and the profile data ( S250). Then, after S248 and S250, the identification device 100c executes an identification process for identifying the type of tooth using the learned model (S52).

As described above, the identification device 100c according to the modified example further determines the tooth type based on the color information of the tooth before being color-coded and the normals generated for each of the plurality of points forming the tooth corresponding to the three-dimensional data. May be identified. Note that the identification device 100c may receive the color information of the tooth before being color-coded but not the normal line information. Alternatively, the normal line information may be input to the identification device 100c, but the color information of the tooth before color division may not be input.

In this way, the identification device 100c can identify the tooth type based on the color information of the tooth before color classification and/or the normal line generated for each of the plurality of points, and thus the tooth type can be more accurately determined. Can be identified.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope. Note that the configurations exemplified in the present embodiment and the configurations exemplified in the modified examples can be appropriately combined.

1 user, 2 target person, 5 network, 10 scanner system, 100, 100a, 100b identification device, 102 scanner interface, 103, 503 display interface, 104 speaker interface, 105, 505 peripheral device interface, 106, 506 network controller, 107 , 507 Media reader, 108 PC display, 109, 509 memory, 110, 510 storage, 112, 512 scan information, 114, 514 estimation model, 114a, 514a learned model, 116, 516 learning data set, 118, 518 Color classification data, 119,519 profile data, 120 identification program, 121,521 learning program, 122,522 three-dimensional data, 124,524 identification result, 127,527 OS, 130,530 arithmetic unit, 200 three-dimensional scanner , 300, 350 display, 400 speaker, 500 server device, 550 removable disk, 601,651 keyboard, 602,652 mouse, 1102 input unit, 1103 output unit, 1119 profile acquisition unit, 1130 identification unit, 1142
Neural network, 1144 parameters.

The present invention relates to an identification device, a tooth type identification system including the identification device, an identification method, and an identification program.

The present invention has been made to solve such a problem, and is an identification device capable of accurately identifying a tooth type, a tooth type identification system including the identification device, an identification method, and an identification program. The purpose is to provide.

According to the present invention, an identification device for identifying a tooth type is provided. Identification device includes an input unit for three-dimensional data including three-dimensional position information in each of a plurality of points constituting the tooth is input, the three-dimensional data inputted from the input unit, the tooth based on the three-dimensional data based the kinds and estimation model including a neural network for estimating an identifying unit for identifying the kind of the tooth, and an output unit for outputting the identification result by the identifying unit, estimation models, is entered by the user a tooth information corresponding to the type of tooth that is, will be learned based on the identification result by the identifying unit.

According to the present invention, a tooth type identification system to identify the type of tooth is provided. Tooth type identification system includes a three-dimensional scanner for acquiring three-dimensional data including three-dimensional position information in each of a plurality of points constituting the tooth using three-dimensional camera, three-dimensional data obtained by the three-dimensional scanner based on, and a for identifying identification device type of the tooth, the identification device includes an input unit for three-dimensional data obtained by the three-dimensional scanner is inputted, and the three-dimensional data inputted from the input unit, the based on the estimated model including a neural network for estimating the type of tooth based on the three-dimensional data includes an identification section for identifying a kind of the tooth, and an output unit for outputting the identification result by the identifying unit, estimation model includes a tooth information corresponding to the type of tooth entered by the user, will be learned on the basis of the result identification by the identification unit.

According to the present invention, an identification method for identifying a tooth type by a computer is provided. The identification method is a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming a tooth, the three-dimensional data input by the input step, and the three-dimensional data based on the three-dimensional data. based on the estimated model including a neural network for estimating the type of tooth, and a step of outputting identifying a kind of the tooth, the identification result by the identifying step, the estimation model of the input by the user a tooth information corresponding to the type of tooth that is, will be learned on the basis of the result identification by the identification unit.

According to the present invention, an identification program for identifying the type of tooth is provided. Identifying program causes a computer to execute the steps of the three-dimensional data including three-dimensional position information in each of a plurality of points constituting the tooth is input, the three-dimensional data input by the step input, the tertiary based on the estimated model including a neural network for estimating the type of tooth based on the original data, to execute the steps of outputting identifying a kind of the tooth, the identification result by the identifying step, estimation model , a tooth information corresponding to the type of tooth entered by the user, will be learned on the basis of the result identification by the identification unit.

According to the present invention, the type of tooth can be accurately identified based on the three-dimensional data including the three-dimensional position information at each of the plurality of points forming the tooth.

Claims (18)

An identification device for identifying the type of tooth,
An input unit for inputting three-dimensional data including teeth,
An identification unit for identifying the type of the tooth based on an estimation model including the three-dimensional data and the neural network including the tooth feature input from the input unit,
An output unit that outputs the identification result by the identification unit,
The identification model is learned based on tooth information corresponding to a tooth type associated with the three-dimensional data, and a tooth type identification result using the three-dimensional data. In the input unit, at least the three-dimensional data corresponding to a plurality of adjacent teeth and gums in the oral cavity is input,
The identification device according to claim 1, wherein the identification unit identifies each type of the plurality of teeth based on the three-dimensional data including the characteristics of each of the plurality of teeth. The identification device according to claim 1 or 2, wherein the three-dimensional data includes three-dimensional position information at each of a plurality of points forming a tooth corresponding to the three-dimensional data. The identification device according to claim 3, wherein the position information includes coordinates of an absolute position based on a predetermined position. The identification device according to any one of claims 1 to 4, which further identifies a tooth type based on a normal line generated for each of a plurality of points forming a tooth corresponding to the three-dimensional data. .. When the tooth corresponding to the three-dimensional data input to the input unit is an incisor of the upper jaw, the three-dimensional image corresponding to the three-dimensional data is a site on the upper lip side, a site on the palate side, and a cutting edge side. Including at least an image of the part,
When the tooth corresponding to the three-dimensional data input to the input unit is a maxillary canine and a molar, the three-dimensional image corresponding to the three-dimensional data is a cheek side portion, a palate side portion, and occlusion. Including at least an image of the part,
When the tooth corresponding to the three-dimensional data input to the input unit is a lower jaw incisor, the three-dimensional image corresponding to the three-dimensional data is a lower lip side portion, a tongue side portion, and a cutting edge side. Including at least an image of the part of
When the tooth corresponding to the three-dimensional data input to the input unit is a lower canine and a molar, the three-dimensional image corresponding to the three-dimensional data is a buccal region, a lingual region, and occlusion. The identification device according to claim 1, further comprising at least an image of the part. The output unit outputs the identification result by the identification unit to a display unit,
The identification according to any one of claims 1 to 5, wherein the display unit displays at least one of an image, a character, a number, an icon, and a symbol corresponding to an identification result by the identification unit. apparatus. The output unit outputs the identification result by the identification unit to a voice output unit,
The identification device according to any one of claims 1 to 7, wherein the voice output unit outputs a voice corresponding to the identification result by the identification unit. The output unit outputs the identification result by the identification unit to a server device,
The identification device according to any one of claims 1 to 8, wherein the server device accumulates and stores an identification result by the identification unit. The estimation model includes at least one of a weighting coefficient and a determination value as a parameter used by the neural network,
The identification device according to any one of claims 1 to 9, wherein the estimation model is learned by updating the parameters based on the tooth information and the identification result by the identification unit. 11. The tooth information according to claim 1, wherein the tooth information includes at least one information of color, letters, numbers, and symbols associated with a tooth type corresponding to the three-dimensional data. The identification device according to item 1. The identification device according to any one of claims 1 to 11, wherein the tooth information is associated with the three-dimensional data so as to be able to specify each range of a plurality of teeth corresponding to the three-dimensional data. The identification device according to any one of claims 1 to 12, wherein the tooth information is associated with each of a plurality of points forming a tooth corresponding to the three-dimensional data. The identification according to any one of claims 1 to 13, wherein the estimation model is learned based on attribute information about a subject who has a tooth, in addition to the tooth information and the identification result by the identification unit. apparatus. The identification device according to claim 14, wherein the attribute information includes information of at least one of age, sex, race, height, weight, and place of residence of the subject. A scanner system for acquiring tooth shape information,
A three-dimensional scanner that acquires three-dimensional data including teeth using a three-dimensional camera,
Based on the three-dimensional data including the characteristics of the tooth acquired by the three-dimensional scanner, an identification device for identifying the type of the tooth,
The identification device is
An input unit for inputting the three-dimensional data,
An identification unit for identifying the type of the tooth based on an estimation model including the three-dimensional data and the neural network including the tooth feature input from the input unit,
An output unit for outputting the identification result by the identification unit,
A scanner system in which the estimation model is learned based on tooth information corresponding to a tooth type associated with the three-dimensional data and an identification result of the tooth type using the three-dimensional data. A method for identifying the type of tooth,
The step of inputting three-dimensional data including teeth,
Identifying the type of the tooth based on an estimated model including the three-dimensional data including the characteristics of the tooth and a neural network,
Outputting the identification result by the identifying step,
An identification method, wherein the estimation model is learned based on tooth information corresponding to a tooth type associated with the three-dimensional data and an identification result of the tooth type using the three-dimensional data. A program for identifying the type of tooth,
The identification program is a computer,
The step of inputting three-dimensional data including teeth,
Identifying the type of the tooth based on an estimated model including the three-dimensional data including the characteristics of the tooth and a neural network,
And executing a step of outputting an identification result by the identifying step,
A program for identification, wherein the estimation model is learned based on tooth information corresponding to a tooth type associated with the three-dimensional data and an identification result of the tooth type using the three-dimensional data.