JP7227188B2 - Identification device, identification system, identification method, and identification program - Google Patents

Description

The present disclosure relates to an identification device, an identification system, an identification method, and an identification program.

2. Description of the Related Art Conventionally, in the field of dentistry, a three-dimensional scanner with a built-in three-dimensional camera for obtaining a three-dimensional shape of a tooth has been known for digital designing of a prosthesis or the like on a computer. For example, Patent Literature 1 discloses a technique for recording the shape of a tooth by capturing an image of the tooth using a three-dimensional camera. An operator such as a dentist can record the three-dimensional shape of the tooth to be imaged by using the three-dimensional camera disclosed in Patent Document 1, and the recorded three-dimensional shape of the tooth can be recorded. While confirming the displayed three-dimensional image, the user can identify the type of tooth according to his/her own knowledge.

JP-A-2000-74635

In this way, conventionally, the operator has identified the type of the tooth based on his/her own knowledge based on the three-dimensional image including the tooth acquired by the three-dimensional camera, but the level of knowledge varies for each operator. Therefore, there is a problem that the accuracy of the identification result varies depending on the level of knowledge of the operator. For example, central incisors and lateral incisors, canines and first premolars, first premolars and second premolars, second premolars and first molars, first premolars and second molars, etc. The accuracy of discrimination between similar teeth is not particularly high.

The present disclosure has been made to solve such problems, and provides an identification device capable of accurately identifying the type of tooth, an identification system including the identification device, an identification method, and an identification program. intended to

According to the present disclosure, an identification device is provided for identifying tooth types. The identification device includes an input unit for inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth where a plurality of teeth are adjacent, and based on the three-dimensional data input from the input unit. , an identification unit for identifying types of teeth included in adjacent teeth, and an output unit for outputting identification results by the identification unit, wherein the identification unit includes a plurality of points constituting a first adjacent tooth to which the plurality of teeth are adjacent. Based on three-dimensional data including three-dimensional position information in each of and a first estimation model including a first neural network, the same number as the first adjacent tooth or a plurality of numbers less than the first adjacent tooth of teeth included in the second adjacent tooth based on a first identification unit that identifies the second adjacent tooth adjacent to the tooth, the identification result of the first identification unit, and a second estimation model that includes a second neural network and a second identification portion for identifying the type.

According to the present disclosure, an identification system is provided for identifying tooth types. The identification system uses a three-dimensional camera to acquire three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other, and a three-dimensional scanner. an identification device that identifies types of teeth included in adjacent teeth based on the acquired three-dimensional data, and the identification device includes an input unit into which the three-dimensional data acquired by the three-dimensional scanner is input; An identification unit that identifies the types of teeth included in adjacent teeth based on the three-dimensional data input from the identification unit, and an output unit that outputs identification results obtained by the identification unit. Based on three-dimensional data containing three-dimensional position information at each of a plurality of points constituting one adjacent tooth and a first estimation model including a first neural network, the same number as the first adjacent tooth or the first Based on a first identification unit that identifies a second adjacent tooth adjacent to a plurality of teeth that are less than one adjacent tooth, a result of identification by the first identification unit, and a second estimation model that includes a second neural network, and a second identification portion for identifying the type of tooth included in the second adjacent tooth.

According to the present disclosure, a computerized identification method for identifying tooth types is provided. The identification method includes a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth where a plurality of teeth are adjacent , as a process executed by a computer , and a step of inputting Based on the input three-dimensional data, identifying the types of teeth included in adjacent teeth; and outputting identification results obtained by the identifying step. Based on three-dimensional data including three-dimensional positional information at each of a plurality of points constituting adjacent teeth and a first estimation model including a first neural network, the same number as the first adjacent tooth or the first Based on a first identification step of identifying a second adjacent tooth adjacent to a plurality of teeth in a number smaller than the number of adjacent teeth, an identification result of the first identification step, and a second estimation model including a second neural network, a second and a second identification step of identifying the types of teeth included in the two adjacent teeth.

According to the present disclosure, an identification program is provided for identifying tooth types. The identification program comprises a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other into a computer; Based on the three-dimensional data, a step of identifying types of teeth included in adjacent teeth and a step of outputting an identification result of the identifying step are executed, wherein the identifying step includes a first adjacent tooth to which a plurality of teeth are adjacent. Based on three-dimensional data containing three-dimensional position information at each of a plurality of points that constitute the and a first estimation model containing a first neural network, the same number as the first adjacent tooth or the first adjacent tooth Based on a first identification step of identifying a second adjacent tooth adjacent to a plurality of teeth having a number less than the number of adjacent teeth, the identification result of the first identification step, and a second estimation model including a second neural network, a second adjacent tooth and a second identification step of identifying the type of tooth contained in the tooth.

According to the present disclosure, it is possible to accurately identify the type of tooth based on three-dimensional data including three-dimensional position information on each of a plurality of points that constitute the tooth.

It is a schematic diagram showing an application example of the identification device according to the present embodiment. 1 is a schematic diagram showing the overall configuration of a system according to an embodiment; FIG. 1 is a schematic diagram showing a hardware configuration of an identification device according to an embodiment; FIG. 2 is a schematic diagram showing a hardware configuration of a server device according to this embodiment; FIG. 1 is a schematic diagram showing a functional configuration of an identification device according to an embodiment; FIG. FIG. 4 is a schematic diagram for explaining the contents of three-dimensional data input to the identification device according to the present embodiment; FIG. 4 is a schematic diagram showing an example of a tooth to be identified in identification processing according to the present embodiment; FIG. 4 is a schematic diagram showing a specific example of an identification unit included in the identification device according to the present embodiment; FIG. 4 is a schematic diagram for explaining generation of learning data according to the present embodiment; FIG. 3 is a schematic diagram for explaining an example of learning data according to the present embodiment; FIG. 4 is a schematic diagram showing an example of color classification data according to the embodiment; FIG. FIG. 4 is a schematic diagram for explaining generation of a trained model based on a learning data set according to the present embodiment; FIG. 4 is a schematic diagram for explaining learning of a first-stage estimation model among estimation models according to the present embodiment; FIG. 5 is a schematic diagram for explaining learning of a second-stage estimation model among the estimation models according to the present embodiment; FIG. 10 is a schematic diagram for explaining learning of a third-stage estimation model among the estimation models according to the present embodiment; 6 is a flowchart for explaining an example of learning processing executed by the identification device according to the present embodiment; 6 is a flowchart for explaining an example of learning processing executed by the server device according to the present embodiment; 4 is a flowchart for explaining an example of service providing processing executed by the identification device according to the present embodiment; FIG. 11 is a flowchart for explaining an example of service providing processing executed by an identification device according to a modification; FIG. FIG. 11 is a schematic diagram for explaining generation of a trained model based on a learning data set according to a modification; FIG. 11 is a flowchart for explaining an example of service providing processing executed by an identification device according to a modification; FIG. FIG. 11 is a schematic diagram for explaining an example of a learning data set according to a modification; FIG. 11 is a flowchart for explaining an example of service providing processing executed by an identification device according to a modification; FIG.

Embodiments of the present disclosure will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are given the same reference numerals, and the description thereof will not be repeated.

[Application example]
An application example of the identification device 100 according to the present embodiment will be described with reference to FIGS. 1 and 2. FIG. FIG. 1 is a schematic diagram showing an application example of an identification device 100 according to this embodiment. FIG. 2 is a schematic diagram showing the overall configuration of the system according to this embodiment.

As shown in FIG. 1, a user 1 uses an identification system 10 to obtain a plurality of adjacent teeth of a subject 2 (hereinafter also referred to as "adjacent teeth") and three-dimensional shape data of gingiva (hereinafter referred to as " (also referred to as "three-dimensional data") can be acquired. The “user” is any person who uses the identification system 10, such as an operator such as a dentist, a dental assistant, a teacher or student at a dental college, a dental technician, a manufacturer's engineer, or a factory worker. may be A “subject” may be any person who is a target of the identification system 10, such as a patient at a dental clinic or a subject at a dental college. "Adjacent teeth" means a group of adjacent teeth, and the same applies to "first adjacent teeth" and "second adjacent teeth", which will be described later.

Identification system 10 according to the present embodiment includes three-dimensional scanner 200 , identification device 100 , display 300 , and speaker 400 . The three-dimensional scanner 200 acquires three-dimensional data of a scanning target by a built-in three-dimensional camera. Specifically, the three-dimensional scanner 200 scans the inside of the oral cavity to generate three-dimensional data as positional information of each of a plurality of points that constitute the teeth and gums to be scanned (for example, X axis, Y-axis along the horizontal direction, and Z-axis along the height direction) are acquired using an optical sensor or the like. Alternatively, the three-dimensional scanner 200 scans a model created by taking an impression of the oral cavity, and obtains positional information (for example, longitudinal X-axis along the direction, Y-axis along the horizontal direction, and Z-axis along the height direction) are acquired using an optical sensor or the like. Based on the three-dimensional data acquired by the three-dimensional scanner 200, the identification device 100 generates a two-dimensional image including a tooth image from an arbitrary viewpoint, or a three-dimensional image including a three-dimensional image of a tooth such as a hologram. is generated, and the generated three-dimensional image is displayed on the display 300 . At least one of the three-dimensional scanner 200 and the identification device 100 may be a movable portable type or a fixed stationary type. For example, if the three-dimensional scanner 200 and the identification device 100 are stationary desktop scanners, the scanned object may be an artificially generated tooth such as a plaster model or a denture.

For example, in order to digitally design, on a computer, a prosthesis that compensates for missing teeth of the subject 2, the user 1 uses the three-dimensional scanner 200 to image the intraoral cavity of the subject 2, thereby obtaining images of adjacent teeth and gums. Acquire 3D intraoral data including The three-dimensional data includes at least three-dimensional positional information for each of a plurality of points forming the tooth. Each time the user 1 takes an intraoral image, three-dimensional data is sequentially acquired, and the three-dimensional image of the intraoral cavity is displayed on the display 300 . The user 1 confirms the three-dimensional image displayed on the display 300 and scans the missing part of the three-dimensional data.

Conventionally, based on a three-dimensional image in which three-dimensional data including the tooth acquired by the three-dimensional scanner 200 is visualized, the user 1 identifies the type of tooth that is being scanned or has been scanned based on his/her own knowledge. . However, since the level of knowledge differs for each user 1, depending on the knowledge of the user 1 may result in variations in the accuracy of the identification results. In orthodontics and the like, when designing orthodontic appliances, the teeth are classified by type while distinguishing between teeth and gingiva in the acquired three-dimensional data containing multiple adjacent teeth ( segmentation) is also required.

Therefore, the identification system 10 according to the present embodiment uses AI (Artificial Intelligence) of the identification device 100 to automatically identify the type of tooth based on the three-dimensional data acquired by the three-dimensional scanner 200. It is configured to perform a process of identifying Note that the process of identifying the type of tooth by the identification device 100 is also referred to as "identification process".

The "types of teeth" are upper right central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, upper left side. central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, lower right central incisors, lateral incisors, canines, 1st premolar, 2nd premolar, 1st molar, 2nd molar, and 3rd molar, mandibular left central incisor, lateral incisor, canine, 1st premolar, 2nd premolar, 3rd molar It refers to each tooth type such as 1 molar, 2nd molar, and 3rd molar.

Specifically, when the user 1 scans the intraoral teeth of the subject 2 using the 3D scanner 200 , intraoral 3D data including at least the teeth and gums is input to the identification device 100 . The identification device 100 executes identification processing for identifying the type of the tooth based on the inputted three-dimensional data including tooth characteristics and an estimation model including a neural network.

The "estimation model" includes a neural network and parameters used by the neural network, tooth information corresponding to the type of tooth associated with the three-dimensional data, and identification of the type of tooth using the three-dimensional data. It is optimized (adjusted) by being learned based on the results.

Specifically, when three-dimensional data corresponding to a plurality of points constituting adjacent teeth are input, the estimation model extracts tooth features using a neural network based on the three-dimensional data, and extracts tooth features. Estimate the type of tooth based on Then, the estimation model does not update the parameters based on the type of tooth estimated by itself and the type of tooth (tooth information) associated with the input three-dimensional data, if both match. If they do not match, the parameters are optimized by updating the parameters so that they match. In this way, the estimation model is learned by optimizing parameters using teacher data including three-dimensional data as input data and tooth types (tooth information) as correct data.

Note that processing for learning such an estimation model is also referred to as “learning processing”. In addition, the estimation model optimized by the learning process is particularly called a "learned model". That is, in the present embodiment, pre-learning estimation models and trained estimation models are collectively referred to as "estimation models", while trained estimation models are also particularly referred to as "learned models".

"Tooth information" includes upper right central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, upper left central incisors Teeth, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars, mandibular right central incisors, lateral incisors, canines, first minors Molars, second premolars, first molars, second molars, and third molars, mandibular left central incisors, lateral incisors, canines, first premolars, second premolars, first molars , second molars, and third molars. In addition, "tooth information" includes the number 1 assigned to the central incisor, the number 2 assigned to the lateral incisor, the number 3 assigned to the canine, the number 4 assigned to the first premolar, the second small A number assigned to each tooth, such as number 5 assigned to the molar, number 6 assigned to the first molar, number 7 assigned to the second molar, number 8 assigned to the third molar, and so on. (eg tooth numbers commonly used in the dental field). In addition, the "tooth information" may include color information assigned to each tooth, or may include symbol information assigned to each tooth.

When identification processing is executed by identification device 100 using a trained model, the identification result is output to display 300 and speaker 400 .

Display 300 displays at least one of images, letters, numbers, icons, and symbols corresponding to the identification result. For example, after the three-dimensional scanner 200 completes the scanning of the second molar corresponding to the number 7 on the right side of the mandible, the display 300 uses the identification result of the tooth by the identification device 100 to display "lower right number 7 Scanning is completed.", an image is displayed to the effect that scanning of the second molar corresponding to number 7 on the right side of the mandible has been completed.

Speaker 400 outputs a sound corresponding to the identification result. For example, after the three-dimensional scanner 200 completes the scanning of the second molar corresponding to the number 7 on the right side of the mandible, the speaker 400 uses the identification result of the teeth by the identification device 100 to say, "right bottom 7 completed. ', outputting a voice indicating that the scanning of the second molar corresponding to the number 7 on the right side of the mandible has been completed.

Furthermore, the identification result by the identification device 100 is output as scan information to the server device 500 arranged in the dental laboratory and the management center together with the three-dimensional data used in the identification processing.

For example, as shown in FIG. 2, an identification system 10 is located at each of a plurality of locals AC. For example, local A and local B are dental clinics, and in the dental clinics, an operator or a dental assistant who is user 1 utilizes identification system 10 to form a plurality of teeth of a patient who is target person 2. Acquire the 3D data corresponding to the points of . Also, the local C is a dental college, and at the dental college, a teacher or a student who is the user 1 acquires intraoral three-dimensional data of a subject who is the subject 2 . The scan information (three-dimensional data, identification results) acquired by each of the locals A to C is output via the network 5 to the server device 500 located in the dental laboratory and management center as the local D.

In the dental laboratory, a dental technician or the like creates a prosthesis or the like to replace the missing tooth portion of the subject 2 based on the scan information obtained from each of the locals A to C. In the management center, the server device 500 accumulates and stores the scan information obtained from each of the locals A to C, and holds it as big data.

Note that the server device 500 is not limited to being arranged in a management center different from the local one in the dental clinic, and may be arranged locally. For example, the server apparatus 500 may be arranged within any one of the locals A to C. FIG. Also, a plurality of identification devices 100 may be arranged within one local, and a server device 500 capable of communicating with the plurality of identification devices 100 may be arranged within the one local. Moreover, the server device 500 may be implemented in the form of a cloud service.

In the dental laboratory, scan information is aggregated from various locations, such as local A-C. For this reason, the scan information held at the dental laboratory may be transmitted to the management center via the network 5, or may be sent to a removable disk 550 such as a CD (Compact Disc) and USB (Universal Serial Bus) memory. may be sent to the management center via

The scan information may also be sent to the management center from each of the local A to C via the removable disk 550 without going through the network 5 . Also, between each of the locals A to C, scan information may be sent to each other via the network 5 or the removable disk 550 .

Each identification device 100 of local A to C has its own estimated model, and identifies the type of tooth using its own estimated model during the identification process. Each local identification device 100 generates a trained model by learning its own estimation model through its own learning process. Furthermore, in the present embodiment, server device 500 also holds an estimation model. The server device 500 generates a trained model by learning an estimated model through learning processing using scan information acquired from the identification device 100 of each of the locals A to C and the dental laboratory. The trained model is distributed to the identification device 100 .

In the present embodiment, both local identification devices 100 and server devices 500 of local A to C execute learning processing, but only local identification devices 100 of local A to C execute learning processing. Alternatively, only the server device 500 may execute the learning process. If only the server device 500 executes the learning process, the estimation model (learned model) held by each local identification device 100 is common among the local identification devices 100. become.

Also, the server device 500 may have the function of identification processing in the identification device 100 . For example, each of the locals A to C transmits the acquired three-dimensional data to the server device 500, and the server device 500 identifies the type of tooth in each based on the three-dimensional data received from each of the locals A to C. Results may be calculated. Then, server device 500 may transmit the respective identification results to each of locals A to C, and each of locals A to C may output the identification results received from server device 500 to a display or the like. In this way, each local A to C and the server device 500 may be configured in the form of a cloud service. In this way, as long as server device 500 holds an estimation model (learned model), each of locals A to C can obtain an identification result without holding an estimation model (learned model). can.

Thus, according to the identification system 10 according to the present embodiment, the type of tooth is automatically identified based on the three-dimensional data acquired by the three-dimensional scanner 200 using the AI of the identification device 100. be. By using AI, it is possible to find not only tooth features that can be extracted by the user 1, but also tooth features that cannot be extracted by the user 1, and the type of tooth can be identified based on the found tooth features. Therefore, the user 1 can accurately identify the type of tooth without relying on his/her own knowledge.

[Hardware Configuration of Identification Device]
An example of the hardware configuration of the identification device 100 according to this embodiment will be described with reference to FIG. FIG. 3 is a schematic diagram showing the hardware configuration of the identification device 100 according to this embodiment. The identification device 100 may be realized, for example, by a general-purpose computer, or by a computer dedicated to the identification system 10 .

As shown in FIG. 3, the identification device 100 includes a scanner interface 102, a display interface 103, a speaker interface 104, a peripheral device interface 105, a network controller 106, and a media reader 107 as main hardware elements. , a PC display 108 , a memory 109 , a storage 110 and an arithmetic unit 130 .

The scanner interface 102 is an interface for connecting the three-dimensional scanner 200 and realizes input/output of data between the identification device 100 and the three-dimensional scanner 200 .

The display interface 103 is an interface for connecting the display 300 and realizes input/output of data between the identification device 100 and the display 300 . Display 300 is configured by, for example, an LCD (Liquid Crystal Display) or an organic ELD (Electroluminescence) display.

The speaker interface 104 is an interface for connecting the speaker 400 and realizes input/output of data between the identification device 100 and the speaker 400 .

The peripheral device interface 105 is an interface for connecting peripheral devices such as the keyboard 601 and the mouse 602, and realizes input/output of data between the identification device 100 and the peripheral devices.

The network controller 106 communicates data to and from each of the devices located at the dental laboratory, the server device 500 located at the management center, and other locally located identification devices 100 via the network 5 . send and receive The network controller 106 supports arbitrary communication methods such as Ethernet (registered trademark), wireless LAN (Local Area Network), and Bluetooth (registered trademark).

The media reading device 107 reads various data such as scan information stored in a removable disk 550, which is a storage medium.

The PC display 108 is a dedicated display for the identification device 100 . PC display 108 is configured by, for example, an LCD or an organic EL display. Although PC display 108 is separate from display 300 in the present embodiment, it may be shared with display 300 .

The memory 109 provides a storage area for temporarily storing program codes, work memory, etc. when the arithmetic unit 130 executes an arbitrary program. The memory 109 is composed of, for example, a volatile memory device such as a DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory).

The storage 110 provides storage areas for storing various data necessary for identification processing, learning processing, and the like. The storage 110 is composed of a non-volatile memory device such as a hard disk or SSD (Solid State Drive), for example.

Storage 110 includes scan information 112, estimated model 114 (learned model), learning data set 116, color classification data 118, profile data 119, identification program 120, learning program 121, OS (Operating System) 127 is stored.

The scan information 112 includes three-dimensional data 122 acquired by the three-dimensional scanner 200 and identification results 124 obtained by identification processing executed based on the three-dimensional data 122 . The identification result 124 is stored in the storage 110 in association with the three-dimensional data 122 used for identification processing. The learning data set 116 is a group of learning data used for learning processing of the estimation model 114 . The color classification data 118 is data used for generating the learning data set 116 and for learning processing. The profile data 119 is attribute information about the target person 2, and is data (for example, medical record information) in which profiles such as age, sex, race, height, weight, and place of residence of the target person 2 are summarized. be. The identification program 120 is a program for executing identification processing. The learning program 121 is a program for executing the learning process of the estimation model 114, and part of it also includes a program for executing the identification process.

The arithmetic unit 130 is an arithmetic entity that executes various types of processing such as identification processing and learning processing by executing various programs, and is an example of a computer. Arithmetic device 130 includes, for example, a CPU (Central Processing Unit) 132, an FPGA (Field-Programmable Gate Array) 134, a GPU (Graphics Processing Unit) 136, and the like.

Arithmetic device 130 may be configured with at least one of CPU 132, FPGA 134, and GPU 136, or may be configured with CPU 132 and FPGA 134, FPGA 134 and GPU 136, CPU 132 and GPU 136, or CPU 132, FPGA 134, and GPU 136. may be Processing units 130 may also be referred to as processing circuitry.

[Hardware Configuration of Server Device]
An example of the hardware configuration of server device 500 according to the present embodiment will be described with reference to FIG. FIG. 4 is a schematic diagram showing the hardware configuration of server device 500 according to the present embodiment. Server device 500 may be implemented by, for example, a general-purpose computer, or may be implemented by a computer dedicated to identification system 10 .

As shown in FIG. 4, the server device 500 includes, as main hardware elements, a display interface 503, a peripheral device interface 505, a network controller 506, a media reader 507, a memory 509, a storage 510, and a computing device. and a device 530 .

The display interface 503 is an interface for connecting the display 350 and realizes input/output of data between the server device 500 and the display 350 . Display 350 is configured by, for example, an LCD or an organic ELD display.

Peripheral device interface 505 is an interface for connecting peripheral devices such as keyboard 651 and mouse 652, and implements data input/output between server device 500 and the peripheral devices.

The network controller 506 transmits and receives data via the network 5 to and from each of the locally located identification device 100 and the device located at the dental laboratory. The network controller 506 may support any communication method such as Ethernet (registered trademark), wireless LAN, Bluetooth (registered trademark), or the like.

The media reader 507 reads various data such as scan information stored in the removable disk 550 .

The memory 509 provides a storage area for temporarily storing program codes, work memory, etc. when the arithmetic unit 530 executes an arbitrary program. Memory 509 comprises, for example, a volatile memory device such as a DRAM or SRAM.

The storage 510 provides storage areas for storing various data necessary for learning processing and the like. The storage 510 is configured with a non-volatile memory device such as a hard disk or SSD, for example.

Storage 510 stores scan information 512 , estimated model 514 (learned model), learning data set 516 , color classification data 518 , profile data 519 , learning program 521 , and OS 527 .

The scan information 512 includes three-dimensional data 522 acquired from the identification device 100 and the dental laboratory locally arranged via the network 5, and an identification result 524 by identification processing executed based on the three-dimensional data 522. . The identification result 524 is stored in the storage 510 in association with the three-dimensional data 522 used for identification processing. A learning data set 516 is a group of learning data used for learning processing of the estimation model 514 . The color classification data 518 is data used for generating the learning data set 516 and for learning processing. The profile data 519 is attribute information about the subject 2, and is data in which the profile of the subject 2 such as age, sex, race, height, weight, place of residence, etc. is summarized (for example, medical record information). . The learning program 521 is a program for executing the learning process of the estimation model 514, and part of it also includes a program for executing the identification process.

The estimation model 514 (learned model) is transmitted to the local identification device 100 and is held as the estimation model 114 (learned model) by the identification device 100 .

The arithmetic device 530 is an arithmetic entity that executes various types of processing such as learning processing by executing various programs, and is an example of a computer. Arithmetic device 530 includes, for example, CPU 532, FPGA 534, GPU 536, and the like.

Note that the arithmetic device 530 may be configured with at least one of the CPU 532, FPGA 534, and GPU 536, or may include the CPU 532 and FPGA 534, the FPGA 534 and GPU 536, the CPU 532 and GPU 536, or the CPU 532, FPGA 534, and GPU 536. may be Processing units 530 may also be referred to as processing circuitry.

[Identification processing by identification device]
An example of identification processing by the identification device 100 according to the present embodiment will be described with reference to FIGS. 5 to 8. FIG. FIG. 5 is a schematic diagram showing the functional configuration of the identification device 100 according to this embodiment. FIG. 6 is a schematic diagram for explaining the contents of three-dimensional data input to the identification device according to this embodiment. FIG. 7 is a schematic diagram showing an example of a tooth to be identified in the identification processing according to this embodiment. FIG. 8 is a schematic diagram showing a specific example of an identification unit included in the identification device according to this embodiment. In FIG. 7, the tooth to be scanned by the three-dimensional scanner 200 is represented by a diagram.

As shown in FIG. 5, the identification device 100 has an input unit 1102, a profile acquisition unit 1119, an identification unit 1101, and an output unit 1103 as functional units related to identification processing. Each of these functions is realized by executing the OS 127 and the identification program 120 by the arithmetic device 130 of the identification device 100 .

Three-dimensional data corresponding to a plurality of points forming adjacent teeth acquired by the three-dimensional scanner 200 is input to the input unit 1102 . Profile acquisition unit 1119 acquires profile data 119 of subject 2 .

The identification unit 1101 is a multistage identification unit including a plurality of identification units. In this embodiment, the identification section 1101 has a first identification section 1131 and a second identification section 1132 .

The first identification unit 1131 includes three-dimensional data including three-dimensional positional information of each of a plurality of points forming adjacent teeth (first adjacent teeth) input to the input unit 1102, and a first estimation model 1141 (first 1 trained model), adjacent teeth (second adjacent teeth) adjacent to a plurality of teeth equal in number to or less than the first adjacent tooth are identified. The second adjacent teeth are adjacent teeth selected from among the teeth included in the first adjacent teeth, and are set in advance by grouping. Further, in the present embodiment, the first identification unit 1131 performs the first estimation based on the three-dimensional data input to the input unit 1102 and the profile data 119 of the subject 2 acquired by the profile acquisition unit 1119. Identify the second adjacent tooth using model 1141 (first trained model). Note that the first identification unit 1131 does not have to use the profile data 119 in identification processing.

The first estimation model 1141 includes a first neural network 1151 and a first parameter 1161 used by the first neural network 1151 . The first parameter 1161 includes a weighting factor used for calculation by the first neural network 1151 and a decision value used for identification decision.

The second identification unit 1132 identifies the type of tooth included in the second adjacent tooth based on the identification result of the second adjacent tooth by the first identification unit 1131 and the second estimation model 1142 .

Second estimation model 1142 includes second neural network 1152 and second parameters 1162 used by second neural network 1152 . Second parameters 1162 include weighting factors used in calculations by second neural network 1152 and decision values used to determine identity.

Note that the first estimation model 1141 and the second estimation model 1142 described above correspond to the estimation model 114 shown in FIG.

The output unit 1103 outputs the identification result of the tooth type by the second identification unit 1132 to the display 300 , the speaker 400 and the server device 500 .

Here, as shown in FIG. 6, the three-dimensional data input to the input unit 1102 includes three-dimensional position information at each point on the tooth and color information at each point on the tooth. The position information includes the coordinates of the absolute position in three dimensions with reference to a predetermined position. For example, the positional information can be obtained by using the center position of each point of the tooth as the origin, the X-axis (for example, the axis along the lateral direction of the tooth), the Y-axis (for example, the axis along the longitudinal direction of the tooth), and the Z-axis. (eg, the axis along the height direction of the tooth). Note that the position information is not limited to coordinates of a three-dimensional absolute position with a predetermined position as the origin, but includes coordinates of a three-dimensional relative position indicating the distance and direction (vector) from an adjacent point, for example. You can stay.

Here, as shown in FIG. 7, when the tooth to be scanned by the three-dimensional scanner 200 is an incisor of the upper jaw, the obtained three-dimensional image is a region on the upper lip side, a region on the palate side, and a region on the incisal side. The oral cavity of a subject 2 is scanned by a user 1 so as to include at least an image of the site. Also, when the teeth to be scanned by the three-dimensional scanner 200 are the canines and molars of the upper jaw, the obtained three-dimensional image should include at least images of the buccal region, the palatal region, and the occlusal region. , the oral cavity of a subject 2 is scanned by a user 1 . When the tooth to be scanned by the three-dimensional scanner 200 is a mandibular incisor, the three-dimensional image obtained includes at least images of the lower lip side region, the lingual region, and the incisal side region. A user 1 scans the oral cavity of a subject 2 . When the teeth to be scanned by the three-dimensional scanner 200 are the canines and molars of the mandible, the resulting three-dimensional image includes at least images of the buccal region, the lingual region, and the occlusal region. 1 scans the oral cavity of a subject 2 .

In general, the teeth of the subject 2 differ in shape and size depending on their type. For example, for maxillary incisors, the upper labial surface is generally U-shaped, whereas for maxillary canines, the buccal surface is generally pentagonal. Each tooth has a characteristic shape and size according to its type, and the identification unit 1101 uses the estimation model 114 based on three-dimensional data in which these characteristic shapes and sizes are quantified. Identify the type of tooth corresponding to the three-dimensional data.

In the first neural network 1151, the value of position information included in the three-dimensional data input to the input unit 1102 is input to the input layer. In the first neural network 1151, for example, the intermediate layer multiplies the value of the input positional information by a weighting factor or adds a predetermined bias, and performs calculation using a predetermined function. The calculation result is compared with the judgment value. Then, in the first neural network 1151, the result of the calculation and determination is output from the output layer as the identification result. Any method may be used for the calculation and determination by the first neural network 1151 .

In the second neural network 1152, a value corresponding to the identification result by the first identification section 1131 is input to the input layer. In the second neural network 1152, for example, the intermediate layer multiplies the value corresponding to the input identification result by a weighting factor or adds a predetermined bias, and performs calculation using a predetermined function. and the calculated result is compared with the decision value. Then, in the second neural network 1152, the result of the calculation and determination is output from the output layer as the identification result. Any method may be used for the calculation and determination by the second neural network 1152 .

In the first neural network 1151 and the second neural network 1152, processing by deep learning is performed because the intermediate layer has a multi-layer structure. In this embodiment, the identification program 120 that performs identification processing specialized for three-dimensional images includes, for example, VoxNet, 3D ShapeNets, Multi-View CNN, RotationNet, OctNet, FusionNet, PointNet, PointNet++, SSCNet, and MarrNet etc. are used, but other programs may be used. Also, an existing one may be applied to the mechanisms of the first neural network 1151 and the second neural network 1152 .

A neural network included in estimation model 514 held by server device 500 has the same configuration as first neural network 1151 and second neural network 1152 .

A specific example of the identification unit 1101 will be described with reference to FIG. Note that FIG. 8 shows an example in which three-dimensional data corresponding to a plurality of points forming the 3rd to 7th adjacent teeth and gingiva on the left side of the mandible are input to the identification unit 1101 . Note that the 3rd to 7th adjacent teeth are an example of the "first adjacent tooth". The three-dimensional data input to the identification unit 1101 may include non-color-processed color information (that is, color information indicating the color of the tooth itself) for the 3rd to 7th adjacent teeth, Grayscaled color information may be included for adjacent teeth numbered 3-7. The identification unit 1101 identifies the third to seventh adjacent teeth on the left side of the mandible based on the input three-dimensional data.

As shown in FIG. 8, identification section 1101 includes identification section 1130a, identification section 1130b, identification section 1130c, estimation model 1140a, estimation model 1140b, and estimation model 1140c.

The identification unit 1130a on the first stage is an example of the first identification unit 1131 shown in FIG. Based on the estimated model 1140a, adjacent teeth are identified that are adjacent to a plurality of teeth equal in number to or less than the adjacent tooth. Estimation model 1140a is an example of first estimation model 1141 shown in FIG. 5, and includes a neural network (not shown) and parameters (not shown) used by the neural network. In FIG. 8, an example will be described in which the identification unit 1130a on the first stage identifies adjacent teeth whose number is smaller than the number of adjacent teeth corresponding to the input three-dimensional data.

Specifically, when the identification unit 1130a receives three-dimensional data corresponding to a plurality of points forming adjacent teeth and gums of numbers 3 to 7 on the left side of the mandible, the identification unit 1130a uses the estimation model 1140a to identify the number 3 Identifies neighboring teeth that are adjacent to a plurality of teeth among teeth numbered 1 through 7. For example, the identification unit 1130a estimates a color a that corresponds to a set of teeth (neighboring teeth) consisting of teeth Nos. 4 to 7 among Nos. 3 to 7 and is an index thereof. Color a is a label associated with a group of teeth (adjacent teeth) in which teeth No. 4 to No. 7 are set. Further, the identification unit 1130a estimates the color b that corresponds to a group of teeth (neighboring teeth) in which the teeth No. 3 to No. 6 are set (adjacent teeth) among the teeth No. 3 to No. 7 and that is an index thereof. Color b is a label associated with a group of teeth (adjacent teeth) in which teeth No. 3 to No. 6 are set. Alternatively, the identification unit 1130a may estimate the color corresponding to any one group of teeth (adjacent teeth). The identification result obtained by the identifying section 1130a is input to the second stage identifying section 1130b. The 4th to 7th adjacent teeth and the 3rd to 6th adjacent teeth are examples of the "second adjacent teeth".

In this way, the identification unit 1130a on the first stage uses the three-dimensional data corresponding to the input three-dimensional data corresponding to a plurality of points constituting the adjacent teeth and gingiva of numbers 3 to 7 and the estimation model 1140a to determine numbers 4 to 4. Discriminate between adjacent teeth in which tooth No. 7 is a set and adjacent teeth in which teeth No. 3 to 6 are in a set. Alternatively, identifier 1130a identifies any one adjacent tooth.

The identification unit 1130b on the second stage is an example of the first identification unit 1131 shown in FIG. Based on the estimated model 1140b, adjacent teeth are identified that are adjacent to a plurality of teeth equal in number to or less than the adjacent tooth. The estimation model 1140b is an example of the first estimation model 1141 shown in FIG. 5, and includes a neural network (not shown) and parameters (not shown) used by the neural network. The identification unit 1130b on the second stage is connected to the rear stage of the identification unit 1130a on the first stage, and includes three-dimensional position information for each of a plurality of points constituting adjacent teeth identified by the identification unit 1130a. Three-dimensional data is input to identification section 1130b. In FIG. 8, an example will be described in which the identification unit 1130b on the second stage identifies adjacent teeth whose number is smaller than the number of adjacent teeth corresponding to the input three-dimensional data.

Specifically, the identification unit 1130b outputs the identification results of the 4th to 7th adjacent teeth output by the identification unit 1130a (for example, the estimation result of the color a, the three-dimensional data corresponding to the 4th to 7th adjacent teeth). data), the estimation model 1140b is used to identify neighboring teeth adjacent to a plurality of teeth among the 4th to 7th neighboring teeth. For example, identification part 1130b corresponds to a group of teeth (adjacent teeth) in which teeth No. 6 and No. 7 are set among No. 4 to No. 7 adjacent teeth, and colors c and no. and 6 corresponding to a set of teeth (adjacent teeth), and corresponding to the color d that is their index, and corresponding to a set of teeth (adjacent teeth) of the 4th and 5th teeth Estimate the color e that is their index. Color c is a label associated with a group of teeth (neighboring teeth) in which teeth No. 6 and 7 are set, and color d is a label associated with a group of teeth in which teeth No. 5 and 6 are a set ( adjacent teeth), and the color e is a label associated with a tooth group (adjacent teeth) in which teeth No. 4 and No. 5 are set. Alternatively, the identification unit 1130b may estimate the color corresponding to any one group of teeth (adjacent teeth). The identification result obtained by the identifying section 1130b is input to the identifying section 1130c on the third stage. The 4th and 5th adjacent teeth, the 5th and 6th adjacent teeth, and the 6th and 7th adjacent teeth are examples of the "second adjacent teeth".

In addition, the identification unit 1130b in the second stage outputs the identification results of the 3rd to 6th adjacent teeth output by the identification unit 1130a (for example, the estimation result of the color b, the tertiary data corresponding to the 3rd to 6th adjacent teeth). Based on the original data), the estimation model 1140b is used to identify adjacent teeth among the 3rd to 6th adjacent teeth that are adjacent to each other. For example, identification unit 1130b corresponds to a group of teeth (adjacent teeth) in which teeth No. 5 and No. 6 are a set of adjacent teeth No. 3 to No. 6, and colors d and No. and 5 correspond to a set of teeth (adjacent teeth), and the color e, which is their index, corresponds to a set of teeth (adjacent teeth) of teeth 3 and 4, and Estimate the color f that is their index. Color f is a label associated with a group of teeth (adjacent teeth) in which teeth No. 3 and No. 4 are set. Alternatively, the identification unit 1130b may estimate the color corresponding to any one group of teeth (adjacent teeth). The identification result obtained by the identifying section 1130b is input to the identifying section 1130c on the third stage. The 3rd and 4th adjacent teeth, the 4th and 5th adjacent teeth, and the 4th and 6th adjacent teeth are examples of the "second adjacent teeth".

Among the second adjacent teeth, the second adjacent teeth identified by the identification unit 1130b based on the identification results of the 4th to 7th adjacent teeth (4th and 5th adjacent teeth, 5th and 6th adjacent teeth, 6 and 7 adjacent teeth) and the second adjacent teeth identified by the identification unit 1130b based on the identification results of the 3rd to 6th adjacent teeth (3rd and 4th adjacent teeth, 4th and 5th adjacent teeth adjacent teeth, adjacent teeth of No. 4 and No. 6), each tooth No. 4 to No. 6 overlaps.

In this way, the identification unit 1130b in the second stage performs 6 Adjacent teeth with teeth No. 7 and No. 7, adjacent teeth with teeth No. 5 and No. 6, adjacent teeth with teeth No. 4 and No. 5, teeth No. 3 and Tooth No. 4 identifies adjacent teeth in a set. Alternatively, identifier 1130b identifies any one adjacent tooth. Note that the identification unit 1130b uses at least one of the input identification results of the 4th to 7th adjacent teeth and the input identification results of the 3rd to 6th adjacent teeth, and the estimation model 1140b. Based on the above, adjacent teeth set of teeth 6 and 7, adjacent teeth set of teeth 5 and 6, and adjacent teeth set of teeth 4 and 5 , and the set of teeth #3 and #4.

The identification unit 1130c on the third stage is an example of the second identification unit 1132 shown in FIG. Identify the type of tooth contained in the tooth. Estimation model 1140c is an example of second estimation model 1142 shown in FIG. 5, and includes a neural network (not shown) and parameters (not shown) used by the neural network. The identification unit 1130c on the third stage is connected to the rear stage of the identification unit 1130b on the second stage, and includes three-dimensional position information for each of a plurality of points constituting adjacent teeth identified by the identification unit 1130b. Three-dimensional data is input to identification section 1130c.

Specifically, the identification unit 1130c outputs the identification results of the 6th and 7th adjacent teeth output by the identification unit 1130b (for example, the estimation result of the color c, the three-dimensional data), the estimated model 1140c is used to identify the tooth type for each of the 6th and 7th adjacent teeth. For example, the identification unit 1130c estimates the color h that is the indicator of the number 6 and the color g that is the indicator of the number 7 among the adjacent teeth numbered 6 and 7. Color h is the label associated with tooth #6, and color g is the label associated with tooth #7. Alternatively, the identification unit 1130c may estimate the color (type) corresponding to any one tooth.

In addition, the identification unit 1130c at the third stage outputs the identification results of the 5th and 6th adjacent teeth output by the identification unit 1130b (for example, the estimation result of the color d, the tertiary data corresponding to the 5th and 6th adjacent teeth). (original data), the estimation model 1140c is used to identify the tooth type for each of the 5th and 6th adjacent teeth. For example, the identification unit 1130c estimates the color i that is the index of number 5 and the color h that is the index of number 6 of the adjacent teeth numbered 5 and 6. Color i is the label associated with tooth #5. Alternatively, the identification unit 1130c may estimate the color (type) corresponding to any one tooth.

In addition, the identification unit 1130c at the third stage uses the identification result (for example, the estimation result of the color e, the tertiary data corresponding to the 4th and 5th adjacent teeth) of the 4th and 5th adjacent teeth output by the identification unit 1130b. (original data), the estimated model 1140c is used to identify the tooth type for each of the 4th and 5th adjacent teeth. For example, the identification unit 1130c estimates the color j that is the index of No. 4 and the color i that is the index of No. 5 among the adjacent teeth No. 4 and No. 5. Color j is the label associated with tooth #4. Alternatively, the identification unit 1130c may estimate the color (type) corresponding to any one tooth.

Furthermore, the identification unit 1130c at the third stage uses the identification results of the third and fourth adjacent teeth output by the identification unit 1130b (for example, the estimation result of the color f, the tertiary (original data), the estimated model 1140c is used to identify the tooth type for each of the 3rd and 4th adjacent teeth. For example, the identification unit 1130c estimates the color k that is the index of number 3 and the color j that is the index of number 4 of the adjacent teeth numbered 3 and 4. Color k is the label associated with tooth #3. Alternatively, the identification unit 1130c may estimate the color (type) corresponding to any one tooth.

In this way, the identification unit 1130c on the third stage obtains the input identification results of the adjacent teeth Nos. 6 and 7, the identification results of the adjacent teeth Nos. 5 and 6, and the adjacent teeth Nos. 4 and 5. , the identification results of adjacent teeth No. 3 and No. 4, and the estimation model 1140c, the types of teeth No. 3 to No. 7 are identified. Alternatively, the identification unit 1130c identifies any one tooth type. The identification unit 1130c receives identification results of adjacent teeth No. 6 and 7, identification results of adjacent teeth No. 5 and 6, identification results of adjacent teeth No. 4 and 5, and 3 Based on at least one of the identification results of adjacent teeth No. 3 and No. 4 and the estimation model 1140c, at least one tooth type among teeth No. 3 to No. 7 is identified. You may

In such a configuration, when three-dimensional data including three-dimensional position information on each of a plurality of points forming adjacent teeth is input, the identification device 100 identifies the positions of the plurality of teeth based on the three-dimensional data. Each feature is extracted using a plurality of multistage estimation models 1141 (1140a, 1140b) and 1142 (1140c), and the type of each of the plurality of teeth is identified based on the features of each of the extracted teeth. can do. The identification device 100 can extract not only generally recognized tooth features but also generally unrecognized tooth features, thereby accurately identifying the type of tooth. .

Further, since the identification apparatus 100 identifies teeth while gradually reducing the number of adjacent teeth using a plurality of multistage estimation models 1141 (1140a, 1140b) and 1142 (1140c), a large number of teeth can be identified. It is possible to identify the type of tooth with higher accuracy than when identifying the type of each tooth at once based on the three-dimensional data corresponding to adjacent teeth adjacent to each other. More specifically, when identifying one tooth at a time, it is necessary to identify only the characteristics of that tooth (or teeth adjacent to that tooth). Since there are 16 types of teeth, when identifying each tooth type consisting of 16 types at once, it is necessary to identify only the characteristics of each tooth (or teeth adjacent to each tooth), and the accuracy of identification is difficult. won't go up On the other hand, when identifying a plurality of teeth, the number of features is greater than when identifying a single tooth, so the accuracy of identification is improved. Furthermore, by making identification of a plurality of teeth multistage, identification is performed only in the plurality of teeth identified in the first stage from the second stage onwards, so that the accuracy is further improved. For example, if the first row identifies four teeth (numbers 4, 5, 6, and 7) and the second row identifies two teeth, the second row identifies three types of teeth (numbers 4 and 7). 5, 5 and 6, 6 and 7) are only required to be identified, resulting in greater accuracy.

Furthermore, when estimating adjacent teeth step by step, the identification device 100 identifies overlapping teeth (each of Nos. 4 to 6) in a plurality of second adjacent teeth a plurality of times. The type of tooth can be identified with high accuracy.

As described above, in the example shown in FIG. 8, the identification unit 1130b on the second stage identifies a smaller number of adjacent teeth than the adjacent teeth corresponding to the input three-dimensional data. Part 1130b may identify the same number of adjacent teeth as the adjacent teeth corresponding to the input three-dimensional data.

For example, the identification unit 1130b in the second stage identifies the 3rd to 6th adjacent teeth using the estimation model 1140b based on the identification results of the 3rd to 6th adjacent teeth output by the identification unit 1130a. . Based on the identification results of the 3rd to 6th adjacent teeth output by the identification unit 1130b, the identification unit 1130c in the third stage uses the estimation model 1140c to identify the 3rd to 6th adjacent teeth. Each tooth type may be identified.

Further, the identification unit 1130b in the second stage identifies the 4th to 7th adjacent teeth using the estimation model 1140b based on the identification results of the 4th to 7th adjacent teeth output by the identification unit 1130a. . Then, the identification unit 1130c in the third stage uses the estimation model 1140c based on the identification results of the 4th to 7th adjacent teeth output by the identification unit 1130b to identify the 4th to 7th adjacent teeth. Each tooth type may be identified.

[Generation of learning data]
An example of generating the learning data set 116 will be described with reference to FIGS. 9 to 11. FIG. FIG. 9 is a schematic diagram for explaining generation of learning data according to the present embodiment. FIG. 10 is a schematic diagram for explaining an example of learning data according to the present embodiment. FIG. 11 is a schematic diagram showing an example of color classification data according to this embodiment.

As shown in FIG. 9, first, three-dimensional data is acquired by the three-dimensional scanner 200 (STEP 1). The three-dimensional data acquired by the three-dimensional scanner 200 includes three-dimensional position information at each of a plurality of points that constitute the teeth and gums corresponding to the three-dimensional data, and color information (RGB) at each of the plurality of points. value). When a three-dimensional image is generated based on the three-dimensional data acquired by the three-dimensional scanner 200, a three-dimensional image including teeth and gums with actual colors is generated as shown in FIG. 9(a). .

Next, noise removal processing is performed in preparation for color-coding processing for each tooth, which will be described later. For example, in the present embodiment, a 3D image corresponding to 3D data is grayscaled (STEP 2). Grayscaling of the three-dimensional image is performed by the user 1 (in this case, a manufacturer engineer who generates learning data, an operator at a manufacturing plant, a dentist, a dental technician, or other person with dental knowledge). will be When the 3D image is grayscaled, a 3D image including the grayscaled teeth is generated as shown in FIG. 9(b). Further, in accordance with the grayscaling of the 3D image, the color information (RGB values) at each point of the tooth corresponding to the 3D data is changed to a value (for example, 192192192) corresponding to the grayscale.

Next, each tooth is color-coded by applying a predetermined color to each tooth included in the three-dimensional image corresponding to the three-dimensional data (STEP 3).

For example, as shown in FIG. 10, by applying color a to a group of teeth (adjacent teeth) consisting of teeth No. 4 to No. 7, adjacent teeth No. 4 to No. 7 and color a and learning data associated with is generated. This learning data is used for learning the estimation model 1140a shown in FIG. That is, the estimation model 1140a is based on the three-dimensional data corresponding to the 3rd to 7th adjacent teeth including the 4th to 7th adjacent teeth as shown in FIG. 9(a) or 9(b). Adjacent teeth No. 7 to No. 7 are estimated and output as color information, and learning is performed based on the result of comparison between the estimation result and color information (color a) which is correct data.

Further, as shown in FIG. 10, by applying color c to a group of teeth (adjacent teeth) in which teeth No. 6 and No. 7 are set, adjacent teeth No. 6 and No. 7 and color c and learning data associated with is generated. This learning data is used for learning the estimation model 1140b shown in FIG. That is, the estimation model 1140b estimates the 6th and 7th adjacent teeth based on the three-dimensional data corresponding to the 4th to 7th adjacent teeth including the 6th and 7th adjacent teeth, and outputs them as color information. Then, learning is performed based on the result of comparison between the estimation result and the color information (color c) that is the correct data.

Further, as shown in FIG. 10, by applying the color g to the tooth number 7, learning data in which the tooth number 7 and the color g are associated is generated. This learning data is used for learning the estimation model 1140c shown in FIG. That is, the estimation model 1140c estimates the 7th tooth based on the three-dimensional data corresponding to the 6th and 7th adjacent teeth including the 7th tooth, and outputs it as color information. is learned based on the result of comparison with the color information (color g).

As shown in FIG. 11, the color classification data 118 for color-coding one tooth or adjacent teeth consisting of a plurality of teeth is used for each region in the oral cavity such as the left lower jaw, the right lower jaw, the left upper jaw, and the right upper jaw. provided for each. FIG. 11 shows color classification data 118 corresponding to the left side of the lower jaw. In each color classification data 118, a tooth number generally used in the dental field and predetermined color information are assigned to each tooth type.

For example, the second molar is assigned number 7 as the tooth number and color g as the color information. The first molar is assigned number 6 as the tooth number and color h as color information. The second premolar is assigned number 5 as the tooth number and color i as the color information. In this way, in each color classification data 118, color information is assigned in advance to each adjacent tooth consisting of one tooth or a plurality of teeth.

The application of color to each tooth is performed by a user 1 (in particular, a person with dental knowledge such as a dentist, dental technician, etc.). Specifically, the user 1 identifies the type of each tooth included in the three-dimensional image based on his/her own knowledge, identifies the color corresponding to the identified tooth type with reference to the color classification data 118, Apply the specified color to the image of the tooth.

For example, when the user 1 identifies that the tooth included in the three-dimensional image is the number 7 tooth, the user 1 applies the color g to the image of the tooth. Further, when the tooth included in the three-dimensional image is identified as tooth number 6, the color h is applied to the image of the tooth. In addition, in FIGS. 9 and 10, each color is represented by hatching for easy understanding.

In addition, according to the color coding of one tooth or adjacent teeth consisting of a plurality of teeth, the color information (RGB values) at each point of the tooth corresponding to the three-dimensional data is changed to a value corresponding to the color applied to each tooth. be done. For example, the color information (RGB value) is "255000000" for each of the position coordinates of the 4th to 7th adjacent teeth painted in color a (red), and 6 is painted in color c (green). The color information (RGB value) is "000255000" for each of the position coordinates of the adjacent teeth No. 7 and No. 7, and the position coordinates of the No. 7 tooth painted in color g (blue) is: The color information (RGB values) becomes "00000255". That is, predetermined color information (RGB values) is associated with each point of the tooth corresponding to the three-dimensional data.

When predetermined color information is associated with each tooth, the three-dimensional data includes position information and color information corresponding to the applied color. Adopted as training data. That is, in the learning data according to the present embodiment, color information corresponding to the type of tooth is associated (labeled) with position information referred to in identification processing. Further, color information is associated with the three-dimensional data so that each range of a plurality of teeth corresponding to the three-dimensional data can be specified. Specifically, the same color information is associated with each piece of position information corresponding to each tooth. A collection of such learning data is held in the identification device 100 as a learning data set 116 .

In this way, there are many advantages in labeling correct data by applying colors to each tooth included in the three-dimensional image by the user 1 when generating learning data. For example, if the labeling is done using simple characters or symbols, the user 1 will find it difficult to recognize the range of each tooth, but if the labeling is done by color coding, the user 1 will be able to distinguish between the tooth to be labeled and the adjacent tooth. , and between the tooth and gingiva to be labeled can be easily recognized by the application of color. In addition, the user 1 applies the color while checking the three-dimensional image from various angles during labeling. becomes easier to recognize.

In this embodiment, the user 1 manually applies the color to each tooth included in the three-dimensional image based on his/her own knowledge, but it is also possible to supplement part of the work with software. . For example, the boundary between the tooth to be labeled and the tooth adjacent to it, and the boundary between the tooth to be labeled and the gingiva may be identified by edge detection, such that: Only teeth that are labeling targets can be extracted.

Generating the learning data set 116 shown in FIGS. 9 and 10 can also be applied to generating the learning data set 516 held by the server device 500 . For example, learning data set 116 shown in FIG. 10 may be applied to learning data set 516 held by server device 500, and color classification data 118 shown in FIG. It may be applied to data 518 .

[Generate trained model]
An example of generating a learned model will be described with reference to FIG. FIG. 12 is a schematic diagram for explaining generation of a trained model based on learning data set 116 according to the present embodiment.

As shown in FIG. 12, the learning data set 116 can be classified into categories based on the profile of the subject 2 who was scanned when the learning data set 116 was generated. For example, age (minor, working generation, elderly), gender (male, female), race (Asian, Western, African), height (less than 150 cm, 150 or more), weight (less than 50 kg, 50 kg) above), and place of residence (resident in Japan, residing outside of Japan) can be assigned a learning data set generated from three-dimensional data including the teeth of the corresponding subject 2 . Note that the stratification of each category can be set as appropriate. For example, with respect to age, for each predetermined age difference (every three years in this case), specifically, more detailed information such as 0 to 3 years old, 4 to 6 years old, 7 to 9 years old, etc. can be stratified into

The identification device 100 generates a trained model by training the estimation model 114 using a plurality of training data sets 116a to 116o that can be classified by category. Note that the learning data may overlap depending on how the categories are classified, but if the learning data overlaps, the estimation model 114 may be trained using only one of the learning data. .

In general, tooth shape differs depending on genetics or living environment such as age, sex, race, height, weight, place of residence, and the like. For example, the permanent teeth of adults are generally larger than the milk teeth of children, and the two have different shapes. In general, male teeth are larger than female teeth, and the two have different shapes. In general, the teeth of Westerners tend to have sharp tips so that they can easily bite off hard meat and bread, whereas the teeth of Japanese people tend to have sharp tips so that they can easily grind soft rice and vegetables. It tends to be smooth. Therefore, by performing learning processing based on profile data as in the present embodiment, it is possible to generate a trained model that can identify the type of tooth in consideration of heredity or living environment.

Note that the generation of the trained model shown in FIG. 12 can also be applied to the generation of the trained model held by the server device 500 . For example, the learning datasets 116a to 116o shown in FIG. 12 may be applied to the learning dataset 516 held by the server device 500, and the estimation model 114 shown in FIG. It may be applied to model 514 .

[Learning of estimation model]
The learning of the estimation model will be described with reference to FIGS. 13 to 15. FIG. The estimation algorithms executed by the identification unit 1101 include, for example, an estimation algorithm for estimating adjacent teeth consisting of four teeth, an estimation algorithm for estimating adjacent teeth consisting of two teeth, and an estimation algorithm for estimating adjacent teeth consisting of two teeth. There are estimation algorithms for estimating teeth.

The estimation algorithm for estimating adjacent teeth consisting of four teeth is, for example, in the case of the lower jaw, the 4th to 7th adjacent teeth on the lower left, the 3rd to 6th adjacent teeth on the lower left, and the 2nd to 5th lower left. lower left 1-4 adjacent teeth, lower right 1 and lower left 1-3 adjacent teeth, lower right 1 and 2 and lower left 1 and 2 adjacent teeth, lower right 1 No. 3 to No. 3 and lower left adjacent teeth, lower right No. 1 to 4 adjacent teeth, lower right No. 2 to 5 adjacent teeth, lower right No. 3 to 6 adjacent teeth, lower right No. 4 and up It contains all pattern estimation algorithms for estimating four-tooth adjacent teeth such as the number 7 adjacent tooth. Furthermore, the estimation algorithm for estimating adjacent teeth consisting of four teeth similarly includes estimation algorithms for all patterns in the case of the upper jaw.

The estimation algorithm for estimating adjacent teeth consisting of two teeth is, for example, the lower left 4th to 7th adjacent teeth, the lower left 4th and 5th adjacent teeth, the lower left 5th and 6th adjacent teeth Includes all pattern estimation algorithms for estimating adjacent teeth consisting of two teeth, such as teeth, lower left 6th and 7th adjacent teeth. Furthermore, the estimation algorithm for estimating the adjacent teeth consisting of two teeth is the lower left 3rd to 6th adjacent teeth, the lower left 2nd to 5th adjacent teeth, the lower left 1st to 4th adjacent teeth, the right Lower 1st and lower left 1st to 3rd adjacent teeth, lower right 1st and 2nd and lower left 1st and 2nd adjacent teeth, lower right 1st to 3rd and lower left 1st adjacent teeth, lower right 1 No. 4 to No. 4 adjacent teeth, lower right No. 2 to No. 5 adjacent teeth, lower right No. 3 to No. 6 adjacent teeth, lower right No. 4 to No. 7 adjacent teeth, and in the case of the upper jaw, all Contains pattern estimation algorithms.

The estimation algorithm for estimating one tooth is, for example, in the case of the lower left 6th and 7th adjacent teeth, all the patterns for estimating one tooth, such as each of the lower left 6th and 7th teeth. contains an estimation algorithm for Further, the estimation algorithm for estimating a single tooth is: lower left 5th and 6th adjacent teeth, lower left 4th and 5th adjacent teeth, lower left 3rd and 4th adjacent teeth, lower left 2nd and 3rd adjacent teeth, 1st and 2nd lower left adjacent teeth, 1st lower left and 1st lower left adjacent teeth, 1st lower right and 2nd lower right adjacent teeth, 2nd lower right and 3rd lower right adjacent teeth Adjacent teeth, lower right 3rd and lower right 4th neighboring teeth, lower right 4th and 5th neighboring teeth, lower right 5th and 6th neighboring teeth, lower right 6th and 7th neighboring teeth, upper jaw also includes all pattern estimation algorithms.

The identification unit 1101 first executes an estimation algorithm for estimating adjacent teeth consisting of four teeth based on three-dimensional data scanned by the three-dimensional scanner 200 at predetermined intervals (every several milliseconds). Specifically, when only one tooth is scanned by the three-dimensional scanner 200, the identification unit 1101 does not have all the data for identification yet, so the identification unit 1101 executes an estimation algorithm for estimating adjacent teeth. Instead, it does not output the identification result, or outputs information indicating that the identification result cannot be output. Scanning by the three-dimensional scanner 200 progresses, and when 4 or more teeth are scanned eventually, or when about 80% of 5 teeth are scanned, or when 4 or more teeth are scanned, the amount of data is acquired. When a predetermined amount of accumulated data that can be estimated to have been obtained is input, the identification unit 1101 executes the estimation algorithm for estimating adjacent teeth consisting of four teeth as described above, and outputs identification results. After that, the identification unit 1101 performs an estimation algorithm for estimating adjacent teeth consisting of two teeth as described above, based on the identification result obtained by executing the estimation algorithm for estimating adjacent teeth consisting of four teeth. Run the estimation algorithm and output the identification result. After that, the identification unit 1101 executes the above-described estimation algorithm for estimating one tooth based on the identification result obtained by executing the estimation algorithm for estimating adjacent teeth consisting of two teeth. and output the identification result.

FIG. 13 is a schematic diagram for explaining learning of first-stage estimation model 1140a of estimation models 114 according to the present embodiment. As shown in FIG. 13, the identification unit 1130a receives the three-dimensional data of a plurality of points forming adjacent teeth and gingiva of Nos. 3 to 7 among the learning data included in the learning data set 116. Then, the estimation model 1140a is learned by learning processing based on the three-dimensional data.

Specifically, when the three-dimensional data of a plurality of points forming adjacent teeth and gums numbered 3 to 7 is input, the identification unit 1130a performs preprocessing such as noise removal on the three-dimensional data. conduct. Next, the identification unit 1130a executes an estimation algorithm for estimating adjacent teeth consisting of four teeth, thereby determining four adjacent teeth (for example, number 4) among the adjacent teeth numbered 3 to 7. to 7 adjacent teeth, 3 to 6 adjacent teeth) are extracted. Next, the identification unit 1130a identifies the four adjacent teeth whose features have been extracted by identification processing. Next, the identification unit 1130a identifies the identification result (eg, estimated color information) obtained by the identification processing, and the color information (eg, adjacent teeth Nos. 4 to 7) which is correct data included in the learning data. and color b) corresponding to adjacent teeth Nos. 3 to 6 are matched, and parameters included in estimation model 1140a are adjusted based on the result of the determination.

Note that the identification unit 1130a identifies a plurality of points that constitute five adjacent teeth other than the 3rd to 7th teeth (for example, the 2nd to 6th adjacent teeth and the 1st to 5th adjacent teeth). Even when three-dimensional data is input, the parameters included in estimation model 1140a are adjusted by executing the same learning process as described above.

As shown in FIG. 14, the identification unit 1130b receives three-dimensional data of a plurality of points forming adjacent teeth Nos. 4 to 7 among the learning data included in the learning data set 116. Based on the three-dimensional data, the estimation model 1140b is learned by learning processing.

When the three-dimensional data of a plurality of points forming adjacent teeth No. 4 to No. 7 are input, the identification unit 1130b performs preprocessing such as noise removal on the three-dimensional data. Next, the identification unit 1130b executes an estimation algorithm for estimating adjacent teeth consisting of two teeth, thereby determining two adjacent teeth (for example, number 6) among the number 4 to number 7 adjacent teeth. and No. 7 adjacent teeth, No. 5 and No. 6 adjacent teeth, No. 4 and No. 5 adjacent teeth). Next, the identification unit 1130b identifies two adjacent teeth from which features have been extracted by identification processing. Next, the identification unit 1130b generates the identification result obtained by the identification process (eg, estimated color information) and the color information (eg, adjacent teeth No. 6 and No. 7) which is correct data included in the learning data. color c corresponding to , color d corresponding to the 5th and 6th adjacent teeth, and color e) corresponding to the 4th and 5th adjacent teeth match, and based on the determination result Adjust parameters included in estimation model 1140b.

Note that the identification unit 1130b can also receive three-dimensional data of a plurality of points that form four adjacent teeth other than the 4th to 7th teeth (for example, the 3rd to 6th adjacent teeth). , the parameters included in the estimation model 1140b are adjusted by executing the same learning process as described above.

As shown in FIG. 15, the identification unit 1130c receives three-dimensional data of a plurality of points forming adjacent teeth Nos. 6 and 7 among the learning data included in the learning data set 116. Based on the three-dimensional data, the estimation model 1140c is learned by learning processing.

When the 3D data of a plurality of points forming the 6th and 7th adjacent teeth are input, the identification unit 1130c performs preprocessing such as noise removal on the 3D data. Next, the identifying unit 1130c executes an estimation algorithm for estimating one tooth, thereby determining one tooth (for example, tooth 7, tooth 6) among adjacent teeth numbered 6 and 7. teeth). Next, the identification unit 1130c identifies the type of one tooth from which the features have been extracted by identification processing. Next, the identification unit 1130c identifies the identification result (eg, estimated color information) obtained by the identification processing, and the color information (eg, the color corresponding to tooth number 7) that is correct data included in the learning data. It is determined whether or not g matches the color h) corresponding to tooth number 6, and the parameters included in the estimation model 1140c are adjusted based on the determination result.

Note that the identification unit 1130c identifies a plurality of points that constitute two adjacent teeth other than the 6th and 7th teeth (for example, the 5th and 6th adjacent teeth, the 4th and 5th adjacent teeth). Even when three-dimensional data is input, the parameters included in the estimation model 1140c are adjusted by executing the same learning process as described above.

In this way, the learning of the first estimation model 1141 (estimation models 1140a and 1140b in this embodiment) by the first identification unit 1131 (identification units 1130a and 1130b in this embodiment) and the second identification unit 1132 (this embodiment In the embodiment, learning of the second estimation model 1142 (estimation model 1140c in this embodiment) is performed by the identification unit 1130c).

In the above example, the identification unit 1130a and the identification unit 1130b are illustrated as the "first identification unit", but the identification device 100 may not have the identification unit 1130a. For example, the identification device 100 has an identification unit 1130b connected to the input unit 1102 as a "first identification unit" and an identification unit 1130c connected to the identification unit 1130b as a "second identification unit". good too. In this case, the 4th to 7th adjacent teeth and the 3rd to 6th adjacent teeth corresponding to the three-dimensional data input to the identification unit 1130b correspond to the "first adjacent tooth". In addition, the 3rd and 4th adjacent teeth, the 4th and 5th adjacent teeth, the 4th and 6th adjacent teeth, the 6th and 7th adjacent teeth corresponding to the three-dimensional data input to the identification unit 1130c corresponds to the "second adjacent tooth".

[Learning processing of identification device]
A learning process executed by the identification device 100 will be described with reference to FIG. 16 . FIG. 16 is a flowchart for explaining an example of the learning process executed by the identification device 100 according to this embodiment. Each step shown in FIG. 16 is realized by executing the OS 127 and the learning program 121 by the arithmetic device 130 of the identification device 100 . Note that the learning process shown in FIG. 16 is performed for each of the plurality of estimation models 1140a, 1140b, and 1140c.

As shown in FIG. 16, the identification device 100 selects learning data to be used for learning from the learning data set 116 (S1). Specifically, the identification device 100 selects one or a plurality of learning data from the learning data sets 116 included in the learning data set group shown in FIG. 12 . Note that the identification device 100 is not limited to automatically selecting learning data, and may use learning data selected by the user 1 for learning processing.

The identification device 100 inputs the three-dimensional data included in the selected learning data and the profile data of the subject 2 who was scanned when the learning data was generated to the estimation model 114 (S2). At this time, the correct answer data labeled with the three-dimensional data is not input to the identification device 100 . The identification device 100 performs identification processing for identifying one tooth or adjacent teeth using the estimation model 114 based on the characteristics of one tooth or adjacent teeth including a plurality of teeth corresponding to the three-dimensional data. (S3). In the identification process, identification device 100 identifies a tooth or adjacent teeth using estimation model 114 based on three-dimensional data as well as profile data.

The identification device 100 updates the parameters 1144 of the estimation model 114 based on the error between the identification result of the identification process and the correct data corresponding to the learning data used in the learning process (S4).

For example, the identification device 100 estimates color information corresponding to one specific tooth as a result of identification based on the position information of a plurality of points forming one specific tooth. Then, the identification device 100 compares the color information (correct data) corresponding to the specific tooth included in the learning data with the color information estimated by itself, and if they match, the parameters of the estimation model 114 are changed. On the other hand, if the answer is incorrect, the parameters of the estimation model 114 are updated so that both match. Alternatively, the identification device 100 estimates color information corresponding to the specific adjacent tooth as a result of identification based on position information of a plurality of points forming adjacent teeth including the specific teeth. Then, the identification device 100 compares the color information (correct data) corresponding to the specific adjacent tooth included in the learning data with the color information estimated by itself, and if they match, the parameters of the estimation model 114 are maintained. On the other hand, if the answer is incorrect, the parameters of the estimation model 114 are updated so that both match.

Note that the identification device 100 estimates the color information corresponding to the specific tooth as a result of identification based on the position information of the specific tooth, and based on the color classification data 118, the type of tooth and the type of tooth corresponding to the color information. A number (correct answer data) may be specified. The identification device 100 identifies the tooth type and tooth number (correct data) assigned to the color information corresponding to the specific tooth included in the learning data, and the tooth type and tooth number estimated by itself. The parameters of the estimation model 114 may be maintained if they are compared, and if they match, the parameters of the estimation model 114 may be updated so that they match if they are incorrect.

Next, the identification device 100 determines whether or not it has learned based on all the learning data (S5). If the identification device 100 has not learned based on all the learning data (NO in S5), the process returns to S1.

On the other hand, if the identification device 100 has learned based on all the learning data (YES in S5), the identification device 100 stores the learned estimation model 114 as a learned model (S6), and terminates this process.

In this way, the identification device 100 provides tooth information (color information, tooth name, tooth number, etc.) corresponding to one tooth type associated with the three-dimensional data included in the learning data, and a plurality of Using tooth information (color information, tooth name, tooth number, etc.) corresponding to adjacent teeth including teeth as correct data, the estimation model 114 is generated based on the results of tooth identification using the three-dimensional data obtained by identification processing. By learning, a trained model can be generated.

Furthermore, since the identification device 100 learns the estimation model 114 in consideration of profile data in addition to learning data in the learning process, it is possible to generate a trained model considering the profile of the subject 2 . Note that the identification device 100 may perform learning processing on each estimation model 114 (1140a, 1140b, 1140c) without using profile data.

[Learning processing of the server device]
A learning process executed by the server device 500 will be described with reference to FIG. 17 . FIG. 17 is a flowchart for explaining an example of learning processing executed by server device 500 according to the present embodiment. Each step shown in FIG. 17 is implemented by the arithmetic device 530 of the server device 500 executing the OS 527 and the learning program 521 . Note that the learning process shown in FIG. 17 is performed for each of a plurality of estimation models.

As shown in FIG. 17, the server device 500 selects learning data to be used for learning from the learning data set (S501). Here, the learning data may be generated using big data accumulated and stored by the server device 500 . For example, the server device 500 generates learning data in advance using the three-dimensional data included in the scan information acquired from the identification devices 100 of the local A to C and the dental laboratory, and the generated learning data may be used to perform the learning process. Note that the server device 500 is not limited to automatically selecting learning data, and may use learning data selected by the user 1 for learning processing.

The server device 500 inputs the three-dimensional data included in the selected learning data and the profile data of the subject 2 who was scanned when the learning data was generated to the estimation model 514 (S502). At this time, the correct answer data labeled with the three-dimensional data is not input to the server device 500 . The server device 500 performs identification processing to identify one tooth or adjacent teeth using the estimation model 514 based on the features of one tooth or adjacent teeth including a plurality of teeth corresponding to the three-dimensional data. (S503). In the identification process, server device 500 identifies a tooth or adjacent teeth using estimation model 514 based on profile data in addition to three-dimensional data.

The server device 500 updates the parameters of the estimation model 514 based on the error between the identification result of the identification processing and the correct data corresponding to the learning data used for learning (S504).

For example, server device 500 estimates color information corresponding to a specific tooth as a result of identification based on the position information of the specific tooth. Server device 500 compares the color information (correct data) corresponding to the specific tooth included in the learning data set with the color information estimated by itself, and if they match, maintains the parameters of estimation model 514. If the answer is incorrect, the parameters of the estimation model 514 are updated so that both match.

Alternatively, server device 500 estimates color information corresponding to one specific tooth as a result of identification based on the position information of a plurality of points forming one specific tooth. Then, server device 500 compares the color information (correct data) corresponding to the specific one tooth included in the learning data with the color information estimated by itself, and if they match, the parameters of estimation model 514 are changed. On the other hand, if the answer is incorrect, the parameters of the estimation model 514 are updated so that both match. Alternatively, the server device 500 estimates color information corresponding to the specific adjacent tooth as a result of identification based on position information of a plurality of points forming adjacent teeth including the specific teeth. Then, the server device 500 compares the color information (correct data) corresponding to the specific adjacent tooth included in the learning data with the color information estimated by itself, and maintains the parameters of the estimation model 514 if they match. On the other hand, if the answer is incorrect, the parameters of the estimation model 514 are updated so that both match.

Server device 500 estimates the color information corresponding to the specific tooth as a result of identification based on the position information of the specific tooth, and based on the color classification data 518, the type of tooth and the type of tooth corresponding to the color information. A number (correct answer data) may be specified. The server device 500 stores the tooth type and tooth number (correct data) assigned to the color information corresponding to the specific tooth included in the learning data, and the tooth type and tooth number estimated by itself. The parameters of the estimation model 514 may be maintained if the comparison is made and if they match, while the parameters of the estimation model 514 may be updated so that the two match if the answer is incorrect.

Next, the server apparatus 500 determines whether or not learning has been performed based on all the learning data (S505). If the server apparatus 500 has not learned based on all the learning data (NO in S505), the process returns to S501.

On the other hand, if the server device 500 has learned based on all the learning data (YES in S505), the server device 500 stores the learned estimation model 514 as a learned model (S506). After that, the server device 500 transmits the generated learned model to each local identification device 100 (S507), and ends this process.

In this way, the server device 500 provides tooth information (color information, tooth name, tooth number, etc.) corresponding to one tooth type associated with the three-dimensional data included in the learning data, and a plurality of Using tooth information (color information, tooth name, tooth number, etc.) corresponding to adjacent teeth including a tooth as correct data, an estimation model 514 is generated based on the identification result of the tooth using the three-dimensional data in the identification process. By learning , a trained model can be generated.

In addition, in the learning process, the server device 500 learns the estimation model 514 in consideration of the profile data in addition to the learning data. Note that server device 500 may perform learning processing on each estimation model 514 without using profile data.

Furthermore, since the server device 500 uses the three-dimensional data included in the scan information acquired from the identification devices 100 of the local A to C and the dental laboratory as learning data used in the learning process, the identification device 100 The learning process can be executed based on a larger amount of learning data than the learning process that is executed each time, and a trained model that can identify the type of tooth with higher accuracy can be generated.

[Service provision processing of identification device]
The service providing process executed by the identification device 100 will be described with reference to FIG. 18 . FIG. 18 is a flowchart for explaining an example of service providing processing executed by the identification device 100 according to this embodiment. Each step shown in FIG. 18 is realized by executing the OS 127 and the identification program 120 by the arithmetic device 130 of the identification device 100 .

As shown in FIG. 18, the identification device 100 determines whether three-dimensional data has been input (S41). For example, the identification device 100 determines whether or not a sufficient amount of three-dimensional data has been input to perform identification processing. If a sufficient amount of three-dimensional data has not been input (NO in S41), the identification device 100 ends this process.

On the other hand, when a sufficient amount of three-dimensional data has been input (YES in S41), the identification device 100 determines whether the user 1 has input the profile data of the subject 2 (S42). If profile data has not been input (NO in S42), identification device 100 inputs three-dimensional data to a first-stage trained model (for example, trained estimation model 1140a) (S43). On the other hand, if the profile data has been input (YES in S42), the identification device 100 inputs the three-dimensional data and the profile data to the learned model in the first stage (S44). Note that the trained model used at this time is not limited to the trained model generated by the identification device 100 in the learning process shown in FIG. There may be.

After S43 and S44, the identification device 100 performs identification processing using the first-stage learned model based on the features of a plurality of teeth corresponding to the input three-dimensional data, and corresponds to the input three-dimensional data. Adjacent teeth including some of the plurality of teeth are identified (S45). At this time, if the profile data has been input to the first learned model in S42, the identification device 100 identifies the type of the tooth using the learned model based on the profile data in addition to the three-dimensional data. do. In this case, the type of tooth can be identified with higher accuracy than when identifying the type of tooth using a trained model based only on three-dimensional data.

Next, the identification device 100 inputs the identification result of the first-stage trained model to the second-stage trained model (for example, the trained estimation model 1140b) (S46). Based on the identification result of the first-stage trained model, the identification device 100 performs identification processing using the second-stage trained model to select one of a plurality of teeth identified by the first-stage trained model. Identify adjacent teeth that include multiple teeth of the part (S47).

Next, the identification device 100 inputs the identification result of the second trained model to the third trained model (for example, the trained estimation model 1140c) (S48). Based on the identification result of the second-stage trained model, the identification device 100 performs identification processing using the third-stage trained model to select one of the plurality of teeth identified by the second-stage trained model. The type of tooth in the part is identified (S49).

After that, the identification device 100 outputs the identification result obtained by the identification processing using the learned model in the third stage to the display 300, the speaker 400, the server device 500, etc. (S50), and ends this processing. .

In this way, the identification device 100 identifies the type of tooth using a learned model based on the characteristics of a plurality of teeth corresponding to the input three-dimensional data, and therefore relies on the knowledge of the user himself to identify the type of tooth. It is possible to identify the type of tooth with higher accuracy than to identify the type.

In addition, since the identification device 100 identifies teeth while gradually reducing the number of adjacent teeth using a plurality of multistage estimation models (learned models), a large number of teeth can be identified as adjacent adjacent teeth. The type of tooth can be identified with higher accuracy than identifying the type of each tooth based on the corresponding three-dimensional data.

Furthermore, since the identification device 100 identifies the tooth type in consideration of profile data in addition to the input three-dimensional data in the identification process, it is possible to identify the tooth type with higher accuracy.

[Main Disclosure]
As described above, the present embodiment includes the following disclosures.

The identification device 100 includes an input unit 1102 to which three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other is input, and a three-dimensional data input from the input unit 1102. An identification unit 1101 for identifying types of teeth included in adjacent teeth based on data, and an output unit 1103 for outputting the identification result of the identification unit 1101. The identification unit 1101 outputs a first adjacent tooth to which a plurality of teeth are adjacent. Three-dimensional data including three-dimensional positional information at each of a plurality of points constituting teeth (for example, adjacent teeth including teeth Nos. 3 to 6 and adjacent teeth including teeth Nos. 4 to 7); 1 Neural network 1151 and a first estimation model 1141 including a neural network 1151, a second adjacent tooth adjacent to a plurality of teeth less than the first adjacent tooth (e.g., the 6th and 7th adjacent teeth, the 5th and No. 6 adjacent tooth, No. 4 and No. 5 adjacent teeth, No. 3 and No. 4 adjacent teeth), the identification result by the first identification unit 1131, and the second neural network 1152 A second identification unit 1132 that identifies the types of teeth included in the second adjacent teeth (for example, teeth 3 to 6, teeth 4 to 7) based on a second estimation model 1142 including including.

In addition, the first identification portion 1131 identifies a plurality of adjacent first adjacent teeth (for example, adjacent teeth including teeth Nos. 3 to 6 and adjacent teeth including teeth Nos. 4 to 7). Based on the three-dimensional data including the three-dimensional positional information at each of the points and the first estimation model 1141 including the first neural network 1151, the same number of teeth as the adjacent teeth are adjacent to the second adjacent tooth It may also identify teeth (eg, adjacent teeth numbered 3-6, adjacent teeth numbered 4-7).

As a result, the identification device 100 configures adjacent teeth using a plurality of multistage estimation models 114 (first estimation model 1141, second estimation model 1142, first trained model, second trained model). Since the type of tooth included in the adjacent tooth can be identified based on three-dimensional data including three-dimensional positional information for each of a plurality of points, it is possible to identify the type of tooth rather than relying on the user's own knowledge. Also, the type of tooth can be identified with high accuracy. Furthermore, since the identification device 100 identifies teeth while gradually reducing the number of adjacent teeth using a plurality of multi-stage estimation models 114, a plurality of teeth correspond to adjacent teeth in three dimensions. The type of tooth can be identified with higher accuracy than the type of each tooth can be identified at once based on the data.

The identification unit of the identification device may be a two-stage identification unit of the first identification unit and the second identification unit, or may be a three-stage identification unit including identification units other than the first identification unit and the second identification unit. may be a multistage identification unit. For example, the first identification unit is not limited to the identification unit connected to the input unit 1102 immediately after the input unit 1102, and is connected to another identification unit (for example, the identification unit 1130a shown in FIG. 8) immediately after the input unit 1102. It may also refer to a connected identifier (eg, identifier 1130b shown in FIG. 8).

Based on the first estimation model 1141 and the three-dimensional data including the three-dimensional positional information of each of the plurality of points forming the first adjacent tooth, the first identification unit 1131 selects a plurality of second teeth as the second adjacent teeth. Identify each of the adjacent teeth (e.g., 6th and 7th adjacent teeth, 5th and 6th adjacent teeth, 4th and 5th adjacent teeth, 3rd and 4th adjacent teeth); The two adjacent teeth include mutually overlapping teeth (for example, teeth No. 4 to No. 6), and the second identification unit 1132 is based on the identification result of the first identification unit 1131 and the second estimation model 1142, A duplicate tooth type included in each of the plurality of second adjacent teeth is identified.

As a result, the identification device 100 uses the second identification unit 1132 to identify overlapping teeth (each of Nos. 4 to 6) among the plurality of second adjacent teeth a plurality of times. Tooth type can be identified.

The first estimation model 1141 is a second adjacent tooth associated with three-dimensional data including three-dimensional position information at each of a plurality of points that constitute the first adjacent tooth (for example, adjacent teeth No. 4 to No. 7). The second estimation model 1142 is learned based on the teeth (for example, the 4th and 5th adjacent teeth) and the estimation results of the second adjacent teeth using the three-dimensional data. The type of tooth included in the second adjacent tooth associated with three-dimensional data including three-dimensional position information for each of a plurality of points constituting the tooth, and the second adjacent tooth using the three-dimensional data is learned based on the results of estimating the types of teeth (for example, teeth Nos. 4 and 5) included in .

Note that the learning of the estimation model 114 may be realized by the learning of the estimation model 514 executed by the server device 500 .

The three-dimensional data input from the input unit 1102 includes three-dimensional positional information for each of a plurality of points forming teeth and gums in the oral cavity.

As a result, the identification device 100 inputs three-dimensional data corresponding to a plurality of adjacent teeth and gingiva in the oral cavity to the estimation model 114 (learned model), thereby identifying each type of the plurality of teeth. Therefore, the type of tooth can be identified more accurately and smoothly than when the type of tooth is identified one by one by relying on the user's own knowledge.

The output unit 1103 outputs the result of identification by the second identification unit 1132 using at least one of colors, characters, numbers, and symbols.

As a result, the identification result by the estimation model 114 (learned model) is output using at least one of colors, letters, numbers, and symbols, so that the user 1 intuitively recognizes the identification result. and improve convenience.

The output unit 1103 outputs the result of identification by the identification unit 1101 to the server device 500, and the server device 500 accumulates and stores the result of identification by the identification unit 1101. FIG.

As a result, big data is formed by accumulating and storing identification results by server device 500, so user 1 can cause server device 500 to execute learning processing using such big data, for example. , it is possible to generate a trained model that can more accurately identify the type of tooth.

The tooth information includes at least one of colors, letters, numbers, and symbols associated with the type of tooth corresponding to the three-dimensional data.

As a result, the identification device 100 can generate a trained model that can more accurately identify the type of tooth based on the colors, letters, numbers, symbols, etc. associated with the type of tooth. .

As shown in FIG. 9, the tooth information is associated with the three-dimensional data so that the range of each of the plurality of teeth corresponding to the three-dimensional data can be specified.

As a result, the user 1 can specify the range of each of the plurality of teeth based on the tooth information, which improves convenience during labeling.

As shown in FIG. 9, tooth information is associated with each of a plurality of points forming a tooth corresponding to three-dimensional data.

As a result, the tooth information is associated with each of the plurality of points forming the tooth corresponding to the three-dimensional data, so that the user 1 can associate the tooth with the tooth information in detail, which improves convenience during labeling. improves.

The estimation model 114 is learned based on attribute information about the subject 2 who has teeth, in addition to the tooth information and the identification result by the identification unit 1101 .

As a result, the identification device 100 can make the estimation model 114 learn based on the attribute information about the subject 2 in addition to the learning data, so that the trained model considering the attribute information of the subject 2 can be generated. can be done.

As shown in FIG. 12, the attribute information includes information on at least one of subject's age, sex, race, height, weight, and place of residence.

As a result, the identification device 100 learns the estimation model 114 based on at least one of the subject's age, sex, race, height, weight, and place of residence in addition to the learning data. Therefore, it is possible to generate a trained model considering the profile of the subject 2.

The identification system 10 uses a three-dimensional camera to acquire three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other, and a three-dimensional scanner 200. and an identification device 100 for identifying types of teeth included in adjacent teeth based on the three-dimensional data acquired by the scanner 200. The identification device 100 receives input of the three-dimensional data acquired by the three-dimensional scanner. an identification unit 1101 for identifying types of teeth included in adjacent teeth based on three-dimensional data input from the input unit 1102; and an output unit 1103 for outputting identification results from the identification unit 1101. A portion 1101 is at each of a plurality of points constituting a first adjacent tooth (for example, adjacent teeth including teeth Nos. 3 to 6 and adjacent teeth including teeth Nos. 4 to 7) to which a plurality of teeth are adjacent. A second adjacent tooth (for example , No. 6 and No. 7 adjacent teeth, No. 5 and No. 6 adjacent teeth, No. 4 and No. 5 adjacent teeth, No. 3 and No. 4 adjacent teeth); Based on the identification result by the unit 1131 and the second estimation model 1142 including the second neural network 1152, the teeth included in the second adjacent teeth (for example, teeth 3 to 6, teeth 4 to 7 and a second identification section 1132 for identifying the type of teeth.

In addition, the first identification portion 1131 identifies a plurality of adjacent first adjacent teeth (for example, adjacent teeth including teeth Nos. 3 to 6 and adjacent teeth including teeth Nos. 4 to 7). Based on the three-dimensional data including the three-dimensional positional information at each of the points and the first estimation model 1141 including the first neural network 1151, the same number of teeth as the adjacent teeth are adjacent to the second adjacent tooth It may also identify teeth (eg, adjacent teeth numbered 3-6, adjacent teeth numbered 4-7).

As a result, the identification system 10 uses the identification device 100 having a plurality of multi-stage estimation models 114 (first estimation model 1141, second estimation model 1142, first trained model, second trained model), Since the type of tooth included in the adjacent tooth can be identified based on the three-dimensional data including the three-dimensional positional information of each of a plurality of points that constitute the adjacent tooth, the user can rely on his/her own knowledge of the tooth. It is possible to identify the type of tooth with higher accuracy than to identify the type. Furthermore, since the identification device 100 identifies teeth while gradually reducing the number of adjacent teeth using a plurality of multi-stage estimation models 114, a plurality of teeth correspond to adjacent teeth in three dimensions. The type of tooth can be identified with higher accuracy than the type of each tooth can be identified at once based on the data.

The identification method comprises a step (S41) of inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to a plurality of teeth (S41), and a step of inputting (S41). identifying the types of teeth included in adjacent teeth based on the obtained three-dimensional data (S45 to S49); Steps (S45 to S49) include a plurality of adjacent teeth that constitute first adjacent teeth (for example, adjacent teeth including teeth No. 3 to 6, adjacent teeth including teeth No. 4 to 7). Based on the three-dimensional data including three-dimensional positional information at each of the points and the first estimation model 1141 including the first neural network 1151, a plurality of teeth that are less in number than the first adjacent teeth are adjacent to the first A first identification step of identifying two adjacent teeth (e.g., 6th and 7th adjacent teeth, 5th and 6th adjacent teeth, 4th and 5th adjacent teeth, 3rd and 4th adjacent teeth) ( S45, S47), the identification result of the first identification step (S45, S47), and the second estimation model 1142 including the second neural network 1152, the tooth included in the second adjacent tooth (for example, No. 3 to and a second identification step (S49) for identifying the type of each tooth No. 6 and each tooth No. 4 to No. 7).

In addition, the first identification step (S45, S47) includes a plurality of adjacent first adjacent teeth (for example, adjacent teeth including teeth Nos. 3 to 6, adjacent teeth including teeth Nos. 4 to 7). and a first estimation model 1141 including a first neural network 1151, a plurality of teeth of the same number as the adjacent teeth are adjacent It may also identify the second adjacent tooth (eg, the 3rd to 6th adjacent teeth, the 4th to 7th adjacent teeth).

As a result, the identification method uses a plurality of multistage estimation models 114 (a first estimation model 1141, a second estimation model 1142, a first trained model, and a second trained model). Since the type of tooth included in the adjacent tooth can be identified based on the three-dimensional data including the three-dimensional positional information of each of the points, it is possible to identify the type of tooth rather than relying on the user's own knowledge. , the type of tooth can be identified with high accuracy. Furthermore, the identification method uses a plurality of multistage estimation models 114 to identify teeth while gradually reducing the number of adjacent teeth. The type of tooth can be identified with higher accuracy than when each type of tooth is identified at once based on .

The identification program 120 inputs to the computing device 130 three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other (S41); Based on the three-dimensional data input in the step (S41), a step (S45 to S49) of identifying the type of tooth included in the adjacent tooth, and a step of outputting the identification result of the identifying step (S45 to S49) ( S50), and the identifying step (S45 to S49) is to identify a first adjacent tooth (for example, adjacent teeth including teeth No. 3 to 6, teeth No. 4 to 7) to which a plurality of teeth are adjacent. Based on three-dimensional data including three-dimensional position information at each of a plurality of points that constitute the adjacent teeth) and the first estimation model 1141 including the first neural network 1151, a number less than the first adjacent teeth adjacent teeth (e.g., 6th and 7th adjacent teeth, 5th and 6th adjacent teeth, 4th and 5th adjacent teeth, 3rd and 4th adjacent teeth) Based on the first identification step (S45, S47) for identifying the first identification step (S45, S47), the identification result of the first identification step (S45, S47), and the second estimation model 1142 including the second neural network 1152, the second adjacent tooth includes and a second identification step (S49) of identifying the types of teeth (eg, teeth Nos. 3 to 6, teeth Nos. 4 to 7).

In addition, the first identification step (S45, S47) includes a plurality of adjacent first adjacent teeth (for example, adjacent teeth including teeth Nos. 3 to 6, adjacent teeth including teeth Nos. 4 to 7). and a first estimation model 1141 including a first neural network 1151, a plurality of teeth of the same number as the adjacent teeth are adjacent It may also identify the second adjacent tooth (eg, the 3rd to 6th adjacent teeth, the 4th to 7th adjacent teeth).

As a result, the identification method uses a plurality of multistage estimation models 114 (a first estimation model 1141, a second estimation model 1142, a first trained model, and a second trained model). Since the type of tooth included in the adjacent tooth can be identified based on the three-dimensional data including the three-dimensional positional information of each of the points, it is possible to identify the type of tooth rather than relying on the user's own knowledge. , the type of tooth can be identified with high accuracy. Furthermore, the identification method uses a plurality of multistage estimation models 114 to identify teeth while gradually reducing the number of adjacent teeth. The type of tooth can be identified with higher accuracy than when each type of tooth is identified at once based on .

[Modification]
The present disclosure is not limited to the above embodiments, and various modifications and applications are possible. Modifications applicable to the present invention will be described below.

(Learning processing during service provision processing)
As shown in FIG. 18, the identification device 100 according to the present embodiment does not execute learning processing in the service provision processing, but as shown in FIG. A learning process may be executed in the process. The identification device 100 may perform so-called federated learning. FIG. 19 is a flowchart for explaining an example of service providing processing executed by the identification device 100a according to the modification. 19 are the same as the processes of S41 to S50 shown in FIG. 18, only the processes after S50 will be described in FIG.

As shown in FIG. 14, the identification device 100a outputs the identification result by the processing of S41 to S50, and then executes the learning processing at the time of service provision. Specifically, after S50, the identification device 100a determines whether correct data for error correction has been input (S61a). For example, when the type of tooth, which is the identification result output in S50, is different from the type of tooth that was actually scanned, the identification device 100a allows the user 1 to identify the type of tooth that was actually scanned. It is determined whether or not the input has corrected the error.

If the correct data for error correction is not input (NO in S61a), the identification device 100a ends this process. On the other hand, when the correct data for error correction is input (YES in S61a), the identification device 100a gives a reward based on the identification result and the correct data (S62).

For example, the smaller the degree of dissociation between the identification result and the correct answer data, the smaller the negative points given as a reward, and the greater the degree of dissociation between the two, the larger the negative points given as a reward. Just do it. Specifically, if the tooth output as the identification result and the tooth input as the correct answer data are adjacent to each other, the identification device 100a gives a small negative point, and if the two are separated, a negative point is given. Give a big minus point. In this way, the identification device 100a gives different rewards depending on the degree of dissociation between the identification result and the correct answer data. Note that the reward is not limited to minus points, and may be plus points.

The identification device 100a updates the parameters of the trained model (for example, at least one of the first parameter 1161 and the second parameter 1162) based on the given reward (S63). For example, the identification device 100a updates the parameters of the learned model so that the negative points given as a reward approach zero. After that, the identification device 100a terminates this process.

In this way, the identification device 100a according to the modification executes learning processing even in the service providing processing. Therefore, the more the user 1 uses the identification device 100a, the more the accuracy of the identification processing improves, and the more accurately the type of tooth is identified. be able to. Such processing is a kind of so-called collaborative learning.

(Generation of trained models for each category)
The identification device 100 according to the present embodiment, as shown in FIG. 12, uses a learning data set group including a plurality of learning data sets 116a to 116o classified for each category to learn the estimation model 114. Thus, one trained model is generated, but as shown in FIG. 20, the identification device 100b according to the modification classifies each of the plurality of learning data sets classified for each category into may be used to train the estimation model 114 to generate a trained model for each category. FIG. 20 is a schematic diagram for explaining generation of a trained model based on the learning data set according to the modification.

As shown in FIG. 20, the learning data set 116 is classified and held for each category based on the profile of the subject 2 who was scanned when the learning data set 116 was generated. For example, learning data sets are assigned to six categories based on age (minors, working generation, elderly) and sex (male, female).

The identification device 100b generates trained models 114p-114u for each category by causing the estimation model 114 to learn using each of the plurality of learning data sets 116p-116u classified for each category.

In this way, the identification device 100b according to the modification can generate a plurality of trained models 114p to 114u classified by category. Able to identify different types of teeth.

The generation of the trained models 114p to 114u shown in FIG. 20 can also be applied to the generation of trained models held by the server device 500. FIG. For example, the learning data sets 116p-116u shown in FIG. It may be applied to the retained trained model.

(Service provision processing using trained model for each category)
A service providing process executed by the identification device 100 using the trained models 114p to 114u for each category will be described with reference to FIG. FIG. 21 is a flowchart for explaining an example of service providing processing executed by the identification device 100b according to the modification. Each step shown in FIG. 21 is realized by executing the OS 127 and the identification program 120 by the arithmetic device 130 of the identification device 100b.

As shown in FIG. 21, the identification device 100b determines whether three-dimensional data has been input (S41). For example, the identification device 100b determines whether or not a sufficient amount of three-dimensional data to perform identification processing has been acquired. If the identification device 100b has not acquired a sufficient amount of three-dimensional data (NO in S41), the process ends.

On the other hand, when a sufficient amount of three-dimensional data has been input (YES in S41), the identification device 100b determines whether or not the user 1 has input the profile data of the subject 2 (S42). After that, the identification device 100b selects a learned model corresponding to the profile data from among the group of learned models shown in FIG. 20 (S61b). For example, if the subject 2 is an elderly female, the identification device 100b selects the learned model 114u.

After that, identification device 100b inputs the three-dimensional data to the first stage trained model (for example, trained estimation model 1140a) (S62b). The identification device 100 performs identification processing using a first-stage trained model based on the features of the plurality of teeth corresponding to the input three-dimensional data, among the plurality of teeth corresponding to the input three-dimensional data. , adjacent teeth including some teeth are identified (S45). After that, the processes shown in S46 to S50 are the same as the processes shown in S46 to S50 shown in FIG.

In this way, the identification device 100b according to the modification can perform identification processing using a trained model that is most suitable for the profile of the subject 2, so that more detailed analysis according to the profile of the subject 2 , the type of tooth can be identified with higher accuracy.

(Profile output)
The identification device 100 according to the present embodiment identifies the type of tooth through identification processing. However, as shown in FIGS. 12 and 20, considering that the trained model is generated based on the learning data set that considers the profile of the subject 2, three-dimensional data is input to the trained model in the identification process. By doing so, a profile of the owner of the tooth may be output as an identification result based on the characteristics of the tooth corresponding to the three-dimensional data. In this way, the profile of the unidentified subject 2 found in a disaster or incident can be specified from the three-dimensional data including teeth.

(learning process)
Identification device 100 according to the present embodiment updates the parameters (first parameter 1161 and second parameter 1162) of estimation model 114 by learning processing. The neural network (first neural network 1151, second neural network 1152) may be updated (for example, the algorithm of the neural network may be updated). Server device 500 according to the present embodiment updates the parameters of estimation model 514 by learning processing, but is not limited to updating the parameters, and the neural network is updated by learning processing (for example, , the algorithm of the neural network is updated).

(identification using normal and/or color information)
FIG. 22 is a schematic diagram for explaining an example of a learning data set according to a modification. The identification device 100c according to the modification learns the estimation model 114 by inputting the actual color information of the tooth into the estimation model 114 in addition to the position information included in the three-dimensional data acquired by the three-dimensional scanner 200. You may

For example, as described with reference to FIG. 9, the learning data set includes position information, which is input data for the estimation model 114, and color information after color coding associated with tooth types, which is correct data. In addition to this, as shown in FIG. 22, tooth color information before color-coding may also be included. In the learning process, in addition to the positional information, the color information of the teeth before color-coding is input to the estimation model 114, so that a learned model is generated in consideration of the actual color information of the teeth. good too.

Furthermore, in the learning data set, in addition to position information, which is input data for the estimation model 114, and color information after color coding associated with tooth types, which is correct data, as shown in FIG. Normal information that can be calculated based on the position information may also be included. Then, in the learning process, by inputting the normal information in addition to the position information to the estimation model 114, the learned model may be generated in consideration of the normal information.

Normal information can be calculated, for example, as follows. For example, one of a plurality of points forming a tooth is focused on, and a normal to the point of interest is generated based on a plurality of points belonging to a predetermined range near the point of interest. Specifically, the normal to the point of interest can be generated using principal component analysis for a plurality of points belonging to a predetermined range near the point of interest. Principal component analysis can generally be performed by computing a variance-covariance matrix. Eigenvectors are calculated in the calculation of the variance-covariance matrix, and the principal component direction is generated as the normal to the point of interest. Since the method of generating a normal to a point in the point group is well known, other generally known techniques may be used.

In this way, by adding normal line information to the learning data set, the identification device can also learn which side of the tooth that each point constitutes is the surface for each of the plurality of points that constitute the tooth. . In addition, the identification device can learn features of a shape such as a dent based only on a small number of point groups belonging to a predetermined range near the point of interest.

The learning data set may contain both color information and normal line information of the tooth before color-coding, or may contain only one of them.

Next, the service providing process for executing the identification process using the trained model trained based on the learning data set containing the tooth color information and normal information before color-coding will be described with reference to FIG. 18 . FIG. 23 is a flowchart for explaining an example of service providing processing executed by the identification device 100c according to the modification.

As shown in FIG. 23, the identification device 100c according to the modification additionally executes the processing of S61c, unlike the service provision processing executed by the identification device 100 shown in FIG. That is, after the three-dimensional data is input (YES in S41), the identification device 100c generates normal lines at a plurality of points forming the tooth based on the position information included in the input three-dimensional data (S61c). ). In addition to the position information, the input three-dimensional data includes color information of the teeth before color-coding.

After that, if the identification device 100c determines that the profile data has not been input (NO in S42), it inputs the normal information in addition to the three-dimensional data (position information and color information) to the learned model in the first stage. (S62c). On the other hand, if the identification device 100c determines that the profile data has been input (YES in S42), the identification device 100c inputs the normal information in addition to the three-dimensional data (position information, color information) and profile data to the learned model (S63c). ). After S62c and S63c, the identification device 100c executes identification processing for identifying adjacent teeth using the learned model of the first stage (S45).

In this way, the identification device 100c according to the modified example further determines the type of tooth based on the color information of the tooth before color-coding and the normal lines generated for each of the plurality of points forming the tooth corresponding to the three-dimensional data. may be identified. Note that the identification device 100c may be input with color information of teeth before color separation, but without input of normal line information. Alternatively, the identification device 100c may receive normal line information but not tooth color information before color-coding.

In this way, the identification device 100c can identify the type of tooth based on the color information of the tooth before color-coding and/or the normal lines generated for each of the plurality of points. can be identified.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope of equivalents of the scope of claims. Note that the configurations exemplified in this embodiment and the configurations exemplified in the modifications can be combined as appropriate.

1 user, 2 subject, 5 network, 10 identification system, 100, 100a, 100b, 100c identification device, 102 scanner interface, 103, 503 display interface, 104 speaker interface, 105, 505 peripheral device interface, 106, 506 network controller , 107,507 media reader, 108 PC display, 109,509 memory, 110,510 storage, 112,512 scan information, 114,514,1140a, 1140b, 1140c estimation model, 116,516 training data set, 118, 518 color classification data 119,519 profile data 120 identification program 121,521 learning program 122,522 three-dimensional data 124,524 identification result 127,527 OS 130,530 arithmetic unit 200 three-dimensional Scanner, 300, 350 Display, 400 Speaker, 500 Server Device, 550 Removable Disk, 601, 651 Keyboard, 602, 652 Mouse, 1101, 1130a, 1130b, 1130c Identification Unit, 1102 Input Unit, 1103 Output Unit, 1119 Profile Acquisition Unit , 1131 first identification unit, 1132 second identification unit, 1141 first estimation model, 1142 second estimation model, 1144 parameters, 1151 first neural network, 1152 second neural network, 1161 first parameters, 1162 second parameters.

Claims (9)

An identification device for identifying the type of tooth,
an input unit for inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth where a plurality of teeth are adjacent;
an identification unit that identifies types of teeth included in the adjacent teeth based on the three-dimensional data input from the input unit;
An output unit that outputs the identification result by the identification unit,
The identification unit
Based on three-dimensional data including three-dimensional position information at each of a plurality of points constituting a first adjacent tooth adjacent to a plurality of teeth and a first estimation model including a first neural network, the first adjacent tooth A first identification unit that identifies a second adjacent tooth that is adjacent to a plurality of teeth that are the same number as or less than the first adjacent tooth;
An identification device, comprising a second identification unit that identifies the type of tooth included in the second adjacent tooth based on the identification result of the first identification unit and a second estimation model including a second neural network. The first identifying unit, based on three-dimensional data including three-dimensional position information of each of a plurality of points constituting the first adjacent tooth and the first estimation model, selects a plurality of points as the second adjacent tooth. identifying each of the second adjacent teeth;
the plurality of second adjacent teeth includes teeth that overlap each other;
The second identification unit identifies the type of the overlapping tooth included in each of the plurality of second adjacent teeth based on the identification result of the first identification unit and the second estimation model. 1. The identification device according to 1. 3. The identification unit identifies types of teeth included in the adjacent teeth based on normal lines generated for each of a plurality of points corresponding to the three-dimensional data input from the input unit. Item 3. The identification device according to item 2. The first estimation model uses the second adjacent tooth associated with three-dimensional data including three-dimensional positional information of each of the plurality of points constituting the first adjacent tooth, and the three-dimensional data. It is learned based on the estimation result of the second adjacent tooth,
The second estimation model includes a type of tooth included in the second adjacent tooth associated with three-dimensional data including three-dimensional position information at each of a plurality of points constituting the second adjacent tooth; 4. The identification device according to any one of claims 1 to 3, wherein the identification device is learned based on the result of estimating the type of tooth included in the second adjacent tooth using three-dimensional data. The three-dimensional data input from the input unit according to any one of claims 1 to 4, wherein the three-dimensional data includes three-dimensional position information at each of a plurality of points that constitute teeth and gums in the oral cavity. identification device. The identification according to any one of claims 1 to 5, wherein the output unit outputs identification results by the second identification unit using at least one of colors, letters, numbers, and symbols. Device. An identification system for identifying tooth types, comprising:
A three-dimensional scanner that uses a three-dimensional camera to acquire three-dimensional data including three-dimensional position information at each of a plurality of points that constitute adjacent teeth adjacent to each other;
an identification device that identifies the type of tooth included in the adjacent tooth based on the three-dimensional data acquired by the three-dimensional scanner;
The identification device
an input unit into which three-dimensional data acquired by the three-dimensional scanner is input;
an identification unit that identifies types of teeth included in the adjacent teeth based on the three-dimensional data input from the input unit;
an output unit that outputs a result of identification by the identification unit;
The identification unit
Based on three-dimensional data including three-dimensional position information at each of a plurality of points constituting a first adjacent tooth adjacent to a plurality of teeth and a first estimation model including a first neural network, the first adjacent tooth A first identification unit that identifies a second adjacent tooth that is adjacent to a plurality of teeth that are the same number as or less than the first adjacent tooth;
An identification system, comprising a second identification unit that identifies the type of tooth included in the second adjacent tooth based on the identification result of the first identification unit and a second estimation model including a second neural network. A computer-based identification method for identifying tooth types, comprising:
The identification method includes, as processing executed by the computer,
a step of inputting three-dimensional data including three-dimensional positional information at each of a plurality of points forming adjacent teeth adjacent to each other;
identifying the types of teeth included in the adjacent teeth based on the three-dimensional data input in the inputting step;
and outputting an identification result from the identifying step,
The identifying step includes:
Based on three-dimensional data including three-dimensional position information at each of a plurality of points constituting a first adjacent tooth adjacent to a plurality of teeth and a first estimation model including a first neural network, the first adjacent tooth a first identification step of identifying a second adjacent tooth adjacent to a plurality of teeth equal in number to or less than the first adjacent tooth;
An identification method, comprising a second identification step of identifying the type of tooth included in the second adjacent tooth based on the identification result of the first identification step and a second estimation model including a second neural network. An identification program for identifying tooth types,
The identification program, in a computer,
a step of inputting three-dimensional data including three-dimensional position information at each of a plurality of points forming adjacent teeth adjacent to each other;
a step of identifying the types of teeth included in the adjacent teeth based on the three-dimensional data input in the step of inputting;
a step of outputting an identification result obtained by the identifying step;
The identifying step includes:
Based on three-dimensional data including three-dimensional position information at each of a plurality of points constituting a first adjacent tooth adjacent to a plurality of teeth and a first estimation model including a first neural network, the first adjacent tooth a first identification step of identifying a second adjacent tooth adjacent to a plurality of teeth equal in number to or less than the first adjacent tooth;
A program for identification, comprising a second identification step of identifying the type of tooth included in the second adjacent tooth based on the identification result of the first identification step and a second estimation model including a second neural network.

Images