WO2021149530A1

WO2021149530A1 - Information processing system, information processing method, and program

Info

Publication number: WO2021149530A1
Application number: PCT/JP2021/000625
Authority: WO
Inventors: 恭平新田; 佐藤　大輔; エンリコリナルディ
Original assignee: Ａｒｉｔｈｍｅｒ株式会社
Priority date: 2020-01-21
Filing date: 2021-01-12
Publication date: 2021-07-29
Also published as: JP7390669B2; JPWO2021149530A1

Abstract

[Problem] Conventional information processing systems have had a problem in that producing a dental restoration is very difficult, and takes time and effort. [Solution] An information processing system 100 comprises: a learning device storage unit 111 storing a learning device for acquiring restoration information for representing the shape of a dental restoration corresponding to certain teeth, obtained by using dentition shape information including the shapes of two or more teeth that are in mutual proximity; a first acquisition unit 151 for acquiring subject shape information including the shapes of one or more proximity teeth that are in proximity to a tooth of interest, and subject identification information for identifying a tooth the shape of which is included in the subject shape information on the basis of the subject shape information; and a second acquisition unit 152 for acquiring restoration information for representing the shape of a dental restoration corresponding to the tooth of interest from the subject identification information and the subject shape information by using the learning device. The information processing system makes it easy to acquire restoration information.

Description

Information processing system, information processing method, and program

The present invention relates to an information processing system, an information processing method, and a program that output information for producing a dental restoration used for one or more teeth to be restored.

Conventionally, dental restorations such as insert teeth and crowns used for patients in the dental field are manually created by dental technicians for each patient. In recent years, CAD / CAM technology has also come to be used in the dental field. That is, dental restorations are produced by a 3D printer, a milling machine, or the like using data modeled by a dental technician using CAD (see, for example, Patent Document 1).

It should be noted that a technique for segmenting the point cloud obtained as the above-mentioned tooth scan data for each tooth has been published (see, for example, Non-Patent Document 1).

Japanese Unexamined Patent Publication No. 2000-107203

By the way, the work of producing a dental restoration as in the past has a problem that it is difficult and time-consuming. That is, it is necessary to adjust the shape of each dental restoration according to the patient, whether it is produced manually by a worker such as a dental technician or modeling is performed using CAD. Therefore, the worker is required to have skillful skills and knowledge, and the work performed by the worker is also troublesome.

The information processing system of the first invention is an information processing system that outputs information for producing a dental restoration used for one or more attention teeth to be restored, and is a shape of two or more teeth adjacent to each other. Learning that stores a learning device for acquiring restoration information for representing the shape of a dental restoration corresponding to a part of two or more teeth obtained by using the dentition shape information including Acquires target shape information including the container storage unit and the shape of one or more adjacent teeth close to the tooth of interest, and target identification information for identifying the tooth whose shape is included in the target shape information. The first acquisition unit and the learning device are used to acquire the restoration information for representing the shape of the dental restoration corresponding to the tooth of interest from the target identification information and the target shape information acquired by the first acquisition unit. It is an information processing system equipped with an acquisition unit.

With such a configuration, it is possible to easily obtain restoration information for representing the shape of the dental restoration.

Further, the information processing system of the second invention includes, for the first invention, a reference shape information storage unit for storing reference shape information indicating a reference shape prepared in advance corresponding to the tooth of interest. The restoration information is information corresponding to the difference in shape between the reference shape and the shape of the dental restoration corresponding to the tooth of interest, and the second acquisition unit uses the restoration information and the reference shape information to perform dental restoration. It is an information processing system that acquires 3D data representing the shape of an object.

With such a configuration, it is possible to acquire more accurate 3D data representing the shape of the dental restoration.

Further, in the information processing system of the third invention, the learning device is information obtained by machine learning performed by using two or more learning target information with respect to the first or second invention, and is a learning target. The information includes dentition shape information, dentition identification information that identifies each of the two or more teeth included in the dentition shape information, and the shape of the dental restoration applied to some of the two or more teeth. It is an information processing system that includes restoration information for representing.

With such a configuration, it is possible to acquire highly accurate restoration information by machine learning.

Further, in the information processing system of the fourth invention, in contrast to the third invention, the learning device storage unit is provided with restoration information for expressing the shape of the dental restoration corresponding to one or more attention teeth. A learning device obtained by machine learning using two or more learning target information including dentition identification information and dentition shape information including two or more tooth shapes close to the attention tooth is stored for each attention tooth. The second acquisition unit uses a learning device corresponding to the target identification information acquired by the first acquisition unit, and uses a machine learning method from the target identification information and the target shape information acquired by the first acquisition unit. It is an information processing system that acquires restoration information.

With such a configuration, it is possible to acquire more accurate restoration information.

Further, in the information processing system of the fifth invention, with respect to the third or fourth invention, the second acquisition unit is a multidimensional vector based on the target identification information and the target shape information acquired by the first acquisition unit. By generating the first vector information representing the above and performing dimensional reduction (dimensional compression) on the generated first vector information, the second vector representing a feature vector having a lower dimension than the vector represented by the first vector information. It is an information processing system that generates information and acquires restoration information using the generated second vector information.

With such a configuration, restoration information can be acquired at a higher speed.

Further, in the information processing system of the sixth invention, for any of the third to fifth inventions, the learning unit evaluates the shape including the evaluation of the interference between the dental restoration and the surrounding teeth. The parameters of the cost function, including the penalty term, have been adjusted based on.

With such a configuration, it is possible to set rules regarding interference between the dental restoration and the surrounding teeth.

Further, in the information processing system of the seventh invention, when the second acquisition unit acquires 3D data with respect to the sixth invention, the learning unit receives the target shape information acquired by the first acquisition unit. Evaluation is performed based on the restoration information or 3D data acquired by the second acquisition unit, and the parameter of the penalty term is changed based on the evaluation result.

With such a configuration, it is possible to acquire restoration information according to the rules regarding interference between the dental restoration and the surrounding teeth.

Further, in the information processing system of the eighth invention, for any one of the first to seventh inventions, the tooth row shape information is the tooth shape corresponding to the dental restoration and all the teeth adjacent to the tooth. Information representing at least a part of the shape in the oral cavity including the shape of the tooth, and the target shape information is the shape of at least a part of the oral cavity including the shape of the tooth of interest and the shape of a adjacent tooth close to the tooth of interest. It is an information processing system that is information representing.

Further, in the information processing system of the ninth invention, with respect to any of the first to eighth inventions, the target shape information is information indicating a point group representing the shape of at least a part of the oral cavity, and the first The acquisition unit identifies the area including the point group corresponding to each tooth among the point groups indicated by the target shape information, and obtains the target identification information in which the tooth identifier that identifies the tooth is associated with the specified area for each tooth. It is an information processing system to acquire.

With such a configuration, it is possible to acquire more accurate restoration information by using the target identification information that identifies the shape of each tooth.

Further, in the information processing system of the tenth invention, with respect to the ninth invention, the first acquisition unit sets a point group including a point group corresponding to each tooth among the point groups indicated by the target shape information. Estimated based on the relationship between each point included in the group and the points around it, the point group included in the estimated area and the point group not included in the area are displayed on the display in different display modes and the user. This is an information processing system that acquires information on the display mode input by the user and identifies an area including a point group corresponding to each tooth based on the information input by the user.

With such a configuration, the user can easily perform an operation of specifying the shape of each tooth.

Further, in the information processing system of the eleventh invention, with respect to the tenth invention, the first acquisition unit displays a group of points included in each specified area on a display and points included in each area. It is an information processing system that acquires labeling information input by a user for a group and acquires target identification information based on the information input by the user.

With such a configuration, the user can easily perform an operation of associating a tooth identifier that identifies the tooth with a specified area for each tooth.

Further, the information processing system of the twelfth invention is an information processing system that outputs information for producing a dental restoration used for one or more attention teeth to be restored, and is an arbitrarily selected selection. A learner that stores a learner adjusted by a plurality of vector information obtained from information indicating the shape of each of a plurality of teeth adjacent to the tooth and output information obtained from information indicating the shape of the selected tooth. Using the storage unit and the learner, the output information corresponding to the attention tooth is acquired based on the plurality of vector information obtained from the information indicating the shapes of the plurality of teeth adjacent to the attention tooth, and the acquired output is obtained. It is an information processing system including a restoration information acquisition unit that acquires restoration information for expressing the shape of a dental restoration corresponding to a tooth of interest from information.

According to the information processing system according to the present invention, restoration information can be easily acquired.

The figure which shows the outline of the dental restoration production system which concerns on Embodiment 1. Diagram showing the configuration of the information processing system Flow chart showing an example of the operation of the information processing system Flow chart showing an example of learning device acquisition and storage processing performed by the information processing system Flow chart showing an example of identification information acquisition processing performed by the information processing system Flow chart showing an example of output information acquisition processing performed by the information processing system The figure explaining the specific example of the 3D data which shows the shape of the dental restoration acquired by the 2nd acquisition part. The figure which shows an example of the operation performed by the 1st acquisition part which concerns on Embodiment 2. A diagram illustrating a specific example of an annotation tool that can be used by a user in the information processing system. Overview of the computer system according to the above embodiment Block diagram of the computer system

Hereinafter, embodiments of the information processing system and the like will be described with reference to the drawings. In addition, since the components with the same reference numerals perform the same operation in the embodiment, the description may be omitted again.

The terms used below are generally defined as follows. It should be noted that the meanings of these terms should not always be interpreted as shown here, and should be interpreted based on the explanations, for example, when they are explained individually below.

A dental restore is a repair or prosthesis to be placed in the oral cavity in dental treatment, for example, a crown (including an insert tooth and an artificial tooth attached to a dental implant), an inlay, a bridge, and the like. Is. It may be interpreted that other prostheses such as dentures are included.

A tooth of interest is a tooth that is the target of treatment using a dental restoration. The tooth of interest may be a completely missing tooth in the dentition, whether or not it is an existing tooth in the patient being treated. Further, the tooth of interest may be an artificially shaped tooth or a part of the tooth. The tooth of interest can be arbitrarily selected according to the purpose of using the information processing system, etc., but is not limited to this.

The 3D data is information representing a three-dimensional shape, and is composed of information representing, for example, a point cloud (which may be meshed), a line, a surface, a voxel, and the like. The 3D data may be information in a format used in a specific CAD, or information in a general-purpose intermediate file format that can be used in various CADs.

For a certain item, the identifier is a character or code that uniquely indicates the item. The code is, for example, alphanumeric characters or other symbols, but is not limited to this. The identifier is, for example, a code string that does not have a specific meaning by itself, but any kind of information can be used as long as it can identify the corresponding item. That is, the identifier may be the name of what it indicates, or it may be a combination of codes so as to uniquely correspond to each other.

The tooth identifier uniquely identifies the patient's tooth, for example. For example, a number represented by the so-called universal system, which is represented by the dental notation, can be used as the tooth identifier, but the present invention is not limited to this.

The acquisition may include acquiring the matters input by the user or the like, or the information stored in the own device or another device (the information may be stored in advance, or the device concerned). It may include acquiring information (which may be information generated by information processing performed in). Acquiring the information stored in the other device may include acquiring the information stored in the other device via API or the like, or the document file provided by the other device. It may include acquiring the content (including the content of the web page).

To output information means to display on a display, project using a projector, print with a printer, output sound, transmit to an external device, store in a recording medium, process to another processing device or other program. It is a concept that includes delivery of results. Specifically, for example, it includes enabling information to be displayed on a web page, transmitting it as an e-mail or the like, and outputting information for printing.

Information reception means receiving information input from input devices such as keyboards, mice, and touch panels, receiving information transmitted from other devices via wired or wireless communication lines, optical disks, magnetic disks, and semiconductors. It is a concept including acceptance of information read from a recording medium such as a memory.

Regarding various types of information stored in information processing systems, etc., updating means changing the stored information, adding new information to the stored information, and updating the stored information. It is a concept that includes the fact that part or all of it is erased.

(Embodiment 1)

Hereinafter, the dental restoration production system including the information processing system according to the first embodiment will be described.

FIG. 1 is a diagram showing an outline of the dental restoration production system 900 according to the first embodiment.

As shown in FIG. 1, the dental restoration production system 900 includes an information processing system 100, a dental scanning system 910, and a modeling device 920. The dental restoration production system 900 is used to produce a dental restoration used for one or more notable teeth to be restored.

The dental scan system 910 includes, for example, a terminal device, a dental scanner connected to the terminal device, and the like. The dental scan system 910 uses a dental scanner to generate 3D data representing the shape of the patient's oral cavity. The dental scanner may be an intraoral scanner or a scanner that reads a mold taken from the oral cavity. The dental scan system 910 transmits the generated 3D data to the information processing system 100. In the present embodiment, the 3D data transmitted to the information processing system 100 is data indicating a point cloud, but is not limited to this.

The information processing system 100 acquires 3D data transmitted from the dental scan system 910 and performs a process as described later. In this embodiment, the information processing system 100 generates 3D data showing the shape of the dental restoration. The information processing system 100 shows the shape of the dental restoration reflecting the content of the editing operation based on the editing operation of the user (for example, a worker who produces the dental restoration such as a dental technician). 3D data may be generated. The information processing system 100 outputs 3D data indicating the shape of the dental restoration to the modeling apparatus 920.

The modeling device 920 is a device that models a dental repair object having a three-dimensional shape using 3D data. The modeling apparatus 920 is, for example, a known dental 3D printer or milling machine, but is not limited thereto. The dental restoration is modeled based on the 3D data indicating the shape of the dental restoration output by the information processing system 100. As a result, the user can obtain the dental restoration modeled by the modeling device 920 by utilizing the dental restoration product production system 900.

In the present embodiment, the devices included in the dental restoration production system 900 can communicate with each other via a network such as the Internet or a LAN, but the present invention is not limited to this. For example, one device may be directly connected to another device by a wired or wireless communication path. In addition, the dental restoration production system 900 may include each of the above-mentioned devices and other devices.

Further, in the dental restoration production system 900, the editing operation of the 3D data generated by the information processing system 100 may be performed by the user using another terminal device different from the information processing system 100. .. Further, even if the 3D data indicating the shape of the dental restoration reflecting the content of the editing operation and the output of the 3D data to the modeling device 920 are performed by another terminal device different from the information processing system 100. good.

The electronic computers used in the dental scan system 910 and the information processing system 100 include personal computers and server devices, as well as portable information terminal devices such as so-called smartphones and tablet-type information terminal devices. Equipment can be used. In the following examples, it is assumed that a so-called personal computer having a keyboard, a display, or the like (not shown) is used as the electronic computer used in the information processing system 100, but the description is not limited to this. The dental scan system 910 and the information processing system 100 may be configured by one device, may be configured by a plurality of devices that operate in cooperation with each other, or may be built in other devices. It may be an electronic computer or the like. The server may be a so-called cloud server, an ASP server, or the like, and the type thereof does not matter.

FIG. 2 is a diagram showing the configuration of the information processing system 100.

As shown in FIG. 2, the information processing system 100 includes a storage unit 110, a reception unit 120, a reception unit 130, a processing unit 140, an output unit 160, and a transmission unit 170.

The storage unit 110 includes a learning device storage unit 111, a reference shape information storage unit 112, and a learning target information storage unit 113.

The storage unit 110 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. For example, information acquired by the receiving unit 120 and the processing unit 140 is stored in each unit of the storage unit 110, but the process of storing the information or the like in each unit of the storage unit 110 is not limited to this. For example, information or the like may be stored in the storage unit 110 via a recording medium, or information or the like transmitted via a communication line or the like may be stored in the storage unit 110. Alternatively, the information or the like input via the input device may be stored in the storage unit 110.

The learning device is stored in the learning device storage unit 111. In the present embodiment, the learning device is obtained by machine learning of the learning unit 141 as described later, for example. In this embodiment, the learner may be referred to as a classifier or a trained model. The learner is used to obtain restoration information to represent the shape of the dental restoration. Details of the learner and its use will be described later.

In the present embodiment, the learning device is stored in association with a tooth identifier that identifies one or more teeth of interest. In other words, the learning device is stored in association with the attention tooth for each specific one or more attention teeth. When producing a dental restoration for one or more teeth of interest to be restored, a learning device corresponding to the tooth of interest is used based on the tooth identifier. Since a learning device is prepared for each tooth of interest, it is possible to acquire restoration information for expressing the shape of the dental restoration with high accuracy by using the learning device corresponding to the tooth of interest. The learner does not have to be associated with the tooth identifier. For example, the learner may be one that can be used to obtain restoration information for any of two or more teeth of interest, regardless of which tooth of interest. For example, the learning device may be information prepared for each tooth region (maxillary side, mandibular side, left and right, etc.).

The reference shape information storage unit 112 stores reference shape information indicating a reference shape prepared in advance for each tooth. The reference shape information of each tooth is stored, for example, in association with a tooth identifier that identifies the corresponding tooth. The reference shape information is, for example, 3D data of the reference shape, but may be a parameter used to generate 3D data of the reference shape according to a predetermined processing method. The reference shape is a shape that can be used as a template when repairing the tooth of interest. For each tooth, a plurality of reference shape information may be prepared according to an index (for example, height, weight, etc.) representing the gender, age, and physique of the patient.

The learning target information storage unit 113 stores two or more learning target information used for machine learning performed by the learning unit 141, which will be described later. One learning target information includes, for example, restoration information for representing the shape of a dental restoration applied to one or more teeth of interest, dentition shape information including the shapes of two or more teeth adjacent to each other, and teeth. Includes dentition identification information that identifies each of the two or more teeth included in the row shape information.

The dentition shape information is information representing at least a part of the shape in the oral cavity including the shape of the tooth corresponding to the dental restoration and the shape of all the teeth adjacent to the tooth in the present embodiment. In the present embodiment, the dentition shape information is, for example, data having the same contents as the 3D data transmitted from the dental scan system 910. It does not matter whether the dentition shape information is the 3D data itself transmitted from the dental scanning system 910. The dentition shape information does not include the shape of the tooth corresponding to the dental restoration, but may include the shape of the tooth close to the tooth corresponding to the dental restoration. The "proximity tooth" means an arbitrary tooth in contact with one tooth, and includes an occlusal tooth and an adjacent tooth.

The dentition identification information is information for identifying a tooth whose shape is included in the dentition shape information with respect to the dentition shape information. In the present embodiment, the dentition identification information is, for example, information for specifying a portion of the dentition shape information corresponding to the tooth (for example, for each of the teeth whose shape is included in the dentition shape information). , Information that specifies the range indicating the shape of the tooth) and the tooth identifier that identifies the tooth include information associated with each other. That is, in one learning target information, based on the dentition shape information and the dentition identification information, each tooth corresponding to the dentition shape information is associated with a tooth identifier to indicate the shape of the tooth. The data can be identified.

The format of each data of the dentition shape information and the dentition identification information does not matter. Both may be separate information, and by associating a tooth identifier with each point of the point group included in the dentition shape information, the point indicating the tooth in the point group can be associated with each tooth. It may be configured so that the range can be specified. For example, the dentition shape information is information including coordinate information of each point in a state where individual points can be identified, and the dentition identification information is information in which individual points are associated with tooth identifiers. good. In this case, the information recorded as a pair of the coordinates of each point and the tooth identifier (for example, point group data in which which tooth is labeled) is the dentition shape information. It is also possible to grasp it as information that is also dentition identification information. Here, the state in which the points can be identified may be, for example, a state in which the points can be identified by including an identifier for identifying the points as information on the individual points, or in the dentition shape information. The points may be identifiable based on the order in which the information of the individual points exists. Further, for example, the dentition shape information is information including coordinate information of a plurality of points, and the dentition identification information is information for specifying a space in which a point group indicating each tooth exists and a tooth identifier for identifying the tooth. It may be information associated with.

Restoration information is information that represents the shape of the dental restoration. In the present embodiment, the restoration information is, for example, information corresponding to the difference in shape between the reference shape and the shape of the dental restoration corresponding to one tooth. In other words, in the present embodiment, the restoration information is a parameter group indicating the amount of deformation from the reference shape to the shape of the dental restoration for one tooth. That is, based on the restoration information and the reference shape information of one tooth, it is possible to generate 3D data showing the shape of the dental restoration when the tooth is the tooth of interest.

Note that the restoration information may include only information related to the shape of a part of the dental restoration. For example, the restoration information includes only information related to the shape of the portion of the attention tooth to be restored that faces another tooth (for example, an occlusal tooth to be meshed up and down or an adjacent adjacent tooth). It may be a thing. In that case, the shape of the other part of the dental restoration can be, for example, a shape based on the shape of the corresponding part of the reference shape. Further, the restoration information may be 3D data showing the shape of the dental restoration as it is.

The learning target information is prepared in advance based on, for example, a case where treatment has been performed in the past, and is stored in the learning target information storage unit 113. In the present embodiment, the restoration information included in the learning target information is used, for example, for modeling the reference shape information stored in the reference shape information storage unit 112 and the dental restoration used for treatment. It can be obtained based on 3D data.

It should be noted that the restoration information used when the dental restoration is produced in the dental restoration production system 900 (the information may reflect the result of the editing operation by the user, or the second acquisition unit 152 described later). (It may be the information acquired by) is associated with the dentition shape information and the dentition identification information used as inputs at that time, and may be stored in the learning target information storage unit 113. .. Such accumulation processing may be performed by, for example, the processing unit 140 or the like.

The receiving unit 120 receives the information transmitted from another device. The receiving unit 120 stores the received information in, for example, the storage unit 110. The receiving unit 120 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

The reception unit 130 receives various input operations to the information processing system 100 performed by the user. The reception unit 130 uses, for example, information input using an input means (not shown) connected to the information processing system 100 or a reading device (for example, a code reader) (for example, a code reader) connected to the information processing system 100 (not shown). Accepts the information input by the input operation performed (including, for example, the information read by the device). The reception unit 130 may receive information related to an input operation or the like transmitted via another device connected via a network or the like. The received information is stored in, for example, the storage unit 110.

The input means that can be used for inputting information that can be accepted by the reception unit 130 may be any input means such as a numeric keypad, a keyboard, a mouse, or a menu screen. The reception unit 130 can be realized by a device driver for input means such as a numeric keypad or a keyboard, control software for a menu screen, or the like.

The processing unit 140 includes a learning unit 141, a first acquisition unit 151, and a second acquisition unit 152. The processing unit 140 performs various processes such as, for example, the processes performed by each unit of the processing unit 140 as follows. The processing unit 140 can usually be realized from an MPU (including a CPU and / or a GPU), a memory, or the like. The processing procedure of the processing unit 140 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The learning unit 141 acquires two or more learning target information, and generates and acquires a learning device by performing machine learning using the acquired learning target information. In other words, in the present embodiment, the learning device is obtained by machine learning performed by using two or more learning target information. The learner inputs input information based on information indicating the shapes of two or more teeth that are close to each other, such as dentition shape information, and information that identifies each of the two or more teeth, such as dentition identification information, and restores the material. It is a learning device that outputs output information corresponding to object information. That is, the learner acquires restoration information for expressing the shape of the dental restoration applied to some of the two or more teeth by using the information including the shapes of two or more teeth that are close to each other. It can be said that it is information for doing.

In the present embodiment, the learning unit 141 generates (learns) a learning device by using a machine learning method, for example, as follows. That is, the learning unit 141 generates input information based on the dentition shape information and the dentition identification information for each of the two or more learning target information. Then, the information of the combination of the input information and the output information obtained from each of the two or more learning target information is given to the module for configuring the learning device of machine learning to generate and acquire the learning device. The learning unit 141 stores the acquired learning device in the learning device storage unit 111. A machine learning method that can be applied to a regression problem that outputs numerical data from some numerical data can be used, for example, deep learning (for example, Deep Feed Forward Neural Networks, etc.), random forest, polypoly regression, etc. SGD regression, LASSO regression, Ridge regression and the like can be applied. For machine learning, functions in various known machine learning frameworks and various existing libraries can be used.

In the present embodiment, machine learning is performed using input information with dimension reduction and output information based on restoration information. For dimensionality reduction, for example, a known principal component analysis (PCA) method can be used, but is not limited thereto. In the present embodiment, the learning unit 141 multidimensionally characterizes the features of two or more teeth included in the dentition shape information, for example, based on the dentition identification information and the dentition shape information included in each learning target information. Generates first vector information represented as a vector of. Then, by reducing the dimension of the generated first vector information, the second vector information representing the feature vector having a lower dimension than the vector represented by the first vector information is generated. Further, the learning unit 141 generates the third vector information from the restoration information by performing the dimension reduction in the same manner as the second vector is generated. The learning unit 141 generates a learning device by using two or more pieces of information on the combination of the second vector information and the third vector information as the information of the combination of the input information and the output information. Since the learning device is generated using the information obtained by reducing the dimensions in this way, the amount of calculation required for performing machine learning can be reduced. In addition, the amount of calculation required for processing performed using the learner can be reduced. Before performing the dimension reduction, mesh registration described later is performed.

Note that the combination of the first vector information and the information representing the feature vector indicating the restoration information may be used as the information of the combination of the input information and the output information.

Here, in the present embodiment, the learning unit 141 optimizes by adjusting the parameters of the cost function including the penalty term so that the evaluation regarding the shape including the evaluation of interference is the highest in the machine learning. .. For example, the learning unit 141 evaluates the interference between the dental restoration and the surrounding teeth based on the learning target information used for learning, and changes the parameters of the penalty term (function and / or value) based on the evaluation result. By doing so, optimization is performed. The parameter change of the penalty term based on the evaluation result can be set to correspond to the rule set regarding the interference between the dental restoration and the surrounding teeth. There are various rules regarding interference with surrounding teeth, such as those related to the symmetry of the left and right teeth (such as making them symmetrical on the left and right) and those that make the upper and lower teeth mesh with each other. Things can be adopted. By changing the parameter of the penalty term according to the evaluation result by a method corresponding to the predetermined rule, it is possible to generate a learning device for obtaining restoration information reflecting the predetermined rule.

The first acquisition unit 151 acquires the target shape information and the target identification information for identifying each of the attention tooth and the adjacent tooth (occlusal tooth and adjacent tooth) corresponding to the target shape information.

The target shape information is 3D data transmitted from the dental scan system 910, and is information including the shape of the tooth of interest and the shape of one or more adjacent teeth close to the tooth of interest. The target shape information is information indicating a point cloud representing the shape of at least a part of the oral cavity of the patient in the present embodiment, and more specifically, the shape of the tooth of interest and all the adjacent teeth close to the tooth of interest. Information including the shape of. The target shape information does not have to include the shape of the tooth of interest.

The target identification information is information for identifying the tooth whose shape is included in the target shape information with respect to the target shape information. In the present embodiment, the target identification information is, for example, information for specifying a portion of the target shape information corresponding to the tooth (for example, the tooth) for each tooth whose shape is included in the target shape information. Information that specifies the range indicating the shape of the tooth) and the tooth identifier that identifies the tooth are associated with each other. That is, based on the target shape information and the target identification information, it has become possible to specify 3D data indicating the shape of the tooth by associating it with a tooth identifier for each tooth corresponding to the target shape information. There is. It can be said that the target identification information is information that labels which part of the target shape information corresponds to which tooth.

The format of each data of the target identification information and the target shape information does not matter. Both may be separate information, and by associating a tooth identifier with each point of the point group included in the target identification information, the range of points indicating the tooth in the point group is associated with each tooth. May be configured to be able to identify. That is, the distinction between the target identification information and the target shape information does not have to be strict, and indicates information in which a tooth identifier is associated with each point in the point group or a tooth shape divided for each tooth. Information including 3D data (for example, point group data labeled which tooth corresponds to) may be interpreted as information in which target identification information and target shape information are combined. For example, the target shape information may be information including coordinate information of each point in a state where individual points can be identified, and the target identification information may be information in which individual points are associated with tooth identifiers. In this case, it is also possible to grasp the information recorded as a pair of the coordinates of each point and the tooth identifier as information that is both the target shape information and the target identification information. Here, the state in which points can be identified may be, for example, a state in which points can be identified by including an identifier for identifying the points as information on each point, or individual points can be identified in the target shape information. The points may be identifiable based on the order in which the information of the points exists. Further, for example, the target shape information is information including coordinate information of a plurality of points, and the target identification information includes information for specifying a space in which a point cloud indicating each tooth exists and a tooth identifier for identifying the tooth. It may be associated information or the like. In the present embodiment, the target shape information and the dentition shape information are similar 3D data in that they both include the shape of one tooth and the shape of a tooth adjacent to the tooth. Further, the target identification information and the dentition identification information both include a portion (range) indicating the shape of each tooth in the 3D data and information for identifying which tooth the portion indicates. In terms of inclusion, it can be said to be similar information. That is, the combination of the dentition shape information and the dentition identification information and the combination of the target identification information and the target shape information are both the shape of the tooth and which tooth is the tooth for each of the plurality of teeth. It can be said that the information includes the information that identifies the tooth. In other words, among the information including the tooth shape and the information for identifying which tooth the tooth is for each of the plurality of teeth, the information used for the processing in the learning unit 141 is the dentition shape information or the tooth. It is called column identification information, and what is acquired by the first acquisition unit 151 is called target shape information or target identification information.

In the present embodiment, the first acquisition unit 151 generates and acquires the target identification information based on the target shape information acquired from the dental scan system 910. The first acquisition unit 151 specifies a region including the point cloud corresponding to each tooth in the point cloud indicated by the target shape information. Then, the first acquisition unit 151 generates the target identification information by associating the specified region with the tooth identifier that identifies the tooth for each tooth.

In the present embodiment, the target identification information may be generated by the first acquisition unit 151 by using, for example, a machine learning method as follows. That is, a machine learning method is used for a learning device that inputs target shape information in advance and outputs target identification information (which may be information indicating a labeled point cloud) for the target shape information. Configure using. Specifically, for example, there are two or more pairs of target shape information and target identification information (for example, a pair of information indicating an unlabeled point group and information indicating a labeled point group). The two or more sets of information are prepared and given to a module for configuring a learning device for machine learning to configure the learning device, and the configured learning device is stored in the storage unit 110. In the present embodiment, it is particularly preferable to use a framework capable of segmenting a point cloud and classifying each segment as a machine learning method. For example, known frameworks such as "PointNet ++" (http://stanford.edu/￣rqi/pointnet2/) and "PointNet" (http://stanford.edu/￣rqi/pointnet/) can be used. can.
Further, the first acquisition unit 151 may be realized by combining the Dynamic Graph Convolutional Neural Network (DGCNN) and the Mask R-CNN. In DGCNN, the gums and dentition are separated. Then, the Mask R-CNN identifies each tooth constituting the dentition by using an image obtained by projecting the dentition from a plurality of viewpoints.

The first acquisition unit 151 may be configured to acquire the target identification information generated in advance corresponding to the target shape information by receiving it from an external device or the like.

The second acquisition unit 152 uses the learning device stored in the learning device storage unit 111 to obtain the shape of the dental restoration corresponding to the tooth of interest from the target identification information and the target shape information acquired by the first acquisition unit 151. Generates and obtains restoration information to represent. In addition, the second acquisition unit 152 generates and acquires 3D data representing the shape of the dental restoration using the restoration information.

In the present embodiment, the restoration information is generated as follows, for example. That is, the second acquisition unit 152 generates input information to be input to the learner based on the information acquired by the first acquisition unit 151. When the input information is generated, the second acquisition unit 152 inputs the input information to the learning device stored in the learning device storage unit 111, generates and acquires the output information. In this case, the second acquisition unit 152 acquires the learning device corresponding to the target identification information acquired by the first acquisition unit 151 from the learning device storage unit 111, and generates output information using the learning device. In other words, the second acquisition unit 152 generates output information using the learner acquired from the learner storage unit 111 using the tooth identifier of the tooth that is the tooth of interest. Then, the second acquisition unit 152 generates and acquires the restoration information based on the output information. That is, the second acquisition unit 152 acquires the restoration information by using the machine learning method. A machine learning method that can be applied to a regression problem that outputs numerical data from some numerical data can be used, for example, deep learning (for example, Deep Feed Forward Neural Networks, etc.), random forest, polypoly regression, etc. SGD regression, LASSO regression, Ridge regression and the like can be applied. For machine learning, functions in various known machine learning frameworks and various existing libraries can be used.

More specifically, the generation of input information and the generation of restoration information based on output information are performed using the same method as that performed in the learning unit 141. For example, the second acquisition unit 152 generates the first vector information representing a multidimensional vector based on the target identification information and the target shape information acquired by the first acquisition unit 151. Then, by reducing the dimension of the generated first vector information, the second vector information representing the feature vector having a lower dimension than the vector represented by the first vector information is generated and used as the input information. Further, for example, the second acquisition unit 152 performs the inverse transformation of the dimension reduction performed at the time of generating the input information on the output information obtained by using the learner, and generates the inverse transformed information as the restoration information. .. In this embodiment, one first vector information or second vector information may be generated based on the shape of each of the two or more adjacent teeth. Further, two or more first vector information and two or more second vector information corresponding to two or more adjacent teeth may be generated.

Here, in the present embodiment, when the first vector information is generated at the time of generating the input information, the second acquisition unit 152 performs mesh registration (sometimes referred to as point cloud registration). Mesh registration is performed, for example, as follows.

That is, the second acquisition unit 152 uses the reference shape information (hereinafter referred to as a template) of the corresponding tooth stored in the reference shape information storage unit 112 for the 3D data indicating each tooth in the point group to make a point. Adjust the number of groups to the predetermined number corresponding to the template. Then, the second acquisition unit 152 adjusts the position of each point in the point cloud so that the entire shape follows the template. Specifically, while maintaining the overall shape of each tooth, each point in the point cloud should not be too far from a nearby point, and the deviation between the point cloud data and the template should be reduced. In addition, the deviation between the point cloud data and the marker coordinates of the template should be reduced. This makes it possible to acquire highly accurate restoration information. As a specific method of mesh registration, a known method can be used.
However, in the known mesh registration, each vertex of the template is associated with the nearest vertex of the point cloud data (3D data) corresponding to each tooth before being moved. As a result, the point cloud data corresponding to each tooth can be expressed by the number of point clouds in the template. However, by expressing the 3D data of each tooth with a template, the unevenness expressed by the 3D data of each tooth may be filled. Therefore, among the vertices of the point cloud data corresponding to each tooth, the vertices whose curvature is equal to or less than the threshold (the convex direction is positive and the concave direction is negative) are associated with the vertices of the template from the vertices of the point cloud data. For other data, mesh registration may be performed by associating the vertices of the template with the vertices of the point cloud data. As a result, it is possible to generate restoration information with good meshing.

It should be noted that the learning unit 141 also performs mesh registration on the dentition identification information and the dentition shape information in the same manner when generating the first vector information at the time of generating the input information performed during machine learning. It should be.

In addition, acquisition of 3D data representing the shape of the dental restoration using the restoration information is performed, for example, as follows. That is, when the second acquisition unit 152 acquires the restoration information corresponding to the attention tooth, the second acquisition unit 152 acquires the reference shape information of the attention tooth from the reference shape information storage unit 112. Then, the second acquisition unit 152 generates 3D data indicating the shape of the dental restoration by using the acquired reference shape information and the restoration information.

In the present embodiment, when the second acquisition unit 152 acquires the 3D data, the learning unit 141 generates a learning device using the 3D data. In this case, the learning unit 141 interferes with the dental restoration and the surrounding teeth based on the target shape information acquired by the first acquisition unit 151 and the restoration information or 3D data acquired by the second acquisition unit. To evaluate. Then, the learning unit 141 changes the parameter of the penalty term of the cost function based on the evaluation result. As a result, the restoration information subsequently generated by using the learner reflects a predetermined rule.

The output unit 160 outputs information by transmitting information to another device using the transmission unit 170 or the like, or outputs information by displaying the information on a display device provided in the information processing system 100, for example. To do. The output unit 160 may or may not include an output device such as a display or a speaker. The output unit 160 can be realized by the driver software of the output device, the driver software of the output device, the output device, or the like.

In the present embodiment, the output unit 160 outputs the 3D data acquired by the second acquisition unit 152.

The transmission unit 170 transmits the information to other devices constituting the dental restoration production system 900 via the network. The transmission unit 170 transmits, for example, the information output by the output unit 160. The transmission unit 170 is usually realized by wireless or wired communication means, but may be realized by broadcasting means.

FIG. 3 is a flowchart showing an example of the operation of the information processing system 100.

The information processing system 100 performs an operation related to the output of 3D data of the dental restoration, for example, as follows.

(Step S101) The learning unit 141 acquires a learning device by the "learning device acquisition / storage process" described later, and stores the learning device in the learning device storage unit 111. The process according to step S101 can save labor when the learner is stored in the learner storage unit 111 in advance.

(Step S102) The first acquisition unit 151 acquires target shape information including the shape of the tooth of interest.

(Step S103) The first acquisition unit 151 performs a process of extracting information on each tooth by a "identification information acquisition process" described later, and acquires the target identification information based on the target shape information.

(Step S104) The second acquisition unit 152 acquires output information (output information acquisition process).

(Step S105) The second acquisition unit 152 acquires the restoration information by performing inverse transformation on the dimension-reduced information which is the output information.

(Step S106) The second acquisition unit 152 acquires the reference shape information of the tooth of interest from the reference shape information storage unit 112.

(Step S107) The second acquisition unit 152 acquires 3D data indicating the shape of the dental restoration for the tooth of interest based on the acquired restoration information and the reference shape information.

(Step S108) The output unit 160 outputs the acquired 3D data so that it can be handled by CAD. As a result, the user can edit the data on the CAD by using the CAD / CAM system, and can model the dental restoration by the modeling device 920.

FIG. 4 is a flowchart showing an example of the learning device acquisition / accumulation process performed by the information processing system 100.

(Step S121) The learning unit 141 acquires two or more learning target information including restoration information related to the same attention tooth from the learning target information storage unit 113.

(Step S122) The learning unit 141 executes mesh registration based on the dentition identification information and the dentition shape information of each learning target information. Further, the learning unit 141 generates the first vector information based on the information generated by the mesh registration.

(Step S123) The learning unit 141 reduces the dimension of the first vector information and generates the second vector information. The second vector information is input information to the learner.

(Step S124) The learning unit 141 reduces the dimension of the vector indicating the restoration information of each learning target information, and generates the third vector information. The third vector information becomes the output information of the learner.

(Step S125) The learning unit 141 performs machine learning using a combination of the second vector information and the third vector information for each learning target information. The learning unit 141 acquires a learning device by performing machine learning.

(Step S126) The learning unit 141 stores the acquired learning device in the learning device storage unit 111 in association with the tooth identifier of the tooth of interest. The process returns to the process shown in FIG.

FIG. 5 is a flowchart showing an example of the identification information acquisition process performed by the information processing system 100.

(Step S141) The first acquisition unit 151 acquires from the storage unit 110 a learner configured by machine learning using the learning data that inputs the point cloud data and outputs the labeled point cloud data. ..

(Step S142) The first acquisition unit 151 inputs the acquired target shape information into the acquired learner.

(Step S143) The first acquisition unit 151 acquires the target identification information which is the output of the learner. As a result, the first acquisition unit 151 can acquire, for example, the target shape information (target identification information) in which the point cloud is labeled. The process returns to the process shown in FIG.

FIG. 6 is a flowchart showing an example of the output information acquisition process performed by the information processing system 100.

(Step S161) The second acquisition unit 152 acquires the reference shape information from the reference shape information storage unit 112 for each tooth corresponding to the target identification information.

(Step S162) The second acquisition unit 152 performs mesh registration for each tooth corresponding to the target identification information using the acquired reference shape information.

(Step S163) The second acquisition unit 152 generates the first vector information using the information generated by the mesh registration.

(Step S164) The second acquisition unit 152 reduces the dimension of the first vector information and generates the second vector information. The second vector information is input information to the learner.

(Step S165) The second acquisition unit 152 acquires a learning device, which is a learning device corresponding to the tooth of interest, from the learning device storage unit 111.

(Step S166) The second acquisition unit 152 inputs the input information to the acquired learner and acquires the output information. The process returns to the process shown in FIG.

FIG. 7 is a diagram illustrating a specific example of 3D data showing the shape of the dental restoration acquired by the second acquisition unit 152.

In FIG. 7, one attention tooth E and its adjacent teeth A, B, C, and D are schematically shown. The attention tooth E is, for example, one of the lower teeth, and the proximity teeth A and D are teeth located on both sides of the attention tooth E so as to be adjacent to the attention tooth E. Proximity teeth B and C are teeth located on the upper side, and can be said to be adjacent to the attention tooth E in the vertical direction, and are subject to engagement with the attention tooth E.

As a dental restoration used for such a noteworthy tooth E, it slightly interferes with the adjacent teeth A and D adjacent to the left and right, and does not interfere with the engagement with the adjacent teeth B and C adjacent to the upper and lower sides. It is required to have a shape. In the present embodiment, the learning device corresponding to the attention tooth E can obtain 3D data of the dental restoration having an appropriate shape for the attention tooth E.

That is, the learning device is generated by the learning target information including the dentition shape information corresponding to the adjacent teeth A, B, C, and D and the restoration information corresponding to the attention tooth E. Then, the second vector information obtained by the first acquisition unit 151 performing processing such as dimension reduction using the target shape information corresponding to the proximity teeth A, B, C, and D (in the figure, the proximity teeth A, Using λ_1, λ_2, ..., λ_n (underscore indicates that the character following it is a subscript) shown for each of B, C, and D as input information, the characteristics of the tooth E of interest are described by the learner. The output information (λbar_11, λbar_2, ..., λbar_n (bar means the horizontal line above λ in FIG. 7)), which is the vector shown, is obtained. By performing processing such as inverse transformation of dimension reduction on the output information, it is possible to generate restoration information corresponding to the shape of the dental restoration. When obtaining output information for one noteworthy tooth, the combination of the dentition shape information corresponding to the close tooth used when obtaining the learner and the target shape information corresponding to the close tooth when obtaining the input information. The combination must match. That is, when there are a plurality of types of learners having different combinations of dentition shape information used for one tooth of interest, a combination of dentition shape information corresponding to the combination of target shape information to be used is used. The learning device generated by using the learning device may be used. Further, in the example shown in the figure, the second vector information for each of the adjacent teeth A, B, C, and D is shown, and thus two or more second vector information is used as the input information. Alternatively, one second vector information may be constructed based on the shapes of the adjacent teeth A, B, C, and D, and may be used as input information.

As described above, in the present embodiment, a plurality of second vector information obtained from the dentition shape information of a plurality of teeth adjacent to the arbitrarily selected selected tooth and the dentition shape of the selected tooth. One or two or more second vector information obtained from the target shape information of a plurality of adjacent teeth close to the attention tooth is input to the learner adjusted by the output information obtained from the information, and the attention tooth is input. The corresponding output information is generated. Then, from the generated output information, restoration information for expressing the shape of the dental restoration corresponding to the tooth of interest is generated. The generation of the output information from the dentition shape information of the selected tooth is performed by, for example, the PCA, and the generation of the restoration information from the output information is performed, for example, by the inverse transformation of the PCA. According to this embodiment, 3D data representing the shape of the dental restoration can be easily output. That is, it is possible to eliminate the need for the modeling work of the dental restoration by the worker, which conventionally required a considerable amount of man-hours, to significantly reduce the man-hours required for modeling, and to perform high-precision dentistry by machine learning. 3D data representing the shape of the restoration can be obtained. In addition, in the conventional method, the skill differs depending on the worker, and the degree of completion of the dental restoration tends to differ depending on the worker who produces the dental restoration (the quality of the dental restoration is personal). ). On the other hand, according to the present embodiment, the information processing system 100 can output 3D data representing the shape of the dental restoration according to the scan data of each patient based on the learning device, so that the dentistry can be output. It is possible to reduce the personal part in the production process of the restoration.

Further, according to the present embodiment, by using the penalty term of the cost function of machine learning, it is possible to obtain 3D data according to the rule regarding the interference between the dental restoration and the surrounding teeth. .. A learning device can be prepared for each tooth, and restoration information can be generated using the learning device corresponding to the tooth of interest. Therefore, it is possible to output 3D data representing the shape of the dental restoration with higher accuracy.

In the present embodiment, when the learning device is generated or the restoration information is generated by the second acquisition unit, the input information generated by using the attribute information such as the patient's attribute may be used. For example, classification information on each patient's gender, age, natural anthropological classification, lifestyle-related information, and the like may be used. In this case, among the reference shape information stored in the reference shape information storage unit 112, the reference shape information corresponding to the attribute information may be used for generating the restoration information.

Note that the processing in this embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded on a recording medium such as a CD-ROM and disseminated.

The software that realizes the information processing system 100 in this embodiment is the following program. That is, this program is a program for outputting information for producing a dental restoration used for one or more attention teeth to be restored, and is a dentition shape including two or more tooth shapes adjacent to each other. It is provided with a learning device storage unit for storing a learning device for acquiring restoration information for representing the shape of a dental restoration corresponding to a part of two or more teeth obtained by using the information. A computer possessed by an information processing system is used to identify target shape information including the shape of the attention tooth and the shape of one or more adjacent teeth adjacent to the attention tooth and target shape information corresponding to each of the attention tooth and the proximity tooth. The first acquisition unit that acquires the identification information, and the restoration information for expressing the shape of the dental restoration corresponding to the tooth of interest from the target identification information and the target shape information acquired by the first acquisition unit using the learning device. This is a program for functioning as a second acquisition unit.

(Embodiment 2)

The outline of the second embodiment will be described with respect to the parts different from the first embodiment described above. In the second embodiment, an information processing system having basically the same configuration as that of the first embodiment is used, but a part of the operation when acquiring the target identification information is different from that of the first embodiment. There is.

That is, in the second embodiment, the user can use the annotation tool which is the software realized by the operation of the first acquisition unit 151. The first acquisition unit 151 can acquire the target identification information based on the user's input operation using the annotation tool and the target shape information.

That is, in the second embodiment, the first acquisition unit 151 confirms the relationship between each point included in the point cloud and the surrounding points with respect to the acquired target shape information. Then, by identifying the part of the point cloud having a predetermined relationship and performing segmentation of the point cloud, the area including the point cloud corresponding to each tooth is estimated from the point cloud indicated by the target shape information. ..

In the second embodiment, the first acquisition unit 151 calculates the curvature (for example, the minimum curvature) at each vertex of the mesh formed by using the point cloud, and classifies the point cloud based on the calculated curvature. .. Specifically, the first acquisition unit 151 specifies a portion having a curvature larger than a predetermined first threshold value (a portion having a radius of curvature smaller than a predetermined second threshold value). Then, a process of converting the specified portion into a single line in the three-dimensional space is performed, and the boundary to be divided is determined. For the process of single-tracking, for example, it is preferable to use a library such as "Skeletonize" (https://scikit-image.org/docs/dev/auto_expamples/edges/prot_skeleton.html). Not limited. By performing the process of estimating the boundary for dividing based on such curvature, it is possible to accurately estimate the boundary between individual teeth and the boundary between teeth and gums.

In the above case, the annotation tool may accept the adjustment operation for changing the first threshold value or the second threshold value by the user, and classify the point cloud according to the received adjustment operation. In this case, even if the user interface of the annotation tool is provided with a slider bar having a slider that the user can slide using a pointing device or the like, the threshold value is set according to the position of the slider in the slider bar. good. Further, the output unit 160 accepts the adjustment operation while outputting the shape indicated by the target shape information and the information indicating the area to be divided to the display device, and immediately displays on the display device each time the adjustment operation is performed. The mode may be changed according to the adjustment operation.

When the point cloud is divided and the area including the point cloud corresponding to each tooth is estimated, the annotation tool is used by the user to perform a labeling operation for associating which tooth is indicated for each area. Is done using. Based on the result of the labeling operation, the point cloud of each region is associated with the tooth identifier.

Specifically, for example, the first acquisition unit 151 displays the point cloud included in the estimated area and the point cloud not included in the area on the display by the output unit 160 in different display modes, and the user may display the point cloud. Get the input annotation information. Displaying in different display modes includes, for example, different display colors, different dot sizes, different background colors and different background patterns, but is not limited to these. The annotation information is information that identifies the tooth indicated by each region. The first acquisition unit 151 can specify the area including the point group corresponding to each tooth based on the annotation information input by the user. The first acquisition unit 151 associates the point cloud included in each region with the tooth identifier corresponding to the region based on the annotation information.

As described above, in the second embodiment, the target identification information corresponding to the target shape information can be acquired by using the annotation tool.

Also in the second embodiment, similarly to the first embodiment, the first acquisition unit 151 performs mesh registration with respect to the target shape information, but the present invention is not limited to this.

FIG. 8 is a diagram showing an example of the operation performed by the first acquisition unit 151 according to the second embodiment.

In FIG. 8, among the operations performed by the first acquisition unit 151, the operations related to the acquisition of the target identification information using the annotation tool are shown. The process shown in FIG. 8 is performed as a process of acquiring target identification information (step S103) in the process shown in FIG. 3 executed in the second embodiment.

(Step S241) The first acquisition unit 151 reads the acquired target shape information.

(Step S242) The first acquisition unit 151 calculates the curvature at each vertex of the read target shape information.

(Step S243) The first acquisition unit 151 acquires the set threshold value of the curvature.

(Step S244) The first acquisition unit 151 calculates a dividing line that divides the area of the point cloud based on the acquired threshold value. Further, the first acquisition unit 151 receives an adjustment operation from the user regarding the dividing line, and recalculates the dividing line based on the adjustment operation.

(Step S245) The first acquisition unit 151 divides the area based on the calculated dividing line. As a result, the area belonging to each tooth and the area belonging to the gums and the like are partitioned.

(Step S246) The first acquisition unit 151 accepts the area integration operation by the user and reflects it in the point cloud section.

(Step S247) The first acquisition unit 151 accepts a labeling operation by the user. Thereby, which tooth each region belongs to is associated.

(Step S248) The first acquisition unit 151 acquires the target identification information based on the acceptance result of the labeling operation. As a result, the labeling content and the target shape information are acquired in association with each other.

FIG. 9 is a diagram illustrating a specific example of an annotation tool that can be used by a user in the information processing system 100.

FIG. 9 shows, for example, an example of an operation screen of the annotation tool displayed on the display device of the information processing system 100. On the operation screen, the display data G1 showing the read target shape information in the 3D space and the operation column G2 including two or more buttons associated with various commands for operating the data are displayed. included. In the present embodiment, the data included in the target shape information can be manipulated or divided by operating the buttons of the operation column G2 or performing the operation of partially selecting each data in the display data G1. You can adjust the line. The display mode of the display data G1 is appropriately changed according to the operation performed by the user, the operation applied to the target shape information, and the like. As a result, the user's attention can be focused on the portion where the display mode has been changed, and the user can be made aware that the operation has been reflected. Therefore, the user can perform an intuitive operation.

In this specific example, the operation column G2 includes a slider bar G3 for changing a threshold value related to curvature. The user can intuitively adjust the threshold value by adjusting the position of the slider on the slider bar G3.

The operation column G2 also includes a merge button G4 for joining regions. For example, the user can combine two or more selected areas into one area by operating the merge button G4 after performing an operation of selecting two or more areas included in the display data G1. .. The combined region may be decomposed.

In this specific example, the operation screen includes a label selection tray G11 and a label assignment button G12. The label selection tray G11 includes, for example, two or more label selection buttons corresponding to each tooth identifier that can be used for labeling by the user. For example, the user performs an operation of selecting an area included in the display data G1 and then an operation of selecting a label selection button on the label selection tray G11. Then, by operating the label assignment button G12 in that state, a tooth identifier corresponding to the selected label selection button can be assigned (labeled) to the selected area.

The operation screen of the annotation tool is not limited to this, and can be set as appropriate.

Also in the second embodiment, the same effect as that of the first embodiment can be obtained. In the second embodiment, as described above, the annotation tool can be used to accept operations related to point cloud classification and labeling by the user. Therefore, even when processing the target shape information including a plurality of connected teeth, which has conventionally taken a long time for the user to process, the operation of specifying the shape of each tooth can be easily performed. .. In addition, the user can easily perform an operation of associating a tooth identifier that identifies the tooth with a specified area for each tooth. By accepting the user's operation using the GUI in this way, the user can perform the operation more intuitively.

The software that realizes the annotation tool in the second embodiment is the following program. That is, this program is a program for processing the target shape information indicating the point cloud representing the shape of at least a part of the oral cavity, and among the point clouds indicated by the target shape information on the computer of the information processing system 100. The area including the point cloud corresponding to each tooth is estimated based on the relationship between each point included in the point cloud and the surrounding points, and the point cloud included in the estimated area and the point cloud not included in the area are estimated. Are displayed on the display in different display modes. In addition, this program causes the computer to acquire information input by the user (selected label, etc.), and identifies an area including a point group corresponding to each tooth based on the information input by the user. The target identification information in which the tooth identifier that identifies the tooth is associated with the specified area is acquired for each tooth.

Note that the annotation tool may be able to accept area integration operations.

The annotation tool according to the second embodiment is not limited to the information processing system 100 having a function of generating restoration information using a learning device, and processes related to 3D data such as a point cloud (including meshed data). It may be feasible in the various devices that perform. In this case, the correspondence between the point cloud included in each area and the identifier associated with the area can be output and used in other devices.

(others)

FIG. 10 is an overview view of the computer system 800 according to the above embodiment. FIG. 11 is a block diagram of the computer system 800.

In these figures, the configuration of a computer that executes the program described in the present specification and realizes the information processing system and the like of the above-described embodiment is shown. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

The computer system 800 includes a computer 801 including a CD-ROM drive, a keyboard 802, a mouse 803, and a monitor 804.

In addition to the CD-ROM drive 8012, the computer 801 is connected to the MPU 8013, the bus 8014 connected to the CD-ROM drive 8012, the ROM 8015 for storing programs such as the bootup program, and the MPU 8013. It includes a RAM 8016 for temporarily storing program instructions and providing a temporary storage space, and a hard disk 8017 for storing application programs, system programs, and data. Although not shown here, the computer 801 may further include a network card that provides a connection to the LAN.

The program that causes the computer system 800 to execute the functions of the information processing system and the like according to the above-described embodiment may be stored in the CD-ROM 8101, inserted into the CD-ROM drive 8012, and further transferred to the hard disk 8017. Alternatively, the program may be transmitted to the computer 801 via a network (not shown) and stored on the hard disk 8017. The program is loaded into RAM 8016 at run time. The program may be loaded directly from the CD-ROM 8101 or the network.

The program does not necessarily have to include an operating system (OS) or a third-party program that causes the computer 801 to execute functions such as the information processing system of the above-described embodiment. The program need only include a portion of the instruction that calls the appropriate function (module) in a controlled manner to obtain the desired result. It is well known how the computer system 800 works, and detailed description thereof will be omitted.

In the above program, in the transmission step of transmitting information and the receiving step of receiving information, processing performed by hardware, for example, processing performed by a modem or interface card in the transmission step (only performed by hardware). Processing that is not done) is not included.

Further, in the above embodiment, the two or more components existing in one device may be physically realized by one medium.

Further, in the above embodiment, each process (each function) may be realized by centralized processing by a single device (system), or may be realized by distributed processing by a plurality of devices. You may. When the distributed processing is performed by a plurality of devices, it is also possible to grasp the entire system composed of the plurality of devices performing the distributed processing as one "device". In the above-described embodiment, the information processing system acquires 3D data representing the shape of the dental restoration by using the acquired restoration information, but the present invention is not limited to this. The information processing system acquires restoration information for representing the shape of the dental restoration, and outputs the restoration information and the information generated based on the restoration information to an external device or the like as information for producing the dental restoration. You may try to do it. In the external device, for example, a dental CAD / CAM system or the like is used to acquire 3D data representing the shape of the dental restoration or to obtain the dental restoration based on the restoration information and the information generated based on the restoration information. 3D data for modeling may be acquired.

Further, in the above embodiment, the transfer of information performed between the respective components depends on, for example, one of the components when the two components that transfer the information are physically different. It may be performed by outputting information and accepting information by the other component, or if the two components that pass the information are physically the same, one component. It may be performed by moving from the processing phase corresponding to the above to the processing phase corresponding to the other component.

Further, in the above embodiment, information related to the processing executed by each component, for example, information received, acquired, selected, generated, transmitted, or received by each component. In addition, information such as threshold values, mathematical formulas, and addresses used by each component in processing may be temporarily or for a long period of time in a recording medium (not shown) even if it is not specified in the above description. In addition, each component or a storage unit (not shown) may store information on a recording medium (not shown). Further, each component or a reading unit (not shown) may read the information from the recording medium (not shown).

Further, in the above embodiment, when the information used in each component or the like, for example, the information such as the threshold value and the address used in the processing by each component and various setting values may be changed by the user, the above Although not specified in the description, the user may or may not be able to change the information as appropriate. When the information can be changed by the user, the change is realized by, for example, a reception unit (not shown) that receives a change instruction from the user and a change unit (not shown) that changes the information in response to the change instruction. You may. The reception unit (not shown) may accept the change instruction from, for example, an input device, information transmitted via a communication line, or information read from a predetermined recording medium. ..

The present invention is not limited to the above embodiments, and various modifications can be made, and these are also included in the scope of the present invention.

An embodiment may be configured by appropriately combining the above-mentioned plurality of embodiments. For example, each component of any of the above embodiments may be optionally replaced or combined with a component of another embodiment. In addition, some components and functions may be omitted from the above-described embodiments.

In the above-described embodiment, the learning device is a learning device obtained by machine learning, but the learning device is not limited to this.

The learner may be, for example, a table showing the correspondence between the input vector based on the input information indicating the shapes of two or more teeth including the attention tooth and the restoration information applied to the attention tooth. good. In this case, the second acquisition unit may acquire the restoration information corresponding to the feature vector based on the target shape information from the table. In addition, the second acquisition unit generates a vector that approximates the feature vector based on the target shape information by using two or more input vectors in the table and parameters that weight each input vector, and each used for the generation. The restoration information and parameters corresponding to the input vector may be used to acquire the restoration information applied to the tooth of interest.

Further, the learner has a relationship between, for example, an input vector based on input information indicating the shape of two or more teeth including the tooth of interest and information for generating restoration information applied to the tooth of interest. It may be a function representing. In this case, the second acquisition unit may, for example, obtain information corresponding to the feature vector based on the target shape information by a function, and acquire the restoration information using the obtained information.

As described above, the information processing system according to the present invention has the effect of being able to easily output 3D data representing the shape of the dental restoration, and is useful as an information processing system or the like.

100 Information processing system 110 Storage unit 111 Learning device storage unit 112 Reference shape information storage unit 120 Reception unit 130 Reception unit 140 Processing unit 141 Learning unit 151 First acquisition unit 152 Second acquisition unit (Example of restoration information acquisition unit)
160 Output unit 170 Transmission unit 900 Dental restoration production system 910 Dental scanning system 920 Modeling equipment

Claims

An information processing system that outputs information for producing a dental restoration used for one or more attention teeth to be restored.
Acquire restoration information for representing the shape of a dental restoration corresponding to a part of the two or more teeth obtained by using the dentition shape information including the shapes of two or more teeth adjacent to each other. A learning device storage unit that stores learning devices for
A first acquisition unit that acquires target shape information including the shape of one or more adjacent teeth close to the attention tooth and target identification information for identifying a tooth whose shape is included in the target shape information.
Using the learning device, the second acquisition unit that acquires the restoration information corresponding to the attention tooth from the target identification information and the target shape information acquired by the first acquisition unit, and the second acquisition unit.
Information processing system equipped with.
A reference shape information storage unit for storing reference shape information indicating a reference shape prepared in advance corresponding to the attention tooth is provided.
The restoration information is information corresponding to the difference between the reference shape and the shape of the dental restoration corresponding to the attention tooth.
The second acquisition unit acquires 3D data representing the shape of the dental restoration by using the restoration information and the reference shape information.
The information processing system according to claim 1.
The learning device is information obtained by machine learning performed using two or more learning target information.
The learning target information is
The dentition shape information and
The dentition identification information for identifying each of the two or more teeth included in the dentition shape information, and the dentition identification information.
Includes restoration information to represent the shape of the dental restoration applied to some of the two or more teeth.
The information processing system according to claim 1 or 2.
In the learning device storage unit, restoration information for expressing the shape of the dental restoration corresponding to one or more attention teeth, the dentition identification information, and the shape of two or more teeth close to the attention tooth are provided. A learning device obtained by machine learning using two or more learning target information including the including tooth row shape information is stored for each tooth of interest.
The second acquisition unit uses a learning device corresponding to the target identification information acquired by the first acquisition unit to obtain the restoration information from the target identification information and the target shape information acquired by the first acquisition unit. get,
The information processing system according to claim 3.
The second acquisition unit generates first vector information representing a multidimensional vector based on the target identification information and the target shape information acquired by the first acquisition unit, and with respect to the generated first vector information. Second vector information is generated by performing dimension reduction, and the restoration information is acquired using the generated second vector information.
The information processing system according to claim 3 or 4.
The learning unit adjusts the parameters of the cost function including the penalty term based on the evaluation of the shape, including the evaluation of the interference between the dental restoration and the surrounding teeth.
The information processing system according to any one of claims 3 to 5.
When the second acquisition unit acquires the 3D data, the learning unit uses the target shape information acquired by the first acquisition unit and the restoration information or the 3D data acquired by the second acquisition unit. Based on the above, the evaluation is performed, and the parameter of the penalty term is changed based on the evaluation result.
The information processing system according to claim 6.
The dentition shape information is information representing at least a part of the shape in the oral cavity including the shape of the tooth corresponding to the dental restoration and the shape of all the teeth adjacent to the tooth.
The target shape information is information representing at least a part of the shape in the oral cavity, including the shape of the tooth of interest and the shape of a tooth close to the tooth of interest.
The information processing system according to any one of claims 1 to 7.
The target shape information is information indicating a point cloud representing the shape of at least a part of the oral cavity.
The first acquisition unit specifies an area including a point group corresponding to each tooth in the point group indicated by the target shape information, and associates a tooth identifier that identifies the tooth with the specified area for each tooth. Acquire the target identification information
The information processing system according to any one of claims 1 to 8.
The first acquisition unit
The area including the point cloud corresponding to each tooth in the point cloud indicated by the target shape information is estimated based on the relationship between each point included in the point cloud and the surrounding points.
The point cloud included in the estimated area and the point cloud not included in the area are displayed on the display in different display modes, and information on the display mode input by the user is acquired.
Identify the area containing the point group corresponding to each tooth based on the information entered by the user.
The information processing system according to claim 9.
The first acquisition unit displays the point cloud included in each of the specified areas on the display, acquires the labeling information input by the user for the point cloud included in each area, and uses the information input by the user as the information. Acquire the target identification information based on
The information processing system according to claim 10.
An information processing system that outputs information for producing a dental restoration used for one or more attention teeth to be restored.
Learning adjusted by a plurality of vector information obtained from information indicating the shape of each of a plurality of teeth adjacent to an arbitrarily selected selected tooth and output information obtained from information indicating the shape of the selected tooth. The learning device storage unit where the device is stored and
Using the learner, the output information corresponding to the attention tooth is acquired based on a plurality of vector information obtained from the information indicating the shape of each of the plurality of teeth adjacent to the attention tooth, and the acquired output information is obtained. The restoration information acquisition unit that acquires restoration information for expressing the shape of the dental restoration corresponding to the attention tooth from
Information processing system equipped with.
Acquire restoration information for representing the shape of a dental restoration corresponding to a part of the two or more teeth obtained by using the dentition shape information including the shapes of two or more teeth adjacent to each other. Produces dental restorations used for one or more notable teeth to be restored, which are realized by the first acquisition unit and the second acquisition unit of the information processing system including the learning device storage unit for storing the learning device. It is an information processing method that outputs information for
The first acquisition unit identifies the target shape information including the shapes of one or more adjacent teeth adjacent to the attention tooth and the target identification of the target shape information for identifying the teeth whose shape is included in the target shape information. The first acquisition step to acquire information and
Restoration information for the second acquisition unit to represent the shape of the dental restoration corresponding to the attention tooth from the target identification information and the target shape information acquired by the first acquisition unit using the learning device. Including the second acquisition step to acquire
Information processing method.
A program for outputting information for producing a dental restoration used for one or more attention teeth to be restored.
Acquire restoration information for representing the shape of a dental restoration corresponding to a part of the two or more teeth obtained by using the dentition shape information including the shapes of two or more teeth adjacent to each other. A computer in an information processing system equipped with a learning device storage unit in which a learning device for storing a learning device is stored.
The first to acquire the target shape information including the shapes of one or more adjacent teeth adjacent to the attention tooth and the target identification information for identifying the teeth whose shape is included in the target shape information for the target shape information. With one acquisition department
Using the learning device, the second acquisition unit acquires the restoration information for expressing the shape of the dental restoration corresponding to the attention tooth from the target identification information and the target shape information acquired by the first acquisition unit. To function as
program.