EP4789217A1

EP4789217A1 - Combined orthodontic and dental restorative treatments

Info

Publication number: EP4789217A1
Application number: EP24791527.5A
Authority: EP
Inventors: Jonathan D. Gandrud; James D. Hansen; David K. Cinader, Jr.; Lois F. Duerst; Donna J. STENBERG
Original assignee: Solventum Intellectual Properties Co
Current assignee: Solventum Intellectual Properties Co
Priority date: 2023-10-05
Filing date: 2024-10-04
Publication date: 2026-08-12
Also published as: WO2025074322A1; CN121942047A

Abstract

Systems and techniques for automatically determining when a first treatment (e.g., orthodontic treatment) is recommended prior to performing a second treatment (e.g., dental restorative treatment) are disclosed including receiving a three-dimensional (3D) representation of a patient's dentition with at least one of gums of the patient or one or more teeth of the patient, computing one or more oral care metrics based on the 3D representation of the patient's dentition, comparing the one or more oral care metrics with a respective target threshold, determining based on the comparison a requirement for orthodontic treatment prior to a dental restorative treatment of one or more target pre-restoration teeth, and generating, based on the determining, a rendering of one or more indications recommending orthodontic treatment for the patient prior to dental restorative treatment on the one or more target pre-restoration tooth.

Description

COMBINED ORTHODONTIC AND DENTAL RESTORATIVE TREATMENTS

Related Documents

[0001] The entire disclosure of PCT Application No. PCT/IB2022/057373 is incorporated herein by reference. The entire disclosures of each of PCT Applications with Publication Nos. WO2022123402A1, WO2021245480A1, and W02020026117A1 are incorporated herein by reference. The entire disclosure of each of the following Provisional U.S. Patent Applications is incorporated herein by reference: 63/432,627; 63/366,492; 63/366,495; 63/352,850; 63/366,490; 63/366,494; 63/370,160; 63/366,507; 63/352,877; 63/366,514; 63/366,498; 63/366,514; 63/264,914; and 63/432,627.

Technical Field

[0002] This disclosure relates to methods of performing combined orthodontic and dental restorative treatments of a patient, and the configurations and training of computer implemented methods to determine when such combined treatments are required.

Summary

[0003] The present disclosure describes systems and techniques for digital dentistry or digital orthodontics. Techniques of this disclosure may receive one or more three-dimensional (3D) representations of a patient’s dentition comprising at least one of the gums of the patient or one or more teeth of the patient. The techniques may predict that an orthodontic treatment is required for the patient prior to the initiation of a dental restorative treatment on one or more aspects of the patient’s dentition (e.g., such determining may be based on the one or more aspects of the patient’s dentition prior to initiation of the dental restorative treatment). The techniques may automatically generate, using a trained machine learning model and in response to determining that orthodontic treatment is required for the patient, one or more transforms to place at least one tooth of the patient’s dentition into a pose for an orthodontic setup. The trained machine learning (ML) model may be trained using the following operations. The ML model may receive a plurality of historical cohort patient case data, wherein the historical cohort patient case data includes one or more representations of one or more reference tooth movements for one or more patients and one or more representations of the patients’ dental anatomy. The ML model may predict one or more transforms to apply to one or more teeth of the 3D representation of the patient’s dentition. The ML model may quantify the difference between a representation of the one or more transforms predicted by the generator and a representation of one or more reference tooth movements (or transforms). The ML model may generate a loss value based on the quantifying, and modify the machine learning model based, at least in part, on the loss value. In some instances, the anatomy of one or more patient cases that is used in training may have required dental restorative treatment, and orthodontic treatment may have been required prior to that dental restorative treatment. The trained ML model may generate one or more generated transforms. Such transforms may place the teeth of the patient into poses which correspond to intermediate stages or final setups. The intermediate stages may each sequentially depict the teeth of the patient with progressive modification in the poses of the teeth for attaining the generated final setup. Techniques described herein may compute one or more oral care metrics based on the 3D representation of the patient’s dentition, the one or more oral care metrics quantifying at least one of the shape of one or more teeth of the patient’s dentition or the spatial relationship between two or more teeth of the patient’s dentition. The techniques may further compare the values of the one or more oral care metrics to corresponding thresholds and proceed to generate an indication of the value of the one or more oral care metrics, wherein the indication of the value of the one or more of the oral care metrics is any of above, below, or equal to the corresponding threshold. A generated orthodontic setup (e.g., a setup generated by the techniques described herein) may be indicative of the teeth of the patient’s dentition having attained a threshold oral care metric value suitable for initiating dental restorative treatment of the patient. The techniques disclosed herein may receive a plurality of historical cohort patient case data, where case data comprises at least one or more three- dimensional (3D) representations of one or more patients’ dentitions comprising at least one of the gums of the patient or one or more teeth of the one or more patients, wherein at least one of the cases in the plurality required orthodontic treatment prior to the initiation of dental restorative treatment. The techniques may train a machine learning model to output an indication of whether orthodontic treatment is needed prior to dental restorative treatment. The techniques described herein may perform dental restorative treatment by performing operations such as automated restoration design generation. The dental restorative treatment may comprise 3D printing an appliance (e.g., a 3D printed matrix such as the 3M FILTEK Matrix) which may be used to shape dental composite in the patient’s mouth. The techniques described herein may perform orthodontic treatment which may comprise 3D printing one or more aligner trays, 3D printing one or more indirect bonding trays, or 3D printing one or more fixture models. Such orthodontic treatment may in some instances correct at least one of excessive overjet, an end-to-end bite, a crossbite, an over-eruption condition, a crowding condition, excessive overbite or a deep bite. Oral care appliances described herein may, in some instances, be 3D printed.

[0004] Techniques of this disclosure may compute one or more oral care metrics based on the 3D representation of the patient’ s dentition and may compare the one or more oral care metrics with a respective target threshold. The techniques may subsequently predict based on the comparison a requirement for orthodontic treatment prior to a dental restorative treatment of one or more target pre-restoration teeth, and generate, based on the determining, a rendering of one or more indications recommending orthodontic treatment for the patient prior to dental restorative treatment on the one or more target pre-restoration teeth. Techniques described herein may generate one or more oral care appliance designs based at least in part on the one or more indications and fabricate one or more oral care appliances based at least in part on the one or more oral care appliance designs. The one or more oral care metrics may quantify at least one of the shape of one or more teeth of the patient’s dentition or the spatial relationship between two or more teeth of the patient’s dentition. The target thresholds described herein may be computed based, at least in part, on one or more historical oral care metrics computed from patient cases which have undergone orthodontic treatment prior to the initiation of dental restorative treatment on at least one target pre-restoration tooth. The techniques disclosed herein may generate one or more indications recommending orthodontic treatment in near real-time while a patient is in a clinical environment. When the techniques described herein render an indication in the affirmative that orthodontic treatment must be performed prior to a dental restorative treatment, a trained setups prediction model may be automatically used by the one or more processors to generate one or more final setups or one or more intermediate stages. The final setups or intermediate stages generated by the techniques of this disclosure may be used to generate one or more orthodontic appliances. When the techniques described herein render an indication in the affirmative that orthodontic treatment must be performed prior to a dental restorative treatment, a trained restoration design generation model is automatically used by the one or more processors to generate one or more restoration designs. Such generated restoration designs may be used to generate of one or more dental restoration appliances. Such generated restoration designs may be used to generate of one or more of an inlay, an onlay, a bridge, a crown or a veneer. Techniques of this disclosure may perform orthodontic treatment to modify the poses of one or more teeth of the patient.

[0005] Techniques of this disclosure may use a generator network that comprises one or more neural networks initially trained to render one or more indications recommending orthodontic treatment for a patient prior to dental restorative treatment. The techniques may automatically generate based on the output of the generator network one or more indications recommending orthodontic treatment for a patient prior to dental restorative treatment on one or more aspects of the patient’s dentition. The techniques may further train the generator network based on the one or more indications by performing the following operations. The techniques may receive a plurality of historical cohort patient case data, wherein the case data comprises at least a three-dimensional (3D) representation of a patient’s dentition comprising at least one of gums of the patient or one or more teeth of the patient. One or more cases in the plurality may have one or more associated reference indications recommending orthodontic treatment for a patient prior to dental restorative treatment on one or more aspects of the patient’s dentition. The techniques may predict, using the generator network, one or more indications recommending orthodontic treatment for a patient prior to dental restorative treatment on one or more aspects of the patient’s dentition. The techniques may quantify, using the generator network, a difference between an indication predicted by the generator network and a reference indication associated with aspects of the patient’s dentition included in the plurality of patient case data. The techniques may generate a loss value based on the quantified difference. The techniques may modify the generator network based at least in part on the loss value. The ML model may be trained on cohort patient cases which required dental restorative treatment, and such that orthodontic treatment was required prior to that dental restorative treatment. The techniques may generate one or more oral care appliance designs based at least in part on the one or more predicted indications and fabricate one or more oral care appliances based at least in part on the one or more oral care appliance designs. The techniques may receive one or more oral care metrics and provide the oral care metrics as inputs to the generator network. The techniques may output one or more indications recommending orthodontic treatment, such that the one or more indications are generated in near real-time while a patient is in a clinical environment. The techniques may render a predicted indication in the affirmative that orthodontic treatment must be performed prior to a dental restorative treatment, a trained restoration design generation model is automatically used by the one or more processors to generate one or more restoration designs. The techniques may generate one or more dental restoration designs based at least in part on the predicted indication. The techniques may generate one or more of a fixture model, an inlay, an onlay, a bridge, a crown or a veneer based at least in part on the predicted indication. Techniques described herein may generate a latent representation of the inputted 3D representation of the patient’s dentition, for example, using a trained representation generation module. The 3D representation of the patient’s dentition may comprise one or more mesh elements, for which mesh element feature vectors may be computed.

Brief Description of Drawings

[0006] FIG. 1 shows a patient case 100 which benefits from orthodontic treatment before the application of dental restorative treatment.

[0007] FIG. 2 shows a patient case 200 which benefits from orthodontic treatment before the application of dental restorative treatment.

[0008] FIG. 3 shows a patient case 300 which benefits from orthodontic treatment before the application of dental restorative treatment.

[0009] FIG. 4 shows a method of using orthodontic treatment to correct malocclusions and prepare a patient’s dentition for dental restorative treatment.

[0010] FIG. 5 shows a method of using oral care metrics to predict whether orthodontic treatment should precede dental restorative treatment.

[0011] FIG. 6 shows a method of using a trained machine learning model to predict whether orthodontic treatment should precede dental restorative treatment. [0012] FIG. 7 shows a reconstruction autoencoder.

[0005] FIG. 8 A shows a patient’s dentition which contains a target tooth 810 which is to undergo dental restorative treatment, but is not yet sufficiently within the target envelope 805.

[0006] FIG. 8B shows a patient’s dentition which contains a target tooth 825 which has undergone some orthodontic treatment, but is not quite into position yet.

[0007] FIG. 8C shows a patient’s dentition which contains a target tooth 835 which is in position within the target envelope 840.

[0008] FIG. 8D shows a patient’s dentition in which the target tooth 855 has completed dental restorative treatment.

[0009] FIG. 9A shows a patient’s dentition with a maloccluded tooth which must be moved before dental restorative treatment can begin.

[0010] FIG. 9B shows that the malocclusion from FIG. 9A has been resolved.

FIG. 9C shows the patient’s dentition after dental restorative treatment. Detailed Description [0013] Described herein are techniques for digital oral care, including digital orthodontics and digital dentistry (e.g., dental restoration), and combinations of digital orthodontics and digital dentistry. For example, techniques of this disclosure my predict that a first treatment is to be performed (e.g., clear tray aligner, or bracket-based orthodontic treatment) before a second treatment is to be performed (e.g., dental restorative treatment). In some examples, techniques of this disclosure may perform dental restorative treatment of a patient, where one or more pre-restoration teeth have first been made accessible for dental restorative treatment by orthodontic treatment. Orthodontic treatment (e.g., with a succession of aligner trays, or the use of indirect bonding trays & brackets) may be applied to move one or more teeth when it is determined that the one or more pre-restoration teeth are not yet accessible for dental restorative treatment. In response to determining that orthodontic treatment must precede dental restorative treatment, a ML model may be used to automatically generate an orthodontic setup (e.g., intermediate stage or final setup), which may be used in the generation of aligner trays, indirect bonding trays or other oral care appliances. Such an ML model may, in some implementations, also automatically generate IPR cut planes. Further techniques described herein may use oral care metrics or machine learning models to make a prediction that orthodontic treatment must precede dental restorative treatment. The oral care metrics may be compared to target ranges or thresholds, as a part of the prediction. Further techniques of this disclosure may train ML models to predict those target ranges or thresholds. Still further techniques of this disclosure may train ML models to determine whether bracket-based treatment is to be performed (e.g., to approximately or coarsely align the teeth) before CTA-based treatment is performed (e.g., to more-precisely align the teeth). The ML models of this disclosure may be trained on cohort patient case data from historical patient cases. [0014] In some implementations, a final setup may be used to generate, at least in part, one or more intermediate stages. Each stage may be used in the generation of a clear tray aligner (e.g., by 3D printing a fixture model onto which a plastic aligner tray is thermoformed, or by directly 3D printing the aligner tray). Such aligners may incrementally move the patient's teeth from the initial or maloccluded poses to the final poses represented by the final setup.

[0015] According to particular implementations, a patient’s malocclusions can be sufficiently corrected using a first treatment so that a second treatment method (e.g., treatment using a dental restoration appliance, such as FILTEK Matrix) may commence. The combination of first and second treatments results in improved outcomes that cannot be achieved through either first or second treatments alone, and systems and techniques of this disclosure may advantageously predict at which stage during the first treatment should the second treatment commence. For instance, a patient’s maloccluded dentition may include one or more teeth which are misshapen or small, and may also include one or more teeth which are in misalignment, and so a wholistic combination of orthodontic and/or dental restorative treatments (e.g., which may include one or more predictions than a first treatment is clinically advisable before a second treatment, or one or more predictions of the extent or stage of treatment at which the a transition may be made between the first and second treatments) provides a better outcome than either treatment alone. According to one example, a first treatment may generate sufficient space in proximity to a target tooth to allow the second treatment to be most effective.

[0016] In other words, a patient’s maloccluded dentition (the pre-treatment arrangement of teeth) may not allow sufficient space around a target tooth (e.g., a tooth which is to receive composite material as a part of dental restoration treatment) for a dental restoration appliance to be fitted or coupled to those teeth, and so the first treatment method is applied to remove malocclusions to the mesial or distal sides of the target tooth (e.g., so that a FILTEK Matrix can be inserted into the mouth and used to form dental composite). An example of a dental restoration appliance, the FILTEK Matrix, may be generated to tightly fit over one or more teeth which are adjacent to a target tooth, and then be used to form dental composite into a desired tooth shape (e.g., to form a veneer). That dental composite then undergoes curing, for example, using a UV light. Therefore, the functioning of the FILTEK Matrix is facilitated if adjacent maloccluded teeth are moved out of the way. In some implementations, techniques of this disclosure may predict at which stage of treatment the patient’s malocclusions have been sufficiently corrected to enable dental restorative treatment to begin, for example, using oral care metrics and/or trained ML models.

[0017] In such implementations, one or more oral care metrics may be computed for the patient’s dentition (e.g., maloccluded dentition, or one or more intermediate, or final stages of orthodontic treatment). Such oral care metrics may quantify the extent of the patient’s malocclusion and may indicate whether the first treatment is sufficiently complete, so that the second treatment may begin. In some examples, an ML model may be trained to predict one or more threshold values 502 of one or more oral care metrics which are to be attained through the course of a first treatment method (e.g., bracket-based treatment, among others), before a second treatment method (e.g., CTA-based treatment, or dental restorative treatment, among others) is to commence. Stated another way, oral care metrics may enable techniques of this disclosure to predict at which stage of treatment a patient’s dentition has improved sufficiently that the dentition is ready for the first treatment to end, and the second treatment to begin. One particular example includes training the ML model to predict a range of values for an alignment metric (among other oral metrics described herein) which is to be attained through the course of a first treatment method (e.g., CTA-based treatment or bracket-based treatment), although other examples are also possible. Such a model may be trained on historical cohort patient case data which has been annotated with one or more ground truth values (e.g., one or more values indicates thresholds 502 for one or more oral care metrics). ML models include encoder-decoder structure-based models (e.g., models based upon variational autoencoders with optional normalizing flows), denoising diffusion probabilistic models, multi-layer perceptron-based models, among others ML models described herein.

[0018] In some implementations, an ML model to perform automated setups prediction may be trained for use in generating one or more clear tray aligners (or other orthodontic device, such as one or more indirect bonding trays). In some implementations, an ML model to perform automated hardware (e.g., bracket, hook, attachment, or button, etc.) placement on one or more 3D representations of teeth may be trained (e.g., as a part of generating an indirect bonding tray, or other appliance, which may be used to deliver physical hardware to the patient’s teeth).

[0019] In some examples, the first treatment may initially use orthodontic brackets to alleviate crowding in the patient’s dentition (e.g., for a treatment period of one or more months, such as a 6-month period). After the teeth have moved sufficiently that there is no material (or impeding) superimposition of a given tooth over the neighboring teeth (front to back or side to side), then a second treatment, such as CTA treatment, may start. In some instances, CTA treatment may commence in spite of a superimposition between two adjacent teeth, provided that there is also a gap between those teeth, or if there is a gap front-to-back between superimposed teeth.

[0020] Orthodontic treatment may involve moving teeth, extracting teeth or performing other modifications to the patient’s dentition and/or smile and may be used in combination to achieve desired results. For instance, when moving teeth, the orthodontic treatment of a patient may include a combination of treatments by orthodontic brackets and/or orthodontic aligners. In some instances, the patient may first undergo orthodontic treatment using a bracket-based method (e.g., using brackets which are delivered by an indirect bonding tray) and then complete orthodontic treatment using clear tray aligners (CTA). In such instances, the orthodontic brackets may be used to move the teeth into poses (e.g., which may in some instances comprise a course adjustment) which are suitable for additional orthodontic adjustment using CTA (e.g., which may in some instances comprise a more-precise adjustment or a more-predictable outcome than other techniques). Some patients may prefer to switch to CTA-based orthodontic treatment as soon as it is clinically feasible for aesthetic reasons. In other instances, a patient may first undergo orthodontic treatment by CTA and then complete treatment with a bracket-based method.

[0021] A training dataset may contain one or more patient malocclusions (e.g., tooth meshes and/or corresponding transforms), one or more vectors of one or more oral care metrics (e.g., which may be computed for the malocclusion or at a later stage of orthodontic treatment), or one or more ground truth data. Ground truth data may include information regarding the type, characteristics or model version information of brackets or aligners which were used in orthodontic treatment for one or more patients. Ground truth data may include one or more oral care metrics values which correspond to the state of one or more patients’ dentitions in an historical patient dataset at which a transition was made between a first and a second treatment. Such a ground truth oral care metric value may, in some implementations, be compared to a predicted oral care threshold as a part of a loss calculation module (e.g., to train an ML model to predict an oral care metric threshold 502). Ground truth data may include a count of CTA stages that one or more historical patients underwent before beginning dental restorative treatment. Ground truth data may also include information regarding the duration of time (e.g., in days, etc.) for which each respective patient underwent a first treatment (e.g., orthodontic treatment with brackets - such as brackets which were delivered using an indirect bonding tray, etc.), before switching to a second treatment (e.g., orthodontic treatment by clear tray aligners), or vice versa. In some instances, ground truth data may include a range of treatment times during which it would be suitable for the a particular patient in the historical patent dataset to switch from orthodontic treatment using brackets to orthodontic treatment using clear tray aligners (or vice versa) for a particular case. For example, the ground truth may contain labels which indicate that while the particular patient switched from bracketbased orthodontic treatment to CTA-based orthodontic treatment after 3 months, the patient would have been eligible to switch treatments anytime between 2.5 months and 4 months (e.g., the practitioner may specify a range of dates or a span of time). The ground truth data may include one or more oral care metrics which quantify the state of one or more historical patients’ dentition (e.g., at the mal stage, an intermediate stage, or at the final stage of treatment).

[0022] The training data may include orthodontic metrics (as described herein) which are computed, for example, on the patient’s maloccluded teeth, and/or after a period of first treatment (e.g., either bracket-based treatment or CTA-based treatment). The metrics may describe physical relationships between two or more teeth (e.g., orthodontic metrics or some restoration design metrics), or aspects of the shape and/or structure of an individual tooth (e.g., some restoration design metrics). These oral care metrics may provide an ML model with information about the state of the patient’s dentition at one or more time points during treatment, enabling that ML model to predict when a first treatment has sufficiently concluded so that a second treatment may commence (e.g., a CTA-based treatment when the first treatment was bracket-based, or a bracketbased treatment when the first treatment was CTA-based). In some implementations, the ML model may be trained to predict the count of days, weeks or months (or some other unit of time) for which the patient should undergo bracket-based treatment, before switching to CTA-based treatment (or vice-versa). Stated another way, one or more aspects of the training data may be provided to a machine learning model (e.g., a neural network based ML model), which may be trained to generate one or more predictions of a duration of time a first treatment is to be initially performed before then switching to a second treatment. According to particular implementations, based at least in part on one or more of the oral care metrics, the ML model may also be configured to predict which first treatment should be selected and which second treatment should follow. The ML models of this disclosure may be trained, at least in part, through the execution of one or more loss modules, which may quantify the difference between predicted values (e.g., predicted determinations as described herein, etc.) and corresponding ground truth values.

[0023] This disclosure pertains to digital oral care, which encompasses the fields of digital dentistry and digital orthodontics. This disclosure generally describes methods of processing three-dimensional (3D) representations of oral care data. It should be understood, without loss of generality, that there are various types of 3D representations. One type of 3D representation is a 3D geometry. A 3D representation may include, be, or be part of one or more of a 3D polygon mesh, a 3D point cloud (e.g., such as derived from a 3D mesh), a 3D voxelized representation (e.g., a collection of voxels - for sparse processing), or 3D representations which are described by mathematical equations. Although the term “mesh” is used frequently throughout this disclosure, the term should be understood, in some implementations, to be interchangeable with other types of 3D representations. A 3D representation may describe elements of the 3D geometry and/or 3D structure of an object.

[0024] It should be understood, without the loss of generality, that the techniques of this disclosure which apply to final setups are also applicable to intermediate staging in orthodontic treatment, particularly geometric deep learning (GDL) Setups, reinforcement learning (RL) Setups, variational autoencoder (VAE) Setups, Capsule Setups, multilayer perceptron (MLP) Setups, Diffusion Setups, pose transfer (PT) Setups, Similarity Setups, force directed graphs (FDG) Setups, Transformer Setups, Setups Comparison, or Setups Classification. The Metrics Visualization aspects of this disclosure may also be configured to visualize data from both final setups and intermediate stages. MLP Setups, VAE Setups and Capsule Setups each fall within the scope of Autoencoder Setups. Some implementations of MLP Setups may fall within the Scope of Transformer Setups. Representation Setups refers to any of MLP Setups, VAE Setups, Capsule Setups and any other setups prediction machine learning model which uses an autoencoder to create the representation for at least one tooth.

[0025] Each of the setups prediction techniques of this disclosure is applicable to the fabrication of clear tray aligners and indirect bonding trays. The setups predictions techniques may also be applicable to other products that involve final teeth poses, also. A pose may comprise a position (or location) and a rotation (or orientation).

[0026] A 3D mesh is a data structure which may describe the geometry or shape of an object related to oral care, including but not limited to a tooth, a hardware element, or a patient’s gum tissue. A 3D mesh may include one or more mesh elements such as one or more of vertices, edges, faces and combinations thereof. In some implementations, mesh element may include voxels, such as in the context of sparse mesh processing operations. Various spatial and structural features may be computed for these mesh elements and be provided to the predictive models of this disclosure, with the predictive models of this disclosure providing the technical advantage of improving data precision in the form of the models of this disclosure outputting more accurate predictions.

[0027] A patient’s dentition may include one or more 3D representations of the patient’s teeth, gums and/or other oral anatomy. An orthodontic metric (OM) may, in some implementations, quantify the relative positions and/or orientations of at least one 3D representation of a tooth relative to at least one other 3D representation of a tooth. A restoration design metric (RDM) may, in some implementations, quantify the at least one aspect of the structure and/or shape of a 3D representation of a tooth. An orthodontic landmark (OL) may, in some implementations, locate one or more points or other structural regions of interest on a 3D representation of a tooth. An OL may, in some implementations, be used in the creation of an orthodontic or dental appliance, such as a clear tray aligner or a dental restoration appliance. A mesh element may, in some implementations, comprise at least one constituent element of a 3D oral care representation. For example, in the case of a tooth that is represented by a 3D mesh, mesh elements may include at least: vertices, edges, faces and voxels. A mesh element feature may, in some implementations, quantify some aspect of a 3D representation in proximity to or in relation with one or more mesh elements, as described elsewhere in this disclosure. Orthodontic procedure parameters (OPP) may, in some implementations, specify at least one value which defines at least one aspect of planned orthodontic treatment for the patient (e.g., specifying desired target attributes of a final setup in final setups prediction). Orthodontic Doctor preferences (ODP) may, in some implementations, specify at least one typical value for an OPP, which may, in some instances, be derived from past cases which have been treated by one or more oral care practitioners. Restoration Design Parameters (RDP) may, in some implementations, specify at least one value which defines at least one aspect of planned dental restoration treatment for the patient (e.g., specifying desired target attributes of a tooth which is to undergo treatment with a dental restoration appliance). Doctor Restoration Design Preferences (DROP) may, in some implementations, specify at least one typical value for an RDP, which may, in some instances, be derived from past cases which have been treated by one or more oral care practitioners. 3D oral care representations may include, but are not limited to: 1) a set of mesh element labels which may be applied to the 3D mesh elements of teeth/gums/hardware/appliance meshes (or point clouds) in the course of mesh segmentation or mesh cleanup; 2) 3D representation(s) for one or more teeth/gums/hardware/appliances for which shapes have been modified (e.g., trimmed, distorted, or filled-in) in the course of mesh segmentation or mesh cleanup; 3) one or more coordinate systems (e.g., describing one, two, three or more coordinate axes) for a single tooth or a group of teeth (such as a full arch - as with the LDE coordinate system); 4) 3D representation(s) for one or more teeth for which shapes have been modified or otherwise made suitable for use in dental restoration; 5) 3D representation(s) for one or more dental restoration appliance components; 6) one or more transforms to be applied to one or more of: dental restoration appliance library component placement relative to one or more teeth, a tooth to be placed for an orthodontic setup (either final setup or intermediate stage), a hardware element to be placed relative to one or more teeth or the like; 7) an orthodontic setup; 8) a 3D representation of a hardware element (such as facial bracket, lingual bracket, orthodontic attachment, button, hook, bite ramp, etc.) to be placed relative to one or more teeth, etc.; 8) a 3D representation of a bonding pad for a hardware element (which may be generated for a specific tooth by outlining a perimeter on the tooth, specifying a thickness to form a shell, and then subtracting-out the tooth via a Boolean operation); 9) 3D representation of a clear tray aligner (CTA); 10) the location or shape of a CTA trimline (e.g., described as either a mesh or polyline); 11) archform that describes the contours or layout of an arch of teeth (e.g., described as a 3D polyline or as a 3D mesh or surface), which may follow the incisal edges one or more teeth, which may follow the facial surfaces of one or more teeth, which may in some implementations correspond to the maloccluded arch and in other implementations correspond to the final setup arch (the effects of malocclusion on the shape of the archform may be diminished by smoothing or averaging of the shape of the archform), which may be described by one or more control points and/or a spline; 12) 3D representation of a fixture models (e.g., depictions of teeth and gums for use in thermoforming clear tray aligners, or depictions of teeth/gums/hardware for use in thermoforming indirect bonding trays); 13) one or more latent space vectors (or latent capsules) produced by the 3D encoder stage of a 3D autoencoder which has been trained on the reconstruction of oral care meshes (e.g., a variational autoencoder which has been trained for tooth reconstruction); 14) one or more oral care metrics values (e.g., such as orthodontic metrics or restoration design generation metrics) for one or more teeth; 15) one or more landmarks (e.g., 3D points) which describe the shapes and/or geometrical attributes of one or more teeth, other dentition structures or hardware structures (e.g., to be used in orthodontic setups creation or restoration appliance component generation or placement); 16) 3D representation created by scanning (e.g., optically scanning, CT scanning or MRI scanning) a 3D printed part corresponding to one or more teeth/gums/hardware/appliances (e.g., a scanned fixture model); 17) 3D printed aligners (including optionally local thickness, reinforcing rib geometry, flap positioning, or the like) 18) 3D representation of the patient's dentition that was captured chairside by a clinician or medical practitioner (e.g., in a context where the 3D representation is validated chairside, before the patient leaves the clinic, so that errors can be detected and re-scans performed as necessary); 19) dental restoration tooth design (e.g. , for veneers, crowns, bridges or dental restoration appliances); 20) 3D representations of one or more teeth for use in digital oral care treatment; 21) other 3D printed parts pertaining to oral care procedures or other fields; 22) IPR cut surfaces; 23) one or more orthodontic setups transforms associated with one or more IPR cut surfaces; 24) a (digital) pontic tooth design which may fill at least a portion of the space between teeth to allow room in an orthodontic setup for an erupting tooth to later emerge from the gums; or 25) a component of a fixture model (e.g., comprising fixture model components such as interproximal webbing, block-out, bite locks, bite ramps, interproximal reinforcement, gingival ridges, torque points, power ridges, pontic tooth or dimples, among others). In some instances, 3D oral care representations may include a prediction of a patient’s smile, which may show the post-treatment teeth, lips, cheeks, nose, and/or other facial features in relation to each other. In implementations where 3D oral care representations include a prediction of a patient’s smile, the predictions may be presented in several formats, including as 2D images or as 3D representations.

[0028] The techniques of this disclosure may be advantageously combined. For example, the Setups Comparison tool may be used to compare the output of the GDL Setups model against ground truth data, compare the output of the RL Setups model against ground truth data, compare the output of the VAE Setups model against ground truth data and compare the output of the MLP Setups model against ground truth data. With each of these setups prediction models compared against ground truth data, it may be possible to determine which model gives the best performance on a certain dataset or within a given problem domain. Furthermore, the Metrics Visualization tool can enable a global view of the final setups and intermediate stages produced by one or more of the setups prediction models, with the advantage of enabling the selection of the best setups prediction model. The Metrics Visualization tool, furthermore, enables the computation of metrics which have a global scope over a set of intermediate stages. These global metrics may, in some implementations, be consumed as inputs to the neural networks for predicting setups (e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, among others). The global metrics may also be consumed by FDG Setups. The local metrics from this disclosure (i.e., a local metric is a metric which may be computed for one stage or setup of treatment, rather than over several stages or setups) may, in some implementations, be consumed by the neural networks herein for predicting setups, with the advantage of improving predictive results. The metrics described in this disclosure may, in some implementations, be visualized using the Metric Visualization tool.

[0029] The VAE and MAE models for mesh element labelling and mesh in-filling can be advantageously combined with the setups prediction neural networks, for the purpose of mesh cleanup ahead of or during the prediction process. In some implementations, the VAE for mesh element labelling may be used to flag mesh elements for further processing, such as metrics calculation, removal or modification. In some instances, such flagged mesh elements may be used as inputs to a setups prediction neural network, to inform that neural network about important mesh features, attributes or geometries, with the advantage of improving the performance of the resulting setups prediction model. In some implementations, mesh in-filling may cause the geometry of a tooth to become more nearly complete, enabling the better functioning of a setups prediction model (i.e., improved correctness of prediction on account of better-formed geometry). In some instances, a neural network to classify a setup (i.e., the Setups Classifier) may aid in the functioning of a setups prediction neural network, because the setups classifier tells that setups prediction neural network when the predicted setup is acceptable for use and can be outputted. A Setups Classifier (e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups and FDG Setups, among others) may aid in the generation of final setups and also in the generation of intermediate stages. Furthermore, a Setups Classifier neural network may be combined with the Metrics Visualization tool. In other implementations, a Setups Classification neural network may be combined with the Setups Comparison tool (e.g., the Setup Comparison tool may output an indication of how a setup produced in part by the Setups Classifier compares to a setup produced by another setups prediction method). In some implementations, the VAE for mesh element labelling may identify one or more mesh elements for use in a metrics calculation. The resulting metrics outputs may be visualized by the Metrics Visualization tool.

[0030] In some examples, the Setups Classifier neural network may aid in the setups prediction technique described in U.S. Patent Application No. US20210259808A1 (which is incorporated herein by reference in its entirety) or the setups prediction technique described in PCT Application with Publication No. WO2021245480A1 (which is incorporated herein by reference in its entirety) or in PCT Application No. PCT/IB2022/057373. The Setups Classifier would help one or more of those techniques to know when the predicted final setup is most nearly correct. In some instances, the Setups Classifier neural network may output an indication of how far away from final setup a given setup is (i.e., a progress indicator).

[0031] In some implementations, the latent space embedding vector(s) from the reconstruction VAE can be concatenated with the inputs to the setups prediction neural network described in published PCT application No. WO2021245480A1. The latent space vectors can also be incorporated as inputs to the other setups prediction models: GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups and Diffusion Setups, among others. The advantage is to impart the essential tooth mesh characteristics to that neural network, hence improving the generated setups prediction.

[0032] In some examples, the various setups prediction neural networks of this disclosure may work together to produce the setups required for orthodontic treatment. For example, the GDL Setups model may produce a final setup, and the RL Setups model may use that final setup as input to produce a series of intermediate stages setups. Alternatively, the VAE Setups model (or the MLP Setups model) may create a final setup which may be used by a RL Setups model to produce a series of intermediate stages setups. In some implementations, a setup prediction may be produced by one setups prediction neural network, and then taken as input to another setups prediction neural network for further improvements and adjustments to be made. In some implementations, such improvements may be performed in iterative fashion.

[0033] In some implementations, a setups validation model, such as the model disclosed in US Provisional Application No. US63/366495, may be involved in this iterative setups prediction loop. First a setup may be generated (e.g., using a model trained for setups prediction, such as GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups and FDG Setups, among others), then the setup undergoes validation. If the setup passes validation, the setup may be outputted for use. If the setup fails validation, the setup may be sent back to one or more of the setups prediction models for corrections, improvements and/or adjustments. In some instances, the setups validation model may output an indication of what is wrong with the setup, enabling the setups generation model to make an improved version upon the next iteration. The process iterates until done.

[0034] Generally speaking, in some implementations, two or more of the following techniques of the present disclosure may be combined in the course of orthodontic and/or dental treatment: GDL Setups, Setups Classification, Reinforcement Learning (RL) Setups, Setups Comparison, Autoencoder Setups (VAE Setups or Capsule Setups), VAE Mesh Element Labeling, Masked Autoencoder (MAE) Mesh Infilling, Multi-Layer Perceptron (MLP) Setups, Metrics Visualization, Imputation of Missing Oral Care Parameters Values, Tooth Classification Using Latent Vector, FDG Setups, Pose Transfer Setups, Restoration Design Metrics Calculation, Neural Network Techniques for Dental Restoration & Orthodontics (e.g., 3D Oral Care Representation Generation or Modification using Transformers), Landmark-based (LB) Setups, Diffusion Setups, Imputation of Tooth Movement Procedures, Capsule Autoencoder Segmentation, Diffusion Segmentation, Similarity Setups, Validation of Oral Care Representations (e.g., using autoencoders), Coordinate System Prediction, Restoration Design Generation or 3D Oral Care Representation Generation of Modification Using Denoising Diffusion Models.

[0035] A 3D representation may be produced using a 3D scanner, such as an intraoral scanner, a computerized tomography (CT) scanner, ultrasound scanner, a magnetic resonance imaging (MRI) machine or a mobile device which is enabled to perform stereophotogrammetry. A 3D representation may describe the shape and/or structure of a subject. A 3D representation may include one or more 3D mesh, 3D point cloud, and/or a 3D voxelized representation, among others. A 3D mesh includes edges, vertices, or faces. Though interrelated in some instances, these three types of data are distinct. The vertices are the points in 3D space that define the boundaries of the mesh. These points would alternatively be described as a point cloud but for the additional information about how the points are connected to each other, as described by the edges. An edge is described by two points and can also be referred to as a line segment. A face is described by a number of edges and vertices. For instance, in the case of a triangle mesh, a face comprises three vertices, where the vertices are interconnected to form three contiguous edges. Some meshes may contain degenerate elements, such as non-manifold mesh elements, which may be removed, to the benefit of later processing. Other mesh pre-processing operations are possible in accordance with aspects of this disclosure. 3D meshes are commonly formed using triangles, but may in other implementations be formed using quadrilaterals, pentagons, or some other n-sided polygon. In some implementations, a 3D mesh may be converted to one or more voxelized geometries (i.e., comprising voxels), such as in the case that sparse processing is performed. The techniques of this disclosure which operate on 3D meshes may receive as input one or more tooth meshes (e.g., arranged in one or more dental arches). Each of these meshes may undergo pre-processing before being input to the predictive architecture (e.g., including at least one of an encoder, decoder, pyramid encoder-decoder and U-Net). This pre-processing may include the conversion of the mesh into lists of mesh elements, such as vertices, edges, faces or in the case of sparse processing - voxels. For the chosen mesh element type or types, (e.g., vertices), feature vectors may be generated. In some examples, one feature vector is generated per vertex of the mesh. Each feature vector may contain a combination of spatial and/or structural features, as specified in the following table:

Table 1

[0036] Table 1 discloses non-limiting examples of mesh element features. In some implementations, color (or other visual cues/identifiers) may be considered as a mesh element feature in addition to the spatial or structural mesh element features described in Table 1. As used herein (e.g., in Table 1), a point differs from a vertex in that a point is part of a 3D point cloud, whereas a vertex is part of a 3D mesh and may have incident faces or edges. In some implementations, color may be considered as a mesh element feature in addition to the spatial or structural mesh element features described in Table 1. Consistent with Table 1, a voxel may also have features which are computed as the aggregates of the other mesh elements (e.g., vertices, edges and faces) which either intersect the voxel or, in some implementations, are predominantly or fully contained within the voxel. Rotating the mesh may not change structural features but may change spatial features. And, as described elsewhere in this disclosure, the term “mesh” should be considered in a non-limiting sense to be inclusive of 3D mesh, 3D point cloud and 3D voxelized representation. In some implementations, apart from mesh element features, there are alternative methods of describing the geometry of a mesh, such as 3D keypoints and 3D descriptors. Examples of such 3D keypoints and 3D descriptors are found in TONIONI A, et al. in Learning to detect good 3D keypoints, International Journal of Computer Vision, 2018 Vol. 126, at 1-20. 3D keypoints and 3D descriptors may, in some implementations, describe extrema (either minima or maxima) of the surface of a 3D representation. In some implementations, one or more mesh element features may be computed, at least in part, via deep feature synthesis (DFS), e.g. as described in: J. M. Kanter and K. Veeramachaneni, Deep feature synthesis: Towards automating data science endeavors, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 2015, at 1-10, doi: 10. 1109/DSAA.2015.7344858.

[0037] Predictive models which may operate on feature vectors of the aforementioned features include but are not limited to: GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, Tooth Classification, Setups Classification, Setups Comparison, VAE Mesh Element Labeling, MAE Mesh In-filling, Mesh Reconstruction Autoencoder, Validation Using Autoencoders, Mesh Segmentation, Coordinate System Prediction, Mesh Cleanup, Restoration Design Generation, Appliance Component Generation & Placement, and Archform Prediction. Such feature vectors may be presented to the input of a predictive model. In some implementations, such feature vectors may be presented to one or more internal layers of a neural network which is part of one or more of those predictive models.

[0038] Examples of oral care metrics include Orthodontic Metrics (OM) and Restoration Design Metrics (RDM). Techniques of this disclosure may train machine learning models (e.g., using cohort patient case data) to generate oral care metrics. Such ML models may generate representations of one or more of the patient’s teeth using representation generation modules. Such representations (e.g., latent representations) may be inputted to an ML model which may be trained to leam oral care metric values from the one or more teeth. Such generated oral care metrics may be outputted for use by other techniques of this disclosure. [0039] RDM may describe the shape and/or form of one or more 3D representations of teeth for use in dental restoration. One use case example is in the creation of one or more dental restoration appliances. Another use case example is in the creation of one or more veneers (such as a zirconia veneer). Some RDM may quantify the shape and/or other characteristics of a tooth. Other RDM may quantify relationships (e.g., spatial relationships) between two or more teeth. RDM differ from restoration design parameters (RDP) in that restoration design metrics define a current state of a patient's dentition, whereas restoration design parameters serve as specifications to a machine learning or other optimization model to generate desired tooth shapes and/or forms. RDM describe the shapes of the teeth currently (e.g., in a starting or mal condition). Restoration design parameters specify how an oral care provider (such as a dentist or dental technician) intends for the teeth to look after the completion of restoration treatment. Either or both of RDM and RDP may be inputted a neural network or other machine learning or optimization algorithm for the purpose of dental restoration. In some implementations, RDM may be computed on the pre-restoration dentition of the patient (i.e., the primary implementation). In other implementations, RDM may be computed on the post-restoration dentition of the patient. A restoration design may comprise one or more teeth and may be referred to as a restoration arch. Restoration design generation may involve the generation of an improved geometry and/or structure of one or more teeth in a restoration arch.

[0040] Aspects of RDM calculation are described below. In some implementations, RDM may be measured, for example, through locating landmarks in the teeth (or gums, hardware and/or other elements of the patient's dentition), and the measurements of distances between those landmarks, or otherwise made in relation to those landmarks. In some implementations, one or more neural networks or other machine learning models may be trained to identify or extract one or more RDM from one or more 3D representations of teeth (or gums, hardware and/or other elements of the patient's dentition). Techniques of this disclosure may use RDM in various ways. For instance, in some implementations, one or more neural networks or other machine learning models may be trained to classify or label one or more setups, arches, dentitions or other sets of teeth based at least in part on RDM. As such, in these examples, ROMs form a part of the training data used for training these models.

[0041] Aspects of a tooth mesh reconstruction autoencoder (e.g., a variational autoencoder optionally utilizing normalizing flows) may be used in accordance with techniques of this disclosure are described below. An autoencoder for restoration design generation is disclosed in US Provisional Application No. US63/366514. This autoencoder (e.g., a variational autoencoder or VAE) takes as input a tooth mesh (or other 3D representation) that reflects a mal state (i.e., the pre-restoration tooth shape). The encoder component of the autoencoder converts that tooth mesh to a latent form (e.g., a latent vector). Modifications may be applied to this latent vector (e.g., based on a mapping of the latent space through prior experiments), for the purpose of altering the geometry and/or structure of the eventual reconstructed mesh. Additional vectors may, in some implementations, be included with the latent vector (e.g., through concatenation), and the resulting concatenation of vectors may be reconstructed by way of the decoder component of the autoencoder into a reconstructed tooth mesh which is a facsimile of the input tooth mesh.

[0042] RDM and RDP may also be used as neural network inputs in the execution phase, in accordance with aspects of this disclosure. In some implementations, one or more RDM may be concatenated with the input to the encoder, for the purpose of telling the encoder specific information about the input 3D tooth representation. In some implementations, one or more RDM may be concatenated with the latent vector, before reconstruction, for the purpose of providing the decoder component with specific information about the input 3D tooth representation. Furthermore, in some implementations, one or more restoration design parameters (RDP) may be concatenated with the input to the encoder component, for the purpose of providing the encoder specific information about the input 3D tooth representation. Likewise, in some implementations, one or more restoration design parameters (RDP) may be concatenated with the latent vector, before reconstruction, for the purpose of providing the decoder specific information about the input 3D tooth representation.

[0043] In this way, either or both of RDM and RDP may be introduced to the functioning of an autoencoder (e.g., a tooth reconstruction autoencoder), and serve to influence the geometry and/or structure of the reconstructed restoration design (i.e., influence the shape of the tooth on the output of the autoencoder). In some implementations, the variational autoencoder of US Provisional Application No. US63/366514 may be replaced by a capsule autoencoder (e.g., instead of converting the tooth mesh to a latent vector, the tooth mesh is converted to one or more latent capsules).

[0044] The following RDM may be measured and used in the creation of either or both of dental restoration appliances and veneers {veneers are a type of dental restoration appliance}, with the objective of making the resulting teeth natural looking. Symmetry is generally a preferred facet. There may be differences between patients based on demographic differences. The generation of dental restoration appliances may benefit from some or all of the following RDM. Shade and translucency may pertain, in particular, to the creation of veneers, though some implementations of dental restoration appliances may also consider this information.

[0045] Examples of inter-tooth RDM are enumerated below:

[0046] 1) Bilateral Symmetry & Ratios: A measure of the symmetry between one or more teeth and one or more other teeth on opposite sides of the dental. For example, for a pair of corresponding teeth, a measure of the width of each tooth. In one instance, the one tooth is of normal width, and the other tooth is too narrow. In another instance, both teeth are of normal width. The following is a list of attributes that can be measured for a tooth, and compared to the corresponding measurement for one or more corresponding teeth: a) width - mesial to distal distance; b) length - gingival to incisal distance; c) diagonal - distance across the tooth, e.g., from the mesial gingival comer to the distal incisal comer (this measure is one of many that can be used to quantify the shape of teeth beyond length and width). Ratios between a and b may be computed, such as a/b or b/a. Such ratios can be indicative of whether spatial symmetry exists (e.g., by measuring the ratio a/b on the left side and measuring the ratio a/b on the right side, then compare the left and right ratios). In some implementations, where spatial symmetry is "off¹, the length, width and/or ratios may not match. Such a ratio may, in some implementations, be computed relative to a standard. A number of esthetic standards are available in the dental literature. Examples include Golden Proportion and Recurring Esthetic Dental Proportion. In some implementations, spatial symmetry may be measured on a pair of teeth, where one tooth is on the right side of the arch, and the other tooth is on the left side of the arch.

[0047] 2) Proportions of Adjacent Teeth: Measure the width proportions of adjacent teeth as measured as a projection along an arch onto a plane (e.g., a plane that is situated in front of the patient's face). The ideal proportions for use in the final restoration design can be, for example, the so-called golden proportions. The golden proportions relate adjacent teeth, such as central incisors and lateral incisors. This metric pertains to the measuring of these proportions as the proportions exist in the pre-restoration mal dentition. The ideal golden proportions are 1.6, 1, 0.6, for the central incisor, lateral incisor and cuspid, on a particular side (either left or right) for a particular arch (e.g., the upper arch). If one or more of these proportion values is off (e.g., in the case of "peg laterals"), the patient may wish for dental restoration treatment to correct the proportions.

[0048] 3) Arch Discrepancies: A measure of any size discrepancies between the upper arch and lower arch, for example, pertaining to the widths of the teeth, for the purpose of dental restoration. For example, techniques of this disclosure may make adjacent tooth width proportion measurements in the upper arch and in the lower arch. In some implementations, Bolton analysis measurements may be made by measuring upper widths, lower widths, and proportions between those quantities. Arch discrepancies may be described in absolute measurements (e.g., in mm or other suitable units) or in terms of proportions or ratios, in various implementations.

[0049] 4) Midline: A measure of the midline of the maxillary incisors, relative to the midline of the mandibular incisors. Techniques of this disclosure may measure the midline of the maxillary incisors, relative to the midline of the nose (if data about nose location is available).

[0050] 5) Proximal Contacts: A measure of the size (area, volume, circumference, etc.) of the proximal contact between adjacent teeth. In the ideal circumstance, the teeth touch along the mesial/distal surfaces and the gums fill in gingivally to where the teeth touch. Black triangles may form if the gum tissue fails to fill the space below the proximal contact. In some instances, the size of the proximal contact may get progressively shorter for teeth located farther towards the posterior of the arch. In an ideal scenario, the proximal contact would be long enough so that there is an appropriately sized incisal embrasure and the gum tissue fills in the area below or gingival to the contact.

[0051] 6) Embrasure: In some implementations, techniques of this disclosure may measure the size

(area, volume, circumference, etc.) of an embrasure, the gap between teeth at either of the gingival or incisal edge. In some implementations, techniques of this disclosure may measure the symmetry between embrasures on opposite sides of the arch. An embrasure is based at least in part on the length of the length of the contact between teeth, and/or at least in part on the shape of the tooth. In some instances, the size of the embrasure may get progressively longer for teeth located farther towards the posterior of the arch.

[0052] Examples of Intra-tooth RDM are enumerated below, continuing with the numbering of other RDM listed above.

[0053] 7) Length & Width: A measure of the length of a tooth relative to the width of that tooth. This metric may reveal, for example, that a patient has long central incisors. Width and length are defined as: a) width - mesial to distal distance; b) length - gingival to incisal distance; c) other dimensions of tooth body - the portions of tooth between the gingival region and the incisal edge. In some implementations, either or both of a length and a width may be measured for a tooth and compared to the length and/or width of one or more teeth.

[0054] 8) Tooth Morphology: A measure of the primary anatomy of the tooth shape, such as line angles, buccal contours, and/or incisal angles & embrasures. The frequency and/or dimensions may be measured. In some implementations, the observed primary tooth shape aspects may be matched to one or more known styles. Techniques of this disclosure may measure secondary anatomy of the tooth shape, such as mamelon grooves. For instance, the frequency and/or dimensions may be measured. In some implementations, the observed secondary tooth shape aspects may be matched to one or more known styles. In some examples, techniques of this disclosure may measure tertiary anatomy of the tooth shape, such as perikymata or striations. For instance, the frequency and/or dimensions may be measured. In some implementations, the observed tertiary tooth shape aspects may be matched to one or more known styles.

[0055] 9) Shade & Translucency: A measure of tooth shade and/or translucency. Tooth shade is often described by the Vita Classical or 3D Master shade guide. Tooth translucency is described by transmittance or a contrast ratio. Tooth shade and translucency may be evaluated (or measured) based on one or more of the following kinds of data pertaining to teeth: the incisal edge, incisal third, body and gingival third. The enamel layer translucency is general higher than the dentin or cementum layer. Shade and translucency may, in some implementations, be measured on a per-voxel (local) basis. Shade and translucency may, in some implementations, be measured on a per-area basis, such as an incisal area, tooth body area, etc. Tooth body may pertain to the portions of the tooth between the gingival region and the incisal edge. [0056] 10) Height of Contour: A measure of the contour of a tooth. When viewed from the proximal view, all teeth have a specific contour or shape, moving from the gingival aspect to the incisal. This is referred to as the facial contour of the tooth. In each tooth, there is a height of contour, where that shape is the most pronounced. This height of contour changes from the teeth in the anterior of the arch to the teeth in the posterior of the arch. In some implementations, this measurement may take the form of fitting against a template of known dimensions and/or known proportions. In some implementations, this measurement may quantify a degree of curvature along the facial tooth surface. In some implementations, measure the location along the contour of the tooth where the height of the curvature is most pronounced. This location may be measured as a distance away from the gingival margin or a distance away from the incisal edge, or a percentage along the length of the tooth.

[0057] PCT Application with Publication No. W02020026117A1 is incorporated herein by reference in its entirety. W02020026117A1 lists some examples of Orthodontic Metrics (OM). Further examples are disclosed herein. The orthodontic metrics may be used to quantify the physical arrangement of an arch of teeth for the purpose of orthodontic treatment (as opposed to restoration design metrics - which pertain to dentistry and describe the shape and/or form of one or more pre-restoration teeth, for the purpose of supporting dental restoration). These orthodontic metrics can measure how badly maloccluded the arch is, or conversely the metrics can measure how correctly arranged the teeth are. In some implementations, the GDL Setups model (or RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups and FDG Setups) may incorporate one or more of these orthodontic metrics, or other similar or related orthodontic metrics. In some implementations, such orthodontic metrics may be incorporated into the feature vector for a mesh element, where these per-element feature vectors are fed into the setups prediction network as inputs. In some implementations, such orthodontic metrics may be directly consumed by a generator, an MLP, a transformer, or other neural network as direct inputs (such as presented in one or more input vectors of real numbers S, such as described elsewhere in this disclosure. The use of such orthodontic metrics in the training of the generator may improve the performance (i.e., correctness) of the resulting generator, resulting in predicted transforms which place teeth more nearly in the correct final setups poses than would otherwise be possible. Such orthodontic metrics may be consumed by an encoder structure or by a U-Net structure (in the case of GDL Setups). Such orthodontic metrics may be consumed by an autoencoder, variational autoencoder, masked autoencoder or regularized autoencoder (in the case of the VAE Setups, VAE Mesh Element Labelling, MAE Mesh In-Filling). Such orthodontic metrics may be consumed by a neural network which generates action predictions as a part of a reinforcement learning RL Setups model. Such orthodontic metrics may be consumed by a classifier which applies a label to a setup arch (e.g., labels such as mal, staging or final setup). This description is non- limiting, as the orthodontic metrics may also be incorporated in other ways into the various techniques of this disclosure.

[0058] The various loss calculations of the present disclosure may, in some examples, incorporate one or more orthodontic metrics, with the advantage of improving the correctness of the resulting neural network. An orthodontic metric may be used to directly compare a predicted example to the corresponding ground truth example (such as is done with the metrics in the Setups Comparison description). In other examples, one or more orthodontic metrics may be taken from this section and incorporated into a loss computation. Such an orthodontic metric may be computed on the predicted example, and then the orthodontic metric would also be computed on the ground truth example. These two orthodontic metrics results would then be consumed by the loss computation, with the advantage of improving the performance of the resulting neural network. In some implementations, one or more orthodontic metrics pertaining to the alignment of two or more adjacent teeth may be computed and incorporated into a loss function, for example, to train, at least in part, a setups prediction neural network. In some implementations, such an orthodontic metric may reward the network for aligning the mesial surface of one tooth with distal surface of adjacent tooth. Backpropagation is an exemplary algorithm by which a neural network may be trained using one or more loss values.

[0059] In some implementations, one or more orthodontic metrics may be used to evaluate the predicted output of a neural network, such as a setups prediction. Such a mefric(s) may enable the training algorithm to determine how close the predicted output is to an acceptable output, for example, in a quantified sense. In some implementations, this use of an orthodontic metric may enable a loss value to be computed which does not depend entirely on a comparison to a ground truth. In some implementations, such a use of an orthodontic metric may enable loss calculation and network training to proceed without the need for a comparison against a ground truth example. The advantage of such an approach is that loss may be computed based on a general principle or specification for the predicted output (such as a setup) rather than tying loss calculation to a specific ground truth example (which may have been defined by a particular doctor, clinician, or technician, whose treatment philosophy may differ from that of other technicians or doctors). In some implementations, such an orthodontic metric may be defined based on a FID (Frechet Inception Distance) score.

[0060] The following is a description of some of the orthodontic metrics which are used to quantify the state of a set of teeth in an arch for the purpose of orthodontic treatment. These orthodontic metrics indicate the degree of malocclusion that the teeth are in at a given stage of clear tray aligner treatment.

[0061] An orthodontic metric that can be computed using tensors may be especially advantageous when training one of the neural networks of the present disclosure, since tensor operations may promote efficient computations. The more efficient (and faster) the computation, the faster the rate at which training can proceed.

[0062] In some examples, an error pattern may be identified in one or more predicted outputs of an ML model (e.g., a transformation matrix for a predicted tooth setup, a labelling of mesh elements for mesh cleanup, an addition of mesh elements to a mesh for the purpose of mesh in-filling, a classification label for a setup, a classification label for a tooth mesh, etc.). One or more orthodontic metrics may be selected to become an input to the next round of ML model training, to address any pattern of errors or deficiencies which may be identified in the one or more predicted outputs.

[0063] Some OM may be defined relative to an archfrom coordinate frame, the LDE coordinate system. In some implementations, a point may be described using an LDE coordinate frame relative to an archform, where L, D and E correspond to: 1) Length along the curve of the archform, 2) Distance away from the archform, and 3) distance in the direction perpendicular to the L and D axes (which may be termed Eminence), respectively.

[0064] Various of the OM and other techniques of the present disclosure may compute collisions between 3D representations (e.g., of oral care objects, such as teeth). Such collisions may be computed as at least one of: 1) penetration distance between 3D tooth representations, 2) count of overlapping mesh elements between 3D tooth representations, and 3) volume of overlap between 3D tooth representations. In some implementations, an OM may be defined to quantify the collision of two or more 3D representations of oral care structures, such as teeth. Some optimization algorithms, such as setups prediction techniques, may seek to minimize collisions between oral care structures (such as teeth).

[0065] Between-arch orthodontic metrics:

[0066] Six (6) metrics for the comparison of two or more arches are listed below. Other suitable comparison orthodontic metrics are found elsewhere in this disclosure, such as in the section for the Setups Comparison technique.

1. Rotation geodesic distance (rotation between predicted example and ground truth setup example)

2. Translation distance (gap between predicted example and ground truth setup example)

3. Normalized translation distance

4. 3D alignment error that measures the distance between predicted mesh elements and ground truth mesh elements, in units of mm.

5. Normalized 3D alignment

6. Percent overlap (% overlap) by volume (alternatively % overlap by mesh elements) of predicted example and corresponding ground truth example

[0067] Within-arch orthodontic metrics: [0068] Alignment - A 3D tooth orientation vector may be calculated using the tooth's mesial-distal axis. A 3D vector, which may be tangent vector to the archform at the position of the tooth may also be calculated. The XY components (i.e., which may be 2D vectors) may then be used to compare the orientation of the archform at the tooth's location to the tooth's orientation in XY space. Cosine similarity may be used to calculate the 2D orientation difference (angle) between the archform tangent and the tooth's mesial-distal axis.

[0069] Arch Symmetry - For each left-right pair of teeth (e.g., lower left lateral incisor & lower right lateral incisor) the absolute difference may be calculated between each tooth’s X-coordinate and the global coordinate reference frame’s X-axis. This delta may indicate the arch asymmetry for a given tooth pair. The result of such a calculation may be the mean X-axis delta of one or more tooth-pairs from the arch. This calculation may, in some implementations, be performed relative to the Y-axis with y-coordinates (and/or relative to the Z axis with Z-coordinates).

[0070] Archform D-axis Differences - May compute the D dimension difference (i.e., the positional difference in the facial-lingual direction) between two arch states, for one or more teeth. May, in some implementations, return a dictionary of the D-direction tooth movement for each tooth, with tooth UNS number as the key. May use the LDE coordinate system relative to an archform.

[0071] Archform (Lower) Length Ratio - May compute the ratio between the current lower arch length and the arch length as it was in the original maloccluded lower arch.

[0072] Archform (Upper) Length Ratio - May compute the ratio between the current upper arch length and the arch length as it was in the original maloccluded upper arch.

[0073] Archform Parallelism (Full arch) - For at least one local tooth coordinate system origin in the upper arch, the one or more nearest origins (e.g., tooth local coordinate system origins) in the lower arch. In some implementations, the two nearest origins may be used. May compute the straight line distance from the upper arch point to the line formed between the origins of the two teeth in the opposing (lower) arch. May return the standard deviation of the set of “point-to-line" distances mentioned above, where the set may be composed of the point-to-line distances for each tooth in the arch.

[0074] Archform Parallelism (Individual tooth) - This metric may share some computational elements with the archform parallclism global orthodontic metric, except that this metric may input the mean distance from a tooth origin to the line formed by the neighboring teeth in opposing arches (e.g., a tooth in the upper arch and the corresponding tooth in the lower arch). The mean distance may be computed for one or more such pairs of teeth. In some implementations, this may be computed for all pairs of teeth. Then the mean distance may be subtracted from the distance that is computed for each tooth pair. This OM may yield the deviation of a tooth from a “typical” tooth parallelism in the arch. [0075] Buccolingual Inclination - For at least one molar or premolar, find the corresponding tooth on the opposite side of the same arch (i.e., for a tooth on the left side of the arch, find the same type of tooth on the right side and vice versa). This OM may compute an n-element list for each tooth (e.g. n may equal 2). This list may contain at least the tooth IDs of the teeth in each pair of teeth (e.g., LeftLowerFirstMolar and RightLowerFirstMolar in a list = [left_tooth_idx_l, right_tooth_idx_2]). Such an n-element vector may be computed for each molar and each premolar in the upper and lower arches. The buccal cusps maybe identified on the molars and premolars on each of the left and right sides of the arch. Draw a line between the buccal cusps of the left tooth and the buccal cusps on the right tooth. Make a plane using this line and the z-axis of the arch. The lingual cusps may be projected onto the plane (i.e., at this point the angle of inclination may be determined). By performing an additional projection, the approximate vertical distance between the lingual cusps and the buccal cusps may be computed. This distance may be used as the buccolingual inclination OM.

[0076] Canine Overbite - The upper and lower canines may be identified. The first premolar for the given side of the mouth may be identified. On a given side of the arch, a distance may be computed between the upper canine and the lower canine, and also between the upper pre-molar and the lower pre-molar. The average (or median, or mode or some other statistic) may be computed for the measured distances. The z- component of this result indicates the degree of overbite. Overbite may be computed between any tooth in one arch and the corresponding tooth in the other arch.

[0077] Canine Overjet Contact - May calculate the collisions (e.g., collision distances) between pairs of canines on opposing arches.

[0078] Canine Overjet Contact KDE - May take an orthodontic metric score for the current patient case as input, and may convert that score into to a log-likelihood using a previously trained kernel density estimation (KDE) model or distribution. This operation may yield information about where in the distribution of "typical" values this patient case lies.

[0079] Canine Overjet - This OM may share some computational steps with the canine overbite OM. In some implementations, average distances may be computed. In some implementations, the distance calculation may compute the Euclidean distance of the XY components of a tooth in the upper arch and a tooth in the lower arch, to yield overjet (i.e., as opposed to computing the difference in Z-components, as may be performed for canine overbite). Oveijet may be computed between any tooth in one arch and the corresponding tooth in the other arch.

[0080] Canine Class Relationship (also applies to first, second and third molars) - This OM may, in some implementations comprise two functions (e.g., written in Python). get_canine_landmarks(): Get landmarks for each tooth which may be used to compute the class relationship, and then, in some implementations, map those landmarks onto the global coordinate space so that measurements may be made between teeth. class_relationship_score_by_side(): May compute the average position of at least one landmark on at least one tooth in the lower arch, and may compute the same for the upper arch. Then may compute the vector from the upper arch landmark position to the lower arch landmark position, and finally projects this vector onto the lower arch to yield a quantification (e.g., as a scalar) of the amount of delta in “arch 1-axis" position there is. This OM may compute how far forward or behind the tooth is positioned on the 1-axis relative to the tooth or teeth of interest in the opposing arch.

[0081] Crossbite - Fossa in at least one upper molar may be located by finding the halfway point between distal and mesial marginal ridge saddles of the tooth. A lower molar cusp may lie between the marginal ridges of the corresponding upper molar. This OM may compute a vector from the upper molar fossa midpoint to the lower molar cusp. This vector may be projected onto the d-axis of the archform, yielding a lateral measure of distance from the cusp to the fossa. This distance may define the crossbite magnitude.

[0082] Edge Alignment - This OM may identify the leftmost and rightmost edges of a tooth, and may identify the same for that tooth’s neighbor.

The OM may then draw a vector from the leftmost edge of the tooth to the leftmost edge of the tooth’s neighbor.

The OM may then draw a vector from the rightmost edge of the tooth to the rightmost edge of the tooth’s neighbor.

The OM may then calculates the linear fit error between the two vectors.

Such a calculation may involve making two vectors:

Vec tooth = right_tooths_leftside to left_tooths_leftside

Vec_neighbor = right_tooths_rightside to left_tooths_leftside

And then may involve computing the dot-product of these two vectors and subtracting the result from 1. (i.e., EdgeAlignment score = 1 - abs(dot(Vec_tooth, Vec_neighbor)) ).

A score of 0 may indicate perfect alignment. A score of 1 may mean perpendicular alignment.

[0083] Incisor Interarch Contact KDE - May identify the deviation of the IncisorlnterarchContact from the mean of a modeled distribution of such statistics across a dataset of one or more other patient cases.

[0084] Leveling - May compute a measure of leveling between a tooth and its neighbor. This OM may calculate the difference in height between two or more neighboring teeth. For molars, this OM may use the midpoint between the mesial and distal saddle ridges as the height of the molar. For non-molar teeth, this OM may use the length of the crown from gums to tip. In some implementations, the tip may be the origin of the local coordinate space of the tooth. Other implementations may place the origin in other locations. A simple subtraction between the heights of neighboring teeth may yield the leveling delta between the teeth (e.g., by comparing Z components).

[0085] Midline - May compute the position of the midline for the upper incisors and/or the lower incisors, and then may compute the distance between them.

[0086] Molar Interarch Contact KDE - May compute a molar interarch contact score (i.e., a collision depth or other type of collision), and then may identify where that score lies in a pre-defined KDE (distribution) built from representative cases.

[0087] Occlusal Contacts - For a particular tooth from the arch, this OM may identify one or more landmarks (e.g., mesial cusp, or central cusp, etc.). Get the tooth transform for that tooth. For each cusp on the current tooth, the cusp may be scored according to how well the cusp contacts the neighboring (corresponding) tooth in the opposite arch. A vector may be found from the cusp of the tooth in question to the vertical intersection point in the corresponding tooth of the opposing arch. The distance and/or direction (i.e., up or down) to the opposing arch may be computed. A list may be returned that contains the resulting signed distances, one for each cusp on the tooth in question.

[0088] Overbite - The upper and lower central incisors may be compared along the z-axis. The difference along the z-axis may be used as the overbite score.

[0089] Overjet - The upper and lower central incisors may be compared along the y-axis. The difference along the y-axis may be used as the overjet score.

[0090] Molar Interarch Contact - May calculate the contact score between molars, and may use collision measurement(s) (such as collision depth).

[0091] Root Movement d - The tooth transforms for an initial state and a next state may be received. The archform axes at a point L along the archform may be computed. This OM may return a distance moved along the d-axis. This may be accomplished by projecting the root pivot point onto the d-axis.

[0092] Root Movement 1 - The tooth transforms for an initial state and a next state may be received. The archform axes at a point L along the archform may be computed. This OM may return a distance moved along the 1-axis. This may be accomplished by projecting the root pivot point onto the 1-axis.

[0093] Spacing - May compute the spacing between each tooth and its neighbor. The transforms and meshes for the arch may be received. The left and right edges of each tooth mesh may be computed. One or more points of interest may be transformed from local coordinates into the global arch coordinate frame. The spacing may be computed in a plane (e.g., the XY plane) between each tooth and its neighbor to the "left". May return an array of one or more Euclidean distances (e.g., such as in the XY plane) which may represent the spacing between each tooth and its neighbor to the left. [0094] Torque - May compute torque (i.e., rotation around and axis, such as the x-axis). For one or more teeth, one or more rotations may be converted from Euler angles into one or more rotation matrices. A component (such as a x-component) of the rotations may be extracted and converted back into Euler angles. This x-component may be interpreted as the torque for a tooth. A list maybe returned which contains the torque for one or more teeth, and may be indexed by the UNS number of the tooth.

[0095] Because generator networks of this disclosure can be implemented as one or more neural networks, the generator may contain an activation function. When executed, an activation function outputs a determination of whether or not a neuron in a neural network will fire (e.g., send output to the next layer). Some activation functions may include: binary step functions, or linear activation functions. Other activation functions impart non-linear behavior to the network, including: sigmoid/logistic activation functions, Tanh (hyperbolic tangent) functions, rectified linear units (ReLU), leaky ReLU functions, parametric ReLU functions, exponential linear units (ELU), softmax function, swish function, Gaussian error linear unit (GELU), or scaled exponential linear unit (SELU). A linear activation function may be well suited to some regression applications (among other applications), in an output layer. A sigmoid/logistic activation function may be well suited to some binary classification applications (among other applications), in an output layer. A softmax activation function may be well suited to some multiclass classification applications (among other applications), in an output layer. A sigmoid activation function may be well suited to some multilabel classification applications (among other applications), in an output layer. A ReLU activation function may be well suited in some convolutional neural network (CNN) applications (among other applications), in a hidden layer. A Tanh and/or sigmoid activation function may be well suited in some recurrent neural network (RNN) applications (among other applications), for example, in a hidden layer. There are multiple optimization algorithms which can be used in the training of the neural networks of this disclosure (such as in updating the neural network weights), including gradient descent (which determines a training gradient using first-order derivatives and is commonly used in the training of neural networks), Newton's method (which may make use of second derivatives in loss calculation to find better training directions than gradient descent, but may require calculations involving Hessian matrices), and conjugate gradient methods (which may yield faster convergence than gradient descent, but do not require the Hessian matrix calculations which may be required by Newton's method). In some implementations, additional methods may be employed to update weights, in addition to or in place of the techniques described above. These additional methods include the Levenberg-Marquardt method and/or simulated annealing. The backpropagation algorithm is used to transfer the results of loss calculation back into the network so that network weights can be adjusted, and learning can progress.

[0096] Neural networks contribute to the functioning of many of the applications of the present disclosure, including but not limited to: GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, Tooth Classification, Setups Classification, Setups Comparison, VAE Mesh Element Labeling, MAE Mesh In-filling, Mesh Reconstruction Autoencoder, Validation Using Autoencoders, imputation of oral care parameters, 3D mesh segmentation (3D representation segmentation), Coordinate System Prediction, Mesh Cleanup, Restoration Design Generation, Appliance Component Generation & Placement, or Archform Prediction. The neural networks of the present disclosure may embody part or all of a variety of different neural network models. Examples include the U-Net architecture, multi-later perceptron (MLP), transformer, pyramid architecture, recurrent neural network (RNN), autoencoder, variational autoencoder, regularized autoencoder, conditional autoencoder, capsule network, capsule autoencoder, stacked capsule autoencoder, denoising autoencoder, sparse autoencoder, conditional autoencoder, long/short term memory (LSTM), gated recurrent unit (GRU), deep belief network (DBN), deep convolutional network (DCN), deep convolutional inverse graphics network (DCIGN), liquid state machine (LSM), extreme learning machine (ELM), echo state network (ESN), deep residual network (DRN), Kohonen network (KN), neural Turing machine (NTM), or generative adversarial network (GAN). In some implementations, an encoder structure or a decoder structure may be used. Each of these models provides one or more of its own particular advantages. For example, a particular neural networks architecture may be especially well suited to a particular ML technique. For example, autoencoders are particularly suited to the classification of 3D oral care representations, due to the ability to convert the 3D oral care representation into a form which is more easily classifiable.

[0097] In some implementations, the neural networks of this disclosure can be adapted to operate on 3D point cloud data (alternatively on 3D meshes or 3D voxelized representation). Numerous neural network implementations may be applied to the processing of 3D representations and may be applied to training predictive and/or generative models for oral care applications, including: PointNet, PointNet++, SO-Net, spherical convolutions, Monte Carlo convolutions and dynamic graph networks, PointCNN, ResNet, MeshNet, DGCNN, VoxNet, 3D-ShapeNets, Kd-Net, Point GCN, Grid-GCN, KCNet, PD-Flow, PU-Flow, MeshCNN and DSG-Net. Oral care applications include, but are not limited to: setups prediction (e.g., using VAE, RL, MLP, GDL, Capsule, Diffusion, etc. which have been trained for setups prediction), 3D representation segmentation, 3D representation coordinate system prediction, element labeling for 3D representation clean-up (VAE for Mesh Element labeling), in-filling of missing elements in 3D representation (MAE for Mesh In-Filling), dental restoration design generation, setups classification, appliance component generation & placement, archform prediction, imputation of oral care parameters, setups validation, or other validation applications and tooth 3D representation classification.

[0098] Some implementations of the techniques of this disclosure incorporate the use of an autoencoder. Autoencoders that can be used in accordance with aspects of this disclosure include but are not limited to: AtlasNet, FoldingNet and 3D-PointCapsNet. Some autoencoders may be implemented based on PointNet.

[0099] Systems of this disclosure may train ML models with representation learning. The advantages of representation learning include the fact that the generative network (e.g., neural network that predicts a transform for use in setups prediction) is guaranteed to receive input with a known size and/or standard format, as opposed to receiving input with a variable size or structure. Representation learning may produce improved performance over other methods, since noise in the input data may be reduced (e.g., since the representation generation model extracts the important aspects of an inputted representation (e.g., a mesh or point cloud) through loss calculations or network architectures chosen for that purpose). Such loss calculation methods may include KL-divergence loss, reconstruction loss or other losses disclosed herein. Representation learning may reduce the size of the dataset required for training a model, since the representation model learns the representation, enabling the generative network to focus on learning the generative task. The result may be improved model generalization because meaningful features of the input data (e.g., local and/or global features) are made available to the generative network. In some instances, transfer learning may first train a representation generation model. That representation generation model (in whole or in part) may then be used to pre-train a subsequent model, such as a generative model (e.g., that generates transform predictions). A representation generation model may benefit from taking mesh element features as input, to improve the understanding of the structure and/or shape of the inputted 3D oral care representations in the training dataset.

[00100] Representation learning may be applied to setups prediction techniques of this disclosure by training a neural network to learn a representation of the teeth, and then using another neural network to generate transforms for the teeth (e.g., which may effectuate movements of one or more teeth). Transforms may, in some instances, effect tooth movements. Some implementations may use a VAE or a Capsule Autoencoder to generate a representation of the essential characteristics of the one or more meshes related to the oral care domain (including, in some instances, information about the structures of the tooth meshes). Then that representation (either a latent vector or a latent capsule) may be used as input to a module which generates the one or more transforms for the one or more teeth. These transforms may in some implementations place the teeth into final setups poses. These transforms may in some implementations place the teeth into intermediate staging poses. In some implementations, a transform may be described by a 9x1 transformation vector (e.g., that specifies a translation vector and a quaternion). In other implementations, a transform may be described by a transformation matrix (e.g., a 4x4 affine transformation matrix).

[00101] In some implementations, systems of this disclosure may implement a principal components analysis (PCA) on an oral care mesh and use the resulting principal components as at least a portion of the representation of the oral care mesh in subsequent machine learning and/or other predictive or generative processing.

[00102] Systems of this disclosure may implement end-to-end training. Some of the end-to-end training-based techniques of this disclosure may involve two or more neural networks, where the two or more neural networks are trained together (i.e., the weights are updated concurrently during the processing of each batch of input oral care data). End-to-end training may, in some implementations, be applied to setups prediction by concurrently training a neural network which learns a representation of the teeth, along with a neural network which generates the tooth transforms.

[00103] According to some of the transfer learning-based implementations of this disclosure, a neural network (e.g., a U-Net) may be trained on a first task (e.g., such as coordinate system prediction). The neural network trained on the first task may be executed to provide one or more of the starting neural network weights for the training of another neural network that is trained to perform a second task (e.g., setups prediction). The first network may learn the low-level neural network features of oral care meshes and be shown to work well at the first task. The second network may exhibit faster training and/or improved performance by using the first network as a starting point in training. Certain layers may be trained to encode neural network features for the oral care meshes that were in the training dataset. These layers may thereafter be fixed (or be subjected to minor changes over the course of training) and be combined with other neural network components, such as additional layers, which are trained for one or more oral care tasks (such as setups prediction). In this manner, a portion of a neural network for one or more of the techniques of the present disclosure (e.g., setups prediction) may receive initial training on another task, which may yield important learning in the trained network layers. This encoded learning may then be built upon with further task-specific training of another network.

[00104] In accordance with this disclosure, transfer learning may be used for setups prediction, as well as for other oral care applications, such as mesh classification (e.g., tooth or setups classification), mesh element labeling, mesh element in-filling, procedure parameter imputation, mesh segmentation, coordinate system prediction, restoration design generation, mesh validation (for any of the applications disclosed herein). In some implementations, a neural network trained to output predictions based on oral care meshes may first be partially trained on one of the following publicly available datasets, before being further trained on oral care data: Google PartNet dataset, ShapeNet dataset, ShapeNetCore dataset, Princeton Shape Benchmark dataset, ModelNet dataset, ObjectNet3D dataset, ThingilOK dataset (which is especially relevant to 3D printed parts validation), ABC: A Big CAD Model Dataset For Geometric Deep Learning, ScanObjectNN, VOCASET, 3D-FUTURE, MCB: Mechanical Components Benchmark, PoseNet dataset, PointCNN dataset, MeshNet dataset, MeshCNN dataset, PointNet++ dataset, PointNet dataset, or PointCNN dataset. [00105] In some implementations, a neural network which was previously trained on a first dataset (either oral care data or other data) may subsequently receive further training on oral care data and be applied to oral care applications (such as setups prediction). Transfer learning maybe employed to further train any of the following networks: GCN (Graph Convolutional Networks), PointNet, ResNet or any of the other neural networks from the published literature which are listed above.

[00106] In some implementations, a first neural network may be trained to predict coordinate systems for teeth (such as by using the techniques described in WO2022123402A1 or US Provisional Application No. US63/366492). A second neural network may be trained for setups prediction, according to any of the setups prediction techniques of the present disclosure (or a combination of any two or more of the techniques described herein). Transfer learning may transfer at least a portion of the knowledge or capability of the first neural network to the second neural network. As such, transfer learning may provide the second neural network an accelerated training phase to reach convergence. In some implementations, the training of the second network may, after being augmented with the transferred learning, then be completed using one or more of the techniques of this disclosure.

[00107] A neural network may be trained to consider one or more inputs using an attention mechanism. This attention mechanism may be implemented using attention gates, attention layers, multi-headed attention mechanisms, or other attention mechanisms. One or more of the neural networks models of this disclosure may have attention gates integrated within. Attention gate integration provides the enhancement of enabling the associated neural network architecture to focus resources on one or more input values. In some implementations, an attention gate may be integrated with a U-Net architecture, with the advantage of enabling the U-Net to focus on certain inputs, such as input flags which correspond to teeth which are meant to be fixed during orthodontic treatment (or which require other special handling). An attention gate may also be integrated with an encoder or with an autoencoder (such as VAE or capsule autoencoder) to improve resource efficiency, in accordance with aspects of this disclosure.

[00108] Representation learning approaches may train some neural networks to represent tooth data (e.g., as latent representations) and other neural networks to predict tooth transforms based on those representations. The tooth transforms may place one or more teeth of the patient into poses which are suitable for orthodontic setups (e.g., final setups or intermediate stages). Neural networks for setups prediction may, in some implementations, consume mesh element features (e.g., a mesh element feature vector for each mesh element of the patient’s dentition) or oral care metrics as inputs, to improve the ability of the one or more neural networks of the setups prediction model to understand, encode or interpret the shape and/or structure of the inputted 3D oral care representations of the patient’s dentition. Automated setups prediction may, in some instances, be performed in response to determining that orthodontic treatment must be performed prior to dental restoration of one or more pre-restoration teeth. [00109] A setups prediction neural network may require a training dataset of hundreds or thousands of cohort patient cases, to ensure that the neural network is able to encode the distribution of patient cases which are likely to be encountered in clinical treatment. Some patient cases in the training dataset may meet one or more of the conditions described herein regarding a requirement to first perform orthodontic treatment prior to dental restorative treatment. Stated another way, the anatomy of one or more of the patient cases in the training dataset may require dental restorative treatment, and in some examples, orthodontic treatment may be required prior to that dental restorative treatment (e.g., in keeping with examples described herein). FIG. 10 describes dental anatomy which may benefit from the placement or application of composite dental restorations or composite fillings (which may be used to train ML models according to techniques of this disclosure). The example shown in FIG. 10 (which may be used to train ML models according to techniques of this disclosure), a patient did not like the small size of his lateral incisors. Composite restorations were used to reshape those teeth so that his smile was more esthetic, symmetrical and appealing.

[00110] In another representative example (which may be used to train ML models according to techniques of this disclosure), a filling is done in tooth #2 to replace an older silver restoration. A filling is done in tooth #3 to remove decay in the distal pita and replace a broken-down silver filling.

[00111] In another representative example (which may be used to train ML models according to techniques of this disclosure), a filling is done in tooth #18 due to a lingual and buccal crack.

[00112] In another representative example (which may be used to train ML models according to techniques of this disclosure), the patient has a chipped tooth #24, and a filling is placed to restore the tooth to the proper shape.

[00113] In another representative example (which may be used to train ML models according to techniques of this disclosure), a filling is done to fill a hole made in a crown to do a root canal.

[00114] In another representative example (which may be used to train ML models according to techniques of this disclosure), a filling is needed in tooth #10’s incisal edge due to an enamel fracture that happened when an aligner attachment was removed.

[00115] In another representative example (which may be used to train ML models according to techniques of this disclosure), a patient has some uneven incisal edges, some chipped edges and some edges which are worn down. The patient wants a more even smile. Composite restorations are done to correct the shapes of the teeth.

[00116] In another representative example (which may be used to train ML models according to techniques of this disclosure), a patient has worn-down teeth and wants to restore the teeth to correct height for esthetic and functional (bite) reason. Composite restorations are done to restore the correct shape and height. [00117] In another representative example (which may be used to train ML models according to techniques of this disclosure), a patient has older discolored restorations on her front teeth. She wants to replace the restorations with the appropriate shade so that color is the same throughout.

[00118] In another representative example (which may be used to train ML models according to techniques of this disclosure), restoration are done on the front teeth to close the space between the teeth. The patient considers the space aesthetic, and so restoration is done.

[00119] In another representative example (which may be used to train ML models according to techniques of this disclosure), a composite restoration is done along the gumline of tooth #11 where the gum tissue has receded.

[00120] Representation learning approaches may train some neural networks to represent tooth data (e.g., as latent representations), which may, in some instances, undergo modification. Other neural networks maybe trained to generate tooth restoration designs based, at least in part, on those latent representations. Transformers, U-Nets or autoencoders may be trained to produce such representations, among other neural networks. Neural networks for restoration design generation may, in some implementations, consume mesh element features (e.g., a mesh element feature vector for each mesh element of the patient’s dentition) or oral care metrics as inputs, to improve the ability of the one or more neural networks of the restoration design generation model to understand, encode or interpret the shape and/or structure of the inputted 3D oral care representations of the patient’s dentition.

[00121] In some instances, restoration design generation may be combined with orthodontic treatment. For example, orthodontic treatment, such as with clear aligners or indirect bonding trays (which may be used to position appliances or hardware on the teeth), may in some instances, be applied to move one or more teeth so that a target pre-restoration tooth is made more accessible for restoration treatment. Stated another way, as seen in FIG. 9, upon the moving of one or more adjacent teeth 910, a target pre-restoration tooth 905 may be made more accessible for treatment by a dental restoration appliance, such as the 3M FILTER Matrix which assists in the placement of a direct composite restoration or to permit access for placement of an indirect restoration such as an inlay, onlay, bridge, crown or veneer (e.g., such as a zirconia veneer that has been milled or a zirconia veneer that has been 3D printed). FIG. 9B shows target teeth 920 and 925 with occlusions which permit dental restorative treatment to proceed. FIG. 9C shows the completion of treatment. An arch may be produced using an intra-oral scanner, using a CT-scanner, or a scan of a 3D model (e.g., such as a dental model made by taking an impression of the patient's dentition, or a 3D printed model). A scanned arch may be segmented, undergo mesh cleanup or coordinate system prediction. Setups prediction techniques such as GDL Setups and others disclosed herein may be applied in the generation of final setups and/or intermediate stages. These setups may be assembled into fixture models, each of which may then be 3D printed, which may be followed by the thermoforming and trimming of an aligner tray for orthodontic treatment. The one or more aligner trays may be applied to the patient’s dentition, in succession, ahead of dental restoration treatment. In some instances, adjacent maloccluded teeth may be moved out of the way and/or space may be created so that a dental restoration appliance may be securely put into place and dental restoration treatment may be effected. Final setups may also be used to plan treatments with brackets and wires as the position and orientation of brackets on the teeth predicts the final position of the teeth (e.g., such as with an indirect bonding tray which is used to deliver brackets or other hardware onto the teeth). FIG. 4 describes a method where orthodontic treatment is required prior to dental restorative treatment to make one or more pre-restoration teeth accessible to dental restorative treatment. The patient may first undergo intraoral scanning 400 (or undergoes impressioning and subsequent scanning of that impression) by a clinician. The intraoral scan may undergo 402 mesh cleanup, mesh segmentation or coordinate system prediction, resulting in 3D representations of the patient’s teeth 404. Techniques described herein may be executed to predict 406 whether orthodontic treatment must precede dental restorative treatment (e.g., predict whether the patient’s teeth are currently too maloccluded for dental restorative treatment to proceed). If the patient’s teeth are not too maloccluded, then dental restorative treatment may proceed 408 (e.g., using a dental restoration appliance - which may involve automated restoration design generation). If the patient’s teeth are too maloccluded, then orthodontic treatment must first be applied 410 (e.g., using aligner trays, or brackets & indirect bonding trays). The accuracy of such orthodontic treatment may be improved through automatic setups prediction, according to techniques described herein. In some instances, subsequent to the completion of orthodontic treatment, the patient may then undergo an optional further round of intraoral scanning 412, and the process may iterate until the patient’s teeth are predicted to be in an acceptable alignment to undergo dental restorative treatment.

[00122] Restoration planning may be done at various levels of detail. Space can be created within the arch in preparation for restoration treatment, such as to create space around a tooth, space between two or more teeth and/or resolution of a tooth mispositioning that could reduce survivability of a restoration, such as a crossbite. Planning for the design of restorations may occur prior to tooth movement or after tooth movement. For instance, tooth movements and tooth restoration plans may be planned early in treatment, but updated mid-treatment based on actual attainment of the plan. In particular, high resolution mid treatment scans may be used to design the final contours of the restorations. In this way, the restoration contours may compensate for variability in the precise attainment of the orthodontic correction. Techniques of this disclosure for automated restoration design generation (e.g., such as using reconstruction autoencoder - such as a variational autoencoder with optional normalizing flows) may be performed to generate the target shape of a tooth for restoration treatment. [00123] Some patients have malocclusions which necessitate orthodontic pre-treatment before dental restorative treatment may be applied (e.g., when the anterior teeth have a bite situation which makes restorative treatment difficult). In ideal occlusal situations, the lower arch ideally should be smaller than the upper arch (e.g., the lower arch sits inside the upper arch) and the upper anterior teeth ideally should have some overjet (e.g., horizontal overlap) and some overbite (e.g., vertical overlap) with the lower anterior teeth (e.g., so that there may be a bite that is functional and less likely experience tooth wear or breakage). The lower canines ideally should be located anterior (towards the midline) of the upper canines when the patient bites down. Such a configuration of teeth may enable normal movements in chewing and speaking. A non-limiting list of conditions for orthodontic pre-treatment includes the conditions described in FIGs. 1-3, the subsequent examples, or the following conditions:

[00124] 1) Worn teeth that have over-erupted over time, resulting in teeth that are too short and in malocclusion.

[00125] 2) Non-symmetrical growth of the two arches resulting in Class 2 or 3 occlusions or non-ideal occlusions.

[00126] 3) Teeth that are too big for the arch and thus erupt in a crowded fashion.

[00127] 4) Teeth that are too small and therefore create spaces.

[00128] 5) Teeth that are normally sized but erupt in arches that are proportionally too small.

[00129] 6) Teeth that erupt out-of-sequence and therefore erupt non-ideally.

[00130] 7) Teeth that are genetically missing or unable to erupt resulting in spacing and movement of other teeth in the arch (e.g., the adjacent or adjourning teeth).

[00131] Orthodontic treatment may include traditional brackets (e.g., where brackets are placed on the patient’s teeth manually by a clinician), indirect bonding (e.g., where brackets are placed with the aid of an indirect bonding tray - such as a tray that was thermoformed onto a 3D printed fixture model of teeth with bracket shapes in position on the teeth), or orthodontic aligners. The 3M Digital Bonding System is an example of treatment with indirect bonding trays. The 3M CLARITY Aligners are an example of treatment with orthodontic aligner trays.

[00132] Techniques of this disclosure may, in some instances, be used to predict whether orthodontic treatment has progressed sufficiently for the restoration to be performed (e.g., such as seen in FIGs. 8A- 8D). For instance, a target envelope may be defined for the position and/or orientation of a target prerestoration tooth 810. A target pre-restoration tooth 810 (and/or one or more surrounding teeth) undergoes orthodontic treatment (see the progress of tooth 810, to tooth 825 and finally to tooth 835), and eventually enters the target envelop 840 which has been designated for that tooth (e.g., by an ML model that is trained to identify such an envelope), after which the pre-restoration tooth 835 is ready to be restored. In some instances, a dental restoration appliance may be generated, which is used to apply a veneer to pre-restoration tooth 835, producing restored tooth 855.

[00133] FIG. 6 describes a machine learning (ML) model 604 which may be trained to identify when orthodontic pre-treatment is advisable before dental restorative treatment begins. The ML model may output an indication 608 that orthodontic treatment is advised to precede dental restorative treatment (e.g., with composite veneers - such as composite veneers which are formed through the use of a dental restoration appliance), or the ML model may output an indication 606 that dental restorative treatment may proceed without intervening orthodontic treatment. Such an ML model may be trained on 3D representations of patient dentitions 600 (e.g., teeth and/or gums), including one or more positive examples of patient cases requiring orthodontic pre-treatment, or one or more negative patient cases which do not require orthodontic pre-treatment. In some implementations, an oral care metrics module 602 may compute one or more oral care metrics (e.g., Alignment or others disclosed herein) on the dentition 600 (e.g., mal, intermediate stage, or final stage, etc.), and subsequently provide the oral care metrics to the ML model 604. The oral care metrics quantify aspects of the patient’s dentition, and may aid the ML model 604 in making the determination of whether or not orthodontic treatment is advisable before dental restorative treatment begins. In some instances, one or more mesh element features (e.g., such as those described herein) may be computed by a mesh element feature module for one or more teeth (or other 3D representations of patient dentition). For example, a mesh element feature vector may be computed for each mesh element in each tooth of a patient case. Such mesh element feature vector may enable an (optional) representation generation module 610 (which consumes the patient dataset as input) to better understand the shape and/or structure of the patient’ s dental anatomy, enabling that the encoder 710 which is contained within representation generation module 610 to generate a better latent representation 715 (e.g., a representation which describes the shape and/or structure of the tooth in a reduced dimensionality). In some implementations, such a latent representation 715 may be reconstructed using a decoder 720 into a facsimile of the original 3D representation of the patient’s dentition (e.g., using a reconstruction autoencoder, shown in FIG. 7). The quality of the latent representation may, in some instances, be demonstrated by computing a reconstruction error between the original 705 and reconstructed 725 3D representations. The method 700 in FIG. 7 shows a 3D representation of a tooth 705, which may be encoded by an encoder 710 into a latent representation 715. The latent representation may be reconstructed by a decoder 720 into a reconstructed tooth 725.

[00134] The following is a description of non-limiting positive and negative training data examples for a machine learning classifier which may be trained to identify patient cases which require orthodontic pretreatment before dental restorative treatment may be applied. [00135] Training data scenario #1: positive examples - photographs ofpatient cases that match at least one of the conditions for orthodontic pre-treatment. Negative examples - photographs of patient cases which are ready for restorative treatment without the need for orthodontic pre-treatment.

[00136] Training data scenario #2: positive examples - 3D representations (e.g., 3D meshes or 3D point clouds, etc.) of patient case data that match at least one of the conditions for orthodontic pre-treatment. Negative examples - 3D representations (e.g., 3D meshes or 3D point clouds, etc.) of patient case data which are ready for restorative treatment without the need for orthodontic pre-treatment.

[00137] Training data scenario #3: positive examples - 2D representations (e.g., digital rendering of a 3D model - such as 2D raster image views of 3D tooth representations, or digital photographs of the patient’s dentition) of patient case data that match at least one of the conditions for orthodontic pretreatment. Negative examples - 2D representations (e.g., digital rendering of a 3D model - such as 2D raster image views of 3D tooth representations, or digital photographs of the patient’s dentition) of patient case data which are ready for restorative treatment without the need for orthodontic pre-treatment.

[00138] Representation learning may, in some instances, be used to predict whether the patient’s dentition warrants orthodontic pre-treatment before restoration is performed. In some instances, patient data (e.g., photographs, 3D representations or 2D representations of those 3D representations) may undergo latent encoding using a neural network, such as a reconstruction autoencoder or transformer. In some implementations, the encoder portion of a reconstruction autoencoder may convert the patient data into a latent representation which may have reduced dimensionality. This latent representation may be provided as input to another machine learning model which has been trained to classify that latent representation into classes such as: requires orthodontic pre-treatment or does not require orthodontic pre-treatment.

[00139] Techniques of this disclosure may generate one or more indications (e.g., a Boolean, categorical or real-valued output) to specify whether or not one or more stages of orthodontic treatment should precede dental restorative treatment. Such an indication may be saved to RAM, FLASH memory, a hard drive. Such an indication may be transmitted by electronic means (e.g., over SMS) or be displayed to a clinician. In some examples, training data scenario #2 may additionally undergo metrics calculation, as described in FIG. 5. One or more of the oral care metrics 504 described herein may be computed for 3D patient case data 500 (e.g., 3D meshes or 3D point clouds of the patient’s teeth and/or gums, and/or transforms). For example, one or more of overbite, ovcrjct. alignment metrics (among others described herein) may be computed. In some instances, thresholds may be predicted by an ML model, based upon inputs such as oral care metrics values which are part of a training dataset. Thresholds 502 and oral care metrics may be provided to the decision module 506, which may output an indication of whether orthodontic treatment should precede dental restorative treatment for the patient. When an oral care metric value falls beyond or within one or more pre-determined thresholds (e.g., thresholds or ranges predicted by an ML model), an indication may be outputted 510 which indicates that the patient case may require orthodontic pre-treatment. These metrics may also be used to predict whether orthodontic treatment has progressed to a state where the restoration may be performed. When the decision module 506 predicts that orthodontic treatment is not required prior to dental restorative treatment, then an indication 508 that dental restorative treatment may proceed may be outputted.

[00140] In some implementations, an ML model may be trained to generate one or more target ranges (or thresholds) 502 corresponding to one or more oral care metrics. For example, one or more 3D representations of the patient’s dentition may be provided to the ML model, and then the ML model may generate one or more target ranges (or thresholds) corresponding to one or more oral care metrics. The one or more target ranges or thresholds 502 may measure aspects of the patient’s dentition, and/or may correspond to the transition from a first treatment to a second treatment. The one or more ranges or thresholds 502 may be provided as inputs to the method described in FIG. 5.

[00141] The ML models of this disclosure may be trained, at least in part, by the calculation of one or more loss values which quantify the difference between predicted and ground truth (or expected) outputs. In some examples, loss calculation may compute the difference between a predicted target threshold for an oral care metric and a corresponding ground truth value for an oral care metric (e.g., provided by historical patient case data), and then that loss value may be used to train, at least in part, the ML model. In further examples, an ML model may be trained to generate one or more target ranges (or thresholds) 502 such that amaloccluded tooth 905 is allowed sufficient space for dental restorative treatment (e.g., tooth 910 is moved using orthodontic treatment and takes the pose of tooth 920).

[00142] Techniques of this disclosure may, in some instances, be used in a time -constrained setting where fast processing is essential to the treatment of the patient. For example, in some instances, techniques of this disclosure may be used in a clinical setting to generate an oral care appliance for the patient while the patient remains in the clinical environment (e.g., while the patient waits in the chair for one or more orthodontic aligners to be generated - for example by 3D printing). In some examples, an intraoral scan may be captured from the patient while the patient remains in the clinical environment. The infraoral scan may undergo mesh cleanup, mesh segmentation, and/or coordinate system prediction. Such operations may be performed while the patient remains in the clinical environment. Techniques of this disclosure may be used to predict whether orthodontic treatment must precede dental restorative treatment (e.g., using oral care metrics measurements or an ML model which has been trained for that purpose) of one or more prerestoration teeth. When it is predicted that orthodontic treatment must precede dental restorative treatment, an automated setups prediction model may be executed to generate intermediate stages and/or final setups. These setups may be used in the generation of an orthodontic appliance. In some instances, such an orthodontic appliance and be sent home with the patient. [00143] Techniques of this disclosure may, in some instances, be used to predict whether orthodontic treatment has progressed sufficiently for the restoration to be performed (e.g., such as seen in FIG. 8A-FIG. 8D). For instance, the dentition 800 shows a target envelope 805 which may be defined for the position and/or orientation of a tooth. A maloccluded pre-restoration tooth 810 (or one or more surrounding teeth) undergoes orthodontic treatment. Eventually, in dentition 815, the tooth 825 enters the target envelop 820 which has been designated for that tooth (e.g., by an ML model that is trained to identify such an envelope), after which the pre-restoration tooth 835 is centered in the target envelop 840 (as shown in dentition 830) and is ready to be restored. The dentition 845 in FIG. 8D shows the post-restoration tooth 855 to be centered on the target envelope 850.

[00144] In some implementations, oral care metrics (described herein) may be computed on 3D representations of the patient’s dentition to detect the conditions which are described in FIGs. 1-3 or in the subsequent examples, and subsequently predict a requirement for orthodontic treatment to precede dental restorative treatment (e.g., to perform operations including automated setups prediction to facilitate the generation of aligner trays for orthodontic treatment, prior to dental restorative treatment). In some implementations, an ML model may be trained on 3D representations of patient dentitions in a dataset of one or more cohort patient cases to detect the conditions which are described in FIGs. 1-3 or in the subsequent examples, and subsequently predict a requirement for orthodontic treatment to precede dental restorative treatment. In some implementations, one or more oral care metrics may be provided to such an ML model, to improve the ability of that model to detect the conditions which are described in FIGs. 1-3 or the subsequent examples, and subsequently predict a requirement for orthodontic treatment to precede dental restorative treatment.

[00145] FIG. 1 shows a patient case 100 (which may be used to train ML models according to techniques of this disclosure) where end-to-end bite is present. The lower six teeth 125 (in particular the central incisors) need to be moved in, or the upper central incisors 105 need to be moved out, to create overjet (to enable composite to be added to the upper central incisors) before upper central incisor 120 may receive restorative treatment.

[00146] FIG. 2 shows a patient case 200 (which may be used to train ML models according to techniques of this disclosure) where the upper and lower arches of the left side of the mouth are in crossbite. This crossbite must be corrected. Teeth (e.g., all of the teeth 205 on the left side from the midline going back) need to be moved before the teeth 210 on the left may undergo restorative treatment.

[00147] FIG. 3 shows a patient case 300 (which may be used to train ML models according to techniques of this disclosure) where the upper left lateral incisor is crowded out of position and rotated slightly. This crowding condition must be corrected. The upper left lateral incisor 310 needs to be brought into alignment before the patient may undergo restorative treatment at site 305 on the upper left lateral incisor 310.

[00148] In another representative example (which may be used to train ML models according to techniques of this disclosure), the upper four anterior teeth (e.g., the central and lateral incisors) are in an end-to-end bite with the lower anterior teeth. Before the upper central incisors may be reshaped and/or lengthened, at least one of the upper incisors or the lower incisors need to be moved out of the way to correct the end-to-end bite.

[00149] In another representative example (which may be used to train ML models according to techniques of this disclosure), the upper lateral incisors are in end-to-end bite with the lower canines and do not have the proper overjet. Teeth need to be moved before the upper lateral incisors may undergo restorative treatment.

[00150] In another representative example (which may be used to train ML models according to techniques of this disclosure), the upper left canine has erupted into a lower space. This over-eruption condition must be corrected. The left upper canine needs to be orthodontically pushed back into position before the patient may undergo correct esthetic restorative treatment.

[00151] In another representative example (which may be used to train ML models according to techniques of this disclosure), the upper teeth are worn and short due to the bite. There is inadequate overjet between the upper teeth and the lower teeth. These tooth positions need to be corrected (e.g., moving at least one of the upper or the lower teeth) before the upper teeth may undergo restorative treatment.

[00152] In another representative example (which may be used to train ML models according to techniques of this disclosure), the upper four anterior teeth are hitting end-to-end with the lower anterior teeth. This end-to-end condition needs to be corrected before the upper four anterior teeth may undergo restorative treatment.

[00153] In another representative example (which may be used to train ML models according to techniques of this disclosure), the overbite is beyond a threshold overlap (e.g., a deep bite is present and needs to be corrected), and worn teeth are in occlusion (e.g., in contact) then orthodontic treatment may be required to reposition worn teeth out of occlusion to enable dental restorative treatment (e.g., with a dental restoration appliance). Appropriate overbite must be created.

[00154] In another representative example (which may be used to train ML models according to techniques of this disclosure), one or more of the patient’s teeth have worn short and/or have over-erupted, then orthodontic treatment may be required ahead of dental restorative treatment. This over-eruption condition must be corrected.

[00155] In another representative example (which may be used to train ML models according to techniques of this disclosure), one or more of the patient’s teeth are too narrow or too small in size for the proper overjet (e.g., which may occur when upper spacing and lower crowding are present), then orthodontic treatment may be required before dental restorative treatment may be properly accomplished. The crowding condition must be corrected.

[00156] Aspects of the present disclosure can provide a technical solution to the technical problem of determining, using one or more 3D representations of a patient’s dentition, whether orthodontic treatment of the patient must precede dental restorative treatment of the patient (e.g., determining whether a target pre-restoration tooth is accessible to dental restoration treatment, given the physical relationships of nearby teeth in the arch). In particular, by practicing techniques disclosed computing systems specifically adapted to perform mesh processing for oral care appliance generation are improved. For example, aspects of the present disclosure improve the performance of a computing system having a 3D representation of the patient’s dentition by reducing the consumption of computing resources. In particular, aspects of the present invention reduce computing resource consumption by decimating 3D representations of the patient’s dentition (e.g., reducing the counts of mesh elements used to describe aspects of the patient’s dentition) so that computing resources are not unnecessarily wasted by processing excess quantities of mesh elements. That is, aspects of the present invention provide for more efficient allocation of computing resources.

[00157] Furthermore, aspects of the present disclosure may need to be executed in a time-constrained manner, such as when an oral care appliance must be generated for a patient immediately after intraoral scanning (e.g., while the patient waits in the clinician’s office). As such, aspects of the present disclosure are necessarily rooted in the underlying computer technology of mesh processing for oral care appliance generation and cannot be performed by a human, even with the aid of pen and paper. For instance, implementations of the present disclosure must be capable of: 1) storing thousands or millions of mesh elements of the patient’s dentition in a manner that can be processed by a computer processor; 2) performing calculation on thousands or millions of mesh elements, e.g., to quantify aspects of the shape and or/structure of an individual tooth in the 3D representation of the patient’s dentition; and 3) determining, based on performing various comparisons, whether the patient must undergo an orthodontic treatment prior to the initiation of a dental restorative treatment on one or more aspects of the patient’s dentition and do so during the course of a short office visit.

[00158] Techniques described herein may be trained to generate transforms which may place the patient’s teeth into poses suitable for use in orthodontic setups (e.g., intermediate stages or final setups), according to the requirements of the oral care arguments which may, in some implementations, be provided to the generative model. Oral care arguments may include oral care parameters as disclosed herein, or other real-valued, text-based or categorical inputs which specify intended aspects of the one or more 3D oral care representations which are to be generated. In some instances, oral care arguments may include oral care metrics, which may describe intended aspects of the one or more 3D oral care representations which are to be generated. Oral care arguments are specifically adapted to the implementations described herein. For example, the oral care arguments may specify the intended the designs (e.g., including shape and/or structure) of 3D oral care representations which may be generated (or modified) according to techniques described herein. In short, implementations using the specific oral care arguments disclosed herein generate more accurate 3D oral care representations than implementations that do not use the specific oral care arguments. In some instances, a text encoder may encode a set of natural language instructions from the clinician (e.g., generate a text embedding). A text string may comprise tokens. An encoder for generating text embeddings may, in some implementations, apply either mean-pooling or max-pooling between the token vectors. In some instances, a transformer (e.g., BERT or Siamese BERT) may be trained to extract embeddings of text for use in digital oral care (e.g., by training the transformer on examples of clinical text, such as those given below). In some instances, such a model for generating text embeddings may be trained using transfer learning (e.g., initially trained on another corpus of text, and then receive further training on text related to digital oral care). Some text embeddings may encode text at the word level. Some text embeddings may encode text at the token level. A transformer for generating a text embedding may, in some implementations, be trained, at least in part, with a loss calculation which compares predicted outputs to ground truth outputs (e.g., softmax loss, multiple negatives ranking loss, MSE margin loss, cross-entropy loss or the like). In some instances, the non-text arguments, such as real values or categorical values, may be converted to text, and subsequently embedded using the techniques described herein. The following are examples of natural language instructions that may be issued by a clinician to the generative models described herein: 1) “Generate a bracket setup to finish with 2 mm overbite, 2 mm of overjet, apply Class II elastics and apply L2-2 .5 mm incisal to ideal”, 2) “Adjust the setup to include IPR of .3mm L4-4, refract to close spaces for increased overjef ’, or 3) “Generate a setup to intrude posterior teeth to allow for a 2 mm overbite with autorotation, apply lower IPR as needed for overjet, expand and procline for space to align.” [00159] Techniques of this disclosure may, in some implementations, use PointNet, PointNet++, or derivative neural networks (e.g., networks trained via transfer learning using either PointNet or PointNet++ as a basis for training) to extract local or global neural network features from a 3D point cloud or other 3D representation (e.g., a 3D point cloud describing aspects of the patient’s dentition - such as teeth or gums). Techniques of this disclosure may, in some implementations, use U-Nets to extract local or global neural network features from a 3D point cloud or other 3D representation.

[00160] 3D oral care representations are described herein as such because 3-dimensional representations are currently state of the art. Nevertheless, 3D oral care representations are intended to be used in a non-limiting fashion to encompass any representations of 3 -dimensions or higher orders of dimensionality (e.g., 4D, 5D, etc.), and it should be appreciated that machine learning models can be trained using the techniques disclosed herein to operate on representations of higher orders of dimensionality.

[00161] In some instances, input data may comprise 3D mesh data, 3D point cloud data, 3D surface data, 3D polyline data, 3D voxel data, or data pertaining to a spline (e.g., control points). An encoderdecoder structure may comprise one or more encoders, or one or more decoders. In some implementations, the encoder may take as input mesh element feature vectors for one or more of the inputted mesh elements, to improve the ability of the encoder to generate a representation of the input data. Examples of encoderdecoder structures include U-Nets, autoencoders or transformers (among others). A representation generation module may comprise one or more encoder-decoder structures (or portions of encoders-decoder structures - such as individual encoders or individual decoders). A representation generation module may generate an information-rich (optionally reduced-dimensionality) representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.

[00162] A U-Net may comprise an encoder, followed by a decoder. The architecture of a U-Net may resemble a U shape. The encoder may extract one or more global neural network features from the input 3D representation, zero or more intermediate-level neural network features, or one or more local neural network features (at the most local level as contrasted with the most global level). The output from each level of the encoder may be passed along to the input of corresponding levels of a decoder (e.g., by way of skip connections). Like the encoder, the decoder may operate on multiple levels of global-to-local neural network features. For instance, the decoder may output a representation of the input data which may contain global, intermediate or local information about the input data. The U-Net may, in some implementations, generate an information-rich (optionally reduced-dimensionality) representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.

[00163] An autoencoder may be configured to encode the input data into a latent form. An autoencoder may train an encoder to reformat the input data into a reduced-dimensionality latent form in between the encoder and the decoder, and then train a decoder to reconstruct the input data from that latent form of the data. A reconstruction error may be computed to quantify the extent to which the reconstructed form of the data differs from the input data. The latent form may, in some implementations, be used as an informationrich reduced-dimensionality representation of the input data which may be more easily consumed by other generative or discriminative machine learning models. In most scenarios, an autoencoder may be trained to input a 3D representation, convert that 3D representation into a latent form (e.g., a latent embedding), and then reconstruct a close facsimile of that input 3D representation at the output.

[00164] A transformer may be trained to use self-attention to generate, at least in part, representations of its input. A transformer may encode long-range dependencies (e.g., encode relationships between a large number of inputs). A transformer may comprise an encoder or a decoder. Such an encoder may, in some implementations, operate in a bi-directional fashion or may operate a self-attention mechanism. Such a decoder may, in some implementations, may operate a masked self-attention mechanism, may operate a cross-attention mechanism, or may operate in an auto-regressive manner. The self-attention operations of the transformers described herein may, in some implementations, relate different positions or aspects of an individual 3D oral care representation in order to compute a reduced-dimensionality representation of that 3D oral care representation. The cross-attention operations of the transformers described herein may, in some implementations, mix or combine aspects of two (or more) different 3D oral care representations. The auto-regressive operations of the transformers described herein may, in some implementations, consume previously generated aspects of 3D oral care representations (e.g., previously generated points, point clouds, transforms, etc.) as additional input when generating a new or modified 3D oral care representation. The transformer may, in some implementations, generate a latent form of the input data, which may be used as an information-rich reduced-dimensionality representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.

[00165] In some implementations, an encoder-decoder structure may first be trained as an autoencoder. In deployment, one or more modifications may be made to the latent form of the input data. This modified latent form may then proceed to be reconstructed by the decoder, yielding a reconstructed form of the input data which differs from the input data in one or more intended aspects. Oral care arguments, such as oral care parameters or oral care metrics may be supplied to the encoder, the decoder, or may be used in the modification of the latent form, to guide the encoder-decoder structure in generating a reconstructed form that has desired characteristics (e.g., characteristics which may differ from that of the input data).

[00166] Techniques of this disclosure may, in some instances, be trained using federated learning. Federated learning may enable multiple remote clinicians to iteratively improve a machine learning model (e.g., validation of 3D oral care representations, mesh segmentation, mesh cleanup, other techniques which involve labeling mesh elements, coordinate system prediction, non-organic object placement on teeth, appliance component generation, tooth restoration design generation, techniques for placing 3D oral care representations, setups prediction, generation or modification of 3D oral care representations using autoencoders, generation or modification of 3D oral care representations using transformers, generation or modification of 3D oral care representations using diffusion models, 3D oral care representation classification, imputation of missing values), while protecting data privacy (e.g., the clinical data may not need to be sent “over the wire” to a third party). Data privacy is particularly important to clinical data, which is protected by applicable laws. A clinician may receive a copy of a machine learning model, use a local machine learning program to further train that ML model using locally available data from the local clinic, and then send the updated ML model back to the central hub or third party. The central hub or third party may integrate the updated ML models from multiple clinicians into a single updated ML model which benefits from the learnings of recently collected patient data at the various clinical sites. In this way, a new ML model may be trained which benefits from additional and updated patient data (possibly from multiple clinical sites), while those patient data are never actually sent to the 3rd party. Training on a local in-clinic device may, in some instances, be performed when the device is idle or otherwise be performed during off- hours (e.g., when patients are not being treated in the clinic). Devices in the clinical environment for the collection of data and/or the training of ML models for techniques described here may include intra-oral scanners, CT scanners, X-ray machines, laptop computers, servers, desktop computers or handheld devices (such as smart phones with image collection capability). In addition to federated learning techniques, in some implementations, contrastive learning may be used to train, at least in part, the ML models described herein. Contrastive learning may, in some instances, augment samples in a training dataset to accentuate the differences in samples from difference classes and/or increase the similarity of samples of the same class.

[00167] Machine learning models such as: U-Nets, encoders, autoencoders, pyramid encoder-decoders, transformers, or convolution & pooling layers, may be trained as a part of a method for hardware (or appliance component) placement. Representation learning may train a first module to encode an embedded representation of a 3D oral care representation (e.g., converting a mesh or point cloud into a latent form using an autoencoder, or using a U-Net, encoder, transformer, block of convolution & pooling layers or the like). That representation may comprise a reduced dimensionality form and/or information-rich version of the inputted 3D oral care representation. In some implementations, the generation of a representation may be aided by the calculation of a mesh element feature vector for one or more mesh elements (e.g., each mesh element). In some implementations, a representation may be computed for a hardware element (or appliance component). Such representations are suitable to be inputted to a second module, which may perform a generative task, such as transform prediction (e.g., a transform to place a 3D oral care representation relative to another 3D oral care representation, such as to place a hardware element or appliance component relative to one or more teeth) or 3D point cloud generation. Such a transform may comprise an affine transformation matrix, translation vector or quaternion or the like. Machine learning models which may be trained to predict a transform to place a hardware element (or appliance component) relative to elements of patient dentition include: MLP, transformer, encoder, or the like. Systems of this disclosure may be trained for 3D oral care appliance placement using past cohort patient case data. The past patient data may include at least: one or more ground truth transforms and one or more 3D oral care representations (such as tooth meshes, or other elements of patient dentition). In the instance where a U- Net (among other neural networks) is trained to generate the representations of tooth meshes, the mesh convolution and/or mesh pooling techniques described herein enjoy invariance to rotations/translations/scaling of that tooth mesh. Pose transfer techniques may be trained for hardware or appliance component placement. Reinforcement learning techniques may be trained for hardware or appliance component placement.

[00168] Ortho-to-restore treatment may combine orthodontic and dental restorative treatments, according to examples contained herein. Dental restoration may occur after orthodontic treatment, before orthodontic treatment, or anytime during orthodontic treatment. Techniques of this disclosure may, in some instances, operate in a time-constrained manner, such as while the patient waits in a clinician's office. In some instances, techniques of this disclosure may automatically predict when orthodontic treatment is warranted before dental restorative treatment is applied. In some instances, techniques of this disclosure may automatically determine when dental restorative treatment is warranted after orthodontic treatment has been completed, to complete a custom smile for the patient. For example, techniques of this disclosure may perform automatic setups prediction (e.g., using an ML model), which may predict the poses of the patient's teeth and/or roots after the completion of orthodontic treatment, and then the techniques may further predict that additional treatment (e.g., dental restorative treatment) is warranted to complete a target custom smile for the patient. In some instances, one or more oral care metrics (e.g., leveling, or others described herein) may be computed on the patient's dentition, or the patient's dentition may be provided as input to a trained ML model which has been trained to determine whether dental restorative treatment is warranted to generate a target custom smile for the patient.

[00169] For example, the patient may wish, for aesthetic purposes, to have the upper central incisors made level with the upper cuspids, and further to have the upper lateral incisors intruded (or extruded) so that the upper lateral incisors extend in the incisal direction until the upper lateral incisors are target distance (e.g., 0.5 mm, among others) from the central incisor incisal edge, in the gingival direction, though other target configurations are possible (among other examples of custom smiles). In some implementations, techniques of this disclosure may compute one or more oral care metrics and compare those oral care metrics against one or more thresholds or ranges of acceptable values. When a particular oral care metric is found to be beyond the prescribed threshold, or outside of the ranges of acceptable values, then the techniques may output one or more indications that dental restorative treatment is warranted. In other implementations, techniques of this disclosure may train an ML model. The patient’s dentition may be provided to the ML model, and the ML model may generate one or more indications regarding whether dental restorative treatment is warranted for the patient. In some implementations, oral care metrics may be provided to the ML model. In some implementations, mesh element features may be computed for one or more mesh elements of the patient’s dentition, and subsequently be provided to the ML model. The automated techniques of this disclosure (e.g., oral care metric-based and/or ML model-based techniques) may be trained to determine when the patient's post-orthodontic dentition does not meet the predetermined customization target. When the patient has already undergone the full extent of orthodontic treatment, and yet still has not achieved the target smile, the dental restorative treatment may be applied achieve the target custom smile. For example, when one or both lateral incisors are physically too short for orthodontic attachments to be applied, or when one or both lateral incisors has already been extruded to the fullest extent that can be achieved through orthodontic treatment (e.g., due physical limitations), then techniques of this disclosure may generate one or more indications that subsequent dental restorative treatment is warranted in order to achieve a target custom smile. This subsequent dental restorative treatment may be additive in nature, such as using veneers which are formed by a FILTEK Matrix or other dental restoration appliance. In some instances, dental restoration design generation (e.g., using a trained ML model, such as an autoencoder) may be performed to generate one or more options for aesthetic and/or clinically suitable 3D representations for one or more teeth which are to be restored. The mention of specific tooth types in this example should be understood, according to particular examples of patient treatment, to be inclusive of other tooth types (e.g., to consider dental restorative treatment on molars, bicuspids, cuspids, etc.). That is, the techniques of this disclosure may be applied to any tooth type to achieve the desired results.

[00170] In some examples, an ML model may be trained to predict whether further dental restorative treatment is warranted after the completion of orthodontic treatment. The training dataset may contain historical data for multiple patients (e.g., tens of thousands of patients), each of which includes the patient’s 3D dentition, and/or one or more historical labels indicating whether dental restorative treatment was applied subsequent to the completion of orthodontic treatment, among other possible ground truth labels. The patient’s dentition may include one or more of the initial dentition, and the target custom dentition (e.g., the final setup after the completion of orthodontic treatment, or the final target shapes of one or more teeth after dental restorative treatment). During training, the ML may generate one or more predictions (or indications) regarding the need for further treatment using a second technique (e.g., dental restorative treatment) after a first technique (e.g., orthodontic treatment) has been applied. The one or more predicted indications may be compared to corresponding ground truth data from the training dataset (e.g., to ground truth data which indicates whether subsequent treatment was historically applied to the case), and one or more loss values may be computed.

[00171] The one or more loss values may then be used to train, at least in part, the ML model. For example, the one or more loss values may be used to update the numerous weights of a neural network (when the ML model includes at least one neural network), enabling the neural network to learn the distribution of the training dataset, and to generate outputs which more closely match corresponding ground truth values as each additional epoch of training takes place. Training may proceed until the ML model reaches a pre-determined cross-validation accuracy, or until the model performance plateaus. Oral care metrics may, in some implementations, be provided to the ML model, to give the ML model additional information about the physical dimensions (or other aspects) of the patient’s dentition. An example of such an oral care metric includes one or more measurements of the distance between a lateral incisor incisal edge and the adjacent central incisor incisal edge, among other oral care metrics described herein. In some implementations, the ML model may be trained to generate one or more recommended modifications to one or more teeth (e.g., to predict that a lateral incisor should be lengthened by 0.5 mm in order to achieve a target custom smile).

[00172] The ML models of this disclosure (e.g., ML models that predict that dental restorative treatment is warranted after orthodontic treatment completes, or ML models that predict when orthodontic treatment is warranted before dental restorative treatment is performed, etc.) may be trained on historical training datasets which contain data for multiple patients (e.g., thousands, tens of thousands, or hundreds of thousands of patients). A dataset for an example patient may include (at least) 2D or 3D representations of the patient's dentition, and/or one or more ground truth data. Examples of ground truth data include one or more of 1) target orthodontic setups, or 2) labels indicating when a first treatment is required before a second treatment (e.g., either orthodontics before dental restorative treatment, or dental restorative treatment before orthodontic treatment), and the like. An ML model may be trained, at least in part, through the calculation of one or more loss values (e.g., LI, L2, cross entropy loss, or others described herein) that compares predicted outputs to corresponding ground truth data. For example, data for an historical patient may include the patient's 3D dentition (e.g., tooth and/or root meshes), and one or more ground truth labels indicating whether a first treatment preceded a second treatment (e.g., whether orthodontic treatment was first performed, and then followed by dental restorative treatment to complete the target custom smile). As a result, it should be apparent to one of ordinary skill in the art that due, a least in part, to the real or near- real time implementations of the techniques disclosed herein, performing the describes techniques would be both impracticable and prone to error for a human to perform, whether in the mind or with the aid of pen and paper.

Claims

What is claimed is:

1. A system comprising: one or more computer processors; non-transitory computer-readable storage having stored thereon instructions that when executed by the one or more processors cause the one or more processors to: receive a three-dimensional (3D) representation of a patient’s dentition comprising at least one of one or more teeth of the patient or gums of the patient; predict that an orthodontic treatment is required for the patient prior to the initiation of a dental restorative treatment on one or more aspects of the patient’s dentition, wherein the determining is based on the one or more aspects of the patient’s dentition prior to initiation of the dental restorative treatment; automatically generate, using a trained machine learning model and in response to determining that orthodontic treatment is required for the patient, one or more transforms to place at least one tooth of the patient’s dentition into a pose for an orthodontic setup, wherein the trained machine learning model was trained using operations comprising: receiving a plurality of historical cohort patient case data, wherein the historical cohort patient case data includes one more representations of one or more reference tooth movements for one or more patients and one or more representations of the patients’ dental anatomy; predicting by the machine learning model one or more transforms to apply to one or more teeth of the 3D representation of the patient’s dentition; quantifying the difference between a representation of the one or more transforms predicted by the generator and a representation of one or more reference tooth movements; generating a loss value based on the quantifying; modifying the machine learning model based at least in part on the loss value, wherein the anatomy of one or more patient cases required dental restorative treatment and orthodontic treatment was required prior to that dental restorative treatment; and output the one or more generated transforms.

2. The system of claim 1, wherein the generated orthodontic setup is a final setup.

3. The system of claim 2, wherein the instructions cause the processor to generate one or more intermediate stages, each sequentially depicting the teeth of the patient with progressive modification in the poses of the teeth for attaining the generated final setup.

4. The system of claim 1, wherein the determining comprises: computing one or more oral care metrics based on the 3D representation of the patient’s dentition, the one or more oral care metrics quantifying at least one of the shape of one or more teeth of the patient’s dentition or the spatial relationship between two or more teeth of the patient’s dentition; comparing the values of the one or more oral care metrics to corresponding thresholds; and generating an indication of the value of the one or more oral care metrics, wherein the indication of the value of the one or more of the oral care metrics is any of above, below, or equal to the corresponding threshold.

5. The system of claim 4, wherein the generated orthodontic setup is indicative of the teeth of the patient’s dentition having attained a threshold oral care metric value suitable for initiating dental restorative treatment of the patient.

6. The system of claim 1, wherein the determining comprises:

Receiving a plurality of historical cohort patient case data, where case data comprises at least a three-dimensional (3D) representation of a patient’s dentition comprising at least one of one or more teeth of the patient or gums of the patient, wherein at least one of the cases in the plurality required orthodontic treatment prior to the initiation of dental restorative treatment, and training a machine learning model to output an indication of whether orthodontic treatment is needed prior to dental restorative treatment.

7. The system of claim 1, wherein the dental restorative treatment comprises automated restoration design generation.

8. The system of claim 1, wherein the dental restorative treatment comprises 3D printing an appliance which is used to shape dental composite in the patient’s mouth.

9. The system of claim 1, wherein the orthodontic treatment comprises at least one of 3D printing one or more aligner trays, 3D printing one or more indirect bonding trays, or 3D printing one or more fixture models.

10. The system of claim 1, wherein the orthodontic treatment is designed to correct at least one of excessive overjet, an end-to-end bite, a crossbite, an over-eruption condition, a crowding condition, excessive overbite or a deep bite.

11. A system comprising: one or more computer processors; non-transitory computer-readable storage having stored thereon instructions that when executed by the one or more processors cause the one or more processors to: receive a three-dimensional (3D) representation of a patient’s dentition comprising at least one of gums of the patient or one or more teeth of the patient; compute one or more oral care metrics based on the 3D representation of the patient’s dentition; compare the one or more oral care metrics with a respective target threshold; predict based on the comparison a requirement for orthodontic treatment prior to a dental restorative treatment of one or more target pre-restoration teeth; and generate, based on the determining, a rendering of one or more indications recommending orthodontic treatment for the patient prior to dental restorative treatment on the one or more target prerestoration tooth.

12. The system of claim 11, wherein the instructions further cause the one or more processors to: generate one or more oral care appliance designs based at least in part on the one or more indications; and fabricate one or more oral care appliances based at least in part on the one or more oral care appliance designs.

13. The system of claim 11, wherein the one or more oral care metrics quantify at least one of the shape of one or more teeth of the patient’s dentition or the spatial relationship between two or more teeth of the patient’s dentition.

14. The system of claim 11, wherein orthodontic treatment may modify the poses of one or more teeth of the patient.

15. The system of claim 11, wherein one or more of the target thresholds are computed based, at least in part, on one or more historical oral care metrics computed from patient cases which have undergone orthodontic treatment prior to the initiation of dental restorative treatment on at least one target prerestoration tooth.

16. The system of claim 11, wherein the one or more indications recommending orthodontic treatment are generated in near real-time while a patient is in a clinical environment.

17. The system of claim 11, wherein when an indication is rendered in the affirmative that orthodontic treatment must be performed prior to a dental restorative treatment, a trained setups prediction model is automatically used by the one or more processors to generate one or more final setups or one or more intermediate stages.

18. The system of claim 17, wherein the one or more final setups or one or more intermediate stages are used to generate one or more orthodontic appliances.

19. The system of claim 11, wherein when an indication is rendered in the affirmative that orthodontic treatment must be performed prior to a dental restorative treatment, a trained restoration design generation model is automatically used by the one or more processors to generate one or more restoration designs.

20. The system of claim 19, wherein the one or more generated restoration designs are used to generate of one or more dental restoration appliances or one or more of an inlay, an onlay, a bridge, a crown or a veneer.