WO2024127314A1 - Imputation de valeurs de paramètres ou de valeurs métriques dans des soins buccaux numériques - Google Patents

Imputation de valeurs de paramètres ou de valeurs métriques dans des soins buccaux numériques Download PDF

Info

Publication number
WO2024127314A1
WO2024127314A1 PCT/IB2023/062708 IB2023062708W WO2024127314A1 WO 2024127314 A1 WO2024127314 A1 WO 2024127314A1 IB 2023062708 W IB2023062708 W IB 2023062708W WO 2024127314 A1 WO2024127314 A1 WO 2024127314A1
Authority
WO
WIPO (PCT)
Prior art keywords
oral care
tooth
model
setups
implementations
Prior art date
Application number
PCT/IB2023/062708
Other languages
English (en)
Inventor
Jonathan D. Gandrud
Michael Starr
Seyed Amir Hossein Hosseini
Joseph C. DINGELDEIN
Original Assignee
3M Innovative Properties Company
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 3M Innovative Properties Company filed Critical 3M Innovative Properties Company
Publication of WO2024127314A1 publication Critical patent/WO2024127314A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/45—For evaluating or diagnosing the musculoskeletal system or teeth
    • A61B5/4538—Evaluating a particular part of the muscoloskeletal system or a particular medical condition
    • A61B5/4542—Evaluating the mouth, e.g. the jaw
    • A61B5/4547—Evaluating teeth
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
    • A61C7/00—Orthodontics, i.e. obtaining or maintaining the desired position of teeth, e.g. by straightening, evening, regulating, separating, or by correcting malocclusions
    • A61C7/002—Orthodontic computer assisted systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101—Computer-aided simulation of surgical operations
    • A61B2034/105—Modelling of the patient, e.g. for ligaments or bones
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A61B5/1072—Measuring physical dimensions, e.g. size of the entire body or parts thereof measuring distances on the body, e.g. measuring length, height or thickness

Definitions

  • Patent Applications is incorporated herein by reference: 63/432,627; 63/366,492; 63/366,495; 63/352,850; 63/366,490; 63/366,494; 63/370,160; 63/366,507; 63/352,877; 63/366,514; 63/366,498; 63/366,514; and 63/264,914.
  • This disclosure relates to configurations and training of machine learning models to improve the accuracy of automatically predicting oral care parameters for use as inputs to a setups prediction model.
  • Training data for the ML model may include past patient cases, each of which may include multiple ground truth oral care parameter values.
  • a patient case may contain orthodontic procedure parameter values such as ⁇ Teeth To Move ⁇ , ⁇ Tooth Movement Restrictions ⁇ , and ⁇ Crossbite ⁇ , and then a neural network may proceed to impute values for one or more other orthodontic procedure parameters, such as ⁇ Correction to Class I ⁇ .
  • Systems of this disclosure may also impute missing oral care metrics values, based upon one or more available oral care metrics values for a patient case of interest (e.g., a patient case that is being treated). Such imputed values may then be provided to machine learning models for the prediction of 3D oral care representations (e.g., orthodontic setups prediction models for final setups or intermediate stages). Linear regression, principal component analysis and other machine learning models (such as neural networks) may be trained and deployed to impute missing values (e.g., such as oral care parameter values or oral care metrics values).
  • Techniques of this disclosure may train a machine learning (ML) model to generate imputed oral care argument values.
  • the methods involve receiving historical oral care argument values associated with past patient cases and generating training data sets from these values.
  • the ML model is then trained using the training data to predict imputed oral care argument values for a patient currently being treated.
  • the trained ML model is deployed for use.
  • the imputed oral care arguments may be provided to a trained ML model for generating oral care appliances.
  • the methods may receive ground truth oral care argument values and compare them with predicted oral care argument values generated by a partially trained ML model. Loss values may be determined based on the comparison, and the training process may involve modifying the weights of the partially trained ML model.
  • Historical oral care argument values may be rearranged into a vector format.
  • Masks may be applied to a vector, for example to flag dimensions of the vector to be ignored or to be imputed during ML model operation.
  • the ML model may include representation generation and imputation modules.
  • the representation generation module may, in some implementations, encode the vector into a lowerdimensional latent representation, which may then be provided to the imputation module for predicting oral care argument values.
  • the oral care arguments may include oral care parameters (e.g., orthodontic procedure parameters, restoration design parameters) or oral care metrics (e.g., orthodontic metrics, restoration design metrics).
  • oral care parameters e.g., orthodontic procedure parameters, restoration design parameters
  • oral care metrics e.g., orthodontic metrics, restoration design metrics
  • the ML model for oral care argument imputation may, in some implementations, contain one or more ML models, such as neural networks, support vector machines, regression models, principal component analysis models, decision trees, random forests, gradient boosting algorithms, Gaussian process-based models, k-nearest neighbors models, or Naive Bayes models.
  • ML models such as neural networks, support vector machines, regression models, principal component analysis models, decision trees, random forests, gradient boosting algorithms, Gaussian process-based models, k-nearest neighbors models, or Naive Bayes models.
  • the methods can be performed in near real-time during a patient encounter in a clinical context.
  • the methods may execute the trained imputation ML model to predict imputed oral care argument values for a patient case. This involves obtaining a set of oral care argument values for the patient case, flagging the oral care argument values to be imputed, providing available oral care arguments as input data to the ML model, and/or executing the model to generate imputed oral care argument values to substitute for the missing values.
  • methods of this disclosure may train machine learning (ML) models to generate (or impute) oral care argument values, based upon historical oral care argument values (e.g., vectors of such values) associated with past cohort patient cases.
  • the deployed ML model may impute missing oral care argument values associated with a patient case of interest (e.g., a patient case which is encountered in the course of clinical treatment).
  • Oral care arguments may include oral care metrics, oral care parameters, or tooth movement procedures.
  • Imputed oral care metrics may include orthodontic metrics, restoration design metrics or other measurements of the dimensions, shape or structure of 3D representations for use in generating 3D representations in digital oral care.
  • Imputed oral care parameters may include procedure parameters, dental restoration parameters, or other parameters which may be provided to a generative ML model to influence that model to generate a 3D oral care representation which may be used in the generation of an oral care appliance.
  • an ignore mask (or other specification of oral care arguments to ignore) may be provided to the predictive models (e.g., specifying which historical oral care arguments the ML models should leave-out of consideration when imputing new oral care argument values).
  • an imputation mask (or other specification of which oral care argument values are to be imputed) may be provided to the predictive models described herein.
  • the methods may, in some implementations, include generative models. The methods may be trained according to either supervised or unsupervised techniques, according to various implementations.
  • the ML models may contain neural networks (e.g., transformers or autoencoders), support vector machines (SVM), regression models (e.g., linear regression model or a logistic regression model), principal component analysis (PCA) models, decision trees, random forests, gradient boosting implementations, Gaussian process-based models, k-nearest neighbors (KNN) models, Naive Bayes models, or other ML models described herein.
  • a vector of oral care arguments may be provided to the encoder module of an encoderdecoder structure (e.g., a reconstruction autoencoder).
  • the encoder module may generate a latent vector representation that is a reduced-dimensionality representation of the execution-phase input data (e.g., of the masked vector of oral care arguments with was provided as the input).
  • the latent vector which is generated by the encoder module may be modified (e.g., by a latent vector modification module), and the modified latent vector may be reconstructed by the decoder module of the reconstruction autoencoder.
  • the reconstructed vector of oral care arguments may include one or more imputed values.
  • Inputs to the ML- based imputation methods described herein may include: available oral care parameter values, patient age data, patient health history data, a malocclusion type, patient demographic data, one or more arch-specific tooth counts, overjet data, overbite quantity data, a class relationship for the patient case of interest, a case difficulty classification, tooth crowding data for the patient case of interest, or a Bolton analysis associated with the patient case of interest.
  • Tooth crowding data may include collision depth information, tooth size information, or dental dimension data.
  • FIG. 1 shows a method of augmenting training data for use in training machine learning (ML) models of this disclosure.
  • FIG. 2 shows a method of training an ML model to impute oral care arguments.
  • FIG. 3 shows a method of using a trained ML model to impute oral care arguments.
  • the machine learning techniques described herein may receive a variety of input data, as described herein, including tooth meshes for one or both dental arches of the patient.
  • the tooth data may be presented in the form of 3D representations, such as meshes or point clouds. These data may be preprocessed, for example, by arranging the constituent mesh elements into lists and computing an optional mesh element feature vector for each mesh element. Such vectors may impart valuable information about the shape and/or structure of an oral care mesh to the machine learning models described herein.
  • Additional inputs may be received as the input to the machine learning models described herein, such as one or more oral care metrics.
  • Oral care metrics may be used for measuring one or more physical aspects of an oral care mesh (e.g., physical relationships within a tooth or between different teeth).
  • an oral care metric may be computed for either or both of a malocclusion oral care mesh example and/or a ground truth oral care mesh example which is then used in the training of the machine learning models described herein.
  • the metric value may be received as the input to the machine learning models described herein, as a way of training that model or those models to encode a distribution of such a metric over the several examples of the training dataset.
  • the network may then receive metric value(s) as input, to assist in training the network to link that inputted metric value to the physical aspects of the ground truth oral care mesh which is used in loss calculation.
  • Such a loss calculation may quantify the difference between a prediction and a ground truth example (e.g., between a predicted oral care mesh and a ground truth oral care mesh).
  • a ground truth example e.g., between a predicted oral care mesh and a ground truth oral care mesh.
  • the network techniques of this disclosure may, through the course of loss calculation and subsequent backpropagation, train the network to encode a distribution of a given metric.
  • one or more oral care arguments may be defined to specify one or more aspects of an intended 3D oral care representation (e.g., a 3D mesh, a polyline, a 3D point cloud or a voxelized geometry), which is to be generated using the machine learning models described herein that have been trained for that purpose.
  • oral care arguments may be defined to specify one or more aspects of a customized vector, matrix or any other numerical representation (e.g., to describe 3D oral care representations such as control points for a spline, an archform, a transform to place a tooth or appliance component relative to another 3D oral care representation, or a coordinate system), which is to be generated using the machine learning models described herein that have been trained for that purpose.
  • a customized vector, matrix or other numerical representation may describe a 3D oral care representation which conforms to the intended outcome of the treatment of the patient.
  • Oral care arguments may include oral care metrics or oral care parameters, among others.
  • Oral care arguments may specify one or more aspects of an oral care procedure - such as orthodontic setups prediction or restoration design generation, among others.
  • one or more oral care parameters may be defined which correspond to respective oral care metrics.
  • Oral care arguments can be provided as the input to the machine learning models described herein and be taken as an instruction to that module to generate an oral care mesh with the specified customization, to place an oral care mesh for the generation of an orthodontic setup (or appliance), to segment an oral care mesh, or to clean up an oral care mesh, to name a few examples.
  • This interplay between oral care metrics and oral care parameters may also apply to the training and deployment of other predictive models in oral care as well.
  • the predictive models of the present disclosure may, in some implementations, may produce more accurate results by the incorporation of one or more of the following inputs: archform information V, interproximal reduction (IPR) information U, tooth dimension information P, tooth gap information Q, latent capsule representations of oral care meshes T, latent vector representations of oral care meshes A, procedure parameters K (which may describe a clinician’s intended treatment of the patient), doctor preferences L (which may describe the typical procedure parameters chosen by a doctor), flags regarding tooth status M (such as for fixed or pinned teeth), tooth position information N, tooth orientation information O, tooth name/dental notation R, oral care metrics S (comprising of at least one of oral care metrics and restoration design metrics).
  • IPR interproximal reduction
  • Systems of this disclosure may, in some instances, be deployed in a clinical context (such as a dental or orthodontic office) for use by clinicians (e.g., doctors, dentists, orthodontists, nurses, hygienists, oral care technicians).
  • clinicians e.g., doctors, dentists, orthodontists, nurses, hygienists, oral care technicians.
  • Such systems which are deployed in a clinical context may enable clinicians to process oral care data (such as dental scans) in the clinic environment, or in some instances, in a "chairside" context (where the patient is present in the clinical environment).
  • a non-limiting list of examples of techniques may include: segmentation, mesh cleanup, coordinate system prediction, CTA trimline generation, restoration design generation, appliance component generation or placement or assembly, generation of other oral care meshes, the validation of oral care meshes, setups prediction, removal of hardware from tooth meshes, hardware placement on teeth, imputation of missing values, clustering on oral care data, oral care mesh classification, setups comparison, metrics calculation, or metrics visualization.
  • the execution of these techniques may, in some instances, enable patient data to be processed, analyzed and used in appliance creation by the clinician before the patient leaves the clinical environment (which may facilitate treatment planning because feedback may be received from the patient during the treatment planning process).
  • Systems of this disclosure may automate operations in digital orthodontics (e.g., setups prediction, hardware placement, setups comparison), in digital dentistry (e.g., restoration design generation) or in combinations thereof. Some techniques may apply to either or both of digital orthodontics and digital dentistry. A non-limiting list of examples is as follows: segmentation, mesh cleanup, coordinate system prediction, oral care mesh validation, imputation of oral care parameters, oral care mesh generation or modification (e.g., using autoencoders, continuous normalizing flows, transformers or denoising diffusion models), metrics visualization, appliance component placement or appliance component generation or the like. In some instances, systems of this disclosure may enable a clinician or technician to process oral care data (such as scanned dental arches).
  • the systems of this disclosure may enable orthodontic treatment planning, which may involve setups prediction as at least one operation.
  • Systems of this disclosure may also enable restoration design generation, where one or more restored tooth designs are generated and processed in the course of creating oral care appliances.
  • Systems of this disclosure may enable either or both of orthodontic or dental treatment planning, or may enable automation steps in the generation of either or both of orthodontic or dental appliances. Some appliances may enable both of dental and orthodontic treatment, while other appliances may enable one or the other.
  • a missing oral care metric (e.g., orthodontic metric or restoration design metric - examples of each of which are described herein) may be imputed.
  • one or more known metrics values may be used as the inputs to such an imputation engine.
  • one or more statistics of one or more metrics in a dataset of cohort patient cases may be used in such an imputation engine.
  • a setups prediction model may take orthodontic metrics as input.
  • such orthodontic metrics include one or more of overbite left cuspid or overbite_left_centralincisor. Upon encountering a patient case that lacks an upper (or lower) left cuspid, then the overbite left cuspid metric may be undefined.
  • Systems of this disclosure may be trained to impute a value to this “missing” overbite_left_cuspid metric, for example using the mode (or median or average or some other statistic) of the histogram of the overbite_left_cuspid metric over a dataset of cohort patient cases.
  • Two or more metrics may be plotted onto two or more orthogonal axes, e.g., in the case of XY axes, a metric A may be plotted to the y-axis, and a metric B may be plotted to the x-axis.
  • a value for metric A may be imputed using the techniques of this disclosure.
  • a principal component analysis (PCA) model may be used to impute one or more missing oral care metrics (e.g., metric A), for example, using one or more available oral care metrics values as input (e.g., metric B).
  • PCA may also be used to impute missing oral care parameters.
  • PCA may be used to compute one or more major or minor axes in a plot of metrics values, which may be described by, for example, a linear formula. In some instances, such a linear formula may be determined using linear regression (or another type of regression).
  • a known metric value (e.g., for a metric B) along the x-axis may be mapped to an imputed metric value for metric A, along the y-axis.
  • Such a technique may also be used to predict an unknown oral care parameter based on one or more known oral care parameter values.
  • Techniques of this disclosure may require a training dataset of hundreds or thousands of cohort patient cases, to ensure that the neural network is able to encode the distribution of patient cases which are likely to be encountered in clinical treatment.
  • a cohort patient case may include a set of tooth crown meshes, a set of tooth root meshes, or a data fde containing attributes of the case (e.g., a JSON file).
  • a typical example of a cohort patient case may contain up to 32 crown meshes (e.g., which may each contain tens of thousands of vertices or tens of thousands of faces), up to 32 root meshes (e.g., which may each contain tens of thousands of vertices or tens of thousands of faces), multiple gingiva mesh (e.g., which may each contain tens of thousands of vertices or tens of thousands of faces) or one or more JSON files which may each contain tens of thousands of values (e.g., objects, arrays, strings, real values, Boolean values or Null values).
  • values e.g., objects, arrays, strings, real values, Boolean values or Null values
  • aspects of the present disclosure can provide a technical solution to the technical problem of predicting, based upon patient treatment plan data and/or the patient’s dentition, one or more missing oral care arguments (e.g., for use in specifying an intended output of a generative oral care model - such as orthodontic setups generation or restoration design generation).
  • computing systems specifically adapted to perform oral care argument prediction for oral care appliance generation are improved.
  • the ML methods described herein are intrinsically linked to the underlying computing technology. The methods may encode the distribution of a large dataset of patient case data (e.g. , containing thousands, millions, or tens of millions of individual oral care argument values).
  • aspects of the present disclosure may need to be executed in a time-constrained manner, such as when an oral care appliance must be generated for a patient immediately after intraoral scanning (e.g., while the patient waits in the clinician’s office). As such, aspects of the present disclosure are necessarily rooted in the underlying computer technology of oral care argument imputation and cannot be performed by a human, even with the aid of pen and paper.
  • implementations of the present disclosure must be capable of: 1) training a machine learning model based upon thousands or millions of patient treatment plans (e.g., each comprising dozens or hundreds of oral care parameters); and 2) imputing, based on the machine learning model, one or more missing oral care parameters so that orthodontic setups may be generated for use in oral care appliance generation, and do so during the course of a short office visit.
  • This disclosure pertains to digital oral care, which encompasses the fields of digital dentistry and digital orthodontics.
  • This disclosure generally describes methods of processing three-dimensional (3D) representations of oral care data.
  • 3D representation is a 3D geometry.
  • a 3D representation may include, be, or be part of one or more of a 3D polygon mesh, a 3D point cloud (e.g., such as derived from a 3D mesh), a 3D voxelized representation (e.g., a collection of voxels - for sparse processing), or 3D representations which are described by mathematical equations.
  • 3D representation may describe elements of the 3D geometry and/or 3D structure of an object.
  • a first arch S 1 includes a set of tooth meshes arranged (e.g., using transforms) in their positions in the mouth, where the teeth are in the mal positions and orientations.
  • a second arch S2 includes the same set of tooth meshes from S 1 arranged (e.g., using transforms) in their positions in the mouth, where the teeth are in the ground truth setup positions and orientations.
  • a third arch S3 includes the same meshes as SI and S2, which are arranged (e.g., using transforms) in their positions in the mouth, where the teeth are in the predicted final setup poses (e.g., as predicted by one or more of the techniques of this disclosure).
  • S4 is a counterpart to S3, where the teeth are in the poses corresponding to one of the several intermediate stages of orthodontic treatment with clear fray aligners.
  • GDL geometric deep learning
  • RL reinforcement learning
  • VAE variational autoencoder
  • MLP multilayer perceptron
  • PT pose transfer
  • FDG force directed graphs
  • MLP Setups, VAE Setups and Capsule Setups each fall within the scope of Autoencoder Setups. Some implementations of MLP Setups may fall within the Scope of Transformer Setups.
  • Representation Setups refers to any of MLP Setups, VAE Setups, Capsule Setups and any other setups prediction machine learning model which uses an autoencoder to create the representation for at least one tooth.
  • setups prediction techniques of this disclosure is applicable to the fabrication of clear tray aligners and/or indirect bonding trays.
  • the setups predictions techniques may also be applicable to other products that involve final teeth poses, also.
  • a pose may comprise a position (or location) and a rotation (or orientation).
  • a 3D mesh is a data structure which may describe the geometry or shape of an object related to oral care, including but not limited to a tooth, a hardware element, or a patient’s gum tissue.
  • a 3D mesh may include one or more mesh elements such as one or more of vertices, edges, faces and combinations thereof.
  • mesh element may include voxels, such as in the context of sparse mesh processing operations.
  • Various spatial and structural features may be computed for these mesh elements and be provided to the predictive models of this disclosure, with the predictive models of this disclosure providing the technical advantage of improving data precision in the form of the models of this disclosure outputting more accurate predictions.
  • a patient’s dentition may include one or more 3D representations of the patient’s teeth (e.g., and/or associated transforms), gums and/or other oral anatomy.
  • An orthodontic metric may, in some implementations, quantify the relative positions and/or orientations of at least one 3D representation of a tooth relative to at least one other 3D representation of a tooth.
  • a restoration design metric may, in some implementations, quantify at least one aspect of the structure and/or shape of a 3D representation of a tooth.
  • An orthodontic landmark (OL) may, in some implementations, locate one or more points or other structural regions of interest on a 3D representation of a tooth.
  • An OL may, in some implementations, be used in the generation of an orthodontic or dental appliance, such as a clear tray aligner or a dental restoration appliance.
  • a mesh element may, in some implementations, comprise at least one constituent element of a 3D representation of oral care data.
  • mesh elements may include at least: vertices, edges, faces and voxels.
  • a mesh element feature may, in some implementations, quantify some aspect of a 3D representation in proximity to or in relation with one or more mesh elements, as described elsewhere in this disclosure.
  • Orthodontic procedure parameters may, in some implementations, specify at least one value which defines at least one aspect of planned orthodontic treatment for the patient (e.g., specifying desired target attributes of a final setup in final setups prediction).
  • Orthodontic Doctor preferences may, in some implementations, specify at least one typical value for an OPP, which may, in some instances, be derived from past cases which have been treated by one or more oral care practitioners.
  • Restoration Design Parameters may, in some implementations, specify at least one value which defines at least one aspect of planned dental restoration treatment for the patient (e.g., specifying desired target attributes of a tooth which is to undergo treatment with a dental restoration appliance).
  • Doctor Restoration Design Preferences may, in some implementations, specify at least one typical value for an RDP, which may, in some instances, be derived from past cases which have been treated by one or more oral care practitioners.
  • 3D oral care representations may include, but are not limited to: 1) a set of mesh element labels which may be applied to the 3D mesh elements of teeth/gums/hardware/appliance meshes (or point clouds) in the course of mesh segmentation or mesh cleanup; 2) 3D representation(s) for one or more teeth/gums/hardware/appliances for which shapes have been modified (e.g., trimmed, distorted, or filled-in) in the course of mesh segmentation or mesh cleanup; 3) one or more coordinate systems (e.g., describing one, two, three or more coordinate axes) for a single tooth or a group of teeth (such as a full arch - as with the LDE coordinate system); 4) 3D representation(s) for one or more teeth for which shapes have been modified or otherwise made suitable for use in
  • Systems of this disclosure may, in some instances, be deployed in a clinical context (such as a dental or orthodontic office) for use by clinicians (e.g., doctors, dentists, orthodontists, nurses, hygienists, oral care technicians).
  • clinicians e.g., doctors, dentists, orthodontists, nurses, hygienists, oral care technicians.
  • Such systems which are deployed in a clinical context may enable clinicians to process oral care data (such as dental scans) in the clinic environment, or in some instances, in a "chairside" context (where the patient is present in the clinical environment).
  • a non-limiting list of examples of techniques may include: segmentation, mesh cleanup, coordinate system prediction, CTA trimline generation, restoration design generation, appliance component generation or placement or assembly, generation of other oral care meshes, the validation of oral care meshes, setups prediction, removal of hardware from tooth meshes, hardware placement on teeth, imputation of missing values, clustering on oral care data, oral care mesh classification, setups comparison, metrics calculation, or metrics visualization.
  • the execution of these techniques may, in some instances, enable patient data to be processed, analyzed and used in appliance creation by the clinician before the patient leaves the clinical environment (which may facilitate treatment planning because feedback may be received from the patient during the treatment planning process).
  • the Setups Comparison tool may be used to compare the output of the GDL Setups model against ground truth data, compare the output of the RL Setups model against ground truth data, compare the output of the VAE Setups model against ground truth data and compare the output of the MLP Setups model against ground truth data.
  • the Metrics Visualization tool can enable a global view of the final setups and intermediate stages produced by one or more of the setups prediction models, with the advantage of enabling the selection of the best setups prediction model.
  • the Metrics Visualization tool furthermore, enables the computation of metrics which have a global scope over a set of intermediate stages. These global metrics may, in some implementations, be consumed as inputs to the neural networks for predicting setups (e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, among others). The global metrics may also be provided to FDG Setups.
  • GDL Setups e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, among others.
  • the global metrics may also be provided to FDG Setups.
  • the local metrics from this disclosure may, in some implementations, be consumed by the neural networks herein for predicting setups, with the advantage of improving predictive results.
  • the metrics described in this disclosure may, in some implementations, be visualized using the Metric Visualization tool.
  • the VAE and MAE models for mesh element labelling and mesh in-fdling can be advantageously combined with the setups prediction neural networks, for the purpose of mesh cleanup ahead of or during the prediction process.
  • the VAE for mesh element labelling may be used to flag mesh elements for further processing, such as metrics calculation, removal or modification.
  • flagged mesh elements may be provided as inputs to a setups prediction neural network, to inform that neural network about important mesh features, attributes or geometries, with the advantage of improving the performance of the resulting setups prediction model.
  • mesh in-filling may cause the geometry of a tooth to become more nearly complete, enabling the better functioning of a setups prediction model (i.e., improved correctness of prediction on account of better-formed geometry).
  • a neural network to classify a setup i.e., the Setups Classifier
  • the setups classifier tells that setups prediction neural network when the predicted setup is acceptable for use and can be provided to a method for aligner tray generation.
  • a Setups Classifier (e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups and FDG Setups, among others) may aid in the generation of final setups and also in the generation of intermediate stages.
  • a Setups Classifier neural network may be combined with the Metrics Visualization tool.
  • a Setups Classification neural network may be combined with the Setups Comparison tool (e.g., the Setup Comparison tool may output an indication of how a setup produced in part by the Setups Classifier compares to a setup produced by another setups prediction method).
  • the VAE for mesh element labelling may identify one or more mesh elements for use in a metrics calculation. The resulting metrics outputs may be visualized by the Metrics Visualization tool.
  • the Setups Classifier neural network may aid in the setups prediction technique described in U.S. Patent Application No. US20210259808A1 (which is incorporated herein by reference in its entirety) or the setups prediction technique described in PCT Application with Publication No. WO2021245480A1 (which is incorporated herein by reference in its entirety) or in PCT Application No. PCT/IB2022/057373 (which is incorporated herein by reference in its entirety).
  • the Setups Classifier would help one or more of those techniques to know when the predicted final setup is most nearly correct.
  • the Setups Classifier neural network may output an indication of how far away from final setup a given setup is (i.e., a progress indicator).
  • the latent space embedding vector(s) from the reconstruction VAE can be concatenated with the inputs to the setups prediction neural network described in WO2021245480A1.
  • the latent space vectors can also be incorporated as inputs to the other setups prediction models: GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups and Diffusion Setups, among others.
  • the advantage is to impart the reconstruction characteristics (e.g., latent vector dimensions of a tooth mesh) to that neural network, hence improving the generated setups prediction.
  • the various setups prediction neural networks of this disclosure may work together to produce the setups required for orthodontic treatment.
  • the GDL Setups model may produce a final setup, and the RL Setups model may use that final setup as input to produce a series of intermediate stages setups.
  • the VAE Setups model (or the MLP Setups model) may create a final setup which may be used by an RL Setups model to produce a series of intermediate stages setups.
  • a setup prediction may be produced by one setups prediction neural network, and then taken as input to another setups prediction neural network for further improvements and adjustments to be made. In some implementations, such improvements may be performed in iterative fashion.
  • a setups validation model such as the model disclosed in US Provisional Application No. US63/366495, may be involved in this iterative setups prediction loop.
  • a setup may be generated (e.g., using a model trained for setups prediction, such as GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups and FDG Setups, among others), then the setup undergoes validation. If the setup passes validation, the setup may be outputted for use. If the setup fails validation, the setup may be sent back to one or more of the setups prediction models for corrections, improvements and/or adjustments.
  • the setups validation model may output an indication of what is wrong with the setup, enabling the setups generation model to make an improved version upon the next iteration. The process iterates until done.
  • GDL Setups Setups Classification, Reinforcement Learning (RL) Setups, Setups Comparison, Autoencoder Setups (VAE Setups or Capsule Setups), VAE Mesh Element Labeling, Masked Autoencoder (MAE) Mesh Infilling, Multi-Layer Perceptron (MLP) Setups, Metrics Visualization, Imputation of Missing Oral Care Parameters Values, Tooth Classification Using Latent Vector, FDG Setups, Pose Transfer Setups, Restoration Design Metrics Calculation, Neural Network Techniques for Dental Restoration and/or Orthodontics (e.g, 3D Oral Care Representation Generation or Modification using Transformers), Landmark-based (LB) Setups, Diffusion Setups, Imputation of Tooth Movement Procedures, Capsule Autoencoder Segmentation, D
  • Oral care parameters may include one or more values that specify orthodontic procedure parameters, or restoration design parameters (RDP), as described herein.
  • Oral care parameters may define one or more intended aspects of a 3D oral care representation, and may be provided to an ML model to promote that ML model to generate output which may be used in the generation of oral care appliances that are suitable for the treatment of a patient.
  • Other types of values include doctor preferences and restoration design preferences, as described herein.
  • Doctor preferences and restoration design preferences may define the typical treatment choices or practices of a particular clinician. Restoration design preferences are subjective to a particular clinician, and so differ from restoration design parameters.
  • doctor preferences or restoration design preferences may be computed by unsupervised means, such as clustering, which may determine the typical values that a clinician uses in patient treatment. Those typical values may be stored in a datastore, and recalled to be provided to an automated ML model as default values (e.g., default values which may be modified before execution of the model).
  • one clinician may prefer one value for a restoration design parameter (RDP), while another clinician may prefer a different value for that RDP, when faced with a similar diagnosis or treatment protocol.
  • RDP restoration design parameter
  • Procedure parameters and/or doctor preferences may, in some implementations, be provided to a setups prediction model for orthodontic treatment, for the purpose of improving the customization of the resulting orthodontic appliance.
  • Restoration design parameters and doctor restoration preferences may in some implementations be used to design tooth geometry for use in the creation of a dental restoration appliance, for the purpose of improving the customization of that appliance.
  • some implementations of ML prediction models of this disclosure, in orthodontic treatment may also take as input a setup (e.g., an arrangement of teeth).
  • an ML prediction model of this disclosure may take as input a final setup (i.e., final arrangement of teeth), such as in the case of a prediction model trained to generate intermediate stages.
  • these preferences are referred to as doctor restoration preferences, but it is intended to be used in a non-limiting sense. Specifically, it should be appreciated that these preferences may be specified by any treating or otherwise appropriate medical professional and are not intended to be limited to doctor preferences per se (i.e., preferences from someone in possession of an M.D. or equivalent degree).
  • An oral care professional or clinician such as a dentist or orthodontist, may specify information about patient treatment in the form of a patient-specific set of procedure parameters.
  • an oral care professional may specify a set of general preferences (aka doctor preferences) for use over a broad range of cases, to use as default values in the set of procedure parameters specification process.
  • Oral care parameters may in some implementations be incorporated into the techniques described in this disclosure, such as one or more of GDL Setups, VAE Setups, RL Setups, Setups Comparison, Setups Classification, VAE Mesh Element Labelling, MAE Mesh In-Filling, Validation Using Autoencoders, Imputation of Missing Procedure Parameters Values, Metrics Visualization, or FDG Setups.
  • One or more of these models may take as input one or more procedure parameters vector K and/or one or more doctor preference vectors L.
  • one or more of these models may introduce to one or more of a neural network’s hidden layers one or more procedure parameters vector K and/or one or more doctor preferences vectors L.
  • one or more of these models may introduce either or both of K and L to a mathematical calculation, such as a force calculation, for the purpose of improving that calculation and the ultimate customization of the resulting appliance to the patient.
  • Some implementations of a neural network for predicting a setup may incorporate information from an oral care professional (aka doctor). This information may influence the arrangement of teeth in the final setup, bringing the positions and orientations of the teeth into conformance with a specification set by the doctor, within tolerances.
  • oral care parameters may be fed directly into the generator network as a separate input alongside the mesh data.
  • oral care parameters may be incorporated into the feature vector which is computed for each mesh element before the mesh elements are input to the generator for processing.
  • Some implementations of a VAE Setup model may incorporate oral care parameters into the setups predictions.
  • the procedure parameters K and/or the doctor preference information L may be concatenated with the latent space vector C.
  • a doctor’s preferences (e.g., in an orthodontic context) and/or doctor’s restoration preferences may be indicated in a treatment form, or they could be based upon characteristics in treatment plans such as final setup characteristics (e.g., amount of bite correction or midline correction in planned final setups), intermediate staging characteristics (e.g., treatment duration, tooth movement protocols, or overcorrection strategies), or outcomes (e.g., number of revisions/refinements).
  • Orthodontic procedure parameters may specify one or more of the following (with possible values shown in ⁇ ).
  • Non-limiting categorical values for some example OPP are described below.
  • a real value may be specified for one or more of these OPP.
  • the Overbite OPP may specify a quantify of overbite (e.g., in millimeters) which is desired in a setup, and may be received as input of a setups prediction model to provide that setups prediction model information about the amount of overbite which is desired in the setup.
  • Some implementations may specify a numerical value for the Overjet OPP, or other OPP.
  • one or more OPP may be defined which correspond to one or more orthodontic metrics (OM).
  • OM orthodontic metrics
  • a numerical value may be specified for such an OPP, for the purpose of controlling the output of a setups prediction model.
  • Tooth Movement Restrictions for each tooth, indicate if tooth is ⁇ DoNotMove, Missing, ToBeExtracted, Primary /Erupting, Clear ⁇
  • doctor can specify an archform - selected from a set of options or custom-designed]
  • Other orthodontic procedure parameters may be defined, such as those which may be used to place standardized brackets at prescribed occlusal heights on the teeth.
  • one or more orthodontic procedure parameters may be defined to specify at least one of the 2 nd and 3 rd order rotation angles to be applied to a tooth (i.e., angulation and torque, respectively), which may enable a target setup arrangement where crown landmarks lie within a threshold distance of a common occlusal plane, for example.
  • one or more orthodontic procedure parameters may be defined to specify the position in global coordinates where at least one landmark (e.g., a centroid) of a tooth crown (or root) is to be placed in a setup arrangement of teeth.
  • an oral care parameter may be defined which corresponds to an oral care metric.
  • an orthodontic procedure parameter may be defined which corresponds to an orthodontic metric (e.g., to specify at the input of a setups prediction model an amount of a certain metric which is desired to appear in a predicted setup).
  • Doctor preferences may differ from orthodontic procedure parameters in that doctor preferences pertain to an oral care provider and may comprise of the means, modes, medians, minimums, or maximums (or some other statistic) of past settings associated with an oral care provider’s treatment decisions on past orthodontic cases.
  • Procedure parameters may pertain to a specific patient, and describe the needs of a particular patient’s treatment.
  • Doctor preferences may pertain to a doctor and the doctor’s past treatment practices, whereas procedure parameters may pertain to the treatment of a particular patient.
  • Doctor preferences (or “treatment preferences”) may specify one or more of the following (with some non-limiting possible values shown in ⁇ ). Other possible values are found elsewhere in this disclosure.
  • Doctor preferences may specify one or more of the following (with other possible values found elsewhere in this disclosure).
  • Protocol_A ⁇ protocol_A, protocol_B, protocol_C ⁇
  • Archform information V may be provided as an input to any of the GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups and Diffusion Setups prediction neural networks. In some implementations, archform information V may be provided directly to one or more internal neural network layers in one or more of those setups applications.
  • the additional procedure parameters may include text descriptions of the patient’s medical condition and of the intended treatment.
  • Such text descriptions may be analyzed via natural language processing operations, including tokenization, stop word removal, stemming, n-gram formation, text data vectorization, bag of words analysis, term frequency inverse document frequency (TF-IDF) analysis, sentiment analysis, naive Bayes classification, and/or logistic regression classification.
  • TF-IDF term frequency inverse document frequency
  • the outputs of such analysis techniques may be used as input to one or more of the neural networks of this disclosure with the advantage of customizing and improving the predicted outputs (e.g., the predicted setups or predicted mesh geometries).
  • a dataset used for training one or more of the neural network models of this disclosure may be filtered conditionally on one or more of the orthodontic procedure parameters described in this section.
  • patient cases which exhibit outlier values for one or more of these procedure parameters may be omitted from a dataset (alternatively used to form a dataset) for training one or more of the neural networks of this disclosure.
  • One or more procedure parameters and/or doctor preferences may be provided to a neural network during training. In this manner the neural network may be conditioned on the one or more procedure parameters and/or doctor preferences.
  • Examples of such neural networks include a conditional generative adversarial network (cGAN) and/or a conditional variational autoencoder (cVAE), either of which may be used for the various neural network-based applications of this disclosure.
  • the restoration treatment of the patient may involve the specification of one or more of the following: restoration guidelines, restoration design parameters, and/or restoration rules for modifying one or more aspects of a patient’s dentition.
  • restoration guidelines for modifying one or more aspects of a patient’s dentition.
  • restoration rules for modifying one or more aspects of a patient’s dentition.
  • One or more of many possible factors may be considered in designing a 3D restoration, whether from an esthetic standpoint and/or from a technical standpoint. For instance, from an esthetic standpoint, the dental and facial midlines and angulation may provide overall guidance, as does the amount of tooth seen by others when the lips are at rest and/or when smiling. After those criteria are considered, a set of “golden proportions” may also inform the esthetic design of overall tooth sizes.
  • Tooth-to-tooth proportion may be configured to reflect these “golden proportions,” which are: 1.618: 1.0:0.618 for the central incisor, lateral incisor and the canine, respectively.
  • a real value may be specified for one or more of the RDP and be received at the input of a dental restoration design prediction model (e.g., a machine learning model to predict the final tooth shape at the completion of restoration design).
  • one or more RDP may be defined which correspond to one or more restoration design metrics (RDM).
  • Tooth i.e., malocclusion
  • orientation i.e., rotation and inclination
  • tooth-to-tooth proportion a variety of tooth shapes may be leveraged to match the overall esthetic of the patient's face and smile.
  • tooth shapes may be generally rectangular with squared edges, or they may be generally ovoid with rounded edges.
  • tooth-to-tooth proportions may be manipulated to invoke a different overall esthetic.
  • 3D Dental CAD programs often provide libraries of different tooth “styles” to choose from and offer designers the ability to tune the result to best match the esthetic and medical requirements of the doctor and patient.
  • Tooth length, width, and width-to-length esthetic relationships may be specified for one or more teeth.
  • the length for a maxillary central incisor may be set to 11 mm, and the width- to-length esthetic relationship may be set to either 70% or 80%.
  • the lateral incisors may be between 1.0 mm and 2.5 mm shorter than the central incisors.
  • Canine teeth may, in some instances, be between 0.5 mm and 1.0 mm shorter than the central incisors. Other proportions and measurements are possible for various teeth.
  • restorations produced from a given material must be of sufficient thickness to have the necessary mechanical strength required for long term use.
  • the tooth width and shape must be designed to provide a suitable contact with the adjacent teeth.
  • a neural network engine of this disclosure may incorporate as an input one or more of accepted “golden proportion” guidelines for the size of teeth, accepted “ideal” tooth shapes, patient preferences, practitioner preferences, etc.
  • Restoration design parameters may be used to encode aspects of smile design guidelines described herein, such as parameters which pertain to the intended dimensions of a restored tooth.
  • Nonlimiting examples of restoration design parameters are described in T able 1.
  • Restoration design parameters are intended as instructions and/or specifications which describe the shape and/or form that one or more teeth should assume after the completion of dental restoration treatment.
  • One or more RDP may be received by a neural network or other machine learning or optimization algorithm for dental restoration design, with the advantage of providing guidance to that optimization algorithm.
  • Some neural networks may be trained for dental restoration design generation, such as some examples of a GAN or an autoencoder.
  • a dental restoration design may be used to define the target shapes of one or more teeth for the generation of a dental restoration appliance.
  • a dental restoration design may be used to define the target teeth shapes for the generation of one or more veneers.
  • a partial list of tooth dimensions may include: length, width, height, circumference, diameter, diagonal measure, volume — any of which dimensions may be normalized in comparison to another tooth or teeth.
  • one or more restoration design parameters may be defined which pertain to a gap between two or more teeth, and the amount, if any, of the gap which the patient wishes to remain after treatment (e.g., such as when a patient wishes to retain a small gap between the upper central incisors).
  • Additional restoration design parameters may include the parameters specified in the following table. In the event that one of these parameters contradicts another, the following order may determine precedence (i.e., let the first parameter in the following list be considered authoritative). If a parameter value is not specified, then that parameter may be ignored.
  • default values may be introduced for one or more parameters. Such default values may be determined, for example, through clustering of prior patient cases.
  • a golden proportion guideline may specify one or more numbers pertaining to the widths of adjacent teeth, such as: ⁇ 1.6, 1, 0.6 ⁇ .
  • Tooth-to-tooth proportions may also be defined between other pairs of teeth as well. Proportions may be made relative to tooth widths, heights, diagonals, etc. Angle lines, incisal angles and buccal contours may describe primary aspects of tooth macro shape. Mamelon grooves may be vertical macro textures on the front of a tooth, and sometimes may take a V-shape. Striations or perikymata may be a horizontal micro texture on the teeth. Symmetry may be generally desired. There may be differences between male and female patients. Doctor Restoration Design Preferences (DRDP):
  • Parameters may be defined to encode doctor restoration design preferences (DRDP), as pertains to various use case scenarios. These use case scenarios, may reflect information about the treatment preferences of one or more doctors, and directly affect the characteristics of one or more teeth in a dental restoration design or a veneer.
  • DRDP may describe preferred or habitually involved values or ranges of values of RDP for a doctor or other treating medical professional. In some instances, such value or ranges of values may be derived from historical patient cases that were treated by that doctor or medical professional.
  • a DRDP may be defined which is derived from a RDP (e.g., such as Width- to-length esthetic relationship) or from a RDM.
  • Representation generation neural networks based on autoencoders, U-Nets, transformers, other types of encoder-decoder structures, convolution and/or pooling layers, or other models may benefit from the use of oral care arguments (e.g., oral care metrics or oral care parameters).
  • oral care metrics e.g., orthodontic metrics or restoration design metrics
  • Each oral care metric describes distinct information about the patient’s dentition that may not be redundantly present in other input data that are provided to the neural network.
  • an “Overbite” metric may quantify the overlap between the upper and lower central incisors along the vertical Z-axis, information which may not otherwise, in some implementations, be readily ascertainable by a traditional neural network.
  • the oral care metrics provide refined information about the patient’s dentition that a traditional neural network (e.g., a representation generation neural network) may not be adequately trained or configured to extract the oral care metrics described herein.
  • a neural network which is specifically trained to generate oral care metrics may overcome such a shortcoming, because, for example loss may be computed in such a way as to facilitate accurate oral care metrics prediction.
  • Mesh oral care metrics may provide a processed version of the structure and/or shape of the patient’s dentition, data which may not otherwise be available to the neural network. This processed information is often more accessible, or more amenable for encoding by the neural network.
  • a system implementing the techniques disclosed herein has been utilized to run a number of experiments on 3D representations of teeth. For example, oral care metrics have been provided to a representation generation neural network which is based on a U-Net model. Based on experiments, it was found that systems using oral care metrics (e.g., “Overbite”, “Overjet” and “Canine Class Relationship” metrics) were at least 2.5% more accurate than systems that did not. Furthermore, training converges more quickly when the oral care metrics are used.
  • W02020026117A1 lists some examples of Orthodontic Metrics (OM). Further examples are disclosed herein.
  • OM Orthodontic Metrics
  • the orthodontic metrics may be used to quantify the physical arrangement of an arch of teeth for the purpose of orthodontic treatment (as opposed to restoration design metrics - which pertain to dentistry and describe the shape and/or form of one or more pre-restoration teeth, for the purpose of supporting dental restoration). These orthodontic metrics can measure how badly maloccluded the arch is, or conversely the metrics can measure how correctly arranged the teeth are.
  • the GDL Setups model may incorporate one or more of these orthodontic metrics, or other similar or related orthodontic metrics.
  • such orthodontic metrics may be incorporated into the feature vector for a mesh element, where these perelement feature vectors are fed into the setups prediction network as inputs.
  • such orthodontic metrics may be directly consumed by a generator, an MLP, a transformer, or other neural network as direct inputs (such as presented in one or more input vectors of real numbers S, such as described elsewhere in this disclosure.
  • Such orthodontic metrics may be consumed by an encoder structure or by a U-Net structure (in the case of GDL Setups).
  • Such orthodontic metrics may be consumed by an autoencoder, variational autoencoder, masked autoencoder or regularized autoencoder (in the case of the VAE Setups, VAE Mesh Element Labelling, MAE Mesh In-Filling).
  • Such orthodontic metrics may be consumed by a neural network which generates action predictions as a part of a reinforcement learning RL Setups model.
  • Such orthodontic metrics may be consumed by a classifier which applies a label to a setup arch (e.g., labels such as mal, staging or final setup).
  • a label e.g., labels such as mal, staging or final setup.
  • This description is non-limiting, as the orthodontic metrics may also be incorporated in other ways into the various techniques of this disclosure.
  • the various loss calculations of the present disclosure may, in some examples, incorporate one or more orthodontic metrics, with the advantage of improving the correctness of the resulting neural network.
  • An orthodontic metric may be used to directly compare a predicted example to the corresponding ground truth example (such as is done with the metrics in the Setups Comparison description).
  • one or more orthodontic metrics may be taken from this section and incorporated into a loss computation.
  • Such an orthodontic metric may be computed on the predicted example, and then the orthodontic metric would also be computed on the ground truth example. These two orthodontic metrics results would then be consumed by the loss computation, with the advantage of improving the performance of the resulting neural network.
  • one or more orthodontic metrics pertaining to the alignment of two or more adjacent teeth may be computed and incorporated into a loss function, for example, to train, at least in part, a setups prediction neural network.
  • such an orthodontic metric may influence the network to align the mesial surface of one tooth with distal surface of adjacent tooth.
  • Backpropagation is an exemplary algorithm by which a neural network may be trained using one or more loss values.
  • one or more orthodontic metrics may be used to evaluate the predicted output of a neural network, such as a setups prediction. Such a metric(s) may enable the training algorithm to determine how close the predicted output is to an acceptable output, for example, in a quantified sense. In some implementations, this use of an orthodontic metric may enable a loss value to be computed which does not depend entirely on a comparison to a ground truth. In some implementations, such a use of an orthodontic metric may enable loss calculation and network training to proceed without the need for a comparison against a ground truth example.
  • loss may be computed based on a general principle or specification for the predicted output (such as a setup) rather than tying loss calculation to a specific ground truth example (which may have been defined by a particular doctor, clinician, or technician, whose treatment philosophy may differ from that of other technicians or doctors).
  • a specific ground truth example which may have been defined by a particular doctor, clinician, or technician, whose treatment philosophy may differ from that of other technicians or doctors.
  • such an orthodontic metric may be defined based on a FID (Frechet Inception Distance) score.
  • An orthodontic metric that can be computed using tensors may be especially advantageous when training one of the neural networks of the present disclosure, because tensor operations may promote efficient computations. The more efficient (and faster) the computation, the faster the rate at which training can proceed.
  • an error pattern may be identified in one or more predicted outputs of an ML model (e.g., a transformation matrix for a predicted tooth setup, a labelling of mesh elements for mesh cleanup, an addition of mesh elements to a mesh for the purpose of mesh in-filling, a classification label for a setup, a classification label for a tooth mesh, etc.).
  • One or more orthodontic metrics may be selected to become an input to the next round of ML model training, to address any pattern of errors or deficiencies which may be identified in the one or more predicted outputs.
  • Some OM may be defined relative to an archfrom coordinate frame, the LDE coordinate system.
  • a point may be described using an LDE coordinate frame relative to an archform, where L, D and E correspond to: 1) Length along the curve of the archform, 2) Distance away from the archform, and 3) distance in the direction perpendicular to the L and D axes (which may be termed Eminence), respectively.
  • OM and other techniques of the present disclosure may compute collisions between 3D representations (e.g., of oral care objects, such as teeth). Such collisions may be computed as at least one of: 1) penetration distance between 3D tooth representations, 2) count of overlapping mesh elements between 3D tooth representations, and 3) volume of overlap between 3D tooth representations.
  • an OM may be defined to quantify the collision of two or more 3D representations of oral care structures, such as teeth.
  • Some optimization algorithms, such as setups prediction techniques may seek to minimize collisions between oral care structures (such as teeth).
  • a 3D tooth orientation vector may be calculated using the tooth's mesial-distal axis.
  • a 3D vector which may be tangent vector to the archform at the position of the tooth may also be calculated.
  • the XY components i.e., which may be 2D vectors
  • Cosine similarity may be used to calculate the 2D orientation difference (angle) between the archform tangent and the tooth's mesial-distal axis.
  • the absolute difference may be calculated between each tooth’s X-coordinate and the global coordinate reference frame’s X-axis.
  • This delta may indicate the arch asymmetry for a given tooth pair.
  • the result of such a calculation may be the mean X-axis delta of one or more tooth-pairs from the arch. This calculation may, in some implementations, be performed relative to the Y-axis with y-coordinates (and/or relative to the Z axis with Z-coordinates).
  • Archform D-axis Differences May compute the D dimension difference (i.e., the positional difference in the facial-lingual direction) between two arch states, for one or more teeth. May, in some implementations, return a dictionary of the D-direction tooth movement for each tooth, with tooth UNS number as the key. May use the LDE coordinate system relative to an archform.
  • Archform (Lower) Length Ratio - May compute the ratio between the current lower arch length and the arch length as it was in the original maloccluded lower arch.
  • Archform (Upper) Length Ratio - May compute the ratio between the current upper arch length and the arch length as it was in the original maloccluded upper arch.
  • Archform Parallelism (Full arch) - For at least one local tooth coordinate system origin in the upper arch, the one or more nearest origins (e.g., tooth local coordinate system origins) in the lower arch.
  • the two nearest origins may be used. May compute the straight line distance from the upper arch point to the line formed between the origins of the two teeth in the opposing (lower) arch. May return the standard deviation of the set of “point-to-line” distances mentioned above, where the set may be composed of the point-to-line distances for each tooth in the arch.
  • This metric may share some computational elements with the archform parallclism global orthodontic metric, except that this metric may input the mean distance from a tooth origin to the line formed by the neighboring teeth in opposing arches (e.g., a tooth in the upper arch and the corresponding tooth in the lower arch). The mean distance may be computed for one or more such pairs of teeth. In some implementations, this may be computed for all pairs of teeth. Then the mean distance may be subtracted from the distance that is computed for each tooth pair. This OM may yield the deviation of a tooth from a “typical” tooth parallelism in the arch.
  • Buccolingual Inclination For at least one molar or premolar, find the corresponding tooth on the opposite side of the same arch (i.e., for a tooth on the left side of the arch, find the same type of tooth on the right side and vice versa).
  • This OM may compute an n-element list for each tooth (e.g. n may equal 2).
  • Such an n-element vector may be computed for each molar and each premolar in the upper and lower arches.
  • the buccal cusps maybe identified on the molars and premolars on each of the left and right sides of the arch. Draw a line between the buccal cusps of the left tooth and the buccal cusps on the right tooth. Make a plane using this line and the z-axis of the arch. The lingual cusps may be projected onto the plane (i.e., at this point the angle of inclination may be determined). By performing an additional projection, the approximate vertical distance between the lingual cusps and the buccal cusps may be computed. This distance may be used as the buccolingual inclination OM.
  • Canine Overbite The upper and lower canines may be identified.
  • the first premolar for the given side of the mouth may be identified.
  • a distance may be computed between the upper canine and the lower canine, and also between the upper pre-molar and the lower pre-molar.
  • the average (or median, or mode or some other statistic) may be computed for the measured distances.
  • the z-component of this result indicates the degree of overbite.
  • Overbite may be computed between any tooth in one arch and the corresponding tooth in the other arch.
  • Canine Overjet Contact - May calculate the collisions (e.g., collision distances) between pairs of canines on opposing arches.
  • Canine Overjet Contact KDE - May take an orthodontic metric score for the current patient case as input, and may convert that score into to a log-likelihood using a previously trained kernel density estimation (KDE) model or distribution. This operation may yield information about where in the distribution of "typical" values this patient case lies.
  • KDE kernel density estimation
  • Canine Overjet - This OM may share some computational steps with the canine overbite OM.
  • average distances may be computed.
  • the distance calculation may compute the Euclidean distance of the XY components of a tooth in the upper arch and a tooth in the lower arch, to yield ovcrjct (i.e., as opposed to computing the difference in Z-components, as may be performed for canine overbite).
  • Overjet may be computed between any tooth in one arch and the corresponding tooth in the other arch.
  • Canine Class Relationship (also applies to first, second and third molars) -
  • This OM may, in some implementations comprise two functions (e.g., written in Python).
  • get_canine_landmarks() Get landmarks for each tooth which may be used to compute the class relationship, and then, in some implementations, map those landmarks onto the global coordinate space so that measurements may be made between teeth.
  • class_relationship_score_by_side() May compute the average position of at least one landmark on at least one tooth in the lower arch, and may compute the same for the upper arch.
  • This OM may compute how far forward or behind the tooth is positioned on the 1-axis relative to the tooth or teeth of interest in the opposing arch.
  • Crossbite - Fossa in at least one upper molar may be located by finding the halfway point between distal and mesial marginal ridge saddles of the tooth.
  • a lower molar cusp may he between the marginal ridges of the corresponding upper molar.
  • This OM may compute a vector from the upper molar fossa midpoint to the lower molar cusp. This vector may be projected onto the d-axis of the archform, yielding a lateral measure of distance from the cusp to the fossa. This distance may define the crossbite magnitude.
  • This OM may identify the leftmost and rightmost edges of a tooth, and may identify the same for that tooth’s neighbor.
  • the OM may then draw a vector from the leftmost edge of the tooth to the leftmost edge of the tooth’s neighbor.
  • the OM may then draw a vector from the rightmost edge of the tooth to the rightmost edge of the tooth’s neighbor.
  • the OM may then calculates the linear fit error between the two vectors.
  • Such a calculation may involve making two vectors:
  • Vec tooth right_tooths_leftside to left_tooths_leftside
  • Vec_neighbor right_tooths_rightside to left_tooths_leftside
  • EdgeAlignment score 1 - abs(dot(Vec_tooth, Vec_neighbor)) ).
  • a score of 0 may indicate perfect alignment.
  • a score of 1 may mean perpendicular alignment.
  • Incisor Interarch Contact KDE - May identify the deviation of the IncisorlnterarchContact from the mean of a modeled distribution of such statistics across a dataset of one or more other patient cases.
  • Leveling - May compute a measure of leveling between a tooth and its neighbor.
  • This OM may calculate the difference in height between two or more neighboring teeth. For molars, this OM may use the midpoint between the mesial and distal saddle ridges as the height of the molar. For non-molar teeth, this OM may use the length of the crown from gums to tip. In some implementations, the tip may be the origin of the local coordinate space of the tooth. Other implementations may place the origin in other locations. A simple subtraction between the heights of neighboring teeth may yield the leveling delta between the teeth (e.g., by comparing Z components).
  • Midline - May compute the position of the midline for the upper incisors and/or the lower incisors, and then may compute the distance between them.
  • Molar Interarch Contact KDE - May compute a molar interarch contact score (i.e., a collision depth or other type of collision), and then may identify where that score lies in a pre-defined KDE (distribution) built from representative cases.
  • a molar interarch contact score i.e., a collision depth or other type of collision
  • this OM may identify one or more landmarks (e.g., mesial cusp, or central cusp, etc.). Get the tooth transform for that tooth. For each cusp on the current tooth, the cusp may be scored according to how well the cusp contacts the neighboring (corresponding) tooth in the opposite arch. A vector may be found from the cusp of the tooth in question to the vertical intersection point in the corresponding tooth of the opposing arch. The distance and/or direction (i.e., up or down) to the opposing arch may be computed. A list may be returned that contains the resulting signed distances, one for each cusp on the tooth in question.
  • landmarks e.g., mesial cusp, or central cusp, etc.
  • Overbite The upper and lower central incisors may be compared along the z-axis. The difference along the z-axis may be used as the overbite score.
  • Overjet The upper and lower central incisors may be compared along the y-axis. The difference along the y-axis may be used as the overjet score.
  • Molar Interarch Contact - May calculate the contact score between molars, and may use collision measurement(s) (such as collision depth).
  • Root Movement d The tooth transforms for an initial state and a next state may be received.
  • the archform axes at a point L along the archform may be computed.
  • This OM may return a distance moved along the d-axis. This may be accomplished by projecting the root pivot point onto the d-axis.
  • Root Movement 1 The tooth transforms for an initial state and a next state may be received.
  • the archform axes at a point L along the archform may be computed.
  • This OM may return a distance moved along the 1- axis. This may be accomplished by projecting the root pivot point onto the 1-axis.
  • Spacing May compute the spacing between each tooth and its neighbor.
  • the transforms and meshes for the arch may be received.
  • the left and right edges of each tooth mesh may be computed.
  • One or more points of interest may be transformed from local coordinates into the global arch coordinate frame.
  • the spacing may be computed in a plane (e.g., the XY plane) between each tooth and its neighbor to the "left”.
  • Torque - May compute torque (i.e., rotation around and axis, such as the x-axis). For one or more teeth, one or more rotations may be converted from Euler angles into one or more rotation matrices. A component (such as a x-component) of the rotations may be extracted and converted back into Euler angles. This x- component may be interpreted as the torque for a tooth. A list maybe returned which contains the torque for one or more teeth, and may be indexed by the UNS number of the tooth.
  • Oral care arguments may include oral care parameters, or oral care metrics.
  • oral care metrics include Orthodontic Metrics (OM) and Restoration Design Metrics (RDM).
  • OM Orthodontic Metrics
  • RDM Restoration Design Metrics
  • OM Orthodontic Metrics
  • RDM Restoration Design Metrics
  • One use case example is in the creation of one or more dental restoration appliances.
  • Another use case example is in the creation of one or more veneers (such as a zirconia veneer).
  • Some RDM may quantify the shape and/or other characteristics of a tooth.
  • Other RDM may quantify relationships (e.g., spatial relationships) between two or more teeth.
  • RDM differ from restoration design parameters (RDP) in that restoration design metrics define a current state of a patient's dentition, whereas restoration design parameters serve as specifications to a machine learning or other optimization model to generate desired tooth shapes and/or forms.
  • RDM describe the shapes of the teeth currently (e.g., in a starting or mal condition).
  • Restoration design parameters specify how an oral care provider (such as a dentist or dental technician) intends for the teeth to look after the completion of restoration treatment.
  • RDM and RDP may be provided to a neural network or other machine learning or optimization algorithm for the purpose of dental restoration.
  • RDM may be computed on the prerestoration dentition of the patient (i.e., the primary implementation).
  • RDM may be computed on the post-restoration dentition of the patient.
  • a restoration design may comprise one or more teeth and may be referred to as a restoration arch.
  • Restoration design generation may involve the generation of an improved geometry and/or structure of one or more teeth in a restoration arch.
  • RDM may be measured, for example, through locating landmarks in the teeth (or gums, hardware and/or other elements of the patient's dentition), and the measurements of distances between those landmarks, or otherwise made in relation to those landmarks.
  • one or more neural networks or other machine learning models may be trained to identify or extract one or more RDM from one or more 3D representations of teeth (or gums, hardware and/or other elements of the patient's dentition). Techniques of this disclosure may use RDM in various ways.
  • one or more neural networks or other machine learning models may be trained to classify or label one or more setups, arches, dentitions or other sets of teeth based at least in part on RDM.
  • ROMs form a part of the training data used for training these models.
  • a tooth mesh reconstruction autoencoder may be used in accordance with techniques of this disclosure are described below.
  • An autoencoder for restoration design generation is disclosed in US Provisional Application No. US63/366514.
  • This autoencoder e.g., a variational autoencoder or VAE
  • the encoder component of the autoencoder encodes that tooth mesh into a latent form (e.g., a latent vector).
  • Modifications may be applied to this latent vector (e.g., based on a mapping of the latent space through prior experiments), for the purpose of altering the geometry and/or structure of the eventual reconstructed mesh. Additional vectors may, in some implementations, be included with the latent vector (e.g., through concatenation), and the resulting concatenation of vectors may be reconstructed by way of the decoder component of the autoencoder into a reconstructed tooth mesh which is a facsimile of the input tooth mesh.
  • RDM and RDP may also be used as neural network inputs in the execution phase, in accordance with aspects of this disclosure.
  • one or more RDM may be concatenated with the input to the encoder, for the purpose of telling the encoder specific information about the input 3D tooth representation.
  • one or more RDM may be concatenated with the latent vector, before reconstruction, for the purpose of providing the decoder component with specific information about the input 3D tooth representation.
  • one or more restoration design parameters (RDP) may be concatenated with the input to the encoder component, for the purpose of providing the encoder specific information about the input 3D tooth representation.
  • one or more restoration design parameters (RDP) may be concatenated with the latent vector, before reconstruction, for the purpose of providing the decoder specific information about the input 3D tooth representation.
  • either or both of RDM and RDP may be introduced to the functioning of an autoencoder (e.g., a tooth reconstruction autoencoder), and serve to influence the geometry and/or structure of the reconstructed restoration design (i.e., influence the shape of the tooth on the output of the autoencoder).
  • an autoencoder e.g., a tooth reconstruction autoencoder
  • the variational autoencoder of US Provisional Application No. US63/366514 may be replaced by a capsule autoencoder (e.g., instead of encoding the tooth mesh into a latent vector, the tooth mesh is encoded into one or more latent capsules).
  • clustering or other unsupervised techniques may be performed on RDM to cluster one or more setups, arches, dentitions or other sets of teeth based on the restoration characteristics of the teeth.
  • Such clusters may be useful in treatment planning, as the clusters provide insight into categories of patients with different treatment needs. This information may be instructive to clinicians as they learn about possible treatment options.
  • best practices may be identified (such as default RDP values) for patient cases that fall into one or another cluster (e.g., as determined by a similarity measure, as in k-NN). After a new case is classified into a particular cluster, information about the relevant best practices may be provided to the clinician who is responsible for processing the case. Such default values may, in some instances, undergo further tuning or modifications.
  • Case Assignment Such clusters may be used to gain further insight into the kinds of patient cases which exist in a dataset. Analysis of such clusters may reveal that patient treatment cases with certain RDM values (or ranges of values) may take less time to treat (or alternatively more time to treat). Cases which take more time to treat (or are otherwise more difficult) may be assigned to experienced or senior technicians for processing. Cases which take less time to treat may be assigned to newer or less- experienced techniques for processing. Such an assignment may be further aided by finding correlations between RDM values for certain cases and the known processing durations associated with those cases.
  • RDM values or ranges of values
  • the following RDM may be measured and used in the creation of either or both of dental restoration appliances and veneers ⁇ veneers are a type of dental restoration appliance ⁇ , with the objective of making the resulting teeth natural looking. Symmetry is generally a preferred facet. There may be differences between patients based on demographic differences. The generation of dental restoration appliances may benefit from some or all of the following RDM. Shade and translucency may pertain, in particular, to the creation of veneers, though some implementations of dental restoration appliances may also consider this information.
  • Bilateral Symmetry and/or Ratios A measure of the symmetry between one or more teeth and one or more other teeth on opposite sides of the dental. For example, for a pair of corresponding teeth, a measure of the width of each tooth. In one instance, the one tooth is of normal width, and the other tooth is too narrow. In another instance, both teeth are of normal width.
  • Such ratios can be indicative of whether spatial symmetry exists (e.g., by measuring the ratio a/b on the left side and measuring the ratio a/b on the right side, then compare the left and right ratios).
  • spatial symmetry is "off 1 , the length, width and/or ratios may not match.
  • Such a ratio may, in some implementations, be computed relative to a standard.
  • a number of esthetic standards are available in the dental literature. Examples include Golden Proportion and Recurring Esthetic Dental Proportion.
  • spatial symmetry may be measured on a pair of teeth, where one tooth is on the right side of the arch, and the other tooth is on the left side of the arch.
  • Proportions of Adjacent Teeth Measure the width proportions of adjacent teeth as measured as a projection along an arch onto a plane (e.g., a plane that is situated in front of the patient's face).
  • the ideal proportions for use in the final restoration design can be, for example, the so-called golden proportions.
  • the golden proportions relate adjacent teeth, such as central incisors and lateral incisors. This metric pertains to the measuring of these proportions as the proportions exist in the prerestoration mal dentition.
  • the ideal golden proportions are 1.6, 1, 0.6, for the central incisor, lateral incisor and cuspid, on a particular side (either left or right) for a particular arch (e.g., the upper arch). If one or more of these proportion values is off (e.g., in the case of "peg laterals"), the patient may wish for dental restoration treatment to correct the proportions.
  • Arch Discrepancies A measure of any size discrepancies between the upper arch and lower arch, for example, pertaining to the widths of the teeth, for the purpose of dental restoration. For example, techniques of this disclosure may make adjacent tooth width proportion measurements in the upper arch and in the lower arch. In some implementations, Bolton analysis measurements may be made by measuring upper widths, lower widths, and proportions between those quantities. Arch discrepancies may be described in absolute measurements (e.g., in mm or other suitable units) or in terms of proportions or ratios, in various implementations.
  • Midline A measure of the midline of the maxillary incisors, relative to the midline of the mandibular incisors. Techniques of this disclosure may measure the midline of the maxillary incisors, relative to the midline of the nose (if data about nose location is available).
  • Proximal Contacts A measure of the size (area, volume, circumference, etc.) of the proximal contact between adjacent teeth.
  • the teeth touch along the mesial/distal surfaces and the gums fill in gingivally to where the teeth touch. Black triangles may form if the gum tissue fails to fill the space below the proximal contact.
  • the size of the proximal contact may get progressively shorter for teeth located farther towards the posterior of the arch.
  • the proximal contact would be long enough so that there is an appropriately sized incisal embrasure and the gum tissue fills in the area below or gingival to the contact.
  • Embrasure In some implementations, techniques of this disclosure may measure the size
  • an embrasure (area, volume, circumference, etc.) of an embrasure, the gap between teeth at either of the gingival or incisal edge.
  • techniques of this disclosure may measure the symmetry between embrasures on opposite sides of the arch.
  • An embrasure is based at least in part on the length of the length of the contact between teeth, and/or at least in part on the shape of the tooth. In some instances, the size of the embrasure may get progressively longer for teeth located farther towards the posterior of the arch.
  • Length and/or Width A measure of the length of a tooth relative to the width of that tooth.
  • This metric may reveal, for example, that a patient has long central incisors. Width and length are defined as: a) width - mesial to distal distance; b) length - gingival to incisal distance; c) other dimensions of tooth body - the portions of tooth between the gingival region and the incisal edge. In some implementations, either or both of a length and a width may be measured for a tooth and compared to the length and/or width of one or more teeth.
  • Tooth Morphology A measure of the primary anatomy of the tooth shape, such as line angles, buccal contours, and/or incisal angles and/or embrasures.
  • the frequency and/or dimensions may be measured.
  • the observed primary tooth shape aspects may be matched to one or more known styles.
  • Techniques of this disclosure may measure secondary anatomy of the tooth shape, such as mamelon grooves. For instance, the frequency and/or dimensions may be measured.
  • the observed secondary tooth shape aspects may be matched to one or more known styles.
  • techniques of this disclosure may measure tertiary anatomy of the tooth shape, such as perikymata or striations. For instance, the frequency and/or dimensions may be measured.
  • the observed tertiary tooth shape aspects may be matched to one or more known styles.
  • Shade and/or Translucency A measure of tooth shade and/or translucency. Tooth shade is often described by the Vita Classical or 3D Master shade guide. Tooth translucency is described by transmittance or a contrast ratio. Tooth shade and translucency may be evaluated (or measured) based on one or more of the following kinds of data pertaining to teeth: the incisal edge, incisal third, body and gingival third. The enamel layer translucency is general higher than the dentin or cementum layer. Shade and translucency may, in some implementations, be measured on a per-voxel (local) basis. Shade and translucency may, in some implementations, be measured on a per-area basis, such as an incisal area, tooth body area, etc. Tooth body may pertain to the portions of the tooth between the gingival region and the incisal edge.
  • Height of Contour A measure of the contour of a tooth. When viewed from the proximal view, all teeth have a specific contour or shape, moving from the gingival aspect to the incisal. This is referred to as the facial contour of the tooth. In each tooth, there is a height of contour, where that shape is the most pronounced. This height of contour changes from the teeth in the anterior of the arch to the teeth in the posterior of the arch. In some implementations, this measurement may take the form of fitting against a template of known dimensions and/or known proportions. In some implementations, this measurement may quantify a degree of curvature along the facial tooth surface. In some implementations, measure the location along the contour of the tooth where the height of the curvature is most pronounced. This location may be measured as a distance away from the gingival margin or a distance away from the incisal edge, or a percentage along the length of the tooth.
  • tooth shape-based inputs may be provided to a neural network for setups prediction, in accordance with aspects of this disclosure.
  • non-shape-based inputs can be used as inputs, such as a tooth name or designation, as it pertains to dental notation.
  • a vector R of flags may be provided to the neural network, where a ‘ 1’ value indicates that the tooth is present and a ‘0’ value indicates that the tooth is absent from the patient case (though other value-significance combinations are possible in accordance with aspects of this disclosure).
  • the vector R may comprise a 1-hot vector, where each element in the vector corresponds to a tooth type, name or designation.
  • Identifying information about a tooth can be provided to the predictive neural networks of this disclosure, with the advantage of enabling the neural network to become trained to handle different teeth in tooth-specific ways.
  • the setups prediction model may learn to make setups transformations predictions for a specific tooth designation (e.g., upper right central incisor, or lower left cuspid, etc.).
  • the mesh cleanup autoencoders either for labelling mesh element or for in-filling missing mesh data
  • the autoencoder may be trained to provide specialized treatment to a tooth according to that tooth’s designation, in this manner.
  • Tooth designation/name may be defined, for example, according to the Universal Numbering System, Palmer System, or the FDI World Dental Federation notation (ISO 3950).
  • a vector R may be defined as an optional input to the setups prediction neural networks of this disclosure, where there is a 0 in the vector element corresponding to each of the wisdom teeth, and a 1 in the elements corresponding to the following teeth: UR7, UR6, UR5, UR4, UR3, UR2, UR1, ULI, UL2, UL3, UL4, UL5, UL6, UL7, LL7, LL6, LL5, LL4, LL3, LL2, LL1, LR1, LR2, LR3, LR4, LR5, LR6, LR7 [0095]
  • the position of the tooth tip may be provided to a neural network for setups predictions.
  • one or more vectors S of the orthodontic metrics described elsewhere in this disclosure may be provided to a neural network for setups predictions.
  • the advantage is an improved capacity for the network to become trained to understand the state of a maloccluded setup and therefore be able to predict a more accurate final setup or intermediate stage.
  • the neural networks may take as input one or more indications of interproximal reduction (IPR) U, which may indicate the amount of enamel that is to be removed from a tooth during the course orthodontic treatment (either mesially or distally).
  • IPR information e.g., quantity of IPR that is to be performed on one or more teeth, as measured in millimeters, or one or more binary flags to indicate whether or not IPR is to be performed on each tooth identified by flagging
  • the vector(s) and/or capsule(s) resulting from such a concatenation may be provided to one or more of the neural networks of the present disclosure, with the technical improvement or added advantage of enabling that predictive neural network to account for IPR.
  • IPR is especially relevant to setups prediction methods, which may determine the positions and poses of teeth at the end of treatment or during one or more stages during treatment. It is important to account for the amount of enamel that is to be removed ahead of predicted tooth movements.
  • one or more procedure parameters K and/or doctor preferences vectors L may be introduced to a setups prediction model.
  • one or more optional vectors or values of tooth position N e.g., XYZ coordinates, in either tooth local or global coordinates
  • tooth orientation O e.g., pose, such as in transformation matrices or quaternions, Euler angles or other forms described herein
  • dimensions of teeth P e.g., length, width, height, circumference, diameter, diagonal measure, volume - any of which dimensions may be normalized in comparison to another tooth or teeth
  • distance between adjacent teeth Q may be used to describe the intended dimensions of a tooth for dental restoration design generation.
  • tooth dimensions P such as length, width, height, or circumference may be measured inside a plane, such as the plane that intersects the centroid of the tooth, or the plane that intersects a center point that is located midway between the centroid and either the incisal-most extent or the gingival-most extent of the tooth.
  • the tooth dimension of height may be measured as the distance from gums to incisal edge.
  • the tooth dimension of width may be measured as the distance from the mesial extent to the distal extent of the tooth.
  • the circularity or roundness of the tooth cross-section may be measured and included in the vector P. Circularity or roundness may be defined as the ratio of the radii of inscribed and circumscribed circles.
  • the distance Q between adjacent teeth can be implemented in different ways (and computed using different distance definitions, such as Euclidean or geodesic).
  • a distance QI may be measured as an averaged distance between the mesh elements of two adjacent teeth.
  • a distance Q2 may be measured as the distance between the centers or centroids of two adjacent teeth.
  • a distance Q3 may be measured between the mesh elements of closest approach between two adjacent teeth.
  • a distance Q4 may be measured between the cusp tips of two adjacent teeth. Teeth may, in some implementations, be considered adjacent within an arch. Teeth may, in some implementations, also be considered adjacent between opposing arches.
  • any of QI, Q2, Q3 and Q4 may be divided by a term for the purpose of normalizing the resulting value of Q.
  • the normalizing term may involve one or more of: the volume of a tooth, the count of mesh elements in a tooth, the surface area of a tooth, the cross-sectional area of a tooth (e.g., as projected into the XY plane), or some other term related to tooth size.
  • Other information about the patient’s dentition or treatment needs may be concatenated with the other input vectors to one or more of MLP, GAN, generator, encoder structure, decoder structure, transformer, VAE, conditional VAE, regularized VAE, 3D U-Net, capsule autoencoder, diffusion model, and/or any of the neural networks models listed elsewhere in this disclosure.
  • the vector M may contain flags which apply to one or more teeth.
  • M contains at least one flag for each tooth to indicate whether the tooth is pinned.
  • M contains at least one flag for each tooth to indicate whether the tooth is fixed.
  • M contains at least one flag for each tooth to indicate whether the tooth is pontic.
  • Other and additional flags are possible for teeth, as are combinations of fixed, pinned and pontic flags.
  • a flag that is set to a value that indicates that a tooth should be fixed is a signal to the network that the tooth should not move over the course of treatment.
  • the neural network loss function may be designed to be penalized for any movement in the indicated teeth (and in some particular cases, may be heavily penalized).
  • a flag to indicate that a tooth is pontic informs the network that the tooth gap is to be maintained, although that gap is allowed to move.
  • M may contain a flag indicating that a tooth is missing.
  • the presence of one or more fixed teeth in an arch may aid in setups prediction, because the one or more fixed teeth may provide an anchor for the poses of the other teeth in the arch (i.e., may provide a fixed reference for the pose transformations of one or more of the other teeth in the arch).
  • one or more teeth may be intentionally fixed, so as to provide an anchor against which the other teeth may be positioned.
  • a 3D representation (such as a mesh) which corresponds to the gums may be introduced, to provide a reference point against which teeth can be moved.
  • one or more of the optional input vectors K, L, M, N, O, P, Q, R, S, U and V described elsewhere in this disclosure may also be provided to the input or into an intermediate layer of one or more of the predictive models of this disclosure.
  • these optional vectors may be provided to the MLP Setups, GDL Setups, RL Setups, VAE Setups, Capsule Setups and/or Diffusion Setups, with the advantage of enabling the respective model to generate setups which better meet the orthodontic treatment needs of the patient.
  • such inputs may be provided, for example, by being concatenated with one or more latent vectors A which are also provided to one or more of the predictive models of this disclosure.
  • such inputs may be introduced, for example, by being concatenated with one or more latent capsules T which are also provided to one or more of the predictive models of this disclosure.
  • K, L, M, N, O, P, Q, R, S, U and V may be introduced to the neural network (e.g., MLP or Transformer) directly in a hidden layer of the network.
  • the neural network e.g., MLP or Transformer
  • K, L, M, N, O, P, Q, R, S, U and V may be introduced directly into the internal processing of an encoder structure.
  • a setups prediction model (such as GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, PT Setups, Similarity Setups and Diffusion Setups) may take as input one or more latent vectors A which correspond to one or more input oral care meshes (e.g., such as tooth meshes).
  • a setups prediction model (such as GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups and Diffusion Setups) may take as input one or more latent capsules T which correspond to one or more input oral care meshes (e.g., such as tooth meshes).
  • a setups prediction method may take as input both of A and T.
  • Various loss calculation techniques are generally applicable to the techniques of this disclosure (e.g., GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, Setups Classification, Tooth Classification, VAE Mesh Element Labelling, MAE Mesh In-Filling and the imputation of procedure parameters).
  • Losses include LI loss, L2 loss, mean squared error (MSE) loss, cross entropy loss, among others.
  • Losses may be computed and used in the training of neural networks, such as multi-layer perceptron’s (MLP), U-Net structures, generators and discriminators (e.g., for GANs), autoencoders, variational autoencoders, regularized autoencoders, masked autoencoders, transformer structures, or the like. Some implementations may use either triplet loss or contrastive loss, for example, in the learning of sequences.
  • MLP multi-layer perceptron’s
  • U-Net structures such as generators and discriminators (e.g., for GANs), autoencoders, variational autoencoders, regularized autoencoders, masked autoencoders, transformer structures, or the like.
  • Some implementations may use either triplet loss or contrastive loss, for example, in the learning of sequences.
  • Losses may also be used to train encoder structures and decoder structures.
  • a KL- Divergence loss may be used, at least in part, to train one or more of the neural networks of the present disclosure, such as a mesh reconstruction autoencoder or the generator of GDL Setups, which the advantage of imparting Gaussian behavior to the optimization space.
  • This Gaussian behavior may enable a reconstruction autoencoder to produce a better reconstruction (e.g., when a latent vector representation is modified and that modified latent vector is reconstructed using a decoder, the resulting reconstruction is more likely to be a valid instance of the inputted representation).
  • There are other techniques for computing losses which may be described elsewhere in this disclosure. Such losses may be based on quantifying the difference between two or more 3D representations.
  • MSE loss calculation may involve the calculation of an average squared distance between two sets, vectors or datasets. MSE may be generally minimized. MSE may be applicable to a regression problem, where the prediction generated by the neural network or other machine learning model may be a real number.
  • a neural network may be equipped with one or more linear activation units on the output to generate an MSE prediction.
  • Mean absolute error (MAE) loss and mean absolute percentage error (MAPE) loss can also be used in accordance with the techniques of this disclosure.
  • Cross entropy may, in some implementations, be used to quantify the difference between two or more distributions.
  • Cross entropy loss may, in some implementations, be used to train the neural networks of the present disclosure.
  • Cross entropy loss may, in some implementations, involve comparing a predicted probability to a ground truth probability.
  • Other names of cross entropy loss include “logarithmic loss,” “logistic loss,” and “log loss”.
  • a small cross entropy loss may indicate a better (e.g., more accurate) model.
  • Cross entropy loss may be logarithmic.
  • Cross entropy loss may, in some implementations, be applied to binary classification problems.
  • a neural network may be equipped with a sigmoid activation unit at the output to generate a probability prediction.
  • cross entropy may also be used.
  • a neural network trained to make multi -class predictions may, in some implementations, be equipped with one or more softmax activation functions at the output (e.g., where there is one output node for class that is to be predicted).
  • Other loss calculation techniques which may be applied in the training of the neural networks of this disclosure include one or more of: Huber loss, Hinge loss, Categorical hinge loss, cosine similarity, Poisson loss, Logcosh loss, or mean squared logarithmic error loss (MSLE). Other loss calculation methods are described herein and may be applied to the training of any of the neural networks described in the present disclosure.
  • One or more of the neural networks of the present disclosure may, in some implementations, be trained, at least in part by a loss which is based on at least one of: a Point-wise Mesh Euclidean Distance (PMD) and an Earth Mover’s Distance (EMD).
  • PMD Point-wise Mesh Euclidean Distance
  • EMD Earth Mover’s Distance
  • Some implementations may incorporate a Hausdorff Distance (HD) calculation into the loss calculation.
  • HD Hausdorff Distance
  • Computing the Hausdorff distance between two or more 3D representations may provide one or more technical improvements, in that the HD not only accounts for the distances between two meshes, but also accounts for the way that those meshes are oriented, and the relationship between the mesh shapes in those orientations (or positions or poses).
  • Hausdorff distance may improve the comparison of two or more tooth meshes, such as two or more instances of a tooth mesh which are in different poses (e.g., such as the comparison of predicted setup to ground truth setup which may be performed in the course of computing a loss value for training a setups prediction neural network).
  • Reconstruction loss may compare a predicted output to a ground truth (or reference) output.
  • all joints_target is a 3D representation (e.g., a 3D mesh or point cloud) corresponding to ground truth data (e.g., a ground truth tooth restoration design, or a ground truth example of some other 3D oral care representation).
  • all joints jredicted is a 3D representation (e.g., a 3D mesh or point cloud) corresponding to generated or predicted data (e.g., a generated tooth restoration design, or a generated example of some other kind of 3D oral care representation).
  • Other implementations of reconstruction loss may additionally (or alternatively) involve L2 loss, mean absolute error (MAE) loss or Huber loss terms.
  • NLP natural language processing
  • One example application of NLP is the generation of new text based upon prior words or text.
  • Transformers have in turn provided significant improvements over GRU, LSTM and other such RNN-based NLP techniques due to an important attribute of the transformer model, which has the property of multi-headed attention.
  • the NLP concept of multi-headed attention may describe the relationship between each word in a sentence (or paragraph or document or corpus of documents) and each other word in that sentence (or paragraph or document or corpus of documents). These relationships may be generated by a multiheaded attention module, and may be encoded in vector form.
  • This vector may describe how each word in a sentence (or paragraph or document or corpus of documents) should attend to each other word in that sentence (or paragraph or document or corpus of documents).
  • RNN, LSTM and GRU models process a sequence, such a sentence, one word at a time from the start to the end of the sequence. Furthermore, the model may only account for a given subset (called a window) of the sentence when making a prediction.
  • transformer-based models may, in some instances, account for the entirety of the preceding text by processing the sequence in its entirety in a single step.
  • Transformer, RNN, LSTM, and GRU models can all be adapted for use in predictive models in digital dentistry and digital orthodontics, particularly for the setup prediction task.
  • an exemplary transformer model for use with 3D meshes and 3D transforms in setups prediction may be adapted from the Bidirectional Encoder Representation from Transformers (BERT) and/or Generative Pre-Training (GPT) models.
  • a GPT (or BERT) model may first be trained on other data, such as text or documents data, and then be used in transfer learning. Such a transfer learning process may receive a previously trained GPT or BERT model, and then do further training using data comprising 3D oral care representations.
  • Such transfer learning may be performed to train oral care models such as: segmentation, mesh cleanup, coordinate system prediction, setups prediction, validation of 3D oral care representations, transform prediction for placement of oral care meshes (e.g., teeth, hardware, appliance components, fixture model components), tooth restoration design generation (or generation of other 3D oral care representations - such as appliance components, fixture models or archforms), classification of 3D oral care representations, imputation of missing oral care parameters, clustering of clinicians or clustering of clinician preferences, or the like.
  • oral care models such as: segmentation, mesh cleanup, coordinate system prediction, setups prediction, validation of 3D oral care representations, transform prediction for placement of oral care meshes (e.g., teeth, hardware, appliance components, fixture model components), tooth restoration design generation (or generation of other 3D oral care representations - such as appliance components, fixture models or archforms), classification of 3D oral care representations, imputation of missing oral care parameters, clustering of clinicians or clustering of clinician preferences, or the like.
  • Oral care data may comprise one or more of (or combinations of): 3D representations of tooth (e.g., meshes, point clouds or voxels), sections of tooth meshes (such as subsets of mesh elements), tooth transforms (such as in matrix, vector and/or quaternion form, or combinations thereof), transforms for appliance components, transforms for fixture model components, and mesh coordinate system definitions (such as represented by transforms, for example, transformation matrices) and/or other 3D oral care representations described herein.
  • 3D representations of tooth e.g., meshes, point clouds or voxels
  • sections of tooth meshes such as subsets of mesh elements
  • tooth transforms such as in matrix, vector and/or quaternion form, or combinations thereof
  • transforms for appliance components transforms for fixture model components
  • mesh coordinate system definitions such as represented by transforms, for example, transformation matrices
  • Transformers may be trained for generating transforms to position teeth into setups poses (or to place appliance components for use in appliance generation or to place fixture model components for use in fixture model generation). Some implementations may operate in an offline prediction context, and some implementations operation in an online reinforcement learning (RL) context.
  • RL online reinforcement learning
  • a transformer may be initially trained in an offline context and then undergo further fine-tuning training in the online context.
  • the transformer may be trained from a dataset of cohort patient case data.
  • the transformer may be trained from either a physics model, or a CAD model, for example.
  • the transformer may learn from static data, such as transformations (e.g., trajectory transformer).
  • the transform may provide a mapping from malocclusion to setup (e.g., receiving transformation matrices as input and generating transformation matrices as output).
  • transformers may be trained to process 3D representations, such as 3D meshes, 3D point clouds or voxels (e.g., using a decision transformer) takes as input geometry (e.g., mesh, point cloud, voxels etc.), outputs transformations.
  • the decision transformer may be coupled with a representation generation module that encodes representation of the patient’s dentition (e.g., teeth), such as a VAE, a U-Net, an encoder, a transformer encoder, a pyramid encoder-decoder or a simple dense or fully connected network, or a combination thereof.
  • a representation generation module e.g., VAE, the U-Net, the encoder, the pyramid encoder-decoder or the dense network for generating the tooth representation
  • VAE the U-Net
  • the representation generation module may be trained on all teeth in both arches, only the teeth within the same arch (either upper or lower), only anterior teeth, only posterior teeth, or some other subset of teeth.
  • such a model may be trained on each individual tooth (e.g., an upper right cuspid), so that the model is trained or otherwise configured to generate highly accurate representations for an individual tooth.
  • an encoder structure may encode such a representation.
  • a decision transformer may learn in an online context, in an offline context or both.
  • An online decision transformer may be trained (e.g., using RL techniques) to output action, state, and/or reward.
  • transformations may be discretized, to allow for piecewise or stepwise actions.
  • a transformer may be trained to process an embedding of the arch (i.e., to predict transforms for multiple teeth concurrently), to predict a setup.
  • embeddings of individual teeth may be concatenated into a sequence, and then input into the transformer.
  • a VAE may be trained to perform this embedding operation
  • a U-Net may be trained to perform such an embedding
  • a simple dense or fully connected network may be trained, or a combination thereof.
  • the transformer-based techniques of this disclosure may predict an action for an individual tooth, or may predict actions for multiple teeth (e.g., predict transformations for each of multiple teeth).
  • a 3D mesh transformer may include a transformer encoder structure (which may encode oral care data), and may be followed by a transformer decoder structure.
  • the 3D mesh transformer encoder may encode oral care data into a latent representation, which may be combined with attention information (e.g., to concatenate a vector of attention information to the latent representation).
  • the attention information may help the decoder focus on the relevant oral care data during the decoding process (e.g., to focus on tooth order or mesh element connectivity), so that the transformer decoder can generate a useful output for the 3D mesh transformer (e.g., an output which may be used in the generation of an oral care appliance).
  • Either or both of the transformer encoder or transformer decoder may generate a latent representation.
  • the output of the transformer decoder may be reconstructed using a decoder into, for example, one or more tooth transforms for a setup, one or more mesh element labels for segmentation, coordinate systems transforms for use in coordinate system generation, or one or more points of a point cloud or voxels or other mesh elements for another 3D representation).
  • a transformer may include modules such as one or more of: multi-headed attention modules, feed forward modules, normalization modules, linear modules, and softmax modules, and convolution models for latent vector compression, and/or representation.
  • the encoder may be stacked one or more times, thereby further encoding the oral care data, and enabling different representations of the oral care data to be learned (e.g., different latent representations). These representations may be embedded with attention information (which may influence the decoder’s focus to the relevant portions of the latent representation of the oral care data) and may be fed into the decoder in continuous form (e.g., as a concatenation of latent representations - such as latent vectors). In some implementations, the encoded output of the encoder (e.g., latent representations) may be used by downstream processing steps in the generation of oral care appliances.
  • the generated latent representation may be reconstructed into transforms (e.g., for the placement of teeth in setups, or the placement of appliance components or fixture model components), or may be reconstructed into 3D representations (e.g., 3D point clouds, 3D meshes or others disclosed herein).
  • the latent representation which is generated by the transformer e.g., containing continuously encoded attention information
  • Continuously encoded attention information may include attention information which has undergone processing by multiple multi-headed attention modules within the transformer encoder or transformer decoder, to name one example.
  • a loss may be computed for a particular domain using data from that domain. The loss calculation may train the transformer decoder to accurately reconstruct the latent representation into the output data structure pertaining to a particular domain.
  • the decoder when the decoder generates a transform for an orthodontic setup, the decoder may be configured with outputs that describe, for example, the 16 real values which comprise a 4x4 transformation matrix (other data structures for describing transforms are possible). Stated a different way, the latent output generated by the transformer encoder (or transformer decoder) may be used to predict setups tooth transforms for one or more teeth, to place those teeth in setup positions (e.g., either final setups or intermediate stages). Such a transformer encoder (or transformer decoder) may be trained, at least in part using a reconstruction loss (or a representation loss, among others described herein) function, which may compare predicted transforms to ground truth (or reference) transforms.
  • a reconstruction loss or a representation loss, among others described herein
  • the decoder when the decoder generates a transform for a tooth coordinate system, the decoder may be configured with outputs that describe, for example, the 16 real values which comprise a 4x4 transformation matrix (other data structures for describing transforms are possible). Stated a different way, the latent output generated by the transformer encoder (or transformer decoder) may be used to predict local coordinate systems for one or more teeth. Such a transformer encoder (or transformer decoder) may be trained, at least in part using a representation loss (or a reconstruction loss, among others described herein) function, which may compare predicted coordinate systems to ground truth (or reference) coordinate systems.
  • a representation loss or a reconstruction loss, among others described herein
  • the decoder when the decoder generates a 3D point cloud (or other 3D representation - such as 3D mesh, voxelized representation, or the like), the decoder may be configured with outputs that describe, for example, one or more 3D points (e.g., comprising XYZ coordinates). Stated a different way, the latent output generated by the transformer encoder (or transformer decoder) may be used to predict mesh elements for a generated (or modified) 3D representation.
  • Such a transformer encoder may be trained, at least in part using a reconstruction loss (or an LI, L2 or MSE loss, among others described herein) function, which may compare predicted 3D representations to ground truth (or reference) 3D representations.
  • a reconstruction loss or an LI, L2 or MSE loss, among others described herein
  • the decoder may be configured with outputs that describe, for example, labels for one or more mesh elements.
  • the latent output generated by the transformer encoder (or transformer decoder) may be used to predict mesh element labels for mesh segmentation or mesh cleanup.
  • Such a transformer encoder (or transformer decoder) may be trained, at least in part using a cross entropy loss (or others described herein) function, which may compare predicted mesh element labels to ground truth (or reference) mesh element labels.
  • Multi-headed attention and transformers may be advantageously applied to the setups- generation problem.
  • Multi-headed attention is a module in a 3D transformer encoder network which computes the attention weights for the provided oral care data and produces an output vector with encoded information on how each example of oral care data should attend to each other oral care data in an arch.
  • An attention weight is a quantification of the relationship between pairs of oral care data.
  • a 3D representation of oral care data (e.g., comprising voxels, a point cloud, or a 3D mesh composed of vertices, faces or edges) may be provided to the transformer.
  • the 3D representation may describe the patient's dentition, a fixture model (or components of a fixture model), an appliance (or components of an appliance), or the like.
  • a transformer decoder (or a transformer encoder) may be equipped with multi-head attention. Multi-headed attention may enable the transformer decoder (or transformer encoder) to attend to different portions of the 3D representation of oral care data.
  • multi-headed attention may enable the transformer to attend to mesh elements within local neighborhoods (or cliques), or to attend to global dependencies between mesh elements (or cliques).
  • multi-headed attention may enable a transformer for setups prediction (e.g., a setups prediction model which is based on a transformer) to generate a transform for a tooth, and to substantially concurrently attend to each of the other teeth in the arch while that transform is generated.
  • the transform for each tooth may be generated in light of the poses of one or more other teeth in the arch, leading to a more accurate transform (e.g., a transform which conforms more closely to the ground truth or reference transform).
  • a transformer model may be trained to generate a tooth restoration design.
  • Multi-headed attention may enable the transformer to attend to multiple portions of the tooth (or to the surfaces of the adjacent teeth) while the tooth undergoes the generative process.
  • the transformer for restoration design generation may generate the mesh elements for the incisal edge of an incisor while, at least substantially concurrently, attending to the mesh elements of the mesial, distal, facial or lingual surfaces of the incisor.
  • the result may be the generation of mesh elements to form an incisal edge for the tooth which merges seamlessly with the adjacent surfaces of the tooth.
  • one or more attention vectors may be generated which describe how aspects of the oral care data interacts with other aspects of the oral care data associated with the arch.
  • the one or more attention vectors may be generated to describe how one or more portions of a tooth T1 interact with one or more portions of a tooth T2, a tooth T3, a tooth T4, and so one.
  • a portion of a mesh may be described as a set of mesh elements, as defined herein.
  • the interacting portions of tooth T1 and tooth T2 may be determined, in part, through the calculation of mesh correspondences, as described herein.
  • any of these models may be advantageously applied to the task of setups transform prediction, such as in the models described herein.
  • a transformer may be particularly advantageous in that a transformer may enable the transforms for multiple teeth, or even an entire arch to be generated at once, rather than individually, as may be the case with some other models, such as an encoder structure.
  • attention-free transformers may be used to make predictions based on oral care data.
  • One implementation of the GDL Setups neural network model may include a representation generation module (e.g., containing a U-Net structure, an autoencoder encoder, a transformer encoder, another type of encoder-decoder structure, or an encoder, etc.) which may provide its output to a module which is trained to generate tooth transformers (e.g., a set of fully connected layers with optional skip connections, or an encoder structure) to generate the prediction of a transform for each individual tooth.
  • Skip connections may, in some implementations, connect the outputs of a particular layer in a neural network to the inputs of another later in the neural network (e.g., a layer which is not immediately adjacent to the originating layer).
  • the transform-generation module may handle the transform prediction one tooth at a time.
  • Other implementations may replace this encoder structure with a transformer (e.g., transformer encoder or transformer decoder), which may handle all the predictions for all teeth substantially concurrently.
  • a transformer may be configured to receive a large number of input values, larger than some other neural network models (e.g., than a typical MLP). This is because an increased number of inputs may be accommodated by the transformer, the predictions corresponding to those inputs may be generated substantially concurrently.
  • the representation generation module may provide its output to the transformer, and the transformer may generate the setups transforms for all of the several teeth at once, with the technical advantage of improved accuracy (because the transforms for each tooth is generated in light of the transform for each of the adjacent or nearby teeth - leading to fewer collisions and better conformance with the goals of treatment).
  • a transformer may be trained to output a transformation, such as a transform encoded by a 4x4 matrix (or some other size), a quaternion, a translation vector, Euler angles or some other form.
  • the transformation may place a tooth into a setups pose, may place a fixture model component into a pose suitable for fixture model generation, or may place an appliance component into a pose suitable for appliance generation (e.g., dental restoration appliance, clear tray aligner, etc.).
  • the transform may define a coordinate system for aspects of the patient’s dentition, such as a tooth mesh (e.g., a local coordinate system for a tooth).
  • the inputs to the transformer may first be encoded using a neural network (e.g., a latent representation or embedding may be generated), such as one or more linear layers, and/or one or more convolutional layers.
  • the transformer may first be trained on an offline dataset, and subsequently be trained using a secondary actor-critic network, which may enable online reinforcement learning.
  • Transformers may, in some implementations, enable large model capacity and/or enable an attention mechanism (e.g., the capability to pay attention and respond to certain inputs).
  • the attention mechanisms e.g., multi-headed attention
  • the attention mechanisms that are found within transformers may enable intra-sequence relationships to be encoded into neural network features.
  • Intra-sequence relationships may be encoded, for example, by associating an order number (e.g., 1, 2, 3, etc.) with each tooth in an arch, or by associating an order number with each mesh element in a 3D representation (e.g., of a tooth).
  • intra-sequence relationships may be encoded, for example, by associating an order number (e.g., 1, 2, 3, etc.) with each element in the latent vector.
  • Transformers may be scaled by increasing the number of attention heads and/or by increasing the number of transformer layers. Stated differently, one or more aspects of a transformer may be independently trained to handle discrete tasks, and later combined to allow the resulting transformer to perform all of the tasks for which the individual components had been trained, without degrading the predictive accuracy of the neural network. Scaling a convolutional network may be more difficult, because the models may be less malleable or may be less interchangeable.
  • Convolution has an ability to be rotation and translation invariant, which leads to improved generalization, because a convolution model may not need to account for the manner in which the input data in rotated or translated.
  • Transformers have an ability to be permutation invariant, because intra- sequence relationships may be encoded into neural network features.
  • transformers may be combined with convolution-based neural networks, such as by vertically stacking convolution layers and attention layers.
  • Stacking transformer blocks with convolutional blocks enables the resulting structure to have the translation invariance of convolution, and also the permutation invariance of a transformer.
  • Such stacking may improve model capacity and/or model generalization.
  • CoAtNet is an example of a network architecture which combines convolutional and attention-based elements and may be applied to the processing of oral care data.
  • a network for the modification or generation of 3D oral care representations may be trained, at least in part, from CoAtNet (or another model that combines convolution and self-attention/transformers) using transfer learning.
  • the neural networks of this disclosure may exploit one or more benefits of the operation of parameter tuning, whereby the inputs and parameters of a neural network are optimized to produce more data-precise results.
  • One parameter which may be tuned is neural network learning rate (e.g., which may have values such as 0. 1, 0.01, 0.001, etc.).
  • Data augmentation schemes may also be tuned or optimized, such as schemes where “shiver” is added to the tooth meshes before being input to the neural network (i.e., small random rotations, translations and/or scaling may be applied to vary the dataset and make the neural network robust to variations in data).
  • a subset of the neural network model parameters available for tuning are as follows: o Learning rate (LR) decay rate (e.g., how much the LR decays during a training run) o Learning rate (LR). The floating-point value (e.g., 0.001) that is used by the optimizer.
  • LR schedule e.g., cosine annealing, step, exponential
  • Voxel size for cases with sparse mesh processing operations
  • Dropout % e.g., dropout which may be performed in a linear encoder
  • LR decay step size e.g. , decay every' 10 or 20 or 30 epochs
  • Model scaling which may increase or decrease the count of layers and/or the count of parameters per layer.
  • Parameter tuning may be advantageously applied to the training of a neural network for the prediction of final setups or intermediate staging to provide data precision-oriented technical improvements. Parameter tuning may also be advantageously applied to the training of a neural network for mesh element labeling or a neural network for mesh in-filling. In some examples, parameter tuning may be advantageously applied to the training of a neural network for tooth reconstruction. In terms of classifier models of this disclosure, parameter tuning may be advantageously applied to a neural network for the classification of one or more setups (i.e., classification of one or more arrangements of teeth). The advantage of parameter tuning is to improve the data precision of the output of a predictive model or a classification model. Parameter tuning may, in some instances, provide the advantage of obtaining the last remaining few percentage points of validation accuracy out of a predictive or classification model.
  • Various neural network models of this disclosure may draw benefits from data augmentation. Examples include models of this which are trained on 3D meshes, such as GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, FDG Setups, Setups Classification, Setups Comparison, VAE Mesh Element Labeling, MAE Mesh In-filling, Mesh Reconstruction VAE, and Validation Using Autoencoders.
  • Data augmentation such as by way of the method shown in FIG. 1, may increase the size of the training dataset of dental arches.
  • Data augmentation can provide additional training examples by adding random rotations, translations, and/or rescaling to copies of existing dental arches.
  • data augmentation may be carried out by perturbing or jittering the vertices of the mesh, in a manner similar to that described in (“Equidistant and Uniform Data Augmentation for 3D Objects”, IEEE Access, Digital Object Identifier 10.1109/ACCESS.2021.3138162).
  • the position of a vertex may be perturbed through the addition of Gaussian noise, for example with zero mean, and 0. 1 standard deviation. Other mean and standard deviation values are possible in accordance with the techniques of this disclosure.
  • FIG. 1 shows a data augmentation method that systems of this disclosure may apply to 3D oral care representations.
  • a non-limiting example of a 3D oral care representation is a tooth mesh or a set of tooth meshes.
  • Tooth data 100 e.g., 3D meshes
  • the systems of this disclosure may generate copies ofthe tooth data 100 (102).
  • the systems of this disclosure may apply one or more stochastic rotations to the tooth data 100 (104).
  • the systems of this disclosure may apply stochastic translations to the tooth data 100 (106).
  • the systems of this disclosure may apply stochastic scaling operations to the tooth data 100 (108).
  • the systems of this disclosure may apply stochastic perturbations to one or more mesh elements of the tooth data 100 (110).
  • the systems of this disclosure may output augmented tooth data 112 that are formed by way of the method of FIG. 1.
  • generator networks of this disclosure can be implemented as one or more neural networks
  • the generator may contain an activation function.
  • an activation function When executed, an activation function outputs a determination of whether or not a neuron in a neural network will fire (e.g., send output to the next layer).
  • Some activation functions may include: binary step functions, or linear activation functions.
  • Other activation functions impart non-linear behavior to the network, including: sigmoid/logistic activation functions, Tanh (hyperbolic tangent) functions, rectified linear units (ReLU), leaky ReLU functions, parametric ReLU functions, exponential linear units (ELU), softmax function, swish function, Gaussian error linear unit (GELU), or scaled exponential linear unit (SELU).
  • a linear activation function may be well suited to some regression applications (among other applications), in an output layer.
  • a sigmoid/logistic activation function may be well suited to some binary classification applications (among other applications), in an output layer.
  • a softmax activation function may be well suited to some multiclass classification applications (among other applications), in an output layer.
  • a sigmoid activation function may be well suited to some multilabel classification applications (among other applications), in an output layer.
  • a ReLU activation function may be well suited in some convolutional neural network (CNN) applications (among other applications), in a hidden layer.
  • CNN convolutional neural network
  • a Tanh and/or sigmoid activation function may be well suited in some recurrent neural network (RNN) applications (among other applications), for example, in a hidden layer.
  • RNN recurrent neural network
  • gradient descent which determines a training gradient using first-order derivatives and is commonly used in the training of neural networks
  • Newton's method which may make use of second derivatives in loss calculation to find better training directions than gradient descent, but may require calculations involving Hessian matrices
  • additional methods may be employed to update weights, in addition to or in place of the techniques described above. These additional methods include the Levenberg-Marquardt method and/or simulated annealing.
  • the backpropagation algorithm is used to transfer the results of loss calculation back into the network so that network weights can be adjusted, and learning can progress.
  • Neural networks contribute to the functioning of many of the applications of the present disclosure, including but not limited to: GDL Setups, RL Setups, VAE Setups, Capsule Setups, MLP Setups, Diffusion Setups, PT Setups, Similarity Setups, Tooth Classification, Setups Classification, Setups Comparison, VAE Mesh Element Labeling, MAE Mesh In-filling, Mesh Reconstruction Autoencoder, Validation Using Autoencoders, imputation of oral care parameters, 3D mesh segmentation (3D representation segmentation), Coordinate System Prediction, Mesh Cleanup, Restoration Design Generation, Appliance Component Generation and/or Placement, or Archform Prediction.
  • GDL Setups RL Setups
  • VAE Setups Capsule Setups
  • MLP Setups Diffusion Setups
  • PT Setups Similarity Setups, Tooth Classification, Setups Classification, Setups Comparison, VAE Mesh Element Labeling,
  • the neural networks of the present disclosure may embody part or all of a variety of different neural network models. Examples include the U-Net architecture, multi-later perceptron (MLP), transformer, pyramid architecture, recurrent neural network (RNN), autoencoder, variational autoencoder, regularized autoencoder, conditional autoencoder, capsule network, capsule autoencoder, stacked capsule autoencoder, denoising autoencoder, sparse autoencoder, conditional autoencoder, long/short term memory (LSTM), gated recurrent unit (GRU), deep belief network (DBN), deep convolutional network (DCN), deep convolutional inverse graphics network (DCIGN), liquid state machine (LSM), extreme learning machine (ELM), echo state network (ESN), deep residual network (DRN), Kohonen network (KN), neural Turing machine (NTM), or generative adversarial network (GAN).
  • U-Net architecture multi-later perceptron (MLP), transformer, pyramid architecture, recurrent
  • an encoder structure or a decoder structure may be used.
  • Each of these models provides one or more of its own particular advantages.
  • a particular neural networks architecture may be especially well suited to a particular ML technique.
  • autoencoders are particularly suited to the classification of 3D oral care representations, due to the ability to encode the 3D oral care representation into a form which is more easily classifiable.
  • the neural networks of this disclosure can be adapted to operate on 3D point cloud data (alternatively on 3D meshes or 3D voxelized representation).
  • Numerous neural network implementations may be applied to the processing of 3D representations and may be applied to training predictive and/or generative models for oral care applications, including: PointNet, PointNet++, SO-Net, spherical convolutions, Monte Carlo convolutions and dynamic graph networks, PointCNN, ResNet, MeshNet, DGCNN, VoxNet, 3D-ShapeNets, Kd-Net, Point GCN, Grid-GCN, KCNet, PD-Flow, PU-Flow, MeshCNN and DSG-Net.
  • Oral care applications include, but are not limited to: setups prediction (e.g., using VAE, RL, MLP, GDL, Capsule, Diffusion, etc. which have been trained for setups prediction), 3D representation segmentation, 3D representation coordinate system prediction, element labeling for 3D representation clean-up (VAE for Mesh Element labeling), in-filling of missing elements in 3D representation (MAE for Mesh In-Filling), dental restoration design generation, setups classification, appliance component generation and/or placement, archform prediction, imputation of oral care parameters, setups validation, or other validation applications and tooth 3D representation classification.
  • setups prediction e.g., using VAE, RL, MLP, GDL, Capsule, Diffusion, etc. which have been trained for setups prediction
  • 3D representation segmentation e.g., 3D representation coordinate system prediction
  • element labeling for 3D representation clean-up VAE for Mesh Element labeling
  • MAE Mesh In-Filling
  • dental restoration design generation setup
  • Autoencoders that can be used in accordance with aspects of this disclosure include but are not limited to: AtlasNet, FoldingNet and 3D-PointCapsNet. Some autoencoders may be implemented based on PointNet.
  • Representation learning may be applied to setups prediction techniques of this disclosure by training a neural network to learn a representation of the teeth, and then using another neural network to generate transforms for the teeth.
  • Some implementations may use a VAE or a Capsule Autoencoder to generate a representation of the reconstruction characteristics of the one or more meshes related to the oral care domain (including, in some instances, information about the structures of the tooth meshes).
  • that representation (either a latent vector or a latent capsule) may be used as input to a module which generates the one or more transforms for the one or more teeth.
  • These transforms may in some implementations place the teeth into final setups poses.
  • These transforms may in some implementations place the teeth into intermediate staging poses.
  • a transform may be described by a 9x1 transformation vector (e.g., that specifies a translation vector and a quaternion).
  • a transform may be described by a transformation matrix (e.g., a 4x4 affine transformation matrix).
  • systems of this disclosure may implement a principal components analysis (PCA) on an oral care mesh, and use the resulting principal components as at least a portion of the representation of the oral care mesh in subsequent machine learning and/or other predictive or generative processing.
  • PCA principal components analysis
  • An autoencoder may be trained to generate a latent form of a 3D oral care representation.
  • An autoencoder may contain a 3D encoder (which encodes a 3D oral care representation into a latent form), and/or a 3D decoder (which reconstructs that latent from into a facsimile of the inputted 3D oral care representation).
  • 3D encoders and 3D decoders the term 3D should be interpreted in a non-limiting fashion to encompass multi-dimensional modes of operation.
  • systems of this disclosure may train multi-dimensional encoders and/or multi-dimensional decoders.
  • Systems of this disclosure may implement end-to-end training.
  • Some of the end-to-end training-based techniques of this disclosure may involve two or more neural networks, where the two or more neural networks are trained together (i.e., the weights are updated concurrently during the processing of each batch of input oral care data).
  • End-to-end training may, in some implementations, be applied to setups prediction by concurrently training a neural network which learns a representation of the teeth, along with a neural network which generates the tooth transforms.
  • a neural network (e.g., a U-Net) may be trained on a first task (e.g., such as coordinate system prediction).
  • the neural network trained on the first task may be executed to provide one or more of the starting neural network weights for the training of another neural network that is trained to perform a second task (e.g., setups prediction).
  • the first network may learn the low-level neural network features of oral care meshes and be shown to work well at the first task.
  • the second network may exhibit faster training and/or improved performance by using the first network as a starting point in training.
  • Certain layers may be trained to encode neural network features for the oral care meshes that were in the training dataset.
  • These layers may thereafter be fixed (or be subjected to minor changes over the course of training) and be combined with other neural network components, such as additional layers, which are trained for one or more oral care tasks (such as setups prediction).
  • additional layers which are trained for one or more oral care tasks (such as setups prediction).
  • a portion of a neural network for one or more of the techniques of the present disclosure may receive initial training on another task, which may yield important learning in the trained network layers. This encoded learning may then be built upon with further task-specific training of another network.
  • transfer learning may be used for setups prediction, as well as for other oral care applications, such as mesh classification (e.g., tooth or setups classification), mesh element labeling, mesh element in-filling, procedure parameter imputation, mesh segmentation, coordinate system prediction, restoration design generation, mesh validation (for any of the applications disclosed herein).
  • mesh classification e.g., tooth or setups classification
  • mesh element labeling e.g., mesh element in-filling
  • procedure parameter imputation e.g., mesh element in-filling
  • mesh segmentation e.g., coordinate system prediction
  • restoration design generation for any of the applications disclosed herein.
  • a neural network trained to output predictions based on oral care meshes may first be partially trained on one of the following publicly available datasets, before being further trained on oral care data: Google PartNet dataset, ShapeNet dataset, ShapeNetCore dataset, Princeton Shape Benchmark dataset, ModelNet dataset, ObjectNet3D dataset, ThingilOK dataset (which is especially relevant to 3D printed parts validation), ABC: A Big CAD Model Dataset For Geometric Deep Learning, ScanObjectNN, VOCASET, 3D-FUTURE, MCB: Mechanical Components Benchmark, PoseNet dataset, PointCNN dataset, MeshNet dataset, MeshCNN dataset, PointNet++ dataset, PointNet dataset, or PointCNN dataset.
  • a neural network which was previously trained on a first dataset may subsequently receive further training on oral care data and be applied to oral care applications (such as setups prediction).
  • Transfer learning maybe employed to further train any of the following networks: GCN (Graph Convolutional Networks), PointNet, ResNet or any of the other neural networks from the published literature which are listed above.
  • a first neural network may be trained to predict coordinate systems for teeth (such as by using the techniques described in WO2022123402A1 or US Provisional Application No. US63/366492).
  • a second neural network may be trained for setups prediction, according to any of the setups prediction techniques of the present disclosure (or a combination of any two or more of the techniques described herein).
  • Transfer learning may transfer at least a portion of the knowledge or capability of the first neural network to the second neural network. As such, transfer learning may provide the second neural network an accelerated training phase to reach convergence.
  • the training of the second network may, after being augmented with the transferred learning, then be completed using one or more of the techniques of this disclosure.
  • Systems of this disclosure may train ML models with representation learning.
  • representation learning include the fact that the generative network (e.g., neural network that predicts a transform for use in setups prediction) can be configured to receive input with a known size and/or standard format, as opposed to receiving input with a variable size or structure.
  • Representation learning may produce improved performance over other techniques, because noise in the input data may be reduced (e.g., because the representation generation model extracts hierarchical neural network features and/or reconstruction characteristics of an inputted representation (e.g., a mesh or point cloud) through loss calculations or network architectures chosen for that purpose).
  • Reconstruction characteristics may comprise values in of a latent representation (e.g., a latent vector) that describe aspects of the shape and/or structure of the 3D representation that was provided to the representation generation module that generated the latent representation.
  • the weights of the encoder module of a reconstruction autoencoder may be trained to encode a 3D representation (e.g., a 3D mesh, or others described herein) into a latent vector representation (e.g., a latent vector).
  • the capability to encode a large set (e.g., hundreds, thousands or millions) of mesh elements into a latent vector may be learned by the weights of the encoder.
  • Each dimension of that latent vector may contain a real number which describes some aspect of the shape and/or structure of the original 3D representation.
  • the weights of the decoder module of the reconstruction autoencoder may be trained to reconstruct the latent vector into a close facsimile of the original 3D representation.
  • the capability to interpret the dimensions of the latent vector, and to decode the values within those dimensions may be learned by the decoder.
  • the encoder and decoder neural network modules are trained to perform the mapping of a 3D representation into a latent vector, which may then be mapped back (or otherwise reconstructed) into a 3D representation that is substantially similar to an original 3D representation for which the latent vector was generated.
  • examples of loss calculation may include KL-divergence loss, reconstruction loss or other losses disclosed herein.
  • Representation learning may reduce the size of the dataset required for training a model, because the representation model learns the representation, enabling the generative network to focus on learning the generative task.
  • the result may be improved model generalization because meaningful neural network features of the input data (e.g., local and/or global features) are made available to the generative network.
  • a first network may learn the representation, and a second network may make the predictive decision.
  • each of the networks may generate more accurate results for their respective tasks than with a single network which is trained to both learn a representation and make a decision.
  • transfer learning may first train a representation generation model. That representation generation model (in whole or in part) may then be used to pre-train a subsequent model, such as a generative model (e.g., that generates transform predictions).
  • a representation generation model may benefit from taking mesh element features as input, to improve the capability of a second ML module to encode the structure and/or shape of the inputted 3D oral care representations in the training dataset.
  • One or more of the neural networks models of this disclosure may have attention gates integrated within. Attention gate integration provides the enhancement of enabling the associated neural network architecture to focus resources on one or more input values.
  • an attention gate may be integrated with a U-Net architecture, with the advantage of enabling the U-Net to focus on certain inputs, such as input flags which correspond to teeth which are meant to be fixed (e.g,. prevented from moving) during orthodontic treatment (or which require other special handling).
  • An attention gate may also be integrated with an encoder or with an autoencoder (such as VAE or capsule autoencoder) to improve predictive accuracy, in accordance with aspects of this disclosure.
  • attention gates can be used to configure a machine learning model to give higher weight to aspects of the data which are more likely to be relevant to correctly generated outputs.
  • attention gates or mechanisms
  • utilizes aspects of the data that are more likely to be relevant to correctly generated outputs the ultimate predictive accuracy of those machine learning models is improved.
  • the quality and makeup of the training dataset for a neural network can impact the performance of the neural network in its execution phase.
  • Dataset filtering and outlier removal can be advantageously applied to the training of the neural networks for the various techniques of the present disclosure (e.g., for the prediction of final setups or intermediate staging, for mesh element labeling or a neural network for mesh in-filling, for tooth reconstruction, for 3D mesh classification, etc.), because dataset filtering and outlier removal may remove noise from the dataset.
  • dataset filtering and outlier removal may remove noise from the dataset.
  • the mechanism for realizing an improvement is different than using attention gates, that ultimate outcome is that this approach allows for the machine learning model to focus on relevant aspects of the dataset, and may lead to improvements in accuracy similar to improvements in accuracy realized vis-a-vis attention gates.
  • a patient case may contain at least one of a set of segmented tooth meshes for that patient, a mal transform for each tooth, and/or a ground truth setup transform for each tooth.
  • a patient case may contain at least one of a set of segmented tooth meshes for that patient, a mal transform for each tooth, and/or a set of ground truth intermediate stage transforms for each tooth.
  • a training dataset may exclude patient cases which contact passive stages (i.e., stages where the teeth of an arch do not move).
  • the dataset may exclude cases where passive stages exist at the end of treatment.
  • a dataset may exclude cases where overcrowding is present at the end of treatment (i.e., where the oral care provider, such as an orthodontist or dentist) has chosen a final setup where the tooth meshes overlap to some degree.
  • the dataset may exclude cases of a certain level (or levels) of difficulty (e.g., easy, medium and hard).
  • the dataset may include cases with zero pinned teeth (or may include cases where at least one tooth is pinned).
  • a pinned tooth may be designated by a technician as they design the treatment to stop the various tools from moving that particular tooth.
  • a dataset may exclude cases without any fixed teeth (conversely, where at least one tooth is fixed).
  • a fixed tooth may be defined as a tooth that shall not move in the course of treatment.
  • a dataset may exclude cases without any pontic teeth (conversely, cases in which at least one tooth is pontic).
  • a pontic tooth may be described as a “ghost” tooth that is represented in the digital model of the arch but is either not actually present in the patient’s dentition or where there may be a small or partial tooth that may benefit from future work (such as the addition of composite material through a dental restoration appliance).
  • the advantage of including a pontic tooth in a patient case is to leave space in the arch as a part of a plan for the movements of other teeth, in the course of orthodontic treatment.
  • a pontic tooth may save space in the patient’s dentition for future dental or orthodontic work, such as the installation of an implant or crown, or the application of a dental restoration appliance, such as to add composite material to an existing tooth that is too small or has an undesired shape.
  • the dataset may exclude cases where the patient does not meet an age requirement (e.g., younger than 12). In some implementations, the dataset may exclude cases with interproximal reduction (IPR) beyond a certain threshold amount (e.g., more than 1.0 mm).
  • the dataset to train a neural network to predict setups for clear tray aligners (CTA) may exclude patient cases which are not related to CTA treatment.
  • the dataset to train a neural network to predict setups for an indirect bonding tray product may exclude cases which are not related to indirect bonding tray treatment.
  • the dataset may exclude cases where only certain teeth are treated. In such implementations, a dataset may comprise of only cases where at least one of the following are treated: anterior teeth, posterior teeth, bicuspids, molars, incisors, and/or cuspids.
  • FIG. 2 describes an example method to impute designated oral care parameters (e.g., using a trained machine learning model) in accordance with aspects of this disclosure.
  • Techniques of this disclosure are directed to the imputation of new oral care parameters or oral care metrics, such as the technique depicted in FIG. 2.
  • the imputed values may be used in patient treatment, for example, in the automated generation of orthodontic setups or the automated generation of a tooth restoration design.
  • the ML model may have been trained on a dataset of cohort patient cases.
  • Oral care arguments comprise either or both of oral care parameters and/or oral care metrics.
  • Oral care parameters may include either or both of orthodontic procedure parameters and/or restoration design parameters.
  • Oral care metrics may include either or both of orthodontic metrics and/or restoration design metrics.
  • a subset of procedure parameters values (or oral care metrics) for a particular patient may be provided to a machine learning (ML) model for oral care argument imputation which has been trained on historical patient data to impute values for one or more oral care arguments (e.g., to receive values for oral care arguments which are present in the training dataset, and to subsequently suggest values for oral care arguments which are not present in the training dataset).
  • Oral care arguments which may be imputed include oral care parameters, oral care metrics, and/or tooth movement procedures (as described herein).
  • Oral care parameters include orthodontic procedure parameters, restoration design parameters, or other values which may be provided to ML models to influence the generation of 3D oral care representations.
  • Oral care metrics may quantify aspects of the shape, dimensions, and/or structure of a 3D representation which is used in digital oral care treatment (e.g., orthodontic metrics, dental restoration metrics, or other measurements of 3D representations for use in digital oral care), and may be provided to ML models to influence the generation of 3D oral care representations.
  • Such an imputation ML module may use either supervised or unsupervised techniques to make such recommendations.
  • the ML-based techniques may include but are not limited to the techniques described herein with respect to natural language processing.
  • An ML module of this type may impute missing values and/or provide categorical recommendations (e.g., using a neural network), textbased recommendations (e.g., using a transformer), and/or real-valued or integer (numerical) recommendations (e.g., using a neural network, PCA model, or linear regression).
  • categorical recommendations e.g., using a neural network
  • textbased recommendations e.g., using a transformer
  • real-valued or integer (numerical) recommendations e.g., using a neural network, PCA model, or linear regression.
  • an ML-based imputation methods of this disclosure may receive a subset of restoration design parameters values for a particular patient (e.g., “Tooth width at base,” “Dental restoration style,” “Style”).
  • the imputation methods may be trained on historical cohort patient case data to impute values for one or more missing restoration design parameters (e.g., “Tooth width at incisal edge”).
  • the methods may generate recommended values for restoration design parameters according to the distribution of such values in the training dataset.
  • the methods may generate recommended values for restoration design parameters which are missing from a trial patient case, according to the distribution of those values present in the training dataset.
  • the ML Model for oral care argument imputation may use either supervised or unsupervised learning paradigms to provide such recommendations, according to various implementations.
  • supervised ML models which may be trained to output these recommendations include, but are not limited to, regression models (such as linear regression), decision trees, random forests, boosting, Gaussian process, k-nearest neighbors (KNN), logistic regression, principal component analysis (PCA), Naive Bayes, gradient boosting algorithms (e.g., GBM, XGBoost, LightGBM and CatBoost) or support vector machines (SVMs), or a fully connected neural network model that has been trained for classification.
  • a multilayer perceptron (MLP), a transformer, autoencoder, or other neural network may be trained to predict missing oral care arguments given a set of oral care arguments of known values.
  • an imputation ML model may be trained (e.g., as seen in FIG. 2) to encode the distribution of cohort patient case data.
  • Each patient case values for one or more oral care arguments e.g., oral care parameters and/or oral care metrics
  • 3D representations of the patient’s anatomy e.g., dental anatomy
  • one or more oral care metrics e.g., “Alignment” or “Proximal Contacts”
  • may be computed for a patient case e.g., computed based upon the 3D representations of the patient’s anatomy.
  • a patient case initially has a list of values for one or more available oral care arguments, and a vector of oral care arguments may subsequently be generated from that list, while maintaining an order of oral care arguments which is consistent with the other patient cases in the training dataset.
  • the vectors of oral care arguments may maintain a consistent order between patient cases (e.g., each patient case may have a vector of oral care arguments, and each vector may have the same order). If a patient case lacks a value for a particular oral care argument, then that oral care argument may have a NULL value, or the like.
  • An example patient case may contain the 28 tooth meshes that describe the shapes of the patient’s teeth, and may also contain a one or more oral care arguments (e.g., which may be arranged into a vector with an order that is consistent with other cohort patient cases) which are to be provided to an automated setups prediction model.
  • oral care arguments may include oral care parameters (e.g., “Teeth To Move,” “Overbite,” “Correction to Class I (canine and molar),” or others described herein) and/or oral care metrics (e.g., “Alignment,” “Buccolingual Inclination,” “Canine Overbite,” “Midline,” or others described herein).
  • Oral care parameters may be provided to a generative ML model (e.g., setups prediction, restoration design generation, appliance component generation, fixture model component generation, and other implementations described herein) to influence the generative operations of that model.
  • Oral care parameters may include integer values, real values, categorical values, text-based values (e.g., text-based instructions or the doctor’s notes), or others described herein.
  • Oral care metrics may quantify aspects of the dimensions, shape, or structure of a 3D representation which is to be provided to a model for generating an oral care appliance.
  • Oral care metrics generally include real values, or others described herein.
  • the training dataset may contain hundreds, thousands, or tens of thousands of such patient cases, each of which has values for a particular set of oral care arguments, and custom values for those oral care arguments.
  • the set of oral care arguments for a particular patient case in the training dataset may undergo masking (e.g., one or more dimensions of the vector of oral care arguments may be assigned a masking token or masking value).
  • An “ignore masking” token may flag a dimension in the vector to be ignored by the imputation ML model (e.g., to influence the imputation ML model not to consider that dimension while generating predictive inferences).
  • An “imputation masking” token may flag a dimension in the vector to be imputed by the imputation ML model (e.g., to influence the imputation ML model to generate a value for that dimension in the vector, according to the distribution of oral care argument values in the training dataset).
  • Either or both of an imputation mask and an ignore mask may be applied to a vector of oral care arguments, and the masked vector of oral care arguments may subsequently be provided to an imputation ML model (e.g., comprising a transformer encoder or transformer decoder, followed by an MLP, an encoder or a decoder).
  • GPT encoders or GPT decoders may be used, among other imputation ML models described herein.
  • the transformer encoder or transformer decoder may encode the distribution of the training dataset, and generate a latent representation, which may then be reconstructed by a subsequent neural network module (e.g., an MLP, encoder, or decoder).
  • the reconstructed latent representation may comprise a vector of values which corresponds to the values that were flagged by the imputation mask.
  • the imputation ML model may generate a vector of values for the one or more oral care arguments that were flagged to imputation by the imputation mask. This generated vector of oral care argument values may be compared to a corresponding ground truth vector of oral care arguments (e.g., the set of oral care arguments that were removed by that application of the imputation mask).
  • a loss value may be computed as a result of the comparing (e.g., cross-entropy, LI, L2 or others described herein), and used to train, at least in part, the neural networks of the imputation ML model (e.g., transformer encoder, transformer decoder, MLP, encoder, decoder, or others described herein).
  • the neural networks may be trained end-to-end.
  • a training dataset 200 may comprise one or more patient cases, each of which may contain one or more oral care arguments (e.g., oral care metrics or oral care parameters).
  • oral care arguments e.g., oral care metrics or oral care parameters
  • the oral care metrics may be computed for a patient case in training dataset 200, according to techniques of this disclosure.
  • the dataset may, in some implementations undergo outlier removal (204), or the removal of noisy or anomalous cases.
  • Each case of the training dataset 200 may have a corresponding imputation mask 202 and a corresponding ignore mask 224, all of which may be provided to the training of an ML model for oral care argument imputation 212.
  • the ignore mask is applied (226) to the vector of oral care arguments, assigning the “ignore token” to zero or more dimensions of the vector of oral care arguments.
  • the ignore token may influence the imputation ML model to ignore those dimensions of the oral care arguments vector when making predictive inferences.
  • the imputation mask may be applied (206) to the vector of oral care arguments, flag the dimensions which contain oral care arguments which are to be predicted by the imputation ML model. Stated another way, the imputation mask influences the imputation ML model to impute oral care argument values for the dimensions which are flagged by the imputation mask token.
  • the result of the mask application steps is full-length vector 208 of oral care arguments.
  • the step 206 wherein the imputation mask is applied to the vector of oral care arguments may generate: 1) a masked copy of the vector of oral care arguments 208 in which zero or more dimensions have received the ignore token (to designate those zero or more oral care arguments to be ignored by the imputation ML model), and one or more dimensions have received the imputation token (to designate those one or more oral care arguments to undergo imputation), and/or 2) a vector 210 which contains only the oral care arguments that were flagged by the imputation mask (e.g., the subset of oral care arguments that are to be predicted or imputed by the ML model for oral care argument imputation 212).
  • the model 212 may comprise a first module 214 which generates one or more latent representations 216, and a second module 218 which generates the imputed oral care argument values.
  • the first module 214 may contain, for example, one or more transformer encoders, one or more U-Nets, one or more transformer decoders, one or more autoencoder encoders, or other neural networks which are configured to generate latent representations.
  • the one or more latent representations 216 may be provided to the second module 218, which may reconstruct the one or more latent representations into a vector 220 of imputed (or imputed) oral care argument values.
  • Differences between the vector 220 and the ground truth oral care arguments vector 222 may be computed by a loss function (e.g., cross-entropy or others disclosed herein).
  • the loss may be used to train, at least in part, the component ML models (e.g., neural networks) within of the ML model for oral care argument imputation 212.
  • the second module 218 may comprise one or more encoders, one or more multi-layer perceptrons (MLP), one or more decoders, one or more transformer encoders, one or more transformer decoders.
  • MLP multi-layer perceptrons
  • the second module 218 may contain a transformer encoder, which generates a latent representation that is then provided to a decoder, which generates the vector of oral care arguments (e.g., a vector containing one or more imputed values).
  • the second module 218 may contain a transformer decoder, which generates a latent representation that is then provided to a decoder, which generates the vector of oral care arguments (e.g., a vector containing one or more imputed values).
  • a trained ML model for oral care argument imputation 308 may be used in deployment as described in FIG. 3.
  • the vector of oral care arguments 300 for a patient case and an imputation mask 302 which flags which dimensions in the vector 300 are to be imputed are provided to a module which applies (304) the mask to the vector.
  • the length and ordering of oral care arguments in vector 300 are consistent with that used in the training dataset in FIG. 2.
  • the masked vector 306 of oral care arguments is provided to the ML model for oral care argument imputation 308.
  • the first module 310 generates a latent representation 312 of the masked oral care argument vector 306.
  • the latent representation 312 is provided to the second module 314, which generates imputed oral care argument values for the one or more masked dimensions in vector 306.
  • the vector 316 of imputed oral care arguments is sent to the output.
  • unsupervised ML models which may be trained to impute oral care arguments in accordance with aspects of this disclosure.
  • Unsupervised ML models include clustering techniques such as K-means clustering, density -based spatial clustering of applications with noise (DBSCAN), Gaussian mixture models, Balance Iterative Reducing and Clustering using Hierarchies (BIRCH), Affinity Propagation clustering, Mean-Shift clustering, Ordering Points to Identify the Clustering Structure (OPTICS), Agglomerative Hierarchy clustering, and spectral clustering.
  • Other applicable unsupervised models include collaborative filtering, and techniques embodied in the open- source GroupLens codebase.
  • a latent vector can be interpreted as a manifold.
  • a manifold may be defined as a surface in n-dimensions, which may describe spaces that locally resemble Euclidean space of a particular dimension but may have global structure that is more complex than Euclidean space. Stated another way, a manifold is a space that can be approximated by flat spaces in small regions, despite the global surface having more complex structure, such as curves.
  • a latent vector of this disclosure may, in some instances, have small regions which are approximately flat. For example, every point on an n-dimensional surface may represent an n-dimensional coordinate defining the "n" latent parameters in that latent vector. This manifold (or surface) may, in some instances, have a boundary.
  • This boundary may then define the limits of "applicable” oral care arguments, vs “inapplicable” oral care arguments for a particular application.
  • the manifold may not have a boundary globally, but boundaries may, in some instances, be defined for local regions of the manifold.
  • a manifold of “applicable” oral care arguments may be learned, and an incomplete subset of oral care arguments for a particular patient case may be mapped onto this manifold to identify values (e.g., according to the distribution of the training dataset) for the incomplete subset of oral care parameters.
  • An autoencoder or other manifold learner may be used to encode a latent manifold (or latent representation) of a vector of oral care arguments, allowing the recovery of corrupt or missing data from the vector of oral care arguments.
  • Autoencoders such as Variational Autoencoders (VAE), Capsule Autoencoders, Masked Autoencoders, or Denoising Autoencoders (among others) may be used to impute missing oral care arguments, according to techniques of this disclosure (e.g., for either oral care parameters or oral care metrics).
  • VAE Variational Autoencoders
  • Capsule Autoencoders Masked Autoencoders
  • Denoising Autoencoders (among others) may be used to impute missing oral care arguments, according to techniques of this disclosure (e.g., for either oral care parameters or oral care metrics).
  • a VAE may be trained on oral care arguments.
  • the encoder portion of the VAE may be trained to encode the oral care argument into one or more latent space vectors (which reduce the arguments data into a lowerdimensional space which still retains the reconstruction characteristics of the input data).
  • Some implementations may encode text, categorical values and/or real number values into latent vectors, and other applications of this disclosure provide conversions of mesh, point cloud and/or voxel data into latent space vectors.
  • the decoder of the VAE may be trained to reconstruct the latent vector into a facsimile (within an acceptable margin of reconstruction loss or reconstruction error) of the inputted oral care arguments vector.
  • a reconstruction error may be computed, according to techniques of this disclosure, to compare the reconstructed oral care argument vector to the input oral care argument vector, to compare the two and quantify a difference (or “delta”) between the two vectors.
  • the predictive ML models of this disclosure may modify or update the one or more latent space vectors according to a previously performed mapping of the latent space. These modifications to the one or more latent space vectors may cause the reconstructed output to differ from the input in advantageous ways.
  • the reconstructed oral care argument set may, in some instances, include new and/or different values for one or more oral care arguments.
  • the reconstructed oral care arguments set may include one or more new oral care arguments that were not present in the input set of oral care arguments, with the advantage of imputing new oral care arguments to be used in treating one or more patients.
  • a recommender system or ML imputation module (e.g., for imputing oral care arguments) of this disclosure may be trained on past oral care arguments from other cohort patient cases that included oral care arguments that were predictive of the missing oral care arguments in the current patient case and/or shared similar demographic characteristics that were predictive of the missing oral care arguments in the current patient case (e.g., available procedure parameter values, patient age, information about patient health history, malocclusion type, gender, number of anterior teeth and posterior teeth by arch, amount of overjet, amount of overbite, class relationship, information about whether case is easy/medium/hard, information about tooth crowding, such as collision depths, tooth sizes and/or dimensions, Bolton analysis, which may involve computing arch length by summing tooth sizes, etc.).
  • available procedure parameter values e.g., patient age, information about patient health history, malocclusion type, gender, number of anterior teeth and posterior teeth by arch, amount of overjet, amount of overbite, class relationship, information about whether case is easy/medium/hard, information about tooth crowding, such
  • clustering may be performed on these values, for the purpose of imputing missing oral care arguments.
  • clustering may also incorporate information from the state of occlusion of the maloccluded teeth of the patient, such as a measure of the severity of the malocclusion.
  • the extent of malocclusion may be quantified by one or more of the oral care metrics described herein.
  • One or more of the oral care metrics described herein may be used to train, at least in part, a recommender system or ML imputation module for oral care arguments.
  • Table 2 shows an example data structure to hold procedure parameter data.
  • Table 3 shows an example data structure to hold restoration design parameter data.
  • systems of this disclosure may conditionally train a VAE on available oral care argument data, one or more tooth latent vectors B, one or more tooth meshes (which have been converted to one or more mesh element lists), flags M about missing/pinned teeth, tooth position info N, tooth orientation info O, tooth name/designation info R, orthodontic metrics S, tooth dimension info P and/or tooth gap info Q data, for the purpose of imputing one or more oral care parameters (either missing values for oral care arguments that are present in the input set, or oral care arguments that are altogether missing from the input dataset).
  • the VAE may be conditioned on such inputs for the purpose of imputing one or more oral care arguments.
  • Capsule Autoencoder-based models may be trained for the imputation of missing oral care arguments.
  • Some implementations of the techniques described herein may train a capsule-encoder to encode input data (e.g., a vector of oral care argument data) into one or more latent capsules T.
  • this capsule-encoder may be a 3D capsule-encoder which reconstructs tooth oral care mesh data into a latent form.
  • this capsule-encoder may be trained for text processing and encode text into a latent form which may be processed using the predictive techniques described herein.
  • a latent capsule describes the reconstruction characteristics of the input data (e.g., the available oral care arguments) and can be reconstructed into a facsimile of the available oral care argument data (assuming no changes were made to the latent capsule).
  • the capsule autoencoder-based techniques of this disclosure may execute one or more arithmetic operations on the one or more latent capsules, such that the reconstructed vector of oral care arguments has one or more imputed and/or improved oral care argument values.
  • a capsule-decoder may be used to reconstruct the latent capsule.
  • Such a capsule autoencoderbased technique may also be trained for the imputation or modification of one or more oral care parameters, one or more doctor preferences, or one or more oral care metrics.
  • ML models may be trained to impute tooth movement procedures (e.g., operations or procedures for the movement of teeth in the course of orthodontic treatment).
  • Systems of this disclosure may impute one or more tooth movement procedures.
  • Such a tooth movement imputation model may be applied to a patient case of interest.
  • the techniques of the disclosure may train an ML model (e.g., a recommender system or the method described in FIG.
  • the training of such an ML model may, in some implementations, use a natural language processing (NLP) operation called entity recognition, that identifies entities from doctors’ notes and unstructured written texts and, classifies those entities based on a list of categories. Categories may include: expressions, teeth movement procedures (tip, torque, rotation, translation, etc.) and procedure parameters (0.5mm, 0.05mm, etc.). With the identification of similar patient cases, the methods may impute one or more tooth movement procedures, and in some implementations, also impute one or more procedure parameters, according to data that were extracted from approved patient cases.
  • the techniques of the disclosure provide technical improvements or advancements with respect to the clinical treatment plan by identifying further work or adjustments commonly required by doctors.
  • the technical improvements provided by the techniques of this disclosure may include enhanced data precision with respect to the recommended tooth movement procedures, and potentially reduced processing time.
  • a deep learning implementation of a bi-directional long-short time memory (LSTM) model may be used to cover past labels and/or cover future labels on the data.
  • the LSTM model of this disclosure may use annotated training data (e.g., labeled examples of tooth movement procedures, or procedure parameters in past patient cases).
  • RNN recurrent neural network
  • text transformer in place of the LSTM model.
  • the system may recommend using tooth movement procedures and/or oral care parameters that are not present in a trial patient case.
  • aspects of the method may enable automation of multiple tooth movement procedures, and may also predict tooth movement procedures by computing correlations among historical cohort patient cases.
  • an autoencoder (e.g., which is trained for imputation) may be conditionally trained on procedure parameter data K, doctor preference data L, flags M about missing/pinned teeth, tooth position info N, tooth orientation info O, tooth name/designation info R, orthodontic metrics S, tooth dimension info P and/or tooth gap info Q data, for the purpose of imputing one or more tooth movement procedures.
  • Non-limiting examples of tooth movement procedures include Boolean flags regarding whether to perform one or more of the following: proclination, tooth packing, or archform expansion. Proclination is the tilting of the teeth outward toward the anterior.
  • Tooth packing is sliding the teeth along the archform until the teeth make contact (e.g., packing the teeth to remove interproximal space).
  • Archform expansion is the pushing of the canines, premolars and molars outward in the facial (cheek) direction such that the archform broadens and creates space for crowding to be addressed.
  • Techniques of this disclosure may train an encoder-decoder structure to reconstruct a 3D oral care representation which is suitable for oral care appliance generation.
  • An encoder-decoder structure may comprise at least one encoder or at least one decoder.
  • Non-limiting examples of an encoder-decoder structure include a 3D U-Net, a transformer, a pyramid encoder-decoder or an autoencoder, among others.
  • Non-limiting examples of autoencoders include a variational autoencoder, a regularized autoencoder, a masked autoencoder or a capsule autoencoder.
  • Techniques of this disclosure may, in some implementations, use PointNet, PointNet++, or derivative neural networks (e.g., networks trained via transfer learning using either PointNet or PointNet++ as a basis for training) to extract local or global neural network features from a 3D point cloud or other 3D representation (e.g., a 3D point cloud describing aspects of the patient’s dentition - such as teeth or gums).
  • Techniques of this disclosure may, in some implementations, use U-Nets to extract local or global neural network features from a 3D point cloud or other 3D representation.
  • 3D oral care representations are described herein as such because 3-dimensional representations are currently state of the art. Nevertheless, 3D oral care representations are intended to be used in a non-limiting fashion to encompass any representations of 3-dimensions or higher orders of dimensionality (e.g., 4D, 5D, etc.), and it should be appreciated that machine learning models can be trained using the techniques disclosed herein to operate on representations of higher orders of dimensionality.
  • input data may comprise 3D mesh data, 3D point cloud data, 3D surface data, 3D polyline data, 3D voxel data, or data pertaining to a spline (e.g., control points).
  • An encoderdecoder structure may comprise one or more encoders, or one or more decoders.
  • the encoder may take as input mesh element feature vectors for one or more of the inputted mesh elements. By processing mesh element feature vectors, the encoder is trained in a manner to generate more accurate representations of the input data.
  • the mesh element feature vectors may provide the encoder with more information about the shape and/or structure of the mesh, and therefore the additional information provided allows the encoder to make better-informed decisions and/or generate more-accurate latent representations of the mesh.
  • encoder-decoder structures include U-Nets, autoencoders or transformers (among others).
  • a representation generation module may comprise one or more encoder-decoder structures (or portions of encoders-decoder structures - such as individual encoders or individual decoders).
  • a representation generation module may generate an information-rich (optionally reduced-dimensionality) representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.
  • a U-Net may comprise an encoder, followed by a decoder.
  • the architecture of a U-Net may resemble a U shape.
  • the encoder may extract one or more global neural network features from the input 3D representation, zero or more intermediate-level neural network features, or one or more local neural network features (at the most local level as contrasted with the most global level).
  • the output from each level of the encoder may be passed along to the input of corresponding levels of a decoder (e.g., by way of skip connections).
  • the decoder may operate on multiple levels of global-to-local neural network features. For instance, the decoder may output a representation of the input data which may contain global, intermediate or local information about the input data.
  • the U-Net may, in some implementations, generate an information-rich (optionally reduced-dimensionality) representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.
  • An autoencoder may be configured to encode the input data into a latent form.
  • An autoencoder may train an encoder to reformat the input data into a reduced-dimensionality latent form in between the encoder and the decoder, and then train a decoder to reconstruct the input data from that latent form of the data.
  • a reconstruction error may be computed to quantify the extent to which the reconstructed form of the data differs from the input data.
  • the latent form may, in some implementations, be used as an information-rich reduced-dimensionality representation of the input data which may be more easily consumed by other generative or discriminative machine learning models.
  • an autoencoder may be trained to input a 3D representation, encode that 3D representation into a latent form (e.g., a latent embedding), and then reconstruct a close facsimile of that input 3D representation as the output.
  • a latent form e.g., a latent embedding
  • a transformer may be trained to use self-attention to generate, at least in part, representations of its input.
  • a transformer may encode long-range dependencies (e.g., encode relationships between a large number of inputs).
  • a transformer may comprise an encoder or a decoder. Such an encoder may, in some implementations, operate in a bi-directional fashion or may operate a self-attention mechanism.
  • Such a decoder may, in some implementations, may operate a masked self-attention mechanism, may operate a cross-attention mechanism, or may operate in an auto-regressive manner.
  • the self-attention operations of the transformers described herein may, in some implementations, relate different positions or aspects of an individual 3D oral care representation in order to compute a reduced-dimensionality representation of that 3D oral care representation.
  • the cross-attention operations of the transformers described herein may, in some implementations, mix or combine aspects of two (or more) different 3D oral care representations.
  • the auto-regressive operations of the transformers described herein may, in some implementations, consume previously generated aspects of 3D oral care representations (e.g., previously generated points, point clouds, transforms, etc.) as additional input when generating a new or modified 3D oral care representation.
  • the transformer may, in some implementations, generate a latent form of the input data, which may be used as an information-rich reduced-dimensionality representation of the input data, which may be more easily consumed by other generative or discriminative machine learning models.
  • an encoder-decoder structure may first be trained as an autoencoder. In deployment, one or more modifications may be made to the latent form of the input data. This modified latent form may then proceed to be reconstructed by the decoder, yielding a reconstructed form of the input data which differs from the input data in one or more intended aspects. Oral care arguments, such as oral care parameters or oral care metrics may be supplied to the encoder, the decoder, or may be used in the modification of the latent form, to influence the encoder-decoder structure in generating a reconstructed form that has desired characteristics (e.g., characteristics which may differ from that of the input data).
  • Techniques of this disclosure may, in some instances, be trained using federated learning.
  • Federated learning may enable multiple remote clinicians to iteratively improve a machine learning model (e.g., validation of 3D oral care representations, mesh segmentation, mesh cleanup, other techniques which involve labeling mesh elements, coordinate system prediction, non-organic object placement on teeth, appliance component generation, tooth restoration design generation, techniques for placing 3D oral care representations, setups prediction, generation or modification of 3D oral care representations using autoencoders, generation or modification of 3D oral care representations using transformers, generation or modification of 3D oral care representations using diffusion models, 3D oral care representation classification, imputation of missing values), while protecting data privacy (e.g., the clinical data may not need to be sent “over the wire” to a third party).
  • a machine learning model e.g., validation of 3D oral care representations, mesh segmentation, mesh cleanup, other techniques which involve labeling mesh elements, coordinate system prediction, non-organic object placement on teeth, appliance component generation, tooth restoration
  • a clinician may receive a copy of a machine learning model, use a local machine learning program to further train that ML model using locally available data from the local clinic, and then send the updated ML model back to the central hub or third party.
  • the central hub or third party may integrate the updated ML models from multiple clinicians into a single updated ML model which benefits from the learnings of recently collected patient data at the various clinical sites. In this way, a new ML model may be trained which benefits from additional and updated patient data (possibly from multiple clinical sites), while those patient data are never actually sent to the 3rd party.
  • Training on a local in-clinic device may, in some instances, be performed when the device is idle or otherwise be performed during off-hours (e.g., when patients are not being treated in the clinic).
  • Devices in the clinical environment for the collection of data and/or the training of ML models for techniques described herein may include intra-oral scanners, CT scanners, X-ray machines, laptop computers, servers, desktop computers or handheld devices (such as smart phones with image collection capability).
  • contrastive learning may be used to train, at least in part, the ML models described herein. Contrastive learning may, in some instances, augment samples in a training dataset to accentuate the differences in samples from difference classes and/or increase the similarity of samples of the same class.
  • a local coordinate system for a 3D oral care representation such as a tooth
  • a 3D oral care representation such as a tooth
  • transforms e.g., an affine transformation matrix, translation vector or quaternion
  • Systems of this disclosure may be trained for coordinate system prediction using past cohort patient case data.
  • the past patient data may include at least: one or more tooth meshes or one or more ground truth tooth coordinate systems.
  • Machine learning models such as: U-Nets, encoders, autoencoders, pyramid encoder-decoders, transformers, or convolution and/or pooling layers, may be trained for coordinate system prediction.
  • Representation learning may determine a representation of a tooth (e.g., encoding a mesh or point cloud into a latent representation, for example, using a U-Net, encoder, transformer, convolution and/or pooling layers or the like), and then predict a transform for that representation (e.g., using a trained multilayer perceptron, transformer, encoder, transformer, or the like) that defines a local coordinate system for that representation (e.g., comprising one or more coordinate axes).
  • a representation of a tooth e.g., encoding a mesh or point cloud into a latent representation, for example, using a U-Net, encoder, transformer, convolution and/or pooling layers or the like
  • a transform for that representation e.g., using a trained multilayer perceptron, transformer, encoder, transformer, or the like
  • a local coordinate system for that representation e.g., comprising one or more coordinate axes.
  • the mesh convolutional techniques described herein can leverage invariance to rotations, translations, and/or scaling of that tooth mesh to generate predications that techniques that are not invariant to the rotations, translations, and/or scaling of that tooth mesh cannot generate.
  • Pose transfer techniques may be trained for coordinate system prediction, in the form of predicting a transform for a tooth.
  • Reinforcement learning techniques may be trained for coordinate system prediction, in the form of predicting a transform for a tooth.
  • Machine learning models such as: U-Nets, encoders, autoencoders, pyramid encoderdecoders, transformers, or convolution and/or pooling layers, may be trained as a part of a method for hardware (or appliance component) placement.
  • Representation learning may train a first module to determine an embedded representation of a 3D oral care representation (e.g., encoding a mesh or point cloud into a latent form using an autoencoder, or using a U-Net, encoder, transformer, block of convolution and/or pooling layers or the like). That representation may comprise a reduced dimensionality form and/or information-rich version of the inputted 3D oral care representation.
  • a representation may be aided by the calculation of a mesh element feature vector for one or more mesh elements (e.g., each mesh element).
  • a representation may be computed for a hardware element (or appliance component).
  • Such representations are suitable to be provided to a second module, which may perform a generative task, such as transform prediction (e.g., a transform to place a 3D oral care representation relative to another 3D oral care representation, such as to place a hardware element or appliance component relative to one or more teeth) or 3D point cloud generation.
  • transform prediction e.g., a transform to place a 3D oral care representation relative to another 3D oral care representation, such as to place a hardware element or appliance component relative to one or more teeth
  • 3D point cloud generation e.g., a transform to place a 3D oral care representation relative to another 3D oral care representation, such as to place a hardware element or appliance component relative to one or more teeth
  • Such a transform may comprise an affine transformation matrix, translation vector or quatern
  • Machine learning models which may be trained to predict a transform to place a hardware element (or appliance component) relative to elements of patient dentition include: MLP, transformer, encoder, or the like.
  • Systems of this disclosure may be trained for 3D oral care appliance placement using past cohort patient case data.
  • the past patient data may include at least: one or more ground truth transforms and one or more 3D oral care representations (such as tooth meshes, or other elements of patient dentition).
  • the mesh convolution and/or mesh pooling techniques described herein leverage invariance to rotations, translations, and/or scaling of that tooth mesh to generate predications that techniques that are not invariant to the rotations, translations, and/or scaling of that tooth mesh cannot generate.
  • Pose transfer techniques may be trained for hardware or appliance component placement.
  • Reinforcement learning techniques may be trained for hardware or appliance component placement.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Epidemiology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Psychiatry (AREA)
  • Theoretical Computer Science (AREA)
  • Primary Health Care (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Physiology (AREA)
  • Fuzzy Systems (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Rheumatology (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)

Abstract

L'invention concerne des systèmes et des techniques pour entraîner un modèle d'apprentissage automatique (ML) à générer des valeurs d'argument de soins buccaux imputées. Le procédé consiste à recevoir des valeurs d'argument de soins buccaux historiques associées à d'anciens cas de patient et à utiliser des circuits de traitement d'un dispositif informatique pour générer un ou plusieurs ensembles de données d'apprentissage à partir de ces valeurs. Le modèle ML est ensuite entraîné à l'aide des données d'entraînement, lui permettant de prédire les valeurs d'argument de soin buccaux imputées associées à un cas de patient se rapportant à un patient actuellement traité. Le modèle ML entraîné est configuré pour générer des prédictions précises et fiables, améliorant l'efficience et l'efficacité de la planification de traitement de soins buccaux. En déployant le modèle ML entraîné, des professionnels de soins de santé peuvent bénéficier d'une prise de décision plus efficace et de soins personnalisés pour le patient. Ces systèmes et techniques sont un outil précieux pour optimiser des stratégies de traitement de soins buccaux et améliorer les résultats du patient.
PCT/IB2023/062708 2022-12-14 2023-12-14 Imputation de valeurs de paramètres ou de valeurs métriques dans des soins buccaux numériques WO2024127314A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202263432627P 2022-12-14 2022-12-14
US63/432,627 2022-12-14
US202363460583P 2023-04-19 2023-04-19
US63/460,583 2023-04-19

Publications (1)

Publication Number Publication Date
WO2024127314A1 true WO2024127314A1 (fr) 2024-06-20

Family

ID=89378530

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2023/062708 WO2024127314A1 (fr) 2022-12-14 2023-12-14 Imputation de valeurs de paramètres ou de valeurs métriques dans des soins buccaux numériques

Country Status (1)

Country Link
WO (1) WO2024127314A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020026117A1 (fr) 2018-07-31 2020-02-06 3M Innovative Properties Company Procédé de génération automatisée de configurations finales de traitement orthodontique
EP3620130A1 (fr) * 2018-09-04 2020-03-11 Promaton Holding B.V. Planification d'un traitement orthodontique automatisé utilisant l'apprentissage profond
WO2021245480A1 (fr) 2020-06-03 2021-12-09 3M Innovative Properties Company Système pour générer un traitement d'aligneur orthodontique par étapes
WO2022057373A1 (fr) 2020-09-18 2022-03-24 苏州浪潮智能科技有限公司 Procédé de gestion de disque à double port, appareil et terminal et support de stockage
US20220172430A1 (en) * 2019-02-27 2022-06-02 3Shape A/S Method for generating objects using an hourglass predictor
WO2022123402A1 (fr) 2020-12-11 2022-06-16 3M Innovative Properties Company Traitement automatisé de balayages dentaires à l'aide d'un apprentissage profond géométrique
US20220296344A1 (en) * 2019-09-05 2022-09-22 Dentsply Sirona Inc. Method, system and devices for instant automated design of a customized dental object

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020026117A1 (fr) 2018-07-31 2020-02-06 3M Innovative Properties Company Procédé de génération automatisée de configurations finales de traitement orthodontique
US20210259808A1 (en) 2018-07-31 2021-08-26 3M Innovative Properties Company Method for automated generation of orthodontic treatment final setups
EP3620130A1 (fr) * 2018-09-04 2020-03-11 Promaton Holding B.V. Planification d'un traitement orthodontique automatisé utilisant l'apprentissage profond
US20220172430A1 (en) * 2019-02-27 2022-06-02 3Shape A/S Method for generating objects using an hourglass predictor
US20220296344A1 (en) * 2019-09-05 2022-09-22 Dentsply Sirona Inc. Method, system and devices for instant automated design of a customized dental object
WO2021245480A1 (fr) 2020-06-03 2021-12-09 3M Innovative Properties Company Système pour générer un traitement d'aligneur orthodontique par étapes
WO2022057373A1 (fr) 2020-09-18 2022-03-24 苏州浪潮智能科技有限公司 Procédé de gestion de disque à double port, appareil et terminal et support de stockage
WO2022123402A1 (fr) 2020-12-11 2022-06-16 3M Innovative Properties Company Traitement automatisé de balayages dentaires à l'aide d'un apprentissage profond géométrique

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
"Equidistant and Uniform Data Augmentation for 3D Objects", IEEE ACCESS, DIGITAL OBJECT IDENTIFIER
CHEN BEIJIA ET AL: "OrthoAligner: Image-Based Teeth Alignment Prediction via Latent Style Manipulation", IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, vol. 29, no. 8, 11 April 2022 (2022-04-11), USA, pages 3617 - 3629, XP093128182, ISSN: 1077-2626, DOI: 10.1109/TVCG.2022.3166159 *
THANATHORNWONG BHORNSAWAN: "Bayesian-Based Decision Support System for Assessing the Needs for Orthodontic Treatment", HEALTHCARE INFORMATICS RESEARCH, vol. 24, no. 1, 31 January 2018 (2018-01-31), pages 22 - 28, XP093128353, ISSN: 2093-3681, DOI: 10.4258/hir.2018.24.1.22 *
YUN HYE SUN ET AL: "A semi-supervised learning approach for automated 3D cephalometric landmark identification using computed tomography", PLOS ONE, vol. 17, no. 9, 28 September 2022 (2022-09-28), US, pages e0275114, XP093128333, ISSN: 1932-6203, DOI: 10.1371/journal.pone.0275114 *

Similar Documents

Publication Publication Date Title
US20240320382A1 (en) Neural network-based generation and placement of tooth restoration dental appliances
US20210259808A1 (en) Method for automated generation of orthodontic treatment final setups
CN116583243A (zh) 使用几何深度学习对牙科扫描图进行自动化处理
JP5015765B2 (ja) 歯科データマイニング
US20220296344A1 (en) Method, system and devices for instant automated design of a customized dental object
CN109310488A (zh) 用于估计牙齿修复体的形状、位置以及取向中的至少一个的方法
Del Real et al. Use of automated artificial intelligence to predict the need for orthodontic extractions
Liao et al. Automatic tooth segmentation of dental mesh based on harmonic fields
WO2024127315A1 (fr) Techniques de réseau neuronal pour la création d'appareils dans des soins buccodentaires numériques
WO2023242757A1 (fr) Génération de géométrie pour des appareils de restauration dentaire et validation de cette géométrie
WO2024127318A1 (fr) Débruitage de modèles de diffusion pour soins buccaux numériques
WO2024127311A1 (fr) Modèles d'apprentissage automatique pour génération de conception de restauration dentaire
WO2024127314A1 (fr) Imputation de valeurs de paramètres ou de valeurs métriques dans des soins buccaux numériques
WO2024127313A1 (fr) Calcul et visualisation de métriques dans des soins buccaux numériques
WO2024127303A1 (fr) Apprentissage par renforcement pour configurations finales et organisation intermédiaire dans des aligneurs de plateaux transparents
WO2024127304A1 (fr) Transformateurs pour configurations finales et stadification intermédiaire dans des aligneurs de plateaux transparents
WO2024127306A1 (fr) Techniques de transfert de pose pour des représentations de soins bucco-dentaires en 3d
WO2024127302A1 (fr) Apprentissage profond géométrique pour configurations finales et séquençage intermédiaire dans le domaine des aligneurs transparents
WO2024127309A1 (fr) Autoencodeurs pour configurations finales et étapes intermédiaires d'aligneurs transparents
WO2024127308A1 (fr) Classification de représentations 3d de soins bucco-dentaires
WO2024127316A1 (fr) Autocodeurs pour le traitement de représentations 3d dans des soins buccodentaires numériques
WO2023242771A1 (fr) Validation de configurations de dents pour des aligneurs en orthodontie numérique
WO2024127310A1 (fr) Autocodeurs pour la validation de représentations de soins buccodentaires 3d
WO2023242763A1 (fr) Segmentation de maillage et validation de segmentation de maillage en dentisterie numérique
WO2023242765A1 (fr) Validation de modèle d'appareil pour des aligneurs en orthodontie numérique

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23828819

Country of ref document: EP

Kind code of ref document: A1