WO2020184288A1 - 治療後の表情表出時の顔面形態予測方法及びシステム - Google Patents

治療後の表情表出時の顔面形態予測方法及びシステム Download PDF

Info

Publication number
WO2020184288A1
WO2020184288A1 PCT/JP2020/008895 JP2020008895W WO2020184288A1 WO 2020184288 A1 WO2020184288 A1 WO 2020184288A1 JP 2020008895 W JP2020008895 W JP 2020008895W WO 2020184288 A1 WO2020184288 A1 WO 2020184288A1
Authority
WO
WIPO (PCT)
Prior art keywords
case
facial
patient
morphology
expression
Prior art date
Application number
PCT/JP2020/008895
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
千尋 谷川
Original Assignee
国立大学法人大阪大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立大学法人大阪大学 filed Critical 国立大学法人大阪大学
Priority to JP2021504950A priority Critical patent/JPWO2020184288A1/ja
Publication of WO2020184288A1 publication Critical patent/WO2020184288A1/ja

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
    • A61C19/00—Dental auxiliary appliances
    • A61C19/04—Measuring instruments specially adapted for dentistry
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
    • A61C7/00—Orthodontics, i.e. obtaining or maintaining the desired position of teeth, e.g. by straightening, evening, regulating, separating, or by correcting malocclusions

Definitions

  • the present invention relates to a method and a system for predicting the facial morphology of a patient after treatment by arithmetic processing.
  • the present invention relates to a method and a system capable of quantitatively predicting the facial morphology at the time of facial expression expression such as a smile after orthodontic treatment with high accuracy.
  • the human face has a strong influence in obtaining the psychological satisfaction that oneself is socially accepted.
  • facial expressions play an important role as a nonverbal communication means for transmitting emotions and thoughts in social life. Therefore, in modern orthodontic treatment, it is recognized that improving the morphology of the soft tissue of the face is one of the important therapeutic purposes from a social psychological standpoint.
  • a dentist decides on a treatment policy for a patient with an irregular occlusion, whether it is a tooth extraction or a non-extraction, and whether surgery is necessary or camouflage treatment (treatment without surgery) is sufficient. It is indispensable to objectively evaluate the patient's three-dimensional facial morphology and to predict the prognosis of the facial morphology in order to accurately judge.
  • the prediction of facial changes after orthodontic treatment is the hard tissue (dental skeleton) and soft tissue of the patient before orthodontics shown on the head X-ray standard photograph (also referred to as “cephalog” or simply “cephaloc”). It is based on the profile (muscle and skin). For example, software that can visualize and simulate the profile after treatment by performing image processing display such as moving hard tissue on a two-dimensional cephalo image displayed on a monitor and moving soft tissue accordingly. Wear is widespread.
  • Patent Document 1 describes a method of predicting the appearance of a face in a postoperative front view from a preoperative frontal head X-ray standard photograph and a normal facial photograph of a patient in surgical orthodontic treatment of a jaw deformed patient. It is disclosed.
  • the conventional prediction algorithm using cephalo is constructed on the premise that the amount of movement of hard tissues such as teeth and jawbones and soft tissues such as skin is in a simple proportional relationship, and the proportionality constant is also determined by specialist doctors. Designated based on subjectivity or experience. Therefore, the prediction results of facial changes vary among medical professionals, and the prediction accuracy is not guaranteed in a quantitative and objective sense.
  • An object of the present invention is to provide a technique capable of quantitatively predicting a facial morphology in consideration of a patient's facial expression after treatment.
  • the present invention is a method of predicting the facial morphology at the time of facial expression expression after treatment of a patient by an arithmetic process executed by a computer device, and the arithmetic processing performs the treatment.
  • the post-treatment facial expression expression-approximate case facial morphology normalized based on the post-treatment facial expression expression case facial morphology data acquired by using a three-dimensional measuring device at the time of post-treatment facial expression expression.
  • the step of calculating the difference NCS post-pre the step of calculating the resting patient facial morphology model PFMr normalized based on the patient facial morphology data PF of the evaluation target patient, and the resting patient facial morphology model PFMr.
  • the facial expression table predicted after the treatment of the patient to be evaluated by adding the approximate case vector average difference NCS post-pre. It is a method of predicting facial morphology at the time of facial expression expression after treatment, which includes a step of calculating a predicted facial morphology model PFMs-prd at the time of appearance.
  • the case facial morphological data CF for each of the patient patients is the pretreatment resting case facial morphological data CFr-pre and the pretreatment facial expression expression case facial morphological data CFs-pre.
  • the pretreatment case morphology change amount CDpre in which the case feature vector CV is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre.
  • the patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression, and the patient characteristics. It is preferable that the vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs.
  • the calculation process calculates a normalized pre-correction hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the evaluation target patient, and the correction.
  • a predetermined number of cases are selected in the order in which the approximate case feature vector NCV is closer to the patient feature vector PV.
  • the calculation process includes a step of classifying the case feature vector CV into a plurality of case classes by performing a clustering process, and a step of calculating the cluster center of gravity G for each case class.
  • the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is the approximate case feature. It may be selected as the vector NCV.
  • the present invention is a system for predicting the facial morphology at the time of facial expression expression after treatment of a patient, and is realized by arithmetic processing of a computer device, a feature vector extraction means, an approximate case selection means, and a prediction model.
  • a process of extracting a set of multidimensional case feature vectors CV having a plurality of preselected feature variables as elements, and a new patient who is considering treatment (the new patient) A process of extracting a multidimensional patient feature vector PV having the plurality of feature variables as elements based on the patient facial morphology data PF acquired by using a three-dimensional measuring device from the “patient to be evaluated”).
  • the approximate case selection means executes a process of selecting a plurality of approximate case feature vectors NCV that are close to the patient feature vector PV from the set of the case feature vectors CV for the plurality of the case patients.
  • the predictive model calculation means measures three-dimensionally at rest before treatment for each case patient (the selected case patient is referred to as an "approximate case patient") corresponding to the plurality of selected approximate case feature vectors NCV.
  • Pretreatment resting case acquired using the device Process to calculate the pretreatment resting approximate case facial morphology model NCMR-pre normalized based on the facial morphology data, and the post-treatment facial expression table for each of the approximate case patients.
  • Post-treatment facial expression-appearing case acquired using a three-dimensional measuring device at the time of delivery Normalized post-treatment facial expression-appearing approximate case Facial morphology model NCMs-post is calculated based on the facial morphology data, and the pre-treatment
  • the process of calculating the vector average of the resting approximate case facial morphology model NCMr-pre to obtain the pretreatment approximate case vector average NCSpre and the vector average of the post-treatment approximate case facial morphology model NCMs-post are calculated.
  • the process of calculating the resting patient face morphology model PFMr which is normalized based on the patient face morphology data PF of the patient, and the resting patient face morphology model PFMr
  • the facial expression after treatment is executed, which is the process of calculating the predicted facial morphology model PFMs-prd at the time of facial expression expression, which is predicted after the treatment of the patient to be evaluated. It is a facial expression prediction system at the time of expression.
  • the case facial morphology data CF for each of the patient patients is the pretreatment resting case facial morphological data CFr-pre and the pretreatment facial expression expression case facial morphological data CFs-pre.
  • the pretreatment case morphology change amount CDpre in which the case feature vector CV is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre.
  • the patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression, and the patient characteristics. It is preferable that the vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs.
  • the prediction model calculation means calculates a normalized pre-correction hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the evaluation target patient.
  • a predetermined number of cases are selected in order in which the approximate case feature vector NCV is closer to the patient feature vector PV.
  • the approximate case selection means calculates the cluster center of gravity G for each of the case classes and the process of classifying the case feature vector CV into a plurality of case classes by performing the clustering process.
  • the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is the case feature vector CV. It may be selected as the approximate case feature vector NCV.
  • the present invention it is possible to easily and quantitatively predict the facial morphology at the time of facial expression expression after treatment of a patient. Therefore, it is possible to contribute to an appropriate judgment of the treatment policy in consideration of the facial expression of the patient.
  • FIG. 1 illustrates the schematic configuration of the facial morphology prediction system after orthodontic treatment.
  • various processing means described later are mainly realized by arithmetic processing of the computer device 10.
  • a large-capacity database 20, an input device 30, an output device 40, and the like are connected to the computer device 10.
  • the database 20 may be a hard disk or an optical disk directly connected to the computer device 10, or may be a data server or storage in a hospital, for example, which can be accessed from the computer device 10 via a network. Further, the database 20 may be provided in, for example, a cloud data center on a wide area network.
  • the database 20 contains X-ray photograph data, three-dimensional facial morphology data, and the like measured from a plurality of treated patients (past patients who have undergone the treatment are referred to as "case patients").
  • the primary data, the facial morphology model normalized based on the primary data, the intermediate data such as the feature vector which is a multivariate quantity extracted from the features, and the case data including the predicted facial morphology model which is the evaluation data are stored. It is preferable that the access permission to the case data such as the database 20 is restricted only to a specific person (for example, the doctor in charge) who is permitted to share and use the data.
  • the computer device 10 uses the case data stored in the database 20 to execute arithmetic processing for facial morphology prediction, which will be described later.
  • the input device 30 includes an operation input device linked with a human interface, such as a keyboard, a mouse, and a touch panel. Further, the input device 30 may be a device having a function of inputting data acquired or processed by another system to the computer device 10 via an information storage medium or a network.
  • the output device 40 includes, for example, an image display that three-dimensionally visualizes and displays predicted facial morphology data and a model. Further, the output device 40 may be a writing device of an information storage medium for providing data to another system or a communication device capable of outputting data to the outside via a network.
  • the facial morphology prediction system allows the patient's facial photograph data and three-dimensional facial morphological data taken in a hospital laboratory or the like to be taken into the computer device 10 via the database 20 or directly. It is configured. Therefore, this system includes a digital camera 61 and a three-dimensional measuring device 62.
  • a three-dimensional measuring device 62 a general optical measuring device such as a three-dimensional camera, a three-dimensional scanner, or a three-dimensional laser profiler can be used for a predetermined part such as the entire face of the patient or the occlusal portion.
  • the facial morphology data of these patients may be input from the input device 30 via the information storage medium, or may be input to the system via, for example, a hospital network.
  • the data of the X-ray photograph and the cephalo image may be input to the computer device 10 and / or the database 20.
  • the system may be equipped with an X-ray inspection apparatus 63, for example, via a hospital network.
  • the X-ray inspection device 63 may include an inspection device capable of taking a panoramic X-ray photograph of a patient's occlusal portion, CT image data, and the like.
  • case data The "case data" accumulated in the database 20 will be specifically described.
  • N indicates the number of case patients (that is, the number of cases).
  • the case facial morphology data CF for each case patient is, in detail, "pretreatment resting case facial morphological data CFr-pre”, “pretreatment facial expression expression case facial morphological data CFs-pre”, and “post-treatment resting”. Includes “case facial morphology data CFr-post” and “case facial morphology data CFs-post at the time of facial expression after treatment”.
  • FIG. 2 shows an example of case facial morphology data CF.
  • the “non-expressed facial expression” face specifically means a resting face
  • the “expressed facial expression” face specifically refers to a smiling facial expression. It shall refer to the face of.
  • Pretreatment resting case facial morphology data CFr-pre refers to facial morphology data obtained by three-dimensionally measuring the resting face in a state in which the case patient does not express a facial expression before treatment.
  • Case facial morphology data at the time of facial expression expression before treatment CFs-pre refers to facial morphological data obtained by three-dimensionally measuring a face in which a case patient expresses a smiling facial expression, for example, before treatment.
  • Post-treatment resting case facial morphology data CFr-post refers to facial morphological data obtained by three-dimensionally measuring the resting face of a case patient who does not express a facial expression after treatment.
  • Case facial morphology data CFs-post at the time of facial expression expression before treatment refers to facial morphology data obtained by three-dimensionally measuring, for example, a face in which a case patient expresses a smiling facial expression after treatment.
  • the number of three-dimensional facial morphology data acquired differs depending on the size of the face of each patient.
  • the position of the origin differs depending on the standing position of the patient who was photographed.
  • a morphological model that converts three-dimensional facial morphological data into a normalized facial morphological model in order to enable quantitative comparison and statistical processing of the facial morphology of each patient.
  • the conversion means 110 is provided.
  • the morphological modeling means 110 is normal, for example, by extracting predetermined anatomical feature points from the patient's three-dimensional facial morphology data and arranging the feature points on polygons having the same number of points and the same topological structure.
  • the morphological model constructed by such a method is generally called a "homology model", and for example, an HBM (Homologous Body Modeling) program provided by AIST (National Institute of Advanced Industrial Science and Technology) can be used.
  • HBM Homologous Body Modeling
  • the morphological modeling means 110 normalizes the above-mentioned case facial morphological data CF for each case patient and performs a process of constructing a “case facial morphological model CFM”.
  • FIG. 3 shows an example of a case facial morphology model CFM.
  • the model data obtained by normalizing the pretreatment resting case facial morphology data CFr-pre is referred to as the pretreatment resting case facial morphology model CFMr-pre, and the pretreatment facial expression expression case facial morphology data CFs.
  • the model data obtained by normalizing -pre is called “pretreatment facial expression expression case facial morphology model CFMs-pre”
  • the model data obtained by normalizing the post-treatment facial expression data CFr-post is referred to as "post-treatment resting time”.
  • the model data obtained by normalizing the case facial morphology data CFs-post at the time of facial expression expression after treatment is referred to as "case facial morphology model CFMs-post”.
  • the case data further includes the "case morphological change amount CD" for N case patients.
  • the case morphology change amount CD for each case patient includes, in detail, "pre-treatment case morphology change amount CD pre” and "post-treatment case morphology change amount CD post”.
  • the morphological change amount calculation means 120 calculates the change amount of the pretreatment resting case facial morphology model CFMr-pre and the pretreatment facial expression expression case facial morphological model CFMs-pre to obtain the “pretreatment case morphological change amount CDpre”. To get. Further, the morphological change amount calculation means 120 calculates the amount of change in the post-treatment resting case facial morphology model CFMr-post and the post-treatment facial expression expression case facial morphological model CFMs-post to obtain a “post-treatment case morphological change amount”. Get "CDpost”.
  • FIG. 4 shows an example in which the case morphology change amount CDpre is obtained from the case facial morphology models CFMr-pre and CFMs-pre before treatment.
  • the "morphological change amount” includes information on the amount of change in soft tissue and the direction when the patient changes from a resting face to a smiling face, and these can be displayed as three-dimensional image data. ..
  • the feature vector extraction means 130 obtains a multidimensional "case morphology feature vector CFV" having a plurality of “feature variables” (values of feature parameters) selected in advance from the case face morphology model CFM for each case patient. Perform the extraction process.
  • the "feature parameter” is a geometric parameter that characteristically represents a morphology such as a human face, and is selected in advance by a specialist, for example, based on his / her experience and knowledge.
  • a specialist for example, based on his / her experience and knowledge.
  • FIG. 5 shows an example of feature parameters selected in the human face outline.
  • the human face can recognize some inflection points in its morphology.
  • the corner of a boundary line such as an eye or nose, the most prominent position in three dimensions, the most recessed position, or the like can be selected.
  • inflection points are referred to as "landmarks" and are used in the definition of feature parameters.
  • the landmark is not particularly limited as long as it is not an inflection point but can be geometrically defined, such as the center point of a straight line connecting two inflection points.
  • the outline of the face can be extracted as follows. First, a surface normal at each pixel of the three-dimensional surface data is calculated by an arithmetic program customized for measuring the facial morphology from the frontal image of the face. In addition, the angle formed by the z-axis and the face normal of the face is also calculated for each coordinate of the face surface. Each coordinate point where the angle formed by the z-axis and the face normal is, for example, 60 degrees is extracted, and the line connecting these points is used as the outline of the face. The angle that defines the outline of the face is preferably an angle between 45 degrees and 90 degrees.
  • a feature parameter is the distance between landmarks.
  • the feature parameter v1 shown in FIG. 5 is defined as the distance between the outer corners of the eyes Ex (
  • the distance between the landmark for example, the line connecting the outermost end Zy'of the face and the chin point Gn
  • the landmark for example, the cheek point Go'
  • Another example of a feature parameter is the angle of the line connecting the landmarks.
  • the angle of the feature parameter v4 is determined by the positional relationship between the outermost end Zy'of the face, the cheek point Go', and the cheek.
  • the characteristic parameter of the distance may be a dimensionless quantity.
  • ) can be adopted as a feature parameter.
  • deviations with respect to a plurality of average values and ratios with respect to the average may be considered as feature parameters.
  • FIGS. 6 to 8 a plurality of feature parameters are selected from a cross section based on three-dimensional data obtained by photographing a specific part of a human face. These cross sections are created by data processing based on anatomical measurement points after determining the three-dimensional coordinate system.
  • FIG. 6 shows, as an example, a yz cross section when the subject's face is cut at a line connecting the outer corner of the eye Ex and the corner point Ch of the mouth.
  • the angle (v7) of the mouth angle Ch with the outer corner Ex as the base point in the z-axis direction the angle (v8) of the cheek protrusion P (Ex-Ch) in the cross section with the outer corner Ex as the base point, the outer corner Ex and the mouth angle Ch.
  • the length of the outer curve (v12), the area closed by the outer curve (v13), and the like can be selected as feature parameters.
  • FIG. 7 shows an xz cross section when the subject's face is cut in a horizontal plane passing through the subnasal point Sn.
  • FIG. 8 illustrates an xz cross section when the subject's face is cut in a horizontal plane passing through the most apex point Pm of the nose.
  • the amount of protrusion of the facial part in the z direction (v14, v18), the angle of the apex (v16, v20), and the amount of protrusion (v17, v22, v23) at various cross-sectional positions.
  • the angle of the concave point (v21) and the like can be selected as feature parameters.
  • the cross section that characterizes the facial morphology may be a cross section that passes through, for example, the glabellar point Gla, the nose root point N, the upper lip point Ls, the lower lip point Li, and the chin point Sm. Further, a difference or a ratio with respect to the z average value of a specific part may be added to the feature parameter.
  • the feature vector extraction means 130 performs a process of measuring feature variables corresponding to each of a plurality of selected and set feature parameters from the patient's three-dimensional facial morphology data.
  • the feature vector extraction means 130 extracts an n-dimensional "feature vector V" having the measured n feature variables v as vector elements.
  • the feature vector extraction means 130 is a multidimensional "case morphology feature vector" based on the case facial morphology data CF before and after the treatment for each case patient and at rest and when the smiling facial expression is expressed. Perform the process of extracting "CFV".
  • FIG. 9 shows an example of extracting the case morphology feature vector CFV from the case face morphology model CFM.
  • the feature vector extracted based on the pretreatment resting case facial morphology data CFr-pre is called the pretreatment resting case morphological feature vector CFVr-pre, and is used as the pretreatment facial expression expression case facial morphology data CFs-pre.
  • the feature vector extracted based on this is called “pretreatment facial expression expression case morphology feature vector CFVs-pre”
  • the feature vector extracted based on post-treatment resting case facial morphology data CFr-post is called "post-treatment rest”.
  • the feature vector extracted based on the case facial morphology data CFs-post at the time of facial expression expression after treatment is called "case morphology feature vector CFVs-post at the time of facial expression expression after treatment”.
  • the feature vector extraction means 130 can also extract a multidimensional "case morphology change amount feature vector CDV" from the case morphology change amount CD showing the soft tissue change amount at rest and when smiling for each case patient.
  • FIG. 10 shows an example of extracting the case morphology change amount feature vector CDV from the case morphology change amount CD.
  • the feature vector extracted based on the pretreatment case morphological change CDpre is called “pretreatment case morphological change feature vector CDVpre”
  • the feature vector extracted based on the posttreatment case morphological change CDpost is called “post-treatment case”. It is called "morphological change amount feature vector CDV post”.
  • the "case feature vector CV" used as the basal variate when selecting the "approximate case patient” described later is the above-mentioned pretreatment resting case morphology feature vector CFVr-pre and the pretreatment facial expression expression case morphology feature.
  • Example 1 Pretreatment resting case Morphological feature vector CFVr-pre (Example 2) Case morphology feature vector CFVs-pre when facial expression is expressed before treatment (Example 3) Pretreatment case morphological change feature vector CDVpre (Example 4) Pretreatment resting case morphological feature vector CFVr-pre + pretreatment facial expression expression case morphological feature vector CFVs-pre (Example 5) Pretreatment resting case morphological feature vector CFVr-pre + pretreatment case morphological change feature vector CDVpre (Example 6) Case morphological feature vector CFVs-pre + pretreatment case morphological change feature vector CDVpre And so on.
  • the database 20 contains a set of case feature vector CVs CV (1) and CV (2), which are extracted from the case morphology feature vector CFV and / or the case morphology deformation amount feature vector CDV and correspond to N case patients. ), CV (3), ..., CV (N) are made into knowledge.
  • a set of case feature vectors CVs for a plurality of case patients may be clustered and knowledgeed in the database 20.
  • a general vector quantization method such as the Lloyd method or the k-means method can be used.
  • the "distance" between the vectors may be either the Euclidean distance or the Manhattan distance.
  • the primary cluster center of gravity G * (l) at the shortest distance is searched from each case feature vector CV, and the group of the case feature vector CV having the shortest distance center of gravity G * (l) as an element is used. Reorganize the next cluster CL ** (l). Then, the secondary cluster center of gravity G ** (l) is also obtained in the secondary cluster CL ** (l), and the tertiary cluster center of gravity G *** (l) is obtained from the group of the case feature vector CV at the shortest distance.
  • the process of optimizing the number of clusters may be performed by the following algorithm.
  • the calculation is performed, and the minimum distance Dc (l) min is obtained.
  • the inter-cluster distance Dc which is the average value of the minimum distance Dc (l) min of each cluster, is calculated by the mathematical formula (1).
  • the inter-cluster distance Dc (for example, D 3 , D 4 , ..., D 12 ) is obtained, and the change shown in the formula (2) is obtained.
  • C + 1 which is obtained by adding 1 to C having the maximum ⁇ Dc, can be determined as the optimum number of clusters.
  • the data of the case feature vector CV classified into each case class CL and the cluster center of gravity G thereof are knowledgeable in the database 20.
  • FIG. 11 is a block diagram showing an outline of the face morphology prediction method
  • FIG. 12 is a flowchart thereof.
  • step S1 the face of a new patient under consideration for treatment (the new patient is referred to as an "evaluation target patient”) is measured using a three-dimensional measuring device 62 before treatment, and the patient's facial morphology. Acquire the data PF.
  • the state in which the patient does not express a facial expression that is, the "resting patient facial morphology data PFr" which is the facial morphological data obtained by measuring the face at rest, and the facial morphological data measuring the face when the smiling facial expression is expressed.
  • At least two types of patient facial morphology data PFs of "facial expression expression patient facial morphology data PFs" are acquired.
  • the morphology modeling means 110 calculates a "patient facial morphology model PFM" normalized by using, for example, the above-mentioned homology model algorithm, based on the patient facial morphology data PF.
  • the facial morphology model constructed based on the resting patient facial morphology data PFr is called “resting patient facial morphology model PFMr”
  • the facial morphological model constructed based on the facial expression expression patient facial morphology data PFs is called “facial expression expression”.
  • patient facial morphology model PFMs When referred to as "patient facial morphology model PFMs".
  • the morphological change amount calculation means 120 obtains the "patient morphological change amount PD" by calculating the change amount of the resting patient facial morphology model PFMr and the facial expression expression patient facial morphological model PFMs.
  • the feature vector extraction means 130 extracts a multidimensional "patient feature vector PV" having a plurality of feature variables as elements from the patient facial morphology model PFM and / or the patient morphology change amount PD.
  • the feature vector extracted based on the resting patient facial morphology model PFMr is called “resting patient morphological feature vector PFVr”, and the feature vector extracted based on the facial expression expression patient facial morphology model PFMs is called “facial expression expression”.
  • patient morphological feature vector PFVs When referred to as “patient morphological feature vector PFVs”.
  • patient morphology change amount feature vector PDV the feature vector extracted based on the patient morphology change amount feature vector PDV.
  • patient feature vector PV is any one feature vector selected from the group of resting patient morphology feature vector PFVr, facial expression expression patient morphology feature vector PFVs, and patient morphology change feature vector PDV, or An extended feature vector can be obtained by combining the feature variables of these two or more feature vectors.
  • Example 1 Resting patient morphology feature vector PFVr (Example 2) Patient morphology feature vector PFVs when expressing facial expressions (Example 3) Patient morphology change characteristic vector PDV (Example 4) Resting patient morphology feature vector PFVr + Facial expression expression patient morphology feature vector PFVs (Example 5) Resting patient morphology feature vector PFVr + patient morphology change feature vector PDV (Example 6) Patient morphology feature vector PFVs + patient morphology change feature vector PDV at the time of facial expression expression And so on.
  • the case feature vector CV selected here that approximates the patient feature vector PV is referred to as an “approximate case feature vector NCV”.
  • the population of the case feature vector CV is narrowed down to cases in which, for example, gender, age, treatment site, hard tissue (dentition, etc.) are common or similar to the patient to be evaluated.
  • Example 1-1 For example, when “resting patient morphology feature vector PFVr" is adopted as the patient feature vector PV, the case feature vector CV becomes "pretreatment resting case morphology feature vector CFVr-pre".
  • the morphological change amount feature vector of Example 1-3 includes information such as the deformation amount of the soft tissue of the face, the deformation direction, and the softness of the tissue as the feature amount, an approximate case is selected using this as the basal variation. As a result, the accuracy of facial morphology prediction when the smiling facial expression of the evaluation target patient is expressed can be increased.
  • the approximate case selection means 140 can select the case feature vector CV having a predetermined number of cases k in ascending order of distance from the patient feature vector PV.
  • the approximate number of cases k is a number determined by an empirical judgment of a specialist doctor or the like.
  • the set CV (j) of the case feature vector CV corresponding to N case patients is knowledgeable in the database 20.
  • the approximate case selection means 140 selects approximate cases having the number of cases k in ascending order of distance (
  • the "distance" between the vectors may be either the Euclidean distance or the Manhattan distance.
  • the approximate case selection means 140 may select the case feature vector CV belonging to the case class CL having the cluster center of gravity G closest to the patient feature vector PV as the approximate case feature vector NCV.
  • ) is selected.
  • the "distance" in this case may be either the Euclidean distance or the Manhattan distance.
  • the approximate case selection means 140 selects a set of all the case feature vectors CV belonging to the approximate case class NCL as the approximate case feature vector NCV.
  • step S3 of FIG. 12 the prediction model calculation means 150 predicts after the treatment of the evaluation target patient based on the patient face morphology model PFM of the evaluation target patient, and the “predicted face” at the time of expressing a smiling expression is predicted.
  • the morphological model PFMs-prd is calculated.
  • Example 3-1 According to this Example 3-1 as shown in FIG. 13, the predicted facial morphology model PFMs-prd at the time of smiling, which is predicted after treatment, is based on the patient facial morphological model PFMs at the time of smiling of the evaluation target patient. Calculate.
  • step S11 a set of "pretreatment facial expression expression approximate case facial morphology model NCMs-pre” normalized based on pretreatment facial expression expression case facial morphology data NCFs-pre for each approximate case patient. Is calculated. Further, in step S12, a normalized “post-treatment facial expression expression approximate case facial morphology model NCMs-post” is calculated based on the post-treatment facial expression expression case facial morphology data NCFs-post for each approximate case patient. To do.
  • step S13 the vector average of the set of the approximate case facial morphology model NCMs-pre at the time of expressing the facial expression before treatment is calculated to obtain the "pretreatment approximate case vector average NCApre”. Further, in step S14, the vector average of the set of the approximate case facial morphology model NCMs-post at the time of facial expression expression after treatment is calculated to obtain the “post-treatment approximate case vector average NCApost”.
  • step S15 the "approximate case vector average difference NCApost-pre" is calculated by subtracting the pretreatment approximate case vector average NCApre from the post-treatment approximate case vector average NCApost.
  • step S16 the approximate case facial morphology vector average difference NCA post-pre calculated in step S15 is added to the patient facial morphology model PFMs when the facial expression of the patient to be evaluated is expressed.
  • the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression which is predicted after the treatment of the patient to be evaluated, is calculated.
  • the prediction model calculation means 150 obtains the prediction face morphology model PFMs-prd at the time of smile, which is predicted after the treatment, based on the patient face morphology model PFMr at rest of the evaluation target patient. You may calculate.
  • step S21 a set of normalized “pretreatment resting approximate case facial morphology model NCMR-pre” is calculated based on the pretreatment resting case facial morphology data NCFr-pre for each approximate case patient.
  • step S22 a normalized “post-treatment facial expression expression approximate case facial morphology model NCMs-post” is calculated based on the post-treatment facial expression expression case facial morphology data NCFs-post for each approximate case patient.
  • step S23 the vector average of the set of the pretreatment approximate case facial morphology model NCMr-pre is calculated to obtain the "pretreatment approximate case vector average NCApre”. Further, in step S24, the vector average of the set of the approximate case facial morphology model NCMs-post at the time of facial expression expression after treatment is calculated to obtain the “post-treatment approximate case vector average NCApost”.
  • step S25 the "approximate case vector average difference NCApost-pre" is calculated by subtracting the pretreatment approximate case vector average NCApre from the post-treatment approximate case vector average NCApost.
  • step S26 the approximate case facial morphology vector average difference NCA post-pre calculated in step S25 is added to the resting patient facial morphology model PFMr of the patient to be evaluated.
  • the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression, which is predicted after the treatment of the patient to be evaluated, is calculated.
  • step S4 (see FIG. 12) of displaying the predicted patient facial morphology model
  • an image of the tooth alignment after treatment is output on a display or the like together with the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression. It is preferable to display it on the device 40.
  • the arithmetic processing by the computer device 10 is a step of calculating a normalized pre-orthodontic hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the tooth skeleton of the patient to be evaluated, and the pre-orthodontic hardness. Prediction in the step of predicting the corrected hard tissue morphology model HMpost of the patient to be evaluated based on the tissue morphology model HMpre, and the prediction facial morphology model PFMs-prd at the time of expressing a smiling expression after treatment evaluated in step S3. The step of incorporating the corrected hard tissue morphology model HMpost, which has been performed, can be included.
  • the image data of the hard tissue is, for example, three-dimensional image data obtained by measuring the tooth portion of the patient with a three-dimensional measuring device, a CT image including the tooth skeleton of the occlusal part, a cephalo image, a panoramic X photograph, and the like.
  • the HMpre may be constructed by combining these two-dimensional and / or three-dimensional image data.
  • the tooth alignment predicted after the treatment is displayed on the image of the predicted facial morphology when the smiling facial expression is expressed.
  • a three-dimensional dentition prediction image or a two-dimensional dentition prediction image with the mouth portion of the predicted smile, it is possible to make a prediction including the appearance of the tooth alignment.
  • the three-dimensional facial morphology of a patient after orthodontic treatment can be predicted easily and quantitatively.
  • the facial morphology at the time of expressing the smiling facial expression can be predicted in advance before the treatment, it is necessary to make an appropriate judgment as to whether or not the treatment plan creates a "good smile" that leads to aesthetic improvement of the facial expression for the patient. Can contribute.
  • the facial morphology prediction system and the facial morphology prediction method according to the present invention can be used not only for orthodontic treatment but also for surgical treatment of patients with jaw deformities, for example. It can also be used, for example, to predict when maxillofacial surgery (including oral surgery and plastic surgery) is performed alone, orthodontic treatment, or jointly with jaw prosthesis treatment. Furthermore, its application can be expected for the prediction of age-related changes in facial morphology.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Epidemiology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Engineering & Computer Science (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Image Processing (AREA)
PCT/JP2020/008895 2019-03-13 2020-03-03 治療後の表情表出時の顔面形態予測方法及びシステム WO2020184288A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021504950A JPWO2020184288A1 (enrdf_load_stackoverflow) 2019-03-13 2020-03-03

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019046443 2019-03-13
JP2019-046443 2019-03-13

Publications (1)

Publication Number Publication Date
WO2020184288A1 true WO2020184288A1 (ja) 2020-09-17

Family

ID=72427823

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/008895 WO2020184288A1 (ja) 2019-03-13 2020-03-03 治療後の表情表出時の顔面形態予測方法及びシステム

Country Status (2)

Country Link
JP (1) JPWO2020184288A1 (enrdf_load_stackoverflow)
WO (1) WO2020184288A1 (enrdf_load_stackoverflow)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040029068A1 (en) * 2001-04-13 2004-02-12 Orametrix, Inc. Method and system for integrated orthodontic treatment planning using unified workstation
JP2011086266A (ja) * 2009-10-19 2011-04-28 Canon Inc 特徴点位置決め装置、画像認識装置、その処理方法及びプログラム
JP2014513824A (ja) * 2011-02-22 2014-06-05 モルフェウス カンパニー リミテッド 顔面補正イメージ提供方法およびそのシステム
WO2017069231A1 (ja) * 2015-10-23 2017-04-27 国立大学法人大阪大学 人体における治療後の形態予測方法及びシステム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040029068A1 (en) * 2001-04-13 2004-02-12 Orametrix, Inc. Method and system for integrated orthodontic treatment planning using unified workstation
JP2011086266A (ja) * 2009-10-19 2011-04-28 Canon Inc 特徴点位置決め装置、画像認識装置、その処理方法及びプログラム
JP2014513824A (ja) * 2011-02-22 2014-06-05 モルフェウス カンパニー リミテッド 顔面補正イメージ提供方法およびそのシステム
WO2017069231A1 (ja) * 2015-10-23 2017-04-27 国立大学法人大阪大学 人体における治療後の形態予測方法及びシステム

Also Published As

Publication number Publication date
JPWO2020184288A1 (enrdf_load_stackoverflow) 2020-09-17

Similar Documents

Publication Publication Date Title
US11617633B2 (en) Method and system for predicting shape of human body after treatment
US9788917B2 (en) Methods and systems for employing artificial intelligence in automated orthodontic diagnosis and treatment planning
US10568716B2 (en) Methods and systems for employing artificial intelligence in automated orthodontic diagnosis and treatment planning
Yuan et al. Personalized design technique for the dental occlusal surface based on conditional generative adversarial networks
Moss et al. Three‐dimensional assessment of treatment outcomes on the face
KR20190020756A (ko) 치아 수복물의 형상, 위치 및 배향 중 적어도 하나를 추정하는 방법
US10007988B2 (en) Systems and methods for approximating the soft tissue profile of the skull of an unknown subject
Barone et al. Computer-aided modelling of three-dimensional maxillofacial tissues through multi-modal imaging
JP2016085490A (ja) 顔形態の評価システム及び評価方法
Park et al. A three-dimensional parametric adult head model with representation of scalp shape variability under hair
Tuan et al. Shape Prediction of Nasal Bones by Digital 2D‐Photogrammetry of the Nose Based on Convolution and Back‐Propagation Neural Network
JP2025512821A (ja) 歯の位置の決定、及び人工ニューラルネットワークを用いた２ｄリスライス画像の生成
KR101715567B1 (ko) 사상체질전문가의 체질 진단에서의 시진(視診) 또는 망진(望診) 상의 오류를 교정하기 위한 얼굴 분석 시스템
CN119445106A (zh) 基于计算机视觉的隐形矫治器佩戴监测和调整系统及方法
WO2020184288A1 (ja) 治療後の表情表出時の顔面形態予測方法及びシステム
CN118840625A (zh) 一种使用图像分析口腔健康的监测方法
Amirkhanov et al. Visual analytics in dental aesthetics
D'Alessio et al. Measure and comparison of facial attractiveness indices through photogrammetry and statistical analysis
Zhang et al. Dental artificial intelligence systems: a review of various data types
Nemeth et al. Quantitative analysis of facial shape in children to support respirator design
Ridel et al. Shape analysis of the nasal complex among South African groups from CBCT scans
Tanikawa et al. Knowledge-dependent pattern classification of human nasal profiles
Paulose et al. Smartening up with artificial intelligence in dentistry: A review
Nayak et al. Artificial intelligence in orthodontics
Marcolin Miscellaneous expertise of 3D facial landmarks in recent literature

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20770797

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021504950

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20770797

Country of ref document: EP

Kind code of ref document: A1