WO2020184288A1

WO2020184288A1 - Method and system for predicting facial morphology in facial expression after treatment

Info

Publication number: WO2020184288A1
Application number: PCT/JP2020/008895
Authority: WO
Inventors: 千尋谷川
Original assignee: 国立大学法人大阪大学
Priority date: 2019-03-13
Filing date: 2020-03-03
Publication date: 2020-09-17
Also published as: JPWO2020184288A1

Abstract

This method for quantitatively predicting the facial morphology of a patient in consideration of an expression after treatment comprises the steps of: acquiring patient morphology models PFM of a resting state and a smiling state of a patient being evaluated; extracting a patient feature vector PV from the patient morphology models PFM; extracting case feature vectors CV from facial morphology models CFM of patients of a plurality of cases for the resting state and the smiling state thereof and before and after treatment; selecting a similar case, the case feature vector CV of which has a small distance to the patient feature vector PV; computing a vector mean difference NCA post-pre of the facial morphology models of the smiling state of the patient of the similar case before and after treatment; and acquiring a predicted morphology model PFMs-prd of a predicted smiling face after treatment by adding the vector mean difference NCA post-pre of the patient of the similar case to the patient morphology models PFM of the resting state or the smiling state of the patient being evaluated.

Description

Facial morphology prediction method and system when facial expression is expressed after treatment

The present invention relates to a method and a system for predicting the facial morphology of a patient after treatment by arithmetic processing. In particular, the present invention relates to a method and a system capable of quantitatively predicting the facial morphology at the time of facial expression expression such as a smile after orthodontic treatment with high accuracy.

The human face has a strong influence in obtaining the psychological satisfaction that oneself is socially accepted. In addition, facial expressions play an important role as a nonverbal communication means for transmitting emotions and thoughts in social life. Therefore, in modern orthodontic treatment, it is recognized that improving the morphology of the soft tissue of the face is one of the important therapeutic purposes from a social psychological standpoint.

For example, when a dentist decides on a treatment policy for a patient with an irregular occlusion, whether it is a tooth extraction or a non-extraction, and whether surgery is necessary or camouflage treatment (treatment without surgery) is sufficient. It is indispensable to objectively evaluate the patient's three-dimensional facial morphology and to predict the prognosis of the facial morphology in order to accurately judge.

Conventionally, the prediction of facial changes after orthodontic treatment is the hard tissue (dental skeleton) and soft tissue of the patient before orthodontics shown on the head X-ray standard photograph (also referred to as "cephalog" or simply "cephaloc"). It is based on the profile (muscle and skin). For example, software that can visualize and simulate the profile after treatment by performing image processing display such as moving hard tissue on a two-dimensional cephalo image displayed on a monitor and moving soft tissue accordingly. Wear is widespread.

For example, Patent Document 1 describes a method of predicting the appearance of a face in a postoperative front view from a preoperative frontal head X-ray standard photograph and a normal facial photograph of a patient in surgical orthodontic treatment of a jaw deformed patient. It is disclosed.

However, the conventional prediction algorithm using cephalo is constructed on the premise that the amount of movement of hard tissues such as teeth and jawbones and soft tissues such as skin is in a simple proportional relationship, and the proportionality constant is also determined by specialist doctors. Designated based on subjectivity or experience. Therefore, the prediction results of facial changes vary among medical professionals, and the prediction accuracy is not guaranteed in a quantitative and objective sense.

Therefore, in order to eliminate the bias of facial morphology / function evaluation, which has been dependent on the subjective evaluation of the surgeon, as much as possible, the inventors have made knowledge of the measurement information of a large number of patients in the past and after the treatment. We have attempted to study and establish a more quantitative and objective evaluation method for facial morphology (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2014-171702 International Publication No. 2017/069231

Through the preliminary analysis of the study, the inventors found that there are cases of disease-specific morphological facial distortion (appearance / facial expression disorder) at rest and at rest, and at rest. It was also found that three-dimensional face measurement when expressing a smiling facial expression is effective in detecting this distortion. As described above, in orthodontic treatment, improvement of facial expression is recognized as one of the important therapeutic purposes. Nevertheless, there was a problem that it was difficult for even specialized dentists to predict and examine in advance whether or not the treatment plan would produce a “good smile”.

An object of the present invention is to provide a technique capable of quantitatively predicting a facial morphology in consideration of a patient's facial expression after treatment.

In order to solve the above-mentioned problems, the present invention is a method of predicting the facial morphology at the time of facial expression expression after treatment of a patient by an arithmetic process executed by a computer device, and the arithmetic processing performs the treatment. Multiple feature variables selected in advance based on the case facial morphology data CF obtained from a plurality of patients (the past patients who have undergone the treatment are referred to as "case patients") using a three-dimensional measuring device. The step of extracting a set of multidimensional case feature vectors CV and the acquisition from a new patient who is considering treatment (the new patient is called an "evaluation target patient") using a three-dimensional measuring device. From the step of extracting the multidimensional patient feature vector PV having the plurality of feature variables as elements and the set of the case feature vector CVs for the plurality of the case patients, based on the patient facial morphology data PF. A step of selecting a plurality of approximate case feature vectors NCV that are close to the patient feature vector PV, and each case patient corresponding to the plurality of selected approximate case feature vectors NCV (the selected case patient is referred to as an “approximate case patient”). With regard to (), the step of calculating the pretreatment resting approximate case facial morphology model NC Pr-pre normalized based on the pretreatment resting case facial morphology data acquired using a three-dimensional measuring device at the time of pretreatment resting. For each of the above-mentioned approximate cases, the post-treatment facial expression expression-approximate case facial morphology normalized based on the post-treatment facial expression expression case facial morphology data acquired by using a three-dimensional measuring device at the time of post-treatment facial expression expression. The step of calculating the model NCMs-post, the step of calculating the vector average of the pretreatment resting approximate case facial morphology model NCMr-pre to obtain the pretreatment approximate case vector average NCSpre, and the posttreatment facial expression expression approximation. The step of calculating the vector average of the case facial morphology model NCMs-post to obtain the post-treatment approximate case vector average NCSpost and the approximate case vector average obtained by subtracting the pretreatment approximate case vector average NCSpre from the post-treatment approximate case vector average NCSpost. In the step of calculating the difference NCS post-pre, the step of calculating the resting patient facial morphology model PFMr normalized based on the patient facial morphology data PF of the evaluation target patient, and the resting patient facial morphology model PFMr. , The facial expression table predicted after the treatment of the patient to be evaluated by adding the approximate case vector average difference NCS post-pre. It is a method of predicting facial morphology at the time of facial expression expression after treatment, which includes a step of calculating a predicted facial morphology model PFMs-prd at the time of appearance.

In the facial morphology prediction method, the case facial morphological data CF for each of the patient patients is the pretreatment resting case facial morphological data CFr-pre and the pretreatment facial expression expression case facial morphological data CFs-pre. The pretreatment case morphology change amount CDpre in which the case feature vector CV is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre. The patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression, and the patient characteristics. It is preferable that the vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs.

Further, in the facial morphology prediction method, the calculation process calculates a normalized pre-correction hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the evaluation target patient, and the correction. The step of predicting the post-correction hard tissue morphology model HMpost of the patient to be evaluated based on the pre-hard tissue morphology model HMpre, and the post-correction hard tissue morphology model HMpost in the predicted facial morphology model PFMs-prd at the time of facial expression expression. It is preferable to further include a step of incorporating and displaying.

Further, in the facial morphology prediction method, it is preferable that a predetermined number of cases are selected in the order in which the approximate case feature vector NCV is closer to the patient feature vector PV.

Further, in the facial morphology prediction method, the calculation process includes a step of classifying the case feature vector CV into a plurality of case classes by performing a clustering process, and a step of calculating the cluster center of gravity G for each case class. Of the cluster centers of gravity G for each of the classified case classes, the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is the approximate case feature. It may be selected as the vector NCV.

Further, the present invention is a system for predicting the facial morphology at the time of facial expression expression after treatment of a patient, and is realized by arithmetic processing of a computer device, a feature vector extraction means, an approximate case selection means, and a prediction model. A case obtained by the feature vector extraction means from a plurality of treated patients (a past patient who has undergone the treatment is referred to as a "case patient") using a three-dimensional measuring device, including at least a calculation means. Based on the facial morphology data CF, a process of extracting a set of multidimensional case feature vectors CV having a plurality of preselected feature variables as elements, and a new patient who is considering treatment (the new patient) A process of extracting a multidimensional patient feature vector PV having the plurality of feature variables as elements based on the patient facial morphology data PF acquired by using a three-dimensional measuring device from the “patient to be evaluated”). Executed, the approximate case selection means executes a process of selecting a plurality of approximate case feature vectors NCV that are close to the patient feature vector PV from the set of the case feature vectors CV for the plurality of the case patients. The predictive model calculation means measures three-dimensionally at rest before treatment for each case patient (the selected case patient is referred to as an "approximate case patient") corresponding to the plurality of selected approximate case feature vectors NCV. Pretreatment resting case acquired using the device Process to calculate the pretreatment resting approximate case facial morphology model NCMR-pre normalized based on the facial morphology data, and the post-treatment facial expression table for each of the approximate case patients. Post-treatment facial expression-appearing case acquired using a three-dimensional measuring device at the time of delivery Normalized post-treatment facial expression-appearing approximate case Facial morphology model NCMs-post is calculated based on the facial morphology data, and the pre-treatment The process of calculating the vector average of the resting approximate case facial morphology model NCMr-pre to obtain the pretreatment approximate case vector average NCSpre and the vector average of the post-treatment approximate case facial morphology model NCMs-post are calculated. The process of obtaining the post-treatment approximate case vector average NCSpost, the process of calculating the approximate case vector average difference NCSpost-pre obtained by subtracting the pretreatment approximate case vector average NCSpre from the post-treatment approximate case vector average NCSpost, and the evaluation target. The process of calculating the resting patient face morphology model PFMr, which is normalized based on the patient face morphology data PF of the patient, and the resting patient face morphology model PFMr By adding the approximate case vector average difference NCS post-pre, the facial expression after treatment is executed, which is the process of calculating the predicted facial morphology model PFMs-prd at the time of facial expression expression, which is predicted after the treatment of the patient to be evaluated. It is a facial expression prediction system at the time of expression.

In the facial morphology prediction system, the case facial morphology data CF for each of the patient patients is the pretreatment resting case facial morphological data CFr-pre and the pretreatment facial expression expression case facial morphological data CFs-pre. The pretreatment case morphology change amount CDpre in which the case feature vector CV is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre. The patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression, and the patient characteristics. It is preferable that the vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs.

Further, in the facial morphology prediction system, the prediction model calculation means calculates a normalized pre-correction hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the evaluation target patient. The process of predicting the post-correction hard tissue morphology model HMpost of the patient to be evaluated based on the pre-correction hard tissue morphology model HMpre, and the post-correction hard tissue morphology model in the predicted facial morphology model PFMs-prd at the time of facial expression expression. It is preferable to further execute the process of incorporating and displaying the HMpost.

Further, in the facial morphology prediction system, it is preferable that a predetermined number of cases are selected in order in which the approximate case feature vector NCV is closer to the patient feature vector PV.

Further, in the facial morphology prediction system, the approximate case selection means calculates the cluster center of gravity G for each of the case classes and the process of classifying the case feature vector CV into a plurality of case classes by performing the clustering process. Among the cluster centers of gravity G for each of the classified case classes, the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is the case feature vector CV. It may be selected as the approximate case feature vector NCV.

According to the present invention, it is possible to easily and quantitatively predict the facial morphology at the time of facial expression expression after treatment of a patient. Therefore, it is possible to contribute to an appropriate judgment of the treatment policy in consideration of the facial expression of the patient.

It is a block diagram which illustrates the schematic structure of the face morphology prediction system. It is a figure which illustrates the case face morphology data CF. It is a figure which illustrates the case face morphology model CFM. It is a figure which illustrates the case morphology change amount CDpre before the treatment. It is a figure which illustrates the feature parameter selected from the front face of a human being. It is a face vertical cross-sectional view which further shows the selection example of a feature parameter. It is a face cross-sectional view which further shows the selection example of a feature parameter. It is a face cross-sectional view which further shows the selection example of a feature parameter. It is a figure which illustrates the case morphology feature vector CFV extracted from the case face morphology model CFM. It is a figure which illustrates the case morphology change amount feature vector CDV extracted from the case morphology change amount CD. It is a figure for demonstrating the outline of the face morphology prediction method. It is a flowchart which illustrates the outline of the face morphology prediction method. It is a figure for demonstrating the Example of the prediction model construction which concerns on this invention. It is a flowchart which shows the prediction model construction process by Example 3-1. It is a flowchart which shows the prediction model construction process by Example 3-2.

(Outline explanation of facial morphology prediction system)
FIG. 1 illustrates the schematic configuration of the facial morphology prediction system after orthodontic treatment. In the face morphology prediction system, various processing means described later are mainly realized by arithmetic processing of the computer device 10. A large-capacity database 20, an input device 30, an output device 40, and the like are connected to the computer device 10. The database 20 may be a hard disk or an optical disk directly connected to the computer device 10, or may be a data server or storage in a hospital, for example, which can be accessed from the computer device 10 via a network. Further, the database 20 may be provided in, for example, a cloud data center on a wide area network.

As will be described later, the database 20 contains X-ray photograph data, three-dimensional facial morphology data, and the like measured from a plurality of treated patients (past patients who have undergone the treatment are referred to as "case patients"). The primary data, the facial morphology model normalized based on the primary data, the intermediate data such as the feature vector which is a multivariate quantity extracted from the features, and the case data including the predicted facial morphology model which is the evaluation data are stored. It is preferable that the access permission to the case data such as the database 20 is restricted only to a specific person (for example, the doctor in charge) who is permitted to share and use the data.

The computer device 10 uses the case data stored in the database 20 to execute arithmetic processing for facial morphology prediction, which will be described later. The input device 30 includes an operation input device linked with a human interface, such as a keyboard, a mouse, and a touch panel. Further, the input device 30 may be a device having a function of inputting data acquired or processed by another system to the computer device 10 via an information storage medium or a network. The output device 40 includes, for example, an image display that three-dimensionally visualizes and displays predicted facial morphology data and a model. Further, the output device 40 may be a writing device of an information storage medium for providing data to another system or a communication device capable of outputting data to the outside via a network.

In addition, the facial morphology prediction system allows the patient's facial photograph data and three-dimensional facial morphological data taken in a hospital laboratory or the like to be taken into the computer device 10 via the database 20 or directly. It is configured. Therefore, this system includes a digital camera 61 and a three-dimensional measuring device 62. As the three-dimensional measuring device 62, a general optical measuring device such as a three-dimensional camera, a three-dimensional scanner, or a three-dimensional laser profiler can be used for a predetermined part such as the entire face of the patient or the occlusal portion. The facial morphology data of these patients may be input from the input device 30 via the information storage medium, or may be input to the system via, for example, a hospital network.

In the facial morphology prediction system of the present embodiment, since the hard tissue (dental skeleton) of the patient is photographed, the data of the X-ray photograph and the cephalo image may be input to the computer device 10 and / or the database 20. For this purpose, the system may be equipped with an X-ray inspection apparatus 63, for example, via a hospital network. In addition to the above-mentioned cephalo image, the X-ray inspection device 63 may include an inspection device capable of taking a panoramic X-ray photograph of a patient's occlusal portion, CT image data, and the like.

(Explanation of case data)
The "case data" accumulated in the database 20 will be specifically described. Here, the case data is a set CF (j = 1,2, ..., N) of "case facial morphology data CF" in which the faces of a plurality of case patients are measured using a three-dimensional measuring device 62 as primary data. )including. Here, N indicates the number of case patients (that is, the number of cases).

The case facial morphology data CF for each case patient is, in detail, "pretreatment resting case facial morphological data CFr-pre", "pretreatment facial expression expression case facial morphological data CFs-pre", and "post-treatment resting". Includes "case facial morphology data CFr-post" and "case facial morphology data CFs-post at the time of facial expression after treatment".

FIG. 2 shows an example of case facial morphology data CF. In the present embodiment, the “non-expressed facial expression” face specifically means a resting face, and the “expressed facial expression” face specifically refers to a smiling facial expression. It shall refer to the face of.

Pretreatment resting case facial morphology data CFr-pre refers to facial morphology data obtained by three-dimensionally measuring the resting face in a state in which the case patient does not express a facial expression before treatment.
Case facial morphology data at the time of facial expression expression before treatment CFs-pre refers to facial morphological data obtained by three-dimensionally measuring a face in which a case patient expresses a smiling facial expression, for example, before treatment.
Post-treatment resting case facial morphology data CFr-post refers to facial morphological data obtained by three-dimensionally measuring the resting face of a case patient who does not express a facial expression after treatment.
Case facial morphology data CFs-post at the time of facial expression expression before treatment refers to facial morphology data obtained by three-dimensionally measuring, for example, a face in which a case patient expresses a smiling facial expression after treatment.

By the way, the number of three-dimensional facial morphology data acquired differs depending on the size of the face of each patient. In addition, the position of the origin differs depending on the standing position of the patient who was photographed. In the facial morphology prediction system of the present embodiment, a morphological model that converts three-dimensional facial morphological data into a normalized facial morphological model in order to enable quantitative comparison and statistical processing of the facial morphology of each patient. The conversion means 110 is provided. The morphological modeling means 110 is normal, for example, by extracting predetermined anatomical feature points from the patient's three-dimensional facial morphology data and arranging the feature points on polygons having the same number of points and the same topological structure. Performs arithmetic processing to build a three-dimensional facial morphology model. The morphological model constructed by such a method is generally called a "homology model", and for example, an HBM (Homologous Body Modeling) program provided by AIST (National Institute of Advanced Industrial Science and Technology) can be used.

The morphological modeling means 110 normalizes the above-mentioned case facial morphological data CF for each case patient and performs a process of constructing a “case facial morphological model CFM”. FIG. 3 shows an example of a case facial morphology model CFM.

More specifically, the model data obtained by normalizing the pretreatment resting case facial morphology data CFr-pre is referred to as the pretreatment resting case facial morphology model CFMr-pre, and the pretreatment facial expression expression case facial morphology data CFs. The model data obtained by normalizing -pre is called "pretreatment facial expression expression case facial morphology model CFMs-pre", and the model data obtained by normalizing the post-treatment facial expression data CFr-post is referred to as "post-treatment resting time". The model data obtained by normalizing the case facial morphology data CFs-post at the time of facial expression expression after treatment is referred to as "case facial morphology model CFMs-post".
In the database 20, a set CFM (j = 1,2, ..., N) of the case facial morphology model CFM corresponding to the calculated case patient N is accumulated.

The case data further includes the "case morphological change amount CD" for N case patients. In the present embodiment, the case morphology change amount CD for each case patient includes, in detail, "pre-treatment case morphology change amount CD pre" and "post-treatment case morphology change amount CD post".

The morphological change amount calculation means 120 calculates the change amount of the pretreatment resting case facial morphology model CFMr-pre and the pretreatment facial expression expression case facial morphological model CFMs-pre to obtain the “pretreatment case morphological change amount CDpre”. To get. Further, the morphological change amount calculation means 120 calculates the amount of change in the post-treatment resting case facial morphology model CFMr-post and the post-treatment facial expression expression case facial morphological model CFMs-post to obtain a “post-treatment case morphological change amount”. Get "CDpost". FIG. 4 shows an example in which the case morphology change amount CDpre is obtained from the case facial morphology models CFMr-pre and CFMs-pre before treatment.

That is, the "morphological change amount" includes information on the amount of change in soft tissue and the direction when the patient changes from a resting face to a smiling face, and these can be displayed as three-dimensional image data. ..

Then, in the database 20, a set CDpre (j = 1,2, ..., N) of the pretreatment case morphology change amount CDpre corresponding to the calculated N case patients and a set CDpost of the posttreatment case morphology change amount CDpost are stored. (j = 1,2, ..., N) is accumulated.

(Explanation of knowledge base)
The feature vector extraction means 130 obtains a multidimensional "case morphology feature vector CFV" having a plurality of "feature variables" (values of feature parameters) selected in advance from the case face morphology model CFM for each case patient. Perform the extraction process.

Here, the "feature parameter" is a geometric parameter that characteristically represents a morphology such as a human face, and is selected in advance by a specialist, for example, based on his / her experience and knowledge. Here, some explanations will be added about the feature parameters and feature variables.

FIG. 5 shows an example of feature parameters selected in the human face outline. As shown in FIG. 5, the human face can recognize some inflection points in its morphology. For such an inflection point, the corner of a boundary line such as an eye or nose, the most prominent position in three dimensions, the most recessed position, or the like can be selected. In the present specification, such inflection points are referred to as "landmarks" and are used in the definition of feature parameters. The landmark is not particularly limited as long as it is not an inflection point but can be geometrically defined, such as the center point of a straight line connecting two inflection points.

The outline of the face can be extracted as follows. First, a surface normal at each pixel of the three-dimensional surface data is calculated by an arithmetic program customized for measuring the facial morphology from the frontal image of the face. In addition, the angle formed by the z-axis and the face normal of the face is also calculated for each coordinate of the face surface. Each coordinate point where the angle formed by the z-axis and the face normal is, for example, 60 degrees is extracted, and the line connecting these points is used as the outline of the face. The angle that defines the outline of the face is preferably an angle between 45 degrees and 90 degrees.

One example of a feature parameter is the distance between landmarks. For example, the feature parameter v1 shown in FIG. 5 is defined as the distance between the outer corners of the eyes Ex (| Ex-Ex |). Also, as in v3, the distance between the landmark (for example, the line connecting the outermost end Zy'of the face and the chin point Gn) and the landmark (for example, the cheek point Go') There may be. Another example of a feature parameter is the angle of the line connecting the landmarks. For example, the angle of the feature parameter v4 is determined by the positional relationship between the outermost end Zy'of the face, the cheek point Go', and the cheek.

The characteristic parameter of the distance may be a dimensionless quantity. For example, a width (| Ch-Ch | / | Ex-Ex |) in which the corner width of the mouth (| Ch-Ch |) is normalized by the distance between the outer corners of the eyes (| Ex-Ex |) can be adopted as a feature parameter. In addition, deviations with respect to a plurality of average values and ratios with respect to the average may be considered as feature parameters.

Further, as shown in FIGS. 6 to 8, a plurality of feature parameters are selected from a cross section based on three-dimensional data obtained by photographing a specific part of a human face. These cross sections are created by data processing based on anatomical measurement points after determining the three-dimensional coordinate system. FIG. 6 shows, as an example, a yz cross section when the subject's face is cut at a line connecting the outer corner of the eye Ex and the corner point Ch of the mouth. For example, the angle (v7) of the mouth angle Ch with the outer corner Ex as the base point in the z-axis direction, the angle (v8) of the cheek protrusion P (Ex-Ch) in the cross section with the outer corner Ex as the base point, the outer corner Ex and the mouth angle Ch. The length of the outer curve (v12), the area closed by the outer curve (v13), and the like can be selected as feature parameters.

As an additional example, FIG. 7 shows an xz cross section when the subject's face is cut in a horizontal plane passing through the subnasal point Sn. Similarly, FIG. 8 illustrates an xz cross section when the subject's face is cut in a horizontal plane passing through the most apex point Pm of the nose. As shown in these figures, the amount of protrusion of the facial part in the z direction (v14, v18), the angle of the apex (v16, v20), and the amount of protrusion (v17, v22, v23) at various cross-sectional positions. , The angle of the concave point (v21) and the like can be selected as feature parameters. Although not shown, the cross section that characterizes the facial morphology may be a cross section that passes through, for example, the glabellar point Gla, the nose root point N, the upper lip point Ls, the lower lip point Li, and the chin point Sm. Further, a difference or a ratio with respect to the z average value of a specific part may be added to the feature parameter.

The feature vector extraction means 130 performs a process of measuring feature variables corresponding to each of a plurality of selected and set feature parameters from the patient's three-dimensional facial morphology data. The feature vector extraction means 130 extracts an n-dimensional "feature vector V" having the measured n feature variables v as vector elements.

As described above, the feature vector extraction means 130 is a multidimensional "case morphology feature vector" based on the case facial morphology data CF before and after the treatment for each case patient and at rest and when the smiling facial expression is expressed. Perform the process of extracting "CFV". FIG. 9 shows an example of extracting the case morphology feature vector CFV from the case face morphology model CFM.

The feature vector extracted based on the pretreatment resting case facial morphology data CFr-pre is called the pretreatment resting case morphological feature vector CFVr-pre, and is used as the pretreatment facial expression expression case facial morphology data CFs-pre. The feature vector extracted based on this is called "pretreatment facial expression expression case morphology feature vector CFVs-pre", and the feature vector extracted based on post-treatment resting case facial morphology data CFr-post is called "post-treatment rest". The feature vector extracted based on the case facial morphology data CFs-post at the time of facial expression expression after treatment is called "case morphology feature vector CFVs-post at the time of facial expression expression after treatment". Refer to.

Further, the feature vector extraction means 130 can also extract a multidimensional "case morphology change amount feature vector CDV" from the case morphology change amount CD showing the soft tissue change amount at rest and when smiling for each case patient.

FIG. 10 shows an example of extracting the case morphology change amount feature vector CDV from the case morphology change amount CD. The feature vector extracted based on the pretreatment case morphological change CDpre is called "pretreatment case morphological change feature vector CDVpre", and the feature vector extracted based on the posttreatment case morphological change CDpost is called "post-treatment case". It is called "morphological change amount feature vector CDV post".

The "case feature vector CV" used as the basal variate when selecting the "approximate case patient" described later is the above-mentioned pretreatment resting case morphology feature vector CFVr-pre and the pretreatment facial expression expression case morphology feature. Either one feature vector selected from the group of vector CFVs-pre and pretreatment case morphological change feature vector CDVpre, or an extended feature vector obtained by combining the feature variables of two or more feature vectors. it can.

That is, as a specific example of the "case feature vector CV",
(Example 1) Pretreatment resting case Morphological feature vector CFVr-pre
(Example 2) Case morphology feature vector CFVs-pre when facial expression is expressed before treatment
(Example 3) Pretreatment case morphological change feature vector CDVpre
(Example 4) Pretreatment resting case morphological feature vector CFVr-pre + pretreatment facial expression expression case morphological feature vector CFVs-pre
(Example 5) Pretreatment resting case morphological feature vector CFVr-pre + pretreatment case morphological change feature vector CDVpre
(Example 6) Case morphological feature vector CFVs-pre + pretreatment case morphological change feature vector CDVpre
And so on.

In this way, the database 20 contains a set of case feature vector CVs CV (1) and CV (2), which are extracted from the case morphology feature vector CFV and / or the case morphology deformation amount feature vector CDV and correspond to N case patients. ), CV (3), ..., CV (N) are made into knowledge. Hereinafter, the set of the case feature vector CV is referred to as CV (j = 1,2, ..., N) or CV (j).

A set of case feature vectors CVs for a plurality of case patients may be clustered and knowledgeed in the database 20. As the clustering process, a general vector quantization method such as the Lloyd method or the k-means method can be used.

For example, according to the k-means method, the clustering process of the case feature vector CV can be performed as follows. First, the number C1 of the primary cluster is arbitrarily set, and a temporary cluster CL ^* (l = 1,2, ..., C1) is assigned to the vector space of n dimensions (n is the number of feature variables v). Next, the average of the case feature vector CV belonging to each primary cluster CL ^* (l) is calculated to obtain the primary cluster center of gravity G ^* (l = 1,2, ..., C1). Then, the distance D ^* (l, i) = | G ^* (l) -CV (i) | between each center of gravity G ^* (l) of the obtained C1 and all the case feature vectors CV is obtained. Here, the "distance" between the vectors may be either the Euclidean distance or the Manhattan distance.

Next, the primary cluster center of gravity G ^* (l) at the shortest distance is searched from each case feature vector CV, and the group of the case feature vector CV having the shortest distance center of gravity G ^* (l) as an element is used. Reorganize the next cluster CL ^** (l). Then, the secondary cluster center of gravity G ^** (l) is also obtained in the secondary cluster CL ^** (l), and the tertiary cluster center of gravity G ^*** (l) is obtained from the group of the case feature vector CV at the shortest distance. The case feature vector CV of each case patient is classified into C1 cluster (case class) CL (l = 1,2, ..., C1) converged by repeating such a cluster reorganization cycle. Can be done.

Subsequently, the process of optimizing the number of clusters (that is, the case class) may be performed by the following algorithm.
First, the number C of candidate clusters is set in a reasonable range such as 3, 4, ..., 12, and the center of gravity Gc (l =) of each cluster classified by the number of each cluster.

Find

1,2,…, C). The distance Dc (l, j) between each center of gravity Gc (l) obtained in each cluster and the case feature vector CV (j = 1,2, ..., N) belonging to the cluster of each center of gravity Gc (l) is set. The calculation is performed, and the minimum distance Dc (l) min is obtained.

The inter-cluster distance Dc, which is the average value of the minimum distance Dc (l) min of each cluster, is calculated by the mathematical formula (1).

For each candidate C (for example, 3, 4, ..., 12) of the number of clusters, the inter-cluster distance Dc (for example, D ₃ , D ₄ , ..., D ₁₂ ) is obtained, and the change shown in the formula (2) is obtained. C + 1, which is obtained by adding 1 to C having the maximum ΔDc, can be determined as the optimum number of clusters.

The case class classification means 180 performs such a clustering process on the case feature vector CV (j = 1,2, ..., N) of each case patient, thereby performing C cluster centers of gravity G (l = 1,2). , ..., C) can be calculated and classified into C case classes CL (l = 1,2, ..., C). The data of the case feature vector CV classified into each case class CL and the cluster center of gravity G thereof are knowledgeable in the database 20.

(Explanation of facial morphology prediction method)
By utilizing the above-mentioned knowledge base of case data in the above-mentioned facial morphology prediction system by the method described below, it is possible to quantitatively predict the facial morphology at the time of facial expression expression after treatment of a patient. FIG. 11 is a block diagram showing an outline of the face morphology prediction method, and FIG. 12 is a flowchart thereof.

First, in step S1, the face of a new patient under consideration for treatment (the new patient is referred to as an "evaluation target patient") is measured using a three-dimensional measuring device 62 before treatment, and the patient's facial morphology. Acquire the data PF. At this time, the state in which the patient does not express a facial expression, that is, the "resting patient facial morphology data PFr" which is the facial morphological data obtained by measuring the face at rest, and the facial morphological data measuring the face when the smiling facial expression is expressed. At least two types of patient facial morphology data PFs of "facial expression expression patient facial morphology data PFs" are acquired.

The morphology modeling means 110 calculates a "patient facial morphology model PFM" normalized by using, for example, the above-mentioned homology model algorithm, based on the patient facial morphology data PF. The facial morphology model constructed based on the resting patient facial morphology data PFr is called "resting patient facial morphology model PFMr", and the facial morphological model constructed based on the facial expression expression patient facial morphology data PFs is called "facial expression expression". When referred to as "patient facial morphology model PFMs".

Further, the morphological change amount calculation means 120 obtains the "patient morphological change amount PD" by calculating the change amount of the resting patient facial morphology model PFMr and the facial expression expression patient facial morphological model PFMs.

Next, the feature vector extraction means 130 extracts a multidimensional "patient feature vector PV" having a plurality of feature variables as elements from the patient facial morphology model PFM and / or the patient morphology change amount PD.

The feature vector extracted based on the resting patient facial morphology model PFMr is called "resting patient morphological feature vector PFVr", and the feature vector extracted based on the facial expression expression patient facial morphology model PFMs is called "facial expression expression". When referred to as "patient morphological feature vector PFVs". Further, the feature vector extracted based on the patient morphology change amount PD is referred to as "patient morphology change amount feature vector PDV".

The above-mentioned "patient feature vector PV" is any one feature vector selected from the group of resting patient morphology feature vector PFVr, facial expression expression patient morphology feature vector PFVs, and patient morphology change feature vector PDV, or An extended feature vector can be obtained by combining the feature variables of these two or more feature vectors.

That is, as a specific example of the "patient feature vector PV",
(Example 1) Resting patient morphology feature vector PFVr
(Example 2) Patient morphology feature vector PFVs when expressing facial expressions
(Example 3) Patient morphology change characteristic vector PDV
(Example 4) Resting patient morphology feature vector PFVr + Facial expression expression patient morphology feature vector PFVs
(Example 5) Resting patient morphology feature vector PFVr + patient morphology change feature vector PDV
(Example 6) Patient morphology feature vector PFVs + patient morphology change feature vector PDV at the time of facial expression expression
And so on.

Next, in step S2, the approximate case selection means 140 performs a process of selecting an approximate case using the case feature vector CV for a plurality of case patients as a basal variable. Specifically, a plurality of case feature vector CVs that are close to the patient feature vector PV are selected from the set CV (j = 1,2, ..., N) of the case feature vector CV. The case feature vector CV selected here that approximates the patient feature vector PV is referred to as an “approximate case feature vector NCV”.

It is preferable that the population of the case feature vector CV is narrowed down to cases in which, for example, gender, age, treatment site, hard tissue (dentition, etc.) are common or similar to the patient to be evaluated.

The combination of the patient feature vector PV and the case feature vector CV whose distance is compared with it is the same. Examples of their combinations are given below.

(Example 1-1)
For example, when "resting patient morphology feature vector PFVr" is adopted as the patient feature vector PV, the case feature vector CV becomes "pretreatment resting case morphology feature vector CFVr-pre".

(Example 1-2)
Further, when "patient morphology feature vector PFVs at the time of facial expression expression" is adopted as the patient feature vector PV, the case feature vector CV becomes "case morphology feature vector CFVs-pre at the time of facial expression expression before treatment".

(Example 1-3)
Further, when the "patient morphology change amount feature vector PDV" is adopted as the patient feature vector PV, the case feature vector CV becomes the "pretreatment case morphology change amount feature vector CDV pre".

In particular, since the morphological change amount feature vector of Example 1-3 includes information such as the deformation amount of the soft tissue of the face, the deformation direction, and the softness of the tissue as the feature amount, an approximate case is selected using this as the basal variation. As a result, the accuracy of facial morphology prediction when the smiling facial expression of the evaluation target patient is expressed can be increased.

A specific example of the process of selecting the approximate case feature vector NCV in step S2 of FIG. 12 will be described.

(Example 2-1)
The approximate case selection means 140 can select the case feature vector CV having a predetermined number of cases k in ascending order of distance from the patient feature vector PV. The approximate number of cases k is a number determined by an empirical judgment of a specialist doctor or the like.
As described above, the set CV (j) of the case feature vector CV corresponding to N case patients is knowledgeable in the database 20. The approximate case selection means 140 selects approximate cases having the number of cases k in ascending order of distance (| CV (j) -PV |) between each case feature vector CV (j) and the extracted patient feature vector PV. .. Here, the "distance" between the vectors may be either the Euclidean distance or the Manhattan distance.

(Example 2-2)
Further, the approximate case selection means 140 may select the case feature vector CV belonging to the case class CL having the cluster center of gravity G closest to the patient feature vector PV as the approximate case feature vector NCV.
As described above, the data of the case feature vector CV classified into each case class CL and the cluster center of gravity G thereof are knowledgeable in the database 20. The approximate case selection means 140 is a patient feature vector PV among the center of gravity G (l = 1,2, ..., C) of the case feature vector CV belonging to each case class CL (l = 1,2, ..., C). The approximate case class NCL having the vector centroid G with the closest distance (| G (l) -PV |) is selected. The "distance" in this case may be either the Euclidean distance or the Manhattan distance.
Then, the approximate case selection means 140 selects a set of all the case feature vectors CV belonging to the approximate case class NCL as the approximate case feature vector NCV.

Next, in step S3 of FIG. 12, the prediction model calculation means 150 predicts after the treatment of the evaluation target patient based on the patient face morphology model PFM of the evaluation target patient, and the “predicted face” at the time of expressing a smiling expression is predicted. The morphological model PFMs-prd "is calculated. An example of the prediction model construction process will be described below with reference to FIGS. 13 to 15.

(Example 3-1)
According to this Example 3-1 as shown in FIG. 13, the predicted facial morphology model PFMs-prd at the time of smiling, which is predicted after treatment, is based on the patient facial morphological model PFMs at the time of smiling of the evaluation target patient. Calculate.

The prediction model construction process according to the present embodiment will be described in more detail based on the flowchart of FIG. First, in step S11, a set of "pretreatment facial expression expression approximate case facial morphology model NCMs-pre" normalized based on pretreatment facial expression expression case facial morphology data NCFs-pre for each approximate case patient. Is calculated.
Further, in step S12, a normalized “post-treatment facial expression expression approximate case facial morphology model NCMs-post” is calculated based on the post-treatment facial expression expression case facial morphology data NCFs-post for each approximate case patient. To do.

In step S13, the vector average of the set of the approximate case facial morphology model NCMs-pre at the time of expressing the facial expression before treatment is calculated to obtain the "pretreatment approximate case vector average NCApre". Further, in step S14, the vector average of the set of the approximate case facial morphology model NCMs-post at the time of facial expression expression after treatment is calculated to obtain the “post-treatment approximate case vector average NCApost”.

In step S15, the "approximate case vector average difference NCApost-pre" is calculated by subtracting the pretreatment approximate case vector average NCApre from the post-treatment approximate case vector average NCApost.

In step S16, the approximate case facial morphology vector average difference NCA post-pre calculated in step S15 is added to the patient facial morphology model PFMs when the facial expression of the patient to be evaluated is expressed. As a result, the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression, which is predicted after the treatment of the patient to be evaluated, is calculated.

(Example 3-2)
Further, as shown in FIG. 13, the prediction model calculation means 150 obtains the prediction face morphology model PFMs-prd at the time of smile, which is predicted after the treatment, based on the patient face morphology model PFMr at rest of the evaluation target patient. You may calculate.

The prediction model construction process according to the present embodiment will be described in more detail based on the flowchart of FIG. First, in step S21, a set of normalized “pretreatment resting approximate case facial morphology model NCMR-pre” is calculated based on the pretreatment resting case facial morphology data NCFr-pre for each approximate case patient.
Further, in step S22, a normalized “post-treatment facial expression expression approximate case facial morphology model NCMs-post” is calculated based on the post-treatment facial expression expression case facial morphology data NCFs-post for each approximate case patient. To do.

In step S23, the vector average of the set of the pretreatment approximate case facial morphology model NCMr-pre is calculated to obtain the "pretreatment approximate case vector average NCApre". Further, in step S24, the vector average of the set of the approximate case facial morphology model NCMs-post at the time of facial expression expression after treatment is calculated to obtain the “post-treatment approximate case vector average NCApost”.

In step S25, the "approximate case vector average difference NCApost-pre" is calculated by subtracting the pretreatment approximate case vector average NCApre from the post-treatment approximate case vector average NCApost.

In step S26, the approximate case facial morphology vector average difference NCA post-pre calculated in step S25 is added to the resting patient facial morphology model PFMr of the patient to be evaluated. As a result, the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression, which is predicted after the treatment of the patient to be evaluated, is calculated.

Next, in step S4 (see FIG. 12) of displaying the predicted patient facial morphology model, an image of the tooth alignment after treatment is output on a display or the like together with the predicted facial morphology model PFMs-prd at the time of expressing a smiling facial expression. It is preferable to display it on the device 40.

In that case, the arithmetic processing by the computer device 10 is a step of calculating a normalized pre-orthodontic hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the tooth skeleton of the patient to be evaluated, and the pre-orthodontic hardness. Prediction in the step of predicting the corrected hard tissue morphology model HMpost of the patient to be evaluated based on the tissue morphology model HMpre, and the prediction facial morphology model PFMs-prd at the time of expressing a smiling expression after treatment evaluated in step S3. The step of incorporating the corrected hard tissue morphology model HMpost, which has been performed, can be included.

The image data of the hard tissue is, for example, three-dimensional image data obtained by measuring the tooth portion of the patient with a three-dimensional measuring device, a CT image including the tooth skeleton of the occlusal part, a cephalo image, a panoramic X photograph, and the like. The HMpre may be constructed by combining these two-dimensional and / or three-dimensional image data.

By narrowing down the case data common to the hard tissue structure for which orthodontic treatment is planned and executing the above-mentioned prediction model construction process, the tooth alignment predicted after the treatment is displayed on the image of the predicted facial morphology when the smiling facial expression is expressed. Can be made to. For example, by combining a three-dimensional dentition prediction image or a two-dimensional dentition prediction image with the mouth portion of the predicted smile, it is possible to make a prediction including the appearance of the tooth alignment. In addition, while moving the hard tissue on the image, it is possible to visualize and simulate how the smile after treatment is improved by following the movement.

According to the facial morphology prediction system and the facial morphology prediction method described above, the three-dimensional facial morphology of a patient after orthodontic treatment can be predicted easily and quantitatively. In particular, in the present embodiment, since the facial morphology at the time of expressing the smiling facial expression can be predicted in advance before the treatment, it is necessary to make an appropriate judgment as to whether or not the treatment plan creates a "good smile" that leads to aesthetic improvement of the facial expression for the patient. Can contribute.

The facial morphology prediction system and the facial morphology prediction method according to the present invention can be used not only for orthodontic treatment but also for surgical treatment of patients with jaw deformities, for example. It can also be used, for example, to predict when maxillofacial surgery (including oral surgery and plastic surgery) is performed alone, orthodontic treatment, or jointly with jaw prosthesis treatment. Furthermore, its application can be expected for the prediction of age-related changes in facial morphology.

10 Computer device 20 Database 30 Input device 40 Output device 62 Three-dimensional measurement device 63 X-ray inspection device 110 Morphological modeling means 120 Morphological change amount calculation means 130 Feature vector extraction means 140 Approximate case selection means 150 Predictive model calculation means CF Case face Morphological data CFM Case facial morphology model CD Case morphological change CV Case feature vector CFV Case morphological feature vector CDV Case morphological change Feature vector PF Patient facial morphological data PFM Patient facial morphological model PFMs-prd Predicted facial morphological model at the time of facial expression expression PD Patient morphology change PV Patient feature vector PDV Patient morphology change Feature vector NCV Approximate case Feature vector NCApre Pre-treatment approximate case vector Average NCApost Post-treatment approximate case vector Average NCApost-pre Approximate case vector Average difference N Cases Number of patients (number of cases) ) CL case class NCL approximate case class G cluster center of gravity

Claims

It is a method of predicting the facial morphology at the time of facial expression expression after treatment of a patient by arithmetic processing executed by a computer device.
The arithmetic processing
A plurality of preselected patients based on case facial morphology data CF obtained from a plurality of treated patients (past patients who have undergone the treatment are referred to as "case patients") using a three-dimensional measuring device. Steps to extract a set of multidimensional case feature vector CVs with feature variables as elements,
Based on the patient facial morphology data PF obtained from a new patient who is considering treatment (the new patient is referred to as an "evaluation target patient") using a three-dimensional measuring device, the plurality of feature variables are elements. Steps to extract the multidimensional patient feature vector PV
A step of selecting a plurality of approximate case feature vectors NCV that are close to the patient feature vector PV from the set of the case feature vectors CV for the plurality of the case patients.
Treatment acquired by using a three-dimensional measuring device at rest before treatment for each of the selected case patients (the selected case patient is referred to as "approximate case patient") corresponding to the plurality of selected approximate case feature vectors NCV. Pre-resting case Normalized treatment based on facial morphology data Pre-resting approximate case Facial morphology model NCMr-pre calculation step and
For each of the approximate case patients, a post-treatment facial expression expression approximate case facial morphology model normalized based on the post-treatment facial expression expression case facial morphology data acquired using a three-dimensional measuring device at the time of post-treatment facial expression expression. Steps to calculate NCMs-post and
A step of calculating the vector average of the pretreatment approximate case facial morphology model NCMR-pre to obtain the pretreatment approximate case vector average NC Spre.
The step of calculating the vector average of the approximate case facial morphology model NCMs-post at the time of expressing the facial expression after the treatment to obtain the vector average NCSpost of the approximate case after the treatment.
A step of calculating the approximate case vector average difference NCSpost-pre obtained by subtracting the pretreatment approximate case vector average NCSpre from the post-treatment approximate case vector average NCSpost.
A step of calculating a resting patient facial morphology model PFMr normalized based on the patient facial morphology data PF of the patient to be evaluated.
By adding the approximate case vector average difference NCSpost-pre to the resting patient facial morphology model PFMr, the predicted facial expression model PFMs-prd at the time of facial expression expression, which is predicted after the treatment of the patient to be evaluated, is calculated. A method for predicting facial morphology at the time of facial expression expression after treatment, including steps to be performed.
The case facial morphology data CF for each of the case patients includes pretreatment resting case facial morphological data CFr-pre and pretreatment facial expression expression case facial morphological data CFs-pre.
The case feature vector CV is extracted based on the pretreatment case morphology change amount CDpre, which is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre. Being done
The patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression expression.
The treatment according to claim 1, wherein the patient feature vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs. How to predict facial morphology when expressing facial expressions later.
The arithmetic processing
A step of calculating a normalized pre-orthodontic hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the patient to be evaluated, and
A step of predicting the post-correction hard tissue morphology model HMpost of the patient to be evaluated based on the pre-correction hard tissue morphology model HMpre, and
The facial expression at the time of facial expression after treatment according to claim 1 or 2, further comprising a step of incorporating and displaying the corrected hard tissue morphology model HMpost in the predicted facial morphology model PFMs-prd at the time of facial expression expression. Morphological prediction method.
The method for predicting facial morphology at the time of facial expression expression after treatment according to claim 1 or 2, wherein a predetermined number of cases are selected in the order in which the approximate case feature vector NCV is closer to the patient feature vector PV.
The arithmetic processing
A step of classifying the case feature vector CV into a plurality of case classes by performing clustering processing, and
Including a step of calculating the cluster center of gravity G for each of the case classes.
Of the cluster centers of gravity G for each of the classified case classes, the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is defined as the approximate case feature vector NCV. The method for predicting facial morphology at the time of facial expression expression after treatment according to claim 1 or 2, which is selected.
It is a system that predicts the facial morphology at the time of facial expression after treatment of a patient.
It includes at least a feature vector extraction means, an approximate case selection means, and a prediction model calculation means realized by arithmetic processing of a computer device.
The feature vector extraction means
A plurality of preselected patients based on case facial morphology data CF obtained using a three-dimensional measuring device from a plurality of treated patients (past patients who have undergone the treatment are referred to as "case patients"). Processing to extract a set of multidimensional case feature vector CVs with feature variables as elements,
Based on the patient facial morphology data PF obtained from a new patient who is considering treatment (the new patient is referred to as an "evaluation target patient") using a three-dimensional measuring device, the plurality of feature variables are elements. Execute the process of extracting the multidimensional patient feature vector PV
The approximate case selection means executes a process of selecting a plurality of approximate case feature vectors NCV that are close to the patient feature vector PV from the set of the case feature vectors CV for the plurality of the case patients.
The predictive model calculation means
Treatment acquired by using a three-dimensional measuring device at rest before treatment for each of the selected case patients (the selected case patient is referred to as "approximate case patient") corresponding to the plurality of selected approximate case feature vectors NCV. Treatment normalized based on pre-resting case facial morphology data Processing to calculate pre-resting approximate case facial morphology model NCMR-pre,
For each of the approximate case patients, a post-treatment facial expression expression approximate case facial morphology model normalized based on the post-treatment facial expression expression case facial morphology data acquired by using a three-dimensional measuring device at the time of post-treatment facial expression expression. Processing to calculate NCMs-post and
The process of calculating the vector average of the pretreatment approximate case facial morphology model NCMR-pre to obtain the pretreatment approximate case vector average NC Spre,
The process of calculating the vector average of the approximate case facial morphology model NCMs-post at the time of expressing the facial expression after treatment to obtain the vector average NCSpost of the approximate case after treatment.
The process of calculating the approximate case vector average difference NCSpost-pre obtained by subtracting the pretreatment approximate case vector average NCSpre from the post-treatment approximate case vector average NCSpost.
A process of calculating the resting patient facial morphology model PFMr normalized based on the patient facial morphology data PF of the evaluation target patient, and
By adding the approximate case vector average difference NCSpost-pre to the resting patient facial morphology model PFMr, the predicted facial expression model PFMs-prd at the time of facial expression expression, which is predicted after the treatment of the patient to be evaluated, is calculated. A facial morphology prediction system at the time of facial expression expression after treatment, which executes the processing to be performed.
The case facial morphology data CF for each of the case patients includes pretreatment resting case facial morphological data CFr-pre and pretreatment facial expression expression case facial morphological data CFs-pre.
The case feature vector CV is extracted based on the pretreatment case morphology change amount CDpre, which is the change amount of the pretreatment resting case facial morphology data CFr-pre and the pretreatment facial expression expression case facial morphology data CFs-pre. Being done
The patient facial morphology data PF for the evaluation target patient includes the resting patient facial morphology data PFr of the evaluation target patient and the patient facial morphology data PFs at the time of facial expression expression.
The treatment according to claim 6, wherein the patient feature vector PV is extracted based on the patient morphology change amount PD, which is the change amount of the resting patient face morphology data PFr and the facial expression expression patient face morphology data PFs. Facial morphology prediction system when expressing facial expressions later.
The predictive model calculation means
A process of calculating a normalized pre-orthodontic hard tissue morphology model HMpre based on image data obtained by photographing the hard tissue including the teeth of the patient to be evaluated, and
A process of predicting the post-correction hard tissue morphology model HMpost of the patient to be evaluated based on the pre-correction hard tissue morphology model HMpre, and
The process of incorporating and displaying the corrected hard tissue morphology model HMpost in the predicted facial morphology model PFMs-prd at the time of facial expression expression, according to claim 6 or 7, at the time of facial expression expression after treatment. Facial morphology prediction system.
The facial morphology prediction system at the time of facial expression expression after treatment according to claim 6 or 7, wherein a predetermined number of cases are selected in the order in which the approximate case feature vector NCV is closer to the patient feature vector PV.
The approximate case selection means
A process of classifying the case feature vector CV into a plurality of case classes by performing a clustering process, and
For each of the case classes, the process of calculating the cluster center of gravity G is executed.
Of the cluster center of gravity G for each of the classified case classes, the case feature vector CV belonging to the case class having the cluster center of gravity closest to the patient feature vector PV is defined as the approximate case feature vector NCV. The facial morphology prediction system at the time of facial expression expression after the treatment according to claim 6 or 7, which is selected.