JP2002015307A - Generating method for geometric model - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve better matching without causing a local abnormal deformation on a local section such as the eyes or the mouth. SOLUTION: A standard model is deformed based on measured data to generate a geometric model by this method. The standard model is deformed based on the overall evaluation of two or more functions among a function related to the distance between the standard model and the measured data, a function related to the distance between a feature point defined on the standard model and a feature point defined on the measured data, and a function related to the distance between a contour defined on the standard model and a contour defined on the measured data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for generating a shape model, and is used, for example, for generating a three-dimensional model in the field of computer graphics.

[0002]

2. Description of the Related Art In recent years, three-dimensional CG (three-dimensional computer graphics) technology is often used for movies, games, and the like. In three-dimensional CG, virtual 3
Since a three-dimensional model or a light is arranged and moved in a three-dimensional space, the degree of freedom of expression is high.

Conventionally, a non-contact type three-dimensional measuring device using a light-section method or the like has been put into practical use, and it is possible to relatively easily create three-dimensional data of an object by performing measurement using the device. Can be. However, if the three-dimensional data obtained by the measurement is used as it is for the three-dimensional CG, there are various problems, such as complicated processing for reducing the data amount by thinning the obtained data.

To cope with this problem, there has been proposed a method in which a standard model of an object is prepared, and the standard model is deformed in accordance with the measured three-dimensional data (Japanese Patent Laid-Open No. 5-81377).

In this conventional method, three-dimensional shape information of three-dimensional data obtained by measurement, that is, a three-dimensional point group is used as a fitting target, and the surface of a standard model is fitted to the three-dimensional point group. Let it.

[0006]

However, in the above-described conventional method, although the overall shape almost matches, it is difficult to match local features.

That is, for example, when a three-dimensional model of a human head is created by a conventional method, the completed 3D model is obtained.
In the dimensional model, the overall shape of the head matches the head of the person to be measured.However, for local parts that greatly affect the facial expression, such as the eyes or mouth, there is sufficient It does not lead to a match. Therefore, it has been difficult to express a person's detailed expression.

[0008] Further, for example, when an attempt is made to match the contours and feature points of a face using two-dimensional image information, local deformation occurs in the standard model, which tends to cause abnormal deformation. The present invention has been made in view of the above-described problems, and provides a method of generating a shape model that can better match a local portion such as an eye or a mouth without causing local abnormal deformation. Aim.

[0009]

According to the method of the present invention,
A method of generating a shape model by deforming a standard model based on measurement data, which is a function related to the distance between the standard model and the measurement data, a feature point defined on the standard model and a measurement on the measurement data. Two or more functions related to the distance between the specified feature points and the function related to the distance between the contour defined on the standard model and the contour specified on the measurement data; Deform the standard model based on the comprehensive evaluation of the function.

For example, each function is an energy function. The standard model is deformed by comprehensive evaluation so that the total value of two or more energy functions is minimized. Further, excessive deformation of the standard model is avoided. Is defined, and the standard model is deformed by performing comprehensive evaluation so that the total value of these four energy functions is minimized.

[0011] Preferably, the weight of each function in the comprehensive evaluation is changed according to the reliability of the measurement data itself and the feature points and contours specified on the measurement data. Three-dimensional data is used as the measurement data. It is also possible to use a two-dimensional image as the measurement data.

In the present invention, the term “fitting” refers to a process of transforming a standard model or a series of processes including the process. As the fitting target, the constituent points themselves, the contour, the feature points, and the like of the measurement data are used.

[0013]

FIG. 1 is a block diagram showing a modeling device 1 according to the present invention. In the present embodiment,
An example of generating a three-dimensional model of a human head by deforming (fitting) a standard model created in advance based on measurement data (three-dimensional data or a two-dimensional image) of the human head will be described. .

As shown in FIG. 1, the modeling device 1
It comprises a processing device 10, a magnetic disk device 11, a medium drive device 12, a display device 13, a keyboard 14, a mouse 15, a three-dimensional measuring device 16, and the like.

The processing device 10 includes a CPU, a RAM, an RO,
M, a video RAM, an input / output port, and various controllers. Various functions are realized on the processing device 10 by the CPU executing programs stored in the RAM, the ROM, and the like.

The magnetic disk drive 11 has an OS (Operat
ing System), a modeling program PR for generating the three-dimensional model ML, other programs, a standard model (standard model data) DS, three-dimensional data (three-dimensional measurement data) DT, and reliability of the three-dimensional data DT. Reliability data DR, two-dimensional image (two-dimensional measurement data) FT,
The generated three-dimensional model ML, other data, and the like are stored. These programs and data are loaded into the RAM of the processing device 10 as appropriate.

The modeling program PR includes a measurement process, a rough alignment, a data reduction process, a deformation process,
A program for partial area selection processing and other processing is included.

The medium drive device 12 is a CD-ROM
(CD), a floppy disk FD, or a recording medium such as a magneto-optical disk is accessed to read and write data or programs. An appropriate drive device is used depending on the type of the recording medium. The above-described modeling program PR can be installed from these recording media. Standard model DS, three-dimensional data DT, reliability data DR, and two-dimensional image F
T and the like can also be input via a recording medium.

On the display surface HG of the display device 13,
The various data described above, the three-dimensional model ML generated by the modeling program PR, and other data (images) are displayed.

The keyboard 14 and the mouse 15 are used for inputting data or giving commands to the processing device 10. The three-dimensional measuring device 16 is for obtaining three-dimensional data DT of an object by, for example, a light section method. It is also possible to directly obtain the three-dimensional data DT by the three-dimensional measuring device 16,
It is also possible to indirectly obtain three-dimensional data DT by performing an operation in the processing device 10 or the like on the basis of the data output from 6.

At the same time as the three-dimensional data DT, if necessary, a two-dimensional image FT on the same line of sight of the same object can be obtained. Such a three-dimensional measuring device 16
For example, a known device described in Japanese Patent Application Laid-Open No. 10-206132 can be used.

As another known method for obtaining the three-dimensional data DT of an object, there is a method of using a plurality of cameras arranged with parallax to the object. From multiple images with parallax obtained from those cameras,
The three-dimensional data DT can be obtained by calculation using a stereoscopic method.

In this method, for example, by using three cameras, data for determining the reliability of each point of the three-dimensional data DT can be obtained at the same time. That is, an object is photographed by multi-viewing with three cameras, and three images are obtained. For these three images,
Search for each other's corresponding points. Based on the corresponding points of the two images, three-dimensional data DT is obtained by a known calculation. The other one image is used to obtain the reliability data DR.

For example, as shown in FIG. 10, an object Q is photographed using three cameras A, B, and C, and three images F are obtained.
Acquire A, FB, FC. For each of the images FA, FB, FC, the respective image planes (un, vn) are shown (n = 1, 2, 3). A point QP on the object Q in the three-dimensional space M is projected on points PA, PB, and PC on each image plane.

If the correspondence between the points PA, PB, and PC is determined, the three-dimensional space M ′ can be reconstructed from the correspondence. In the three-dimensional space M ′, the point P
A point QP ′ corresponding to A and PB is obtained. Ideally, a point PC ′ obtained by backprojecting the reconstructed point QP ′ on the three-dimensional space M ′ onto the image plane (u ₃ , v ₃ ) and a point QP on the original three-dimensional space M are The point PC projected on the image plane (u ₃ , v ₃ ) should match.

However, it is difficult to accurately determine the projection transformation, and it is also difficult to accurately determine the correspondence. Therefore, a deviation between the point PC ′ and the point PC is set as an error, and is used as an index of reliability.

For example, an error between the point PC ′ and the point PC is indicated by the number of shifted pixels. If the point PC ′ and the point PC are on the same pixel, the error is “0”. If there is a one pixel shift,
The error is “1”. If there is a two pixel shift, the error is "2". This shifted number of pixels can be used as reliability data DR.

As another method for determining the reliability data DR, for example, a method disclosed in Japanese Patent Application Laid-Open No. 61-125686, or another known method can be used. The modeling device 1 can be configured using a personal computer or a workstation. The programs and data described above can be obtained by receiving them via the network NW.

Next, the overall processing flow of the modeling device 1 will be described with reference to flowcharts.
FIG. 2 is a flowchart showing the overall processing flow of the modeling device 1, FIG. 3 is a flowchart showing a deformation process, FIG. 4 is a diagram showing an example of a standard model DS1, and FIG. 5 acquires three-dimensional data DT from an object. FIG. 6 (A) showing the situation.
FIG. 7B is a diagram showing an outline of the alignment, FIG. 7 is a diagram showing an outline and a feature point extraction process, and FIG. 8 is a diagram schematically showing a plane S of the standard model DS and a point P of the three-dimensional data DT. FIG. 9 is a diagram for explaining a virtual spring for preventing abnormal deformation of the standard model DS, and FIG. 10 is a diagram for explaining an example of a method for acquiring three-dimensional data DT and reliability data DR of an object. FIG. [Preparation of Standard Model] In FIG. 2, first, a standard model DS for an object is prepared (# 11). In the present embodiment, since the object is a human head, it is most similar to the head of the object from among a plurality of standard model groups having various sizes and shapes, all around the head. Prepare a standard model DS1.

The standard model DS may be either a three-dimensional shape model defined by a polygon or a three-dimensional shape model defined by a free-form surface. 3 defined by polygon
In the case of a three-dimensional model, the surface shape is determined by the three-dimensional coordinates of the vertices of each polygon. In the case of a three-dimensional shape model defined by a free-form surface, the shape of the surface is determined by the function defining the surface and the coordinates of each control point.

When the model is a three-dimensional shape model defined by polygons, the vertices of each polygon are described as "constituent points". Further, at the time of fitting the standard model DS, a point used for deforming the standard model is referred to as a “control point”. The positional relationship between the control points and the constituent points of the polygon is arbitrary, and the control points may be set on the surface of the polygon or may be set away from the surface of the polygon. One control point is composed of multiple points (about 3 to 100)
And the associated constituent point moves in accordance with the movement of the control point. When fitting the standard model DS, the entire standard model DS is deformed by moving these control points. Even when the three-dimensional shape model is defined by a free-form surface, the arrangement of control points used for fitting is arbitrary.

The control points are arranged at a high density in a portion having a fine shape such as the outer corner of the eye or the end of the lip and a portion having a sharp change in shape such as the nose or lip. The other parts are uniformly arranged.

In the standard model DS, a characteristic contour RK and a characteristic point TT viewed from a certain direction are set. Contour R
As K, for example, a line of an eyelid, a line of a nose, a line of a lip, a line of a chin, or the like is set in an eye, a nose, a mouth, a chin, or the like. As the characteristic point TT, for example, there is no characteristic in itself, such as a part having an actual characteristic such as an end of an eye or a mouth, a top of a nose, or a lower end of a chin, or a middle part thereof. The part that can be easily specified is selected.

In the standard model DS1 shown in FIG. 4, the chin line, the lip line, and the eyelid line have the contours RK1 to RK3.
Is set as As can be seen in FIG.
Reference numeral 1 denotes an edge portion of the standard model DS1 when viewed from a certain direction. In the standard model DS1 shown in FIG. 4, only a part of the set feature points TT is actually shown in the figure. [Acquisition of Three-Dimensional Data] Next, three-dimensional measurement of the object is performed to acquire three-dimensional data DT (# 12). At this time, a two-dimensional image FT of the object is also acquired at the same time.
Further, as described above, the reliability data DR indicating the reliability of each point of the three-dimensional data DT or information for obtaining the reliability data DR is acquired as necessary.

For example, as shown in FIG. 5, the head of a person as an object is measured (photographed) using a three-dimensional measuring device 16.
I do. Thus, three-dimensional data DT and two-dimensional image FT are obtained.

The three-dimensional data DT and / or the two-dimensional image FT obtained by measuring the object are referred to as “measurement data”.
May be described. Either preparation of the standard model DS and acquisition of the three-dimensional data DT may be performed first or may be performed in parallel. [Schematic alignment] Standard model DS and three-dimensional data D
A rough alignment with T is performed (# 13). In this process, the orientation, size, and position of the standard model DS are changed so that the standard model DS and the three-dimensional data DT substantially match. At this time, the standard model DS is represented by X, Y, Z
The size in each direction can be well matched to the three-dimensional data DT by individually multiplying the magnification to an arbitrary magnification in each direction.

For example, as shown in FIG. 6A, by rotating the standard model DS with respect to the three-dimensional data DT and magnifying the three-dimensional data DT in each direction, as shown in FIG.
Standard model DSa of almost the same size as three-dimensional data DT
Can be obtained. For simplicity,
In the drawing, those not aligned are shown.

As described below, there are three methods of rough positioning, namely, (1) overall rough positioning, (2) local rough positioning, and (3) iterative rough positioning. .

Of these methods, (1) and (3)
Can be performed automatically. In the method (2), it is difficult to automatically extract feature points therein,
Some need to be done manually. In the fitting after the approximate alignment, local deformation of the standard model DS is basically performed. Therefore, when importance is attached to matching the shapes, the methods (1) and (3) are preferable. The method (2) is preferable when it is desired to match the position rather than the shape as in the animation. Further, if the user is accustomed to performing feature point extraction, it is effective to use the method (2) to reduce the processing time. [Overall rough alignment] In overall rough alignment,
The position, direction, and size of the standard model DS are changed so as to minimize the distance between the three-dimensional data DT and the standard model DS.

That is, when the energy function e (si, αi, ti) shown in the following equation (1) is minimized, si, αi,
lead ti. Note that f (si, αi, ti) is an energy function defined in relation to the distance between the three-dimensional data DT and the standard model DS. g (si) is a stabilizing energy function to avoid excessive deformation.

Further, when the three-dimensional data DT is obtained by the three-dimensional measuring device 16, the two-dimensional image F
The initial values of the position, the direction, and the size may be given using T and pattern matching on the two-dimensional image FT.

[0042]

(Equation 1)

K: the number of points constituting the three-dimensional data dk: the distance between the points constituting the three-dimensional data and the surface of the standard model Wsc: weighting parameter for stabilizing magnification S0: initial scale Si: deviation in each direction Double amount (where S3 is the depth direction) αi: Rotation of each direction of the standard model ti: Movement amount of each direction of the standard model Here, the constituent points on the standard model DS are expressed by the following equation (2). Therefore, it moves, and accordingly, the three-dimensional data DT
The distance dk between the point of construction and the surface of the standard model DS changes.

[0044]

(Equation 2)

When the measurement (photographing) by the three-dimensional measuring device 16 is limited to one direction, the magnification in the depth direction (Z direction) may not be accurately obtained. In this case, it is assumed that there is no significant difference between the shape of the three-dimensional data DT and the shape of the standard model DS, and as shown in the following equation (3), the magnification in the X and Y directions (S1, S2) A method of correcting the magnification in the Z direction (S3) is also conceivable.

[0046]

(Equation 3)

Where γ is the weight parameter for the line of sight x3 deformation [local rough positioning] When the above-mentioned global rough positioning is performed automatically, if it does not fit well, a manual However, the local rough positioning described here is a method for performing the manual positioning as easily as possible. In addition, the part which does not work automatically is reset once, and then manually started from the beginning.

In the local rough registration, a characteristic line or point on the three-dimensional data DT is associated with a characteristic line or point on the standard model DS, and a standard is set so that their distance is minimized. Change the position, orientation, and size of model DS. When a line is associated with a line, a point on one line and the shortest point of a perpendicular drawn from the point onto the other line are set as feature points, and a plurality of points on the line are acquired. It shall be.

That is, with respect to the distance between the feature point on the three-dimensional data DT and the corresponding feature point on the standard model DS, the energy function E (si, α
i, ti) is minimized so that t of the standard model DS
derive i, αi, si.

[0050]

(Equation 4)

Where k: number of corresponding feature points Mk: feature points on the standard model after alignment x: feature points on the standard model before alignment Ck: feature points on three-dimensional data Si: standard model Αi: Rotation of the standard model in each direction ti: Movement of the standard model in each direction Also, when the measurement (photographing) by the three-dimensional measuring device 16 is limited to one direction, the depth The scale in the direction (Z direction) may not be obtained accurately. In such a case, a method of correcting the scale in the Z direction by using the following equation (5) is conceivable, as in the case of the above-described overall rough alignment.

[0052]

(Equation 5)

[Extraction of Contour / Feature Point] Three-dimensional data DT
Alternatively, contours and feature points are extracted on the two-dimensional image FT (# 14). When the contour RK and the feature point TT for the standard model DS have been extracted in advance, the contour and the feature point to be arranged at the same position as the contour RK and the feature point TT on the three-dimensional data DT or the corresponding two-dimensional It is arranged on the image (see FIG. 7).

When the contour RK and the feature point TT of the standard model DS have not been extracted in advance, the outline RK and the feature point TT are designated on the standard model DS together with the arrangement on the three-dimensional data DT or the two-dimensional image. [Data Reduction] Next, in order to reduce the calculation amount and the error, the data of the three-dimensional data DT is reduced, and only necessary and highly reliable data is extracted (# 15).
By performing data reduction, the original three-dimensional data D
The amount of calculation can be reduced without breaking the shape of T.

In reducing the data, for example, data outside the area of the object is excluded, and unnecessary data is excluded. For example, the area of the face is determined from the two-dimensional image FT, and only the three-dimensional data DT corresponding to the area is left. Alternatively, the area is determined using the difference in the distance between the object and the background.
In addition, there are various methods such as extracting a face region using information on the approximate alignment. In addition, three-dimensional data DT
If there is reliability data DR, only data having high reliability is left. If there is a lot of data in the neighborhood, the data is thinned out and the density is averaged.

When averaging the density by thinning out the data, only the three-dimensional data DT which satisfies the condition expressed by the following equation (6) is used.

[0057]

(Equation 6)

Here, Pk: constituent point P to r: already used constituent point R _det (x): function representing the density around constituent point x According to the above equation (6), the data Pk of interest If the distance between the data Pr and the data Pr that has been adopted so far is equal to or more than a certain value, the data Pk is adopted. [Modification] The standard model DS is modified (# 16).
Here, each constituent point of the three-dimensional data DT and the standard model D
The energy function e1 defined in relation to the distance to the surface of S is used, and in addition, the standard model D
Energy function e defined in relation to the distance between the feature point of S and the feature point specified for the three-dimensional data DT
3. The energy function e2 defined in relation to the distance between the contour of the standard model DS and the contour specified for the three-dimensional data DT, and the energy function defined to avoid excessive deformation Using e s, an energy function e summing them is evaluated, and a total energy function e
Is deformed so that is minimized.

Note that the total energy function e is e
It is most preferable to use four functions of 1, e2, e3, and es, but it is also possible to use only two of e1 to e3.

Next, each energy function will be described sequentially. [Distance between standard model and three-dimensional data] In FIG.
One of the points forming the three-dimensional data DT is indicated by a point Pk. On the surface S of the standard model DS, the point closest to the point Pk is indicated by Qk. The point Qk is an intersection when a perpendicular is drawn from the point Pk to the surface S. Here, the distance between the point Pk and the point Qk is evaluated.

That is, the difference energy e1 between each point of the three-dimensional data DT and the surface of the standard model DS is obtained by projecting the point Pk on the three-dimensional data DT after data reduction and the surface Pk on the surface S of the standard model DS. Using the squared distance to the point Qk,
It is calculated by the following equation (7).

[0062]

(Equation 7)

T1A: Control point group Pk: Constituent point of reduced three-dimensional data Qk: Projected point from constituent point to model surface K: Number of constituent point after reduced ^dk : From constituent point to model surface Projection direction, d ^k = (Q
k-Pk) / | Qk- Pk | ρk: construction point Pk reliability w (pk): reliability function, w (ρk) = 1 / (α + ρk) n W: Σw (ρk) L: Various energy Adjustment scale for handling in the same unit [distance between contour on standard model and contour on measurement data]
Here, the contour RK specified on the three-dimensional data DT,
Alternatively, the distance between the contour RK specified on the two-dimensional image FT and the contour RK on the standard model DS is evaluated.

The contour RK of the measurement data is three-dimensional data DT
When specified above, a perpendicular is dropped from a point on the contour RK of the three-dimensional data DT to a corresponding contour RK on the standard model DS, and the shortest of the perpendiculars is set as the distance. Note that a plurality of points are specified on the contour RK.

When the contour RK of the measurement data is specified on the two-dimensional image FT, the contour RK of the standard model DS is projected on the two-dimensional image FT using the camera parameters of the camera that has captured the two-dimensional image FT. I do. 2D image F
A perpendicular is dropped from a point on the contour RK of T to the corresponding contour RK of the standard model DS, and the shortest of the perpendiculars is defined as the distance. Note that a plurality of points are specified on the contour RK.

The contour RK of the measurement data is three-dimensional data DT
In the case specified above, the difference energy e2 for each contour RK of the standard model DS is calculated by the following equation (8) using the sum of squares of those distances.

[0067]

(Equation 8)

T2A: control point group pk: contour point on three-dimensional data qk: footstep point from contour point on three-dimensional data to corresponding model contour n: one model contour is associated Number of contour points of three-dimensional data d ^k : direction of projection from contour points of measurement data to corresponding model contour lines, d ^k = (qk-pk) / | qk-pk | l: treats various energies in the same unit When the contour RK of the measurement data is specified on the two-dimensional image FT, the difference energy e2 'for each contour RK of the standard model DS is calculated by the following equation (9).

[0069]

(Equation 9)

T2A: control point group pk: contour point on two-dimensional image qk: footstep point n from contour point on two-dimensional image to corresponding model contour projected on two-dimensional image n: one contour points of the measurement data corresponding to the model contour is attached d ^k: projection direction from the contour points on the two-dimensional image to a corresponding model ^{contour, d k = (qk-pk} ) / | qk-pk | l: Adjustment scale for handling various energies in the same unit The reason why the contour RK of the measurement data is specified on the two-dimensional image FT is that, for example, if the three-dimensional data DT is ambiguous,
This is because if the contour RK is specified on the dimensional data DT, the contour RK itself becomes inaccurate. Therefore, the contour RK is extracted using the two-dimensional image FT instead. [Distance between feature point on standard model and feature point on measurement data corresponding to feature point] By setting feature point TT on measurement data, feature point TT specified on three-dimensional data DT,
Alternatively, the distance between the feature point TT specified on the two-dimensional image FT and the feature point on the standard model DS is evaluated.

The difference energy e3 between the feature point TT on the three-dimensional data DT and the feature point TT on the standard model DS is calculated by the following equation (10) using the square distance of the corresponding feature point TT.

When the feature point TT is specified on the two-dimensional image FT, the standard model D
The feature point TT of S is projected on the two-dimensional image FT, and the difference energy on the two-dimensional image FT is calculated.

[0073]

(Equation 10)

T3A: control point group Fk: feature point of measurement data Gk: feature point on standard model corresponding to feature point of measurement data N: correspondence between feature point of measurement data and feature point on standard model Number L: Adjustment scale for handling various energies in the same unit [Stabilizing energy for avoiding excessive deformation] In addition to the energy of the difference described above, stabilizing energy for avoiding excessive deformation es is introduced.

That is, it is assumed that control points used for deformation are connected by a virtual spring (elastic bar) KB shown in FIG. Based on the constraint of the virtual spring KB, a stabilizing energy es for stabilizing the shape of the surface S of the standard model DS is defined.

The virtual spring does not necessarily need to be set between control points. It is sufficient that the relationship between the control point and the virtual spring is clear. FIG. 9 shows a part of the surface S of the standard model DS to be fitted. Surface S
Is formed by a control point group U = | ui, i = 1... N |. A virtual spring KB is arranged between adjacent control points. The virtual spring KB gives a constraint between the control points by a tensile force, and functions to prevent abnormal deformation of the surface S.

That is, when the interval between adjacent control points u increases, the pulling force by the virtual spring KB increases accordingly. For example, when the point Qk approaches the point Pk and the movement of the point Qk increases the interval between the control points u, the tensile force of the virtual spring KB increases. Point Q
If the interval between the control points u does not change even if k moves, that is, if the relative positional relationship between the control points u does not change, the pulling force by the virtual spring KB does not change. A value obtained by averaging the tensile force by the virtual spring KB over the entire surface S is defined as stabilizing energy es. Therefore, when a part of the surface S protrudes and deforms, the stabilization energy es increases. If the entire surface S moves on average, the stabilization energy es is zero.

The stabilizing energy es is obtained by the following equation (11) according to the state of deformation of the virtual spring KB.

[0079]

[Equation 11]

TsA: Control point group U-m, V-m: Initial value of end point (control point) of virtual spring Um, Vm: End point of virtual spring after deformation L0m: Length of virtual spring in initial state , L0m = | U〜m−V
M: Number of virtual springs c: Spring coefficient L: Adjustment scale for handling various energies in the same unit Therefore, if spring coefficient c is increased, virtual spring KB
Are hard and difficult to deform.

By introducing such a stabilizing energy function es, a constant constraint is imposed on the shape change of the surface S, and excessive deformation of the surface S can be prevented. [Overall Energy Function] As described above, the control point groups T1A, T2A, T3A, and TsA are used for each of the energy functions e1, e2, e3, and e4. Here, these control point groups T1A to TsA are the same, but can be different from each other as described later. These control point groups TA
Is used to transform the standard model DS to obtain a control point group TA that minimizes the total energy function e (TA) shown in the following equation (12).

[0082]

(Equation 12)

Here, e1 (T1A): the difference energy between the constituent points of the three-dimensional data and the model surface e2 ^s (T2A): the difference energy between the contour on the measurement data for each model contour e3 (T3A): the difference energy of the measurement data Energy difference between feature point and feature point on model eS (TSA): Stabilization energy wi, c for avoiding excessive deformation Weight parameter of each energy TA = T1A = T2A = T3A = TSA [Repeated deformation ] Actually, deformation is repeatedly performed (# 1
7). In other words, the control point is moved and the deformation is repeated. The total energy function after the nth deformation is en (TA)
Then, when the condition of the following equation (13) is satisfied,
It is determined that the total energy function en (TA) has converged.

[0084]

(Equation 13)

Now, the overall flow of the deformation process will be described with reference to FIG. First, a set of corresponding points is created between the measurement data and the standard model DS (# 21).
The surface S is deformed (# 22), and the total energy function en (TA) after the deformation is calculated (# 23). The process is repeated until the total energy function en (TA) converges (Yes in # 24).

As a method of judging the convergence of the total energy function en (TA), as described above, a method of converging when the total energy function en (TA) becomes smaller than a predetermined value, It is possible to use a known method such as a method in which convergence is achieved when the rate of change compared with the above becomes a predetermined value or less. [Use of Different Control Points] In the above equation (12), the fitting target (the constituent point of the three-dimensional data DT, the contour R
K, characteristic points TT) with different energy functions e1
Although the same control point group was used for .about.e4, the example shown here uses a different control point group for each fitting object, that is, for each energy function. That is, the control point group T1
A, T2A, T3A, and TsA are different from each other.

In this case, the total energy function e (T
Equation (12) shown above can be used as A). However, the control point groups T1A, T2A, T3 used there
A and TSA are different from each other and have the following relationship.

TA⊃T1A TA⊃T2A TA⊃T3A TA = TSA That is, as described above, since the feature point is a point,
For the energy between the feature points, local movement occurs. For example, three-dimensional data DT and standard model D
When trying to match the eye position with S, the feature point TT
There is a possibility that only the part where is set is strongly pulled and deformed into an irregular shape. In such a case, it is preferable to make the overall movement.

On the other hand, with respect to the constituent points of the three-dimensional data DT, it is desired that the sides of the eyes move finely. However, when only a small number of control points are used, the constituent points do not move finely. Therefore, a large number of control points are used for the constituent points of the three-dimensional data DT, and a small number of control points are used for the feature points. The contour RK has an intermediate amount.

For example, for a portion where the contour RK changes rapidly, the control points are made finer. Stabilizing energy is applied to all control points. Such selection of control points is performed when preparing the standard model DS.

Although the control point groups T1A, T2A, T3A, and TsA are different from each other, some control points included in each control point group are commonly used. [Change of weight according to reliability] (12)
In the formula, the total energy function e (TA) is evaluated assuming that the reliability of each information is equivalent. In the example shown here, the weight is changed according to the reliability of each information. As a result, it is possible to make a determination with emphasis on more reliable information from various information.

It is assumed that the reliability of each information is obtained at the time of three-dimensional measurement or at the time of automatic extraction of contours and feature points. In this case, a control point group TA that minimizes the total energy function e (TA) shown in the following equation (14) is obtained.

[0093]

[Equation 14]

Here, ρi: reliability of information on each energy function ei (TA) W (ρi): reliability function TA = T1A = T2A = T3A = TSA According to the above-described embodiment, three-dimensional data DT or Since the fitting is performed using not only the constituent points of the two-dimensional image FT but also the contour RK or the feature point TT, the local fitting such as the eyes or the mouth is maintained while maintaining the fineness of the entire fitting by the three-dimensional data DT or the like. They can be better matched without any significant abnormal deformation. Therefore, 3 which is closer to the object
A dimensional model can be generated.

In particular, in the field of CG involving partial deformation of the standard model DS, deformation that does not contradict with texture information can be performed. In the embodiment described above, the configuration, the circuit, the processing content, the processing order, the processing timing, the setting of the coefficient, and the like of the modeling device 1 can be appropriately changed according to the gist of the present invention.

[0096]

According to the present invention, a local portion such as an eye or a mouth can be more closely matched without causing local abnormal deformation.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a modeling device according to the present invention.

FIG. 2 is a flowchart showing the flow of the entire processing of the modeling device.

FIG. 3 is a flowchart showing a transformation process.

FIG. 4 is a diagram showing an example of a standard model.

FIG. 5 is a diagram illustrating a state in which three-dimensional data is obtained from an object.

FIG. 6 is a diagram showing a schematic state of alignment.

FIG. 7 is a diagram illustrating a state of a process of extracting a contour and a feature point.

FIG. 8 is a diagram schematically showing a plane of a standard model and points of three-dimensional data.

FIG. 9 is a diagram for explaining a virtual spring for preventing abnormal deformation of the standard model.

FIG. 10 is a diagram illustrating an example of a method for acquiring three-dimensional data and reliability data of an object.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Modeling apparatus 10 Processing apparatus DT 3D data (measurement data) FT 2D image (measurement data) ML 3D model DS Standard model RK Contour TT Feature point TA Control point group Pk Constitution point S surface PR Modeling program

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Hideo Fujii 2-3-1-13 Azuchicho, Chuo-ku, Osaka-shi, Osaka Inside Osaka International Building Minolta Co., Ltd. (72) Inventor Tomasi Kovalchik 5-19 Hiroo, Shibuya-ku, Tokyo -9 Hiroo ON Building Gen-Tech Co., Ltd. F-term (reference) 5B050 BA09 BA12 DA07 EA13 5B057 CA13 CA16 CB13 CB16 CD11 CD20

Claims (4)

[Claims] 1. A method for generating a shape model by deforming a standard model based on measurement data, comprising: a function relating to a distance between the standard model and the measurement data;
A function related to the distance between the feature point defined on the standard model and the feature point specified on the measurement data, and the function between the contour defined on the standard model and the contour specified on the measurement data. A method of generating a shape model, comprising: deforming a standard model based on a comprehensive evaluation of two or more functions among functions related to a distance between them. 2. Each function is an energy function.
The method for generating a shape model according to claim 1, wherein the standard model is deformed by performing a comprehensive evaluation so that a total value of two or more energy functions is minimized. 3. An energy function for avoiding excessive deformation of the standard model is defined, and the standard model is deformed by performing comprehensive evaluation so that the total value of these four energy functions is minimized. The method of generating a shape model according to claim 2. 4. The shape according to claim 1, wherein the weight of each function in the comprehensive evaluation is changed according to the reliability of the measurement data itself and the feature points and contours specified on the measurement data. How to generate the model.