JP2002015338A - Model deforming method and modeling device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a close fit of a hair portion or the like, as when creating a three-dimensional model of a human head. SOLUTION: Matching portions between a first standard model DSA1 and a second standard model DSB1 are matched with each other by means of corresponding points or lines, and when the first standard model is deformed in accordance with measurement data, the position information of the corresponding points or lines on the first standard model is applied to the corresponding points or lines on the second standard model to thereby deform the second standard model.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for deforming a model and a modeling device, and is used, for example, for generating a three-dimensional model in the field of computer graphics.

[0002]

2. Description of the Related Art In recent years, three-dimensional CG (three-dimensional computer graphics) technology is often used for movies, games, and the like. In three-dimensional CG, virtual 3
Since a three-dimensional model or a light is arranged and moved in a three-dimensional space, the degree of freedom of expression is high.

Conventionally, a non-contact type three-dimensional measuring device using a light-section method or the like has been put into practical use, and it is possible to relatively easily create three-dimensional data of an object by performing measurement using the device. Can be. However, if the three-dimensional data obtained by the measurement is used as it is for the three-dimensional CG, there are various problems, such as complicated processing for reducing the data amount by thinning the obtained data.

To cope with this problem, there has been proposed a method in which a standard model of an object is prepared, and the standard model is deformed in accordance with the measured three-dimensional data (Japanese Patent Laid-Open No. 5-81377).

In this conventional method, three-dimensional shape information of three-dimensional data obtained by measurement, that is, a three-dimensional point group is used as a fitting target, and the surface of a standard model is fitted to the three-dimensional point group. Let it.

[0006]

According to the above-described conventional method, in order to generate a three-dimensional model of a human head, three-dimensional data obtained by measuring a human head is previously calculated. A standard model of the prepared human head will be fitted.

However, a human head includes a face portion and a hair portion, and the former is easy to perform three-dimensional measurement, while the latter is difficult. Therefore, when the standard model is fitted, there remains a problem that the hair part does not fit well.

In order to solve this problem, for example, a method of dividing a face portion and a hair portion into a standard model, fitting each of them, and combining them to combine them is conceivable. In this case, it is necessary that the face part and the hair part be well connected without being shifted, and be united without discomfort.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems, and is useful for generating a three-dimensional model of a human head.
The purpose is to make the hair part fit well.

[0010]

According to the method of the present invention, parts corresponding to each other between a first standard model and a second standard model are associated with each other by corresponding points or corresponding lines, and the first standard model and the second standard model are associated with each other. When the standard model is deformed on the basis of the measurement data, the position information of the corresponding point or the corresponding line on the first standard model is applied to the corresponding point or the corresponding line on the second standard model to obtain the second model. Transform the standard model.

For example, the first standard model is a face model, the second standard model is a hair model, and the measurement data is data for a human head. A plurality of second standard models are associated with one first standard model.

The first standard model and the second standard model are set as three-dimensional shape data or two-dimensional images. According to the method for generating a three-dimensional model according to the present invention, a standard model is divided into a standard model Ai and a standard model Bj, and the standard model Ai and the standard model Bj are associated with each other by corresponding points or lines. In addition, the standard model Ai and / or Bi is fitted to the measurement data and deformed, and the other standard model Bi or Ai is used by using the position information of the corresponding point or the corresponding line of one standard model Ai or Bi after deformation. To transform.

In the present invention, the term “fitting” refers to a process of transforming a standard model or a series of processes including the process.

[0014]

FIG. 1 is a block diagram showing a modeling device 1 according to the present invention. In the present embodiment,
An example of generating a three-dimensional model of a human head by deforming (fitting) a standard model created in advance based on measurement data (three-dimensional data or a two-dimensional image) of the human head will be described. .

As shown in FIG. 1, a modeling device 1
It comprises a processing device 10, a magnetic disk device 11, a medium drive device 12, a display device 13, a keyboard 14, a mouse 15, a three-dimensional measuring device 16, and the like.

The processing device 10 includes a CPU, a RAM, an RO,
M, a video RAM, an input / output port, and various controllers. Various functions are realized on the processing device 10 by the CPU executing programs stored in the RAM, the ROM, and the like.

The magnetic disk drive 11 has an OS (Operat
ing System), a modeling program PR for generating the three-dimensional model ML, other programs, a standard model (standard model data) DS, three-dimensional data (three-dimensional measurement data) DT, and reliability of the three-dimensional data DT. Reliability data DR, two-dimensional image (two-dimensional measurement data) FT,
The generated three-dimensional model ML, other data, and the like are stored. These programs and data are loaded into the RAM of the processing device 10 as appropriate.

As the standard models DS, there are provided a first standard model group A (group A) for the face part and a second standard model group B (group B) for the hair part.
Group A includes a plurality of standard models Ai for the face part.
(DSA) is stored. In the group B, a plurality of standard models Bj (DSB) for the hair part are stored. These standard models DSA and standard models DSB are associated with each other by corresponding points or corresponding lines.

The modeling program PR includes measurement processing, rough alignment, data reduction processing, deformation processing,
A program for partial area selection processing and other processing is included.

The medium drive device 12 is a CD-ROM
(CD), a floppy disk FD, or a recording medium such as a magneto-optical disk is accessed to read and write data or programs. An appropriate drive device is used depending on the type of the recording medium. The above-described modeling program PR can be installed from these recording media. Standard model DS, three-dimensional data DT, reliability data DR, and two-dimensional image F
T and the like can also be input via a recording medium.

On the display surface HG of the display device 13,
The various data described above, the three-dimensional model ML generated by the modeling program PR, and other data (images) are displayed.

The keyboard 14 and the mouse 15 are used to input data or give commands to the processing device 10. The three-dimensional measuring device 16 is for obtaining three-dimensional data DT of an object by, for example, a light section method. It is also possible to directly obtain the three-dimensional data DT by the three-dimensional measuring device 16,
It is also possible to indirectly obtain three-dimensional data DT by performing an operation in the processing device 10 or the like on the basis of the data output from 6.

At the same time as the three-dimensional data DT, if necessary, a two-dimensional image FT on the same line of sight for the same object can be obtained. Such a three-dimensional measuring device 16
For example, a known device described in Japanese Patent Application Laid-Open No. 10-206132 can be used.

As another known method for acquiring the three-dimensional data DT of the object, there is a method of using a plurality of cameras arranged with parallax with respect to the object. From multiple images with parallax obtained from those cameras,
The three-dimensional data DT can be obtained by calculation using a stereoscopic method.

In this method, for example, by using three cameras, data for determining the reliability of each point of the three-dimensional data DT can be obtained at the same time. That is, an object is photographed by multi-viewing with three cameras, and three images are obtained. For these three images,
Search for each other's corresponding points. Based on the corresponding points of the two images, three-dimensional data DT is obtained by a known calculation. The other one image is used to obtain the reliability data DR.

For example, as shown in FIG. 10, an object Q is photographed by using three cameras A, B, and C, and three images F are obtained.
Acquire A, FB, FC. For each of the images FA, FB, FC, the respective image planes (un, vn) are shown (n = 1, 2, 3). A point QP on the object Q in the three-dimensional space M is projected on points PA, PB, and PC on each image plane.

Here, assuming that the correspondence between the points PA, PB, and PC is determined, the three-dimensional space M ′ can be reconstructed from the correspondence. In the three-dimensional space M ′, the point P
A point QP ′ corresponding to A and PB is obtained. Ideally, a point PC ′ obtained by backprojecting the reconstructed point QP ′ on the three-dimensional space M ′ onto the image plane (u ₃ , v ₃ ) and a point QP on the original three-dimensional space M are The point PC projected on the image plane (u ₃ , v ₃ ) should match.

However, it is difficult to accurately determine the projection transformation, and it is also difficult to accurately determine the correspondence. Therefore, a deviation between the point PC ′ and the point PC is set as an error, and is used as an index of reliability.

For example, an error between the point PC ′ and the point PC is indicated by the number of shifted pixels. If the point PC ′ and the point PC are on the same pixel, the error is “0”. If there is a one pixel shift,
The error is “1”. If there is a two pixel shift, the error is "2". This shifted number of pixels can be used as reliability data DR.

As another method for judging the reliability data DR, for example, a method disclosed in Japanese Patent Application Laid-Open No. 61-125686 or another known method can be used. The modeling device 1 can be configured using a personal computer or a workstation. The programs and data described above can be obtained by receiving them via the network NW.

Next, the overall processing flow of the modeling device 1 will be described with reference to flowcharts.
FIG. 2 is a flowchart showing the overall processing flow of the modeling device 1, FIG. 3 is a flowchart showing a deformation process, FIG. 4 is a diagram showing an example of a standard model DSA1, and FIG. 5 acquires three-dimensional data DT from an object. Diagram showing the situation, FIG. 6
FIGS. 7A and 7B are schematic views showing a state of alignment, FIG.
FIG. 8 is a diagram showing a state of extraction processing of contours and feature points, FIG. 8 is a diagram schematically showing a surface S of the standard model DS and a point P of the three-dimensional data DT, and FIG. 9 is a diagram for preventing abnormal deformation of the standard model DS. FIG. 10 is a view for explaining an example of a method for acquiring three-dimensional data DT and reliability data DR of an object, and FIG. 11 is a standard face model DSA and a standard hair model. FIG. 12 is a diagram showing a state where DSBs are associated with corresponding points, FIG. 12 is a diagram showing a standard model DSA of a deformed face and a standard model DSB of hair, and FIG.
FIG. 14 is a diagram showing a state in which SA and hair standard model DSB are associated with corresponding lines, and FIG. 14 is a face standard model DS after the deformation.
FIG. 2 is a diagram showing A and a hair standard model DSB. [Preparation of Standard Model] In FIG. 2, first, a standard model DS for an object is prepared (# 11). In the present embodiment, since the object is a human head, it is most similar to the face of the object from among a plurality of standard model groups of the face all around the head having various sizes and shapes. The prepared standard model DSA1 is prepared.

The standard model DS may be either a three-dimensional shape model defined by polygons or a three-dimensional shape model defined by free-form surfaces. 3 defined by polygon
In the case of a three-dimensional model, the surface shape is determined by the three-dimensional coordinates of the vertices of each polygon. In the case of a three-dimensional shape model defined by a free-form surface, the shape of the surface is determined by the function defining the surface and the coordinates of each control point.

When the model is a three-dimensional shape model defined by polygons, the vertices of each polygon are described as "constituent points". Further, at the time of fitting the standard model DS, a point used for deforming the standard model is referred to as a “control point”. The positional relationship between the control points and the constituent points of the polygon is arbitrary, and the control points may be set on the surface of the polygon or may be set away from the surface of the polygon. One control point is composed of multiple points (about 3 to 100)
And the associated constituent point moves in accordance with the movement of the control point. When fitting the standard model DS, the entire standard model DS is deformed by moving these control points. Even when the three-dimensional shape model is defined by a free-form surface, the arrangement of control points used for fitting is arbitrary.

The control points are arranged at a high density in a portion having a fine shape such as the outer corner of the eye and the end of the lip, and in a portion having a sharp change in shape such as the nose and lips. The other parts are uniformly arranged.

In the standard model DS, a characteristic contour RK and a characteristic point TT viewed from a certain direction are set. Contour R
As K, for example, a line of an eyelid, a line of a nose, a line of a lip, a line of a chin, or the like is set in an eye, a nose, a mouth, a chin, or the like. As the characteristic point TT, for example, there is no characteristic in itself, such as a part having an actual characteristic such as an end of an eye or a mouth, a top of a nose, or a lower end of a chin, or a middle part thereof. The part that can be easily specified is selected.

In the standard model DSA1 shown in FIG. 4, the chin line, the lip line, and the eyelid line have the contours RK1 to RK1.
3 is set. As can be seen in FIG.
K1 is a portion that becomes an edge when the standard model DSA1 is viewed from a certain direction. In the standard model DSA1 shown in FIG. 4, only a part of the set feature points TT is actually shown in the figure.

Along with the preparation of the standard face model DSA, the preparation of a standard hair model DSB is also performed. The preparation of the standard model DS is performed by selecting an appropriate model for the target object from a plurality of standard models stored in the magnetic disk device 11. An example of the hair standard model DSB1 is shown in FIGS. 11 and 13 together with the state of association with the standard face model DSA.

In the present embodiment, after fitting the standard model DSA of the face to the measurement data, the standard model DSA of the hair is added to the standard model DSA after the fitting and united.

As described above, there is a corresponding point C between the face standard model DSA and the hair standard model DSB.
T or the corresponding line LT is set, and by making the corresponding points CT or the corresponding lines LT coincide with each other,
Combines well to form a natural shape.

When the standard model DSA is deformed by fitting, the deformation amount of the corresponding point CT or the corresponding line LT is derived, and is applied to the corresponding point CT or the corresponding line LT of the standard model DSB. At this time, in order to eliminate the deviation between the shape of the standard model DSB and the shape of the deformed standard model DSA, each point on the standard model DSB is also deformed according to the amount of deformation of each feature point.

In the case of deformation using the corresponding line LT as shown in FIG. 13, for example, the standard model DSA of the face
1 from a plurality of points on the corresponding line LT
The shortest perpendicular is dropped on the corresponding line LT on SB1, and the standard hair model DSB1 is deformed so that those points coincide with each other.

Thus, the standard hair model DSB
Is deformed corresponding to the deformation of the standard model DSA of the face. FIG. 12 shows examples of the standard models DSA and DSB after the deformation.

As a result, there is no deviation between the standard model DSA and the standard model DSB, they are connected without any gap, and the fitting is performed without a sense of incongruity. When the standard model DSB is deformed, the corresponding point C of the standard model DSB is changed.
Not only the deformation amount of T or the corresponding line LT, but also the standard model D
It is also possible to use measurement data (three-dimensional data DT or two-dimensional image FT) obtained by fitting the SA together.

When the selected standard model DSB is changed to another standard model DSB, the same modification is applied to the changed standard model DSB,
It can equally well be incorporated into the standard model DSA after fitting. [Acquisition of Three-Dimensional Data] Next, three-dimensional measurement of the object is performed to acquire three-dimensional data DT (# 12). At this time, a two-dimensional image FT of the object is also acquired at the same time.
Further, as described above, the reliability data DR indicating the reliability of each point of the three-dimensional data DT or information for obtaining the reliability data DR is acquired as necessary.

For example, as shown in FIG. 5, the head of a person as an object is measured (photographed) using a three-dimensional measuring device 16.
I do. Thus, three-dimensional data DT and two-dimensional image FT are obtained.

The three-dimensional data DT and / or the two-dimensional image FT obtained by measuring the object are referred to as “measurement data”.
May be described. Either preparation of the standard model DS and acquisition of the three-dimensional data DT may be performed first or may be performed in parallel. [Schematic alignment] Standard model DS and three-dimensional data D
A rough alignment with T is performed (# 13). In this process, the orientation, size, and position of the standard model DS are changed so that the standard model DS and the three-dimensional data DT substantially match. At this time, the standard model DS is represented by X, Y, Z
The size in each direction can be well matched to the three-dimensional data DT by individually multiplying the magnification to an arbitrary magnification in each direction.

For example, as shown in FIG. 6A, by rotating the standard model DS with respect to the three-dimensional data DT and magnifying the three-dimensional data DT in each direction, as shown in FIG.
Standard model DSa of almost the same size as three-dimensional data DT
Can be obtained. For simplicity,
In the drawing, those not aligned are shown.

As described below, there are three methods of rough positioning, namely, (1) overall rough positioning, (2) local rough positioning, and (3) iterative rough positioning. .

Of these methods, (1) and (3)
Can be performed automatically. In the method (2), it is difficult to automatically extract feature points therein,
Some need to be done manually. In the fitting after the approximate alignment, local deformation of the standard model DS is basically performed. Therefore, when importance is attached to matching the shapes, the methods (1) and (3) are preferable. The method (2) is preferable when it is desired to match the position rather than the shape as in the animation. Further, if the user is accustomed to performing feature point extraction, it is effective to use the method (2) to reduce the processing time. [Overall rough alignment] In overall rough alignment,
The position, direction, and size of the standard model DS are changed so as to minimize the distance between the three-dimensional data DT and the standard model DS.

That is, when the energy function e (si, αi, ti) shown in the following equation (1) is minimized, si, αi,
lead ti. Note that f (si, αi, ti) is an energy function defined in relation to the distance between the three-dimensional data DT and the standard model DS. g (si) is a stabilizing energy function to avoid excessive deformation.

Further, when the three-dimensional data DT is obtained by the three-dimensional measuring device 16, the two-dimensional image F simultaneously obtained
The initial values of the position, the direction, and the size may be given using T and pattern matching on the two-dimensional image FT.

[0052]

(Equation 1)

K: the number of points constituting the three-dimensional data dk: the distance between the points constituting the three-dimensional data and the surface of the standard model Wsc: weighting parameter for stabilizing magnification magnification S0: initial scale Si: deviation in each direction Αi: rotation in each direction of the standard model ti: movement amount in each direction of the standard model Here, the constituent points on the standard model DS are expressed by the following equation (2). Therefore, it moves, and accordingly, the three-dimensional data DT
The distance dk between the point of construction and the surface of the standard model DS changes.

[0054]

(Equation 2)

When the measurement (photographing) by the three-dimensional measuring device 16 is limited to one direction, the magnification in the depth direction (Z direction) may not be accurately obtained. In this case, it is assumed that there is no significant difference between the shape of the three-dimensional data DT and the shape of the standard model DS, and as shown in the following equation (3), the magnification in the X and Y directions (S1, S2) A method of correcting the magnification in the Z direction (S3) is also conceivable.

[0056]

(Equation 3)

Where γ is the weight parameter for the line-of-sight direction x3 deformation [local rough positioning] When the above-mentioned global rough positioning is automatically performed, if it does not fit well, However, the local rough positioning described here is a method for performing the manual positioning as easily as possible. In addition, the part which does not work automatically is reset once, and then manually started from the beginning.

In the local rough alignment, a characteristic line or point on the three-dimensional data DT is associated with a characteristic line or point on the standard model DS, and a standard is set so that the distance between them is minimized. Change the position, orientation, and size of model DS. When a line is associated with a line, a point on one line and the shortest point of a perpendicular drawn from the point onto the other line are set as feature points, and a plurality of points on the line are acquired. It shall be.

That is, with respect to the distance between a feature point on the three-dimensional data DT and a corresponding feature point on the standard model DS, an energy function E (si, α) shown in the following equation (4) is obtained.
i, ti) is minimized so that t of the standard model DS
derive i, αi, si.

[0060]

(Equation 4)

Where: k: number of corresponding feature points Mk: feature points on the standard model after alignment x: feature points on the standard model before alignment Ck: feature points on three-dimensional data Si: standard model Αi: Rotation of the standard model in each direction ti: Movement of the standard model in each direction Also, when the measurement (photographing) by the three-dimensional measuring device 16 is limited to one direction, the depth The scale in the direction (Z direction) may not be obtained accurately. In such a case, a method of correcting the scale in the Z direction by using the following equation (5) is conceivable, as in the case of the above-described overall rough alignment.

[0062]

(Equation 5)

[Extraction of Contour / Feature Point] Three-dimensional data DT
Alternatively, contours and feature points are extracted on the two-dimensional image FT (# 14). When the contour RK and the feature point TT for the standard model DS have been extracted in advance, the contour and the feature point to be arranged at the same position as the contour RK and the feature point TT on the three-dimensional data DT or the corresponding two-dimensional It is arranged on the image (see FIG. 7).

When the contour RK and the feature point TT of the standard model DS have not been extracted in advance, the outline RK and the feature point TT are specified on the standard model DS together with the arrangement on the three-dimensional data DT or the two-dimensional image. [Data Reduction] Next, in order to reduce the calculation amount and the error, the data of the three-dimensional data DT is reduced, and only necessary and highly reliable data is extracted (# 15).
By performing data reduction, the original three-dimensional data D
The amount of calculation can be reduced without breaking the shape of T.

In reducing data, for example, data outside the area of the object is excluded, and unnecessary data is excluded. For example, the area of the face is determined from the two-dimensional image FT, and only the three-dimensional data DT corresponding to the area is left. Alternatively, the area is determined using the difference in the distance between the object and the background.
In addition, there are various methods such as extracting a face region using information on the approximate alignment. In addition, three-dimensional data DT
If there is reliability data DR, only data having high reliability is left. If there is a lot of data in the neighborhood, the data is thinned out and the density is averaged.

When the density is averaged by thinning out the data, only the three-dimensional data DT which satisfies the condition expressed by the following equation (6) is adopted.

[0067]

(Equation 6)

Here, Pk: constituent point P to r: already used constituent point R _det (x): function representing the density around constituent point x According to equation (6) above, the data Pk of interest If the distance between the data Pr and the data Pr that has been adopted so far is equal to or more than a certain value, the data Pk is adopted. [Modification] The standard model DS is modified (# 16).
Here, each constituent point of the three-dimensional data DT and the standard model D
The energy function e1 defined in relation to the distance to the surface of S is used, and in addition, the standard model D
Energy function e defined in relation to the distance between the feature point of S and the feature point specified for the three-dimensional data DT
3. The energy function e2 defined in relation to the distance between the contour of the standard model DS and the contour specified for the three-dimensional data DT, and the energy function defined to avoid excessive deformation Using e s, an energy function e summing them is evaluated, and a total energy function e
Is deformed so that is minimized.

Note that the total energy function e is e
It is most preferable to use four functions of 1, e2, e3, and es, but it is also possible to use only two of e1 to e3.

Next, each energy function will be described sequentially. [Distance between standard model and three-dimensional data] In FIG.
One of the points forming the three-dimensional data DT is indicated by a point Pk. On the surface S of the standard model DS, the point closest to the point Pk is indicated by Qk. The point Qk is an intersection when a perpendicular is drawn from the point Pk to the surface S. Here, the distance between the point Pk and the point Qk is evaluated.

That is, the difference energy e 1 between each point of the three-dimensional data DT and the surface of the standard model DS is calculated by projecting the point Pk on the three-dimensional data DT after data reduction and the surface Pk on the surface S of the standard model DS. Using the squared distance to the point Qk,
It is calculated by the following equation (7).

[0072]

(Equation 7)

[0073] However, T1A: control point group Pk: construction point of the three-dimensional data after reduction Qk: projection point from the configuration point to the model surface K: number d ^k configuration points after reduction: from the configuration point to the model surface projection direction ^{of, d k = (Qk-Pk} ) / | Qk-Pk | ρk: configuration point Pk of the reliability w (ρk): reliability function, w (ρk) = 1 / (α + ρk) n W: Σw ( ρk) L: Adjustment scale for handling various energies in the same unit [distance between the contour on the standard model and the contour on the measurement data]
Here, the contour RK specified on the three-dimensional data DT,
Alternatively, the distance between the contour RK specified on the two-dimensional image FT and the contour RK on the standard model DS is evaluated.

The contour RK of the measurement data is three-dimensional data DT
When specified above, a perpendicular is dropped from a point on the contour RK of the three-dimensional data DT to a corresponding contour RK on the standard model DS, and the shortest of the perpendiculars is set as the distance. Note that a plurality of points are specified on the contour RK.

When the contour RK of the measurement data is specified on the two-dimensional image FT, the contour RK of the standard model DS is projected on the two-dimensional image FT using the camera parameters of the camera that captured the two-dimensional image FT. I do. 2D image F
A perpendicular is dropped from a point on the contour RK of T to the corresponding contour RK of the standard model DS, and the shortest of the perpendiculars is defined as the distance. Note that a plurality of points are specified on the contour RK.

The contour RK of the measurement data is three-dimensional data DT
In the case specified above, the difference energy e2 for each contour RK of the standard model DS is calculated by the following equation (8) using the sum of squares of those distances.

[0077]

(Equation 8)

T2A: control point group pk: contour point on three-dimensional data qk: footstep from contour point on three-dimensional data to corresponding model contour n: one model contour is assigned The number of contour points of the three-dimensional data d ^k : the direction of projection from the contour points of the measured data to the corresponding model contour, d ^k = (qk-pk) / | qk-pk | l: handles various energies in the same unit When the contour RK of the measurement data is specified on the two-dimensional image FT, the difference energy e2 'for each contour RK of the standard model DS is calculated by the following equation (9).

[0079]

(Equation 9)

T2A: control point group pk: contour point on two-dimensional image qk: footstep point n from contour point on two-dimensional image to corresponding model contour projected on two-dimensional image n: one The number of contour points of the measurement data associated with the model contour d ^k : the direction of projection from the contour points on the two-dimensional image to the corresponding model contour, d ^k = (qk-pk) / | qk-pk | l: Adjustment scale for handling various energies in the same unit The reason why the contour RK of the measurement data is specified on the two-dimensional image FT is that, for example, if the three-dimensional data DT is ambiguous,
This is because if the contour RK is specified on the dimensional data DT, the contour RK itself becomes inaccurate. Therefore, the contour RK is extracted using the two-dimensional image FT instead. [Distance between feature point on standard model and feature point on measurement data corresponding to feature point] By setting feature point TT on measurement data, feature point TT specified on three-dimensional data DT,
Alternatively, the distance between the feature point TT specified on the two-dimensional image FT and the feature point on the standard model DS is evaluated.

The difference energy e3 between the feature point TT on the three-dimensional data DT and the feature point TT on the standard model DS is calculated by the following equation (10) using the square distance of the corresponding feature point TT.

When the feature point TT is specified on the two-dimensional image FT, the standard model D
The feature point TT of S is projected on the two-dimensional image FT, and the difference energy on the two-dimensional image FT is calculated.

[0083]

(Equation 10)

T3A: control point group Fk: feature point of measurement data Gk: feature point on standard model corresponding to feature point of measurement data N: correspondence between feature point of measurement data and feature point on standard model Number L: Adjustment scale for handling various energies in the same unit [Stabilizing energy for avoiding excessive deformation] In addition to the energy of the difference described above, stabilizing energy for avoiding excessive deformation es is introduced.

That is, it is assumed that control points used for deformation are connected by a virtual spring (elastic bar) KB shown in FIG. Based on the constraint of the virtual spring KB, a stabilizing energy es for stabilizing the shape of the surface S of the standard model DS is defined.

Note that the virtual spring does not necessarily need to be set between control points. It is sufficient that the relationship between the control point and the virtual spring is clear. FIG. 9 shows a part of the surface S of the standard model DS to be fitted. Surface S
Is formed by a control point group U = | ui, i = 1... N |. A virtual spring KB is arranged between adjacent control points. The virtual spring KB gives a constraint between the control points by a tensile force, and functions to prevent abnormal deformation of the surface S.

That is, when the interval between adjacent control points u increases, the pulling force of the virtual spring KB increases accordingly. For example, when the point Qk approaches the point Pk and the movement of the point Qk increases the interval between the control points u, the tensile force of the virtual spring KB increases. Point Q
If the interval between the control points u does not change even if k moves, that is, if the relative positional relationship between the control points u does not change, the pulling force by the virtual spring KB does not change. A value obtained by averaging the tensile force by the virtual spring KB over the entire surface S is defined as stabilizing energy es. Therefore, when a part of the surface S protrudes and deforms, the stabilization energy es increases. If the entire surface S moves on average, the stabilization energy es is zero.

The stabilizing energy es is obtained by the following equation (11) according to the state of deformation of the virtual spring KB.

[0089]

[Equation 11]

TsA: Control point group U-m, V-m: Initial value of end point (control point) of virtual spring Um, Vm: End point of virtual spring after deformation L0m: Length of virtual spring in initial state , L0m = | U-m-V-m | M: Number of virtual springs c: Spring coefficient L: Adjustment scale for handling various energies in the same unit Therefore, if spring coefficient c is increased, virtual spring KB
Are hard and difficult to deform.

By introducing such a stabilizing energy function es, a certain constraint is imposed on the shape change of the surface S, and excessive deformation of the surface S can be prevented. [Overall Energy Function] As described above, the control point groups T1A, T2A, T3A, and TsA are used for each of the energy functions e1, e2, e3, and e4. Here, these control point groups T1A to TsA are the same, but can be different from each other as described later. These control point groups TA
Is used to transform the standard model DS to obtain a control point group TA that minimizes the total energy function e (TA) shown in the following equation (12).

[0092]

(Equation 12)

Here, e1 (T1A): the difference energy between the constituent points of the three-dimensional data and the model surface e2 ^s (T2A): the difference energy between the contour on the measurement data for each model contour e3 (T3A): the difference energy of the measurement data Energy difference between feature point and feature point on model eS (TSA): Stabilization energy wi, c for avoiding excessive deformation Weight parameter of each energy TA = T1A = T2A = T3A = TSA [Repeated deformation ] Actually, deformation is repeatedly performed (# 1
7). In other words, the control point is moved and the deformation is repeated. The total energy function after the nth deformation is en (TA)
Then, when the condition of the following equation (13) is satisfied,
It is determined that the total energy function en (TA) has converged.

[0094]

(Equation 13)

Now, the overall flow of the deformation process will be described with reference to FIG. First, a set of corresponding points is created between the measurement data and the standard model DS (Pk in FIG. 8).
And Qk) (# 21).

The surface S is deformed (# 22), and the total energy function en (TA) after the deformation is calculated (# 23). The process is repeated until the total energy function en (TA) converges (Yes in # 24).

As a method of judging the convergence of the total energy function en (TA), as described above, a method of converging when the total energy function en (TA) becomes smaller than a predetermined value, It is possible to use a known method such as a method in which convergence is achieved when the rate of change compared with the above becomes a predetermined value or less.

If measurement data on a hair part is obtained as measurement data, the standard face model D
An energy function e4 related to the distance between the feature point on the SA and the feature point on the hair standard model DSB is newly defined and added to the above-described total energy function e (TA) and simultaneously evaluated. do it. [Use of Different Control Points] In the above equation (12), the fitting target (the constituent point of the three-dimensional data DT, the contour R
K, characteristic points TT) with different energy functions e1
Although the same control point group was used for .about.e4, the example shown here uses a different control point group for each fitting object. That is, the control point groups T1A, T2A, T3A, and TsA are different from each other.

In this case, the total energy function e (T
Equation (12) shown above can be used as A). However, the control point groups T1A, T2A, T3 used there
A and TSA are different from each other and have the following relationship.

TA⊃T1A TA⊃T2A TA⊃T3A TA = TSA That is, as described above, since the feature point is a point,
For the energy between the feature points, local movement occurs. For example, three-dimensional data DT and standard model D
When trying to match the eye position with S, the feature point TT
There is a possibility that only the part where is set is strongly pulled and deformed into an irregular shape. In such a case, it is preferable to make the overall movement.

On the other hand, with respect to the constituent points of the three-dimensional data DT, it is desired that the sides of the eyes move finely. However, when only a small number of control points are used, the constituent points do not move finely. Therefore, a large number of control points are used for the constituent points of the three-dimensional data DT, and a small number of control points are used for the feature points. The contour RK has an intermediate amount.

For example, for a portion where the contour RK changes rapidly, the control points are made finer. Stabilizing energy is applied to all control points. Such selection of control points is performed when preparing the standard model DS.

Although the control point groups T1A, T2A, T3A, and TsA are different from each other, some control points included in each control point group are commonly used. [Change of weight according to reliability] (12)
In the formula, the total energy function e (TA) is evaluated assuming that the reliability of each information is equivalent. In the example shown here, the weight is changed according to the reliability of each information. As a result, it is possible to make a determination with emphasis on more reliable information from various information.

It is assumed that the reliability of each information is obtained at the time of three-dimensional measurement or at the time of automatic extraction of contours and feature points. In this case, a control point group TA that minimizes the total energy function e (TA) shown in the following equation (14) is obtained.

[0105]

[Equation 14]

Here, ρi: reliability of information about each energy function ei (TA) W (ρi): reliability function TA = T1A = T2A = T3A = TSA According to the above-described embodiment, three-dimensional data DT or Since the fitting is performed using not only the constituent points of the two-dimensional image FT but also the contour RK or the feature point TT, the local fitting such as the eyes or the mouth is maintained while maintaining the fineness of the fitting by the three-dimensional data DT or the like. They can be better matched without any significant abnormal deformation. Therefore, 3 which is closer to the object
A dimensional model can be generated.

The standard model DSA of the face and the standard model DSB of the hair are associated with each other by corresponding points CT, and the standard model DSB of the hair is changed according to the deformation of the standard model DSA of the face.
Is deformed, so that when they are combined, there is no deviation between them, and fitting is performed without discomfort.

In the above embodiment, a two-dimensional face model and a plurality of hairstyle models may be prepared, and corresponding points or lines may be defined between the models. In that case, a face image is input, and the face model is deformed to match the face image. The deformation amount of the corresponding point or corresponding line arranged on the face model at that time is applied to the corresponding point or corresponding line of the hairstyle model to deform the hairstyle model and simulate the hairstyle.

In CAD or the like, the correspondence between objects is defined by points or lines, and when a deformation operation is applied to one object, the corresponding point or line on one object is deformed by the deformation operation. The amount may be applied to a corresponding point or line on the other object to deform. As a result, the load on the design can be reduced.

In the above-described embodiment, the configuration, circuit, processing contents, processing order, processing timing, coefficient setting, and the like of the modeling device 1 can be appropriately changed in accordance with the gist of the present invention.

[0111]

According to the present invention, when a three-dimensional model of a human head is generated, it is possible to properly fit a hair portion and the like.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a modeling device according to the present invention.

FIG. 2 is a flowchart showing the flow of the entire processing of the modeling device.

FIG. 3 is a flowchart showing a transformation process.

FIG. 4 is a diagram showing an example of a standard model.

FIG. 5 is a diagram illustrating a state in which three-dimensional data is obtained from an object.

FIG. 6 is a diagram showing a schematic state of alignment.

FIG. 7 is a diagram illustrating a state of a process of extracting a contour and a feature point.

FIG. 8 is a diagram schematically showing a plane of a standard model and points of three-dimensional data.

FIG. 9 is a diagram for explaining a virtual spring for preventing abnormal deformation of the standard model.

FIG. 10 is a diagram illustrating an example of a method for acquiring three-dimensional data and reliability data of an object.

FIG. 11 is a diagram showing a state where a standard model of a face and a standard model of hair are associated with corresponding points.

FIG. 12 is a diagram showing a standard model of a face and a standard model of hair after deformation.

FIG. 13 is a diagram showing a state in which a standard model of a face and a standard model of hair are associated with each other by corresponding lines.

FIG. 14 is a diagram showing a standard model of a face and a standard model of hair after deformation.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Modeling apparatus (3D model generation apparatus) 10 Processing apparatus 11 Magnetic disk apparatus DT 3D data (measurement data) FT 2D image (measurement data) ML 3D model DS standard model DSA Standard model of face (first Standard model, standard model Ai) DSB Standard model of hair (second standard model, standard model Bi) CT corresponding point LT corresponding line RK contour TT feature point TA control point group PR modeling program

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Shinichi Hotta 2-3-1-13 Azuchicho, Chuo-ku, Osaka-shi, Osaka Inside the Osaka International Building Minolta Co., Ltd. (72) Inventor Tomashi Kovalchik 5-19 Hiroo, Shibuya-ku, Tokyo -9 Hiroo ON Building Gen-Tech Co., Ltd. F-term (reference) 5B050 BA07 BA12 EA13 EA27 EA28

Claims (6)

[Claims] A first standard model and a second standard model correspond to each other by corresponding points or corresponding lines, and the first standard model is deformed based on measurement data. At this time, the second standard model is deformed by applying the position information of the corresponding point or the corresponding line on the first standard model to the corresponding point or the corresponding line on the second standard model. How to transform the model. 2. The first standard model is a face model, and the second standard model is a second model.
The method according to claim 1, wherein the standard model is a hair model, and the measurement data is data for a human head. 3. The method according to claim 2, wherein a plurality of second standard models are associated with one first standard model. 4. The model deformation method according to claim 1, wherein said first standard model and said second standard model are set as three-dimensional shape data. 5. A method for generating a three-dimensional model by fitting a three-dimensional standard model to measurement data of an object, comprising: dividing a standard model into a standard model Ai and a standard model Bj.
The standard model Ai and the standard model Bj are associated with each other by corresponding points or corresponding lines, and the standard model Ai and / or Bi are fitted to the measurement data to be deformed. A method of generating a three-dimensional model, wherein the other standard model Bi or Ai is deformed using the position information of a corresponding point or a corresponding line after deformation of Ai or Bi. 6. A modeling device for deforming a model, comprising: a standard model Ai belonging to a first standard model group A;
Means for associating a standard model Bj belonging to a second standard model group B related to the standard model group A with corresponding points or lines, and when one of the standard models Ai or Bi is deformed, Means for deforming the other standard model Bi or Ai by using the position information of the corresponding point or the corresponding line after the deformation.