JP2001297333A - Device and method for processing image and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To change the hairstyle of hair to be expressed for each hair by means of CF corresponding to plural different head forms. SOLUTION: A hairstyle data reading part 21 of a hairstyle mapping processing part 4 reads hairstyle data, which are selected by using a controller, out of a hairstyle data base 7 and outputs them to a mapping position calculating part 22. The mapping position calculating part 22 calculates a position for mapping hair data, which comprise the hairstyle data, (corresponding to one hair) to a three-dimensional(3D) head model. A hair data interpolating processing part 23 interpolates the part, where the hair data are not mapped, of the head. A coordinate transforming part 24 transforms the coordinate of the mapped and interpolated hair data from the object coordinate system of hair data to the object coordinate system of the 3D head model.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and method, and a recording medium, and is used, for example, when synthesizing a hair image expressed by computer graphics with a three-dimensional shape image of a head excluding hair. And a suitable image processing apparatus and method, and a recording medium.

[0002]

2. Description of the Related Art Conventionally, as a method of expressing human hair by computer graphics (hereinafter abbreviated as CG), as shown in FIG. 1A, an entire hairstyle is approximated using a polygon model. A method of attaching the texture of the hair to the surface and a method of rendering using an anisotropic reflection model showing a different reflection intensity depending on a light source position or a viewpoint position are used.

[0003] These methods are described in "Yamana and Suenaga," Hair expression using anisotropic reflection model ", IEICE Technical Report, PRU87-3, pp.115-120,1989". In such a method, the expression of the hairstyle is the shape of the polygon model itself, and the change of the hairstyle is realized by deforming the shape of the polygon model.

[0004] A method of expressing hair by a particle system has been proposed. The particle system is a method of easily expressing a large number of particulate objects by giving characteristics such as the mass of the particles as parameters and generating particles according to a certain rule when handling a large number of particulate objects.

[0005] The applicant of the present invention has proposed a method of approximating each individual hair as a curve as shown in FIG. 1B, for example, as Japanese Patent Application No. 10-164545. Successful expression of hairstyle images.

Also, a method of designing a hairstyle by editing a plurality of hairs whose curves are approximated collectively is described in "Kishi, Saegusa, Morishima," Development of Hair Design System Using Tuft Model ", IEICE Technical Report, MVE97. -102, pp.67-74,1998 ''
It is described in.

[0007]

However, in the method of approximating the entire hairstyle by the polygon model, it is difficult to express a complicated hairstyle, and it is not possible to express the movement (flow) of the hair. .

On the other hand, a method of expressing hair by a particle system or a method of approximating hair one by one by a geometric shape can express an arbitrary hairstyle.
In addition, the movement of the hair can be expressed. However, in the method of expressing hair by a particle system, in order to control the hairstyle, it is necessary to control the parameters of the particles expressing the hair, and it is difficult to intuitively control the hairstyle. It was difficult.

"Kishi, Saegusa, Morishima," Development of Hair Design System Using Tuft Model "", IEICE Technical Report, MVE97-
102, pp. 67-74, 1998, it is possible to generate the intended hairstyle relatively easily,
With this method, it is only possible to change the hair corresponding to the fixed head shape to the intended hairstyle, and it is not possible to change the hair to the intended hairstyle corresponding to a plurality of different head shapes. was there.

[0010] The present invention has been made in view of such a situation, and enables the hairstyle of each hair expressed by CG to be changed corresponding to a plurality of different head shapes. The purpose is to:

[0011]

According to the first aspect of the present invention, there is provided an image processing apparatus comprising: reading means for reading hairstyle data from a database in which hairstyle data is recorded; and one piece of hairstyle data included in the hairstyle data read by the reading means. Mapping means for mapping the hair data corresponding to the hair to a predetermined position of the three-dimensional shape image of the head, wherein the mapping means further converts the hair data expressed in the first coordinate system into the head data. 2D in which the three-dimensional shape image of
And a coordinate conversion means for converting the data into data of the coordinate system.

The coordinate conversion means converts the hair data expressed in the first coordinate system into data in a local coordinate system having an origin on the surface of the head, and then displays a three-dimensional image of the head. It can be converted to the data of the second coordinate system.

The coordinate conversion means converts the hair data expressed in the first coordinate system into data in the second coordinate system in which a three-dimensional shape image of the head is expressed, based on a random number. Thus, at least one of coordinate axis rotation and origin movement can be executed.

[0014] The mapping means may further include a deforming means for arbitrarily deforming the three-dimensional shape image of the head.
The hair data corresponding to the hair of the book can be mapped to a predetermined position of the three-dimensional shape image of the head deformed by the deforming means.

According to a seventh aspect of the present invention, there is provided an image processing method comprising: a reading step of reading hairstyle data from a database in which hairstyle data is recorded; and a method for reading one hair included in the hairstyle data read in the processing of the reading step. Mapping the corresponding hair data to a predetermined position in the three-dimensional shape image of the head. The mapping step further includes: converting the hair data expressed in the first coordinate system to the head data. A coordinate conversion step of converting the data into the data of the second coordinate system in which the three-dimensional shape image is expressed.

According to another aspect of the present invention, there is provided a program for a recording medium, comprising: a reading step of reading hairstyle data from a database in which hairstyle data is recorded; and one hair included in the hairstyle data read in the reading step processing. And mapping the hair data corresponding to the head data to a predetermined position of the three-dimensional shape image of the head. In the processing of the mapping step, the hair data expressed in the first coordinate system is further converted to the head data. A coordinate conversion step of converting the data into a data of a second coordinate system in which a three-dimensional shape image of the part is expressed.

According to a ninth aspect of the present invention, the image processing apparatus converts the first image data represented by the first coordinate system into data of the second coordinate system representing the second image. A coordinate conversion unit that converts the first image data expressed in the first coordinate system into data in the second coordinate system in which the second image is expressed, At least one of rotation of the coordinate axis and movement of the origin is executed based on a random number.

According to a tenth aspect of the present invention, there is provided the image processing method according to the first aspect.
A coordinate conversion step of converting the first image data expressed in the coordinate system of the first coordinate system into data of the second coordinate system in which the second image is expressed. When converting the first image data expressed in the coordinate system of the first coordinate system into data of the second coordinate system in which the second image is expressed, at least one of coordinate axis rotation and origin movement based on a random number It is characterized in that one of them is executed.

According to another aspect of the present invention, a program for a recording medium converts first image data represented in a first coordinate system into data in a second coordinate system representing a second image. And converting the first image data represented in the first coordinate system into data in the second coordinate system representing the second image. Then, at least one of rotation of the coordinate axis and movement of the origin is executed based on a random number.

[0020] The image processing apparatus according to claim 1, and claim 7.
In the image processing method according to the above, and the program of the recording medium according to the eighth aspect, the hairstyle data is read from a database in which the hairstyle data is recorded, and one hair included in the read hairstyle data. Is mapped to a predetermined position in the three-dimensional shape image of the head. In the processing in the mapping step, the hair data expressed in the first coordinate system is further converted into data in the second coordinate system in which a three-dimensional shape image of the head is expressed.

An image processing apparatus according to claim 9, wherein
In the image processing method according to the first aspect, the first image data represented by the first coordinate system is replaced with the second image represented by the second image. Is converted to data of the coordinate system. In this coordinate conversion process, when the first image data expressed in the first coordinate system is converted into data in the second coordinate system in which the second image is expressed, based on a random number, , At least one of coordinate axis rotation and origin movement is executed.

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A hair synthesis system to which the present invention is applied will be described with reference to FIG. This hair synthesis system generates a three-dimensional model of a neck based on a captured image of a neck, and synthesizes a CG of an arbitrary hairstyle based on a captured image of the neck (a portion above the neck). The image processing device 2 generates a hairstyle matching image by using the image processing device 2 and the display device 9 that displays the generated hairstyle matching image.
If this hair synthesis system is installed in, for example, a beauty salon, a barber shop, or the like, it can be used for purposes such as assisting a customer in selecting a hairstyle.

The photographing device 1 of the hair synthesis system is a combination of a three-dimensional shape measuring device such as a stereo camera and a range finder, and a two-dimensional image input device such as a CCD (Charge Coupled Device). In the case of a stereo camera, the three-dimensional shape measuring device of the photographing device 1 performs a predetermined image processing by photographing the neck of a person (customer such as a beauty salon) a plurality of times from different directions, and performs predetermined image processing. A three-dimensional shape of the neck portion (from which the hair portion has been removed) is obtained. In the case of a range finder, the distance is measured by laser scanning the neck of the person to be the subject, and the distance of the face is measured.
Get a dimensional shape. The two-dimensional image input device of the imaging device 1 includes:
An image of the neck of the person to be the subject is taken to obtain a texture corresponding to the three-dimensional shape of the face.

In general, the three-dimensional shape measuring device of the photographing device 1 includes a three-dimensional shape measuring part for darkening the light of the subject (for example, the head of a black hair) and a part reflecting light such as metal. Shape data cannot be obtained. Therefore, from the photographing apparatus 1, the three-dimensional shape data of the part of the neck of the subject, such as the face excluding the hair (hereinafter referred to as the face part), and the three-dimensional shape data of the face part are associated with each other. The texture image data is output.

It should be noted that the three-dimensional shape measuring device and the two-dimensional image input device constituting the photographing device 1 may be plural in order to measure the neck of a person (hereinafter referred to as a customer) as a subject from multiple directions. Good.

The image processing device 2 combines the three-dimensional shape data of the head read from the head database 6 with the three-dimensional shape data of the face input from the photographing device 1 to generate a three-dimensional shape model of the neck. The three-dimensional head reconstruction processing unit 3 to be constructed, the hairstyle mapping processing unit 4 that maps hairstyle data read from the hairstyle database 5 to the three-dimensional shape model of the neck, and rendering to the three-dimensional shape model of the neck to which the hairstyle data is mapped Rendering unit 5 for generating hairstyle matched image data, a head database 6 in which a plurality of three-dimensional shape data of the head are recorded in advance, a hairstyle database 7 in which a plurality of hairstyle data are recorded in advance,
And 3 of the head stored in the head database 6
The controller 8 is operated by a user when instructing selection of dimensional shape data, selection of hairstyle data stored in the hairstyle database 7, and the like.

The hairstyle data stored in the hairstyle database 7 has been edited and generated using a three-dimensional graphics modeler or the like. Therefore, it is possible to generate and store hairstyle data for a hairstyle that is actually difficult to style. The data structure of the hairstyle database 7 is shown in FIG.
1 will be described later.

The display device 9 is an LCD (Liquid Crystal Displ.).
ay) or a CRT (Cathode Ray Tube), etc., and displays a hairstyle matching image input from the image processing apparatus 2.

Next, the operation of the hair synthesis system will be described.
This will be described with reference to the flowchart of FIG. 3 and FIG.
In step S <b> 1, the photographing device 1 measures a three-dimensional shape by photographing or laser-scanning the neck of a customer serving as a subject, and outputs three-dimensional shape data and texture image data of the customer's neck to the image processing device 2. .

As a method of measuring the three-dimensional shape, any method such as a method using a stereo camera and a method using a range finder may be used. However, as described above, the three-dimensional shape measuring device of the photographing device 1 uses the 3
Since the three-dimensional shape data cannot be obtained, in the process of step S1, the three-dimensional shape data and the texture image data of the photographed face of the customer (in FIG. 4, the texture image is pasted on the three-dimensional shape data of the face) (Illustrated as the measured data obtained). When the photographing apparatus 1 is provided with a plurality of three-dimensional shape measuring devices and two-dimensional image input devices, a plurality of sets of three-dimensional shape data and texture image data of the face are output.

In step S 2, the three-dimensional head reconstruction processing unit 2 of the image processing device 2 uses the measurement data of the face (three-dimensional shape data and texture image data) input from the photographing device 1 for the customer. Construct a three-dimensional shape model of the head.

More specifically, as shown in FIG. 4, the head data selected and read from the head database 6 is combined with the measurement data of the customer's face to obtain a complete customer's head 3
Rebuild the dimensional model. The head database 6 is composed of a three-dimensional model of only a plurality of heads of arbitrary shapes. The head data is scaled and aligned based on the size and orientation of the measurement data of the customer's face,
Combined with customer measurement data.

The combination of the head data and the measurement data of the customer may be such that a new polygon is generated between the release edges of both data, or the release edge portion of one data is matched with the other release edge. It may be deformed as follows.

The constructed three-dimensional shape of the customer's head is represented by, for example, a triangular polygon model. The constructed three-dimensional model of the customer's head is obtained by removing the hair from the real image of the customer as shown in FIG.

In step S3, the operation of the user (customer or the like) for selecting the hairstyle data stored in the hairstyle database 7 is received by the controller 8, and the selected hairstyle data is read from the hairstyle database 7 and the hairstyle is read. It is supplied to the mapping processing unit 4.

At the time of this hairstyle selection, an icon or a thumbnail indicating the hairstyle is displayed on the display device 9 to urge the user to make a selection. The selection may be made by moving a cursor or the like on the screen of the display device 9, or may be made by inputting a number or the like indicating a hairstyle.

In step S4, the hair style mapping processing section 4 of the image processing device 2 converts the supplied hair style data into
Customer's head 3 supplied from the three-dimensional head reconstruction processing unit 3
The data is mapped to the dimensional model and output to the rendering unit 5. The hairstyle data mapping process maps the hair data corresponding to each individual hair included in the hairstyle data onto the surface of the head 3D model according to the shape of the head 3D model by the same method as texture mapping. It is. The details of the mapping process will be described later with reference to FIGS.

In step S5, the rendering unit 5
Renders the three-dimensional model of the customer's head to which the hairstyle data is mapped, generates hairstyle matched image data as shown in FIG. The rendering method includes texture mapping of a polygon model for the face of the three-dimensional head model,
For the hair part, this is rendering processing of a curve model. The rendering process of the curve model is described in, for example, “Kobayashi, Morishima, Harashima,“ Simulation Model of Threaded Object and Simulation by CG ””, IEICE Technical Report, PRU90-127, pp.15-20,1
991 ", the details of which are disclosed.

The display device 9 displays an image corresponding to the input hairstyle matching image data. The controller 8 prompts the user (customer or the like) to input whether or not the hairstyle matching image displayed on the image device 9 is satisfied.

At this time, the user can operate the controller 8 to view the hairstyle matching image while changing the viewpoint position and the illumination position. When the viewpoint position or the lighting position changes, the rendering unit 5 executes the rendering process of the head three-dimensional model and the hairstyle data again so as to correspond to the changed condition (the viewpoint position or the lighting position) and displays the display device. 9 is output.

If it is determined in step S5 that the user has received an input indicating that the court is satisfied, the process returns to step S3, and the subsequent processing is repeated. If it is determined in step S5 that the input of the user indicating satisfaction has been received, the process ends.

Next, the details of the hair style mapping processing of the hair style mapping processing section 4 in step S4 will be described.
Before that, regarding the configuration example of the hairstyle mapping processing unit 4,
This will be described with reference to FIG.

The hair style data reading section 21 of the hair style mapping processing section 4 reads out the hair style data selected by the controller 8 from the hair style database 7 and outputs it to the mapping position calculating section 22. Mapping position calculator 2
2 calculates a position at which the hair data (corresponding to one hair) constituting the hairstyle data is mapped to the three-dimensional head model. The hair data interpolation processing unit 23 obtains the mapped hair data by interpolating the hair data around the hairstyle data. Coordinate converter 24
Performs mapping by transforming the interpolated hair data from the object coordinate system of the hair data to the object coordinate system of the three-dimensional head model.

Next, details of the hair style mapping process of the hair style mapping processing section 4 will be described with reference to the flowchart of FIG. It is assumed that the hairstyle data is already supplied from the hairstyle data reading unit 21 to the mapping position calculation unit 22 of the hairstyle mapping processing unit 4 by the process of step S3.

The hairstyle data supplied to the mapping position calculator 22 is a two-dimensional array of hair data approximated by an object having a three-dimensional shape, as shown in FIG.

In the example of FIG. 7A, one hair is approximated by a space curve such as a cubic B-spline curve. In the example of FIG. 7B, one hair is approximated by the polygon model. The address of the two-dimensional array corresponds to the mapping coordinate value given to the three-dimensional head model to which the hairstyle is mapped, similarly to the texture mapping process.

For example, the mapping coordinates (u, v) = (0.1, 0...) On the vertices of a predetermined polygon of the head three-dimensional model.
When 1) is given, the address in the hairstyle data array of the hair data mapped to the vertex is (0.1).
u _max , 0.1 v _max ). Here, u _max is the number of data in the u-direction of the hairstyle data array, and _vmax is the number of data in the v-direction of the hairstyle data array.

Since the hairstyle data is a two-dimensional array of hair data having an arbitrary three-dimensional shape, it is possible to express an arbitrary hairstyle, and it is difficult to actually cut the hairstyle. Can be generated even if the hairstyle data is The method of generating the hairstyle data will be described later with reference to FIG.

First, in step S11, the mapping position calculation unit 22 initializes the vertical count value v of the hairstyle data array to 0.0, and in step S12, converts the horizontal count value u of the hairstyle data array. Initialize to 0.0.

In step S13, the mapping position calculation unit 22 calculates to which position in the three-dimensional head model the hair data represented by the address (u, v) in the hairstyle data array is mapped.

A specific description will be given. FIG. 8 shows an arbitrary triangular polygon on the three-dimensional head model. The vertices A, B, and C of the triangular polygon have attribute vectors A _attr , B _attr , and C _attr representing the attributes of the three-dimensional head model, respectively.

Here, the attribute is composed of the object coordinate value of the three-dimensional head model, the local coordinate axis at the vertex, and the mapping coordinate value of the hair. Assuming that vectors forming arbitrary two sides of the triangular polygon ABC are vectors V1 and V2, the vectors V1 and V2 can be expressed by, for example, the following equation (1) using attribute vectors of respective vertices. V1 = B _attr -A _attr V2 = C _attr -A _attr (1)

The vectors V1 and V2 are converted to a triangular polygon AB
Given a C axis of an arbitrary point P _h on the triangular polygon ABC can be expressed by the following equation (2). _{P h = A attr + aV1 +} bV2 ··· (2)

[0054] In this case, coefficients a, b, as shown in FIG. 8, with respect to one axis from any point P _h, intersect point and the arbitrary point when the lower the line parallel to the other axis the length of the P _h, is normalized by the length of each axis vector.

[0055] vertex A, B, respectively a mapping coordinate values of _{C (u a, v a)} , (u b, v b), (u c, v c) as a point represented by the formula (2) P _{When h} is focused on the elements of the mapping coordinate values, the following equation (3) is established. _{U h = U a + a (} U b + U a) + b (U c -U a) V h = V a + a (V b -V a) + b (V c -V a) ··· (3) Equation (3 ) Holds for any point in the (U, V) space.

[0056] Now, the counter value u = u _p, counter value v
= V if it is _p, the condition for the point P _h on the 3D head model to hair data is mapped is inside the triangular polygon is expressed by the following equation (4). a + b ≦ 1 (4)

That is, an arbitrary triangle of the three-dimensional head model
In the polygon, in equation (3), u _h= U_p, V_h= V_pTo
When the coefficients a and b obtained by the substitution satisfy Expression (4),
Point P _hExists inside the triangular polygon,
(U in the hairstyle data array_p, V_p) Hair data represented by
Will be mapped onto the triangular polygon.
You. Therefore, the head 3D model of the hair data at this time is
The mapping position to Dell is determined by the triangle point to be mapped.
Calculated using equation (2) in Ligon.

Returning to FIG. 6, in step S14, the mapping position calculation unit 22 determines whether or not the triangular polygon of the three-dimensional model of the head to be mapped is a hair generation area. This determination is made based on a flag (positive or negative) indicating whether to map the hair data assigned to each triangular polygon of the three-dimensional head model.

If the mapping flag of the triangular polygon including the mapping position obtained in the process of step S13 is positive, it is determined that the triangular polygon is a hair generation area, and the process proceeds to step S15.

[0060] In step S15, the hair data interpolation processing unit 23 interpolates using hair data stored (u _p, v _p) the hair data to be mapped represented by the hairstyle data array. As the interpolation method, an average value of four neighboring points may be used, a weighted average value of four neighboring points may be used, or the nearest neighbor point may be used without interpolation.

FIG. 9 shows a method of using a weighted average value from four neighboring points. Now, the counter value u = u _p, when the counter value v = v _p, hairstyle data array address of the hair data mapped is _{P a = (m p, n} p) =
_{(U p · u max, v} p · v max) is. As Hp the hair data at point P _a, the 4-point neighborhood, as the point P1 to P4 as shown in FIG. 9, the hair data at each point P1 to P4 to as H1 to H4.

In the weighted averaging method from the four neighboring points, the hair data Hp to be mapped is interpolated from the data of the four neighboring points using the following equation (5). Hp = dm (dn · H2 + (1-dn) H1) + (1-dm) (dn · H4 + (1-dn) H3)) (5)

Here, when the hair data is a space curve such as a cubic B-spline curve as shown in FIG.
The interpolation processing of the hair data is obtained by interpolating the data of each control point. On the other hand, when the hair data is a polygon model as shown in FIG. 7B, the interpolation processing of the hair data is obtained by interpolating each polygon vertex data. In addition,
In the equation (4), dm is a decimal part of m _p , and dn is n _p
Is the decimal part.

Referring back to FIG. 6, in step S16, the coordinate conversion unit 24 converts the interpolated hair data from the object coordinate system of the hair data to the object coordinate system of the three-dimensional head model.

The concept of coordinate conversion will be described with reference to FIG. Let H be any hair data in the hairstyle data array. The hair data H is represented in the object coordinate system (I, J, K) of the hair data. The object coordinate system (I, J, K) of the hair data is arbitrary as long as it is a three-dimensional coordinate system whose origin is the starting point of the hair data H. For example, the horizontal direction of the hairstyle data array is I, and the vertical direction is J. And a direction orthogonal to them as K. Thus, the starting point of the hair data H is (I, J, K) = (0, 0,
0), and the other control points are represented by relative positions from the start point of the hair data.

On the other hand, the three-dimensional head model is represented by the object coordinate system (X, Y, Z). The object coordinate system of the head three-dimensional model is generally a three-dimensional orthogonal coordinate system. For example, with the center of gravity of the head as the origin, the right ear direction is X, the parietal direction is Y, and the nose direction is Z. It is an orthogonal coordinate system such as

[0067] The coordinate transformation unit 24, the object coordinate system of the hair data (I, J, K) the hair data H, represented by the local at the mapping position P _h of the 3D head model which is obtained in step S13 Convert to a coordinate system (T, S, N). Local coordinate system in this respect _{P h (T, S, N} ) is
Any three-dimensional coordinate system with the point _Ph as the origin is arbitrary.
For example, the normal vector at the point P _h and N, Y in the longitudinal direction of the central axis (FIG. 10 of the 3D head model
The tangential vector is determined as the outer product of the radiation direction vector from the axis) to a point P _h and normal vector N is T, the orthogonal coordinate system as the S vector perpendicular to the normal direction N and tangential direction T It is.

Here, it is assumed that the local coordinate system at each vertex of the three-dimensional head model is given in advance to the head database. Therefore, the local coordinate system at the point P _h (T, S, N), as well as the calculation of the mapping position is calculated using equation (2) in the triangle polygon is mapped. Now, an arbitrary control point Ci of the hair data H
In the object coordinate system (I, J, K) of (i, J, K)
j, k) and when the control points Ci is the local coordinate system at the point P _h (T, S, N) in is expressed by the following equation (6). Ci = iT + jS + kN (6)

Here, each unit vector of the local coordinate system (T, S, N) corresponds to the object coordinate system (X,
Y, Z).

Therefore, the control point Ci represented by the equation (6)
Is expressed in the object coordinate system (X, Y, Z) of the three-dimensional head model.

Returning to FIG. 6, in step S17, the mapping position calculating section 22 determines whether or not the counter value u has exceeded 1.0. When it is determined that the counter value u has not exceeded 1.0, the process proceeds to step S18.
In step S18, the mapping position calculation unit 22
Increments the counter value u by the step value du. The step value du is a value indicating at what interval the hair data stored in the hairstyle data array is mapped in the horizontal direction.

Thereafter, the process returns to step S13, and the subsequent processes are repeated. If it is determined in step S17 that the counter value u has exceeded 1.0, the horizontal mapping process is performed for one row of the hairstyle data array. Since it means that the end has been reached, the process proceeds to step S19.

In step S19, the mapping position calculator 22 determines whether the counter value v has exceeded 1.0. When it is determined that the counter value v does not exceed 1.0, the process proceeds to step S20. Step S2
At 0, the mapping position calculation unit 22 increments the counter value v by the step value dv. Step value dv
Is a value indicating at what interval the hair data stored in the hairstyle data array is mapped in the vertical direction.

Thereafter, the flow returns to step S12, and the subsequent processing is repeated. If it is determined in step S19 that the counter value v has exceeded 1.0, it means that the mapping processing for all rows has been completed. Therefore, the hair mapping process ends, and the process proceeds to step S5 in FIG.

In step S14, if the mapping flag of the triangular polygon including the mapping position obtained in the processing of step S13 is negative, it is determined that the triangular polygon is not a hair generation area, and steps S15 and S16 Is skipped.

By specifying the hair generation area using the mapping flag in this manner, all the hair data constituting the hairstyle data read out from the hairstyle database 7 are not mapped, and an arbitrary area of the head three-dimensional model is mapped. It is possible to map only. Therefore, for example, a hairstyle such as a so-called mohawk haircut or a haircut cut into a star shape can be easily generated.

According to the hairstyle mapping process,
Each hairstyle data stored in the hairstyle database 7 is
It is expressed in the object coordinate system of the hair, and can be mapped to a three-dimensional model of the head of an arbitrary shape without depending on the shape of the human head.

Next, the data structure of the hairstyle database 7 will be described with reference to FIG. The hairstyle database 7 includes a plurality of hairstyle data. The hairstyle data is a two-dimensional array as described above, and the number of data u _{max in} the horizontal direction and the number of data v _{max in the} vertical direction of the array are described at the beginning, and thereafter, u _max × v _max pieces of hair data are described. It has been described.

In the hair data, group information (described later) may be described at the head. Hereinafter, when the hair data is approximated by a cubic parametric curve or the like, a plurality of control point data are described. The control point data is expressed in the hair object coordinate system (I, J, K).

The hair data may be three-dimensional polygon data. In this case, a plurality of polygon vertex data is described as hair data.

The group information described in the hair data will be described. The group information can be used in the hair data interpolation process in step S15 of the hair style mapping process (FIG. 6) described above.

The hair data interpolation process when group information is given will be described with reference to FIG. Now, the counter value u = u _p, counter value v = v _p
When it is, the address of hairstyle data array of the hair data mapped _{P a = (m p, n} p) = (u p ·
u _max , v _p · v _max ).

[0083] and Hp the hair data at point P _a, the 4-point near the points P1 to P4 as shown in FIG. 12, each of the hair data of the point P1 to P4 to as H1 to H4. Further, it is assumed that 1 is described as group information in the hair data H1, and 2 is described as group information in the hair data H2 to H4. That is, in this example, only the hair data H1 belongs to the group 1 and the other hair data H2 to H4 belong to the group 2. When group information is given, in the hair data interpolation processing, interpolation is performed using only hair data belonging to the same group.

Specifically, the point P to be mapped first is
Of the four adjacent points of _a, nearest neighbor is determined, the group information is obtained. In the case of FIG. 12, the nearest point is the point P3.
It is. Next, the number of neighboring points belonging to the same group as the nearest neighboring point among the four neighboring points P1 to P4 is obtained.

[0085] 4 of the neighboring point P1 to P4, neighboring points belonging recently to the same group as neighbor if no other closest point of the hair data are directly be used in the point P _a.

Of the four neighboring points P1 to P4, the number of neighboring points belonging to the same group as the nearest point is 2 including the nearest point.
, The linear interpolation value between the two points is used for the interpolation.

Among the four neighboring points P1 to P4, the number of neighboring points belonging to the same group as the nearest point is 3 including the nearest point.
, That is, in the case as shown in FIG. 12, the interpolation value is calculated by the following equation (7). Hp = (1-r) H32 + r · H34 H32 = dn · H2 + (1-dn) H3 H34 = dn · H4 + (1-dn) H3 r = (m p -m32) / (m34-m32) ··· ( 7)

Where dn is the decimal part of n _p and m3
2 is the m address at point P32, and m34 is the point P34.
Is the m address.

In this manner, by describing the group information in the hair data, it is possible to express a hairstyle that causes inconvenience due to interpolation, such as a division of hair. In other words, if the grouping is performed at the boundary, even if the four neighboring points straddle the boundary, unnatural hair may be generated using the hair data on both sides of the boundary. Can be deterred.

Next, an application example of coordinate transformation in the processing of step S16 of the above-described hairstyle mapping processing (FIG. 6) will be described with reference to FIGS.

In the coordinate conversion process of step S16,
It is possible to express the variation of the hairstyle using random numbers. FIG. 13B illustrates the hairstyle variation by rotating the local coordinate system using random numbers in the coordinate transformation, whereas the hairstyle variation illustrated in FIG. 13A is not represented. 4 shows an example of a state.

Specifically, the local coordinate system (T, S,
After N) is rotated around the axis of N with respect to the generated random number, it is converted to the object coordinate system (X, Y, Z) of the head model. By such a process, the direction in which each hair grows can be slightly changed without changing the position where the hair data is mapped.

FIG. 14 (B) shows a variation of the hairstyle by changing the mapping position using random numbers in the coordinate transformation, while the variation of the hairstyle shown in FIG. 14 (A) is not represented. An example of the expressed state is shown.

[0094] Specifically, the local coordinate system (T, S, N) the position of P _h is the origin of, after shifting to the P _n corresponding to the generated random number, the object coordinate system of the head model ( X, Y, Z). By such a process, the position where the hair grows can be finely changed without changing the hairstyle.

It is to be noted that the use of random numbers to rotate the axis of the coordinate system to be transformed or to move the origin to express a natural variation in the synthesized image is not limited to the synthesis of the hair image. It is also possible to apply.

In this hair synthesis system, 3
After the three-dimensional head model is transformed into an arbitrary shape by the three-dimensional head reconstruction processing unit 3, the hairstyle data can be mapped. FIG. 15B shows a state in which the three-dimensional head model shown in FIG. 15A is not deformed, and the three-dimensional head model is enlarged 1.5 times in the Y-axis direction. I have.

The enlargement / reduction processing of the three-dimensional head model is performed by affine transformation processing. At this time, the same affine transformation processing is performed for the local coordinate system (T, S, N) given to an arbitrary polygon vertex of the head three-dimensional model.

As shown in FIG. 15 (B), when the three-dimensional head model is enlarged 1.5 times in the Y-axis direction, the transformation matrix for 1.5 times in the Y-axis direction is T, S , N to obtain a local coordinate system (T ′, S ′, N ′) which is stretched in the vertical direction. By mapping the hair data H to the local coordinate system (T ′, S ′, N ′) thus deformed, the hairstyle is enlarged 1.5 times in the Y-axis direction in accordance with the head. Shape. By such processing, the same hairstyle data can be mapped to a three-dimensional head model deformed into an arbitrary shape.

Next, generation of hairstyle data will be described with reference to FIG.
This will be described with reference to FIG. FIG. 16 shows a cylindrical coordinate system surrounding the head. The cylinder coordinate system is a coordinate system in which an axis passing through the center of the cylinder is the Y axis, a distance from the Y axis on a plane perpendicular to the Y axis is r, and an angle from the reference direction on this plane is θ. .

[0100] Any point P _h of the head surface can be expressed P _h = using cylindrical coordinates _{(y p, r p, θ} p) and. Furthermore, in the case of the cylindrical coordinate system Y = y _p, the contact lines and the head surface determined by theta = theta _p is only always one point, any point P _h of the head surface expressed in (yp, theta] p) can do.

That is, if the shape of the head surface is always convex in the circumferential direction, the above condition (Y = Y in the cylindrical coordinate system)
(the point of contact between the straight line defined by y _p and θ = θ _p and the surface of the head is always one point).

[0102] Considering the scalp portion excluding the face, because in general considered to be always convex toward the circumferential direction, an arbitrary point P _h of the scalp portion of the head model P _h = (Y
p, θp).

The hairstyle data creation is described in “Kishi, Saegusa, Morishima,“ Development of Hair Design System Using Tuft Model ””,
By using the system disclosed in IEICE Technical Report, MVE97-102, pp. 67-74, 1998, it can be performed relatively easily. In that system, hairstyle data is created by editing the hair data generated on a given head model. Hair data output from this system is expressed in the object coordinate system of a given head model.

The hair database 7 of the present invention records hairstyle data stored in a two-dimensional array by mapping this hair data to a space represented by (y, θ) in a cylindrical coordinate system. For conversion from the (y, θ) space to the (u, v) space of a two-dimensional array as shown in FIG. 7, for example, the following equation (8) is used. _{u = θ p / 2π v =} (y p -y min) / (y max -y min) ··· (8)

Here, y _min is the minimum value of the Y axis in the cylindrical coordinate system on the head surface, and y _max is the maximum value of the Y axis in the cylindrical coordinate system on the head surface.

The conversion from the (y, θ) space to the (u, v) space may be non-linear. For example, the closer to the top of the head, the more
By performing the conversion so as to reduce the step in the v direction, it is possible to prevent the sampling near the top of the head from becoming coarse.

The hairstyle data composed of the hair data converted into the (u, v) space as described above is recorded in the hairstyle database 7.

The present invention can be applied to a case where a thread-like object other than hair is represented by CG.

The above-described series of processes of the image processing apparatus 2 can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a recording medium into a possible general-purpose personal computer or the like.

FIG. 17 shows an example of the configuration of a personal computer for realizing the image processing apparatus 2. This personal computer has a CPU (Central Processing Unit) 4
1 is built in. An input / output interface 45 is connected to the CPU 41 via a bus 44. The input / output interface 45 includes an input unit 46 including an input device such as a keyboard and a mouse corresponding to the controller 8.
An image output unit 47 that outputs a hairstyle matching image as a processing result to the display device 9; a storage unit 48 that includes a hard disk drive or the like that stores programs and various data; a communication unit 49 that communicates data via the Internet; A drive 50 for reading and writing data from and to a recording medium such as the magnetic disk 51 to the semiconductor memory 54 is connected. The bus 44 has a ROM (Read Only Memory) 4
2 and a random access memory (RAM) 43 are connected.

An image processing program for causing the personal computer to execute the operation as the image processing apparatus 2 is supplied to the personal computer in a state where it is stored in the magnetic disk 51 to the semiconductor memory 54, read out by the drive 50, and It is installed in a hard disk drive built in the storage unit 48. The image processing program installed in the storage unit 48 is transmitted from the storage unit 48 to the RAM 43 by a command from the CPU 41 corresponding to a command from the user input to the input unit 46.
Is loaded and executed.

In this specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in chronological order according to the described order, but are not necessarily performed in chronological order. Alternatively, it also includes individually executed processing.

In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0114]

As described above, according to the image processing apparatus of the first aspect, the image processing method of the seventh aspect, and the program of the recording medium of the eighth aspect, the hairstyle data is recorded. The hair data corresponding to one hair included in the hairstyle data read from the database is mapped to a predetermined position in the three-dimensional shape image of the head, and is then represented in the first coordinate system. Since the hair data is converted into data of a second coordinate system in which a three-dimensional shape image of the head is represented, the hairstyle of each hair expressed by CG can be converted into a plurality of different head shapes. Can be changed in accordance with.

According to the image processing apparatus of the ninth aspect, the image processing method of the tenth aspect, and the program of the recording medium of the eleventh aspect, the image is represented by the first coordinate system. When converting the first image data into the data of the second coordinate system in which the second image is expressed, at least one of the coordinate axis rotation and the origin movement is converted based on the random number. Thus, it is possible to give a natural variation to an image to be synthesized.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a conventional hair expression.

FIG. 2 is a block diagram illustrating a configuration example of a hair synthesis system to which the present invention has been applied.

FIG. 3 is a flowchart illustrating an operation of the hair synthesis system.

FIG. 4 is a diagram for explaining the operation of the hair synthesis system.

FIG. 5 is a block diagram illustrating a configuration example of a hairstyle mapping processing unit 4 in FIG. 2;

FIG. 6 is a flowchart illustrating processing of a hairstyle mapping processing unit 4;

FIG. 7 is a diagram for explaining hairstyle data in which hair data is two-dimensionally arranged.

FIG. 8 is a diagram for explaining mapping position calculation in step S13 in FIG. 6;

FIG. 9 is a diagram for explaining the hair data interpolation process in step S15 of FIG. 6;

FIG. 10 is a diagram for explaining coordinate conversion in step S16 of FIG. 6;

11 is a diagram showing a data structure of a hairstyle database 7 of FIG.

FIG. 12 is a diagram illustrating an interpolation process using group information included in hair data.

FIG. 13 is a diagram for explaining coordinate axis rotation in coordinate conversion.

FIG. 14 is a diagram for explaining coordinate axis movement in coordinate conversion.

FIG. 15 is a diagram for explaining mapping for an arbitrarily deformed three-dimensional head model.

FIG. 16 is a diagram illustrating a cylindrical coordinate system when hairstyle data is generated.

FIG. 17 is a diagram illustrating a configuration example of a personal computer that realizes the image processing apparatus 2.

[Explanation of symbols]

Reference Signs List 1 shooting device, 2 image processing device, 3 three-dimensional head reconstruction processing unit, 4 hair type mapping processing unit, 5 rendering unit, 6 head database, 7 hair type database, 8 controller, 9 display device, 21
Hairstyle data readout unit, 22 mapping position calculation unit, 23 hair data interpolation processing unit, 24 coordinate conversion unit, 51 magnetic disk, 52 optical disk, 5
3 magneto-optical disks, 54 semiconductor memory

Claims (11)

[Claims] 1. An image processing apparatus for generating a hairstyle matching image by combining a hair image with a three-dimensional shape image of a head, wherein the hairstyle data is read from a database in which hairstyle data is recorded; The hair data corresponding to one hair included in the hair style data read out by the means is stored in the 3D of the head.
Mapping means for mapping to a predetermined position of a three-dimensional shape image, wherein the mapping means further expresses the hair data expressed in a first coordinate system in a three-dimensional shape image of the head. An image processing apparatus, comprising: a coordinate conversion unit that converts data into data of a second coordinate system. 2. The method according to claim 1, wherein the coordinate conversion unit converts the hair data expressed in the first coordinate system into data in a local coordinate system having an origin on the surface of the head. The image processing apparatus according to claim 1, wherein the image data is converted into data of the second coordinate system representing a three-dimensional shape image. 3. The coordinate conversion means converts the hair data expressed in the first coordinate system into data in the second coordinate system in which a three-dimensional shape image of the head is expressed. 2. The image processing apparatus according to claim 1, wherein, when performing, at least one of rotation of a coordinate axis and movement of an origin is executed based on a random number. 4. The mapping means further includes a deformation means for arbitrarily deforming the three-dimensional image of the head, and hair data corresponding to one hair included in the hairstyle data read by the reading means. 2. The image processing apparatus according to claim 1, wherein the mapping is performed at a predetermined position of the three-dimensional shape image of the head deformed by the deforming unit. 5. The image processing apparatus according to claim 1, wherein the hair data corresponding to the one hair is three-dimensional curve data including a plurality of control points. 6. The image processing apparatus according to claim 1, wherein the hair data corresponding to the one hair is polygon data including a plurality of polygons. 7. An image processing method of an image processing apparatus for generating a hairstyle matching image by combining a hair image with a three-dimensional shape image of a head, wherein the hairstyle data is read from a database in which hairstyle data is recorded. And hair data corresponding to one hair included in the hairstyle data read in the processing of the reading step,
And a mapping step of mapping the hair data represented in a first coordinate system to the three-dimensional shape image of the head. An image processing method including a coordinate conversion step of converting data into data of a second coordinate system in which a shape image is expressed. 8. A program for image processing for generating a hairstyle matching image by combining a hair image with a three-dimensional shape image of a head, and reading out the hairstyle data from a database in which hairstyle data is recorded. And hair data corresponding to one hair included in the hairstyle data read in the processing of the reading step,
And a mapping step of mapping the hair data represented in a first coordinate system to the three-dimensional shape image of the head. A recording medium on which a computer-readable program is recorded, comprising a coordinate conversion step of converting data into a second coordinate system in which a shape image is expressed. 9. An image processing apparatus for synthesizing first image data with second image data, wherein the first image data expressed by a first coordinate system is expressed by the second image. Coordinate conversion means for converting the first image data represented in the first coordinate system into the second image. The image processing apparatus according to claim 1, wherein at least one of rotation of a coordinate axis and movement of an origin is executed based on a random number when converting the data into the expressed second coordinate system data. 10. An image processing method of an image processing apparatus for combining first image data with second image data, wherein the first image data expressed in a first coordinate system is converted to the second image data. A coordinate conversion step of converting the image data into data of the second coordinate system, wherein the processing of the coordinate conversion step includes: converting the first image data represented by the first coordinate system into: When converting into the data of the said 2nd coordinate system in which the said 2nd image is represented, at least one of coordinate axis rotation and origin movement is performed based on a random number, The image processing method characterized by the above-mentioned. 11. An image processing program for combining first image data with second image data, wherein the first image data expressed in a first coordinate system is converted to the second image data. A coordinate conversion step of converting the image data into data of the second coordinate system, wherein the processing of the coordinate conversion step includes: converting the first image data represented by the first coordinate system into: When converting into the data of the second coordinate system in which the second image is expressed, at least one of rotation of a coordinate axis and movement of an origin is executed based on a random number, wherein the computer is readable. Media on which various programs are recorded.