JP2003044841A - Device and method for generating model and recording medium with model generation program recorded thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a real and complicated model by a simple operation. SOLUTION: A model generation device is provided with a storing part 1, an image inputting part 2, a user inputting part 3 and a model generating part 5. The model generating part 5 detects the size, position and direction of a target object in an image, calculates the position of an image of each vertex in the case of projecting a rough shape model stored in the storing part 1 to the same size, position and direction which are detected, also calculates the position of an image of each vertex in the case of projecting the same rough shape model to predetermined and regulated size, position and direction, generates an approximate image in the case of looking at the target object from the predetermined and regulated size, position and redirection by deforming the image on the basis of correspondence relation between coordinates on the both images of the calculated same vertexes, and generates a model with an image as image data to be pasted for pasting the approximate image to the rough shape model.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a model generating device and a model generating method for generating a realistic model using an image, and a recording medium recording a model generating program.

[0002]

2. Description of the Related Art Conventionally, various techniques for generating a realistic three-dimensional model using an image have been proposed.

For example, as described in Reference Document 1 (Masahide Kaneko et al., "Detection of Shape Change and Coding of Face Moving Image Based on Three-Dimensional Shape Model", IE87-101), a face image is displayed. , Project a three-dimensional schematic shape model created in advance using the knowledge about the shape of the face, and generate a texture to be attached to the three-dimensional model from the correspondence between the projected coordinates of the three-dimensional model and the image of the face. , Thereby generating a textured three-dimensional model. Then, if necessary, the expression is processed to generate a facial expression, and by projecting from a direction different from the one projected on the input image, a pseudo image viewed from a different direction is generated and displayed. Has been done.

Further, in order to generate a more realistic three-dimensional model, a three-dimensional schematic shape model created in advance using a knowledge about the shape of the target object is projected on the image of the target object, Finely adjust the coordinates of each vertex or feature point on the projected three-dimensional schematic shape model by performing an operation such as deformation so as to match the point corresponding to each vertex or feature point on the target object on the image. Such a method was also adopted.

For example, in Japanese Unexamined Patent Publication No. 4-289976, the operator inputs two-dimensional graphic information of a target three-dimensional object and a three-dimensional basic shape model that approximates the basic shape of the target object. By associating the feature points on the three-dimensional figure information with the control points on the three-dimensional basic shape model,
A method of deforming a dimensional basic shape model to generate a desired three-dimensional shape model is disclosed.

Further, in order to generate a complicated three-dimensional model, a three-dimensional model having a simple shape, a three-dimensional model created by a certain method are combined as a part, or a three-dimensional model formed by combining the three-dimensional models is another one. A method of generating a more complex three-dimensional model by treating the parts as parts and then combining these parts has been adopted.

[0007]

In general, when photographing an image, even if "from the front" or "from this direction" is designated, the image is often photographed in a strictly different direction. is there. For example, when photographing a human face, in many cases, even though the subject is supposed to be facing “front”, in reality, the subject is facing slightly upward or slightly downward due to personal habits. Therefore, in a system premised on pasting an image photographed from the front as texture data, it is necessary to photograph the subject in a strict "front" direction by some method. However, this has a problem in that it is impossible to take a picture easily and a general photographed face cannot be used as an input image.

Even if such a problem is solved, the outline of the rough shape model and the outline of the target object on the actual image may not exactly match. In this case, the outline is generated. There is also a problem that the color of the background or the like is mixed around the contour of the three-dimensional model, and a three-dimensional model that is unnatural in color is generated.

Further, in the above-mentioned conventional technique, in order to prepare a plurality of different shape models, it is necessary to individually prepare textures for all the prepared shape models.
There was also a problem that it took a lot of work.

Further, in the conventional technique for generating a three-dimensional model by combining the above-mentioned two or more parts, how to combine them becomes a problem. The term “combination method” as used herein specifically refers to position, size, etc., but in some cases, the shape itself may be a problem. For example, considering a case where a three-dimensional model of a face and a three-dimensional model of a hairstyle are combined, if the face is too large, the hairstyle is buried in the face and becomes unnatural. Conversely, if the hairstyle is too big,
My hair is floating and it looks unnatural. Further, if the shape on the upper side of the face and the shape on the lower side of the hairstyle (so-called boundary portion) do not match, part of the hair may float or be buried, which is also unnatural. Furthermore, it goes without saying that even if the sizes and shapes match, it becomes unnatural if the positions are displaced.

Conventionally, in order to eliminate such unnaturalness, the operator adjusts the size, position, and shape near the joint each time they are assembled, but this work requires a great deal of labor. There was a problem that it took time.
Further, although a method of preparing an adjustment table in advance and referring to this table by the combined parts to adjust the table is also adopted, there is a similar problem that it takes time and effort to create the table.

Further, in order to generate a more realistic three-dimensional model, it is necessary to prepare more rough shape models, but when the number of rough shape models to be prepared increases,
There is also a problem that the operator's burden of selecting this is increased.

Further, the face is an object having a complicated shape, and it requires a great deal of time and effort to create a realistic three-dimensional model of the face even if the above-mentioned conventional technique is used.

The present invention was devised to solve such a problem, and an object thereof is to provide a model generation device, a model generation method, and a model generation program capable of generating a real and complicated model by a simple operation. It is to provide a recording medium for recording.

[0015]

In order to solve the above-mentioned problems, a model generation device of the present invention is a device for generating a model of a target object included in an image, and is capable of designating an arbitrary position in the image. And a storage unit that stores in advance various data including a schematic shape model of a target object in the image, and image information, position information specified by the position designation unit, and a storage unit that stores the image information. And a model generation unit for generating a model based on various data, wherein the model generation unit detects the size, position and direction of the target object in the input image,
The same schematic shape model is previously stored in the first calculation means for calculating the position on the image of each vertex when the schematic shape model stored in the storage unit is projected in the same detected size, position and direction. Second calculation means for calculating the position on the image of each vertex when projected on the defined size, position and direction, and the first calculation means and the second calculation means.
The image is transformed based on the correspondence between the coordinates of the same vertex on both images calculated by the calculation means, and the approximate image when the target object is viewed from a predetermined size, position, and direction determined in advance. And a means for generating a model with an image as pasted image data for pasting the approximate image onto the schematic shape model.

Further, the model generation method of the present invention is a model generation method by a model generation device that generates a model of a target object included in an image, and detects the size, position and direction of the target object in the image. The step and the outline shape model stored in the storage unit are detected with the same size,
The calculation step to calculate the position of each vertex on the image when projected in the position and direction, and the position on the image of each vertex when projected to the same prescribed shape, position and direction of the same schematic shape model. The image is transformed based on the calculation step for calculating the image and the correspondence between the coordinates of the same vertex on both images calculated by these calculation steps, and the target object has a predetermined size, position, and direction determined in advance. And a step of generating a model with an image as pasted image data for pasting the approximate image on the schematic shape model.

A recording medium recording the model generation program of the present invention is a computer-readable recording medium recording a program for generating a model of a target object included in an image, and the size of the target object in the image. A step of detecting the position and the direction, and a step of calculating the position on the image of each vertex when the schematic shape model stored in the storage unit is projected in the same detected size, position and direction. , A calculation step for calculating the position of each vertex on the image when the same schematic shape model is projected in a predetermined size, position and direction, and both images of the same vertex calculated by these calculation steps By transforming the image based on the correspondence of the coordinates,
A step of generating an approximate image when the target object is viewed from a predetermined size, a position, and a direction, and a step of generating a model with an image as pasted image data for pasting the approximate image on the rough shape model; It is characterized by including.

According to the apparatus, method and recording medium of the present invention having such characteristics, when there is a knowledge about a rough shape model of a target object in advance, the coordinates are calculated from the coordinates of the feature points of the target object in the image. Detecting the largeness, position and direction of the target object, projecting a rough shape model created in advance based on the knowledge about the rough shape model, projecting at the same detected size, position and direction,
The position (coordinates) on the image of each vertex is calculated. Further, the same schematic shape model is projected on an image with a predetermined size, position and direction defined in advance, and the position (coordinates) of each vertex on the image is calculated. Then, by deforming the image based on the correspondence between the coordinates of the same vertex on both images,
An approximate image of the target object viewed from a predetermined size, position, and direction is generated, and an image-attached model is generated as the attached image data for attaching the approximate image to the rough shape model.

According to the model generating device of the present invention,
A means for deforming the image based on the correspondence between the coordinates of the same vertex on both images calculated by the first calculating means and the second calculating means, and a predetermined size for predetermining the general shape model. , The area corresponding to the vicinity of the contour and / or the area outside the contour when projected in the position and direction is filled with the pixel values inside the contour, so that the target object can be viewed from a predetermined size, position and direction. In this case, a means for generating an approximate image is further provided.

Further, according to the model generation method of the present invention,
A step of deforming an image based on the correspondence between the coordinates of the same vertex on both images calculated by the respective calculation steps, and a case in which the rough shape model is projected in a predetermined size, position and direction. By filling the area near the contour and / or the area outside the contour with the pixel values inside the contour, an approximate image is generated when the target object is viewed from a predetermined size, position, and direction determined in advance. And further comprising steps and.

Further, according to the recording medium in which the model generating program of the present invention is recorded, a step of deforming an image based on the correspondence relationship between the coordinates of the same vertex on both images calculated in each of the calculation steps, By filling the area near the contour and / or the area corresponding to the outside of the contour when the schematic shape model is projected in a predetermined size, position, and direction with the pixel value inside the contour, the target object is previously set. And a step of generating an approximate image as viewed from a defined size, position and direction.

According to the apparatus, method and recording medium of the present invention having such characteristics, there is a prior knowledge regarding a rough shape model of a target object, and there is commonality in colors near the contour and other areas. In the case where the image is observed, by determining the pixel value of the pasted image data corresponding to the vicinity of the contour of the generation model from one or a plurality of pixel values in the area inside the contour in the schematic shape model, It is possible to generate a model with a more natural color.

Further, according to the model generating apparatus of the present invention,
The storage unit stores a plurality of schematic shape models in which image pasting rules are fixed in advance, and the model generating unit creates images so as to conform to the preset image pasting rules. A model is selected from a plurality of prepared schematic shape models, and a model with an image is generated by combining the selected schematic shape model and the created image.

According to the apparatus of the present invention having such characteristics, the difference between the plurality of schematic shape models represents the variation of the shape of the target object when there is a knowledge about the general shape model of the target object in advance. , That is, when the outline shape models are common to some extent, the image pasting rule is set so that the position of the feature point in any of the outline shape models corresponds to the position of the feature point on the image data screen. By defining, it becomes possible to use the same pasted image in any schematic shape model.

The model generation device of the present invention is a device for generating a model of a target object by combining a plurality of models, which are component parts of the target object created in advance, according to a predetermined rule. The sizes, positions and directions of the parts to be combined are predetermined, and the shapes of the joints to be combined are made uniform in advance. Further, the model generation device of the present invention enlarges, reduces, or reduces the models generated by combining them as necessary.
It is characterized by further comprising means for rotating, moving, etc.

According to the model generating apparatus of the present invention having such characteristics, when the properties of the parts to be combined are known in advance, for example, in the case of a face and a hairstyle, the joining surface to be combined has a constant shape, for example, the face. If the hairstyle is, the shape of the scalp, etc. is unified, and the model data is created in advance so that the spatial position of the joint surface is also unified. By using the model prepared in this way, no matter which face model and which hairstyle model are combined, the combined head model can be used without adjusting the size, position, and direction. Can be generated. In addition, even if the whole head model generated by combining is enlarged, reduced, moved, rotated, etc.,
No misalignment occurs in the combined joint surface.

Further, the model generation device of the present invention is a device for generating a model of a target object included in an image, and a position designation unit capable of designating each part of the target object included in the image, and A model generation unit that generates a model of the target object based on the image information and the position information designated by the position designation unit, wherein the model generation unit converts the one or more position information designated by the position designation unit. Based on the contour feature of the target portion based on the detected contour feature of the target portion, the contour detection / determination means for determining the shape of the target object from the detected contour feature of the target portion, and the model of the structure corresponding to the determination result are selected, And a model generation means for generating a model.

The model generation method of the present invention is a method for generating a model of a target object included in an image, and the step of designating each part of the target object by the position designating section and the location designating section. Detecting the contour feature of the target region based on one or more position information, determining the shape of the target region from the detected contour feature of the target region, and selecting the model of the structure corresponding to this determination result. And generating a desired model.

A recording medium recording the model generation program of the present invention is a computer-readable recording medium recording a program for generating a model of a target object included in an image, and positions each part of the target object. A step of designating by the designating section and a step of detecting the contour feature of the target site based on the one or more position information designated by the position designating section, and determining the shape of the target site from the detected contour feature of the target site. And a step of selecting a model having a structure corresponding to the determination result and generating a desired model.

According to the apparatus, the method and the recording medium of the present invention having such characteristics, the operator first uses the position specifying unit to detect the characteristics of the face of a target object, for example, a person included in the image. Points (eg, eyes, center of mouth, etc.)
Enter the location information. Next, the contour feature of the face (contour line or the like) is detected from the input feature point information. Next, the shape of, for example, the jaw, which is the target part (for example, the shape of the face is round, oval, square, etc.) is determined from the detection result. Here, for the shape determination, for example, the detected contour line is compared with a plurality of reference face contour lines (reference face contours) that are prepared in advance, and the shape of the reference face contour that is approximated is desired. The face contour shape of

On the other hand, a face model corresponding to the reference face contour is prepared in advance. It should be noted that in the face model, only the contour features are created in advance by a professional designer or the like in accordance with the features of the reference face contour and stored in the storage unit as model structure information.

Finally, the corresponding face model is determined based on the finally obtained reference contour shape. That is, the desired face model may be generated by reading out the face model structure information determined from the storage unit.

In this way, the operator can easily generate and use a model that captures the features of the jaw shape of the face by simply inputting a few feature points such as the eyes and mouth of the face. Becomes

As a method of extracting the contour line from the face contour feature and determining the shape, for example, the method shown in the above-mentioned reference 1, that is, a dynamic contour model is used in the actual image. It is possible to use a method of determining the jaw shape by extracting the contour line of the object and comparing the contour line of the reference face contour with the feature points of the contour distance function.

According to the model generation device of the present invention,
The contour detecting / determining means designates the position information of the characteristic points of the target object by the position designating unit to determine the center position of the target object on the image, and the initial contour near the contour of the target object. Arrangement means, based on the center position of the target object on the image and the initial contour, calculate the color difference between adjacent pixels on a straight line connecting the center coordinates of the target object and each coordinate on the initial contour, A means for creating a color difference map image having the calculated color difference as a pixel value with the coordinate midpoint between the target pixels as a coordinate value, and using the created color difference map image, moving the initial contour according to the active contour model. According to the above, the means for extracting the contour line, the means for calculating the distance function based on the extracted contour line, and the characteristics of the calculated distance function are created in advance from the contour line of the shape of the reference target region. Compare with distance function It allows characterized in that it comprises a means for determining the shape of the target site.

The model generation method of the present invention is a method for generating a model of a target object included in an image, wherein position information of feature points of the target object is designated by a position designating unit, and the target object on the image is designated. Based on the step of determining the center position of the object, the step of placing the initial contour near the contour of the target object, and the center position and initial contour of the target object on the image, the center coordinates and initial contour of the target object Calculating a color difference between adjacent pixels on a straight line connecting each of the above coordinates, and creating a color difference map image having the calculated color difference as a pixel value with the coordinate midpoint between the target pixels as a coordinate value; Using the color difference map image, the initial contour is moved according to the active contour model to extract the contour line, the step of calculating the distance function based on the extracted contour line, and the calculated distance function. It features, by comparing the distance function previously created from the contour line of the shape of the target site as a reference, characterized by comprising a step of determining the shape of the target site.

The recording medium recording the model generation program of the present invention is a computer-readable recording medium recording a program for generating a model of a target object included in an image, and the position of the characteristic point of the target object. Specifying the information by the position designation unit to determine the center position of the target object on the image, placing the initial contour near the contour of the target object, and the center position and the initial position of the target object on the image. Based on the contour, the color difference between adjacent pixels on a straight line connecting the center coordinates of the target object and each coordinate on the initial contour is calculated, and the calculated color difference is calculated using the coordinate midpoint between the target pixels as the coordinate value. A step of creating a color difference map image having pixel values, and a step of extracting a contour line by moving the initial contour according to the active contour model using the created color difference map image. And a step of calculating a distance function based on the extracted contour line, and comparing the characteristics of the calculated distance function with a distance function created in advance from the contour line of the shape of the reference target part. According to the above, a step of determining the shape of the target part is included.

Further, according to the model generating apparatus of the present invention, the contour detecting / determining means detects the feature amount of each part of the target object based on the one or more position information designated by the position designating section. Further including a feature amount extraction detection stage,
The model generation means includes means for deforming the structure of the selected model based on the detected feature amount of each part of the target object.

According to the apparatus, method and recording medium of the present invention having such features, the feature amount of each part of the target object designated by the operator by the position designation unit, for example, each part of the face, for example, the eyes and mouth. The size of each part such as the size and the distance between the parts such as the distance between eyes and the face width are detected. Also, in the same way as above, the model selected by the jaw shape is
It is transformed based on the detected feature amount. That is, even if the same egg-shaped model is selected depending on the jaw shape, it is possible to generate a model that makes the best use of individual characteristics, for example, according to individual differences such as a person with a wide face or a person with a narrow face. Become.

[0040]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. Here, in order to facilitate the description, the target object will be described by focusing on the shape of the face.

FIG. 1 is a system configuration diagram showing an embodiment of a model generation device of the present invention.

The model generation device of this embodiment includes a storage unit 1, an image input unit 2, a user input unit 3, a display unit 4, and a model generation unit (a three-dimensional model generation unit in this embodiment) 5.
It is composed by.

The storage unit 1 stores in advance various data including a schematic shape model of a target object in an image.

As the image input section 2, various input means such as a digital still camera, a scanner and a video capture can be used, but in the present embodiment, not only such a specific input means, , Any still image input means can be used.

The user input section 3 is a block for inputting various kinds of information by the operation of the operator, and here, for example, a pointing device such as a keyboard or a mouse is used, but various other inputs are also made. Method is possible.

The display unit 4 is composed of a CRT, a liquid crystal display or the like, and is used for displaying the generated three-dimensional model and displaying selection candidates when the operator makes selections.

The three-dimensional model generation unit 5 is composed of a CPU 51 which is an arithmetic processing unit, a ROM 52 which stores a three-dimensional model generation program, and a RAM 53 which is used as a work area when the program operates, and is stored in the ROM 52. According to the three-dimensional model generation program, various calculation processes are performed based on the image information input from the image input unit 2, various information input from the user input unit 3, various data stored in the storage unit 1, and the like. This is a block for generating a three-dimensional model.

When the three-dimensional model generation unit 5 is functionally divided, a means for detecting the size, position and direction of the target object in the input image and a schematic shape model stored in the storage unit 1 are provided. , Calculating the position on the image of each vertex when projected on the same detected size, position and direction 1st
Calculation means, second calculation means for calculating the position on the image of each vertex when the same schematic shape model is projected in a predetermined size, position, and direction, and these first calculation means and By deforming the image based on the correspondence between the coordinates of the same vertex on both images calculated by the second calculating means, the target object can be seen from a predetermined size, position and direction. It is configured to include a means for generating an approximate image and a means for generating a textured three-dimensional model as texture map data for pasting the approximate image on the schematic shape model.
In addition, in the vicinity of the contour and // when the schematic shape model is projected in a predetermined size, position and direction defined in advance.
Alternatively, it further includes means for generating an approximate image when the target object is viewed from a predetermined size, position, and direction by filling the area corresponding to the outside of the contour with the pixel value inside the contour. It is composed.

FIG. 2 is a flow chart showing a three-dimensional model generation operation (processing operation of the three-dimensional model generation program stored in the ROM 52) by the model generation device having the above configuration. The following description will be given along this flowchart.

The operator first inputs a face image from the image input section 2 (step S1). The input face image is displayed on the display unit 4. FIG. 3A shows an example of the input image (face image) 301 displayed on the display unit 4.

Next, the operator designates coordinates for the face image 301 (step S2). That is, the coordinates of the eyes, nose, mouth, points on the contour, etc. of the displayed face image 301 are input. FIG. 4 shows an example of coordinate designation, and a cross on the input image 301 is a point whose coordinate is designated by the operator. In this embodiment, a total of 7 points are input, including both eyes, nose, mouth, the tip of the chin on the contour, and the vicinity of the left and right ears.

Next, the size, position and direction of the face are estimated (detected) from the face coordinate information thus input (step S3). Although various methods are conceivable for the method of estimating the size, position, and direction of the face, in the present embodiment, the size of the standard face shape model that is estimated temporarily,
Project on the image in position and direction. Then, the positional shift between the projected coordinates of the eyes, nose, and mouth on the standard face shape model and the point (eye, nose, mouth point) designated by the operator is calculated. Next, the projected size, position, and direction are projected to a position and direction that are slightly deviated from the estimated size, position, and direction, and in the same manner, the projected coordinates of the eyes, nose, and mouth on the standard face shape model and the operator's specified coordinates. Calculate the displacement between the points (eye, nose, mouth points). Then, the size, position, and direction that minimize this deviation are obtained, and the above process is repeated with this as the temporarily estimated size, position, and direction. By repeating such processing, finally, the size, position, and direction when the shift is no longer smaller are set as the size, position, and direction of the face of the input image.

When the size, position and direction of the face are estimated in this way in step S3, standard face shape data is then projected onto the image to generate a texture image (step S4). That is, the standard face shape data is projected on the screen with the size, position and direction of the face estimated in step S3, and the coordinates of all the vertices of the standard face shape data are calculated. A three-dimensional model denoted by reference numeral 303 in FIG. 3B schematically represents the projection result of the standard face shape data according to the estimated position and direction. At the same time, the standard face shape data is projected in parallel from the front on the texture image at a predetermined size and position, and the coordinates of all the vertices of the standard face shape data are calculated. A three-dimensional model indicated by reference numeral 304 in FIG. 3D is a schematic representation of the result of projecting the standard face shape model from the front with this predetermined size and position. That is, it is assumed that the projected coordinates of a certain vertex on the face image correspond to the projected coordinates of the same vertex on the texture image, and based on this correspondence information,
A face image is transferred onto a texture image using an image transformation method. That is, the figure represented by FIG. 3B is transferred while being transformed into the figure represented by FIG. 3D. The image transformation method used here is a well-known technique in the field of image processing, and can be easily realized by a person skilled in the art. Therefore, the description of a more specific method of image transformation is omitted here.

In this case, since the color of the texture near the contour may become unstable depending on the situation, the color near the contour is painted with the color of the object so that the color near the contour becomes stable (step S4). -1). FIG. 5 is a diagram schematically illustrating this operation. That is, the area 502 is set slightly closer to the center than the contour (indicated by reference numeral 501) of the standard face shape data, the pixel values of this area 502 are averaged, and the standard face shape data is set to the average of the pixel values. Pixels near the contour and pixels outside thereof (area indicated by reference numeral 503) are filled. The result of the transfer and the filling is an image that is similar to the pixels obtained by photographing the target object from the front such that the target object has a predetermined size and position. This image is used as a texture image to be attached to the standard face shape data by parallel projection from the front. FIG. 3C shows an input image 301 (FIG. 3A).
An example of the texture image 302 created from (see) is shown.

Next, a model is selected from a plurality of face shape models (rough shape models) prepared. In the present embodiment, first, the face shape model is automatically selected in the automatic face shape model selection step, and then the face shape model is selected by the selection instruction input by the operator (step S5).
-1). In this case, the operator may use the automatically selected face shape model as it is, but the user can select another face shape model by inputting a selection instruction from the user input unit 3. .

This automatic face shape model selection step (step S5-1) is carried out by the following method.

First, edge detection is performed in the vicinity of a portion corresponding to the outer periphery of the face which is the target object in the input image. Various methods are known for this edge detection, and those skilled in the art can easily realize this. Therefore, detailed description of the edge detection will be omitted. In addition, although various methods are conceivable for contour detection including edge detection, these methods can be easily applied to the contour detection of the present embodiment.

Then, the shape of the detected edge and
A comparison is made with the shape of the outer peripheral portion of the selectable face shape model. In the present embodiment, a comparison method is considered in which a line radially extended from the center of the face is considered and the sum of squares of the difference in distance between the edge detected on the line and the outer peripheral portion of the face shape model is obtained. FIG. 6 is a diagram schematically showing the state of this comparison. In the figure, reference numeral 601 is the detected contour, reference numeral 602 is the shape of the outer peripheral portion of the face shape model, reference numeral 603 is the center of the face, reference numeral 604 is one of the lines radially extended from the center, and reference numeral 605 is An intersection of another radially extended line and the detected contour 601, reference numeral 606 is an intersection of the same other radially extended line and the shape 602 of the outer peripheral portion of the facial shape model, and reference numeral 607 is both intersections thereof. 60
5, the distance between 606 is shown. It should be noted that although the description is omitted because the figure becomes complicated, the intersection points with the contour 601 and the intersection point with the shape 602 of the outer peripheral portion of the face shape model are obtained for all the radially extended lines, and the intersection points are obtained. There is no need to calculate the distance between them. Then, the sum of squares of the distances between all these intersections is calculated, and it is calculated with respect to all of the prepared three-dimensional face shape models that can be selected, and the calculated sum of squares becomes the minimum value. Select a 3D shape model. The above is the description of the automatic face shape model selection step. However, various comparison methods other than this method are possible, and it is easily possible to apply those methods to the face shape model selection of the present embodiment.

In the present embodiment, the face shape model (rough shape model) is used for pasting the texture image in advance so that the image projected from the front with a predetermined size and position is used as the texture image. As long as the information is added and the image is projected from the front at a predetermined size and position, a correct face 3D model can be generated regardless of the combination of any image and any model. There is. Therefore, no matter which face shape model is selected, the texture data need only be the one generated above. In other words, the operator does not need to recreate the texture image for each model or fine-tune the paste of the texture image regardless of the model selection result by the automatic model selection step, and the model can be easily selected. It has become.

FIG. 7 is a diagram schematically representing this. That is, no matter which of the plural face shape three-dimensional models (here, two examples 701 and 702 are shown) is selected, by combining with the single texture image 703, the face three-dimensional models 704, 705
Can be obtained.

Next, a hairstyle shape model is selected (step S5-2). In order to match the shape model of the hairstyle with the face shape model, the shape and position of the joint surface are created in advance for all face styles (face shape models) and hairstyles. However, unification here does not necessarily mean that they have exactly the same shape,
For example, it also includes a shape in which the joint surface is formed only by a part of the same shape. In addition, it is possible to consider other shapes that do not cause mismatch when combined, and it is easily possible to apply them to the selection of the hairstyle shape model of the present embodiment.

Since the joint surfaces are unified in advance in this way, a three-dimensional model of an arbitrary hairstyle is selected with an arbitrary three-dimensional model of a face type (face shape three-dimensional model) being selected. Even in such a case, it is possible to generate a three-dimensional model having a hairstyle on the face by simply combining the three-dimensional models without adjusting the size, position and direction.

FIG. 8 is a diagram schematically representing this. Reference numerals 801 to 803 in the drawing schematically represent a three-dimensional model of a hairstyle, and reference numerals 804 and 805 schematically represent a three-dimensional model of a face. The shapes of the joint portions (joint surfaces) of the three-dimensional models of hairstyles 801 to 803 are the same,
The shape of the joint portion of the three-dimensional model of the hairstyle of 803 is configured by the shape of a part of the joint portion of the three-dimensional models 801 and 802 of other hairstyles. Further, reference numerals 806 and 807
Is an example in which a hairstyle three-dimensional model and a face three-dimensional model are combined, reference numeral 806 is an example in which a hairstyle three-dimensional model 803 and a face three-dimensional model 804 are combined, and reference numeral 807 is a hairstyle three. This is an example in which a three-dimensional model 801 and a three-dimensional face model 805 are combined. As described above, even if a three-dimensional model of an arbitrary hairstyle and an arbitrary face is combined, the shapes of the joints of all the three-dimensional models are unified, so that no mismatch of the joints occurs. A preferred three-dimensional head model has been generated.

Finally, a lateral enlargement / reduction operation is performed on the entire head three-dimensional model generated by combining. Process of automatic selection of three-dimensional face shape model (step S
Based on the contour information in the input image acquired in 5-1), if the width of the face in the input image is wider than the standard width set in advance, the face is enlarged in the horizontal direction and When the width of the face is narrower than the standard width set in advance, the width is reduced in the horizontal direction. The operator may use such an enlargement / reduction rate as it is, or may change the enlargement / reduction rate by inputting the enlargement / reduction rate through the user input unit 3.

The texture image thus generated, the face shape model and the hairstyle shape model automatically or selected by the operator, and the head enlargement / reduction ratio automatically or selected by the operator A three-dimensional model of the unit is generated and displayed on the display unit 4 or output from an output unit (not shown) as needed. FIG. 9 shows an example of the three-dimensional head model 901 generated in this manner.

FIG. 10 is a system configuration diagram showing another embodiment of the model generating apparatus of the present invention.

In the present embodiment, the shape of the jaw is determined by detecting the contour features of the jaw of the human face included in the image input from the outside, that is, the image (input image) acquired from the outside by the apparatus. The three-dimensional model of the face corresponding to the human face in the input image is generated based on the shape determination result.

That is, the model generating apparatus according to the present embodiment has an image input unit 11 as an image acquiring unit for acquiring an electronic image, a position specifying unit 12 as a unit for specifying an arbitrary position of the input image, and an input image Contour detecting / determining means 13 for detecting the feature quantity of the jaw to determine the shape of the jaw, 3D model generating means 14 for generating a 3D model according to the feature, and output for outputting the generated 3D model to the outside. Part 15
It is composed by.

The contour detection / judgment means 13 and the three-dimensional model generation means 14 perform arithmetic processing operations thereof by the arithmetic devices 13a and 14a and the storage device (ROM, RAM, etc.) 13.
b and 14b, respectively. The three-dimensional model generation program is stored in the ROM of the storage devices 13b and 14b.
It is stored in.

Further, when the contour detecting / determining means 13 is further functionally divided, the position information of the facial feature points is given to the position specifying unit 1.
2, the means for determining the center position of the face on the image, the means for arranging the initial contour in the vicinity of the face contour, and the face center based on the center position and the initial contour of the face on the image. The color difference between adjacent pixels on the straight line connecting the coordinates and each coordinate on the initial contour is calculated, and the color difference map image having the calculated color difference as the pixel value is created with the coordinate midpoint between the target pixels as the coordinate value. By using the means and the created color difference map image, by moving the initial contour according to the active contour model,
A means for extracting a contour line, a means for calculating a distance function based on the extracted contour line, and a characteristic of the calculated distance function with a distance function previously created from a reference jaw-shaped contour line. By doing so, it is configured to include a means for determining the shape of the jaw.

FIG. 11 is a flow chart showing a three-dimensional model generation operation by the model generation device having the above configuration.
The following description will be given along this flowchart.

First, as a premise, it is assumed that the image input unit 11 stores a target object (hereinafter, referred to as an original image in this embodiment) in the storage devices 13b and 14b.

In this state, the operator first specifies the position information of the feature points (eyes, mouth, etc.) of the face by the position specifying unit 12, and determines the center position of the face on the original image (step S21). ). The center position of the face may be directly designated by the operator, or as shown in FIG. 12, the center coordinates of the right eye, the left eye, and the mouth are designated (reference numerals 121, 122, 123 in the figure).
Then, the center may be calculated as the center position of the face.

After the center position of the face is determined in this way, the initial contour is then arranged near the face contour (step S22). In the initial arrangement, for example, a closed area that surrounds the eye and mouth areas is used as the initial contour. That is, the relative distance between the eyes and the mouth may be statistically checked in advance, and the elliptical contour line surrounding the eyes and the mouth may be arranged. An image in which the center position 131 and the initial contour 132 are thus determined is shown in FIG.

Next, based on the original image thus determined, the center position 131, and the initial contour 132, the color difference between adjacent pixels on the straight line connecting the face center coordinates and each coordinate on the initial contour is calculated. An image (color difference map image) having the calculated color difference as the pixel value is created by using the coordinate midpoint between the target pixels as the coordinate value (step S23).

Here, as a method of calculating the color difference, for example, the difference value is calculated by subtracting the luminance value of each monochromatic light of the image data between pixels, and the difference calculated for each monochromatic light is calculated. The total value is calculated as the color difference. As another method, for example, the pixel data is converted from the luminance value of each monochromatic light into an HSV value represented by hue (H), saturation (S), and luminance (V), and a color difference is obtained 2.
The position of the pixel in the HSV space may be obtained, and the distance value between them may be used as the color difference. Further, instead of between the adjacent pixels, for example, the average color may be obtained for each of a plurality of consecutive pixel units, and the color difference between the average colors may be obtained.

When the color difference is calculated, the detection accuracy of the color difference may be changed by utilizing the fact that the target is a human face.
For example, when the pixel value of two pixels to be compared when calculating the color difference has a value close to the pixel value representing the skin color, it is considered that the pixels at the two points are likely to be pixels within the face contour, and It is possible to reduce the detection accuracy and reduce the influence of noise and the like. On the other hand, both the chin and the neck are highly likely to have a pixel value representing the skin color, and it is better to increase the detection accuracy when detecting the chin boundary which is the boundary between them. Therefore, in the color difference detection on the straight line from the center to the neck direction, the detection accuracy of the color difference is increased to facilitate the detection of the jaw boundary. If the coordinates of the mouth are known, the position of the neck can be estimated from the coordinates.

For example, the face center 131 as shown in FIG.
And a straight line 142 connecting the coordinate point 141 on the initial contour 132
FIG. 15 shows a schematic diagram for obtaining the above color difference. Figure 15
In, the values indicated by reference numeral 151 indicate the arrangement of pixel values on a straight line, and the value indicated by reference numeral 152 indicates the difference in pixel value between two adjacent points. That is, in this example, reference numeral 1
The array indicated by 52 indicates the arrangement of color differences.

Further, after detecting the color difference in this way, a color difference map image more specialized in the face contour shape may be created by further utilizing the characteristic features unique to the human face contour.
For example, assuming that the face has a similar shape of an ellipse,
As shown in (a) and (b), the color difference between one point on the elliptic curves 161 to 163 of arbitrary size centered on the face center and two points adjacent thereto (total of 3 points). Show)
May be averaged and stored again as the color difference of the coordinates to suppress the influence of noise. FIG.
In (b), since the color differences at the three points are 32, 28, and 34, respectively, they are averaged to 32 [(32 + 28 + 3
4) /3=31.33, (rounded up)].

As described above, by using the human face as a constraint condition, the jaw shape feature is specialized even for an input image or a noisy image in which a clear contour line does not appear. It is possible to create a stable color difference map image.

Next, using the color difference map image thus created, the initial contour is moved according to the active contour model to extract (detect) the contour line (step S24).

Here, as the energy function E, for example, the sum E = E1 + E2 + E3 of the internal energy E1 representing the smoothness of the contour line, the energy E2 for contracting the contour, and the image energy E3 characterizing the contour of the object is obtained. Move the contour so as to minimize. Note that the Snake method is used here as a method for performing contour extraction using the active contour model. Snake
The method is described in Reference 2 (M. Kass. “Snakes: A
ctiveContourModels ", Int.
J. Comput. Vision, p. 321, 198
8).

Here, the color difference map image created in step S23 is used for the image energy E3. That is, the image energy E3 (P) at an arbitrary P (x, y) on the image is obtained from the following equation (1) when the color difference value on the color difference map image corresponding to P is D (P). .

E3 (P) = α × (MAX (D) −D (P)) (1) where MAX (D): maximum value of color difference in the color difference map image, coefficient α: energy function E Is the degree of image energy at.

According to this equation (1), the image energy increases as the color difference decreases, so that the contour easily moves in the direction in which the energy is minimized. Conversely, the larger the color difference, the smaller the image energy, and the contour becomes harder to move from there. That is, since the image energy is low at the boundary between the color regions of the object such as the boundary between the face region and the background region, the contour is likely to converge at the boundary between such regions.

As described above, by using the color difference map image obtained in step S23 as the image energy, an energy image in which the features of the jaw shape are added can be created, and an input image in which a clear contour line does not appear It is possible to perform more stable jaw detection even for images with a lot of noise.

Next, a distance function is calculated based on the contour line thus obtained (step S25). That is, the contour line is a known coordinate inside the face, for example, the distance r from the center of the face.
And a direction (angle) θ are represented as a function r = L (θ). FIG. 17 schematically shows this state.

L (θ) may be calculated as r when the value of θ is changed by a unit angle. For example, the range in which the jaw shape is more prominent (the direction of the neck when viewed from the face center) is expressed in units. The angle may be narrowed and the amount of information may be increased as compared with other directions. Further, the distance function may be expressed as a Fourier descriptor represented by the following expression (2), for example.

[0089] Here, A (n): a coefficient representing the curve shape, exp ():
The power of the base of natural logarithm, s: distance on curve, L: total length of closed curve. Details regarding the Fourier descriptor are disclosed in, for example, Reference Document 3 (edited by Mikio Takagi and Yohisa Shimoda, “Image Analysis Handbook”, The University of Tokyo Press, 1991.).

Next, the shape of the jaw is determined by comparing the characteristics of the distance function thus obtained with the reference distance function (step S26). Here, the reference distance function is a distance function created in advance from the reference contour line of the jaw shape. For the contour line of the jaw shape that serves as a reference, an image in which the contour line has been manually detected in advance is classified into similar jaw shapes, for example, base type, round shape, etc. What is averaged may be used.

For comparison of the distance functions, for example, the position of the inflection point on the distance function, the number of inflection points, the slope between the inflection points, etc. are positioned as the features of the distance function, and the distance of the reference contour shape is determined. This is done by comparing each with the characteristics of the function. Before performing the comparison, it is necessary to perform normalization so that the reference distance function and the position match.

The position and number of the inflection points and the inclination between the inflection points are obtained in advance in the case of the reference shape, and the information is stored in the memory (storage device 13b) and obtained in step S25. The information may be compared appropriately with the information of the inflection point of the distance function. Then, the shape having the reference distance function with the closest comparison result is determined as the determination result.

The distance functions can also be compared by simply comparing the sum of differences with the reference distance function. FIG. 18 schematically represents this situation. The symbol z in FIG. 18 indicates the difference from the reference distance function. Here, when the reference distance function is B (θ), the sum of differences Z1 is given by the following expression (3).

[0094] That is, the shape having B (θ) that minimizes Z1 may be determined as the determination shape. In the case of this method, B (θ) for the range of θ is stored in the memory (storage device 13
Although it is necessary to prepare in b), there is an advantage that more detailed shape classification and determination can be easily performed.

If a distance function is expressed using a method of describing a curved line on a plane in the frequency domain, for example, a Fourier descriptor, the Fourier coefficient calculated by this can be positioned as a feature of the distance function. Therefore, by comparing this Fourier coefficient with the Fourier coefficient of the distance function of the reference contour shape, the shape determination can be performed in the same manner as described above.

The reference distance function is represented by a Fourier descriptor,
When the Fourier coefficient is Ab (n), the difference Z2 from the Fourier coefficient of the target distance function is obtained by the following equation (4), and the shape having Ab (n) that minimizes the obtained Z2 is set as the determination shape. decide.

[0097] In general, the lower order term of the Fourier coefficient reflects a rough curve shape, and the higher order term reflects a more detailed curve shape. Therefore, a comparison of lower order terms, ie, equation (4) above
By determining the value of Z2 by reducing the range of n in n, it is possible to obtain a determination result in which influences such as noise and individual differences are eliminated as much as possible.

The above-described processing operation (steps S21 to S26) is performed by the feature detecting / determining means 13. The jaw shape result obtained based on the feature points is sent to the three-dimensional model generation means 14.

The three-dimensional model generating means 14 calculates the 3D model based on the jaw shape determined by the feature detecting / determining means 13.
A dimensional model is determined and a desired three-dimensional model is generated (step S27).

That is, the structural information of the three-dimensional model created by a professional designer or the like according to the type of jaw shape is stored in the storage device 14b in advance, and the three-dimensional model of the face corresponding to the determined jaw shape is stored. Structure information storage device 1
It may be read from 4b and regenerated. The result is C
It is displayed by the output unit 15 such as RT. Here, when the face corresponding to the determined jaw shape is an egg shape, a round shape, and a square shape, an example of generating a three-dimensional model corresponding to the respective shape results is shown in FIG. In FIG. 19, the three-dimensional model indicated by reference numeral 191 is an egg shape, the three-dimensional model indicated by reference numeral 192 is a round shape, and the three-dimensional model indicated by reference numeral 193 is a square shape.

By the above processing operation, the 2
It is possible to detect a more stable jaw shape from the dimension information and determine the shape thereof, and generate a three-dimensional model based on the determination result.

By the way, as described above, even when the same egg-shaped three-dimensional model is selected depending on the jaw shape, there are individual differences such as a person with a wide face and a person with a narrow face. Therefore, in consideration of such individual differences, when the 3D model is generated, the basic 3D model structure prepared in advance is modified by utilizing the separately obtained feature amount of each part of the face. It is conceivable to generate a dimensional model.

In order to do so, the feature detection / determination means 13 shown in FIG. 10 is used not only for detecting the features of the jaw shape but also for each face part such as the distance between both eyes and the distance between the eyes and the mouth. A function for detecting the distance between the two and the distance ratio is added, and the detection result is detected by the three-dimensional model generation means 14 to
It may be used when the dimensional model is generated. In this case, the three-dimensional model generation means 14 generates the three-dimensional model when
At the same time, it is necessary to add a function so that the transformation process can be performed.

That is, the distance between each part is calculated based on the position information of each part designated by the operator in the position designation section 12. Here, in the human face, since the positional relationship and size of each face part are subject to certain constraint conditions, each part such as the positional relationship between both eyes and the mouth, the ratio relationship between the distance between both eyes and the face width, etc. The positional relationship of can be estimated within a certain range. In other words, the positional relationship of each part peculiar to these faces is measured in advance for a plurality of human faces (for example, the distance between both eyes and the face width), and if the average or variance of the ratios is obtained, the face width, etc. It is possible to estimate from the distance between both eyes even if the operator does not input the information. As a result, it is possible to reduce the burden on the operator when detecting the feature amount of each part of the face.

The three-dimensional model of the face is transformed based on the feature amount of each part of the face thus detected. The basic model of the face to be deformed is determined based on the above-mentioned determination result of the jaw shape.

Here, the case where the ratio of the distance between the parts, for example, the ratio of the distance between both eyes and the distance between the eye openings (aspect ratio) is used as the feature amount of each part of the face will be described.

The average value of the ratios obtained in advance for a plurality of human faces is compared with the ratio obtained from the human faces in the input image, and the entire three-dimensional model of the face is enlarged or reduced according to the ratio.

For example, from the human face in the input image, as shown in FIG.
In the case where the three-dimensional model indicated by reference numeral 201 is selected as the reference three-dimensional model, a modification example in which the aspect ratio obtained from the feature amount is longer in the vertical direction than the average ratio is indicated by reference numeral 202 in FIG. A modification in the case where the horizontal direction is long is shown by reference numeral 203 in the figure.

By the above processing operation, the reference three-dimensional model can be deformed based on the position information of each part of the face designated by the operator by the position designation unit 12. As a result, even when the same three-dimensional model is selected, it is possible to easily generate a three-dimensional model that makes the most of the characteristics of the person according to individual differences such as a person with a wide face and a person with a narrow face. Become.

As described above, the model generation device of each of the above-described embodiments realizes the model generation process of the device by the model generation program stored in the storage unit 1 and the storage devices 13b and 14b. This program is stored in a computer-readable recording medium. In the present invention, although not shown, this recording medium may be a program medium that is provided with a program reading device inside the model generating device and can be read by inserting the recording medium into the model generating device, or It may be stored in a storage means such as a program memory inside the apparatus as in the embodiment. In any case, the stored program may be directly accessed and executed, or in any case, the program is read and the read program is downloaded to a main memory (not shown). The program may be executed by executing the program. It is assumed that this download program is stored in the apparatus body in advance.

Here, the program medium is a recording medium which can be separated from the main body, and is a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy disk or a hard disk, a CD-ROM, an MO,
Disc system of optical discs such as MD and DVD, card system such as IC card and optical card, mask ROM, EP
It may be a medium that fixedly carries the program, including a semiconductor memory such as a ROM, an EEPROM, and a flash ROM.

Further, in the present invention, when a means capable of communicating with the outside (a wireless communication function or a wired communication function via a public line such as the Internet) is provided, this is used to connect to the outside. The medium may be a medium that fluidly carries the program so that the program can be downloaded from the medium. When the program is downloaded from the communication network in this manner, the download program may be stored in the apparatus main body in advance, or may be installed from another recording medium.

[0113]

According to the model generating apparatus, the model generating method, and the recording medium recording the model generating program of the present invention, the following various effects are achieved.

That is, even when an image taken from a direction other than a predetermined direction is used as an input image, it is treated in the same manner as an image input from a predetermined direction, and a three-dimensional model is easily generated. be able to.

Further, even when an image taken from a direction other than the predetermined direction is used as the input image, the model can be generated in the same manner as the image input from the predetermined direction, and Even if there is a difference between the rough shape and the shape of the target object, it is possible to prevent mixing of colors other than the target object in the vicinity of the contour of the generated model and generate a higher quality model.

Further, by using images from a predetermined direction, it is only necessary to prepare a single pasted image data for a plurality of models, so that the time and effort required to generate the models are greatly reduced. To be done.

Regarding the creation of the pasted image data,
Since it is possible to use images taken from directions other than the predetermined direction, it is possible to create models from various images. Further, since the combinations of the shape models are created so as to be matched in advance, it is not necessary for the operator to perform adjustment or matching, so that the burden on the operator is greatly reduced. Further, regarding the combination of shape models, when preparing a shape model to be combined, it is possible to create and prepare a model from various images, which greatly reduces the trouble of creating and preparing the model. . Further, since a model having a shape close to the shape of the target object on the input image is automatically selected, it is not necessary for the operator to select a shape model, which reduces the burden on the operator.

Further, the operator can more stably detect the chin and determine the shape of the chin from a two-dimensional face image such as a photographed image only by inputting a few feature points such as the eyes and mouth of the face. ,
A model can be easily generated based on the determination result. Further, the operator can easily generate a model that takes advantage of the features of the face image from a two-dimensional face image such as a photographed image by simply inputting several feature points such as the eyes and mouth of the face. You can

[Brief description of drawings]

FIG. 1 is a system configuration diagram showing an embodiment of a model generation device of the present invention.

FIG. 2 is a flowchart showing a three-dimensional model generation operation by the model generation device of the present invention.

3A is an explanatory diagram showing an example of an image input from an image input unit, FIG. 3B is a diagram schematically showing a projection result of standard face shape data according to an estimated position and direction, and FIG. Is an illustration of the texture image generated from the input image,
FIG. 7D is a diagram schematically showing the result of projecting the standard face shape model from the front with a predetermined size and position.

FIG. 4 is an explanatory diagram showing an example of specifying coordinates of an input image.

FIG. 5A is an explanatory diagram showing an example of an area for obtaining a pixel value near the contour, and FIG. 5B is an explanatory diagram showing an example of an area for painting a color near the contour.

FIG. 6 is an explanatory diagram schematically showing how contours are compared.

FIG. 7 is an explanatory diagram schematically showing how a single texture image is applied to a plurality of schematic face shape models to generate a three-dimensional model.

FIG. 8 is an explanatory diagram schematically showing a method that enables combination of arbitrary three-dimensional models of a face and a hairstyle.

FIG. 9 is an explanatory diagram illustrating an example of a generated three-dimensional model of three heads.

FIG. 10 is a system configuration diagram showing another embodiment of the model generation device of the present invention.

FIG. 11 is a flowchart showing a three-dimensional model generation operation in another embodiment.

FIG. 12 is an explanatory diagram showing an example of position designation.

FIG. 13 is an explanatory diagram showing the arrangement of center coordinates and initial contours in the input image.

FIG. 14 is a diagram for explaining a method of calculating a color difference on a straight line connecting a point on the initial contour and a center coordinate.

FIG. 15 is an explanatory diagram schematically showing an example of calculation of color difference.

16 (a) and 16 (b) are diagrams for explaining a case in which the fact that the face has an elliptical shape is used as a method for performing color difference calculation specialized for the face contour shape.

FIG. 17 is a diagram for explaining a method of calculating a distance function from the extracted face contour line.

FIG. 18 is a diagram illustrating a method of comparing a distance function obtained from an input image with a reference distance function.

FIG. 19 is an explanatory diagram showing an example of a three-dimensional model of a face with respect to a determination result of a jaw shape.

FIG. 20 is an explanatory diagram showing an example in which a three-dimensional model is modified based on a feature amount of a face.

[Explanation of symbols]

1 memory 2 Image input section 3 User input section 4 Display 5 3D model generator 11 Image input section 12 Positioning part 13 Feature detection / determination means 14 Three-dimensional model generation means 13a, 14a arithmetic unit 13b, 14b storage device 15 Output section

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 17/40 G06T 17/40 B (72) Inventor Kazuhiro Saeki 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Inside the Sharp Corporation (72) Toshiya Takahashi 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Prefecture Inside the Sharp Corporation (72) Kyoichi Suzuki 22-22 Nagaike-cho, Abeno-ku, Osaka-shi, Osaka Prefecture Inner F-term (reference) 5B050 AA09 BA12 CA07 DA07 EA06 EA07 EA13 EA19 EA27 EA28 EA30 FA09 5B057 AA20 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB13 CB16 CD11 CD14 CE08 DA07 A09 DC02 DC06 A06 DC16 A06 DC25 A16 DC25 A0 DC16 A0 A6 A0 DC12 A0 DC12 A0 FA67 FA69 GA41 JA11

Claims (19)

[Claims] 1. A device for generating a model of a target object included in an image, including a position specifying unit capable of specifying an arbitrary position in the image, and a schematic shape model of the target object in the image. A storage unit that stores various data in advance, and a model generation unit that generates a model based on the image information, the position information designated by the position designation unit, and the various data stored in the storage unit, The model generation unit projects the size, position and direction of the target object in the input image and the rough shape model stored in the storage unit onto the same detected size, position and direction. The first calculation means for calculating the position of each vertex on the image in the case of
Second calculating means for calculating the position of each vertex on the image when projected in the position and direction, and the coordinates of the same vertex on both images calculated by the first calculating means and the second calculating means. A model generation device comprising: a unit configured to generate an approximate image when the target object is viewed from a predetermined size, position, and direction by deforming the image based on the correspondence relationship. . 2. A means for deforming an image based on a correspondence relationship between coordinates of the same vertex on both images calculated by the first calculating means and the second calculating means, and the general shape model is determined in advance. Stipulated size,
The target object is viewed from a predetermined size, position and direction by filling the area corresponding to the vicinity of the contour and / or the area outside the contour when projected in the position and direction with the pixel value inside the contour. The model generation device according to claim 1, further comprising: a means for generating an approximate image in the case. 3. The model generation device according to claim 1, further comprising means for generating a model with an image as pasted image data for pasting the approximate image onto a rough shape model. 4. The storage unit stores a plurality of schematic shape models in which image pasting rules are fixed in advance, and the model generating unit conforms to the preset image pasting rules. 2. An image is created in step 1, one model is selected from a plurality of prepared rough shape models, and a model with an image is generated by combining the selected rough shape model and the created image. The model generation device according to claim 3. 5. An apparatus for generating a model of a target object by using a plurality of models, which are component parts of a target object created in advance, in combination according to a predetermined rule, and the size of the parts to be combined. A model generation device, characterized in that the position, the direction, and the direction are predetermined, and the shapes of the joints to be combined are made uniform in advance. 6. The model generation device according to claim 5, further comprising means for enlarging, reducing, rotating, moving, etc., the models generated by the combination as necessary. 7. A device for generating a model of a target object included in an image, the position specifying unit capable of specifying each part of the target object included in the image, and the image information and the position specifying unit. And a model generation unit that generates a model of the target object based on the determined position information, wherein the model generation unit determines the contour feature of the target region based on the one or more position information designated by the position designation unit. Contour detection / determination means for detecting and determining the shape of the target object from the detected contour characteristics of the target region, and model generation means for selecting a model of the structure corresponding to the determination result and generating a desired model. A model generation device comprising: 8. The contour detecting / determining means designates the position information of the feature points of the target object by the position designating section to determine the center position of the target object on the image, and Based on the means for arranging the initial contour in the vicinity and the center position and the initial contour of the target object on the image,
A color difference map that calculates the color difference between adjacent pixels on a straight line connecting the center coordinates of the target object and each coordinate on the initial contour, and uses the calculated color difference as the pixel value, with the coordinate midpoint between the target pixels as the coordinate value. A means for creating an image, a means for extracting the contour line by moving the initial contour according to the active contour model using the created color difference map image, and a distance function is calculated based on the extracted contour line. And a means for determining the shape of the target part by comparing the calculated feature of the distance function with a distance function created in advance from the contour line of the shape of the target part serving as a reference. The model generation device according to claim 7. 9. The contour detecting / determining means further includes a feature quantity extraction detecting stage for detecting a feature quantity of each part of the target object based on one or more pieces of position information designated by the position designating unit, 9. The model generation device according to claim 8, wherein the model generation unit includes a unit that deforms the structure of the selected model based on the detected feature amount of each part of the target object. 10. A model generation method by a model generation device for generating a model of a target object included in an image, the method comprising: detecting the size, position and direction of the target object in the image, The calculation step of calculating the position on the image of each vertex when projecting the rough shape model having the same detected size, position and direction, and the prescribed size predetermined for the same rough shape model,
By calculating the position of each vertex on the image when projected in the position and direction, and transforming the image based on the correspondence between the coordinates of the same vertex on both images calculated by these calculation steps. And a step of generating an approximate image when the target object is viewed from a predetermined size, position and direction defined in advance. 11. A step of deforming an image based on a correspondence relationship between coordinates of the same vertex on both images calculated by each of the calculation steps, and a predetermined size for predetermining the schematic shape model,
The target object is viewed from a predetermined size, position and direction by filling the area corresponding to the vicinity of the contour and / or the area outside the contour when projected in the position and direction with the pixel value inside the contour. The method according to claim 10, further comprising the step of generating an approximate image of the case. 12. The method according to claim 1, further comprising the step of generating a model with an image as pasted image data for pasting the approximate image onto the general shape model.
0 or the model generation method according to claim 11. 13. A method for generating a model of a target object included in an image, comprising the step of designating each part of the target object by a position designating unit, and the one or more position information designated by the position designating unit. Detecting the contour feature of the target region based on the step, and determining the shape of the target region from the detected contour feature of the target region; and selecting a structure model corresponding to the determination result to generate a desired model. And a step of creating a model. 14. A method of generating a model of a target object included in an image, comprising: specifying position information of feature points of the target object by a position specifying unit to determine a center position of the target object on the image. And a step of placing an initial contour in the vicinity of the contour of the target object, and based on the center position of the target object on the image and the initial contour,
A color difference map that calculates the color difference between adjacent pixels on a straight line connecting the center coordinates of the target object and each coordinate on the initial contour, and sets the calculated color difference as the pixel value with the coordinate midpoint between the target pixels as the coordinate value. The step of creating an image, the step of extracting the contour line by moving the initial contour according to the active contour model using the created color difference map image, and calculating the distance function based on the extracted contour line Steps,
Comparing the characteristic of the calculated distance function with a distance function prepared in advance from the contour line of the shape of the reference target region, and determining the shape of the target region. Model generation method. 15. A computer-readable recording medium recording a program for generating a model of a target object included in an image, the step of detecting the size, position, and direction of the target object in the image, and a storage unit. The calculation step for calculating the position on the image of each vertex when the schematic shape model stored in is projected in the same detected size, position and direction, and the same schematic shape model with a predetermined size ,
By calculating the position of each vertex on the image when projected in the position and direction, and transforming the image based on the correspondence between the coordinates of the same vertex on both images calculated by these calculation steps. And a step of generating an approximate image when the target object is viewed from a predetermined size, position and direction defined in advance, the recording medium having a model generation program recorded therein. 16. A step of deforming an image based on a correspondence relationship between coordinates of the same vertex on both images calculated by each of the calculation steps, and a predetermined size of the rough shape model set in advance,
By filling the area near the contour and / or the area corresponding to the outside of the contour when projected in the position and direction with the pixel value inside the contour, the target object is viewed from a predetermined size, position, and direction determined in advance. A recording medium storing the model generation program according to claim 15, further comprising a step of generating an approximate image in the case. 17. The method according to claim 15, further comprising the step of generating a model with an image as the pasted image data for pasting the approximate image onto the general shape model.
A recording medium recording the model generation program according to claim 16. 18. A computer-readable recording medium in which a program for generating a model of a target object included in an image is recorded, the step of designating each part of the target object by a position designating unit, and designating by the position designating unit. Detecting contour features of the target portion based on the one or more pieces of positional information obtained, and determining the shape of the target portion from the detected contour features of the target portion; and selecting a model of the structure corresponding to this determination result. And a step of generating a desired model, the recording medium having a model generation program recorded therein. 19. A computer-readable recording medium in which a program for generating a model of a target object included in an image is recorded, wherein position information of feature points of the target object is specified by a position specifying unit, Based on the step of determining the center position of the target object, the step of placing the initial contour in the vicinity of the contour of the target object, and the center position and the initial contour of the target object on the image,
A color difference map that calculates the color difference between adjacent pixels on a straight line connecting the center coordinates of the target object and each coordinate on the initial contour, and sets the calculated color difference as the pixel value with the coordinate midpoint between the target pixels as the coordinate value. The step of creating an image, the step of extracting the contour line by moving the initial contour according to the active contour model using the created color difference map image, and calculating the distance function based on the extracted contour line And a step of determining the shape of the target region by comparing the calculated feature of the distance function with a distance function created in advance from the contour line of the shape of the target region serving as a reference. A recording medium on which the model generation program is recorded.