JP4424111B2 - Model creation device and data distribution system - Google Patents

Description

  The present invention relates to a model creation device and a data distribution system, and in particular, the model creation device of the present invention relates to a device that deforms an image obtained from a human or an animal and obtains a deformed model corresponding to the image. The data distribution system of the present invention also relates to a model distribution system that receives order data, deforms a reference model corresponding to the order, and provides a reference deformation model created at the order transmission source.

  Currently, a service for creating a three-dimensional computer graphics (CG) model that takes advantage of the user's personality for use in communication tools and computer games has entered the practical stage. Specific examples of these services include Non-Patent Document 1 and Non-Patent Document 2.

In these services, the user transmits an image owned by the user, for example, his / her face image, to the service site, and the user designates the position of the face part that is a feature of the face image. The service site creates a model that takes advantage of the user's personality by deforming the model according to this feature or pasting this face image on the model, and providing it to the user who sent the model is doing.
JP2003-281539 Chobit3D, http://www.spacetown.ne.jp/mebius/lib/t-bit3d/chobi.html, as of March 2, 2004 3DMeNow, http://www.biovirtual.com

  However, in order to make full use of the individuality, it is desirable that the features of the facial parts be specified in detail. Thus, in order to reflect the individuality, there is a problem that a great burden is imposed on the user. On the other hand, if a known technique is used for extracting facial parts, it is possible to automatically create a model without user designation, but it often depends on the extraction accuracy, and the shooting conditions become severe. Automation of model creation has not reached a practical level at present.

  In addition, there are various demands for users, such as when it is desired to acquire a model quickly at a low fee or when it is desired to acquire a highly accurate model even at a high fee. In the former request, the information is often displayed only on a small screen of an information device such as a mobile phone, so that the extraction accuracy is low. A user who requests the latter includes a personal computer. The services described above are required to meet all these user requests.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a model creation apparatus and a data distribution system that can solve such drawbacks of the prior art and can respond to user requests even if model creation is automated.

  In order to solve the above-described problems, the present invention provides an image storage means for temporarily storing a supplied face image, and the number of feature points to be extracted for each characteristic face part included in the face image. Feature extraction means for extracting more than the number of models, model storage means for storing a model used as a reference, model deformation means for deforming a reference model based on a plurality of extracted feature points, and a coordinate system for displaying a face image Adjustment means for adjusting the coordinate system for displaying the reference model and a coordinate system for displaying the reference model, and image pasting means for pasting the adjusted face image to the deformed reference model.

  The model creation apparatus of the present invention supplies the face image stored in the image storage means to the feature extraction means, and extracts more feature points in each part by the feature extraction means than the conventional number, so that the model deformation means A model obtained by deforming the reference data based on the extracted feature points and the reference data supplied from the model storage unit is converted into a model having higher accuracy than the conventional model, and is supplied to the image pasting unit. By supplying the face image adjusted in the coordinate system to the coordinate system of the reference model to the image pasting means, and pasting the face image supplied to the model supplied by the image pasting means to create a reference deformation model without deviation Therefore, it is possible to automatically create a reference deformation model in accordance with the face image without performing a complicated part extraction operation by the user.

  In order to solve the above-mentioned problems, the present invention transmits a face image and a reference model as order data, and receives an order data and an information terminal device that receives reference data transformed according to the order data. In the data distribution system in which the reference data modified according to the order data is managed and the server device that transmits the deformed reference data to the ordered information terminal device is connected via the network, the server device Image storage means for temporarily storing the supplied face image; and feature extraction means for extracting the number of feature points to be extracted for each of the characteristic face parts included in the face image more than the conventional number; Model storage means for storing a model to be used as a reference, model deformation means for deforming the reference model based on a plurality of extracted feature points, and a position for displaying a face image An image adjusting unit that performs adjustment to match a system and a coordinate system that displays a reference model, and an image pasting unit that pastes the adjusted face image on the deformed reference model are included as a model creation device. .

  In the data distribution system of the present invention, order data from an information terminal device is supplied to a server device via a network, and a face image stored in an image storage unit by a model creation device included in the server device is supplied to a feature extraction unit. Then, by extracting more feature points in each part with this feature extraction means than the conventional number, the reference data is deformed based on the feature points extracted by the model deformation means and the reference data supplied from the model storage means. The model having a higher accuracy than the conventional model is supplied to the image pasting means, and the image adjusting means supplies the face image in which the coordinate system of the face image is adjusted to the coordinate system of the reference model to the image pasting means. Create a standard deformation model without deviation by pasting the face image supplied to the model supplied by the pasting means, and use the standard deformation model created from the server device. By supplying to the information terminal device, the user's operation burden on the information terminal device is less than before, the cost required to generate the desired reference deformation model in the server device is suppressed, and the model generation is also acquired in a short time can do.

  Next, an embodiment of a model creation device according to the present invention will be described in detail with reference to the accompanying drawings.

  In this embodiment, the model creation device of the present invention is applied to the model creation device 10. The illustration and description of parts not directly related to the present invention are omitted. In the following description, the signal is indicated by the reference number of the connecting line in which it appears.

  As shown in FIG. 1, the model creation apparatus 10 includes an image storage unit 12, a component extraction unit 14, an image adjustment unit 16, a reference model storage unit 18, a model deformation unit 20, an image pasting unit 22, and a model output unit 24. . The image storage unit 12 is a memory having a function of storing supplied image data 26. The image data 26 is an image obtained by photographing a person's face, that is, face image data. The image storage unit 12 also has a function of decompressing or expanding the compression when compressed face image data is supplied. The image storage unit 12 outputs the stored face image data 28 to the component extraction unit 14 and the image adjustment unit 16, respectively.

  The part extraction unit 14 has a function of extracting feature points of face parts from the face image, and from the face image 30 represented by the face image data 28 as shown in FIG. 2, the eyes, eyebrows, nose, mouth The position of the face part such as is extracted or sampled as the feature point 32. In this embodiment, the feature point 32 is represented by a white circle. The feature point is represented by coordinates (X, Y) from the origin by setting the origin 34 of the coordinates in the face image 30 to a position obtained by dividing the distance between both eyes into two. As a method for extracting a facial part, Patent Document 1 which is a known technique is used. When operating with high accuracy, it is desirable to extract feature points in detail at least as shown in FIG. Basically, by extracting feature points in this way, it is possible to provide high-precision data for the face image model to open and close the mouth and move the eyebrows instead of the user's face image. become. The component extraction unit 14 supplies data 36 indicating the feature points of the components on the extracted face to the model deformation unit 20.

  Returning to FIG. 1, the reference model storage unit 18 is a memory having a function of storing at least one reference model. As shown in FIG. 3, the reference data 38 in this embodiment is a face model having polygons created in advance for human use, and the position of each polygon and the model origin are specified. The model origin 40 is set at the center of both eyes of the model as in the case of extracting feature points. In this way, the face image data 28 and the reference data 38 that are supplied are associated with each other by setting the origin at the same position as the previous setting. The reference model storage unit 18 outputs the reference data 38 to the model deformation unit 20.

  In this embodiment, one model for humans is prepared and used. However, a plurality of reference models are prepared, and a method for giving which reference model to use from the outside is used. Also good. For example, even when a face image is a dog or a cat, a model other than a person can be created by preparing a reference model of a dog or a cat and selecting it.

  Referring back to FIG. 1 again, the model deforming unit 20 has a function of deforming the reference face model by changing the polygon of the reference data 38 in accordance with the extracted data 36. The model deforming unit 20 also has a function of calculating a conversion ratio R that absorbs the difference between the coordinate system of the reference model and the coordinate system of the supplied face image. The conversion ratio R is obtained from ML / IL as the distance ML between both eyes of the reference model and the distance IL between both eyes in the face image. The distance IL is obtained based on the number of pixels on the image. The model transformation unit 20 multiplies the coordinates of the extracted feature points by the transformation ratio R, and moves each feature point of the reference model corresponding to the extracted feature points to the corresponding coordinate of the obtained multiplication result.

  If the number of polygons is larger than the number of feature points, the polygon estimated to be closest to the feature points is multiplied by the deformation ratio R and moved to the obtained coordinate position, and the surrounding polygons are moved. While maintaining the positional relationship that does not break the symmetrical relationship with the polygons that have been moved, it is moved so as to maintain a range that does not affect the coordinate positions obtained for other polygons.

  As a result, a reference model in which the polygon is deformed is obtained, and therefore, the model deforming unit 20 outputs the reference deformed model 42 having the characteristics of the supplied face image to the image pasting unit 22. Further, the model deforming unit 20 supplies the deformation ratio R 44 to the image adjusting unit 16.

  The image adjustment unit 16 has a function of adjusting the supplied face image data 28 so as to match the coordinate system of the reference model, and the deformation ratio R 44 into the horizontal width W and height or vertical width H of the face image data 28. Multiply by and perform linear expansion and contraction. This is because it is desired that the supplied face image data 28 be normalized in accordance with inputs of various sizes, that is, different resolutions expressed in pixels. The image adjusting unit 16 supplies the adjusted image data 46 to the image pasting unit 22.

  The image pasting unit 22 has a texture mapping function for matching the origin of the adjusted image data 46 to the face polygon of the supplied reference deformation model 42 and pasting it. Since the size has already been adjusted, everything other than the face protrudes from the polygon. This is expressed as mapped. The protruding part is ignored when the model is displayed. As a result, this part is not visible. As a result, a deformation model similar to the face image data to which only the face of the reference model is supplied is obtained. The image pasting unit 22 supplies the image data obtained in this way to the model output unit 24 as a reference deformation model 48.

  The model output unit 24 is a memory that temporarily stores and outputs the supplied reference deformation model 48. An address of a predetermined memory in the model output unit 24 may be given from the outside, or a fixed address may be held inside the apparatus in advance. Although not shown, the model output unit 24 outputs a stored reference deformation model 50 according to the supplied timing.

Next, the operation of the model creation device 10 will be briefly described. The supplied image data 26 is temporarily stored in the image storage unit 12, and the face image data 28 is supplied to the component extraction unit 14 and the image adjustment unit 16, respectively. The component extraction unit 14 extracts facial feature points from the face image data 28 and supplies the data 36 to the model deformation unit 20. The model deformation unit 20 is also supplied with reference data 38 from the reference model storage unit 18.

  In the model deforming unit 20, a deformation ratio R between the reference model 38 and the data 36 that is the feature point of the face part is obtained, the deformation ratio R is applied to the data 36, and the reference model 38 corresponding to the resulting feature point is obtained. The reference deformation model 42 obtained by deforming the reference model is generated by replacing the position of the feature point with the obtained result.

  The reference deformation model 42 is generated by first obtaining the distance ML between the eyes of the model and the distance IL between the eyes of the face image in units of the number of pixels on the image, respectively, and R = ML / Find IL.

  Next, processing is performed to absorb the deviation between the coordinate system of the reference model and the coordinate system of the supplied face image. The data 36, which is the coordinates of each feature point of the extracted facial part, is (X1, Y1), (X2, Y2), ..., (Xn, Yn), and n represents the number of feature points. The deformation ratio R is applied to the data 36. That is, (X1 × R, Y1 × R), (X2 × R, Y2 × R),... (Xn × R, Yn × R). Each feature point of the reference model 38 and the data 36 is matched, and the polygon of the reference data is moved by replacing the coordinate value indicating the position of the feature point of the reference data 38 with the data 36 in which the deformation ratio R is applied. Let The replaced reference data is reference deformation data 42. The reference deformation data 42 is output to the image pasting unit 22. Further, the deformation ratio R 44 is output to the image adjustment unit 16.

  The image adjustment unit 16 linearly expands and contracts the face image data 28 with the conversion ratio R 44 to obtain adjusted image data 46. The adjusted image data 46 is output to the image pasting unit 22. The image pasting unit 22 texture-maps the adjusted image data 46 to the reference deformation data 42 to obtain a reference deformation model 48.

  By extracting the feature points in this way, the component extraction operation by the user can be eliminated, and the reference deformation model 48 that is the reference model matched to the face image can be automatically created. .

Next, another embodiment of the model creation apparatus 10 will be described. The model creating apparatus 10 in FIG. 4 includes a feature input unit 52, a feature determination unit 54, and a feature output unit 56 in the components shown in FIG. Each component shown in FIG. 1 is given the same reference numeral as that of the previous embodiment, and the description thereof is omitted to avoid complicated explanation. The feature input unit 52 has a function of receiving feature points of facial parts from the outside. The feature input unit 52 is an input interface for inputting the feature point table 58 from the outside and has a memory function. The feature input unit 52 outputs the feature point table 60 to the feature determination unit 54. The feature point table 60 may include edited information such as addition of unextracted points or correction of extracted points. Information editing before input is performed by an operator, that is, an editor. The editor compares the coordinates shown in the feature point table with the face image, confirms whether the image has been correctly extracted, corrects the feature point table as required, and inputs it to the feature determination unit 54.

  The feature determination unit 54 has a determination function for confirming whether or not all feature points to be deformed in the reference model have been extracted. As shown in FIG. 5, the feature point table has feature point serial numbers, coordinates, and extraction flags. The feature point serial numbers are stored in advance as serial numbers of feature points to be deformed in the reference model. The coordinate represents the position of the feature point described above with respect to the origin. In addition, the extraction flag indicates the presence / absence of a feature point matching the serial number by “1” and “0”, respectively. It should be noted that when not extracted, the initial value or the origin X = 0, Y = 0 is entered as the coordinate value.

Returning to FIG. 4, the feature determining unit 54 outputs the feature point table 62 to the feature output unit 56 when at least one of the extraction flags is “0”. In addition, the feature determination unit 54 preferably transmits the extraction result from the component extraction unit 14 to the feature output unit 56 because there is a possibility of erroneous extraction. The feature determination unit 54 may be configured to transmit only the feature point table of the editing result received from the feature input unit 52 or all the feature point tables of the extraction flag “1” to the model transformation unit 20 as the feature point table 64.

  The feature output unit 56 is an output interface that outputs the result extracted by the component extraction unit 14 to the outside, and may have a memory function. The feature output unit 56 outputs a feature point table 62 supplied to the outside of the model creating apparatus 10 as a check feature point table 66.

  Next, the operation in this embodiment will be described. When extracting feature points of facial parts from facial images, the extraction accuracy decreases if there is a bad shooting environment where the incident light is backlit or dark, a situation where the eyes and eyebrows are hidden with long hair, and a situation where the mouth is hidden with beards However, erroneous extraction may be caused, or extraction itself may be impossible.

The feature determination unit 54 inputs not only the data 36 that is the feature point table but also the feature point table 60 so as to compensate for the extraction supplied from the outside of the apparatus, and extracts all the feature points prepared in advance in the reference model. checks are, when the extraction flag of the feature point that matches the serial number "0" is at least one existing "false", and transmits the feature point table 62 in the characteristic output section 56, and outputs to the outside. At this time, the feature determination unit 54 transmits nothing to the model deformation unit 20. As a result, the reference deformation model 48 is not created, and an addition or correction process is performed on the feature point table output outside. When all the feature points used in model transformation are extracted as “true”, the feature point table 64 is sent to the model transformation unit 20 as an extraction result. Thereafter, the reference transformation model is operated by the same operation procedure as in the previous embodiment. Create

  Summarizing the function of the feature determination unit 54, the feature determination unit 54 determines whether at least one of the extracted feature points and the feature points from the feature input unit 52 is sufficient for deformation of the reference model. In response to this determination true, a feature point table is output to the model deforming unit 20, and in response to this determination false, based on at least the feature point table from the feature input unit 52 and the feature point from the component extraction unit 14. One of the generated feature point tables is output to the feature output unit 56.

  Outside the device, the coordinates shown in the feature point table are compared with the face image to check whether it has been extracted correctly, the feature point table 58 is corrected as required, and unextracted points are added to the feature point table 58 To do. The processing operation described above is performed by an editor who handles the model creating apparatus 10.

  By editing the feature points in this way, the correct feature points can be extracted and provided even if problems such as deterioration of the shooting environment occur. A reference deformation model can be reliably created. Further, since feature points are edited only for unextracted feature points and erroneously extracted feature points, the burden can be reduced as compared with a method of specifying all feature points in advance.

  Next, an embodiment in which the data distribution system of the present invention is applied to the model distribution system 70 will be described. The model distribution system 70 includes an information terminal device 72 and a model distribution server 74 as shown in FIG. The information terminal device 72 and the model distribution server 74 are connected to the network 76 and communicate with each other via the network 76. The information terminal device 72 includes a mobile phone and a personal computer. The model distribution server 74 is a server that provides a model distribution service, and includes the model creation device 10. The network 76 preferably uses the Internet.

  The information terminal device 72 specifies the URL (Uniform Resource Locator) of the model distribution server 74 that provides the model distribution service on the browser, and displays the order screen 76 shown in FIG. 7 on the browser as information from the specified URL. The designated information, that is, the order data, is output to the model distribution server 74 through the operations of the reference model selection button 78, the face image file name 80, and the order instruction button 82 displayed as “order”. . On the order screen 76, it is preferable to select only one favorite model by referring to the thumbnail images of a plurality of displayed reference models. The file name 80 of the face image is specified by the image input editor.

  The model distribution server 74 has a built-in model creation device 10, creates a reference deformation model based on the reference model number and face image data supplied as order data to the model creation device 10, and manages the created reference deformation model Then, the reference deformation model is distributed to the ordered information terminal device 72.

  In this way, the model distribution service supplies order data from the user of the information terminal device 72 to the model distribution server 74, creates a reference deformation model according to the order data supplied by the model distribution server 74, and This is a service for providing a reference deformation model to a certain information terminal device 72.

  By operating in this way, the user can acquire a favorite reference deformation model in a short time with a small burden of only transmitting a face image without specifying the feature of the face part. Further, since the model creation is automatically performed, it is advantageous in that the cost on the model distribution service side is small.

  The model creation device 10 creates a feature point table of supplied order data using a device having functions of feature input, feature determination and feature output, and adds a feature point table to this order data, The information is transmitted to the information terminal device 72 of the transmission source. In this case, the information terminal device 72 is provided with a feature point editing tool described later in advance, and the user may receive order data including the feature point table from the model distribution server 74 and the user may edit the feature point table. The feature point editing tool can be downloaded free of charge from, for example, a model distribution service.

  Although not shown, the user designates the updated feature point table on the browser and operates the order instruction button 82 again. The information terminal device 72 transmits the order data with the feature point table to the model distribution server 74. In the model distribution server 74, the feature point table and the face image included in the order data with the feature point table are input to the model creation device 10, the model is created, and the created reference deformation model is sent to the information terminal device 72 of the transmission source. Send.

  According to this, even if an image is taken under bad conditions, the user checks and corrects it, so that it is possible to create a highly accurate model without failure. Further, since the user only has to edit only the unextracted feature points and the erroneously extracted feature points among the many feature points, the burden on the user can be relatively reduced.

  Furthermore, the model distribution service in this case has no burden of editing features in that it does not involve an editor, and can suppress the cost and set the service price at a low price. Therefore, the user can receive this service at a low charge.

Next, another embodiment in which the data distribution system of the present invention is applied to the model distribution system 70 will be described. In the model distribution system 70, as shown in FIG. 8, an editing terminal device 84 is newly added. The editing terminal device 84 has a function of monitoring an unprocessed order folder in the model distribution server 74 and a feature point editing tool via the network 76. The editing terminal device 84 is operated by the editor.

  The monitoring function in the editing terminal device 84 specifies the URL of the model distribution server 74, displays the order data in the unprocessed order folder on the browser of the editing terminal device 84, and can refer to it in the unprocessed order folder. If one or a plurality of order data exists, the order data is designated and acquired.

  The feature point editing tool in the editing terminal device 84 has a function of displaying, correcting, and updating the face image in the order data and the feature point superimposed on the coordinates indicated by the feature point table. The feature points can be clarified, for example, by superimposing them on the face image with red circles. By referring to the screen, the editor can know the feature points displayed at the wrong coordinates, and can correct and move the feature points to the correct coordinates. If no feature point is displayed at the coordinates from which the feature point is extracted, the feature point is added to coordinates that can be matched from the surrounding feature points. The feature point editing tool updates the feature point data in accordance with the change in coordinates, and updates the coordinates by changing the extraction flag in the feature point data to “1” in accordance with the addition of the feature points. Upon completion, the editing terminal device 84 supplies the feature point table to the model creation device 10 of the model distribution server 74 via the network 76.

  Next, a model distribution service in the model distribution system 70 of the present embodiment will be described.

  The information terminal device 72 designates the URL of the model distribution server 74 that provides the model distribution service, and displays the order screen 76 shown in FIG. 9 on the browser. The order screen 76 is basically the same as the screen configuration of FIG. 7, and is different in that an order list 86 is provided next to the order instruction button 82. The order list 86 will be described later. The information terminal device 72 transmits the information selected on the order screen 76 and the order data to the model distribution server 74.

  The model distribution server 74 creates a feature point table based on the order data supplied using the model creation device 10, adds the feature point table to this order data, and stores it in the unprocessed order folder provided in advance. To do. On the other hand, the editing terminal device 84 designates the URL of the model distribution server 74 and displays and monitors the order data in the unprocessed order folder via the network 76. When the order data exists by monitoring, the order data is designated and the order data is acquired from the model distribution server 74 to the editing terminal device 84.

  The editing terminal device 84 superimposes and displays the face image of the order data and the feature point data using a feature point editing tool provided in advance. The editor moves the wrong coordinates to the correct coordinates while visually confirming with the feature point editing tool, and adds the corresponding coordinates as the feature points when there is no feature point to be originally extracted. The editing terminal device 84 transmits the order data including the feature point data updated in this way to the model distribution server 74.

  In the model distribution server 74, after the order data is downloaded to the editing terminal device 84, the downloaded order data is deleted, and the order data is uploaded to the processed order folder of the model distribution server 74 when the order data is uploaded from the editing terminal device 84. Is stored. The model distribution server 74 supplies the updated order data to the model creation device 10 and creates a reference deformation model. The created reference deformation model is held in the model distribution server 74 until a download request from the user is supplied. Here, the model creation apparatus 10 may use the other configuration example described above.

  By the way, if the user operates the order list button 86 after ordering model creation, the information terminal device 72 sends a list display command to the model distribution server 74. Upon receiving this command, the model distribution server 74 searches the unprocessed order folder and the processed order folder for the order data of the user who sent the command.

  When the order data exists in the unprocessed order folder, the model distribution server 74 transmits a response message such as “model creation in progress” to the information terminal device 72, for example. If the order data exists in the processed order folder, the model distribution server 74 transmits a response message such as “model creation complete” to the information terminal device 72, for example. Such a display of the order list is advantageous in grasping the model creation status for each order data because one user may order a plurality of model creations continuously. The user downloads the standard deformation model of the order corresponding to the response message “model creation complete”, and acquires the model.

  Note that the model distribution system 70 may arrange the model creation device 10 in the editing terminal device 84 as shown in FIG. According to this configuration, the model distribution server 74 only has to manage the order data supplied from the information terminal device 72 and the reference modification model that has been created in the unprocessed order folder and the processed order folder. On the other hand, the editing terminal device 84 creates a feature point table from the order data in the unprocessed order folder by the model creation device 10 and makes the order data including the feature point table. The order data is updated using the feature point editing tool for the order data. The editing terminal device 84 creates a reference deformation model by the model creation device 10 based on the updated order data. The created reference deformation model is supplied from the editing terminal device 84 to the processed order folder of the model distribution server 74 via the network 76. The created reference deformation model is held in the model distribution server 74 until a download request from the user is supplied.

  This operation is advantageous in that the user can acquire a desired model by simply transmitting a face image without specifying complicated features of the facial parts and reducing the burden on the user. is there. Moreover, even if an image shot under adverse conditions is used, the editor performs confirmation and correction, so that it is possible to create a highly accurate model without failure. Furthermore, since the editor only has to edit unextracted feature points and erroneously extracted feature points among many feature points, it is possible to process many orders while reducing the burden on the editor. .

  In particular, when the model creation device 10 is provided in the model distribution server 74, model transmission / reception between the server and the editing terminal is not required, so that communication costs can be reduced. On the other hand, if the model creation device 10 is provided in the editing terminal device 84, the model creation processing is executed by the editing terminal device 84, so that the burden on the model distribution server 74 can be suppressed and realized by a low-spec machine.

  Further, the model distribution system 70 shown in FIG. 11 may be configured by combining FIG. 8 and FIG. However, the model creation device 10 disposed in the model distribution server 74 may have the configuration shown in FIG. In this embodiment, the URL designated by the user is accessed via the network 76, and the order screen 76 of FIG. 12 is displayed on the browser of the information terminal device 72. In the order screen 76, an order form area 88 is added to the configuration of FIG. The operation of this embodiment will be briefly described.

  The user selects his or her favorite reference model by using the selection button 78 with reference to a plurality of prepared thumbnail images of the reference model. This selection button 78 can select only one from a plurality of reference models. In addition, the user designates a data file name indicating a face image photographed by a digital camera or the like in advance using an image input editor.

  Next, the user selects one order form by selecting one selection button from the order form area 88. “Automatically create” creates a standard deformation model without editing part feature points and sets the order form number to 1. “Edit on the service side” means that an editor on the service side edits part feature points, creates a reference deformation model from the edited order data, and sets the order form number to 2. Further, “edit by yourself” obtains a feature point table from the model distribution service, requests the creation of a reference deformation model by editing the part feature points by the user himself, and sets the order form number to 3.

  When the user selects the reference model, the face image data file name, and the order form number, collects the order data, and operates the order instruction button 82, the selected order data is sent from the information terminal device 72 to the model distribution server via the network 76. Sent to 74.

  The model distribution server 74 receives order data from the user. When the order form number is 1, the model distribution server 74 creates a reference deformation model as in the first embodiment of the model distribution system 70 and transmits it to the information terminal device 72. When the order form number is 2, the editor edits the feature points as described above, creates a reference deformation model from the obtained order data, and transmits it to the information terminal device 72. When the order form number is 3, the user edits the feature points on the information terminal device 72 as in the first embodiment, requests the model distribution server 74 to create a model again, and sets the reference deformation model on the information terminal device 72. get.

  By operating in this way, if you want to get a model quickly at a user's request, that is, at a low fee, if you do not want to edit but want to get an accurate model, and edit it yourself, get an accurate model at a low fee It can be selected according to the case where the user wants to do so, and usability can be improved.

  In the embodiment of the present invention, it has been described that the coordinates of the component feature points are expressed in two dimensions, and the polygon movement accompanying the model deformation is also performed in two dimensions. It adopts methods such as automatically detecting depth direction coordinates based on face images that have been used multiple times, or adding depth direction information by the editor at the feature point confirmation / feature point correction stage The model may be deformed three-dimensionally.

  Further, the face image used in the model distribution service is not limited to the user's face, and a model may be created using, for example, face images of friends, lovers, pets, and the like.

  In the embodiment of the present invention, the reference deformation model is created by hardware called a model creation device, but may be realized by software processing called a program. Furthermore, in the embodiment to which the present invention is applied, one information terminal device 72 and one editing terminal device 84 are provided, but it goes without saying that the information terminal device 72 and the editing terminal device 84 can be realized by providing a plurality of each.

  With the configuration described above, the model creation device 10 supplies the face image stored in the image storage unit 12 to the feature extraction unit 14, and the feature extraction unit 14 extracts more feature points in each component than the conventional number. By doing so, the image pasting unit 22 is represented with higher accuracy than the conventional model by deforming the reference data based on the feature points extracted by the model deforming unit 20 and the reference data supplied from the reference model storage unit 18. The image adjustment unit 16 supplies the face image adjusted in the coordinate system of the face image to the coordinate system of the reference model to the image pasting unit 22, and the face image supplied to the model supplied by the image pasting unit 22 By creating a reference deformation model that is pasted and has no deviation, a reference deformation model that matches the face image can be automatically created without a complicated user extracting operation.

  The model creation device 10 sets a feature point table in which a serial number for a feature point, coordinates indicating the position of the feature point, and an extraction flag indicating the presence or absence of the feature point are combined into one set, and the feature point table is externally input by the feature input unit 52. The feature point table supplied by the feature output unit 56 is output to the outside, and at least one of the feature points extracted by the feature determination unit 54 and the feature points from the feature input unit 52 is a reference model. Whether or not the deformation is sufficient, and outputs a feature point table to the model deforming unit 20 according to the true of the determination, and at least the feature point table and parts from the feature input unit 52 according to the false of the determination By outputting one of the feature point tables generated based on the feature points from the extraction unit 14 to the feature output unit 56, the correct feature points can be extracted and provided in response to problems caused by deterioration of the shooting environment, etc. Of feature points Since the collection includes only unextracted feature points and mis-extracted feature points, it is easier to create a standard deformation model that takes advantage of the user's personality by reducing the burden compared to the method of specifying all feature points in advance. Can do.

  In the model distribution system 70, the order data from the information terminal device 72 is supplied to the model distribution server 74 via the network 76, and the face image stored in the image storage unit 12 in the model creation device 10 included in the model distribution server 74 Is supplied from the feature model extracted by the model transformation unit 20 and the reference model storage unit 18 by extracting more feature points in each part than the conventional number by the feature extraction unit 14 A model in which the reference data is deformed based on the reference data is represented with higher accuracy than before and is supplied to the image pasting unit 22, and the image adjustment unit 16 adjusts the coordinate system of the face image to the coordinate system of the reference model. The image is supplied to the image pasting unit 22, the face image supplied to the model supplied by the image pasting unit 22 is pasted to create a standard deformation model without deviation, and the standard deformation model created from the model distribution server 74 is By supplying the information terminal device 12 that has been written, the user's operation burden on the information terminal device 12 is less than before, the cost required to generate the desired reference deformation model in the model distribution server 74 is suppressed, and the model generation Can also be obtained in a short time.

It is a block diagram which shows the schematic structure of the Example to which the model creation apparatus of this invention is applied. It is a figure which shows an example of the position of the extracted feature point which the model preparation apparatus of FIG. 1 handles. It is a figure which shows the example of the reference | standard model which the model creation apparatus of FIG. 1 handles. It is a block diagram which shows the schematic structure of the other Example to which the model creation apparatus of this invention is applied. It is a figure which shows an example of the feature point table which the model creation apparatus of FIG. 4 handles. It is a block diagram which shows the schematic structure in the Example which applied the data delivery system of this invention to the model delivery system. It is a figure which shows an example of the order screen displayed on the information terminal device in the model delivery system of FIG. It is a block diagram which shows the schematic structure in the other Example which applied the data delivery system of this invention to the model delivery system. It is a figure which shows an example of the order screen displayed on the information terminal device in the model delivery system of FIG. It is a block diagram which shows the schematic structure in the other Example which applied the data delivery system of this invention to the model delivery system. It is a block diagram which shows the schematic structure in the other Example which applied the data delivery system of this invention to the model delivery system. FIG. 12 is a diagram showing an example of an order screen displayed on the information terminal device in the model distribution system of FIG.

Explanation of symbols

10 Model creation device
12 Image storage
14 Parts extractor
16 Image adjustment section
18 Reference model storage
20 Model deformation part
22 Image pasting part
24 Model output section
70 Model distribution system
72 Information terminal equipment
74 Model distribution server

Claims (3)

Image storage means for temporarily storing the supplied face image;
Component extraction means for extracting feature points for each characteristic face component included in the face image;
Model storage means for storing a model used as a reference;
Model deformation means for deforming the reference model based on the plurality of extracted feature points;
Image adjusting means for performing adjustment to match the coordinate system for displaying the face image and the coordinate system for displaying the reference model;
An image pasting unit for pasting the adjusted facial image on the modified reference model seen including,
The component extraction unit sets a feature point table in which a serial number for the feature point, coordinates indicating the position of the feature point, and a flag indicating the presence or absence of the feature point are grouped together.
The apparatus further includes feature output means for outputting the feature point table to the outside,
It is determined whether or not all of the extracted feature points used in the deformation of the reference model are included, and the feature point table is output to the model deformation unit according to the true of the determination, and the determination is false model generating apparatus according to claim including Mukoto the feature determining means for outputting the feature point table to the feature output unit in accordance with. The model creation apparatus according to claim 1, wherein the feature determination unit includes an input unit that inputs the feature point table to which the feature point is added or modified from the outside. 3. The apparatus according to claim 2, wherein the reference model is a model other than a person such as a dog or a cat .

Images