WO2013180266A1 - Image generation system, image generation application server, image generation method, and program - Google Patents

Abstract

An avatar generation application server receives face image data and acquires feature point data for the object corresponding to each part of the face, compares the feature point data with a feature point database stored in a storage section, and selects the avatar part image most similar to the object. Then, a user terminal device generates an avatar image on the basis of the selected part image and displays the avatar image on a display section. The user edits the displayed avatar image, if necessary, and finalizes the image. In response to this finalization, the avatar generation application server updates the feature point database so as to add information indicating the relationship between the avatar part image and the object when the image is finalized.

Description

Image creation system, image creation application server, image creation method and program

The present invention relates to an image creation system, an image creation application server, an image creation method, and a program.
This application claims priority based on Japanese Patent Application No. 2012-125331 filed in Japan on May 31, 2012 and Japanese Patent Application No. 2012-285883 filed in Japan on December 27, 2012. Is incorporated herein by reference.

In a communication service on a network such as an SNS (social networking service), a user is expressed by an avatar that is a character of his / herself. This avatar is generally represented by an image of a two-dimensional model or a three-dimensional model.

Such an avatar's face can be regarded as a portrait of the user. From this point of view, the following face image recognition system is known as a technique for easily creating and digitizing his own portrait.
That is, this face image recognition system performs matching between a face image obtained by capturing a face and face parts stored in a database in advance, and adds data such as size, position, color, and inclination to the part and encodes it. . The face image recognition system automatically outputs a portrait on a screen or the like based on the encoded data. In addition, the face image recognition system can adjust the displayed face parts so as to be deformed in accordance with a user operation (see, for example, Patent Document 1).

JP 2007-164408 A

When automatically creating an avatar based on a face image as described above, from the viewpoint that the avatar is a caricature, the quality of the avatar will be improved if the avatar's similarity to the avatar creation image is as high as possible. It becomes.

However, in Patent Document 1, no particular consideration is given to improving the avatar similarity to the image of the avatar creation source. For example, in the case of the configuration of Patent Document 1, the above-described similarity depends on the degree of enhancement of the face part database. Therefore, for example, in the configuration of Patent Document 1, if the number of face parts stored in the database is insufficient, face parts having a tendency of low similarity are continuously created.

Some aspects of the present invention provide an image creation system, an image creation application server, an image creation method, and a program in which the similarity of an avatar to an avatar creation source image is improved.

An image creation system as an aspect of the present invention includes an image input unit that inputs image data, an analysis data acquisition unit that acquires analysis data indicating an analysis result by analyzing the input image data, Based on the result of comparing the analysis data and the comparison data stored in the comparison data storage unit, an element image selection unit that selects an element image from among a plurality of element images stored in the element image storage unit is selected. An overall image creation unit that creates an overall image based on an element image, an overall image provision unit that provides the created overall image to a user, and edits the contents of the provided overall image according to an instruction from the user An entire image editing unit, and an update unit that updates the comparison data based on the content of the entire image when the content of the entire image is confirmed according to the content of the entire image being confirmed Provided.

It is a figure which shows the structural example of the avatar preparation system of this embodiment. It is a figure which shows the structural example of the feature point database production | generation apparatus in this embodiment. It is a figure which shows the structural example of the parts image table in this embodiment. It is a figure which shows the example of acquisition of the feature point data by the feature point data acquisition part in this embodiment. It is a figure which shows an example of the representative point coordinate information which the object definition table production | generation part in this embodiment produces | generates. It is a figure which shows an example of the record of the object definition table which the object definition table production | generation part in this embodiment produces | generates. It is a figure which shows the example of a procedure for calculating | requiring the normalization coordinate of the feature point in this embodiment. It is a figure which shows the example of a procedure for calculating | requiring the normalization coordinate of the feature point in this embodiment. It is a figure which shows the example of a procedure for calculating | requiring the normalization coordinate of the feature point in this embodiment. It is a figure which shows the example of a procedure for calculating | requiring the normalization coordinate of the feature point in this embodiment. It is a figure which shows the example of a procedure for calculating | requiring the normalization coordinate of the feature point in this embodiment. It is a figure which shows an example of the normalization coordinate information which a feature point hash table production | generation part produces | generates. It is a figure which shows an example of the shift calculation process for a feature point hash table production | generation part to convert normalized coordinate information into a hash value. It is a figure which shows an example of the coordinate hash information which a feature point hash table production | generation part produces | generates. It is a figure which shows an example of the record of the feature point hash table which a feature point hash table production | generation part produces | generates. It is a figure which shows the structural example of the feature point database which the feature point database production | generation apparatus of this embodiment produces | generates. It is a figure which shows the example of a process sequence which the feature point database production | generation apparatus of this embodiment performs. It is a figure which shows the structural example of the avatar preparation application server and user terminal device in this embodiment. It is a figure which shows an example of the coordinate hash information which the feature point data acquisition part of the avatar preparation application server in this embodiment produces | generates. It is a figure which shows the example of the content of the feature point database which the avatar preparation application server in this embodiment memorize | stores. It is a figure which shows an example of the comparison result of the coordinate hash of the coordinate hash information by the part image selection part of this embodiment, and the coordinate hash of a feature point database. It is a figure which shows the total result of part ID by the parts image selection part of this embodiment. It is a figure which shows an example of the avatar information which the parts image selection part of this embodiment produces | generates. It is a figure which shows the example of a process sequence which the user terminal device and avatar preparation application server in 1st Embodiment perform. It is a figure which shows the example of a process sequence which the avatar image preparation part of a user terminal device performs in 2nd Embodiment. It is a figure which shows the example of a process sequence which the display control part of a user terminal device performs in 2nd Embodiment. It is a figure which shows the example of a process sequence which the avatar image preparation part of a user terminal device performs in 3rd Embodiment. It is a figure which shows the structural example of the avatar preparation application server and user terminal device in a modification.

<First Embodiment>
[Configuration example of avatar creation system]
FIG. 1 shows a configuration example of an avatar creation system (image creation system) of the present embodiment.
The avatar creation system shown in this figure includes a feature point database generation device 100, an avatar creation application server 200, a user terminal device 300, and an avatar utilization application server 400. The avatar creation application server 200, the user terminal device 300, and the avatar utilization application server 400 are connected to the network 500.

The feature point database generation device 100 is a device that generates a feature point database. The feature point database is a database having a structure in which a part image is associated with a feature point for each object. An object corresponds to a predetermined part such as an eye, nose, mouth, or eyebrows extracted for each of a plurality of face image data. The predetermined part corresponding to this object corresponds to the type (part type) of part images such as eyes, nose, mouth, eyebrows and the like that are necessary for creating an avatar. The avatar creation application server 200 stores the feature point database generated by the feature point database generation device 100.

The avatar creation application server (image creation application server) 200 is a server that executes processing related to the avatar creation application.
In the present embodiment, a native avatar creation application runs on the user terminal device 300. For example, the user terminal device 300 transmits operation information to the avatar creation application server 200 in response to a user operation on the avatar creation application. The avatar creation application server 200 executes a predetermined process in response to this operation information.

Then, the avatar creation application of this embodiment automatically creates an avatar similar to the face indicated by the face image data using the face image data acquired by the user terminal device 300. Here, the face image data acquired by the user terminal device 300 is, for example, captured image data generated by capturing the face of the user or the like. Further, for example, it may be image data such as an illustration depicting a portrait of a user or the like.

The face image data acquired by the user terminal device 300 is transmitted to the avatar creation application server 200 in response to being designated as the avatar creation source face image by an operation on the avatar creation application operating on the user terminal device 300. Is done.
The avatar creation application server 200 automatically creates an avatar similar to the face indicated by the face image data received from the user terminal device 300. At this time, the avatar creation application server 200 specifies similar part images for each part in the face image by comparing the received face image data with the feature point database stored by itself, and these specified parts are identified. Create an avatar with an image.

The user terminal device 300 is a network terminal device operated by a user, and specifically, for example, a mobile phone, a smartphone, a tablet terminal, or the like.
As described above, the avatar creation application operates on the user terminal device 300 by connecting the user terminal device 300 to the avatar creation application server 200.
Moreover, the avatar creation application operates on the user terminal device 300 by connecting the user terminal device 300 to an avatar use application server 400 described later.

The avatar use application server 400 is a server that provides an avatar use application to the user of the user terminal device 300. An avatar utilization application is an application which uses the avatar created by the avatar creation application server.
An example of an avatar utilization application is a virtual space community application.

On the screen of the virtual space community application that runs on the user terminal device 300, the user's own avatar that operates the user terminal device 300 and the avatars of other users exist in the virtual space. These avatars are expressed as, for example, talking about tweets posted by the corresponding user. In addition, for example, when a user touches another avatar by operating the screen of the virtual space and, for example, strokes, the appearance of the avatar being pleased is also displayed. In this way, in the virtual space community application, communication with other users can be achieved using an avatar.

[Configuration example of feature point database generation device]
FIG. 2 shows a configuration example of the feature point database generation apparatus 100. In the present embodiment, an operator for generating a feature point database inputs a sample face image to the feature point database generation device 100. The worker is most similar to each of the objects (parts such as eyes, nose, mouth, eyebrows corresponding to the part image for creating the avatar) in the sample face image among the plurality of part images prepared in advance. Judgment is made visually. Then, the operator performs an operation for associating the object in the sample image with the part image determined to be most similar to the object on the feature point database generation apparatus 100. In response to this operation, the feature point database generation device 100 generates a feature point database indicating part images corresponding to the feature points for each object.

2 includes a sample face image data input unit 101, a feature point data acquisition unit 102, a display control unit 103, a part ID acquisition unit 104, a feature point database generation unit 105, a storage unit 106, and a feature. A point database transmission unit 107, an operation unit 108, and a display unit 109 are provided.

The sample face image data input unit 101 inputs sample face image data, which is face image data prepared in advance as a sample for generating a feature point database, from the outside. This sample face image data is obtained, for example, by imaging real human faces collected as sample targets. Further, the sample face image data input unit 101 may input sample face image data via, for example, a predetermined data interface or a network.

The feature point data acquisition unit 102 performs analysis processing including face recognition on the input sample face image data, thereby supporting predefined objects (parts for creating avatars such as eyes, nose, mouth, eyebrows, etc.) To be extracted). In addition, the feature point data acquisition unit 102 obtains feature point data for each extracted object. Note that feature points in an object are defined in advance for each type of object. Further, the feature point data acquisition unit 102 obtains coordinates for each feature point, for example, as feature point data.

The display control unit 103 stores the face image image input by the sample face image data input unit 101 and the part image stored in the part image table 600 stored in the storage unit 106 as a user interface image for generating a feature point database. Are displayed on the display unit 109.

The operator visually determines from the list of part images displayed on the display unit 109 the most similar one for each part corresponding to the object in the face indicated by the face image displayed on the same display unit 109. Then, the operator performs an operation on the operation unit 108 for selecting a part image determined to be most similar for each object from the part image list.

The part ID acquisition unit 104 acquires a part ID associated with the selected part image from the part image table 600 stored in the storage unit 106. That is, the part ID acquisition unit 104 acquires a part ID that specifies a part image determined to be most similar to the object.

The feature point database generation unit 105 is based on the feature point for each object acquired from the face image data by the feature point data acquisition unit 102 and the part ID associated with each object acquired by the part ID acquisition unit 104. A feature point database 700 is generated.

The feature point database generation unit 105 includes an object definition table generation unit 105A and a feature point hash table generation unit 105B.
The object definition table generation unit 105A generates an object definition table 710 included in the feature point database 700.
The feature point hash table generation unit 105B generates a feature point hash table 720 included in the feature point database 700.

The storage unit 106 stores programs and various types of information necessary for realizing the operation as the feature point database generation apparatus 100. In this figure, a part image table 600 and a feature point database 700 among the information stored in the storage unit 106 are shown.

The part image table 600 corresponds to part IDs of images (part images) of plural types respectively corresponding to predetermined parts such as eyes, nose, mouth, eyebrows of avatars prepared in advance for use in avatar creation. It is a table that has been added and stored.

The feature point database 700 is information generated by the feature point database generation unit 105, and is a database having a structure in which a part ID of a part image is associated with a feature point for each object.
An example of hardware supported by the storage unit 106 is an HDD (Hard Disk Drive), a flash memory, or the like.

The feature point database transmission unit 107 transmits the feature point database 700 generated by the feature point database generation unit 105 to the avatar creation application server 200 via a predetermined data interface or network. At this time, the feature point database transmission unit 107 reads and transmits the feature point database 700 stored in the storage unit 106.
The feature point database 700 transmitted in this way is stored in the avatar creation application server 200.

The operation unit 108 collectively indicates operation devices and operators for operating the feature point database generation apparatus 100. An example of an operation device included in the operation unit 108 is, for example, a mouse or a keyboard.
The display unit 109 displays an image according to the control of the display control unit 103.

Note that the functional units such as the sample face image data input unit 101, the feature point data acquisition unit 102, the display control unit 103, the part ID acquisition unit 104, and the feature point database generation unit 105 in the feature point database generation device 100 are in the computer device. This can be realized by causing a CPU (Central Processing Unit) to execute a program.

[Example structure of parts table image]
Here, with reference to FIG. 3, a structural example of the part image table 600 stored in the storage unit 106 will be described.
The part image table 600 shown in FIG. 3 is formed by associating a part ID with a part type ID for each part image.
The parts image is an image of parts corresponding to eyes, nose, mouth, eyebrows and the like for forming an avatar. The data stored as the part image is information that refers to the substance of the image data, such as the file name of the image data.
The part ID is an identifier for uniquely identifying the corresponding part image.
The part type ID is an identifier for uniquely identifying the part type to which the corresponding part image belongs. The part type corresponds to each part of parts such as eyes, nose, mouth, and eyebrows.
Note that the part types of the part images specifically shown in FIG. 3 are common to eyes.
Accordingly, the part type ID associated with these part images is “0” indicating that both are eyes.

In FIG. 3, for convenience of simplifying the illustration, an example of content corresponding to only the eyes is shown, but the part image table 600 corresponds to other part types such as nose, mouth, and eyebrows, for example. Also includes a part image record. That is, the part image table 600 stores part images corresponding to each of a plurality of part types.
A structural example of the feature point database 700 will be described later.

[Feature point database generation example]
Next, a specific example of processing for generating a feature point database executed by the feature point database generating apparatus 100 having the above configuration will be described.

First, the feature point data acquisition unit 102 obtains feature point data for each object (face part) from the face image data input by the sample face image data input unit 101 as follows.
The feature point data acquisition unit 102 associates, for example, the input sample face image data with a two-dimensional absolute coordinate system in the x direction and the y direction. Then, the feature point data acquisition unit 102 obtains coordinates (absolute coordinates (x, y)) in the absolute coordinate system for each feature point of the object obtained from the sample face image data.

FIG. 4 schematically shows an example of acquiring feature point data in the case of an object OBJ as an eye.
Thus, when the object OBJ is an eye, the feature point data acquisition unit 102, for example, as shown in the figure, from the eye image portion recognized from the face image data, each of the absolute coordinate systems of the six feature points ER1 to ER6. The feature point data of the object OBJ is obtained by obtaining the absolute coordinates at.

Next, the object definition table generation unit 105A in the feature point database generation unit 105 generates the object definition table 710 as follows.
First, the object definition table generation unit 105A obtains coordinate information indicating the absolute coordinates of two representative points preset for each part from the absolute coordinates of a plurality of feature points acquired by the feature point data acquisition unit 102. Generate.

The object OBJ shown in FIG. 4 is an eye. The representative points set in advance corresponding to the part of the eye are, for example, the corner of the eye and the head of the eye. Among the feature points ER1 to ER6 shown in FIG. 4, the feature points ER1 and ER4 correspond to the corners of the eyes and the eyes. Therefore, the representative points in FIG. 4 are feature points ER1 and ER4. In the following description, the feature points ER1 and ER4 may be referred to as representative points ER1 and ER4.

Then, the object definition table generating unit 105A acquires the absolute coordinates (x, y) of the representative points ER1 and ER4 from the representative points ER1 to ER6 as representative point coordinate information.
FIG. 5 shows representative point coordinate information 810 indicating the absolute coordinates of the representative points ER1 and ER4 of FIG. The representative point coordinate information 810 shown in this figure indicates that the absolute coordinate of the representative point ER1 is (356, 421) and the absolute coordinate of the representative point ER2 is (53, 383).

In parallel with this, the operator performs an operation of designating a part image visually determined to be most similar to the eye of FIG. 4 in the face image displayed on the display unit 109. In response to this, the part ID acquisition unit 104 acquires a part ID associated with the object as the eye of FIG.
Therefore, the object definition table generation unit 105A uses the representative point coordinate information 810 and the part ID acquired by the part ID acquisition unit 104 to record a record of the object definition table 710 corresponding to the object OBJ as an eye in FIG. Generate.

FIG. 6 shows an example of the contents of the record 711 of the object definition table 710 corresponding to the object OBJ of FIG.
As shown in this figure, the record 711 of the object definition table 710 includes a part type ID, an object ID, a part ID, and representative point coordinates.

The part type ID indicates the type of the part image corresponding to the corresponding object.
Since the object in FIG. 4 is an eye, the type of part image corresponding to this object is also an eye. Accordingly, the part type ID in FIG. 6 stores a value (for example, “0”) indicating that the part image type is eye. From this, it can be said that the part type ID in the record 711 indicates the type of eyes, nose, mouth, eyebrows, etc., for the face part to which the object corresponds.

The object ID is an identifier that uniquely identifies the corresponding object. This object ID is added after being generated by the object definition table generation unit 105A when the record 711 is generated.

The part ID is a part ID indicating a part image associated with the object. This part ID is acquired by the part ID acquisition unit 104.
The representative point coordinates indicate the coordinates of two representative points. In the case of this figure, the contents of the representative point coordinate information 810 of FIG. 5 are reflected. In this case, the representative point coordinates (X0, Y0) indicate the absolute coordinates (356, 421) of the representative point ER1, and the representative point coordinates (X1, Y1) indicate the absolute coordinates (53, 383) of the representative point ER4. .

In parallel with the processing of the object definition table generation unit 105A, the feature point hash table generation unit 105B generates the feature point hash table 720 in the feature point database 700 as follows. The following description is processing for generating the record 721 of the feature point hash table 720 corresponding to the object OBJ in FIG.

For this purpose, the feature point hash table generation unit 105B first converts the absolute coordinates of the feature points ER1 to ER6 acquired by the feature point data acquisition unit 102 into normalized coordinates (Xnz, Ynz) in the normalized coordinate system. .

In the conversion to the normalized coordinates, the feature point hash table generation unit 105B uses a normalized coordinate system.
FIG. 7A shows an example of a normalized coordinate system used by the feature point hash table generation unit 105B. This normalized coordinate system is a two-dimensional coordinate of an X coordinate and a Y coordinate. Here, the X coordinate of the normalized coordinate system is represented by Xnz, and the Y coordinate is represented by Ynz.
In the normalized coordinates, reference coordinates REF1 and REF2 that are normalized in advance for the representative points ER1 and ER4 are set. In the example of FIG. 7A, the reference coordinate REF1 corresponding to the representative point ER1 is set as (8, 5), and the reference coordinate REF2 corresponding to the representative point ER4 is set as (1, 5).

Then, as shown in FIG. 7B, for example, the feature point hash table generation unit 105B makes the absolute coordinates of the feature points ER1 to ER6 match the representative point ER4 with the reference coordinates REF2, and then sets the feature points ER1 to ER6. Place absolute coordinates on a normalized coordinate system.

Next, as shown by arrows A and B in FIG. 7C, the feature point hash table generation unit 105B changes the enlargement ratio using another representative point ER1 as a reference and the reference coordinate REF2 where the representative point ER4 is already arranged. And move to the reference coordinate REF1.
Then, the magnification and rotation angle when the representative point ER1 is moved to the reference coordinate REF1 as described above are also applied to the remaining feature points ER2, ER3, ER5, and ER6 other than the representative points ER1 and ER4. .

As a result, as shown in FIG. 8A, the absolute coordinates of the feature points ER1 to ER6 are converted into normalized coordinates.
However, at the stage of FIG. 8A, the coordinate values in the normalized coordinate system of the remaining feature points ER2, ER3, ER5, and ER6 other than the representative points ER1 and ER4 include values after the decimal point. This means that there is an error with respect to the normalized coordinates that should take an integer value.
Therefore, as shown in FIG. 8B, the feature point hash table generation unit 105B moves the feature points ER2, ER3, ER5, and ER6 to the coordinates of integer values approximated in the normalized coordinate system. As a result, the absolute coordinates of the feature points ER1 to ER6 are converted into normalized coordinates.

Next, the feature point hash table generation unit 105B uses the normalized coordinates of the feature points ER2, ER3, ER5, ER6 other than the representative points ER1, ER4 among the normalized coordinates obtained as shown in FIG. (Hereinafter also referred to as coordinate hash).
For this purpose, the feature point hash table generation unit 105B generates normalized coordinate information 820 indicating the normalized coordinates of the feature points ER2, ER3, ER5, and ER6, for example, as shown in FIG.
The normalized coordinate information 820 in FIG. 9 indicates the feature points ER2, ER3, ER5, and ER6 obtained as a result of the normalization illustrated in FIG. 8B. That is, the normalized coordinate of the feature point ER2 indicates (7, 6), the normalized coordinate of the feature point ER3 indicates (3, 6), the normalized coordinate of the feature point ER5 indicates (3, 4), The normalized coordinates of the feature point ER6 indicate (7, 4).

Then, the feature point hash table generation unit 105B indicates each of the normalized coordinates of the feature points ER2, ER3, ER5, and ER6 of the normalized coordinate information 820 shown in FIG. 9 as the shift calculation coordinates 820A of FIG. Perform a shift operation. In the case of FIG. 10, the shift calculation is performed with the coordinate values Xnz and Ynz in normalized coordinates being 16 bits each.
From the result of the shift operation shown in FIG. 10, the feature point hash table generation unit 105B, for example, as shown in FIG. 11, for example, coordinate hash information 830 in which coordinate hashes for feature points ER2, ER3, ER5, and ER6 are arranged. Is generated.
In the coordinate hash information 830 shown in FIG. 11, coordinate hashes of feature points ER2, ER3, ER5, and ER6 are sequentially arranged from left to right.

Next, the feature point hash table generation unit 105B generates an object ID for uniquely identifying the object OBJ in FIG.
In addition, the feature point hash table generation unit 105B uses the part type ID corresponding to the type of the object OBJ in FIG. 4, the object ID generated as described above, and the contents of the coordinate hash information 830 in FIG. A record 721 shown in FIG.
The record 721 shown in FIG. 12 is to be included in the feature point hash table 720, and indicates the hash value of the normalized coordinates of the feature points ER2, 3, 5, and 6 corresponding to the object OBJ shown in FIG. .

FIG. 13 shows a structure example of the feature point database 700 stored in the storage unit 106. As shown in this figure, the feature point database 700 includes an object definition table 710 and a feature point hash table 720.
The object definition table 710 includes a table for each part (object) type indicated by the part type ID, for example. In addition, the table corresponding to the part type ID is formed by a plurality of records in which the object ID, the part ID, and the representative point coordinates for each object are associated with each other.
The feature point hash table 720 also includes a table for each part type (object type) indicated by the part type ID. In addition, the table corresponding to the part type ID is formed by a plurality of records in which the object ID and the coordinate hash are associated with each object.

Then, the object definition table generation unit 105A registers, as a new record, the record 711 of FIG. 6 that it has generated for the table corresponding to the part type ID = 0 as indicated by the wavy line in the object definition table 710 of FIG. To do.

In addition, the feature point hash table generation unit 105B generates the record 721 of FIG. 12 generated by itself for the table corresponding to the same part type ID = 0, as indicated by the wavy line in the feature point hash table 720 of FIG. Register as a new record.
As a result, a record corresponding to one object OBJ is newly registered in the feature point database 700.

The feature point database generation apparatus 100 performs new registration of the record corresponding to the object OBJ in the feature point database 700 for each object corresponding to each different part type extracted from one sample face image data. In addition, new registration of a record corresponding to a plurality of part types in such sample face image data is performed every time the sample face image data is acquired.
As a result, the feature point database 700 is formed including records (s × p) corresponding to the total number s of input sample face image data and the number of part types p.

[Example of processing procedure for generating feature point database]
Next, an example of a processing procedure executed by the feature point database generation device 100 will be described with reference to the flowchart of FIG.

In the feature point database generation device 100, the sample face image data input unit 101 inputs one sample face image data (step S101).

Next, as described above with reference to FIG. 4, the feature point data acquisition unit 102 performs face recognition on the input sample face image data, and acquires feature point data for each object extracted from the sample face image data. (Step S102).

Also, the display control unit 103 causes the display unit 109 to display the input sample face image data and the part images stored in the part image table 600 stored in the storage unit 106 in a predetermined manner (step S103).

The operator determines the most similar part image for each part corresponding to the object in the sample face image data by viewing the image displayed on the display unit 109 in step S103. Then, an operation of selecting the part image as the most similar to the object is performed.
In response to this operation, the part ID acquisition unit 104 acquires a part ID associated with the selected part image from the part image table 600 (step S104).

In parallel with the part ID acquisition process in step S104, the object definition table generation unit 105A generates representative point coordinate information 810 indicating the absolute coordinates (x, y) of the representative points ER1 and ER4 for each object. (Step S105).

Then, the object definition table generation unit 105A generates a record 711 of the object definition table 710 for each object (step S106).
For this purpose, the object definition table generation unit 105A newly generates an object ID for each object. Then, the object definition table generation unit 105A generates a record 711 for each object using the newly generated object ID, the part ID acquired in step S104, and the representative point coordinate information 810 generated in step S105.

Next, as described with reference to FIGS. 9 to 12, the feature point hash table generation unit 105B generates a record 721 of the feature point hash table 720 using feature point data other than the representative points (step S107).

Then, the object definition table generation unit 105A and the feature point hash table generation unit 105B newly register each record generated in steps S106 and S107 in the feature point database 700 as described with reference to FIG. (Step S108).
That is, the object definition table generation unit 105A newly registers the record 711 for each object generated in step S106 in the object definition table 710 in the feature point database 700. In addition, the feature point hash table generation unit 105B newly registers the record 721 generated in step S107 in the feature point hash table 720 in the feature point database 700.

Next, the feature point database generation unit 105 (the object definition table generation unit 105A or the feature point hash table generation unit 105B) performs a process of newly registering in the feature point database 700 corresponding to all the prepared sample face image data. It is determined whether or not the processing has been completed (step S109).

Here, when the processing has not been completed for all the sample face image data (step S109—NO), the sample face image data input unit 101 returns to step S101 to input the next sample face image data. To do. Thereby, the process of generating the feature point database 700 corresponding to the next sample face image data is started.

On the other hand, when the processing for all the sample face image data is completed (step S109-YES), the feature point database 700 including the records for the objects for all the sample face image data is formed. . Therefore, the feature point database transmission unit 107 reads the feature point database 700 from the storage unit 106 and transmits it to the avatar creation application server 200 (step S110).

[Configuration example of avatar creation application server]
Next, a configuration example of the avatar creation application server 200 will be described with reference to FIG.
The avatar creation application server 200 shown in this figure includes a face image data input unit 201, a feature point data acquisition unit 202, a storage unit 203, a part image selection unit 204, a feature point database update unit 205, and a communication unit 206.

The face image data input unit 201 (image input unit) inputs face image data (image data) as an avatar creation source. The face image data as the avatar creation source is transmitted from the user terminal device 300 connected to the avatar creation application server 200 and operating the avatar creation application, and is received by the communication unit 206. The face image data input unit 201 inputs the face image data received by the communication unit 206 in this way.

The feature point data acquisition unit 202 (analysis data acquisition unit) acquires the analysis data indicating the analysis result by analyzing the input face image data.
That is, the feature point data acquisition unit 202 acquires the feature point data (analysis data) for each object extracted from the face image data by analyzing the input face image data.
The feature point data acquisition unit 202 may have the same configuration as the feature point data acquisition unit 102 in FIG.

In addition, the feature point data acquisition unit 202 of the present embodiment generates, for example, coordinate hash information 910 having a structure shown in FIG. 16 for each object as the feature point data extraction result.
As shown in FIG. 16, the coordinate hash information 910 has a structure in which coordinate hash information is associated with each part type ID.
The part type ID in the coordinate hash information 910 in FIG. 16 indicates the part of the face to which the object corresponds (that is, the type of part image).
The coordinate hash indicates a hash value of coordinates for each feature point other than the representative point obtained for the corresponding object. Note that the coordinate hash calculation method for each feature point may be the same as described with reference to FIGS.

The storage unit (comparison data storage unit, element image storage unit) 203 stores various types of information necessary for the avatar creation application server 200 to realize its functions. In the figure, as such information, a parts image table 600A and a feature point database 700A are shown.

The part image table 600A corresponds to part IDs of images (part images) of a plurality of types respectively corresponding to predetermined parts such as eyes, nose, mouth and eyebrows of an avatar prepared in advance for use in avatar creation. It is a table that has been added and stored. The contents of the parts image table 600A are the same as the parts image table 600 (FIG. 3) stored in the feature point database generation device 100, for example. That is, the storage unit 203 stores a part image corresponding to each of a plurality of part types as a part image table 600A.

The feature point database 700A is obtained by storing the feature point database 700 received from the feature point database generation device 100 in the storage unit 203.

Here, the storage unit 203 may store the feature point database 700 received from the feature point database generation device 100 as an initial feature point database 700A in advance, for example, in a stage before providing an avatar creation application.
In this case, the initial feature point database 700A has the same contents as the feature point database 700. Accordingly, the object definition table 710A and the feature point hash table 720A of the feature point database 700A in the initial stage have the same contents as the object definition table 710 and the feature point hash table 720, respectively.
Then, the feature point database update unit 205 updates the initial feature point database 700A to add a new record (comparison data).

The part image selection unit (element image selection unit) 204, based on the result of comparing the feature point data and the feature point database 700A (comparison data), from the part image table 600A (a plurality of part images) Element image).
At this time, the part image selection unit 204 selects a part image for each of a plurality of part types.
Thereby, the part image selection part 204 can select the part image for every site | parts, such as eyes, a mouth, a nose, for example.

Specifically, the part image selection unit 204 inputs the coordinate hash information 910 (FIG. 16) generated by the feature point data acquisition unit 202 as the feature point data. Then, the part image selection unit 204 compares the coordinate hash in the coordinate hash information 910 with the coordinate hash in the feature point hash table 720A of the feature point database 700A for each feature point.

As a specific example, a case will be described in which the coordinate hash of the coordinate hash information 910 in FIG. 16 is compared with the coordinate hash of the feature point hash table 720A of the feature point database 700A shown in FIG. In the feature point database 700A of FIG. 17, for the convenience of making the illustration simple and easy to understand, in both the object definition table 710A and the feature point hash table 720A, the part type ID = 0 corresponding to the eye part type Only the table corresponding to is shown.

In the feature point hash table 720A of FIG. 17, cells corresponding to feature points whose coordinate hashes match the coordinate hash information 910 of FIG. The part image selection unit 204 recognizes that the coordinate hashes of feature points corresponding to the thick-line cells shown in FIG.
FIG. 18 schematically shows the recognition result. In this figure, for each object with object ID = 0 to 29, the number of feature points (coordinate hash coincidence number) with which coordinate hash coincides is shown by the comparison result.

In this case, the objects in the feature point hash table 720 in FIG. 17 are the objects corresponding to the coordinate hash information 910 in FIG. 16 as the number of feature points matching the coordinate hash information in the coordinate hash information 910 in FIG. The degree of similarity is high.

Therefore, the parts image selection unit 204 first recognizes the maximum value max of the number of coordinate hash matches this time based on the comparison result of FIG. In this case, as can be seen with reference to FIG. 18, the maximum value of the coordinate hash match number is max = 4. This means that an object in which the coordinate hash of the coordinate hash information 910 in FIG. 16 matches the coordinate hash of all feature points is recognized.

Next, the part image selection unit 204 executes a totaling process for totalizing the part IDs whose coordinate hash match number is the maximum value max = 4.
For this purpose, the part image selection unit 204 searches the feature point hash table 720A for an object ID having a maximum number of coordinate hash matches max = 4. Next, the part image selection unit 204 searches the part definition associated with the searched object ID from the object definition table 710A. And the parts image selection part 204 totals the number of searches for every part ID.

FIG. 19 shows the comparison results shown in FIG. 18 and the tabulation results corresponding to the contents of the object definition table 710A shown in FIG. As shown in this figure, the number of searches for part ID = 0 is “1”, the number of searches for part ID = 1 is “0”, and the number of searches for part ID = 2 is “2”. Yes, the number of searches for part ID = 3 is “0”, and the number of searches for part ID = 4 is “1”.

Of these, the part with the largest number of searches is part ID = 2. This means that among the part images stored in the part image table 600A, the part image with the part ID = 2 is most similar to the object of the part corresponding to the part type ID = 0 extracted from the face image data. means.
Therefore, the part image selection unit 204 selects a part image with a part ID = 2 as a part image corresponding to the part type ID = 0.

Although description by illustration is abbreviate | omitted, the part image selection part 204 selects a part image for every object corresponding to other part classification ID based on the result of having compared similarly to the above.
And the part image selection part 204 will produce | generate the avatar information reflecting the selection result of the part images so far, if selection of the part image for every object corresponding to all the part classification IDs is complete | finished. The avatar information is information used to create an avatar image in the user terminal device 300.

FIG. 20 shows a structural example of the avatar information 920 generated by the part image selection unit 204. As shown in this figure, the avatar information 920 has a structure in which a part ID and part image data are associated with each part type ID for every part type required to create an avatar.
The face of the avatar image created by the part image stored in the avatar information 920 is similar to the face of the face image data of the avatar creation source.
And the parts image selection part 204 transmits the avatar information 920 produced | generated as mentioned above with respect to the user terminal device 300 from the communication part 206. FIG.

The avatar creation application on the user terminal device 300 creates an avatar image using the avatar information 920 received from the avatar creation application server 200 as described above, and displays the avatar image.

The face of the avatar image displayed in this way is similar to, for example, the face of the face image data transmitted earlier, but the user, for example, in order to make the face of the avatar image more similar to the face of the face image data An operation for editing the avatar can be performed. The operation for editing the avatar here is to select and change the part image used for creating the avatar from the part image list. Of course, if the user likes the avatar image created by the received avatar information 920, the user does not need to edit.

Then, if the user of the user terminal device 300 edits and obtains a satisfactory avatar face, the user terminal device 300 performs an operation for avatar determination on the avatar creation application. Note that if the user likes the avatar image created by the received avatar information 920 as it is, the user may perform a confirmation operation without editing.

In response to the above confirmation operation, the avatar creation application on the user terminal device 300 transmits a confirmation notification to the avatar creation application server 200. In response to the confirmation notification, the user terminal device 300 transmits the confirmation avatar information indicating the content of the avatar when the confirmation operation is performed.
The confirmed avatar information included in the confirmation notification has a structure in which, for example, an avatar image is omitted from the avatar information 920 in FIG. 20 and a part ID is associated with each part type ID. However, if the contents are changed by editing, the changed contents are reflected.

In the avatar creation application server 200, the feature point database update unit (update unit) 205 stores the content of the avatar image (the entire image) based on the content of the avatar image when the content is determined. The feature point database 700A stored in the unit 203 is updated.

Specifically, the feature point database update unit (update unit) 205 first updates the object definition table 710A as follows to update the feature point database 700A.
That is, the feature point database update unit 205 generates a new object ID for each object extracted from the avatar creation source face image data.
Next, the feature point database update unit 205 newly generates a record 711 of the object definition table 710A associated with the newly generated object ID.
For this purpose, the feature point database update unit 205 associates the newly generated object ID with the part ID indicated by the confirmed avatar information included in the received confirmation notification. Furthermore, the feature point database update unit 205 associates the newly generated object ID with the representative point coordinates that the feature point data acquisition unit 102 acquired from the avatar creation source face image data. In this way, the feature point database update unit 205 generates a record 711 corresponding to the newly generated object ID for each part type.
Then, the feature point database update unit 205 newly registers the newly generated record 711 in the table for each corresponding part type ID in the object definition table 710A.

The feature point database update unit 205 also updates the feature point hash table 720A as follows. Here, the feature point data acquisition unit 202 uses the coordinate hash information 910 having the structure shown in FIG. 16 for creating an avatar. Generated for every part type. The feature hash data acquisition unit 202 holds the coordinate hash information 910 generated in this way.
Therefore, the feature point database update unit 205 associates the newly generated object ID with the coordinate hash of the coordinate hash information 910 generated for the object indicated by the object ID, thereby recording the record 721 of the feature point hash table 720A. Is generated. Also about this record 721, the feature point data acquisition part 202 produces | generates for every part classification.
Then, the feature point database update unit 205 newly registers the newly generated record 721 in the table for each corresponding part type ID in the feature point hash table 720A.
In this way, the feature point database update unit 205 updates the feature point database 700A by updating the object definition table 710 and the feature point hash table 720.

In the avatar creation application server 200, the communication unit 206 communicates with the user terminal device 300, the avatar utilization application server 400, and the like via the network 500.

[Configuration example of user terminal device]
Next, a configuration example of the user terminal device 300 will be described again with reference to FIG.
The user terminal device 300 includes a control unit 301, an operation unit 302, a display unit 303, and a communication unit 304.

The control unit 301 executes processing corresponding to the avatar creation application operating on the user terminal device 300. In this embodiment, the avatar creation application is installed in the user terminal device 300 as, for example, a native application program.

The control unit 301 includes a face image data transmission unit 311, an avatar image creation unit 312, a display control unit 313, and an avatar editing unit 314. These functional units are realized by, for example, a CPU configuring the control unit 301 executing an avatar creation application.

The face image data transmission unit 311 transmits the face image data to the avatar creation application server 200 in response to an operation for instructing the avatar creation application to transmit face image data including the face of the avatar creation source. .

As described above, the face image data is acquired as, for example, image data obtained by imaging the face of the user himself (or another person) by an imaging unit (not shown here) provided in the user terminal device 300, for example. can do. Further, the face image data may be acquired via a network, a data interface, or the like, or by reading from a removable medium loaded in the user terminal device 300. Moreover, designation as face image data for the image data acquired in this way can also be performed by an operation on the avatar creation application on the user terminal device 300.

As described above, the user can edit the avatar created by the avatar creating application server 200 in response to an operation on the avatar creating application operating on the user terminal device 300.

The avatar image creation unit (overall image creation unit) 312 creates an avatar image based on the part image selected by the part image selection unit 204.
That is, the avatar image creation unit 312 generates an avatar image based on the avatar information 920 generated by the part image selection unit 204 of the avatar creation application server 200. The avatar information 920 stores a part image selected by the part image selection unit 204. The avatar image creating unit 312 creates an avatar image using the part images stored in the avatar information 920.

The display control unit (overall image providing unit) 313 provides the user with the avatar image created by the avatar image creating unit 312. Specifically, the display control unit 313 provides the user with the avatar image by causing the display unit 303 to display the avatar image.

When the user looks at the avatar image displayed on the display unit 303 and, for example, the user wants to further resemble the face of the original face image data, the editing operation for changing the avatar is performed. Can be done against.
In this editing operation, for example, as described above, a desired part image is selected from the list of part images displayed on the screen of the avatar creation application, and the part image used for the avatar is changed. It becomes that.

The avatar editing unit (whole image editing unit) 314 edits the avatar according to the avatar editing operation performed as described above. That is, the avatar editing unit 314 edits the content of the avatar image provided to the user according to the user's instruction.
Specifically, the avatar editing unit 314 requests the part image by specifying the part ID of the part image selected by the operation from the avatar creation application server 200.
In response to this request, the avatar creation application server 200 reads the designated part image from the part image table 600A and transmits it to the user terminal device 300. The avatar editing unit 314 changes the part image used so far to the received part image. The avatar image creation unit 312 creates an avatar image in which the change of the part image is reflected, and the display control unit 313 displays the avatar image. Thus, the avatar image is edited in accordance with the avatar editing operation performed by the user.

If the user is satisfied with the edited avatar image as described above, or if the user is satisfied with the avatar image that the avatar creation application has automatically created without editing, the content of the avatar Perform a confirmation operation to confirm

In response to the confirmation operation being performed, the avatar editing unit 314 generates, for example, confirmed avatar information indicating the content of the avatar when the confirmation operation is performed using the avatar information in which the edited content is reflected. To do. Then, a confirmation notification including the confirmed avatar information is transmitted to the avatar creation application server 200.
As described above, the feature point database update unit 205 in the avatar creation application server 200 updates the feature point database 700A using the confirmed avatar information in response to receiving the confirmation notification.

Thus, in the avatar creation system of this embodiment, the avatar creation application server 200 automatically creates an avatar using the face image data and the feature point database 700A.
In addition, the automatically created avatar can be edited on the user terminal device 300. The user terminal device 300 transmits the editing result (including a case where editing is not performed) to the avatar creation application server 200 as a confirmation notification in response to the confirmation operation of the user.
Then, the avatar creation application server 200 updates the feature point database 700A so as to add a record about the object extracted from the face image data that is the creation source of the current avatar according to the received editing result.

In the user terminal device 300, the operation unit 302 collectively indicates operation devices necessary for operating an avatar creation application that operates on the user terminal device 300. For example, when the user terminal device 300 is a mobile phone or a smartphone, the operation unit 302 is various keys or a touch panel provided on the mobile phone or the smartphone.
The display unit 303 displays an image according to the control of the display control unit 313 in the user terminal device 300.

The communication unit 304 communicates with the avatar creation application server 200, the avatar utilizing application server 400, and the like via the network 500.

Thus, in this embodiment, the feature point database 700A is updated every time an avatar is automatically created. This means that each time an avatar is automatically created, information indicating the correspondence between an object and a part image similar to the object increases in the feature point database 700A. For example, since human faces are different from each other, it can be said that the feature amount of an object for each face image data is various even in the same part (part type).
In the present embodiment, records in the feature point database 700A are added based on the contents of different face image data for each opportunity for an avatar to be automatically created. This corresponds to an improvement in the accuracy of the similarity of the part images corresponding to the objects in the feature point database 700A corresponding to the above-described diversity of objects.
Then, the feature point database 700A whose accuracy is improved in this way is used for automatic creation of an avatar, so that the accuracy of automatically creating an avatar similar to the face of the face image data of the creation source is also improved.

In the present embodiment, the feature point database 700A is updated in correspondence with an avatar that is not substantially edited by the user.
For example, when an avatar is automatically created, even if a part image corresponding to an object that does not match all of the feature points is selected, editing is performed because the user has the impression that they are very similar It is conceivable that the confirmation operation may be performed without performing it.
In such a case, the record newly registered in the feature point database 700A may be different from the content of the record already stored in the feature point database 700A without overlapping. At this time, the record newly registered in the feature point database 700A reflects a result determined to be more similar to the user's subjectivity. Therefore, the update of the feature point database 700A corresponding to an avatar that is not substantially edited also leads to an improvement in the accuracy of the feature point database 700A.

Here, in order to improve the accuracy of the feature point database 700A, the following other methods may be employed.
That is, when the feature point database generation apparatus 100 creates the feature point database 700, a huge number of sample face image data is prepared. Then, the feature point database 700 is generated by processing the sample face image data.
However, as described above, when the feature point database generation apparatus 100 creates the feature point database 700, it is necessary for an operator to visually determine similarity. For this reason, the burden on the operator increases as the number of sample face image data increases.
On the other hand, in the case of this embodiment, every time a user creates an avatar using an avatar application, the accuracy of the feature point database 700A is improved, and the burden on the worker increases. There is no.

[Example of processing procedure for updating feature point database]
Next, an example of a processing procedure executed by the user terminal device 300 and the avatar creation application server 200 in order to update the feature point database 700A will be described with reference to a flowchart of FIG.

In the user terminal device 300, the face image data transmission unit 311 causes the communication unit 304 to transmit the face image data to the avatar creation application server 200 in response to an operation instructing transmission of the face image data (step S201).

In response to the transmission of the face image data, the avatar creation application server 200 automatically creates an avatar and transmits the avatar information 920. In response to the avatar information being received by the communication unit 304, the avatar image creating unit 312 creates an avatar image using the received avatar information (step S202).

The display control unit 313 causes the display unit 303 to display the avatar image created in step S202 (step S203).

Further, the avatar editing unit 314 executes the avatar editing process so as to change the avatar information in accordance with the avatar editing operation performed by the user (step S204).

The process in step S204 can be repeatedly executed until, for example, a confirmation operation for confirming the content of the avatar is performed (NO in step S205).

Then, in response to the confirmation operation being performed (step S205—YES), the avatar editing unit transmits a confirmation notification including the confirmed avatar information to the avatar creation application server 200 (step S206).

Further, in the avatar creation application server 200, the face image data transmitted from the user terminal device 300 in step S201 is received by the communication unit 206. In response to this, the face image data input unit 201 inputs the received face image data (step S301).

Next, the feature point data acquisition unit 202 extracts objects by analyzing the face image data, and acquires feature point data for each of these objects (step S302). Then, the feature point data acquisition unit 202 generates the coordinate hash information 910 described in FIG. 16 for each object as the extraction result of the feature point data, and holds these (step S303).

Next, the part image selection unit 204 compares the coordinate hash indicated by the coordinate hash information 910 as the feature point data with the coordinate hash in the feature point hash table 720A of the feature point database 700A for each feature point. Based on the comparison result, the part image selection unit 204 selects a similar part image for each object from the part image table 600A (step S304).

In addition, the part image selection unit 204 generates avatar information 920 based on the selection result of the part image in step S304 (step S305). The avatar information generated in this way indicates the contents of the automatically created avatar.
And the parts image selection part 204 transmits the avatar information produced | generated by step S305 with respect to the user terminal device 300 (step S306).

After the avatar information is transmitted in step S306, a confirmation notification is transmitted from the user terminal device 300 in step S206. In response to the confirmation notification being received by the communication unit 206, the feature point database update unit 205 stores the records 711 in the object definition table 710A and the feature point hash table 720A for each object extracted corresponding to step S302. A new record 721 is generated (step S307).
Then, the feature point database update unit 205 updates the feature point database 700A (step S308). That is, the feature point database update unit 205 newly registers the records 711 and 721 generated in step S307 in the object definition table 710A and the feature point hash table 720A in the feature point database 700A, respectively.

<Second Embodiment>
[Overview]
Next, a second embodiment will be described. In addition, the structure of the feature point database production | generation apparatus 100 in 2nd Embodiment, the avatar preparation application server 200, and the user terminal device 300 may be the same as that of FIG. 2 and FIG. 15, for example.

In the first embodiment, one avatar is created using one face image data as a creation source. On the other hand, the second embodiment corresponds to a case where a plurality of avatars may be created using one face image data as a creation source.

In the second embodiment, the part image selection unit 204 generates a plurality of avatar information corresponding to a plurality of avatars in the following cases.
That is, in the total result of the part IDs shown in FIG. 19, when there are a plurality of part IDs with the largest number of searches, the part image selection unit 204 selects the part images corresponding to the plurality of part IDs. To do. In this way, the part image selection unit 204 selects a plurality of part images from the part images corresponding to the same part type.
When a plurality of part images are selected in this way, the part image selection unit 204 generates a plurality of avatar information reflecting the part IDs of these part images. And the parts image selection part 204 transmits these avatar information with respect to a user terminal device.
And in the user terminal device 300, according to reception of a plurality of avatar information, creation and display of an avatar image are performed as follows.

[Example of processing procedure corresponding to the second embodiment]
The flowchart of FIG. 22 is a processing procedure in the case of corresponding to the second embodiment in which a plurality of avatar information may be transmitted as described above as processing for creating an avatar image in step S202 of FIG. An example is shown.

In this case, the avatar image creation unit 312 of the user terminal device 300 determines whether or not the received avatar information is plural (step S401).
When there is one avatar information (step S401—NO), the avatar image creation unit 312 creates one avatar image corresponding to the received one avatar information (step S402).
On the other hand, when there is a plurality of avatar information (step S401—YES), the avatar image creation unit 312 creates a plurality of avatar images corresponding to each of the received plurality of avatar information (step S403).

Further, the flowchart of FIG. 23 shows an example of a processing procedure in the case of corresponding to the second embodiment as processing for avatar image display in step S203 of FIG.

In this case, the display control unit 313 of the user terminal device 300 determines whether or not there are a plurality of avatar images created in step S202 (step S501).
When there is one created avatar image (step S501-NO), the display control unit 313 displays this one avatar image on the display unit 303 (step S502).
On the other hand, when there is a plurality of avatar information (step S501—YES), the display control unit 313 causes the display unit 303 to display the plurality of created avatar images (step S503).

When a plurality of avatar images are displayed, the user first selects one avatar image from the plurality of avatar images as an avatar editing operation. If necessary, a confirmation operation is performed after performing an operation of changing the part of the selected avatar image. Therefore, the avatar edit part 314 should just transmit the confirmation notification containing the avatar confirmation information about one avatar according to confirmation operation.

As described above, in the second embodiment, the avatar creating application server 200 generates a plurality of avatar information when the similarity of the part images to be selected in the same part type is equal or close. When the user terminal device 300 receives a plurality of avatar information, the user terminal device 300 creates an avatar image for each of the plurality of avatar information and displays the avatar image on the display unit 303.
Thereby, in 2nd Embodiment, a some candidate can be provided to a user as an avatar image supposed to be similar to face image data. Thereby, the user can select what he / she determined to be most similar from among a plurality of avatar image candidates. This result is reflected in the update of the feature point database 700A. As a result, the feature point database 700A reflects, for example, the user's preference tendency with respect to the face of the avatar. That is, it is possible to improve the accuracy of the feature point database 700A so as to suit the user's preference tendency and the like.

<Third Embodiment>
[Overview]
Next, a third embodiment will be described.
In the previous second embodiment, when a plurality of avatar information is received from the avatar creation application server 200, the avatar image creation unit 312 creates a plurality of avatar images based on each of the plurality of avatar information.
However, for example, when the part ID matching frequency in a certain part type ID is high in a plurality of avatar information, the part image of the part ID having a high matching frequency is similar to other part images in that part type. Can be estimated.
Therefore, the avatar image creation unit 312 in the third embodiment does not create an avatar image for each avatar information, but applies a part ID having a high matching frequency to a plurality of avatar information in common. Then, the avatar image creation unit 312 creates an avatar image based on avatar information in which part IDs having a high matching frequency are shared. As a result, the plurality of created avatar images all have high similarity to the original face image data.
That is, when a plurality of part images are selected from the part images corresponding to the same part type based on the result of the part image selection unit 204 comparing the feature point data with the feature point database 700A, an avatar image is created. The unit 312 creates a plurality of avatar images corresponding to each of the selected plurality of part images. In addition, the display control unit 313 causes the display unit 303 to display a plurality of avatar images created as described above.

[Example of processing procedure corresponding to the third embodiment]
The flowchart in FIG. 24 illustrates an example of a processing procedure when the processing for creating an avatar image in step S202 in FIG. 21 corresponds to the third embodiment. In this figure, the same steps as those in FIG. 22 are denoted by the same reference numerals.

In this case, the avatar image creation unit 312 of the user terminal device 300 determines whether or not the received avatar information is plural (step S401).
When there is one avatar information (step S401—NO), the avatar image creation unit 312 creates one avatar image corresponding to the received one avatar information (step S402).

On the other hand, when there is a plurality of avatar information (step S401—YES), the avatar image creation unit 312 determines the part type ID corresponding to the part ID having the highest matching frequency in the received plurality of avatar information (step S401—YES). S403A).
Next, for the part ID corresponding to the part type ID determined in step S403A in the plurality of avatar information, the avatar image creating unit 312 stores the plurality of avatar information so that the part ID having a higher matching frequency is stored in common. It is regenerated (step S403B).
Then, the avatar image creating unit 312 creates a plurality of avatar images corresponding to each of the plurality of avatar information regenerated as described above (step S403C).

Further, the processing of the display control unit 313 in the third embodiment is the same as that in FIG. Thereby, a plurality of avatar images created by the processing of FIG. 24 are displayed on the display unit 303.

<Modification>
Next, a modification of this embodiment will be described.
In the embodiments so far, the case where the avatar creation application is installed in the user terminal device 300 as a native application program has been described as an example. However, the avatar creation application can be configured as a browser-type application (also referred to as a web application).
Thus, as a modification of the present embodiment, a configuration of an avatar creation system corresponding to a browser-type avatar creation application is given.

FIG. 25 shows a configuration example of the avatar creating application server 200A and the user terminal device 300A in the avatar creating system as a modification adopting the browser type avatar creating application as described above. In this figure, the same parts as those in FIG.

The user terminal device 300A shown in this figure includes a web browser unit 315 as a function corresponding to avatar creation.

When the avatar creation application is a browser type, the avatar creation application server 200 </ b> A transmits the file of the avatar creation application to the user terminal device 300 </ b> A connected via the network 500. This file is described in a markup language such as XML (Extensible Markup Language) or HTML (HyperText Markup Language).
The web browser unit 315 of the user terminal device 300A reproduces the file transmitted as described above. By reproducing the file in this manner, the application screen of the avatar creation application is displayed on the display unit 303 of the user terminal device 300A. In the case of this modification, the functions of the face image data transmission unit 311 and the display control unit 313 in FIG. 15 are realized by the web browser unit 315 that reproduces the avatar creation application file as described above.

Then, when the user of the user terminal device 300A operates the application screen displayed as described above using the operation unit 302, the web browser unit 315 sends a command corresponding to the operation to the avatar creation application server 200A. Send to. The avatar creation application server 200A executes a predetermined process related to avatar creation in response to the received command. Thereby, for example, the same operation as that of the embodiment described so far is realized.

Next, the avatar creating application server 200A shown in this figure further includes an avatar image creating unit 211 and an avatar editing unit 212 in addition to the configuration of the avatar creating application server 200 shown in FIG.

The avatar image creation unit 211 creates an avatar image based on the part image selected by the part image selection unit 204. That is, the avatar image creation unit 211 generates an avatar image based on the avatar information 920 generated by the part image selection unit 204 in the avatar creation application server 200A. And the avatar image preparation part 211 produces | generates the file of the avatar preparation application which reflected the created avatar image, and transmits this file to the user terminal device 300A. When the web browser unit 315 of the user terminal device 300A reproduces the file, an application screen on which the created avatar image exists is displayed on the display unit 303.

In addition, the avatar editing unit 212 edits the avatar image created by the avatar image creating unit 211 in accordance with a user instruction (avatar editing operation) performed on the user terminal device 300A.
The user of the user terminal device 300A can perform an avatar editing operation on the application screen on which the avatar image is displayed as described above. In response to the operation for editing the avatar, the web browser unit 315 transmits a command corresponding to the operation from the avatar creation application server 200A to the output.

The avatar editing unit 212 performs avatar editing according to the command transmitted as described above. The avatar editing process itself may be the same as that of the avatar editing unit 314 in FIG. In addition, the avatar image creating unit 211 changes the avatar image based on the avatar information changed according to the editing result of the avatar editing unit 212. In addition, the avatar image creation unit 211 generates a file reflecting the contents of the avatar image changed in this way, and transmits the file to the user terminal device 300A.
And according to a user performing operation for avatar confirmation with respect to an application screen, the web browser part 315 transmits a notification of confirmation with respect to the avatar preparation application server 200A. In response to receiving the confirmation notification, the feature point database update unit 205 updates the feature point database 700A as in the case of FIG.

It should be noted that methods other than those described so far may be adopted as the method for generating the feature point databases 700 and 700A and the method for selecting the part image in each of the above embodiments. Along with this, the structure of the feature point databases 700 and 700A may be changed.

In the above description, the feature amount of the object is feature point data based on the coordinates. However, for example, the color of the object may be further included in the feature amount.

Further, the program for realizing each functional unit shown in FIG. 2, FIG. 15 and FIG. 25 is recorded on a computer-readable recording medium, and the program recorded on this recording medium is read into a computer system and executed. By doing so, generation of feature point databases 700 and 700A, creation of avatars, update of feature point databases 700 and 700A, and the like may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM or a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

An image creation system according to an aspect of the present invention includes an image input unit (201) that inputs image data, and analysis data acquisition that acquires analysis data indicating the analysis result by analyzing the input image data Element image selection for selecting an element image from among a plurality of element images stored in the element image storage unit based on a result of comparing the unit (202) with the analysis data and the comparison data stored in the comparison data storage unit Unit (204), an overall image creation unit (312) that creates an overall image based on the selected element image, and an overall image provision unit (313) that provides the created overall image to the user. A whole image editing unit (314) for editing the contents of the whole image according to the user's instruction, and when the contents of the whole image are confirmed And a update unit that updates the comparison data based on the contents of the serial whole image (205).

In the image creation system as one aspect of the present invention, the comparison data storage unit stores in advance initial comparison data created by using a plurality of image data prepared in advance, and the update The unit (205) may update the initial comparison data so as to add new comparison data.

In the image creation system as one aspect of the present invention, the element image storage unit stores a plurality of element images corresponding to a plurality of element types, and the element image selection unit (204) Element images may be selected for each of the plurality of element types.

In the image creation system as one aspect of the present invention described above, the overall image creation unit (312) includes elements corresponding to the same element type based on a result of comparing the analysis data and the comparison data. When a plurality of element images are selected from the images, a plurality of the whole images are created corresponding to each of the selected plurality of element images, and the whole image providing unit generates the plurality of whole images An image may be provided to the user.

In the image creation system as one aspect of the present invention, the overall image creation unit (312) is most effective when a plurality of element images are selected from element images corresponding to the same element type. For element types corresponding to element images having a high matching frequency, the plurality of whole images may be created using the element image having the highest matching frequency in common.

The image creation application server as one aspect of the present invention includes a communication unit that communicates with a user terminal device via a network, an image input unit (201) that inputs image data, and analyzes the input image data. Then, based on the result of comparing the analysis data acquisition unit (202) that acquires the analysis data indicating the analysis result and the comparison data stored in the comparison data storage unit, the element image storage unit An element image selection unit (204) for selecting an element image from among a plurality of element images to be stored, and the content of the entire image created based on the selected element image in accordance with a user instruction to the user terminal device An update unit (205) that updates the comparison data based on the content of the whole image when it is confirmed in response to the confirmation.

The image creation application server as one aspect of the present invention includes a communication unit that communicates with a user terminal device via a network, an image input unit (201) that inputs image data, and analyzes the input image data. Then, based on the result of comparing the analysis data acquisition unit (202) that acquires the analysis data indicating the analysis result and the comparison data stored in the comparison data storage unit, the element image storage unit An element image selection unit (204) for selecting an element image from a plurality of element images to be stored, an entire image generation unit (312) for generating an entire image based on the selected element image, and the user terminal device An overall image editing unit (314) that edits the content of the created overall image in response to a user instruction made to the user, and the content of the overall image Serial in response to is determined in accordance with an instruction from a user to the user terminal device, and a update unit for updating the comparison data based on the contents of the entire image when it is determined (205).

An image creation method as one aspect of the present invention includes an image input step for inputting image data, and an analysis data acquisition step for acquiring analysis data indicating the analysis result by analyzing the input image data. And an element image selection step for selecting an element image from among a plurality of element images based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit, and based on the selected element image A whole image creating step for creating a whole image, a whole image providing step for providing the created whole image to a user, and a whole image editing for editing the content of the provided whole image in accordance with an instruction from the user And comparing the comparison data based on the content of the whole image when confirmed, according to the step and the content of the whole image being confirmed. And an updating step of new to.

The image creation method as one aspect of the present invention is stored in a communication step of communicating with a user terminal device via a network, an image input step of inputting image data, and the analysis data and comparison data storage unit. An element image selection step for selecting an element image from a plurality of element images based on the result of comparison with the comparison data, and the content of the entire image created based on the selected element image for the user terminal device An update step of updating the comparison data based on the content of the whole image when confirmed in response to being confirmed according to a user instruction.

The image creating method as one aspect of the present invention includes a communication step of communicating with a user terminal device via a network, an image input step of inputting image data, and analyzing the input image data. Select an element image from a plurality of element images based on the result of comparing the analysis data and the comparison data stored in the comparison data storage unit with an analysis data acquisition step for acquiring analysis data indicating the analysis result An element image selection step, an entire image creation step for creating an entire image based on the selected element image,
A whole image editing step for editing the contents of the created whole image in accordance with a user instruction given to the user terminal apparatus; and a contents of the whole image in accordance with a user instruction to the user terminal apparatus An update step of updating the comparison data based on the content of the whole image when it is confirmed.

A program as one aspect of the present invention is stored in a computer in a communication step for communicating with a user terminal device via a network, an image input step for inputting image data, and the analysis data and comparison data storage unit. An element image selection step of selecting an element image from a plurality of element images based on the result of comparison with the comparison data to be obtained, and the content of the entire image created based on the selected element image is the user terminal device The update step of updating the comparison data based on the content of the whole image when confirmed is performed in response to the confirmation according to the user's instruction.

According to another aspect of the present invention, there is provided a program comprising: a communication step of communicating with a user terminal device via a network; an image input step of inputting image data; and analyzing the input image data. The analysis data acquisition step for acquiring the analysis data indicating the analysis result, and the result of comparing the analysis data and the comparison data stored in the comparison data storage unit, the element image from among the plurality of element images An element image selection step to select, an entire image creation step to create an entire image based on the selected element image, and the entire image created in response to a user instruction given to the user terminal device An overall image editing step for editing the contents of the Depending on the contents is confirmed, it is intended for executing an updating step of updating the comparison data based on the contents of the entire image when it is determined.

As described above, according to some aspects of the present invention, the degree of avatar similarity to an avatar creation source image is improved.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design and the like within the scope not departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100 Feature point database generation apparatus 101 Sample face image data input part 102 Feature point data acquisition part 103 Display control part 104 Acquisition part 105 Feature point database generation part 105A Object definition table generation part 105B Feature point hash table generation part 106 Storage part 107 Feature Point database transmission unit 108 operation unit 109 display unit 200 avatar creation application server 201 face image data input unit 202 feature point data acquisition unit 203 storage unit 204 part image selection unit 205 feature point database update unit 206 communication unit 300 user terminal device 301 control Unit 302 Operation unit 303 Display unit 304 Communication unit 311 Face image data transmission unit 312 Avatar image creation unit 313 Display control unit 314 Avatar editing unit 400 Application for avatar use Server 500 network 600 parts image table 600A parts image table 700 feature point database 700A feature point database 710 object definition table 710A object definition table 711 record 720 feature point hash table 720A feature point hash table 721 record 810 representative point coordinate information 820 normalization Coordinate information 820A Shift calculation coordinates 830 Coordinate hash information 910 Coordinate hash information 920 Avatar information

Claims (12)

1. An image input unit for inputting image data;
   By analyzing the input image data, an analysis data acquisition unit that acquires analysis data indicating the analysis result;
   An element image selection unit that selects an element image from among a plurality of element images stored in the element image storage unit, based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
   An overall image creation unit for creating an overall image based on the selected element image;
   A whole image providing unit for providing the created whole image to a user;
   An overall image editing unit that edits the content of the provided entire image according to an instruction of the user;
   And an update unit that updates the comparison data based on the content of the whole image when the content of the whole image is confirmed.
2. The comparison data storage unit
   The initial comparison data created using a plurality of image data prepared in advance is stored in advance,
   The update unit
   The image creation system according to claim 1, wherein the initial comparison data is updated so as to add new comparison data.
3. The element image storage unit
   Storing a plurality of element images corresponding to each of a plurality of element types;
   The element image selection unit
   The image creation system according to claim 1, wherein an element image is selected for each of the plurality of element types.
4. The overall image creation unit
   Based on the result of comparing the analysis data and the comparison data, when a plurality of element images are selected from among the element images corresponding to the same element type, each of the selected plurality of element images corresponds. To create a plurality of the whole images,
   The overall image providing unit includes:
   The image creation system according to claim 3, wherein the plurality of created overall images are provided to a user.
5. The overall image creation unit
   When a plurality of element images are selected from element images corresponding to the same element type, the element image having the highest matching frequency is commonly used for the element type corresponding to the element image having the highest matching frequency. The image creation system according to claim 4, wherein the plurality of whole images are created.
6. A communication unit that communicates with a user terminal device via a network;
   An image input unit for inputting image data;
   An analysis data acquisition unit for acquiring analysis data indicating the analysis result acquired by analyzing the input image data;
   An element image selection unit that selects an element image from among a plurality of element images stored in the element image storage unit based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
   The comparison based on the content of the whole image when the content of the whole image created based on the selected elemental image is decided according to the user's instruction to the user terminal device An image creation application server including at least an update unit for updating data.
7. A communication unit that communicates with a user terminal device via a network;
   An image input unit for inputting image data;
   By analyzing the input image data, an analysis data acquisition unit that acquires analysis data indicating the analysis result;
   An element image selection unit that selects an element image from among a plurality of element images stored in the element image storage unit, based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
   An overall image creation unit for creating an overall image based on the selected element image;
   A whole image editing unit that edits the contents of the created whole image in response to a user instruction given to the user terminal device;
   An update unit that updates the comparison data based on the content of the whole image when the content of the whole image is confirmed according to a user instruction to the user terminal device. Image creation application server.
8. An image input step for inputting image data;
   By analyzing the input image data, an analysis data acquisition step of acquiring analysis data indicating the analysis result;
   An element image selection step of selecting an element image from a plurality of element images based on the result of comparing the analysis data and comparison data stored in the comparison data storage unit;
   A whole image creating step for creating a whole image based on the selected element image;
   A whole image providing step of providing the created whole image to a user;
   A whole image editing step of editing the contents of the provided whole image according to an instruction of the user;
   An image creating method including at least an update step of updating the comparison data based on the content of the whole image when the content of the whole image is confirmed.
9. A communication step for communicating with a user terminal device via a network;
   An image input step for inputting image data;
   An element image selection step of selecting an element image from a plurality of element images based on the result of comparing the analysis data obtained by analyzing the image data and the comparison data stored in the comparison data storage unit; ,
   The content of the entire image created based on the selected element image is determined according to the user's instruction to the user terminal device, and based on the content of the entire image when determined. An image generation method including at least an update step for updating the comparison data.
10. A communication step for communicating with a user terminal device via a network;
    An image input step for inputting image data;
    By analyzing the input the image data, the analysis data acquisition step of acquiring the analysis data indicating the result of the analysis,
    An element image selection step of selecting an element image from a plurality of element images based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
    A whole image creating step for creating a whole image based on the selected element image;
    A whole image editing step for editing the contents of the created whole image in accordance with a user instruction given to the user terminal device;
    An update step of updating the comparison data based on the content of the whole image when the content is confirmed in response to the content of the whole image being confirmed according to a user instruction to the user terminal device. Image creation method.
11. On the computer,
    A communication step for communicating with a user terminal device via a network;
    An image input step for inputting image data;
    An element image selection step of selecting an element image from a plurality of element images based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
    The content of the entire image created based on the selected element image is determined according to the user's instruction to the user terminal device, and based on the content of the entire image when determined. A program for executing an update step for updating comparison data.
12. On the computer,
    A communication step for communicating with a user terminal device via a network;
    An image input step for inputting image data;
    By analyzing the input image data, an analysis data acquisition step for acquiring analysis data indicating the analysis result;
    An element image selection step of selecting an element image from a plurality of element images based on a result of comparing the analysis data and comparison data stored in the comparison data storage unit;
    A whole image creating step for creating a whole image based on the selected element image;
    A whole image editing step of editing the contents of the created whole image in response to a user instruction given to the user terminal device;
    An update step of updating the comparison data based on the content of the whole image when the content is confirmed in response to the content of the whole image being confirmed according to a user instruction to the user terminal device. Program for.
    
    

Images