JP2021043841A - Virtual character generation apparatus and program - Google Patents

Abstract

To provide a virtual character generation apparatus which can simply generate a virtual character with no feeling of strangeness, and a program.SOLUTION: A virtual character generation apparatus 1 includes: an input data receiving unit 11 which receives input data; an image impression specifying unit 15 or a voice impression specifying unit 19 which specifies impression of the input data received by the input data receiving unit 11 by referring to a first storage unit storing first data composed of image data and voice data and an impression discriminator to be used for specifying impression of the first data; a voice selection unit 16 or an image selection unit 20 which selects second data associated with the specified impression by referring to a second storage unit storing multiple pieces of second data of a kind different from the first data and associated with impressions thereof; and a virtual character generation unit 21 which generates a virtual character composed of the input data received by the input data receiving unit 11 and the second data selected by the voice selection unit 16 or the image selection unit 20.SELECTED DRAWING: Figure 1

Description

The present invention relates to a virtual character generator and a program.

Conventionally, an effort has been made to install a display or a microphone at a reception desk of a company, and the displayed virtual character responds to a user on behalf of a person. As the character that serves such customers, a character generated by a device that generates a virtual character that operates in a three-dimensional virtual space (for example, Patent Document 1) may be used.

Japanese Unexamined Patent Publication No. 2011-113135

It is possible to make the virtual character used for customer service as described above suitable for the taste of the customer service partner. By doing so, the other party can easily have a favorable impression on the virtual character. However, when serving a large number of customers, it is necessary to prepare a large number of virtual characters to suit the tastes of each partner, which is troublesome. Further, in the case where the virtual character is made to speak, if the same voice is used, the appearance and the voice of the virtual character do not match, and the person feels uncomfortable.

Therefore, an object of the present invention is to provide a virtual character generation device and a program capable of easily generating a virtual character without a sense of discomfort.

The present invention solves the above problems by the following solutions.
The first invention is a virtual character generation device that generates a virtual character composed of a combination of image data and audio data, the first invention comprising an input data receiving means for receiving input data and image data or audio data. A first storage unit that stores data and an impression classifier used to identify the impression of the first data, and a plurality of second data in which each impression of a type different from the first data is associated with each other are stored. The impression specifying means for specifying the impression of the input data received by the input data receiving means by referring to the second storage unit and the first storage unit, and the impression specifying means with reference to the second storage unit. A virtual character is created by the selection means for selecting the second data associated with the impression specified by the means, the input data received by the input data receiving means, and the second data selected by the selection means. It is a virtual character generation device including a virtual character generation means to be generated.
According to the second invention, in the virtual character generation device of the first invention, the input data receiving means receives image data of one virtual character as the input data, and the second storage unit receives audio data. An audio data storage unit in which the impression matching the audio data is associated in advance, and the image data of the one virtual character received by the input data receiving means is analyzed to obtain an image of the one virtual character. A feature extraction means for extracting image feature data indicating features related to data is provided, and the impression specifying means relates to the image data of the one virtual character based on the image feature data extracted by the feature extraction means. The selection means is a virtual character generation device that identifies an impression, refers to the voice data storage unit, and selects the voice data from the impression specified by the impression specifying means.
According to the third invention, in the virtual character generation device of the second invention, the voice data storage unit associates the voice data with the impression for each gender and age, and the feature extraction means. An estimation means for estimating gender and age based on the extracted image feature data is provided, and the selection means refers to the audio data storage unit and is based on the gender and age estimated by the estimation means. This is a virtual character generation device that selects the voice data.
The fourth invention is the virtual character generation device of the second invention or the third invention, in which the first storage unit is a character feature storage unit in which the image feature data of each virtual character and the impression are associated with each other. The impression specifying means identifies the impression based on the degree of coincidence between the image feature data extracted by the feature extracting means and the image feature data stored in the character feature storage unit. It is a virtual character generator.
According to a fifth aspect of the present invention, in the virtual character generation device of the first invention, the input data receiving means receives one voice data as the input data, and the second storage unit is at least a part of each virtual character. An image data storage unit in which the image data representing the above and the impression matching the image data are previously associated with each other, and the one audio data received by the input data receiving means is analyzed to obtain the one audio. A feature amount calculating means for calculating a feature amount of data is provided, and the impression specifying means identifies the impression related to the one voice data based on the feature amount of the voice data extracted by the feature amount calculating means. The selection means is a virtual character generation device that refers to the image data storage unit and selects the image data from the impression specified by the impression specifying means.
A sixth aspect of the present invention is the virtual character generation device of the fifth invention, wherein the first storage unit is a voice feature storage unit in which a feature amount of the voice data and the impression are associated with each other, and the impression specifying means. Identifyes the impression based on the degree of coincidence between the feature amount of the one voice data calculated by the feature amount calculation means and the feature amount of the voice data stored in the voice feature storage unit. It is a virtual character generator.
According to the seventh invention, in the virtual character generation device of the fifth invention or the sixth invention, the image data storage unit associates the image data with the impression for each part constituting each virtual character. The selection means refers to the image data storage unit, selects the image data from the impression specified by the impression specifying means for each part, and the virtual character generating means selects the image data. Virtual character generation that generates image data of one virtual character by combining the image data selected for each part by means, and generates the virtual character by the generated image data and the one audio data. It is a device.
The eighth invention is a program for operating a computer as any of the virtual character generation devices from the first invention to the seventh invention.

According to the present invention, it is possible to provide a virtual character generation device and a program capable of easily generating a virtual character without a sense of discomfort.

It is a functional block diagram of the virtual character generation apparatus which concerns on this embodiment. It is a figure which shows the example of the storage part of the virtual character generation apparatus which concerns on this embodiment. It is a figure which shows the example of the impression map corresponding to the impression used in the virtual character generation apparatus which concerns on this embodiment. It is a flowchart which shows the voice DB construction process which constructs the voice DB of the virtual character generation apparatus which concerns on this embodiment. It is a flowchart which shows the image DB construction process which constructs the image DB of the virtual character generation apparatus which concerns on this embodiment. It is a flowchart which shows the virtual character generation processing of the virtual character generation apparatus which concerns on this embodiment. It is a flowchart which shows the voice selection process of the virtual character generation apparatus which concerns on this embodiment. It is a figure for demonstrating the voice selection process of the virtual character generation apparatus which concerns on this embodiment. It is a flowchart which shows the image selection process of the virtual character generation apparatus which concerns on this embodiment. It is a figure for demonstrating the image selection process of the virtual character generation apparatus which concerns on this embodiment.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. It should be noted that this is only an example, and the technical scope of the present invention is not limited to this.
(Embodiment)
<Virtual character generator 1>
FIG. 1 is a functional block diagram of the virtual character generation device 1 according to the present embodiment.
FIG. 2 is a diagram showing an example of a storage unit 30 of the virtual character generation device 1 according to the present embodiment.
FIG. 3 is a diagram showing an example of an impression map 50 corresponding to the impression used in the virtual character generation device 1 according to the present embodiment.

The virtual character generation device 1 shown in FIG. 1 is a device that generates a virtual character that does not cause a sense of discomfort in appearance and voice based on input data. More specifically, when the input data is voice data related to the voice spoken by the virtual character, the virtual character generation device 1 selects the image data of the virtual character that matches the voice data. When the input data is the image data of the virtual character, the virtual character generation device 1 selects the voice data of the virtual character that matches the image data.
The virtual character generation device 1 is, for example, a server. The virtual character generation device 1 may also be a personal computer (PC) or the like.

As shown in FIG. 1, the virtual character generation device 1 includes a control unit 10, a storage unit 30, an input unit 37, a display unit 38, and a communication unit 39.
The control unit 10 is a central processing unit (CPU) that controls the entire virtual character generation device 1. The control unit 10 appropriately reads and executes the operating system (OS) and the application program stored in the storage unit 30 to cooperate with the above-mentioned hardware and execute various functions.

The control unit 10 includes an input data receiving unit 11 (input data receiving means), an image processing unit 12, a voice processing unit 17, and a virtual character generating unit 21 (virtual character generating means).
The input data receiving unit 11 is a control unit that receives input data from the input unit 37. The input data receiving unit 11 receives audio data or image data as input data. Here, the voice data is, for example, data related to a human voice. Further, the image data is, for example, image data including a face image of a virtual character. The image data may be an image of a virtual character showing not only the whole body but also the upper body.

The image processing unit 12 is a control unit that performs processing when the input data received by the input data receiving unit 11 is image data. The image data is an image of a virtual character, and may be an image of a person represented by, for example, an animation, or a realistic image such as a photograph of a person. Further, the image of the virtual character may be either a 2D image or a 3D image.

The image processing unit 12 includes an image feature extraction unit 13 (feature extraction means), an attribute estimation unit 14 (estimation means), an image impression identification unit 15 (impression identification means), and a voice selection unit 16 (selection means). Be prepared.
The image feature extraction unit 13 analyzes the image data received as the input data and extracts the image feature data indicating the features related to the image. The image feature extraction unit 13 may extract image feature data from the image data by using, for example, a technique such as deep learning. The image feature data includes, for example, the hair color and hairstyle of the virtual character, the ratio and complexion of eyes, nose, and mouth, clothing and its color, and the like.

The attribute estimation unit 14 estimates the gender and age of the virtual character indicated by the image data. The attribute estimation unit 14 may estimate the gender and age of the virtual character indicated by the image data, for example, based on the image feature data extracted by the image feature extraction unit 13. More specifically, the attribute estimation unit 14 may estimate the gender and age based on, for example, the hairstyle, the color of the hair, the ratio of the eyes, the nose, and the mouth. Further, the attribute estimation unit 14 may recognize a face by using a known image recognition method, and further estimate the gender and age from the characteristics of each part constituting the face.

The image impression specifying unit 15 specifies the impression of the virtual character based on the image data. Here, the impression is related to the visual impression of the virtual character indicated by the image data. Impressions can be obtained, for example, as classifications such as "cute", "gorgeous", and "traditional".
The image impression specifying unit 15 specifies an impression based on, for example, the image feature data extracted by the image feature extracting unit 13. More specifically, the image impression specifying unit 15 includes image feature data extracted by the image feature extraction unit 13 and each image feature data (impression) of the image DB (database) 33 (described later) (first storage unit). The impression corresponding to the image feature data of the image DB 33 having a high degree of matching in comparison with the classifier) is specified as the impression of the image data.

The voice selection unit 16 selects voice data that matches the virtual character indicated by the image data. The voice selection unit 16 is based on the impression specified by the image impression specifying unit 15, and the voice data corresponding to the impression from the voice data stored in the voice DB 32 (described later) (second storage unit) (second data). To select. At that time, the voice selection unit 16 selects voice data corresponding to the gender and age estimated by the attribute estimation unit 14.

The voice processing unit 17 is a control unit that performs processing when the input data received by the input data receiving unit 11 is voice data. The voice data is data used as the voice of a virtual character. The voice data may be data in which the voice of a person including the utterance content is recorded, or data in which the voice of a person not including the utterance content such as "ah" is recorded. Further, the voice data may be, for example, one generated by a hostile generation network (GANs: Generative Adversarial Networks), or one artificially generated by a computer or the like by synthetic voice creation or the like.

The voice processing unit 17 includes a voice feature amount calculation unit 18 (feature amount calculation means), a voice impression specifying unit 19 (impression specifying means), and an image selection unit 20 (selection means).
The voice feature amount calculation unit 18 calculates the feature amount of the voice data received as input data. As the feature amount of the voice data, for example, the feature amount related to the pitch, loudness, and timbre of the voice is calculated. The voice feature amount calculation unit 18 calculates feature amounts related to voice pitch, loudness, and timbre as fundamental frequency, power, and mel frequency cepstrum coefficients (MFCC: Mel-Frequency Cepstrum Coefficients), respectively. The MFCC is a voice feature amount represented by a numerical value in n dimensions (n is, for example, 20).

The voice impression specifying unit 19 identifies the impression of the virtual character based on the voice data. Here, the impression is related to the impression of the voice represented by the voice data, and can be obtained as a classification such as "cute", "gorgeous", "traditional", etc., like the impression of the image data.
The voice impression specifying unit 19 specifies an impression based on, for example, the voice feature amount calculated by the voice feature amount calculation unit 18. More specifically, the voice impression specifying unit 19 is characterized by comparing the feature amount calculated by the voice feature amount calculation unit 18 with each feature amount (impression classifier) of the voice DB 32 (first storage unit). The similarity of the quantities is calculated, and those having a certain threshold value or more are classified, and the classified impression is specified as the impression of the voice data as having a high degree of agreement.

The image selection unit 20 selects image data that matches the virtual character indicated by the voice data. The image selection unit 20 selects image data (second data) corresponding to the impression from the image data stored in the image DB 33 (second storage unit) based on the impression specified by the voice impression specifying unit 19. Here, when the image data for each part is stored in the image DB 33, the image selection unit 20 selects the image data corresponding to the impression for each part.

When the input data received by the input data receiving unit 11 is image data, the virtual character generation unit 21 generates a virtual character composed of the audio data selected by the image processing unit 12 and the received image data. ..
Further, when the input data received by the input data receiving unit 11 is voice data, the virtual character generation unit 21 generates a virtual character composed of the image data selected by the voice processing unit 17 and the received voice data. Generate.

The storage unit 30 is a storage area for a hard disk, a semiconductor memory element, or the like for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
The storage unit 30 includes a program storage unit 31, a voice DB 32 (voice data storage unit, voice feature storage unit), an image DB 33 (image data storage unit, character feature storage unit), a generated character storage unit 34, and utterance contents. A storage unit 35 is provided.
The program storage unit 31 is a storage area for storing various programs. The program storage unit 31 stores the virtual character generation program 31a. The virtual character generation program 31a is a program for performing various functions executed by the control unit 10 of the virtual character generation device 1.

The voice DB 32 is a database that stores voice data and information on voice features and impressions.
FIG. 2A shows an example of items and data included in the voice DB 32.
The voice DB 32 shown in FIG. 2A stores data corresponding to each item of gender, age, voice data, feature amount, and impression data using the voice ID (IDentification) as a key.
The voice ID is identification information for uniquely identifying the voice data.
The gender and age are the gender and age determined from the voice data. Gender is a classification of males or females, and ages are teens, 20s, etc., which is an example.
The voice data is the data of the voice itself.
The feature amount is a feature amount of the voice data calculated by analyzing the voice data. As described above, the feature amount is expressed numerically as the fundamental frequency, power, and MFCC. For example, what is represented by MCFF (20 dimensions) is
MFCC: Represented by a series of 20 numerical values consisting of -14.77, -8.78, -6.70, ..., 2.80, 22, 29.
Impression data is data for identifying an impression determined from voice data. The impression data is the coordinate data of the impression map 50 in this example.

Here, the impression map 50 will be described with reference to FIG.
The impression map 50 shown in FIG. 3 is a map used for both image data and audio data, and is used to express each impression.
The impression map 50 represents an impression in three dimensions. The impression map 50 shows a tendency of being feminine or masculine in the X-axis direction. Further, the impression map 50 shows a tendency of being young or mature in the Y-axis direction, and shows a tendency of animation or real in the Z-axis direction. Then, the X-axis and the Y-axis are associated with classifications related to impressions such as "cute" and "boyish". The impression shown by the X-axis and the Y-axis is an example.
The impression data with the voice ID V001 shown in FIG. 2A is the classifier indicated by the position 51, and the impression data with the voice ID V002 is the classifier indicated by the position 52.

The image DB 33 of FIG. 1 is a database that stores image data and information on image features and impressions.
FIG. 2B shows an example of items and data included in the image DB 33.
The image DB 33 shown in FIG. 2B stores data corresponding to each item of part type, image data, image feature data, and impression data using the image ID as a key.
The image ID is identification information for uniquely identifying the image data.
The part type is the type of the part indicated by the image data. Types of parts include whole body, upper body, face, hairstyle, accessories, clothes, etc. The part type may be blank for the data indicating the basis of the virtual character such as the whole body and the upper body.
The image data is the data of the image itself.
The image feature data is tag information added by analyzing the image data. The image feature data is, for example, a hair color, a hairstyle, or the like. In the example of FIG. 2B, the color is indicated by characters such as black and brown, but for example, it may be represented by a numerical value such as an RGB value.
Impression data is data for identifying an impression determined from image data. In this example, the impression data is the coordinate data of the impression map 50 (FIG. 3) described above. For example, the impression data with the image ID I002 shown in FIG. 2B is the classifier indicated by the position 53 (see FIG. 3).

The generated character storage unit 34 of FIG. 1 is a storage area for storing the virtual character generated by the virtual character generation device 1. The generation character storage unit 34 stores, for example, one voice ID and one or more image IDs in association with each other. By doing so, it becomes clear that the generated virtual character is composed of image data and audio data.
The utterance content storage unit 35 is a storage area for storing the utterance content of the virtual character. The utterance content storage unit 35 stores the utterance content as, for example, a text.

The input unit 37 is, for example, an input device such as a keyboard or a mouse.
The display unit 38 is, for example, a display device such as an LCD (Liquid Crystal Display).
When the virtual character generation device 1 is a server, the input unit 37 and the display unit 38 may be various devices externally attached to the server.
The communication unit 39 is an interface for performing communication with an external device via the communication network N.
Here, the computer means an information processing device provided with a control unit, a storage device, and the like, and the virtual character generation device 1 is an information processing device provided with a control unit, a storage unit, and the like, and is included in the concept of a computer. ..

<Preparation process for virtual character generation device 1>
Next, the processing in the virtual character generation device 1 will be described.
First, a process of preparing the information necessary for generating a virtual character in the virtual character generation device 1 will be described.
FIG. 4 is a flowchart showing a voice DB construction process for constructing the voice DB 32 of the virtual character generation device 1 according to the present embodiment.
FIG. 5 is a flowchart showing an image DB construction process for constructing the image DB 33 of the virtual character generation device 1 according to the present embodiment.

First, in operating the virtual character generation device 1, it is necessary to prepare the voice DB 32 and the image DB 33 in advance. Therefore, the user who is constructing the virtual character generation device 1 prepares a plurality of voice data to be used as the voice of the virtual character in advance. In addition, the user prepares a plurality of image data to be used as an image of the virtual character in advance. As the image data, in addition to the image that is the center of the virtual character such as the whole body and the upper body of the virtual character, a plurality of images for each part to be decorated on the virtual character are prepared.

<Construction of voice DB 32>
When constructing the voice DB 32, the control unit 10 of the virtual character generation device 1 causes the display unit 38 to display an administrator screen (not shown) by the user's operation. Then, the user displays a screen (not shown) for constructing the voice DB 32 from the administrator screen, and inputs data according to the screen.

In step S (hereinafter, simply referred to as “S”) 11 of FIG. 4, the user inputs one voice data out of the prepared plurality of voice data via the input unit 37, thereby causing a virtual character generation device. The control unit 10 of 1 registers the input voice data in the voice DB 32. At that time, the control unit 10 randomly determines, for example, a unique voice ID and associates it with the voice data.
In S12, the user inputs the gender and age of the voice based on the registered voice data via the input unit 37, and the control unit 10 registers the gender and age in the voice DB 32 in association with the voice data. Here, the gender and age of the voice data to be registered are determined by the user.

In S13, the user plots the impression represented by the voice data on, for example, the impression map 50 (see FIG. 3) via the input unit 37. For example, the control unit 10 displays the impression map 50 on the screen. Then, the user operates the input unit 37 to plot the position of any of the impressions shown by the impression map 50. Here, the impression of the voice data to be registered is determined by the user.

In S14, the control unit 10 registers the three-dimensional coordinates, which are the plotted positions, as impression data in the voice DB 32 in association with the voice data. In this example, the impression data is stored as the position indicated by the impression map 50, but a text such as "cute" indicated at the plotted position may be registered.
In S15, the control unit 10 (voice feature amount calculation unit 18) calculates the feature amount of the voice data.
In S16, the control unit 10 registers the calculated feature amount of the voice data in the voice DB 32 in association with the voice data.

In S17, the control unit 10 determines whether or not to end the processing depending on whether or not the registration processing has been performed for all the voice data prepared in advance by the user. When the user performs an operation of closing the screen used for constructing the voice DB 32 when all the voice data is registered, the control unit 10 ends the process by receiving a signal corresponding to the operation of closing the screen. Then judge. When the registration process is performed for all the voice data prepared by the user (S17: YES), the control unit 10 ends this process. On the other hand, when all the prepared voice data has not been registered (S17: NO), the control unit 10 shifts the processing to S11 and performs the same processing on the unprocessed voice data.

<Construction of image DB33>
When constructing the image DB 33, the control unit 10 of the virtual character generation device 1 causes the display unit 38 to display an administrator screen (not shown) by the user's operation. Then, the user displays a screen (not shown) for constructing the image DB 33 from the administrator screen, and inputs data according to the screen.

In S21 of FIG. 5, the user inputs the image data of one of the prepared plurality of image data and the part type indicated by the image data via the input unit 37, thereby causing the virtual character generation device 1. The control unit 10 registers the image data and the part type in the image DB 33. At that time, the control unit 10 randomly determines, for example, a unique image ID and associates it with the image data.
Here, the image data may be a 3D image or a 2D image. Further, the image data includes a whole image showing the whole body of the virtual character and the like, and a partial image which is an image of a part of the virtual character such as a hairstyle, an accessory, and clothes.

In S22, the control unit 10 (image feature extraction unit 13) analyzes the image data registered in the image DB 33 and extracts the image feature data indicating the features related to the image. Then, the control unit 10 registers the extracted image feature data in the image DB 33.
In S23, the user plots the impression of the image data on, for example, the impression map 50 (see FIG. 3) via the input unit 37. Here, the impression of the image data to be registered is determined by the user.
In S24, the control unit 10 registers the three-dimensional coordinates, which are the plotted positions, as impression data in the image DB 33 in association with the image data.

In S25, the control unit 10 determines whether or not to end the processing depending on whether or not the registration processing has been performed for all the image data prepared in advance by the user. When the user performs an operation of closing the screen used for constructing the image DB 33 when all the image data is registered, the control unit 10 ends the process by receiving a signal corresponding to the operation of closing the screen. Then judge. When the registration process is performed for all the image data prepared by the user (S25: YES), the control unit 10 ends this process. On the other hand, when all the prepared image data has not been registered (S25: NO), the control unit 10 shifts the processing to S21 and performs the same processing on the unprocessed image data.

The voice DB 32 can be constructed by the voice DB construction process described above. Further, the image DB 33 can be constructed by the image DB construction process described above. Therefore, when the virtual character generation device 1 receives input data such as voice data or image data, the control unit 10 can perform processing using the constructed voice DB 32 and image DB 33.

<Virtual character generation process>
Next, the generation of a virtual character using the input data will be described.
FIG. 6 is a flowchart showing a virtual character generation process of the virtual character generation device 1 according to the present embodiment.
FIG. 7 is a flowchart showing a voice selection process of the virtual character generation device 1 according to the present embodiment.
FIG. 8 is a diagram for explaining the voice selection process of the virtual character generation device 1 according to the present embodiment.
FIG. 9 is a flowchart showing an image selection process of the virtual character generation device 1 according to the present embodiment.
FIG. 10 is a diagram for explaining an image selection process of the virtual character generation device 1 according to the present embodiment.

For example, when the user inputs the input data via the input unit 37, the control unit 10 (input data reception unit 11) of the virtual character generation device 1 receives the input data in S31 of FIG.
In S32, the control unit 10 determines whether or not the received input data is image data. In the case of image data (S32: YES), the control unit 10 shifts the processing to S33. On the other hand, when it is not image data, that is, when it is audio data (S32: NO), the control unit 10 shifts the processing to S34.
In S33, the control unit 10 (image processing unit 12) performs audio selection processing.

Here, the voice selection process will be described with reference to FIG.
In S41 of FIG. 7, the control unit 10 (image feature extraction unit 13) analyzes the image data and extracts the image feature data indicating the features related to the image.
For example, in the case of the image data 60 shown in FIG. 8, the image feature data extracted from the image data 60 by the control unit 10 is shown in Table 61. The image feature data shown in Table 61 represent features of each portion (part) constituting the image data 60.

In S42 of FIG. 7, the control unit 10 (attribute estimation unit 14) estimates the gender and age of the virtual character indicated by the image data based on the image feature data extracted in S41. As described above, the control unit 10 may recognize the face by using a known image recognition method, and further estimate the gender and age from the characteristics of each part of the face.
In S43, the control unit 10 (image impression specifying unit 15) refers to the image DB 33 and specifies the impression of the virtual character indicated by the image data. For example, the control unit 10 collates the image feature data extracted by S41 with each image feature data registered in the image DB 33, and the position of the impression data associated with the image feature data of the image DB 33 having a high degree of matching. Is specified as an impression of this image data.

In S44, the control unit 10 registers the image data received in the process of S31 of FIG. 6 in the image DB 33. At that time, the control unit 10 may register the part type as the whole body, the upper body, or the like based on the image feature data. Further, the control unit 10 determines and assigns a unique image ID.
By the process of S44, various image data are appropriately added to the image DB 33 and stored as a database. Therefore, the virtual character generation device 1 can register more image data in the image DB 33. The processing of S44 is arbitrary. If it is not desired to register the input data, the control unit 10 may not perform the process of S44.

In S45, the control unit 10 (voice selection unit 16) selects one voice data from the plurality of voice data stored in the voice DB 32 based on the impression specified by S43. The image data and the audio data specify the impression by the position of the impression data using the same impression map 50 (FIG. 3). Therefore, the control unit 10 can select audio data suitable for the image data by selecting audio data corresponding to the impression data at a position close to the coordinate position indicated by the impression data of the image data. After that, the control unit 10 shifts the processing to S35 of FIG.

On the other hand, in S34 of FIG. 6, the control unit 10 (audio processing unit 17) performs image selection processing.
Here, the image selection process will be described with reference to FIG.
In S51 of FIG. 9, the control unit 10 (voice feature amount calculation unit 18) calculates the feature amount of the voice data.
In S52, the control unit 10 (voice impression specifying unit 19) refers to the voice DB 32 and specifies the impression of the virtual character indicated by the voice data. For example, the control unit 10 collates the feature amount of the voice data extracted by S51 with the feature amount registered in the voice DB 32, and the position of the impression data associated with the feature amount of the voice DB 32 having a high degree of matching is determined. The impression to be shown is specified as an impression of this voice data.

In S53, the control unit 10 registers the voice data received in the process of S31 in FIG. 6 in the voice DB 32. At that time, the control unit 10 may register the gender and age of the voice DB 32 having similar feature amounts in S52 as the gender and age of the voice data. Further, the control unit 10 determines and assigns a unique voice ID.
By the process of S53, various voice data are appropriately added to the voice DB 32 and stored as a database. Therefore, more voice data can be registered in the voice DB 32. The processing of S53 is arbitrary. If it is not desired to register the input data, the control unit 10 may not perform the S53 process.

In S54, the control unit 10 (image selection unit 20) selects one image data from the plurality of image data stored in the image DB 33 based on the impression specified in the process of S52. Here, when the image DB 33 stores the image data for each part, the control unit 10 may select one image data from the plurality of image data for each part.

An image in which the parts type is the main (root) of the virtual character such as the whole body and the upper body is indispensable. Therefore, the control unit 10 may select the image data corresponding to the impression data at the position closest to the coordinate position indicated by the impression data of the voice data.
In addition, hairstyles and accessories are not essential parts. Therefore, the control unit 10 may select the image data only when the impression data of the image data is located in a predetermined range from the coordinate position indicated by the impression data of the audio data. FIG. 10A shows a set of hairstyle image data 71, and FIG. 10B shows a set of accessory image data 72.

Further, the image of clothes whose parts type is clothes changes according to the parts type of the main image. When the main image part type is the whole body, the image of the clothes is indispensable, but in the case of the upper body, for example, only the image of the clothes of the upper body is indispensable. Therefore, the control unit 10 may select image data corresponding to the main image. FIG. 10C shows a set 73 of image data of clothes.
After that, the control unit 10 shifts the processing to S35 of FIG.

In S35 of FIG. 6, the control unit 10 (virtual character generation unit 21) generates a virtual character using the data selected in the process of either S33 or S34 and the input data.
In S36, the control unit 10 registers the generated virtual character in the generated character storage unit 34. The control unit 10 may store the voice ID of one voice data and the image ID of one or more image data in association with each other in the generation character storage unit 34. After that, the control unit 10 ends this process.

By this processing, a virtual character using image data conforming to the voice data and a virtual character using the voice data conforming to the image data are generated, so that the generated virtual character does not easily feel a sense of discomfort in appearance and voice. Can be made.
When the virtual character is displayed on, for example, a terminal connected via the communication unit 39, the control unit 10 of the virtual character generation device 1 may, for example, utter the utterance content stored in the utterance content storage unit 35. The utterance data obtained by using the voice data for the data may be output to a terminal or the like. Further, the control unit 10 of the virtual character generation device 1 may output the image data with motion.

As described above, the virtual character generation device 1 of the present embodiment has the following effects.
(1) Select audio data that matches the image data related to the character and generate a virtual character. In addition, image data that matches the voice data related to the voice is selected to generate a virtual character. Therefore, the appearance of the virtual character can be made to have a high affinity with the voice, and it is possible to make it difficult to feel a sense of discomfort.

(2) Image feature data is extracted from the image data related to the character to specify the impression. Further, in the voice DB 32, an impression is associated with the voice data, and the voice data is selected by using the impression specified from the image data. Therefore, the impression of the image data constituting the virtual character and the voice data can be made close to each other. As a result, it is possible to make the appearance and voice of the virtual character less likely to feel uncomfortable.
(3) Image feature data is extracted from the image data related to the character, and the gender and age of the character are estimated. Further, in the voice DB 32, a gender and an age are associated with the voice data, and the voice data is selected using the gender and the age estimated from the image data. Therefore, the image data constituting the virtual character and the voice data can indicate the same gender and age. As a result, it is possible to make the appearance and voice of the virtual character less likely to feel uncomfortable.
(4) The impression of the image data related to the character is specified based on the degree of coincidence between the image feature data of the image data and the image feature data stored in the image DB 33. Therefore, the impression of the image data can be made by using the image data registered in the image DB 33. Therefore, the impression of the image data can be made similar to the accumulated data.

(5) The feature amount is calculated from the voice data related to the character to specify the impression. Further, in the image DB 33, an impression is associated with the image data, and the image data is selected by using the impression specified from the audio data. Therefore, the impression of the image data constituting the virtual character and the voice data can be made close to each other. As a result, it is possible to make the appearance and voice of the virtual character less likely to feel uncomfortable.
(6) The impression of the voice data related to the character is specified based on the degree of coincidence between the feature amount of the voice data and the feature amount stored in the voice DB 32. Therefore, the impression of the voice data can be made by using the voice data registered in the voice DB 32. Therefore, the impression of the voice data can be made similar to the accumulated data.
(7) In the image DB 33, image data is associated with a part type, and image data is selected for each part using the impression specified from the audio data. Then, the image data is generated by combining the image data for each selected part. Therefore, each part of the image data constituting the virtual character can have a unified impression.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the embodiments are merely a list of the most preferable effects arising from the present invention, and the effects according to the present invention are not limited to those described in the embodiments. The above-described embodiment and the modified form described later can be used in combination as appropriate, but detailed description thereof will be omitted.

(Transformed form)
(1) In the present embodiment, an example of receiving input data from an input unit has been described, but the present embodiment is not limited to this. For example, the control unit of the virtual character generation device may accept the input data by transmitting the input data from the terminal connected via the communication unit to the virtual character generation device. Further, the data before processing may be stored in the storage unit, and the data before processing stored in the storage unit may be accepted as input data.

(2) In the present embodiment, the impression is explained using a three-dimensional impression map as an example, but this is an example. There may be yet another dimension. Further, the items constituting each axis are also an example, and may be other items.

(3) In the present embodiment, the image data of the virtual character generated when the voice data is received is not specified in particular, but the voice data may be accepted including the designation. The designation means, for example, the range designation of the image to be displayed as a virtual character such as the whole body and the upper body. By doing so, the image data selected based on the impression of the audio data can be based on the designation. Further, although the main image of the virtual character has been described by taking the whole body and the upper body as an example, only the face image may be used.

1 Virtual character generation device 10 Control unit 11 Input data reception unit 13 Image feature extraction unit 14 Attribute estimation unit 15 Image impression specification unit 16 Voice selection unit 18 Voice feature amount calculation unit 19 Voice impression specification unit 20 Image selection unit 21 Virtual character generation Unit 30 Storage unit 31a Virtual character generation program 32 Voice DB
33 Image DB
37 Input unit 38 Display unit 39 Communication unit

Claims (8)

A virtual character generator that generates a virtual character composed of a combination of image data and audio data.
Input data receiving means for receiving input data and
A first storage unit that stores first data composed of image data or audio data and an impression classifier used to identify the impression of the first data.
A second storage unit that stores a plurality of second data to which impressions of different types from the first data are associated with each other.
With reference to the first storage unit, an impression specifying means for specifying an impression of the input data received by the input data receiving means, and an impression specifying means.
With reference to the second storage unit, a selection means for selecting the second data associated with the impression specified by the impression specifying means, and a selection means.
A virtual character generating means in which a virtual character is generated by the input data received by the input data receiving means and the second data selected by the selection means, and
A virtual character generator comprising. In the virtual character generation device according to claim 1,
The input data receiving means receives the image data of one virtual character as the input data, and receives the image data.
The second storage unit is a voice data storage unit in which the voice data and the impression matching the voice data are previously associated with each other.
A feature extraction means for analyzing the image data of the one virtual character received by the input data receiving means and extracting image feature data indicating features related to the image data of the one virtual character is provided.
The impression specifying means identifies the impression related to the image data of the one virtual character based on the image feature data extracted by the feature extracting means.
The selection means refers to the voice data storage unit and selects the voice data from the impression specified by the impression specifying means.
Virtual character generator. In the virtual character generation device according to claim 2.
In the voice data storage unit, the voice data and the impression are associated with each other for each gender and age.
An estimation means for estimating gender and age based on the image feature data extracted by the feature extraction means is provided.
The selection means refers to the voice data storage unit, and further selects the voice data based on the gender and the age estimated by the estimation means.
Virtual character generator. In the virtual character generator according to claim 2 or 3.
The first storage unit is a character feature storage unit that associates the image feature data of each virtual character with the impression.
The impression specifying means identifies the impression based on the degree of coincidence between the image feature data extracted by the feature extracting means and the image feature data stored in the character feature storage unit.
Virtual character generator. In the virtual character generation device according to claim 1,
The input data receiving means receives one voice data as the input data, and receives the input data.
The second storage unit is an image data storage unit in which image data representing at least a part of each virtual character and the impression matching the image data are associated in advance.
A feature amount calculating means for analyzing the one voice data received by the input data receiving means and calculating the feature amount of the one voice data is provided.
The impression specifying means identifies the impression related to the one voice data based on the feature amount of the voice data extracted by the feature amount calculating means.
The selection means refers to the image data storage unit and selects the image data from the impression specified by the impression specifying means.
Virtual character generator. In the virtual character generation device according to claim 5.
The first storage unit is a voice feature storage unit that associates the feature amount of the voice data with the impression.
The impression specifying means is based on the degree of coincidence between the feature amount of the one voice data calculated by the feature amount calculating means and the feature amount of the voice data stored in the voice feature storage unit. To identify,
Virtual character generator. In the virtual character generation device according to claim 5 or 6.
In the image data storage unit, the image data and the impression are associated with each part constituting each virtual character.
The selection means refers to the image data storage unit, and selects the image data from the impression specified by the impression specifying means for each part.
The virtual character generation means generates image data of one virtual character by combining the image data selected for each part by the selection means, and the generated image data and the one audio data are used. Generate a virtual character,
Virtual character generator. The program for operating a computer as the virtual character generation device according to any one of claims 1 to 7.

Images