JP7496128B2 - Virtual person dialogue system, image generation method, and image generation program - Google Patents

Description

The present invention relates to a virtual person dialogue system, an image generation method using the virtual person dialogue system, and an image generation program for the virtual person dialogue system.

Patent Document 1 discloses an imaging device that corrects face image data to be stored in a memory for face recognition data based on specified specific face image data and face image data used in the correction process, and detects the face of an individual even in images taken from angles or directions other than the front.

Patent document 2 discloses a conversation sentence generation device that selects a sentence that corresponds to an input sentence from a conversation template created in advance, and processes the selected sentence based on the agent information of a virtual agent to generate a response sentence.

JP 2011-76457 A JP 2015-69455 A

In order to generate videos of specific virtual characters that do not actually exist, such as deceased people or celebrities, and to realize realistic dialogue, a huge amount of information about the virtual character, such as images, voice, and personality traits, is required. Furthermore, to integrate this information to generate a virtual character, images are generated using computer graphics, etc., which requires the purchase of large-scale equipment and content, making it difficult for individuals to use. Therefore, there is a need for a system that can generate talking videos of virtual characters with a simple configuration.

One of the objectives of the present invention is to generate speech images of a virtual person using a simple configuration.

To achieve the above object, a virtual person dialogue system according to one aspect of the present invention includes a video model database that stores multiple types of video models of human movements, a video model selection unit that selects a video model to be used to generate a virtual person from data in the video model database, a video processing unit that extracts facial data of the virtual person from a registered information source, a face insertion unit that integrates the facial data into the video model to be used, an audio processing unit that extracts audio from the information source and generates a voice of the virtual person, and a video display processing unit that generates an image of the virtual person speaking based on the video model to be used into which the facial data has been integrated and the voice of the generated virtual person.

The system may further include a personality model database that stores multiple personality models of characters, a personality model selection unit that presents questions about the personality of the virtual character and selects a personality model to be used to generate the virtual character from data in the personality model database based on the answers to the questions, and a dialogue processing unit that generates a message to be spoken by the virtual character based on the personality model to be used.

The personality model selection unit may select the personality model to be used based on a record created by the virtual person.

The system may further include an input unit for inputting a question to the virtual character, and an output unit for outputting a response from the virtual character, and the dialogue processing unit may generate a response to the question and cause the output unit to output the response.

The device may further include a personality model correction unit that corrects the usage personality model based on an evaluation of the message.

In order to achieve the above object, a video generation method according to another aspect of the present invention is a method for generating a video of a virtual person by a virtual person dialogue system having a video model database that stores a plurality of types of video models of human movements, the method comprising:
The method includes a video model selection step of selecting a usage video model to be used for generating the virtual person from data in the video model database, a video processing step of extracting facial data of the virtual person to be generated from a registered information source, a face insertion step of integrating the facial data into the usage video model, an audio processing step of extracting audio from the information source and generating a voice of the virtual person, and a video display processing step of generating an image of the virtual person speaking based on the usage video model into which the facial data has been integrated and the voice of the generated virtual person.

In order to achieve the above-mentioned object, an image generation program according to yet another aspect of the present invention is a computer program for generating an image of a virtual person using a virtual person dialogue system equipped with a video model database that stores multiple types of video models of human motion, and causes a computer to execute a video model selection command for selecting a usage video model to be used to generate the virtual person from data in the video model database, a video processing command for extracting facial data of the virtual person to be generated from a registered information source, a face insertion command for integrating the facial data into the usage video model, an audio processing command for extracting audio from the information source and generating a voice of the virtual person, and a video display processing command for generating an image of the virtual person speaking based on the usage video model into which the facial data has been integrated and the voice of the generated virtual person.
The computer program can be provided by downloading via a network such as the Internet, or can be provided by recording it on various computer-readable recording media such as a CD-ROM.

The present invention makes it possible to generate a video of a virtual person speaking using a simple configuration.

1 is a schematic configuration diagram of a virtual person dialogue system according to the present invention; FIG. 2 is a functional block diagram of the virtual person dialogue system. FIG. 11 is a sequence diagram showing a process in which the virtual person dialogue system determines a video model to be used for generating a virtual person. FIG. 4 is a sequence diagram showing a process in which the virtual character dialogue system generates a voice of a virtual character. FIG. 4 is a sequence diagram showing a process in which the virtual character dialogue system determines a personality model of a virtual character. FIG. 2 is a sequence diagram showing a process in which a user interacts with a virtual character using the virtual character interaction system.

Below, embodiments of the virtual person dialogue system, image generation method, and image generation program of the present invention will be described with reference to the drawings.

● Overview of the Virtual Person Dialogue System The virtual person dialogue system is a system that allows a user to virtually dialogue with a virtual person by playing back the video and voice of a specific virtual person who is not actually there and automatically generating the content of the speech. The person for which the virtual person is generated (hereinafter also referred to as the "target person") is assumed to be a person who has no or limited opportunities to speak due to location or time restrictions, such as a deceased person, a celebrity, or a narrator such as a war veteran, but may be any person. The virtual person is generated based on information about the target person registered by the user and model data to be described later. The virtual person is played on a user terminal 10 (see FIG. 1), and acts, speaks, talks to the user, and answers questions from the user as if it actually exists.

As shown in FIG. 1, a user U communicates with a cloud computer C, which comprises part or all of the configuration of a virtual person dialogue system, via a user terminal 10 to converse with a virtual person K. When the user U logs into the cloud computer C via the user terminal 10 (step s1), an image of the virtual person K is transmitted from the cloud computer C (step s2). When the user U speaks to the virtual person K (step s3), the cloud computer C analyzes the contents of the input message, generates a reply based on the predetermined personality of the virtual person K, and plays it back on the user terminal 10 together with the image (step s4).

As shown in FIG. 2, the virtual person dialogue system 1 according to the present invention (hereinafter also referred to as "this system 1") is configured by connecting a storage device 20, a virtual person generation device 30, and a video generation device 40 via a network NW. This system 1 is connected to a user terminal 10 owned by a customer via the network NW, and can send and receive information between them.

The user terminal 10, the storage device 20, the virtual person generation device 30, and the video generation device 40 may be connected to each other wirelessly or by wire. The storage device 20, the virtual person generation device 30, and the video generation device 40 may be configured as a single device. In addition, some or all of the functions of the storage device 20, the virtual person generation device 30, and the video generation device 40 may be realized on the cloud computer C.

The user terminal 10 is a computer used by a user who interacts with a virtual person, and includes an input unit 11, an output unit 12, a display unit 13, an information source registration unit 14, and a communication processing unit 19. The user terminal 10 is, for example, a personal computer. The user terminal 10 may also be a smartphone or a tablet. There may be one or more user terminals 10 connected to the system 1.

The input unit 11 is a functional unit that allows the user to input a message to the virtual character, and is composed of a keyboard, a touch panel display, a microphone, etc.

The output unit 12 is a functional unit that outputs a message from the virtual person. The output unit 12 is configured with a display that displays the message in text, a speaker that outputs the message as audio, etc.

The display unit 13 of the user terminal 10 may be a flat playback device such as an LCD screen, or a device that plays a virtual character image in three dimensions, such as a head-mounted display type VR display device or a hologram (three-dimensional image) display device. If the user terminal 10 is configured as a device that plays a virtual character image in three dimensions, it is possible to make the interaction with the virtual character more realistic. Furthermore, the display unit 13 may be a projection device that allows multiple users to view the image of one virtual character simultaneously.

The display unit 13 may display a UI unique to the present system 1, or the present system 1 may be linked to an existing chat tool such as SKYPE (registered trademark) so that messages and videos from the virtual person are displayed in the existing tool. With this configuration, it is possible to obtain the feeling of chatting with a real person, and to make the conversation with the virtual person feel realistic.

The information source registration unit 14 is a functional unit that acquires information about the target person, i.e., the information source of the target person. The information source includes, for example, videos, still images, and audio sources that include the target person, as well as record documents such as diaries created by the target person, documents expressing hobbies and preferences, and text data such as SNS. The information source also includes information about possessions such as clothing. The information source may be registered by the user or may be acquired via the Internet. The acquired information source is transmitted to the virtual person generation device 30.

The communication processing unit 19 is a functional unit that transmits and receives information to and from the system 1 via the network NW, and the communication format is arbitrary.

When a user registers information about a target person through the user terminal 10, the virtual person generation device 30 processes the information and determines the image, voice, personality, etc. of the virtual person. The determined virtual person data is stored in the storage device 20 and is called up as appropriate by the video generation device 40. The video generation device 40 generates a video including the image, voice, and message of the virtual person based on the virtual person data, and displays it on the user terminal 10.

Configuration of the storage device The storage device 20 comprises an arithmetic unit such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thus has, as software resources, at least an image model DB 21, a personality model DB 22, a virtual person data storage unit 23, and a communication processing unit 29. In this specification, "DB" is an abbreviation for "database."

The video model DB21 is a storage unit that stores multiple types of video models of human movements. A video model is a video template used to generate an image of a virtual person. A video model is data that configures, in particular, the shape and movement of the torso. In addition, the video model is configured to integrate face data, which will be described later, and move the integrated face data together with the image of the torso.

The video model includes the appearances of multiple types of people with different physiques depending on height, weight, age, etc. The video model includes multiple types of clothing that can be worn by each person and played back. Furthermore, the video model includes various data on the actions of people with each appearance, including data on actions that are often performed during conversation, such as nodding, folding arms, and raising a hand. The video model may be footage of an actual person, or it may be footage modeled using CG, or it may include both.

The personality model DB22 is a storage unit that stores multiple types of personality models for people. The personality model includes, for example, characteristics of answers to questions, and determines the answer policy, such as whether the content is positive or negative, and the emotions expressed in the answer. Furthermore, the personality model is not limited to answers to questions from users, but may also be characteristics of messages according to seasons, time periods, etc. The personality model DB22 may also store responses to questions that are expected in advance and are in line with each personality model. With this configuration, the computational burden of generating responses in accordance with the personality model to standard questions can be reduced.

The virtual character data storage unit 23 is a storage unit that stores information on the image model, personality model, and voice determined for each virtual character. The virtual character data storage unit 23 also stores information known to the virtual character, such as episodes and personal experiences of the target character. The virtual character data is determined and stored by the virtual character generation device 30. The virtual character data is also called up by the video generation device 40 when playing a video of the virtual character.

●Configuration of the virtual character generation device The virtual character generation device 30 comprises a calculation device such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and thereby comprises at least a video processing unit 31, an audio processing unit 32, a personality processing unit 33, and a communications processing unit 39 as software resources.

The video processing unit 31 is a functional unit that extracts appearance data used to generate a virtual person from the data of the target person. The appearance data includes the face, body, hairstyle, clothing, etc. of the target person. The video processing unit 31 also selects a video model to be used to generate the virtual person and determines the video data to be used for the image of the virtual person. The video processing unit 31 may extract the appearance data of the virtual person based on an information source registered via the information source registration unit 14 of the user terminal 10, as well as an information source obtained from the Internet. The video processing unit 31 may also extract appearance data to be used to generate one virtual person based on information sources registered from multiple user terminals 10. When many users, such as celebrities, interact with a common virtual person, each user registers an information source for one virtual person. With this configuration, a virtual person can be generated based on more information sources, enabling more realistic interactions.

The video processing unit 31 has a video acquisition unit 311, a still image acquisition unit 312, a trimming unit 313, an image correction unit 314, a video model selection unit 315, and a face insertion unit 316.

The video acquisition unit 311 is a functional unit that acquires video data. The video acquisition unit 311 acquires videos included in an information source registered in the user terminal 10. The video acquisition unit 311 can also prompt the user to shoot a video through the user terminal 10. Examples of situations in which a video can be shot through the user terminal 10 include a case where the target person is a person close to the user and a virtual person is displayed on another user terminal 10, or a case where a virtual person is generated so that the target person can be interacted with even after the target person has passed away. In this case, the video acquisition unit 311 may display a tutorial on the user terminal 10 to encourage the user to shoot a video.

The still image acquisition unit 312 is a functional unit that acquires still image data. The still image acquisition unit 312 acquires still images contained in an information source registered in the user terminal 10. The still image acquisition unit 312 can also prompt the user to take still images through the user terminal 10. In this case, the still image acquisition unit 312 may cause the user terminal 10 to display a tutorial for encouraging the user to take still images, i.e., photographs. The still image acquisition unit 312 also converts video data into still images and acquires them. The still image acquisition unit 312 extracts images of the target person from various angles and images of various facial expressions, and converts them into still images.

The trimming unit 313 is a functional unit that trims and extracts data of a target person from a still image. The trimming unit 313 may have a face recognition function and be capable of automatically extracting only the face of the target person.

The image correction unit 314 performs color correction and resolution correction on the extracted images to make the quality of the extracted images uniform. The image correction unit 314 may also determine whether the extracted images are clear or not, and exclude unclear images from the extracted data group. The image correction unit 314 may also exclude images with a resolution below a predetermined level from the extracted data group.

The video model selection unit 315 is a functional unit that selects the video model to be used to generate a virtual person from the data in the video model DB 21. The video model selection unit 315 may select the video model that is most similar to the target person based on the appearance data acquired by the video acquisition unit 311, or may present multiple video models to the user terminal 10 and allow the user to select the video model to use. With this configuration, a video of a virtual person can be created using a video model even if a sufficient number of information sources showing the virtual person moving are not registered.

The video model selection unit 315 may determine the clothes of the virtual person to be generated based on the appearance data or on the possession information included in the information source. The video model selection unit 315 may select the clothes of the virtual person from the video model DB 21. That is, if there is an information source in which the target person is wearing that clothing, it is possible to generate an image of the virtual person based on the information source, and even if there is no information source of the target person, it is possible to generate an image of the virtual person based on the possession information. Since the clothing data can be selected from the video model DB 21, even if the data on the clothing of the target person is insufficient, it is possible to easily generate a virtual person. Note that the video model selection unit 315 may compose an image of a virtual person wearing multiple types of clothing, and the clothing may be changed based on the season, time period, or user selection.

The video model selection unit 315 may determine the hairstyle of the virtual person to be generated based on the appearance data, or may select the hairstyle of the virtual person from the video model DB 21. Furthermore, the video model selection unit 315 may compose videos of virtual people with multiple types of hairstyles, and the hairstyle may be changeable.

In the above explanation, it has been assumed that the video processing unit 31 extracts data of a virtual character based on the information source of the target person himself/herself. However, it is also possible to take new videos or still images of a person who resembles the target person and use them to generate a virtual character. Also, it is also possible to partially use the appearance data of a similar person, such as hairstyle or clothing, to generate a virtual character. In other words, the user may be able to select the elements of the appearance data to be used to generate a virtual character.

The face insertion unit 316 is a functional unit that integrates the face data extracted by the video acquisition unit 311, the still image acquisition unit 312, the trimming unit 313, and the image correction unit 314 into the used video model. The face insertion unit 316 integrates the face data into the torso that is configured by the used video model, and creates a full-body image of the virtual person.

The voice processing unit 32 is a functional unit that artificially generates the speaking voice of a virtual person. The voice processing unit 32 includes a voice extraction unit 321 and a voice generation unit 322.

The voice extraction unit 321 is a functional unit that extracts the voice of a target person from an information source. For example, the voice extraction unit 321 may identify the voice of the person that is included for the longest period of time among multiple types of voices included in the information source as the voice of the target person.

The voice generation unit 322 is a functional unit that generates the voice of the virtual character based on the voice extracted by the voice extraction unit 321. The voice generation unit 322 may trim the voice of the target character and edit it so that it can be played back as the voice of the virtual character. The voice generation unit 322 may also select a voice similar to the voice of the target character from voice data prepared in advance and determine it as the voice of the virtual character. Furthermore, the voice generation unit 322 may generate an artificial voice similar to the voice of the target character. Note that if a message from the virtual character is displayed as text, voice generation may not be necessary.

The personality processing unit 33 is a functional unit that determines the personality model of the virtual person. The personality processing unit 33 includes a text data registration unit 331, a personality model selection unit 332, and a personality model correction unit 333.

The text data registration unit 331 is a functional unit that extracts text data from an information source and stores it in the virtual person data storage unit 23. The text data registration unit 331 extracts electronic text data such as the target person's blog or SNS, and stores it in the virtual person data storage unit 23 according to a predetermined rule. The text data registration unit 331 may also read a handwritten document by the target person, such as a diary, convert it into text data, and store it in the virtual person data storage unit 23. Furthermore, the text data registration unit 331 may convert the voice of the target person contained in audio or video data into text data, and store it in the virtual person data storage unit 23.

The personality model selection unit 332 is a functional unit that selects a personality model (hereinafter also referred to as the "used personality model") to be used in generating a virtual character from the personality model DB 22. The personality model selection unit 332 presents a question about the personality of the virtual character via the user terminal 10. When an answer to the question is input from the user terminal 10, the personality model to be used in generating the virtual character is selected from the data in the personality model DB 22 based on the answer.

Multiple personality questions may be presented. Questions may also be presented along a chart that links input answers to the next question. As the user answers the questions, a basic personality assignment is made for the virtual character based on basic personality classifications prepared in advance. If personality assignment were to be made from information about the target character's actual conversation, a huge amount of conversation information would be required. With this system 1, the answers to personality questions can be classified into one of the prepared personalities, so that the personality of the virtual character can be determined with a simple configuration even if information is insufficient.

The personality model of the virtual character may be determined for each scenario pattern according to the type of question from the user. The scenario pattern may be, for example, daily conversation or consultation about a problem. Once a personality model is determined for some scenario patterns, a configuration may be made that allows dialogue in accordance with the scenario pattern. This configuration is simple, since dialogue can be carried out by determining only the personality model for the necessary scenario pattern.

The personality model correction unit 333 is a functional unit that corrects the personality model used selected by the personality model selection unit 332. The personality model correction unit 333 receives an evaluation of the response made by the virtual person from the user terminal 10, and corrects the personality model used based on the evaluation. For example, the user inputs an evaluation of whether the response was appropriate for the target person. The user may also evaluate the virtual person's actions that accompany the response. The personality model correction unit 333 performs automatic learning using AI or the like and corrects the personality model. With this configuration, the personality of the virtual person can be corrected to be closer to the target person. Note that when multiple user terminals 10 simultaneously or at different times have a conversation with one virtual person, the evaluations from the multiple user terminals 10 may be used to correct the personality model of one virtual person. With this configuration, a lot of feedback can be given to the personality model of the virtual person, so that the personality model of the virtual person can be made closer to the personality of the target person, and the accuracy of the conversation can be improved.

The personality model correction unit 333 may also determine whether a message from a virtual person was appropriate based on the user's response to the message, rather than on the user's evaluation, and correct the personality model. The personality model correction unit 333 may convert the content of the user's response into text data and analyze it, or may infer the level of satisfaction from the user's tone of voice.

The communication processing unit 39 is a functional unit that communicates with the user terminal 10, the storage device 20, and the video generating device 40 through the network NW.

Configuration of the Moving Image Generating Device The moving image generating device 40 is a device that displays a moving image of the virtual person generated by the virtual person generating device 30 on the user terminal 10. The moving image generating device 40 includes an image display processing unit 41, a dialogue processing unit 42, and a communication processing unit 49.

The video display processing unit 41 is a functional unit that generates a speech video of a virtual person speaking. The video display processing unit 41 performs modeling processing on the face data extracted from the appearance data, and makes it move in accordance with the speech.

The dialogue processing unit 42 is a functional unit that generates a message to be spoken by the virtual person based on the usage personality model. The content of the message may be a response to a question from the user, or may be words generated based on external information such as the date, season, or time of day, or weather forecasts and news on the Internet. Furthermore, when replying to the user, a response may be generated based on external information such as the date, season, or time of day, or weather forecasts and news on the Internet, in addition to the usage personality model. The dialogue processing unit 42 determines the optimal response using AI.

The message generated by the dialogue processing unit 42 is spoken by a voice generated by the audio processing unit 32, and is played on the user terminal 10 together with the spoken image generated by the image display processing unit 41. The voice of the virtual character may be the lines of the target character extracted from the audio extraction unit 321. It may also be played based on sound source data of a similar voice determined in advance. Furthermore, an artificial voice may be generated and played.

The communication processing unit 49 is a functional unit that communicates with the user terminal 10, the storage device 20, and the virtual person generation device 30 via the network NW.

● Flow of determining the video model to be used The flow of the virtual person generation device 30 determining the video model to be used will be described with reference to FIG. 3. As shown in the figure, first, the information source of the target person is registered from the user terminal 10 and transmitted to the virtual person generation device 30 (step S11). Next, the virtual person generation device 30 extracts appearance data from the information source (step S12). Among the appearance data, moving images are converted into still images (step S13). Next, the registered still images and still images converted from the moving images are trimmed to the target person's images, and the color tone and resolution of the images are corrected (step S14). The trimming and image correction can be performed in any order. At this time, if the quality of the data is below a certain level even after correction, it may be determined that the image is not to be used in the subsequent process.

The virtual person generation device 30 then stores the trimmed and corrected image in the virtual person data storage unit 23 of the storage device 20 (step S15). Based on information about the physique of the stored images, the virtual person generation device 30 refers to the video models stored in the video model DB 21 (step S16), selects the video model that most resembles the appearance of the target person, and displays it on the user terminal 10 (step S17). At this time, multiple candidates for the video model may be displayed on the user terminal 10, and the user terminal 10 may be able to select the video model to be used. Also, the user terminal 10 may be able to select a video model different from the presented video model.

Next, the user terminal 10 accepts input to change individual features of the used video model (step S18). Features include each of the contours, eyes, nose, and mouth, and at this time, a selection regarding the hairstyle and clothing of the virtual model may also be input. Once the features of the used video model have been appropriately changed and the used video model of the virtual person has been finalized, face data extracted from the appearance data is integrated into the used video model (step S19). Next, the used video model with the integrated face data is stored in the virtual person data storage unit 23 of the storage device 20 (step S20).

Flow of generating a voice of a virtual character The flow of generating a voice of a virtual character by the virtual character generation device 30 will be described with reference to Fig. 4. First, when an information source is registered from the user terminal 10 (step S21), the virtual character generation device 30 extracts voice data of the target character from the information source (step S22). The virtual character generation device 30 generates the voice of the virtual character based on the voice data.

● Flow of determining a personality model of a virtual character The flow of determining a personality model of a virtual character by the virtual character generation device 30 will be described with reference to Fig. 5. When an information source is registered from the user terminal 10 (step S31), the virtual character generation device 30 extracts text data such as blogs and SNS from the information source (step S32). At this time, image data such as handwritten diaries is also extracted and converted into text data. Furthermore, sound source data is extracted and the voice of the target character is converted into text data. The extracted text data is stored in the virtual character data storage unit 23 based on a predetermined rule (step S33).

Next, the virtual person generation device 30 causes the user terminal 10 to display questions about the personality of the target person (step S34). At this time, the content of the questions may be determined based on the information source to be registered. Alternatively, the user may be allowed to select a scenario pattern to be registered, and questions corresponding to the scenario pattern may be displayed. The user terminal 10 accepts input of answers to the questions (step S35). At this time, multiple questions may be displayed at once, or steps S34 and S35 may be repeated.

Based on the answers to the questions about personality, the virtual person generation device 30 refers to the personality models stored in the personality model DB 22 (step S36) and determines the personality model to be used (step S37). The determined personality model to be used is then stored in the virtual person data storage unit 23 (step S38).

Flow of interacting with a virtual character A flow of interacting with a virtual character using the virtual character interaction system will be described with reference to FIG. 6. When an ID and a password are input to the user terminal 10 (step S41), the virtual character generation device 30 authenticates the user (step S42), and the user can interact with the virtual character associated with the ID. At this time, the virtual character may send an incoming chat message, a phone call, or an e-mail. Next, data of the virtual character to be interacted with is called from the virtual character data storage unit 23 of the storage device 20, and the data is made available for reference by the video generation device 40 (step S43). That is, an image of the virtual character is displayed on the user terminal 10. The virtual character may speak or move when displayed.

When a question is input to the virtual character from the user terminal 10 (step S44), the video generating device 40 generates a video in which the virtual character responds based on the data of the virtual character.

Specifically, first, the video generating device 40 generates a response text to the question based on the personality model of the virtual person (step S45). The video generating device 40 also generates a response voice that reproduces the response text in the voice of the virtual person (step S46). The response voice may be stored audio data of the target person, or may be an artificial voice that is artificially generated. Furthermore, the video generating device 40 generates a response video that is reproduced when reproducing the response voice (step S47). The generated response voice and response video are transmitted to the user terminal 10 as a response video (step S48). Note that the response voice and response video may be integrated and transmitted to the user terminal 10 as a single data file, or each data file may be transmitted to the user terminal 10. Next, a video of the virtual person is displayed on the user terminal 10 (step S49). That is, the virtual person responds to the question from the user, and a dialogue with the virtual person is established. The process from step S44 to step S49 may be repeated multiple times. This configuration allows for a natural dialogue with the virtual person.

In FIG. 6, the flow of generating a video of a virtual person triggered by inputting a question to the user terminal 10 shown in step S44 has been described, but the configuration may be such that a video of a virtual person is generated when a specific date or time arrives and is displayed on the user terminal 10. The video may also be generated based on external information from the Internet, etc., or based on instructions from an administrator of the virtual person dialogue system 1. The video may be displayed on the user terminal 10 immediately after it is generated, or the video may be generated in advance and displayed on the user terminal 10 in response to a question from the user, a date, a time, external information, or an instruction, etc.

Following step S49, when an evaluation of the video is input from the user terminal 10 (step S50), the virtual person generation device 30 corrects the personality model and stores it in the virtual person data storage unit 23 of the storage device 20 (step S51).

In this way, the virtual character dialogue system of the present invention can generate a speech video of a virtual character with a simple configuration.

1 Virtual person generation system 10 User terminal 20 Storage device 21 Video model DB
30 Virtual person generation device 31 Video processing unit 40 Video generation device 41 Video display processing unit

Claims (6)

A video model database that stores a plurality of video models of human movements;
a video model selection unit that selects a video model to be used for generating a virtual person from the data in the video model database;
a face insertion unit for integrating face data of the virtual person into the usage video model;
a voice processing unit for generating a voice of the virtual person;
a video display processing unit that generates a video of the virtual person speaking based on the usage video model into which the face data is integrated and the generated voice of the virtual person;
A personality model database that stores a plurality of personality models of a person;
a personality model selection unit that selects a personality model to be used for generating the virtual person from data in the personality model database;
a dialogue processor that generates a message to be spoken by the virtual person based on the usage personality model;
a personality model correction unit that corrects the usage personality model based on evaluations of the message transmitted from a plurality of terminals that can receive the message;
Equipped with
the dialogue processing unit determines emotions to be expressed in a message to be spoken by the virtual person based on the usage personality model, and generates the message including the emotions.
Virtual person dialogue system.
the personality model selection unit selects the personality model to be used from data in the personality model database based on a record created by a person who is a target of generating the virtual character;
The virtual person interaction system according to claim 1 .
the personality model selection unit, as the user answers the questions, assigns a basic personality to the virtual character based on a basic personality classification prepared in advance, and classifies a personality model to be used for generating the virtual character into one of the personalities included in the personality model database;
3. A virtual person dialogue system according to claim 1 or 2.
The system further includes an input unit for inputting a question to the virtual person, and an output unit for outputting a response from the virtual person,
the dialogue processor generates a response to the question, and causes the output unit to output the response;
4. A virtual person dialogue system according to claim 1.
A method for generating an image of a virtual person by a virtual person dialogue system including an image model database storing a plurality of types of image models of human actions, and a personality model database storing a plurality of personality models of people, comprising:
a video model selection step of selecting a video model to be used for generating the virtual person from data in the video model database;
a face insertion step of integrating face data of the virtual person into the usage video model;
a voice processing step for generating a voice for said virtual character;
a video display processing step of generating a video of the virtual person speaking based on the usage video model into which the face data has been integrated and the generated voice of the virtual person;
a personality model selection step of selecting a personality model to be used for generating the virtual person from data in the personality model database;
a dialogue processing step of generating a message to be spoken by the virtual person based on the usage personality model;
a personality model correction step of correcting the usage personality model based on evaluations of the message transmitted from a plurality of terminals capable of receiving the message;
Including,
In the dialogue processing step, emotions expressed in a message uttered by the virtual person are determined based on the usage personality model, and the message including the emotions is generated.
Image generation method.
A computer program for generating an image of a virtual person by a virtual person dialogue system including an image model database storing a plurality of types of image models of human actions, and a personality model database storing a plurality of personality models of people, the computer program comprising:
a visual model selection command for selecting a visual model to be used in generating the virtual person from data in the visual model database;
a face insertion command for integrating face data of the virtual person into the usage video model;
voice processing instructions for generating a voice for said virtual character;
a video display processing command for generating a video in which the virtual person speaks based on the usage video model into which the face data has been integrated and the generated voice of the virtual person;
a personality model selection command for selecting a personality model to be used in generating the virtual person from data in the personality model database;
A dialogue processing command for generating a message to be spoken by the virtual person based on the usage personality model;
a personality model correction command for correcting the usage personality model based on evaluations of the message transmitted from a plurality of terminals capable of receiving the message;
Run the following on your computer:
The dialogue processing command determines emotions to be expressed in a message to be spoken by the virtual person based on the usage personality model, and generates the message including the emotions.
Video generation program.

Images