JP2021086618A - Virtual person interaction system, video generation method, and video generation program - Google Patents

Abstract

To provide a virtual person interaction system for generating a video of a virtual person speaking, with a simple configuration, a video generation method using the virtual person interaction system, and a video generation program of the virtual person interaction system.SOLUTION: A virtual person interaction system 1 includes: a video model database 21 which stores multiple kinds of video models for motion of a person; a video model selection unit 315 which selects a video model to be used for generating a virtual person from data in the video model database; a video processing unit 31 which extracts face data of a virtual person to be generated, from a registered information source; a face insertion unit 316 which integrates the face data with the video model; a voice processing unit 32 which extracts sound from the information source to generate voice of the virtual person; and a video display processing unit 41 which generates a video of the virtual person speaking, on the basis of the video model integrated with the face data and the generated voice of the virtual person.SELECTED DRAWING: Figure 2

Description

The present invention relates to a virtual person dialogue system, a video generation method by the virtual person dialogue system, and a video generation program of the virtual person dialogue system.

In Patent Document 1, the face image data to be stored in the face recognition data memory is corrected based on the specified specific face image data and the face image data used for the correction process, and an angle other than the front is obtained. An imaging device that detects an individual's face even in an image of a person's face or a direction is disclosed.

In Patent Document 2, a sentence corresponding to the input sentence is selected from the conversation templates created in advance, and the selected sentence is processed based on the agent information of the virtual agent to generate a response sentence. , A conversational sentence generator is disclosed.

Japanese Unexamined Patent Publication No. 2011-76457 JP-A-2015-69455

In order to generate a video of a specific virtual person who does not actually exist, such as a deceased person or a celebrity, and to realize a realistic dialogue, a huge amount of information about the virtual person such as video, audio, and personality traits is required. Is. In addition, in order to integrate this information and generate a virtual person, it is necessary to purchase large-scale equipment and contents because video is generated using computer graphics etc., so it is not necessary to use it at the individual level. It was difficult. Therefore, there is a need for a system that can generate an utterance video of a virtual person with a simple configuration.

One of the objects of the present invention is to generate an utterance video of a virtual person with a simple configuration.

In order to achieve the above object, the virtual person dialogue system according to one aspect of the present invention generates a virtual person from a video model database that stores a plurality of types of video models in which a person operates and data in the video model database. A video model selection unit that selects the video model to be used for, a video processing unit that extracts the face data of the virtual person from the registered information source, and a face insertion unit that integrates the face data into the video model to be used. Based on the voice processing unit that extracts voice from the information source and generates the voice of the virtual person, the video model used in which the face data is integrated, and the generated voice of the virtual person. A video display processing unit that generates a video spoken by the virtual person is provided.

A personality model database that stores a plurality of personality models of a person and a question about the personality of the virtual person are presented, and based on the answer to the question, a personality model used to generate the virtual person is stored in the personality model database. A personality model selection unit that selects from data and a dialogue processing unit that generates a message spoken by the virtual person based on the usage personality model may be further provided.

The personality model selection unit may select the personality model based on the record created by the virtual person.

An input unit for inputting a question to the virtual person and an output unit for outputting the response of the virtual person are further provided, the dialogue processing unit generates a response to the question, and the output unit outputs the response. May be output.

A personality model correction unit that corrects the personality model based on the evaluation of the message may be further provided.

In order to achieve the above object, the image generation method according to another aspect of the present invention is a method of generating an image of a virtual person by a virtual person dialogue system provided with an image model database that stores a plurality of types of image models in which a person operates. And
A video model selection step of selecting a video model to be used for generating the virtual person from the data in the video model database, and a video processing step of extracting the face data of the virtual person to be generated from the registered information source. , A face insertion step that integrates the face data into the used video model, a voice processing step that extracts voice from the information source and generates a voice of the virtual person, and the used video model in which the face data is integrated. And a video display processing step of generating a video spoken by the virtual person based on the generated voice of the virtual person.

In order to achieve the above object, the video generation program according to still another aspect of the present invention generates a video of a virtual person by a virtual person dialogue system including a video model database that stores a plurality of types of video models in which a person operates. A computer program that selects a video model to be used to generate the virtual person from the data in the video model database, and a face data of the virtual person to be generated from the registered information source. The video processing command to be extracted, the face insertion command to integrate the face data into the video model used, the voice processing command to extract the voice from the information source and generate the voice of the virtual person, and the face data are integrated. A computer is made to execute a video display processing command for generating a video spoken by the virtual person based on the generated video model to be used and the generated voice of the virtual person.
The computer program can be provided by downloading via a network such as the Internet, or can be recorded and provided on various computer-readable recording media such as a CD-ROM.

According to the present invention, it is possible to generate an utterance video of a virtual person with a simple configuration.

It is a schematic block diagram of the virtual person dialogue system which concerns on this invention. It is a functional block diagram of the said virtual person dialogue system. FIG. 5 is a sequence diagram showing a process of determining a video model to be used by the virtual person dialogue system for generating a virtual person. It is a sequence diagram which shows the process which the said virtual person dialogue system generates a voice of a virtual person. It is a sequence diagram which shows the process which the said virtual person dialogue system determines a personality model of a virtual person. It is a sequence diagram which shows the process which a user interacts with a virtual person using the said virtual person dialogue system.

Hereinafter, the virtual person dialogue system, the image generation method, and the embodiment of the image generation program according to the present invention will be described with reference to the drawings.

● Overview of the virtual person dialogue system The virtual person dialogue system simulates the user's dialogue with the virtual person by playing the video and voice of a specific virtual person who is not actually there and automatically generating the utterance content. It is a system that can be done in a targeted manner. The person to be generated as a virtual person (hereinafter, also referred to as "target person") is assumed to be a person who has no or limited opportunity to speak due to space and time restrictions, such as a deceased person, a celebrity, a narrator such as a war-experienced person, etc. However, it can be any person. The virtual person is generated based on the information about the target person registered by the user and the model data described later. The virtual person is played back on the user terminal 10 (see FIG. 1) and operates, speaks, speaks to the user, and answers questions from the user as if it actually exists.

As shown in FIG. 1, the user U interacts with the virtual person K by communicating with the cloud computer C having a part or the whole configuration of the virtual person dialogue system via the user terminal 10. When the user U logs in to the cloud computer C via the user terminal 10 (step s1), the image of the virtual person K is transmitted from the cloud computer C (step s2). When the user U talks to the virtual person K (step s3), the cloud computer C analyzes the content of the input message, generates a response based on the predetermined character of the virtual person K, and generates a response, and the user terminal 10 Play back with the video above (step s4).

As shown in FIG. 2, in the virtual person dialogue system 1 (hereinafter, also referred to as “the present system 1”) according to the present invention, the storage device 20, the virtual person generation device 30, and the moving image generation device 40 are networked. It is configured by being connected via NW. The system 1 is connected to the user terminal 10 owned by the customer via a network NW, and can transmit and receive information to and from each other.

The connection between the user terminal 10, the storage device 20, the virtual person generation device 30, and the moving image generation device 40 may be wireless or wired, respectively. The storage device 20, the virtual person generation device 30, and the moving image generation device 40 may be composed of one device. Further, a part or all of the functions of the storage device 20, the virtual person generation device 30, and the moving image generation device 40 may be realized on the cloud computer C.

The user terminal 10 is a computer used by a user who interacts with a virtual person, and includes an input unit 11, an output unit 12, a display unit 13, an information source registration unit 14, and a communication processing unit 19. The user terminal 10 is, for example, a personal computer. Further, the user terminal 10 may be a smartphone or a tablet. The number of user terminals 10 connected to the system 1 may be singular or plural.

The input unit 11 is a functional unit for inputting a message from a user to a virtual person, and is composed of a keyboard, a touch panel display, a microphone, and the like.

The output unit 12 is a functional unit that outputs a message of a virtual person. The output unit 12 includes a display that displays a message in characters, a speaker that outputs a message by voice, and the like.

The display unit 13 of the user terminal 10 three-dimensionally reproduces an image of a virtual person such as a head-mounted display type VR display device and a hologram (stereoscopic image) display device in addition to a flat playback device such as a liquid crystal screen. It may be a device that does. According to the configuration in which the user terminal 10 is a device that reproduces the image of the virtual person in three dimensions, the dialogue with the virtual person can be made more realistic. Further, the display unit 13 may be a projection device capable of allowing a plurality of users to visually recognize an image of one virtual person at the same time.

The display unit 13 may be displayed by the UI unique to the system 1, or the system 1 may be linked with an existing chat tool such as SKYPE (registered trademark) to send a message or video from a virtual person to the existing tool. It may be displayed. According to this configuration, it is possible to obtain the feeling of chatting with an actual person, and it is possible to make the dialogue with the virtual person realistic.

The information source registration unit 14 is a functional unit that acquires information about the target person, that is, an information source of the target person. The information source includes, for example, moving images including the target person, still images and sound sources, recorded documents such as a diary created by the target person, documents expressing hobbies and tastes, and character data such as SNS. Information sources also include information about property such as clothing. The information source may be registered by the user or acquired through the Internet. The acquired information source is transmitted to the virtual person generation device 30.

The communication processing unit 19 is a functional unit that exchanges information with the system 1 via the network NW, and the communication format is arbitrary.

When the user registers the information of the target person through the user terminal 10, the virtual person generation device 30 processes the information to determine the image, voice, personality, and the like of the virtual person. The determined virtual person data is stored in the storage device 20, and is appropriately called by the moving image generation device 40. The moving image generation device 40 generates a moving image including a video, a voice, and a message of a virtual person based on the virtual person data, and displays the moving image on the user terminal 10.

● Configuration of storage device The storage device 20 includes a computing device such as a CPU (Central Processing Unit) for executing information processing, and a storage device such as RAM (Random Access Memory) and ROM (Read Only Memory). As software resources, it has at least a video model DB 21, a personality model DB 22, a virtual person data storage unit 23, and a communication processing unit 29. In this specification, "DB" is an abbreviation for "database".

The video model DB 21 is a storage unit that stores a plurality of types of video models in which a person operates. A video model is a video template used to generate an image of a virtual person. The video model is data that constitutes the shape and movement of the body in particular. Further, the video model is configured to integrate the face data described later and operate the integrated face data together with the image of the body.

The video model includes the appearances of a plurality of types of people having different physiques according to their height, weight, age, and the like. The video model includes multiple types of clothing that each person can wear and play. In addition, the video model contains various data on the movements of the person of each appearance, including data on movements that are often performed during dialogue, such as nodding, crossing arms, and raising hands. .. The video model may be a video of an actual person, a video modeled by CG, or both may be included.

The personality model DB 22 is a storage unit in which a plurality of types of personality models of a person are stored. The personality model includes, for example, the characteristics of the answer to the question, and determines the policy of the answer such as whether the content is positive or negative, and the emotions and emotions appearing in the answer. Further, the personality model is not limited to the answer to the question from the user, and may be the characteristic of the message according to the season, the time zone, and the like. In the personality model DB 22, answers to presumed questions corresponding to each personality model may be stored together. According to this configuration, it is possible to reduce the calculation processing load of generating a response according to the personality model for a standard question.

The virtual person data storage unit 23 is a storage unit that stores video model, personality model, and voice information determined for each virtual person. Further, the virtual person data storage unit 23 stores information known to the virtual person, for example, information such as an episode of the target person and an experience story. The virtual person data is determined and stored by the virtual person generation device 30. Further, the virtual person data is called by the moving image generation device 40 when playing back the moving image of the virtual person.

● Configuration of virtual person generation device The virtual person generation device 30 includes arithmetic units such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as RAM (Random Access Memory) and ROM (Read Only Memory). As a result, at least a video processing unit 31, a voice processing unit 32, a character processing unit 33, and a communication processing unit 39 are provided as software resources.

The video processing unit 31 is a functional unit that extracts appearance data used for generating a virtual person from the data of the target person. Appearance data is data including the face, body, hairstyle, clothes, etc. of the target person. Further, the video processing unit 31 selects a video model to be used for generating a virtual person, and determines video data to be used for the video of the virtual person. The video processing unit 31 may extract appearance data of a virtual person based on an information source acquired from the Internet in addition to an information source registered via the information source registration unit 14 of the user terminal 10. Good. Further, the video processing unit 31 may extract appearance data used for generating one virtual person based on information sources registered from a plurality of user terminals 10. When many users, such as celebrities, interact with a common virtual person, each user registers an information source for one virtual person. According to this configuration, a virtual person can be generated based on more information sources, and a more realistic dialogue becomes possible.

The video processing unit 31 includes a moving image acquisition unit 311, a still image acquisition unit 312, a trimming unit 313, an image correction unit 314, a video model selection unit 315, and a face insertion unit 316.

The moving image acquisition unit 311 is a functional unit that acquires moving image data. The moving image acquisition unit 311 acquires the moving image included in the information source registered in the user terminal 10. In addition, the moving image acquisition unit 311 can also urge the user to shoot a moving image through the user terminal 10. As a situation in which a moving image can be taken through the user terminal 10, for example, when the target person is a person close to the user and a virtual person is displayed on another user terminal 10, dialogue is possible even after the target person dies. Therefore, it is conceivable to generate a virtual person. In this case, the moving image acquisition unit 311 may display a tutorial for causing the user to shoot a moving image on the user terminal 10.

The still image acquisition unit 312 is a functional unit that acquires still image data. The still image acquisition unit 312 acquires a still image included in the information source registered in the user terminal 10. In addition, the still image acquisition unit 312 can also urge the user to shoot a still image through the user terminal 10. In this case, the still image acquisition unit 312 may display a tutorial for causing the user to take a still image, that is, a photograph, on the user terminal 10. Further, the still image acquisition unit 312 converts the moving image data into a still image and acquires it. The still image acquisition unit 312 extracts images of various angles of the target person and images of various facial expressions and converts them into still images.

The trimming unit 313 is a functional unit that trims and extracts the data of the target person from the still image. The trimming unit 313 may have a face recognition function and can automatically extract only the face of the target person.

The image correction unit 314 performs color tone correction and resolution correction of the extracted image to make the quality of the extracted image uniform. Further, the image correction unit 314 may determine whether or not the extracted image is clear, and may exclude the unclear image from the extracted data group. Further, the image correction unit 314 may exclude an image having a resolution of a predetermined value or less from the extracted data group.

The video model selection unit 315 is a functional unit that selects a video model to be used for generating a virtual person from the data in the video model DB 21. The video model selection unit 315 may select a video model most similar to the target person based on the appearance data acquired by the video acquisition unit 311, or presents a plurality of video models to the user terminal 10 to the user. You may let you select the video model to use. According to this configuration, it is possible to compose a video of a virtual person by a video model without sufficiently registering an information source showing how the virtual person is moving.

Further, the video model selection unit 315 may determine the clothes of the virtual person to be generated based on the appearance data or the possession information included in the information source. Further, the video model selection unit 315 may select the clothes of the virtual person from the video model DB 21. That is, if there is an information source in which the target person is dressed, the image of the virtual person can be generated based on the information source, and even if there is no information source of the target person, it is based on the property information. It is possible to generate an image of a virtual person. Further, since the clothes data can be selected from the video model DB 21, the virtual person can be easily generated even if the data related to the clothes of the target person is insufficient. The video model selection unit 315 may configure images of a virtual person wearing a plurality of types of clothes, and the clothes may be changed based on the time, time zone, or user's selection.

The video model selection unit 315 may determine the hairstyle of the virtual person to be generated based on the appearance data, or may select the hairstyle of the virtual person from the video model DB 21. Further, the image model selection unit 315 may configure an image of a virtual person having a plurality of types of hairstyles so that the hairstyle can be changed.

In the above description, the video processing unit 31 has been described on the assumption that the data of the virtual person is extracted based on the information source of the target person itself, but a moving image or a still image of a person similar to the target person has been described. May be newly photographed and used to generate a virtual person. In addition, appearance data of similar persons such as hairstyles and clothes may be partially used to generate a virtual person. That is, the user may be able to select the element of the appearance data used for generating the virtual person.

The face insertion unit 316 is a functional unit that integrates the face data extracted by the moving image acquisition unit 311, the still image acquisition unit 312, the trimming unit 313, and the image correction unit 314 into the video model to be used. The face insertion unit 316 integrates face data into the body composed of the video model used, and forms a full-body image of a virtual person.

The voice processing unit 32 is a functional unit that artificially generates a voice spoken by a virtual person. The voice processing unit 32 includes a voice extraction unit 321 and a voice generation unit 322.

The voice extraction unit 321 is a functional unit that extracts the voice of the target person from the information source. For example, the voice extraction unit 321 may identify the voice of the person who has been included for the longest time as the voice of the target person among the plurality of types of voices included in the information source.

The voice generation unit 322 is a functional unit that generates the voice of a virtual person based on the voice extracted by the voice extraction unit 321. The voice generation unit 322 may trim the voice of the target person and edit it so that it can be reproduced as the voice of a virtual person. Further, the voice generation unit 322 can also select a voice similar to the voice of the target person from the voice data prepared in advance and determine it as the voice of the virtual person. Further, the voice generation unit 322 may generate an artificial voice similar to the voice of the target person. When displaying a message from a virtual person as text, it is not necessary to generate voice.

The personality processing unit 33 is a functional unit that determines a personality model of a virtual person. The personality processing unit 33 includes a text data registration unit 331, a personality model selection unit 332, and a personality model correction unit 333.

The text data registration unit 331 is a functional unit that extracts text data from an information source and stores it in the virtual person data storage unit 23. The text data registration unit 331 extracts electronic text data such as a blog or SNS of the target person, and stores it in the virtual person data storage unit 23 according to a predetermined rule. Further, the text data registration unit 331 may read a handwritten document by the target person, for example, a diary, etc., convert it into text data, and store it in the virtual person data storage unit 23. Further, the text data registration unit 331 may convert the voice of the target person included in the voice or moving image data into text data and store it in the virtual person data storage unit 23.

The personality model selection unit 332 is a functional unit that selects a personality model (hereinafter, also referred to as a “usable personality model”) used for generating a virtual person from the personality model DB 22. The personality model selection unit 332 presents a question regarding the personality of the virtual person through the user terminal 10. When an answer to the question is input from the user terminal 10, the personality model used for generating the virtual person is selected from the data in the personality model DB 22 based on the answer.

Multiple personality questions may be asked. In addition, the question may be presented along a chart in which the input answer and the next question are linked. When the user answers the question, the basic personality of the virtual person is performed based on the basic classification of the personality prepared in advance. If the personality is based on the information of the actual conversation of the target person, a huge amount of information on the conversation is required. According to this system 1, it is possible to classify into one of the personalities prepared in advance based on the answer to the question about the personality, so even if the information is insufficient, the personality of the virtual person can be determined with a simple configuration. Can be done.

The personality model of the virtual person may be defined for each scenario pattern according to the type of question from the user. The scenario pattern is, for example, daily conversation or consultation for each problem. If the personality model is determined for some scenario patterns, it may be possible to have a dialogue according to the scenario pattern. According to this configuration, it is convenient because it is possible to have a dialogue by determining only the personality model related to the necessary scenario pattern.

The personality model correction unit 333 is a functional unit that corrects the personality model selected by the personality model selection unit 332. The personality model correction unit 333 receives an evaluation for the response made by the virtual person from the user terminal 10, and corrects the personality model based on the evaluation. For example, the user inputs as an evaluation whether or not the content is appropriate for the response of the target person in response to the response. In addition, the action of the virtual person performed with the response may be evaluated. The personality model correction unit 333 automatically learns by AI or the like to correct the personality model. According to this configuration, the character of the virtual person can be corrected to be closer to the target person. When a plurality of user terminals 10 interact with one virtual person at the same time or at different times, the evaluations from the plurality of user terminals 10 are used to correct the personality model of one virtual person. You may. According to this configuration, a lot of feedback can be given to the personality model of the virtual person, so that the personality model of the virtual person can be brought closer to the character of the target person and the dialogue accuracy can be improved.

Further, the personality model correction unit 333 may determine whether or not the message is suitable based on the user's response to the message from the virtual person instead of the evaluation from the user, and correct the personality model. The personality model correction unit 333 may convert the response content of the user into text data and analyze it, or may infer the satisfaction level from the voice of the user.

The communication processing unit 39 is a functional unit that communicates with the user terminal 10, the storage device 20, and the moving image generation device 40 through the network NW.

● Configuration of Movie Generation Device The movie generation device 40 is a device that displays a movie of a virtual person generated by the virtual person generation device 30 on the user terminal 10. The moving image generation device 40 includes a video display processing unit 41, an interactive processing unit 42, and a communication processing unit 49.

The video display processing unit 41 is a functional unit that generates an utterance video spoken by a virtual person. The video display processing unit 41 models the face data extracted from the appearance data and operates it according to the utterance.

The dialogue processing unit 42 is a functional unit that generates a message uttered by a virtual person based on the usage personality model. The content of the message may be a response to a question from the user, or may be a word generated according to the date, season, or time zone, or external information such as weather forecast or news on the Internet. .. Further, in replying to the user, in addition to the usage personality model, the reply may be generated based on the date, season, or time zone, or external information such as weather forecast and news on the Internet. The dialogue processing unit 42 determines the optimum answer by AI.

The message generated by the dialogue processing unit 42 is uttered by the voice generated by the voice processing unit 32, and is reproduced on the user terminal 10 together with the uttered video generated by the video display processing unit 41. As the voice of the virtual person, the dialogue of the target person extracted from the voice extraction unit 321 may be reproduced. Further, it may be reproduced based on the sound source data of a similar voice determined in advance. In addition, artificial audio may be generated and reproduced.

The communication processing unit 49 is a functional unit that communicates with the user terminal 10, the storage device 20, and the virtual person generation device 30 through the network NW.

● Flow of determining the video model to be used The flow of determining the video model to be used by the virtual person generation device 30 will be described with reference to FIG. As shown in the figure, first, the information source of the target person is registered from the user terminal 10 and transmitted to the virtual person generation device 30 (step S11). Next, the virtual person generation device 30 extracts appearance data from the information source (step S12). Of the appearance data, the moving image is converted into a still image (step S13). Next, the image of the target person is trimmed with respect to the registered still image and the still image converted from the moving image, and the color tone and resolution of the image are corrected (step S14). Cropping and image correction are in no particular order. At this time, if the quality of the data is still less than or equal to the predetermined value even after the correction, it may be decided not to use the image in the subsequent process.

Next, the virtual person generation device 30 stores the trimmed and image-corrected image in the virtual person data storage unit 23 of the storage device 20 (step S15). The virtual person generation device 30 refers to the video model stored in the video model DB 21 based mainly on the information about the physique of the stored images (step S16), and the video model most similar to the appearance of the target person. Is selected and displayed on the user terminal 10 (step S17). At this time, a plurality of video model candidates may be displayed on the user terminal 10, and the video model to be used may be selectable by the user terminal 10. Further, a video model different from the presented video model may be selectable by the user terminal 10.

Next, the user terminal 10 receives an input for individually changing the parts of the video model used (step S18). For the parts, selections for the contour, eyes, nose, mouth, etc., and the hairstyle and clothes of the virtual model may be input at this time. When the parts of the video model used are appropriately changed and the video model used by the virtual person is determined, the face data extracted from the appearance data is integrated into the video model used (step S19). Next, the used video model in which the face data is integrated is stored in the virtual person data storage unit 23 of the storage device 20 (step S20).

● Flow of generating a voice of a virtual person With reference to FIG. 4, a flow of generating a voice of a virtual person by the virtual person generation device 30 will be described. First, when the information source is registered from the user terminal 10 (step S21), the virtual person generation device 30 extracts the voice data of the target person from the information source (step S22). The virtual person generation device 30 generates a voice of a virtual person based on the voice data.

● Flow of Determining the Personality Model of a Virtual Person With reference to FIG. 5, a flow of determining a personality model of a virtual person by the virtual person generation device 30 will be described. When the information source is registered from the user terminal 10 (step S31), the virtual person generation device 30 extracts text data such as a blog or SNS from the information source (step S32). At this time, image data such as a handwritten diary is extracted and converted into text data. Furthermore, the sound source data is extracted and the voice of the target person is converted into text data. The extracted text data is stored in the virtual person data storage unit 23 based on a predetermined rule (step S33).

Next, the virtual person generation device 30 causes the user terminal 10 to display a question regarding the character of the target person (step S34). At this time, the content of the question may be determined based on the registered information source. Further, the user may be made to select the scenario pattern to be registered, and the question corresponding to the scenario pattern may be displayed. The user terminal 10 accepts the input of the answer to the question (step S35). At this time, a plurality of questions may be displayed at one time, or steps S34 and S35 may be repeated.

The virtual person generation device 30 refers to the personality model stored in the personality model DB 22 (step S36) based on the answer to the question about the personality, and determines the personality model to be used (step S37). Next, the determined usage personality model is stored in the virtual person data storage unit 23 (step S38).

● Flow of interacting with a virtual person With reference to FIG. 6, a flow of a user interacting with a virtual person using a virtual person dialogue system will be described. When the ID and password are input to the user terminal 10 (step S41), the virtual person generator 30 authenticates the user terminal 10 (step S42), and the user can interact with the virtual person associated with the ID. At this time, an effect such as an incoming chat from a virtual person, an incoming call, or an e-mail may be made. Next, the virtual person data storage unit 23 of the storage device 20 calls up the data of the virtual person to interact with, and the video generation device 40 makes it possible to refer to the data (step S43). That is, an image of a virtual person is displayed on the user terminal 10. The virtual person may speak or act when displayed.

When a question to the virtual person is input from the user terminal 10 (step S44), the moving image generation device 40 generates a moving image in which the virtual person responds based on the data of the virtual person.

Specifically, first, the moving image generation device 40 generates a response text to the question based on the personality model of the virtual person (step S45). In addition, the moving image generation device 40 generates a response voice that reproduces the response text in the voice of a virtual person (step S46). The response voice may be stored sound source data of the target person, or may be artificially generated artificial voice. Further, the moving image generation device 40 generates a response video to be reproduced when the response voice is reproduced (step S47). The generated response voice and response video are transmitted to the user terminal 10 as a response video (step S48). The response audio and the response video may be integrated and transmitted to the user terminal 10 as one data file, or each data file may be transmitted to the user terminal 10. Next, a moving image of the virtual person is displayed on the user terminal 10 (step S49). That is, the virtual person responds to the question from the user, and the dialogue with the virtual person is established. The steps from step S44 to step S49 may be repeated a plurality of times. With this configuration, it is possible to have a natural dialogue with a virtual person.

In FIG. 6, the flow of generating the video of the virtual person triggered by the input of the question to the user terminal 10 shown in step S44 has been described, but the virtual person is based on the predetermined date and time. The moving image may be generated and displayed on the user terminal 10. Further, the moving image may be generated based on external information from the Internet or the like, or the moving image may be generated based on a command from the administrator of the virtual person dialogue system 1. It may be configured to be displayed on the user terminal 10 as soon as the moving image is generated, or the moving image may be generated in advance and the user terminal 10 may be triggered by a question, date, time, external information or command from the user. It may be displayed in.

Following step S49, when the evaluation of the moving image is input from the user terminal 10 (step S50), the virtual person generation device 30 corrects the personality model and stores it in the virtual person data storage unit 23 of the storage device 20 (step S51). ).

As described above, according to the virtual person dialogue system according to the present invention, it is possible to generate an utterance video of a virtual person with a simple configuration.

1 Virtual person generation system 10 User terminal 20 Storage device 21 Video model DB
30 Virtual person generation device 31 Video processing unit 40 Video generation device 41 Video display processing unit

Claims (6)

A video model database that stores multiple types of video models in which humans operate,
From the data in the video model database, a video model selection unit that selects the video model to be used for generating a virtual person, and
A video processing unit that extracts the face data of the virtual person from the registered information source,
A face insertion unit that integrates the face data into the video model used, and
A voice processing unit that extracts voice from the information source and generates the voice of the virtual person,
A video display processing unit that generates a video spoken by the virtual person based on the used video model in which the face data is integrated and the generated voice of the virtual person.
A personality model database that stores multiple personality models of a person,
A personality model selection unit that selects a personality model to be used for generating the virtual person from data in the personality model database based on a record created by the person for which the virtual person is to be generated.
A dialogue processing unit that generates a message uttered by the virtual person based on the usage personality model, and
To prepare
Virtual person dialogue system.
The personality model selection unit presents a question regarding the personality of the virtual person, and selects a personality model to be used for generating the virtual person from the data in the personality model database based on the answer to the question.
The virtual person dialogue system according to claim 1.
An input unit for inputting a question to the virtual person and an output unit for outputting the response of the virtual person are further provided.
The dialogue processing unit generates a response to the question, and outputs the response from the output unit.
The virtual person dialogue system according to claim 1 or 2.
A personality model correction unit that corrects the usage personality model based on the evaluation of the message is further provided.
The virtual person dialogue system according to any one of claims 1 to 3.
It is a method of generating a virtual person's image by a virtual person dialogue system including a video model database that stores a plurality of types of video models in which a person operates and a personality model database that stores a plurality of person's personality models.
A video model selection step for selecting a video model to be used for generating the virtual person from the data in the video model database, and
A video processing step that extracts the face data of the virtual person to be generated from the registered information source, and
A face insertion step that integrates the face data into the video model used,
A voice processing step that extracts voice from the information source and generates the voice of the virtual person,
A video display processing step that generates a video spoken by the virtual person based on the used video model in which the face data is integrated and the generated voice of the virtual person.
A personality model selection step of selecting a personality model to be used for generating the virtual person from data in the personality model database based on a record created by the person for which the virtual person is to be generated, and a personality model selection step.
A dialogue processing step that generates a message spoken by the virtual person based on the usage personality model, and
including,
Video generation method.
A computer program that generates a virtual person's image by a virtual person dialogue system including a video model database that stores a plurality of types of video models in which a person operates and a personality model database that stores a plurality of person's personality models.
A video model selection command for selecting a video model to be used for generating the virtual person from the data in the video model database, and
A video processing instruction that extracts the face data of the virtual person to be generated from the registered information source,
A face insertion command that integrates the face data into the video model used, and
A voice processing instruction that extracts voice from the information source and generates the voice of the virtual person,
A video display processing instruction that generates a video spoken by the virtual person based on the used video model in which the face data is integrated and the generated voice of the virtual person.
A personality model selection command for selecting a personality model to be used for generating the virtual person from data in the personality model database based on a record created by the person for which the virtual person is to be generated.
An interactive processing instruction that generates a message spoken by the virtual person based on the usage personality model, and
Let the computer run
Video generation program.

Images