JP2001357413A - Animation conversation system and server to be used for it - Google Patents

Abstract

PROBLEM TO BE SOLVED: To remotely speak with an actual or virtual person imaged by a computer. SOLUTION: A client 2 is provided with a device for inputting the first message for the virtual person, etc., from a user, a means for transmitting it, a means for receiving the second message and the face animation of the person, etc., for the user from the person, etc., an output device 22b for outputting the second message and a display device 22a for displaying the face animation. A server 3 is provided with a face picture database, a means for receiving the first message, a means for generating the second message, a means for generating operation controlling data for controlling the timing of outputting the second message and the operation of face picture data to be coincident, a means for generating the face animation based on the operation control data and the face picture data and a means for transmitting them.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an animation conversation system that enables a computer to converse with a virtual or imaginary person or the like.
The present invention relates to an animation conversation system that allows a user to access a server from a client via a network and to have a remote conversation.

[0002]

2. Description of the Related Art In recent years, a technology capable of communicating with a real or fictional person, animal, doll, or character virtualized by a computer has been developed.
Researched and developed.

[0003] For example, Japanese Patent Application Laid-Open No. H11-212934 discloses a technique of operating a mouse or a keyboard or the like to input a command to a living creature bred in a virtual space and performing a predetermined operation. Have been.
According to this, a computer performs the same care as a real pet, such as feeding a virtual pet or sleeping, praising, scolding, or playing the virtual pet. With such care, the virtual pet grows, and the user confirms the growth of the virtual pet while confirming the growth of the virtual pet by an image or sound output from a display device or a speaker.
You can experience pet breeding. It is also possible to breed virtual pets remotely via a network.

[0004] A method of matching the output timing of the sound and image of a living body is disclosed in, for example, Japanese Patent Application Laid-Open No. 10-31246.
No. 7, there is a method of outputting a sound in accordance with an image, and a method of outputting an image in accordance with a sound described in JP-A-10-293860. By these methods,
Since the animation in which the image operates can be created and the sound can be output in synchronization with the animation, the user can feel the output image and the sound in a realistic manner. In addition, as a method of creating an animation based on a live-action video, there is an animation synthesis technique based on recognition of a live-action video (NTT Technical Journal 1998. 1).
2 p98-106). This technology automatically creates a caricature from a single photo, and based on the caricature, automatically creates facial expressions with different eyes, mouth open / closed states, and emotions, emotions, and sorrows. This is a method of synthesizing animation.

[0005]

SUMMARY OF THE INVENTION The above-mentioned Japanese Patent Application Laid-Open No.
In the technology of -212934, communication with a virtual pet can be remotely performed via a network. However, in this conventional technique, a virtual pet is controlled by a command input by a user from an input device and is displayed on a display surface, and communication patterns that the user can take with the virtual pet are limited. I have. For example, they cannot talk to a virtual pet and cannot communicate with reality.

The animation creation technology described in Japanese Patent Application Laid-Open No. H10-312467 can create, for example, an animation of a motion when a certain person speaks. This is only output, and the user and the person cannot have a conversation.

[0007] It is an object of the present invention to provide a system, a server, and a client capable of remotely communicating with a real or fictional person or the like virtualized by a computer.

[0008]

A system according to the present invention is an animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer, and a conversation with such a person or the like can be performed. An input device for inputting a first message, which is a message from a user to the person or the like, a means for transmitting the first message, the client or the server, Means for receiving a second message, which is a message to the user, and a face animation of the person or the like, an output device for outputting the second message to the user, and a display device for displaying the face animation. The server stores face image data of the person or the like; and the first message. Means for receiving a message, means for generating the second message, means for generating operation control data for operating face image data in accordance with the second message, and the operation control data and the face image. Means for generating the face animation based on the data and means for transmitting the second message and the face animation are provided.

In another embodiment, the face image data is transmitted from the server to the client, and the client has means for generating a face animation based on the received face image data.

As still another form, a storage means for storing face image data is provided in a client, and the face image data is extracted from the storage means in the client. As the face image data, a three-dimensional model structured such that the head of the person or components such as the mouth, eyes, nose, and ears, and the skin, muscle, skeleton, and the like can operate is preferably used. Various real or fictional characters such as actors, singers, other entertainers or celebrities such as stars, athletes, politicians, ancestors of users, and historical characters can be used as the characters. In addition, not only a person but also an animal, a doll, or a cartoon character can be used.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following three embodiments, various persons HMN are virtualized by a computer,
An animation conversation system 1 in which a user can have a conversation with a person HMN selected from among them will be described as an example. [First Embodiment] FIG. 1 is a block diagram showing the overall configuration of an animation conversation system 1 according to a first embodiment of the present invention, and FIG.
7 is a diagram showing an example of a program stored in the magnetic disk device 37 of the server 3, FIG. 3 is a diagram showing an example of a program stored in the magnetic disk device 37 of the server 3, and FIG. FIG. 5 showing an example of
Is a diagram illustrating an example of a list LST of persons HMN.

As shown in FIG. 1, an animation conversation system 1 includes a client 2, a server 3, a network 4, and the like. The client 2 includes a processing device 21, a display device 22a, a speaker 22b, a mouse 2
3a, a keyboard 23b, a microphone 23c, a communication control device 24, a CD-ROM device 25, a floppy disk device 26, a magnetic disk device 27, and the like.

The processing device 21 includes a CPU 21a, a RAM 2
1b, a ROM 21c, and the like, and executes a series of processing in the client. A program or data is temporarily stored in the RAM 21b.
The M21c stores programs and setting information of client hardware. The CPU 21a executes those programs.

The display device 22a displays an animation of the face of the person HMN, and outputs character data TXT2 to be described later to the user. The speaker 22b outputs audio data SND2 described later to the user as audio. The mouse 23a and the keyboard 23b are used for inputting a first message MG1 that is a message from the user to the person HMN, operating the client 2, and the like.
Microphone 23c inputs first message MG1 by voice.

The communication control device 24 sends the first message M
G1, a second message MG2 that is a message from the person HMN to the user, transmission and reception of face animation data FAD described later, and other data are controlled. CD-R
The OM device 25, the floppy disk device 26, and the magnetic disk device 27 store data and programs.

The server 3 includes a processing device 31 and a display device 3
2, mouse 33a, keyboard 33b, communication control device 3
4, CD-ROM device 35, floppy disk device 3
6 and a magnetic disk device 37.

The processing device 31 includes a CPU 31a, a RAM 3
1b, a ROM 31c, and the like. The configuration and functions of the processing device 31 itself are the same as those of the processing device 21 described above. The communication control device 34 controls transmission and reception of the first message MG1, the second message MG2, the face animation data FAD, and other data.

The network 4 includes a public line, a dedicated line,
It is configured by an AN, a wireless line, the Internet or the like. The client 2 and the server 3 are connected via a network 4.

The first message MG1 is transmitted to the microphone 23c.
, Or character data TXT1 input from the keyboard 33b or the like. The second message MG2 is audio data SND2, character data TXT2, or the like. The face animation data FAD is information of a face animation composed of images of a continuous motion of the face of the person HMN.

As shown in FIG. 2, the magnetic disk device 27 of the client 2 includes an OS 2s which is a basic program of the client 2, a client conversation program 2p which is an application program on the client side of the animation conversation system 1, and a client conversation program 2p. Is stored. The client conversation program 2p performs basic operation processing 2b
s, perform other processing. The basic operation process 2bs is executed by the OS
This is a process for performing processing relating to the selection of the person HMN, cooperation with 2s, input processing of the first message MG1, and the like. These programs and data are appropriately
It is loaded on the AM 21b and executed by the CPU 21a.

As shown in FIG. 3, an OS which is a basic program of the server 3 is installed in the magnetic disk device 37 of the server 3.
3s, server conversation program 3 which is an application program on the server side of animation conversation system 1
p, data 3d as information necessary for the server conversation program 3p and the like are stored.

The server conversation program 3p includes a basic operation processing 3bs, a language recognition conversation engine EG1, and an animation engine EG2. The basic operation process 3bs is a process for coordinating with the OS 3s and controlling the language recognition conversation engine EG1 and the animation engine EG2.

The language recognition conversation engine EG1 is a system that performs a language recognition process 3gn and a conversation generation process 3ki, and is known per se. The language recognition process 3gn
This is a process of analyzing the voice data SND1 and extracting character data TXTa expressed in a natural language such as Japanese or English. The conversation generation process 3ki is a process of generating the voice data SND2 or the character data TXT2.

In order to create the voice data SND2, voice data of the person or a substitute person is obtained in advance for each person HMN. Based on the voice data, voice synthesis is performed by a conversation generation process 3ki.

The animation engine EG2 performs an operation control process 3ds and an animation generation process 3an. The operation control processing 3ds generates operation control data DSD. The operation control data DSD refers to the face image data F of the person HMN in accordance with the timing at which the second message MG2 is output to the speaker 22b or the display device 22a.
This is control information for controlling the GD to operate.
The animation generation processing 3an includes the operation control data DS
This is a process of generating face animation data FAD based on D and face image data FGD.

These programs are appropriately stored in the RAM 31
b and is executed by the CPU 31a. Also, the first message MG used in these processes
1. Face image data FGD, second message MG2, operation control data DSD, and face animation data F
AD and the like are also temporarily stored in the RAM 31b as needed.

As shown in FIG. 4, the magnetic disk drive 37
Is provided with a face image database FDB, a person information database HDB, and a conversation database KDB.

The face image data FGD is stored in the face image database FDB, and each person H is stored in the person information database HDB.
Person information HMJ, which is information such as gender, personality, age, etc., relating to the MN is stored in the conversation database KDB, and grammar for generating a conversation sentence and sentence information BNJ and word information TNJ, which are words, are stored.

The face image data FGD expresses the head of the person HMN by a three-dimensional model structured such that constituent parts such as mouth, eyes, nose, and ears, and skin, muscle, and skeleton can operate. This is the data obtained (see FIG. 8). Various real or fictional characters such as actors, singers, other entertainers or celebrities such as stars, athletes, politicians, ancestors of users, and historical characters can be used as the person HMN. In addition, not only a person but also an animal, a doll, or a cartoon character can be used.

Such face image data FGD can be created by the following various known methods. For example, first, three-dimensional data is obtained by the following method. (1) A method of estimating a structured face image based on a normal face photograph taken in two dimensions. (2) A method of calculating a three-dimensional shape from a plurality of two-dimensional images and data indicating a positional relationship between a camera and a subject which has projected the two-dimensional images (stereo imaging method). (3) A method of three-dimensionally measuring a person or an image using a three-dimensional measuring device. (4) A method of newly creating a three-dimensional CG character.

Next, the obtained three-dimensional data is converted into a structured three-dimensional model. As a conversion method, for example, JP-A-8-297751, JP-A-11-3284
For example, a method disclosed in Japanese Patent Application No. 40-200 or Japanese Patent Application No. 2000-90629 proposed by the present applicant can be used.

Thus, for example, a three-dimensional model of the skin is created. Then, it is more preferable that the three-dimensional skin model is further structured by attaching a muscle and a skeleton. As shown in FIG. 5, the list LST is a database storing a plurality of persons HMN with which the user has a conversation, and corresponds to a person number NUM for identifying each person HMN and the person number NUM. Person name N, which is the name of the person
It has a field such as a sample image SMP showing an example of an AM or a face image. Data about a plurality of persons HMN, such as a person HMN1 and a person HMN2, is stored.

Next, processing and operations performed in the animation conversation system 1 when a conversation with the person HMN is performed will be described with reference to a flowchart. FIG. 6 shows an animation conversation system 1 according to the first embodiment.
7 is a flowchart showing a flow of a process for generating the face animation data FAD and the second message MG2, and FIG. 8 is a diagram showing an example of the face image data FGD1.

As shown in FIG. 6, first, the client 2
, The person HMN to be a conversation partner is
a or the keyboard 23b to select from the list LST (# 11). At this time, the selected person HMN
Is transmitted to the server 3. The list LST is stored in the server 3 via the network 4.
May be provided in advance, may be stored in the magnetic disk device 27 in advance as shown in FIG.
It may be provided by a medium such as an M or a floppy disk.

The server 3 generates an animation of the person HMN before the conversation starts. First, face image data FG corresponding to the received data of the person number NUM
D and the person information HMJ are stored in the face image database FDB.
And extracted from the person information database HDB (#
12).

Next, based on the extracted face image data FGD and the person information HMJ, face animation data FAD is generated (# 13) and transmitted to the client 2 (# 14). In the client 2, the received face animation data FAD is displayed on the display device 22a as the initial state of the person HMN (# 15).

A second message MG2 is generated together with the generation of the face animation data FAD, transmitted to the client 2, the face animation data FAD is displayed on the client 2, and the second message MG2 is output from the speaker 22b. May be.

The method of generating the face animation data FAD and the second message MG2 will be described later. The user is the person HMN displayed on the display device 22a.
And talk to the person HMN. That is, in the client 2, the first message MG1 is input by the microphone 23c, the keyboard 23b, or the like,
The input first message MG1 is transmitted to the server 3 (# 16).

Steps # 13 to # 15 may be omitted, and the user may ask first. In the server 3, the next face animation data FAD and the second message MG2 are generated based on the received first message MG1, face image data FGD, person information HMJ, and the like (# 17), and the generated data is It is transmitted to the client 2 (# 18).

In the client 2, the face animation data FAD and the second message MG2 are output by the display device 22a or the speaker 22b (# 19).

If a disconnection request for canceling the conversation with the person HMN has been issued (Yes in # 20), the process ends, and if no disconnection request has been issued, step # 16 is performed.
And the conversation (dialogue) between the user and the person HMN is repeated.

Here, step # 13 and step #
A method of generating an animation or the like performed in step 17 will be described. Face image data FGD used in the present embodiment
Is a three-dimensional model in which components such as the mouth, eyes, nose, and ears, and skin, muscle, and skeleton are operably structured.

The face image data FGD1 shown in FIG. 8 shows a three-dimensional model of the skin. The three-dimensional skin model is composed of a large number of polygons forming the skin of the face (head) of the person HMN, and a plurality of control points PNT for controlling the movement of the face.

As shown in FIG. 7, the server 3 recognizes the received first message MG1 (# 3).
1). When the first message MG1 is composed of the character data TXT1, it is not necessary to execute the language recognition processing 3gn. When the first message MG1 is composed of the voice data SND1, the language recognition processing engine 3 executes language recognition processing 3gn to generate character data TXTa. However, if the first message has not been received as in step # 13, or if the conversation has been interrupted for a predetermined time, step # 31 is omitted.

To reply to the first message MG2, a second message MG2 is generated. That is, the conversation generation processing 3ki is performed by the language recognition conversation engine EG1.
Is performed to generate character data TXT2 (# 32), and audio data SND2 is generated from the generated character data TXT2 (# 33).

The generation of character data TXT2 is performed with reference to character data TXTa or TXT1, text information BNJ, word information TNJ, and the like. For example, when the character data TXTa or TXT1 is a question “How are you?”, The person H
MN extracts sentence information BNJ likely to respond to such a question from the conversation database KDB, and applies word information TNJ to sentence information BNJ. , Are you all right? "
Generate

From character data TXT2 to audio data SND
A known technique may be used for conversion to 2. However, when the first message is not received as in step # 13 or when the conversation is interrupted for a predetermined time, in step # 32, the personal information HMJ, the text information BNJ, the word information TNJ, etc. are referred to, The character data TXT2 that the person HMN asks the user is generated. For example, "Hello", and the like, "What 'was."

By the animation engine EG2,
The motion control data DSD is generated (# 34), and the face animation data FAD is generated (# 35). The operation control data DSD is obtained by executing the operation control processing 3ds.

For example, Japanese Unexamined Patent Publication No.
Using the technology of No. 0-293860, face image data F
GD can be synchronized with audio data SND2.
Then, by executing the animation generation processing 3an, the face image data FGD is operated based on the operation control data DSD, and the face animation data FAD is generated.

In the case of the face image data FGD1 shown in FIG. 8, the face image data FGD is operated by controlling the control point PNT. In order to transmit the face animation data FAD, compression may be performed using various encoding methods such as MPEG.

As described above, in the first embodiment, the face animation data FAD is generated by the server 3 and transmitted to the client 2, so that the client 2 only needs to receive and display it, and the performance is improved. Alternatively, even if the client 2 has difficulty in generating animations due to low specifications, conversation with the person HMN can be performed. [Second Embodiment] In the second embodiment, the overall configuration of the animation conversation system is the same as in the first embodiment. However, the programs stored in the magnetic disk devices 27 and 37 of the client 2 and the server 3 and the processing contents of the processing devices 21 and 31 are different from those of the first embodiment.

That is, in the first embodiment, the server 3
The face image data FGD extracted from the face image database FDB is temporarily stored in the RAM 31b of the server 3 or the magnetic disk device 37. However, in the second embodiment,
The face image data FGD is transmitted to the client 2 and is temporarily stored in the RAM 21 b or the magnetic disk device 27 of the client 2. Then, the client 2 generates the face animation data FAD based on the operation control data DSD transmitted from the server 3.

FIG. 9 is a diagram showing an example of programs stored in the magnetic disk device 27 in the second embodiment, and FIG. 10 is a diagram showing programs and the like stored in the magnetic disk device 37 in the second embodiment. It is a figure showing an example.

In these figures, parts having the same functions as in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted or simplified. The same applies hereinafter. As shown in FIG. 9, the magnetic disk device 27 stores an animation generation process 3an, which is a means for generating an animation of the face of the person HMN, and the face image data F transmitted from the server 3.
GD and operation control data DSD are stored.

As shown in FIG. 10, the magnetic disk drive 3
The server conversation program 3p stored in 7 includes a basic operation processing 3bs, a language recognition conversation engine EG1, and an animation engine EG2, as in the first embodiment. However, animation engine E
G2 performs the operation control processing 3ds, but does not perform the animation generation processing.

Next, processing and operations performed by the animation conversation system 1 when conducting a conversation with the person HMN will be described with reference to flowcharts. FIG.
Is a flowchart showing a processing flow of the animation conversation system 1 in the second embodiment, and FIG. 12 is a flowchart showing a processing flow of generating the operation control data DSD and the second message MG2.

As shown in FIG. 11, the client 2 selects a person HMN to be a conversation partner from the list LST (# 41). At this time, the selected person HMN
Is transmitted to the server 3. The server 3 that has received the person number NUM reads the corresponding face image data FGD from the face image database FDB,
The data is transmitted to the client 2 (# 42). Such preprocessing for conversation is automatically performed as background processing.

In the server 3, the operation control data DSD
Is generated (# 43) and transmitted to the client 2 (# 44). In the client 2, the face image data F
The GD is operated based on the operation control data DSD, thereby generating the face animation data FAD and, at the same time, displaying it on the display device 22a (# 45).

The server 3 generates the second message MG2 together with the generation of the operation control data DSD, transmits them to the client 2, and transmits the second message MG2 to the client 2.
May display the face animation data FAD and output the second message MG2 from the speaker 22b.

In the client 2, the first message MG1 is input and transmitted to the server 3 (# 4).
6). In the server 3, based on the first message MG1 and the person information HMJ, the operation control data DS
D and the second message MG2 are generated (# 4
7), the generated data is transmitted to the client 2 (# 48).

Then, the face image data FGD is output to the display device 22a while operating based on the operation control data DSD, and the second message MG2 is output to the display device 22a or the speaker 22b in synchronization with the operation (# 4).
9).

Until the disconnection request is issued, the user and the person H
The conversation with the MN is repeated (# 46 to # 50). Here, a method of generating operation control data and the like performed in steps # 43 and # 47 will be described with reference to FIG.

The server 3 recognizes the received first message MG1 (# 61). Character data TXT
2 (# 62), and the generated character data TXT2
Generates the voice data SND2 from the second message M
G2 is generated (# 63). Then, the operation control data DSD is generated by the animation engine EG2 (# 64).

As described above, in the second embodiment, the face image data FGD extracted by the server 3 is transmitted to the client 2, and the operation control data DS
The animation is generated by operating the face image data FGD based on D. As a result, the amount of data communication between the server 3 and the client 2 is reduced, and a high-speed animation can be displayed. [Third Embodiment] In the third embodiment, the overall configuration of the animation conversation system is the same as in the second embodiment. The contents of the programs stored in the magnetic disk devices 27 and 37 are almost the same as those of the second embodiment shown in FIGS. However, each of the magnetic disk devices 27 and 37 of the client 2 and the server 3
Is different from the case of the second embodiment,
The processing contents are slightly different.

That is, in the second embodiment, the face image database FDB is also provided in the client 2, and the client 2 performs extraction and temporary storage of the face image data FGD and generation of face animation data. The server 3 generates the operation control data DSD and the second message MG2 based on the first message MG1 transmitted from the client 2.

FIG. 13 is a diagram showing an example of a database and the like provided in the magnetic disk device 27 of the client 2 and the magnetic disk device 37 of the server 3 in the third embodiment.

As shown in FIG. 13, the face image database FDB is not provided in the magnetic disk device 37 of the server 3, but is provided only in the magnetic disk device 27 of the client 2.

The processing content of the animation conversation system 1 in the third embodiment is almost the same as the flowchart in FIG. 11 for the second embodiment. Next, only the differences will be described.

In step # 42 of FIG. 11, the face image data FGD is
7 and is temporarily stored as it is. It does not send to others. Other processes are the same.

As described above, in the third embodiment, the face image database FDB is provided in the magnetic disk device 2 of the client 2, so that it is not necessary to transmit the face image data FGD from the server 3, and until the start of conversation. Can be shortened.

As in the above-described three embodiments, the server 3 generates the second message MG2, thereby reducing the processing load on the client 2 and having a remote conversation with the person HMN. be able to.

Since the face image data FGD is three-dimensionally structured, facial movements and emotional expressions are rich and natural. Since the face animation that understands the user's speech responds with the emotion expression and the sound expressed by the three-dimensional image, the user can enjoy the interactive conversation.

By selecting the person HMN,
A dialogue service with a historical person or a deceased relative can be realized. For example, by selecting “ancestor” as the person HMN from the face image database FDB, the user can enjoy daily conversation with the face animation of the deceased ancestor.

If the person HMN is a real celebrity,
A conversation between a celebrity and many fans can be realized without disturbing the real life of the celebrity. In the language recognition conversation engine EG1, by setting the conversation content according to the type of the person HMN, such as an ancestor, a celebrity, a historical person, etc., a conversation with the content between the user and the person HMN is made. It can be performed.

Further, by keeping the server 3 running at all times, the user does not need to ask the time and place,
A conversation with the person HMN can be enjoyed. Further, since the maintenance of the conversation database KDB can be performed by the server 3, the client 2 can easily cope with the latest topics and popular words without performing special maintenance.

In the embodiment described above, the person HM
N is generated by the server 3, but only the character data TXT2 is generated by the server 3 and the voice data SND is generated.
2 may be generated by the client 2.

In the embodiment described above, a workstation or a personal computer can be used as the server 3 and the client 2. Further, as the client 3, a device having a communication function,
For example, a mobile phone, a mobile device, and various other devices can be used.

In addition, the configuration, circuit, processing contents, processing order, conversation contents, etc. of the whole or each part of the animation conversation system 1 can be appropriately changed in accordance with the gist of the present invention.

[0079]

According to the present invention, a conversation with a virtual or real person or a virtual person can be remotely performed by a computer. Since the second message is generated by the server, the processing load on the client can be reduced.

According to the second aspect of the present invention, the amount of data communication can be reduced, and a high-speed animation can be displayed. According to the third aspect of the present invention, the time required until the start of the conversation can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the overall configuration of an animation conversation system according to the present invention.

FIG. 2 is a diagram showing a program stored on a client side in the first embodiment.

FIG. 3 is a diagram showing a program stored in a server according to the first embodiment.

FIG. 4 is a diagram showing a database provided in a magnetic disk device of the server.

FIG. 5 is a diagram showing an example of a list of persons.

FIG. 6 is a flowchart illustrating a flow of processing of the animation conversation system according to the first embodiment.

FIG. 7 is a flowchart illustrating a flow of processing for generating face animation data and a second message.

FIG. 8 is a diagram illustrating an example of face image data.

FIG. 9 is a diagram showing a program stored on the client side in the second embodiment.

FIG. 10 is a diagram illustrating a program stored in a server according to the second embodiment.

FIG. 11 is a flowchart illustrating a flow of processing of the animation conversation system according to the second embodiment.

FIG. 12 is a flowchart illustrating a flow of processing for generating operation control data and a second message.

FIG. 13 is a diagram illustrating databases stored in respective magnetic disk devices of a client and a server according to the third embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Animation conversation system 2 Client 21 Processing device 22a Display device (output device) 22b Speaker (output device) 23a Mouse (input device) 23b Keyboard (input device) 23c Microphone (input device) 24 Communication control device 3 Server 31 Processing device 34 Communication control device FDB Face image database (storage means) EG1 Language recognition conversation engine (means for generating second message) EG2 Animation engine (means for generating operation control data, means for generating face animation) 3an animation generation processing ( Means for generating a face animation) 4 Network HMN Person MG1 First message MG2 Second message FAD Face animation data

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/14 G10L 3/00 571U F-term (Reference) 5B050 BA08 BA12 CA07 CA08 EA24 FA10 5C064 AA06 AB04 AC05 AC06 AC12 AC16 AC17 AD08 AD09 AD13 5D015 KK02 KK04 LL06 LL08 5D045 AA20 AB26

Claims (8)

[Claims] 1. An animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be made, comprising: a client and a server; An input device for inputting a first message, which is a message from the user to the person or the like, to the client,
Means for transmitting the first message, means for receiving a second message that is a message from the person or the like to the user and face animation of the person or the like, and an output device that outputs the second message to the user And a display device for displaying the face animation, wherein the server has storage means for storing face image data of the person or the like, and means for receiving the first message,
Means for generating the second message; means for generating operation control data for operating face image data in accordance with the second message; and generating the face animation based on the operation control data and the face image data. An animation conversation system, comprising: means for generating; and means for transmitting the second message and the face animation. 2. An animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be made, comprising: a client and a server; An input device for inputting a first message, which is a message from the user to the person or the like, to the client,
Means for transmitting the first message, an output device for outputting a second message, which is a message from the person or the like to the user, to the user, and the second message, the face of the person or the like as image data Means for receiving face image data and operation control data for operating the face image data in accordance with the second message, and means for generating a face animation of the person or the like based on the operation control data and the face image data And a display device for displaying the face animation. The server includes a storage unit that stores the face image data, a unit that receives the first message, and generates the second message. Means, means for generating the operation control data, and means for transmitting the second message and the operation control data. An animated conversation system characterized by: 3. An animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be made, comprising: a client and a server; The client has a storage unit that stores face image data of the person or the like, an input device that inputs a first message that is a message from the user to the person or the like, and a unit that transmits the first message. An output device that outputs to the user a second message that is a message from the person or the like to the user, and an operation of operating the second message, the face image data, and the face image data in accordance with the second message Means for receiving control data; and the person or the like based on the operation control data and the face image data. Means for generating a face animation,
A display device for displaying the face animation is provided; the server includes a unit that receives the first message, a unit that generates the second message, and a unit that generates the operation control data. Means for transmitting the second message and the operation control data, the system comprising: an animation conversation system; 4. A server used in an animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be performed. Storage means for storing face image data, means for receiving a first message that is a message from a user to the person or the like, means for generating a second message that is a message from the person or the like to the user, Means for generating operation control data for operating the face image data in accordance with the output of the second message; means for generating the face animation based on the operation control data and the face image data; Means for transmitting a message and the face animation. 5. A server used in an animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be performed. Storage means for storing face image data, means for receiving a first message that is a message from a user to the person or the like, means for generating a second message that is a message from the person or the like to the user, Means for generating operation control data for operating the face image data in accordance with the output of the second message; and means for transmitting the second message and the operation control data. Server to do. 6. A server used in an animation conversation system in which a real or imaginary person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be performed. Means for receiving a first message that is a message to the like, means for generating a second message that is a message from the person or the like to the user, and the person or the like according to the output of the second message A server comprising: means for generating operation control data for operating face image data; and means for transmitting the second message and the operation control data. 7. A client used in an animation conversation system in which a real or fictional person, animal, doll, character, or the like is virtualized by a computer so that a conversation with the person or the like can be performed. An input device that inputs a first message that is a message to the like, a unit that transmits the first message, and an output device that outputs a second message that is a message from the person or the like to the user to the user Means for receiving the second message, face image data obtained by converting the face of the person or the like into image data, and operation control data for operating the face image data in accordance with the output of the second message; Means for generating a face animation of the person or the like based on data and the face image data; Clients characterized by comprising a, a display device for displaying the animation. 8. The face image data is a three-dimensional model structured to be operable, and the means for generating a face animation converts a structured part of the three-dimensional model based on the motion control data. The system, server, or client according to any of claims 1, 2, 3, 4, or 7.