JP2003108502A - Physical media communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication system for allowing closer communication when the subject and a mate separated in time or a distance have a conversation. SOLUTION: In this physical media communication system, subject and mating communication terminals 101 and 102 exchange subject and mating voice while displaying subject pseudo-personality on a back face on this side of a virtual space image screen 3, and simultaneously displaying mating pseudo-personality on a front face in the inmost part of the virtual space image screen 3, the subject pseudo-personality is moved according to a talker body action based on the subject voice or a listener body action based on the mating voice inputted from a voice input means 5 via a telecommunication line 4, the mating pseudo- personality is moved according to the talker body action based on the mating vice via the telecommunication line 4 or the listener body action based on the subject voice inputted from the voice input means 5, and the mating voice is outputted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication system used for intimate communication between a person and a partner who are separated from each other by using communication terminals connected via a communication line. Here, the principal and the other party in the present invention are relative names. In addition, the numerical relationship between the person and the other party is basically one-to-one, but one-to-many and many-to-one.
Or, the case of many-to-many is also included.

[0002]

2. Description of the Related Art For example, a communication system called "chat" is recognized as a simple communication means via a computer network, and is widely used in proportion to the use of the Internet. A basic chat communication system on the Internet installs a chat application for transmission / reception on each individual computer serving as a client, and exchanges text or voice in real time with the other party to whom the person has permitted connection.

In recent years, a chat communication system has become available in which the person or the other party is displayed as an avatar (pseudo-personality) on the display of each other so that a feeling closer to an actual conversation can be obtained. There is. Initially, only a bubble was attached to a character that did not move, but recently, there is also a character that causes an avatar to perform a physical action (nodding, blinking, opening / closing of mouth, gesturing action, etc.) according to the text or voice that is exchanged. For example, Japanese Patent Laid-Open No. 2001-160021
"Communication system in virtual space" and Japanese Patent Laid-Open No. 2001-230801
There are "communication system and its method, communication service server and communication terminal device".

[0004] Japanese Unexamined Patent Publication No. 2001-160021 aims to provide a communication system in a virtual space in which the emotion of the sender of a message can be accurately transmitted to the other user with a sense of presence, as in the actual conversation. There is. Specifically, the chat messages from the speaker the user, the avatar management server, to bow the avatar listeners user detects a character string such as "Hello", the user excitement keying speed of the client exceeds a predetermined value When it is determined that the user is in a conversation state, a light state is displayed around the avatar, and when a predetermined time elapses after the operation of the client is stopped, the user's hatefulness is determined and the avatar is inflated. In this manner, the user can confirm the emotions and the physical and mental activity states of the other user with a sense of presence on the display, and can have a conversation while mutually understanding the emotions and the physical state of the other user.

[0005] Further, Japanese Patent Laid-Open No. 2001-230801 discloses a communication in which, in a chat environment, in addition to realizing a realistic dialogue by expressing the individuality and personality of participants, services such as translation and provision of related information can be enjoyed. It is a technology aimed at providing a system. Specifically, the input message is distributed to each terminal through the message receiving unit and the message transmitting unit, and the avatar is stored in the avatar information storage unit based on the input operation instruction signal or message. The motion is controlled by the avatar motion control unit by referring to the information and the action database for each character. In addition, the keyword extraction unit extracts the keyword of the dialogue content, the advertisement processing unit acquires related advertisement information and distributes it to each terminal, or the translation processing unit acquires the translation result of the message as necessary, and Send the translation result to the terminal.

[0006]

As the performance of computers has improved, chat communication systems have begun to use avatars as described above from simple text or voice exchanges, and to give physical activity to avatars.
Many people sensuously recognize that a conversation is not a simple exchange of text or voice, but text and voice and physical movement are integrated into a conversation with a more realistic feeling. Based on that. However, it is important to select the body motion because the body motion that is completely unrelated to the text or the voice impairs the sense of reality.

Looking at the above-mentioned prior art from the viewpoint of the relationship between text or voice and body movements, Japanese Patent Laid-Open No. 2001-160021
(1) Correspond a specific physical action to a specific text, (2) guess the excitement state of the person or the other party from the keystroke speed at which the text is typed, or (3) hate the person or the other party from the operation stop time. It can be seen that it is attempting to correlate the operation of the person or the other party with the physical action of the avatar, for example, by making a determination to cause the avatar to lack. Further, Japanese Patent Laid-Open No. 2001-230801 refers to the personality information and personality action database stored in the avatar information storage unit to control the movement by the avatar motion control unit. It can be said that he is trying to associate the operation with the physical action of the avatar. In the present day when speech recognition technology is incomplete, it must be said that it is still difficult to understand the meaning of conversation from text or voice and select an appropriate physical action, and these prior art means are considered to be in a reasonable range. You can also see.

The sense of unity or reality of the person and the other party in the chat communication system is brought about by a "physical pull-in phenomenon (hereinafter abbreviated as pull-in phenomenon)" based on the fact that the conversation rhythms of the person and the other party are synchronized. . That is, when the conversation rhythm is synchronized, the pull-in phenomenon is expressed, and when the pull-in phenomenon is expressed, the synchronization of the conversation rhythm is further advanced. In this state, the person and the other person can easily understand the contents of each conversation, and the degree of understanding of this conversation also contributes to the creation of a sense of unity and reality. From this, it can be seen that how to develop the pull-in phenomenon, that is, to increase the relevance between the body movement and the pull-in phenomenon is an issue. Therefore, a communication system that enables more intimate communication, for example, a chat communication system when the person and the person who are separated from each other temporally or distancely have a conversation, paying attention to the selection of the physical action to be given to the avatar. I examined it.

[0009]

[Means for Solving the Problems] A communication system developed as a result of study is used when intimate communication is performed by using a communication terminal in which a person and a partner who are separated from each other are connected through a communication line. However, the person and the other party's communication terminal display the person's pseudo personality on the back in front of the virtual space screen, and exchange the person's and the other party's voice while simultaneously displaying the person's pseudo person on the back of the virtual space screen. The personality moves according to the speaker's physical movement based on the person's voice or the listener's physical movement based on the other party's voice input from the voice input means via the communication line, and the other person's pseudo personality is the speaker's physical movement or voice input based on the other party's voice via the communication line. A physical media communication system that moves according to the listener's physical movement based on the person's voice input from the means and outputs the other person's voice A.

The communication terminal may be a dedicated device or a computer. In this case, a computer is used as a voice input / output unit and a generation and control unit of a physical action of a pseudo-personality, and the computers are connected to each other using a local network, a telephone line, or a communication line such as the Internet. Here, the pseudo personality referred to in the present invention means an avatar displayed on a screen such as a display of the computer, and the display or the like is a virtual space screen. Although it is possible to configure the pseudo personality with a robot connected to a computer, assuming that multiple people try to chat at one time, the pseudo personality on the display that can easily change the total number of pseudo personalities is preferable.

The communication system of the present invention utilizes the above-mentioned "pull-in phenomenon" to realize more intimate communication between the person and the other party. First, as a first means for causing the expression of the attraction-in phenomenon, (A) the person's pseudo personality is displayed in the back in front of the virtual space screen, and the other person's pseudo personality is displayed in the front in the back of the virtual space at the same time to talk. Gives the person and the other person a feeling of virtually sharing a space with each other. That is, the virtual space screen displays the person and the other person's pseudo personality in a face-to-face relationship. Although the fact that the opponent's pseudo personality is displayed in front on the back of the screen has been seen in the past, the present invention is different in that the pseudo personality of the other person is displayed on the back in front of the screen. This has the effect of encouraging the person to be drawn into the entire virtual space screen by objectively showing the person's pseudo-personality expressing himself / herself. Further, by visually recognizing the relationship between the body movements of the person himself / herself and the pseudo-personality of the other person, there is an advantage that the attraction-in phenomenon can be better expressed as a whole.

[0012] A specific person and a partner communication terminal, a pseudo personality control unit for making a body motion of the person and a partner pseudo personality based on voice, and a transmission and reception processing unit for exchanging data or voice between the person and the partner communication terminal. , A voice input / output unit for the person or the other party's communication terminal, and the person's body movements made by the pseudo-personality control unit based on the person's voice input from the voice input section of the person's communication terminal or the partner input from the reception processing section of the person's communication terminal Based on the voice, the pseudo-personality control unit creates the pseudo-personality of the virtual space screen according to the listener's physical movements, and the transmission processing unit transmits the self-voice to the reception processing unit of the partner communication terminal and inputs from the reception processing unit of the personal communication terminal. Pseudo-personality control based on the speaker's body motion made by the pseudo-personality control unit based on the other party's voice or the person's voice input from the voice input unit of the personal communication terminal The audio output unit with moving the opponent pseudo personality of the virtual space screen is configured to output the other party voice according to audience body movements make it.

If the computer is used as a communication terminal, the pseudo-personality control unit is the computer itself, the transmission / reception processing unit is the computer communication interface, and the voice input / output unit is a computer-connected or built-in microphone and speaker. As described above, the communication terminal of the present invention can be easily configured by a computer, and basically, it is provided as a program for causing the computer to perform a specific process. Since the chat application is provided as a program even in the conventional chat communication system, the same providing form is used in the communication system of the present invention.

The manifestation of the pull-in phenomenon is essentially the physical actions of both pseudo-personals. Therefore, as a second means of expressing the pull-in phenomenon, (B) the pseudo-personality acts as a speaker and as a listener, and the body movement of the speaker and the body movement of the listener are associated with the person or the other party's voice. By doing so, it was made to have a relation with each physical action of the person and the pseudo-personal person in the conversation with the voice. For example, when the pseudo-personal personality follows the person's voice, it behaves as a speaker, and when it follows the other party's voice, it behaves as a listener, while at the same time, the other person's pseudo-person acts as a listener when the other person's voice follows, and as a speaker when following the other party's voice. When a physical movement as a speaker and a physical movement as a listener are made at the same time for the same pseudo personality, the pseudo personality may be moved according to a preset priority, or the physical movements can be simply combined or excluded. You may move in combination. Even in actual human conversation, there are few clear body movements associated with voices, so some differences in nuances and combinations of body movements are
On the contrary, the reality of conversation using the communication system will be cultivated.

For the above-mentioned person and the other party's communication terminal, a specific movement control section for making the body movements of the person and the other person's pseudo personality based on the designation, and a body movement made by each of the pseudo personality control section and the particular movement control section are synthesized. A body motion synthesis unit and a designated motion input unit for the person or the partner communication terminal are additionally provided, and the speaker body motion or the speaker body motion created by the specific motion control unit based on the person's designation input from the designated motion input unit of the personal communication terminal. A virtual space screen is created by combining the listener's body motion created by the specific motion control unit based on the designation of the other party input from the reception processing unit of the personal communication terminal by the talker body motion created by the pseudo-personality control unit or the listener's body motion and body motion composition unit. The person's pseudo-personality is moved, and the speaker's body motion or personal communication terminal is created by the specific motion control unit based on the designation of the other party input from the reception processing unit of the personal communication terminal. Based on the person's designation input from the designated movement input section, the listener's body movement made by the specific movement control section is synthesized by the speaker's body movement made by the pseudo-personality control section or the listener's body movement and the body movement synthesis section to simulate a virtual space screen partner. You may move your personality.

Although there are few distinct body movements associated with voices, in some cases specific voices (eg, “goodbye”) and body movements (eg, waving) may be associated. Therefore, unlike a voice, such a body motion with a clear correspondence may be caused to move the pseudo-personality by a clear motion designation of the person or the other party. In the above example, the specific motion control unit and the body motion synthesis unit can use the computer itself, and the designated motion input unit can use an existing input device generally found in computers such as a keyboard, a mouse and a joystick. For example, if a particular key on the keyboard is given the meaning of "goodbye," and the person hits this particular key, the pseudo-personal character of the person on the virtual space screen of the person's communication terminal waving On the screen, the other person's pseudo-personality sways (the person himself becomes the other person's perspective). Here, the synthesizing of the body motions created by the pseudo-personality control unit and the specific motion control unit in the body motion synthesizing unit may move the pseudo-personality according to only one of the body motions according to a preset priority order, You may move by simply combining physical movements.

The body movement of the pseudo-personality that causes the pull-in phenomenon differs depending on whether the pseudo-personality behaves as a listener or as a speaker. Therefore, we have clarified the algorithm that determines the nodding action timing from the viewpoint that the body action that synchronizes with the conversation rhythm is the most important for the expression of the pull-in phenomenon, and other body actions are also caused by the same algorithm or the nodding action. The listener's body movements and the speaker's body movements were constructed to follow the algorithm. First, (C) the listener's body movements made by the pseudo-personality control unit consist of a selective combination of head nodding movements, eye blinking movements or body gesturing movements, and (a) nodding movements from voice ON / OFF. The estimated nodding predicted value is executed at the nodding operation timing exceeding the nod threshold, (b) the blink operation is performed at the blink operation timing that is exponentially distributed with the nod operation timing as a starting point, and (c). Gestures are voice ON / OFF
It is executed at the gesture motion timing when the predicted gesture value estimated from the above exceeds the gesture threshold value. Although the body movements other than those mentioned above are not excluded, the combination of the body movements is the main body movement of the listener. The gesture motion is a movement of the whole body (actually the upper body), which is so-called body action, such as a movement of an arm and a twist of the waist.

(A) The nod operation is executed at the nod operation timing when the nod prediction value estimated from ON / OFF of the voice exceeds the nod threshold. The nod prediction value is a temporal variable calculated from a prediction model (MA model (Moving-Average Model) or neural network model) that linearly or non-linearly associates a voice with a nod motion or a gesture motion. In the present invention, the nod operation timing is the time when the nod prediction value that can be continuously calculated from the start of voice input exceeds a predetermined nod threshold.
Specifically, for example, the voice is captured as ON / OFF of the electric signal with time, and the nod prediction value obtained from the ON / OFF of the electric signal with time (for example, the voltage value of the electric signal determined to be ON) is nod. The nod operation timing is derived by comparing with the threshold value. Without analyzing the meaning of such voice, simply turn on
Transmitting and processing voice as an electric signal of / OFF has an advantage that the amount of calculation in each communication terminal is small and a personal computer can make a real-time body motion. When the voice is regarded as an ON / OFF electric signal, the prosody and the intonation indicating changes in the electric signal over time may be considered together.

(B) The blinking operation is executed at the blinking operation timing, which is exponentially distributed with time, starting from the nodling operation timing. The algorithm for determining the blinking operation timing works due to the algorithm for determining the nodling operation timing. By associating the nodling motion and the blinking motion in this way, the occurrence of the pull-in phenomenon can be promoted. Then, (c) the gesture motion is executed at the gesture motion timing when the gesture prediction value estimated from ON / OFF of the voice exceeds the gesture threshold. The algorithm for determining the gesture motion timing is similar to the algorithm for determining the nod motion timing. Here, since it is desirable to perform the gesture motion more frequently than the nodling motion, the gesture prediction value to be calculated or the preset gesture threshold is set so that the timing of the gesture motion is determined more frequently than the nodling motion. It is better to make it relatively lower than the nod operation timing. The blink prediction value doubles as the gesture prediction value,
The blink threshold may double as the gesture threshold.

In contrast to the listener's body movement, (D) the speaker's body movement made by the pseudo-personality control unit is composed of a selective combination of a head swinging motion, a mouth opening / closing motion, an eye blinking motion, or a body gesturing motion. did. Here, the swinging motion of the head means that the nod motion is a motion in the front-back direction of the head, whereas moving the head in any direction results in a relative motion in the left-right direction. Also, the opening / closing operation of the mouth may be driven in proportion to the volume of the sound as in the conventional case. In the present invention, it is important that the body movements of the pseudo-personality that express the withdrawal phenomenon are important, and it is not necessary to force the opening and closing movements of the mouth according to the language. Therefore, of the speaker's body movements excluding the mouth opening / closing movements, (a) swinging movements are performed at swinging movement timings when the estimated swinging value estimated from voice ON / OFF exceeds the swinging threshold, and (b) blinking movements. Is performed at the blink motion timing when the eye blink prediction value estimated from voice ON / OFF exceeds the blink threshold, and (c) The body gesture motion is the gesture prediction value estimated from voice ON / OFF. It is executed at a gesture motion timing that exceeds the gesture threshold.

The swing predicted value, the blink predicted value, and the gesture predicted value are calculated according to the same algorithm as the algorithm for determining the blink motion timing in the listener's physical motion described above. The swing threshold, blink threshold, and gesture threshold also have the same meaning as the blink threshold used to determine the blink motion timing in the listener's body motion. Here, the predicted gesture value and the gesture threshold value of the listener's body movement may be the same as or different from the predicted gesture value and the gesture threshold value of the speaker's body movement. Further, the speaker's body motion uses similar algorithms for the head swinging motion, blinking motion, and gesturing motion, and therefore all prediction values or thresholds may be the same.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a system configuration diagram of a chat communication system to which the present invention is applied, and FIG. 2 is a user A.
Front view of the display 2 of the communication terminal 1 (virtual space screen 3), FIG. 3 is a front view of the display 2 of the communication terminal 1 of the user C (virtual space screen 3), and FIG. 4 is each of the users A and B. FIG. 5 is a control flow chart showing the listener control of the pseudo personality of each communication terminal 1, and FIG. 6 is each communication terminal 1.
5 is a control flowchart showing the speaker control of the pseudo personality of FIG. This example relates to a chat communication system in which the effect of intimate communication is most noticeable by applying the present invention.

The outline of the chat communication system according to the present invention is the same as the various communication systems using the existing Internet 4 as shown in FIG. Communication terminals 1, 1, 1, ... Connected to each other via the Internet 4
Are used by users A, B, C, ... While enjoying the conversation by voice while watching the virtual space screen 3 displayed on the display 2 shown in FIG. 2 or 3. The system configuration shown in FIG. 1 illustrates an example in which the communication terminals 1 are directly connected to each other via the Internet 4. However, when the scale of the chat communication system becomes large, a server that mediates each communication terminal is provided to identify the route of voices exchanged between the communication terminals, and in some cases, a pseudo personality by each communication terminal as described later. It is also possible to have the server centralize the generation of the body motion and transmit the pseudo-personal control data from the server to each communication terminal.

The difference from the conventional chat communication system in the present invention first appears in the virtual space screen 3 displayed on the display 2 of each communication terminal 1. When users A, B, and C are chatting with each other in FIG. 1, the communication terminal 1 of the user A displays the pseudo personality a representing himself as a rear view and a relative display as shown in FIG. The enlarged display is performed, and the pseudo personalities b and c representing the opponents (users B and C) are displayed in front and relatively reduced. This is a depth representation of the virtual space screen 3 using the perspective method, and is a means for making the person or the other person feel the communication space. In this example, the size is also changed between the pseudo-personal characters b and c. This expresses a difference based on, for example, the order in which they participated in chat communication or the familiarity of the user A, by making the sizes of the pseudo personalities b and c different from the pseudo personality a.

Each communication terminal 1, as seen in FIG.
Since the chat communication system is based on voice, voice input / output units 5 and 6 are provided, and in the present example, a designated action input unit 7 for causing the pseudo-personality to perform the designated action is provided. In this example, a computer 13 including a pseudo-personality control unit 8, a specific motion control unit 9, a body motion synthesis unit 10, and transmission / reception processing units 11 and 12 is used as a core, a display 2 for displaying a virtual space screen 3, a voice input unit. Computer as 5
A communication terminal is composed of a microphone connected to 13, a keyboard or mouse connected to the computer 13 as the designated operation input unit 7, and a speaker connected to the computer 13 as the audio output unit 6.

The computer 13 controls the CPU and the like (not shown) by software using the pseudo personality control unit 8, the specific action control unit 9, and the physical action synthesis unit 10 stored in a storage device (not shown) attached to the computer. And functionally configure. These control units 8 and 9 and the synthesizing unit 10 can perform hardware processing by using a dedicated IC or the like, but from the viewpoint of maintenance such as expansion or modification of functions, the general-purpose computer 13 is controlled by software. It is better to configure. Also, a modem connected to or built in the computer 13,
The TA or the network device constitutes the transmission / reception processing units 11 and 12. If the computer has a mouse, keyboard or speaker (if the computer is a laptop)
Alternatively, various input devices (track pad, track point, etc.) may be used instead of the mouse.

According to the present invention, the pseudo-personality displayed on the virtual space screen 3 of each communication terminal 1 behaves as a speaker or a listener based on the voice of the person or the other party, and the virtual space screen 3 is displayed.
The attraction phenomenon that draws the person who is watching and the other party into the conversation rhythm is expressed. The pseudo-personality may be a human body expressed by a simple three-dimensional model. This is because in the present invention, the face-to-face display of the person and the pseudo-personality of the person and the physical action and the specific action of each pseudo-personality are important, and the realness of the image of the pseudo-personality itself does not matter. Further, it is preferable to change the facial expression or the complexion of the pseudo personality, but it is not an essential element of the present invention. Therefore, when the present invention is used as a chat communication system, the pseudo personality may be a three-dimensional or two-dimensional model imitating an animal or an inanimate object. Considering the generation and execution of real-time body movements, a simple pseudo-personality is preferable, but if the image processing capability of the communication terminal is high, for example,
It is also possible to configure a pseudo personality as a live-action model of the person and the other party and change the facial expression and the complexion of the face.

Next, taking the case where the user A speaks to the user C as an example, the generation of a physical action of each communication terminal 1 and the operation of each pseudo personality will be described. The communication terminal 1 of the user A is the own communication terminal 101, and the communication terminal of the user C is the partner communication terminal 1
02, the communication terminals 101 and 102 directly exchange voices without mediating the server. Further, the user A is the personal communication terminal 101.
In the rear display, the pseudo-personality a (speaker, see FIG. 2), and in the partner communication terminal 102, the pseudo-personality a (speaker,
(See FIG. 3), the user C shows the pseudo-personal character c in front view on the personal communication terminal 101 (listener, see FIG. 2), and the pseudo-personal character c in back view on the personal communication terminal 102 (listener, FIG. 3).
Display).

The personal voice of the user A input from the voice input unit 5 (displayed by a thick broken line arrow, the same applies hereinafter) is the personal communication terminal 10
It is sent to the pseudo personality control unit 8 and the transmission processing unit 11. The voice of the user A is the basis for determining, in the pseudo personality control unit 8 of the personal communication terminal 101, (1) the listener's physical action of the opponent pseudo personality c and (2) the speaker's physical action of the pseudo personality a. Becomes Also, from the transmission processing unit 11 to the Internet 4
The personal voice of the user A sent to the partner communication terminal 102 via the personal computer is transmitted by the pseudo personality control unit 8 of the partner communication terminal 102 to (3) the listener's body motion of the personal pseudo personality c, and (4) the pseudo personality of the partner. It is the basis for determining the speaker physical movement of a. As described above, the present invention is based on one voice (in the example, the voice of the user A), and the communication terminal 101.102 of the user and the other party have a total of four persons and the speaker and listener body of the pseudo-personal characters a and c. Determine the action. In reality, since the conversation progresses while the relationship between the user and the other person is exchanged with each other, the speaker physical movement and the listener physical movement are simultaneously determined for each pseudo personality. In this case, the pseudo-personal control unit 8 creates a complex physical motion of the listener's physical motion and the speaker's physical motion,
The pseudo-personality is moved according to the complex physical action.

In order to make the user A feel an intimate communication effect, the pseudo-personality c of the opponent displayed on the virtual space screen 3 of the personal communication terminal 101 may be made to act in accordance with the listener's physical movement. The behavior of the opponent pseudo-personality c as a listener is the listener's body movements centering on the nodding movement, and it is important to determine the nodding movement timing. The nod operation timing is determined according to the following procedure. First, considering the person's voice input to the pseudo-personality control unit 8 as an ON / OFF electric signal, as shown in FIG. 5, the nod prediction value N is first calculated from the nod estimation using the MA model.
Calculate 0. The calculation of the nod prediction value N0 is very simple and can be performed in real time even by the computer 13 having a relatively low processing capacity. Therefore, unlike the conventional chat communication system, the present invention has an advantage that users can be directly connected to each other without using a server.

The nod operation timing is the nod predicted value N.
0 is compared with a predetermined nodling threshold value Na, and if N0 ≧ Na, it is determined as the time timing of the nodling predicted value N0.
Then, the nod operation is executed at the nod operation timing. In this example, considering that there is a possibility that the same nodding motion may not be seen as a natural motion every time, a plurality of motion patterns in which the size and speed of movement of the head are changed are set, and random or other body motions are set. A different operation pattern is appropriately selected according to the relationship of the operation, for example, the strength of the voice. The strength of the voice is a barometer showing the ups and downs of emotions, and it is preferable to determine the strength of the motion pattern according to the strength of the voice. Further, the movement of the same pattern may be changed according to the strength of the voice. There are only a plurality of operation patterns, and the nod operation timing is the time point determined previously. Such a variety of motion patterns is also effective for manifesting the pull-in phenomenon.

In the listener's body motion of the pseudo-personal character c, the blink motion timing is determined by relating the blink motion to the nod motion. Specifically, based on the nodding motion timing determined as described above, the blink motion timing is determined by the exponential distribution calculation over time. Even in this blinking motion,
Since a uniform blinking motion impairs a natural feeling, a plurality of motion patterns are set in advance, and a different motion pattern such as a random motion or another physical motion, for example, the strength of a voice is appropriately selected. Blinking is an incidental aspect of the whole body movement, so for example, the person's voice can be simply turned on / off.
It may be regarded as a signal and executed.

In the gesture motion of the opponent pseudo-personality c, the gesture motion timing is basically determined by the same procedure as the nod motion. Although the motion mode itself differs depending on the nod motion and the gesture motion, when the motion timings are synchronized, the natural feeling is also lost. Therefore, the actual noisy motion timing is shifted from the nodding motion timing by using the same nod prediction value N0 as the determination of the nod motion timing as the gesture threshold while making the gesture threshold Ga to be compared different from the nod threshold N0. . Furthermore, by setting a plurality of specific motion patterns of specific gesture motions to be executed at this gesture motion timing and selecting different motion patterns such as random or other physical motions, for example, according to the strength of the voice, diversity is realized. is doing.

According to the present invention, not only the opponent pseudo-personality c is displayed in front on the same virtual space screen 3, but also the pseudo-personality a
By displaying on the back side, the entire virtual space screen 3 is perceived as a space for intimate communication, and the occurrence of the pull-in phenomenon is promoted. The behavior of the pseudo-personal character a as the speaker follows a speaker control similar to the above listener control procedure, except that the speaker uses a swinging motion of the head instead of the nodding motion and a mouth opening / closing motion. . First, the person's voice input to the pseudo-personality control unit is regarded as an ON / OFF electric signal, and as shown in FIG. 6, the swing predicted value M0 is calculated from the swing estimation of the head using the MA model. Of course, it is preferable that the MA model is different between the listener and the speaker, but the MA model may be the same. Although it is not possible to simply compare the head shaking motion and the nodling motion, it is preferable that the frequency of the head shaking motion is relatively higher than that of the nodding motion.

In the talker's body motion of the pseudo-personality a, the swinging motion and the blinking motion are basically unrelated, and the blinking motion is determined separately by the procedure similar to the swinging motion. That is, the blink prediction value E0 is calculated by blink estimation, and the blink operation timing is determined by comparing this blink prediction value E0 with the blink threshold Ma. Even in this blinking motion, the natural feeling is impaired by the uniform blinking motion, so multiple motion patterns are set and random or other physical motions such as different motion patterns are selected according to the strength of the voice, for example. To do. Since the blinking motion is incidental to the whole body motion, the person's voice may be regarded as a simple ON / OFF signal, for example. This blinking operation can be omitted because the person himself / herself is not seen because the pseudo personality a is displayed on the back of the personal communication terminal 101, but it can be omitted, but the pseudo personality as the speaker displayed on the virtual space screen 3 of the partner communication terminal 102 is displayed. a is required (FIGS. 2 and 3 for comparison).

The mouth opening / closing action peculiar to the person himself / herself as a speaker is closely related to the voice of the speaker.
In the simplest case, it is better to regard the person's voice as an ON / OFF electric signal and follow it. Therefore, in this example, the opening / closing operation of the mouth is performed according to the ON / OFF change of the person's voice in real time. Even in this opening / closing operation, the natural feeling is lost in the uniform blinking operation, so multiple operation patterns are set, and a different operation pattern is selected appropriately according to the strength of the sound, such as random or other body movements. To do. Further, the movement of the same pattern may be changed according to the strength of the voice. This opening / closing operation can be omitted because the person himself / herself is not seen because the pseudo personality a is displayed on the back of the personal communication terminal 101, but the pseudo personality as the speaker displayed on the virtual space screen 3 of the partner communication terminal 102 can be omitted. a is required (FIGS. 2 and 3 for comparison).

The gesture motion of the pseudo-personal character a as a speaker is a body action, and in this example, a predicted motion value M
By using 0 as the gesture prediction value, the gesture motion timing is determined in the same procedure as the determination of the swing motion timing. However, since it is easy to give an unnatural feeling when the swing motion and the gesture motion are synchronized, the gesture threshold value Ga to be compared is made different from the nod threshold value Na while using the nod prediction value N0 as the gesture prediction value. Furthermore, by setting a plurality of specific motion patterns of specific gesture motions to be executed at this gesture motion timing and selecting different motion patterns such as random or other physical motions, for example, according to the strength of the voice, diversity is realized. is doing. Further, the movement of the same pattern may be changed according to the strength of the voice.

Here, when the user A designates a specific action from the designated action input unit 7, the specific action control unit 9 simulates the specific action corresponding to the input designated signal (indicated by a thick arrow in FIG. 4). Give to personality a. Since the designation of the specific action is input in parallel with the voice of the user A, the physical action created by the pseudo-personality control unit 8 and the specific action created by the specific action control unit 9 depend on each other. Therefore, the body movement synthesizing unit 10 synthesizes both movements and moves the pseudo-personal character a as the speaker as an integral movement. The body movement and the specific movement can be combined by selectively assigning or weighting them with priorities.
Actually, the listener's body movement and the talker's body movement are simultaneously generated for a specific pseudo personality, and the pseudo personality control unit 8
Since it is synthesized as a complex body motion in (3), the body motion synthesizer 10 synthesizes the complex body motion and the specific motion.

The voice of the user A is sent from the transmission processing unit 11 as it is to the reception processing unit 12 of the partner communication terminal 102 of the user C via the Internet 4. In the partner communication terminal 102, the pseudo personality control unit 8 that receives the voice of the user A from the reception processing unit 12 determines the talker physical action of the pseudo personality a of the partner and the listener's physical action of the pseudo personality c of the user as the speaker control procedure. Or create according to listener control procedures. Then, the specific motion control unit 9 creates a specific motion of the opponent pseudo-personality a in accordance with the specific motion designation signal transmitted at the same time as the voice of the person, and synthesizes it with the talker's body motion to move the partner pseudo-personality a. . As described above, in the partner communication terminal 102, although the positional relationship between the person and the pseudo-personal characters a and c displayed on the virtual space screen 3 is changed, and the front display and the back display are reversed, the generation of the body motion is described above. There is no difference from what I did. Therefore, a description of a specific physical action or a combination of the physical action and the specific action in the partner communication terminal 102 will be omitted.

As described above, according to the present invention, based on the voice of the person, the speaker's body movement of the person's pseudo personality a in the person's communication terminal 101, the listener's body movement of the person's pseudo personality c, and the person's pseudo person a in the other party's communication terminal 102 are performed. And the listener's body movements of the person's pseudo personality c are determined, and each pseudo personality is moved within the virtual space screen 3 of the respective communication terminals 101, 102. In reality, the person and the other party are in a relative relationship, and the conversation can proceed while the speaker and the listener are alternately exchanged. Therefore, in addition to the person's voice, the person's communication terminal is based on the person's voice. The listener's body motion of the pseudo-personal personality a and the talker's body motion of the partner pseudo-personality c in 101, and the listener's body motion of the partner pseudo-personality a and the talker's body motion of pseudo-personal personality c in the partner communication terminal 102 are added. If there are a plurality of opponents, a plurality of listener physical actions of each pseudo-personality are further generated. When such a talker's body movement and a plurality of listener's body movements are simultaneously produced, the pseudo-personal control unit 8 coordinates each body movement and outputs control data or a signal to the body movement synthesis unit 10 as a composite body movement. Good to do. The body movement coordination may be performed by the body movement synthesis unit 10.

The specific motion is a motion added to the speaker's physical motion of the pseudo-personal personality by intentionally specifying the motion of the speaker or the other party, and the listener's physical motion of the pseudo-personal personality is basically irrelevant. However, depending on the specific action, the listener-specific action of the pseudo-personality of the opponent corresponding to the speaker-identification action of the pseudo-personality of the other person may be defined in advance, and the listener-specific action may be combined with the listener's physical action of the pseudo-personality of the other person. For example, in FIG. 2 and FIG. 3, when the user C terminates the chat communication, he / she shakes his hand to suggest separation.
Designate (action that suggests "goodbye"), synthesize as a speaker-specific action into the speaker's physical action, and simulate the opponent's personality c
(Fig. 2) and the pseudo-personality c (Fig. 3) are moving. In response to the speaker specifying action, the pseudo personality a (FIG. 2) and the pseudo partner personality a (FIG. 3) slightly raise their hands as responding listener specifying actions. It is considered that the specific motion is issued only to a specific partner, or to all communication partners, and a large movement that suppresses the basic listener's physical motion is not preferable. For this reason, the movements of the pseudo-personal personality a (FIG. 2) and the pseudo-personal personality a (FIG. 3) in the example are restrained.

The present invention is most suitable for use in a voice chat communication system as described above, but if a usage form in which the present invention is applied to either the person or the other party, for example, a communication system is used. It can also be used as a service system of a corporate call center that uses, or as a voice reading device for mail. As a service system of a call center, a service person who responds to a customer uses a communication terminal to which the present invention is applied, and a pseudo personality is moved by a reception terminal of the service person based on voices from the service person and the customer. Conventionally,
Although the service personnel's response may be a problem, the use of the present invention is expected to improve the service personnel's response, and the service employee can understand the customer's story better by the occurrence of the pull-in phenomenon. Become.

In the case of being configured as a device for reading a voice of a mail, the text of the mail is converted into a voice by using software for voice synthesis, and the communication terminal of the present invention is used as a stand-alone device. In this case, even if the user (mail recipient) who is the main person does not utter a voice, in the virtual space screen, the pseudo personality moves in response to the pseudo personality of the person who reads the mail, and the virtual personality moves according to the listener's physical movements. It is possible to make users who see the screen feel intimate communication. In this case, unlike the voice of the other party (mail sender), a synthesized voice based on a clear word is used, so that the word can be easily identified. By utilizing this, the meaning of the word by the voice recognition may be used for designating the specific action, and the specific action may be synthesized with the physical action based on the synthetic voice. Such a method of designating a specific action based on the meaning of a word by voice recognition can also be used in the chat communication system and the service system of the call center.

[0044]

As described above, according to the present invention, it is possible to provide a communication system which enables more intimate communication when the person and the other party who are separated from each other in time or distance have a conversation. In recent years, there has been a trend to realize more intimate communication by understanding the meaning of specific words, but in communication systems that involve such human-level intelligent judgment, the performance required of communication terminals is high. However, the communication environment may require a high communication speed. Based on such a situation, the present invention has clarified that intimate communication is brought about by a conversation rhythm, and focuses on expressing a pull-in phenomenon that creates this conversation rhythm. There is a great effect in realizing sufficiently close communication while using the environment.

The reason why the performance of the communication terminal is not excessively required is that the control procedure used for generating the body motion of each pseudo-personality requires only simple processing. The nodding motion or head shaking motion, blinking motion, mouth opening / closing motion, and gesture motion that create the rhythm of conversation based on voice treats the voice as a simple ON / OFF electric signal, so processing time and burden are extremely high. It's small. Further, the reason why only the low-speed communication environment is required is that what is exchanged between the communication terminals is only a voice with a small amount of data, and all the physical motions can be generated on each communication terminal based on this voice. This means that the present invention can be easily applied to a peer-to-peer chat communication system that does not go through a server.

The present invention is most suitable for use in a chat communication system in which both the person and the other party use the same communication terminal, but as described above, even if the present invention is applied to only one person or the other party. It is possible to bring about the effect of more intimate communication with the person who uses the present invention, and indirectly to give the effect of improving the conversation rhythm to the other person. This means that the present invention can be used alone. The present invention can be used as long as it is possible to generate a voice-based pseudo-personal motion, and the voice need not necessarily be the real voice of the person or the other person. From this, for example, an application example as a device for enjoying music more by using music instead of voice can be considered. Thus, the present invention provides a system that provides improved direct conversation or broader sound communication.

[Brief description of drawings]

FIG. 1 is a system configuration diagram of a chat communication system to which the present invention is applied.

FIG. 2 is a front view of a display of a communication terminal of user A.
(A virtual space screen).

FIG. 3 is a front view of a display of a communication terminal of user C.
(A virtual space screen).

FIG. 4 is a communication terminal configuration diagram showing the operation of each unit by displaying the communication terminals of users A and B in contrast.

FIG. 5 is a control flowchart showing listener control of a pseudo personality of each communication terminal.

FIG. 6 is a control flowchart showing speaker control of a pseudo personality of each communication terminal.

[Explanation of symbols]

1 communication terminal 2 display 3 Virtual space screen 4 Internet 5 Voice input section (microphone) 6 Audio output section (speaker) 7 Designated operation input section (keyboard or mouse) 8 Pseudo personality control section 9 Specific operation control unit 10 Body movement synthesis section 11 Transmission processing unit 12 Reception processing unit 13 Computer 101 Personal communication terminal 102 Remote communication terminal A user B user C user a pseudo personality b Pseudo personality c pseudo personality

─────────────────────────────────────────────────── ─── Continued Front Page (51) Int.Cl. ⁷ Identification Code FI Theme Coat (Reference) H04M 1/00 G10L 3/00 Q

Claims (9)

[Claims] 1. A communication system used for intimate communication between a person and a partner who are separated from each other by using communication terminals connected via a communication line,
The person and the other party's communication terminal display the pseudo person's personality on the back in front of the virtual space screen, and exchange the person and the other party's voice while simultaneously displaying the person's pseudo personality on the back side of the virtual space screen. Voice-based speaker physical movement or movement according to the listener's physical movement based on the other party's voice input from the voice input means via the communication line, and the pseudo-personality of the other party is input from the speaker's physical movement based on the other party's voice via the communication line or the voice input means A physical media communication system that moves according to the listener's body movements based on the person's own voice and outputs the other party's voice. 2. The person and the other party communication terminal, a pseudo personality control section for making a body motion of the person and the other party pseudo person based on voice, and a transmission / reception processing section for exchanging data or voice between the person and the other party communication terminal. It consists of a voice input / output unit for the person or the other party's communication terminal, and the speaker's body motion made by the pseudo-personality control unit based on the person's voice input from the voice input section of the person's communication terminal or the partner's voice input from the reception processing section of the person's communication terminal Based on the listener's physical action created by the pseudo-personality control unit, the pseudo-personality of the virtual space screen is moved according to the listener's physical movement, and the transmission processing unit transmits the self-voice to the reception processing unit of the partner communication terminal and inputs from the reception processing unit of the personal communication terminal. Created by the pseudo-personality control unit based on the other party's voice Created by the pseudo-personality control unit based on the speaker's body movement or the person's voice input from the voice input unit of the personal communication terminal 2. The physical media communication system according to claim 1, wherein the voice output unit outputs the voice of the other party while moving the other person's pseudo-personality on the virtual space screen according to the listener's physical movement. 3. The person and the partner communication terminal combine a specific motion control unit that creates physical motions of the person and the other party's pseudo-personality on the basis of designation, and body motions created by the pseudo-personality control unit and the particular motion control unit, respectively. A body motion synthesis unit and a designated motion input unit for the person or the partner communication terminal are additionally provided, and the speaker body motion or the speaker body motion created by the specific motion control unit based on the person's designation input from the designated motion input unit of the personal communication terminal. A virtual space screen is created by combining the listener's body motion created by the specific motion control unit based on the designation of the other party input from the reception processing unit of the personal communication terminal by the talker body motion created by the pseudo-personality control unit or the listener's body motion and body motion composition unit. The person's pseudo personality is moved, and the speaker's physical movement or the designated movement of the personal communication terminal made by the specific movement control unit based on the designation of the other party input from the reception processing unit of the personal communication terminal. Based on the person's designation input from the work input section, the listener's body movements made by the specific movement control section are synthesized by the speaker's body movements or the listener's body movements and the body movements synthesis section made by the pseudo-personality control section, and the other person's pseudo-personality on the virtual space screen is displayed. The physical media communication system according to claim 2, wherein the physical media communication system moves. 4. The listener's body motion made by the pseudo-personality control unit is composed of a selective combination of head nodding motion, eye blinking motion or body gesturing motion, and the nodding motion is voice ON / O.
4. The physical media communication system according to claim 1, wherein the predicted nod value estimated from FF is executed at a nod operation timing exceeding a nod threshold. 5. The physical media communication system according to claim 4, wherein the blinking operation is performed at a blinking operation timing that is exponentially distributed over time with the nodling operation timing as a starting point. 6. The physical media communication system according to claim 4, wherein the gesture motion is executed at a gesture motion timing when a gesture prediction value estimated from ON / OFF of voice exceeds a gesture threshold value. 7. The talker's body motion made by the pseudo-personality control unit is a selective combination of a head swinging motion, a mouth opening / closing motion, an eye blinking motion, or a body gesturing motion, and the shaking motion is voice ON / OFF. The physical media communication system according to claim 1, 2 or 3, which is executed at a swing motion timing when a swing predicted value estimated from OFF exceeds a swing threshold. 8. The physical media communication system according to claim 7, wherein the blinking operation is executed at a blinking operation timing at which the eye blink predicted value estimated from ON / OFF of the voice exceeds the blinking threshold value. 9. The physical media communication system according to claim 7, wherein the gesture motion of the body is executed at a gesture motion timing when a gesture prediction value estimated from ON / OFF of voice exceeds a gesture threshold value.