JP2011147070A - Communication apparatus and communication server - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a user of a communication apparatus to intuitively represent his/her emotion without giving any unpleasant feeling to a communication partner, during communication using an avatar image or the like. <P>SOLUTION: In a communication apparatus 100, a receiving section 101 receives a video image of a communication partner, and an input section 102 is configured for a user of the communication apparatus 100 to input his/her emotion about the communication partner indicated by the received video image, and is comprised of e.g., an acceleration sensor for detecting a direction and a magnitude of acceleration. A setting section 103 sets emotional data corresponding to the emotion, inputted to the input section 102, of the user of the communication apparatus 100 about the communication partner, a conversion section 104 converts the video image of the communication partner in accordance with the emotional data, and an output section 105 outputs the video image received by the receiving section 101 and the video image converted by the conversion section 104. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a communication device and a communication server.

  When a communication device user performs a chat or the like using an avatar image in a virtual space, the communication device user changes the avatar's facial expression etc. by selecting one of a plurality of types of emotion setting buttons prepared in advance. There is a conventional technology that conveys the feelings of communication to the communication partner. With this conventional technique, the communication device user can express his / her emotion to the communication partner by changing the avatar image in accordance with his / her emotion (see, for example, Patent Document 1).

JP 2000-190034 A

  However, in the above prior art, since the emotion of the communication device user is transmitted as it is to the communication partner, there is a possibility that the communication partner is uncomfortable.

  An object of the present invention is to enable a communication device user to intuitively express his / her feelings without causing discomfort to a communication partner in communication using an avatar image or the like.

  The communication apparatus according to the first aspect of the present invention sets a receiving means for receiving video or audio of a communication partner, an input means for inputting emotions to the communication partner, and emotion data corresponding to the emotions. Setting means; and conversion means for converting the video or the sound according to the emotion data.

  The communication apparatus according to the second aspect of the present invention includes a receiving unit that receives video or audio of a plurality of communication partners, a designation unit for designating a communication partner that expresses emotion among the plurality of communication partners, Input means for inputting emotions for a specified communication partner, setting means for setting emotion data corresponding to the emotions, communication partner information indicating the specified communication partner, and the emotion data for the specified communication partner Transmitting means for transmitting to the communication server.

  A communication device according to a third aspect of the present invention includes: a receiving unit that receives video or audio of a plurality of communication partners; an input unit that inputs emotions to any of the plurality of communication partners; Setting means for setting corresponding emotion data; and transmission means for transmitting the emotion data for any of the plurality of communication partners to a communication server.

  A communication server according to a fourth aspect of the present invention is indicated in the communication partner information among the plurality of communication partners according to the receiving means for receiving the communication partner information and the emotion data from a communication device, and the emotion data. Conversion means for converting video or audio of a communication partner, and transmission means for transmitting the converted video or audio to a communication partner other than the communication partner indicated in the communication partner information among the plurality of communication partners. It has.

  A communication server according to a fifth aspect of the present invention includes: a receiving unit that receives the emotion data from a communication device; a specifying unit that specifies a communication partner that is a target of the emotion data; Conversion means for converting the video or audio of the specified communication partner among the communication partners, and transmission means for transmitting the converted video or audio to a communication partner other than the specified communication partner among the plurality of communication partners; It comprises.

  According to the present invention, in communication using an avatar image or the like, the communication device user can intuitively express his / her feelings without causing discomfort to the communication partner.

Configuration diagram of communication apparatus according to Embodiment 1 of the present invention Operational concept diagram of input unit according to Embodiment 1 of the present invention The figure which shows the table which concerns on Embodiment 1 of this invention (Example 1) The figure which shows the operation | movement image of the communication apparatus which concerns on Embodiment 1 of this invention. The figure which shows the table which concerns on Embodiment 1 of this invention (example 2) Configuration diagram of a communication system according to Embodiment 2 of the present invention Configuration diagram of communication apparatus according to Embodiment 2 of the present invention Configuration diagram of communication server according to Embodiment 2 of the present invention Configuration diagram of communication apparatus according to Embodiment 3 of the present invention Configuration diagram of communication server according to Embodiment 3 of the present invention

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a configuration diagram of a communication device 100 according to the present embodiment.

  In the communication device 100, the receiving unit 101 receives a video of a communication partner. For example, when the communication apparatus 100 performs videophone communication via a network, the reception unit 101 receives a communication partner avatar image transmitted from the communication partner communication apparatus. The receiving unit 101 outputs the received video of the communication partner to the output unit 105 and the conversion unit 104.

  The input unit 102 is for the user of the communication device 100 to input emotions to the communication partner indicated by the received video, and is configured by an acceleration sensor that detects the direction and magnitude of acceleration, for example. As shown in FIG. 2, the input unit 102 can detect accelerations in three directions of the X axis, the Y axis, and the Z axis applied to the communication apparatus 100.

  The setting unit 103 sets emotion data corresponding to the emotion of the user of the communication device 100 with respect to the communication partner input to the input unit 102. The setting unit 103 sets the emotion data with reference to a table in which the direction and magnitude of the acceleration indicating the emotion of the user of the communication device 100 and the emotion data are set in association with each other as shown in FIG. The setting unit 103 outputs the set emotion data to the conversion unit 104.

  The conversion unit 104 converts the image of the communication partner according to the emotion data. The conversion unit 104 refers to the table shown in FIG. 3 and converts the video according to the emotion data. The conversion unit 104 sends the converted video to the output unit 105.

  The output unit 105 outputs the video received by the receiving unit 101 and the video converted by the conversion unit 104. In the present embodiment, for example, the output unit 105 includes a display that displays an image.

  Next, the operation of the communication apparatus 100 will be described with reference to FIG.

  As illustrated in FIG. 4, the communication device 100 receives a video 201 transmitted from a communication partner via a communication network.

  At this time, for example, when the user of the communication device 100 expresses the feeling of “anger” with respect to the communication partner, the user of the communication device 100 is assumed to tap the communication device 100 with a small force in the Y-axis direction. Further, when the user of the communication device 100 expresses a feeling of “sadness” with respect to the communication partner, the user of the communication device 100 taps the communication device 100 with a small force in the Z-axis direction. Further, when the user of the communication device 100 expresses an “agreement” feeling to the communication partner, the user of the communication device 100 taps the communication device 100 with a large force in the Z-axis direction. Further, when the user of the communication device 100 expresses an “opposite” feeling with respect to the communication partner, the user of the communication device 100 taps the communication device 100 with a large force in the X-axis direction. The input unit 102 detects the direction and magnitude of the acceleration at this time, and outputs the detection result to the setting unit 103.

  The setting unit 103 sets emotion data with reference to the table shown in FIG. 3 according to the direction and magnitude of the acceleration detected by the input unit 102. And the conversion part 104 converts the image | video of a communicating party with reference to the table shown in FIG. 3 according to emotion data.

  For example, when the direction of acceleration is “Y-axis” and the magnitude is “small”, the setting unit 103 sets emotion data: “anger”. Then, according to the emotion data, the conversion unit 104 enlarges the image around the nose in the video 201 and combines the nose breath image and the anger mark to convert the video 201 into an angry facial expression video 206. To do.

  When the acceleration direction is “Z-axis” and the magnitude is “small”, the setting unit 103 sets emotion data: “sadness”. Then, in accordance with the emotion data, the conversion unit 104 synthesizes a tear image under the eyes in the video 201 to convert the video 201 into a video 204 with a crying expression.

  If the acceleration direction is “Z-axis” and the magnitude is “large”, the setting unit 103 sets emotion data: “agreement”. Then, in accordance with the emotion data, the conversion unit 104 synthesizes the laughing mouth image at the position of the mouth in the video 201, thereby converting the video 201 into a laughing facial expression video 205.

  When the acceleration direction is “X axis” and the magnitude is “large”, the setting unit 103 sets emotion data: “opposite”. Then, according to the emotion data, the conversion unit 104 enlarges a certain range of images around the eyes in the video 201, reduces a certain range of images around the mouth, and further synthesizes a sweat image prepared in advance. By doing so, the video 201 is converted into a video 203 with a surprising expression.

  In the above description, the setting unit 103 sets emotion data according to the direction and magnitude of acceleration detected by the input unit 102 when the user of the communication device 100 taps the communication device 100. However, the input unit 102 detects a hitting pattern instead of the magnitude of acceleration, and the setting unit 103 sets emotion data according to the direction of acceleration and the hitting pattern with reference to the table shown in FIG. Also good.

  That is, for example, when the acceleration direction is “Y-axis” and the number of hits for a predetermined time is “1”, the setting unit 103 sets emotion data: “anger”. When the direction of acceleration is “Z-axis” and the number of hits for a predetermined time is “1”, the setting unit 103 sets emotion data: “sadness”. When the direction of acceleration is “Z-axis” and the number of hits for a predetermined time is “continuous multiple times”, the setting unit 103 sets emotion data: “agreement”. When the direction of acceleration is “X-axis” and the number of hits for a predetermined time is “continuous multiple times”, the setting unit 103 sets emotion data: “opposite”.

  As described above, according to the present embodiment, in communication using an avatar image or the like, the image of the communication partner converted according to the emotion of the communication device user is not transmitted to the communication partner. Emotions can be expressed intuitively without causing discomfort to the communication partner.

(Embodiment 2)
FIG. 6 is a configuration diagram of a communication system according to the present embodiment. As illustrated in FIG. 6, the communication system according to the present embodiment includes a communication server 400 and a plurality of communication devices, for example, communication devices 300 </ b> A to 300 </ b> C, and the communication devices 300 </ b> A to 300 </ b> C communicate with each other via the communication server 400. To communicate. That is, communication apparatuses 300A to 300C transmit each user's video (for example, each user's avatar image) to communication server 400, and communication server 400 distributes the received video to communication apparatuses 300A to 300C.

  FIG. 7 is a configuration diagram of communication apparatus 300 according to the present embodiment. Communication devices 300A to 300C shown in FIG. 6 all have the configuration shown in FIG.

  In the communication device 300, the receiving unit 301 receives a plurality of communication partner images from the communication server 400. For example, when the communication device 300A illustrated in FIG. 6 performs videophone communication via the network with the communication devices 300B and 300C, the receiving unit 301 of the communication device 300A receives the communication device 300B and 300C from the communication partner via the communication server 400. The avatar images of the two communication partners transmitted are received. In addition, the reception unit 301 receives a video converted by the communication server 400 among a plurality of communication partner videos. The receiving unit 301 outputs the received video to the output unit 302.

  The output unit 302 outputs the video received by the receiving unit 301. In the present embodiment, for example, the output unit 302 includes a display that displays an image.

  The input unit 303 is used by the user of the communication device 300 to input feelings for any one of the plurality of communication partners indicated by the received video, and like the input unit 102 of the first embodiment, For example, it is constituted by an acceleration sensor that detects the direction and magnitude of acceleration, and as shown in FIG. 2, it is possible to detect accelerations in three directions of the X-axis, Y-axis, and Z-axis applied to the communication apparatus 300.

  The setting unit 304 sets emotion data corresponding to the emotion of the user of the communication device 300 input to the input unit 303 with respect to the communication partner. As with setting unit 304 in the first embodiment, setting unit 304 is set in association with the direction and magnitude of acceleration indicating the emotion of the user of communication apparatus 300 and emotion data as shown in FIG. Set emotion data by referring to the table. The setting unit 304 outputs the set emotion data to the transmission unit 306.

  Also in the present embodiment, as in the first embodiment, the input unit 303 detects a hitting pattern instead of the magnitude of acceleration, and the setting unit 304 refers to the table shown in FIG. Emotion data may be set according to the direction and the pattern of hitting.

  The communication partner designating unit 305 is used by the user of the communication device 300 to designate a communication partner who wants to express emotion among a plurality of communication partners indicated by the received video, and is configured by, for example, a cross key or a numeric keypad. . A user of the communication device 300 designates a communication partner to express an emotion by operating, for example, a cross key or a numeric keypad. The communication partner specifying unit 305 outputs communication partner information indicating the specified communication partner to the transmission unit 306.

  The transmission unit 306 transmits the communication partner information input from the communication partner specifying unit 305 and the emotion data input from the setting unit 304 (that is, emotion data for the communication partner specified by the communication partner specifying unit 305) to the communication server. 400. Further, the transmission unit 306 transmits the video of the user of the communication device 300 to the communication server 400.

  FIG. 8 is a configuration diagram of the communication server 400 according to the present embodiment. Hereinafter, a case where the communication server 400 communicates with the communication devices 300A to 300C illustrated in FIG. 6 will be described as an example.

  In communication server 400, reception unit 401 receives video (for example, an avatar image of each user) of each user of communication devices 300A to 300C from each of communication devices 300A to 300C. Then, the reception unit 401 outputs the received video to the conversion unit 402 and the transmission unit 403. The receiving unit 401 receives communication partner information and emotion data transmitted from each of the communication devices 300 </ b> A to 300 </ b> C, outputs them to the conversion unit 402, and outputs the communication partner information to the transmission unit 403.

  The conversion unit 402 converts the image of the communication partner indicated in the communication partner information according to the emotion data. Similar to the conversion unit 104 of the first embodiment, the conversion unit 402 refers to the table shown in FIG. 3 or 5 and converts the video according to the emotion data. The conversion unit 402 sends the converted video to the transmission unit 403.

  The transmission unit 403 transmits the video of each user of the communication devices 300A to 300C to each of the communication devices 300A to 300C. The transmission unit 403 transmits the video converted by the conversion unit 402 to a communication device other than the communication partner indicated in the communication partner information among the communication devices 300A to 300C.

  Here, in the communication system illustrated in FIG. 6, for example, when the user of the communication device 300A expresses the feeling of “anger” with respect to the user of the communication device 300C, the transmission unit 306 of the communication device 300A performs communication indicating the communication device 300C. The other party information and emotion data: “anger” is transmitted to the communication server 400.

  The conversion unit 402 of the communication server 400 enlarges the image around the nose in the video of the user of the communication device 300C indicated in the communication partner information according to the emotion data: “anger”, and also displays a nasal breath image and an anger mark. Is converted into an image of a surprised facial expression.

  Then, the transmission unit 403 of the communication server 400 transmits the converted video of the user of the communication device 300C only to the communication devices 300A and 300B without transmitting to the communication device 300C. Therefore, in the video of the user of the communication device 300C, the converted video is newly displayed on the output unit 302 of the communication devices 300A and 300B, while the video before conversion is continuously displayed on the output unit 302 of the communication device 300C. Is displayed.

  As described above, according to the present embodiment, in communication using an avatar image or the like, the image of the communication partner converted according to the emotion of the communication device user is not transmitted to the communication partner. Emotions can be expressed intuitively without causing discomfort to the communication partner.

(Embodiment 3)
FIG. 9 is a configuration diagram of the communication apparatus 500 according to the present embodiment, and FIG. 10 is a configuration diagram of the communication server 600 according to the present embodiment. Note that the communication system according to the present embodiment includes a communication server 600 and a plurality of communication devices, for example, communication devices 500A to 500C, as in the second embodiment (FIG. 6). Communication is performed with each other via the communication server 600. Communication devices 500A to 500C all have the configuration shown in FIG.

  In the communication device 500, the receiving unit 501 receives images of a plurality of communication partners from the communication server 600. For example, when the communication device 500A performs videophone communication over the network with the communication devices 500B and 500C, the receiving unit 501 of the communication device 500A is transmitted from the communication devices 500B and 500C as communication partners via the communication server 600. Receive avatar images of two communication partners. In addition, the reception unit 501 outputs the received image of the communication partner to the output unit 502.

  The output unit 502 outputs the video received by the receiving unit 501. In the present embodiment, for example, the output unit 502 is configured by a display that displays an image.

  The input unit 503 is for the user of the communication device 500 to input feelings for any one of the plurality of communication partners indicated by the received video, and like the input unit 102 of the first embodiment, For example, it is constituted by an acceleration sensor that detects the direction and magnitude of acceleration, and as shown in FIG. 2, it is possible to detect acceleration in three directions of the X-axis, Y-axis, and Z-axis applied to the communication device 500.

  The setting unit 504 sets emotion data corresponding to the emotion of the user of the communication device 500 input to the input unit 503 with respect to the communication partner. As with setting unit 103 in the first embodiment, setting unit 504 is set with the direction and magnitude of acceleration indicating the emotion of the user of communication device 500 and emotion data as shown in FIG. Set emotion data by referring to the table. The setting unit 504 outputs the set emotion data to the transmission unit 505.

  Also in the present embodiment, as in the first embodiment, the input unit 503 detects a hitting pattern instead of the magnitude of acceleration, and the setting unit 504 refers to the table shown in FIG. Emotion data may be set according to the direction and the pattern of hitting.

  Transmission unit 505 transmits emotion data input from setting unit 504 to communication server 600. In addition, the transmission unit 505 transmits the video of the user of the communication device 500 to the communication server 600.

  Hereinafter, a case where the communication server 600 communicates with the communication devices 500A to 500C will be described as an example.

  In communication server 600, reception unit 601 receives video of each user of communication devices 500A to 500C (for example, an avatar image of each user) from each of communication devices 500A to 500C. Then, the reception unit 601 outputs the received video to the conversion unit 602 and the transmission unit 603. The receiving unit 601 receives emotion data transmitted from each of the communication devices 500 </ b> A to 500 </ b> C and outputs the emotion data to the converting unit 602.

  The identifying unit 604 identifies the communication partner that is the target of emotion data by analyzing the received video or audio input to the communication devices 500A to 500C. For example, the identifying unit 604 detects the loudness (volume) of each user's voice input to the communication devices 500A to 500C, and designates a user whose current volume during communication is a predetermined threshold or more as a current speaker, Identify the communication partner that is the target of emotion data. The identification unit 604 outputs communication partner information indicating the identified communication partner to the conversion unit 602 and the transmission unit 603.

  The conversion unit 602 converts the image of the communication partner indicated in the communication partner information according to the emotion data. Similar to the conversion unit 104 of the first embodiment, the conversion unit 602 refers to the table shown in FIG. 3 or 5 and converts the video according to the emotion data. The conversion unit 602 sends the converted video to the transmission unit 603.

  The transmission unit 603 transmits the video of each user of the communication devices 500A to 500C to each of the communication devices 500A to 500C. The transmission unit 603 transmits the video converted by the conversion unit 602 to a communication device other than the communication partner indicated in the communication partner information among the communication devices 500A to 500C.

  Here, for example, when the user of the communication device 500A expresses the feeling of “anger” with respect to the user of the communication device 500C, the transmission unit 505 of the communication device 500A displays the emotion data: “anger” while the user of the communication device 500C speaks. Is transmitted to the communication server 600.

  The specifying unit 604 of the communication server 600 specifies the user of the communication device 500C whose current volume during communication is equal to or greater than a predetermined threshold as the current speaker, that is, the communication partner that is the target of emotion data, and the communication device 500C. Is output to the conversion unit 602 and the transmission unit 603.

  The conversion unit 602 of the communication server 600 enlarges an image around the nose in the video of the user of the communication device 500C indicated in the communication partner information in accordance with emotion data: “anger”, and also displays a nose breath image and an anger mark. Is converted into an image of a surprised facial expression.

  Then, the transmission unit 603 of the communication server 600 transmits the converted video of the user of the communication device 500C only to the communication devices 500A and 500B without transmitting to the communication device 500C. Therefore, in the video of the user of the communication device 500C, the converted video is newly displayed on the output unit 502 of the communication devices 500A and 500B, while the video before conversion is continuously displayed on the output unit 502 of the communication device 500C. Is displayed.

  Thus, according to the present embodiment, since the communication server analyzes the received video or the input voice to each communication device and identifies the communication partner that is the target of emotion data, the user of each communication device This saves you the trouble of specifying the communication partner you want to represent.

  The embodiment of the present invention has been described above.

  In the first to third embodiments, the video of the user of the communication device is converted according to the emotion data. However, in the present invention, the voice of the user of the communication device may be converted according to the emotion data. For example, the converters 104, 402, and 602 distort the voice waveform to shake the voice when “anger” emotion data is input, and adjust the voice pitch when “sadness” emotion data is input. Is increased by 2 octaves. When emotion data of “agreement” is input, the voice is echoed and the sound is echoed. When emotion data of “opposite” is input, the voice speed is reduced to a slower low voice. To do.

  The present invention is useful for a communication system in which chat or the like is performed.

100, 300, 500 Communication device 101, 301, 401, 501, 601 Receiving unit 102, 303, 503 Input unit 103, 304, 504 Setting unit 104, 402, 602 Conversion unit 105, 302, 502 Output unit 305 Communication partner designation Units 306, 403, 505, 603 Transmission unit 400, 600 Communication server 604 Identification unit

Claims (7)

Receiving means for receiving the video or audio of the communication partner;
Input means for inputting emotions to the communication partner;
Setting means for setting emotion data corresponding to the emotion;
Conversion means for converting the video or the sound according to the emotion data;
A communication apparatus comprising: Receiving means for receiving video or audio of a plurality of communication partners;
A designation means for designating a communication partner of the plurality of communication partners who wants to express emotion;
An input means for inputting emotions for a specified communication partner;
Setting means for setting emotion data corresponding to the emotion;
Transmission means for transmitting communication partner information indicating a specified communication partner and the emotion data for the specified communication partner to a communication server;
A communication apparatus comprising: Receiving means for receiving video or audio of a plurality of communication partners;
Input means for inputting emotions to any of the plurality of communication partners;
Setting means for setting emotion data corresponding to the emotion;
Transmitting means for transmitting the emotion data for any of the plurality of communication partners to a communication server;
A communication apparatus comprising: The input means is an acceleration sensor;
The communication apparatus according to claim 1. Receiving means for receiving the communication partner information and the emotion data from the communication device according to claim 2;
According to the emotion data, conversion means for converting the video or audio of the communication partner indicated in the communication partner information among the plurality of communication partners,
Transmitting means for transmitting the converted video or audio to a communication partner other than the communication partner indicated in the communication partner information among the plurality of communication partners;
A communication server comprising: Receiving means for receiving the emotion data from the communication device according to claim 3;
A specifying means for specifying a communication partner as a target of the emotion data;
According to the emotion data, conversion means for converting video or audio of the specified communication partner among the plurality of communication partners,
Transmitting means for transmitting the converted video or audio to a communication partner other than the specified communication partner among the plurality of communication partners;
A communication server comprising: The identifying means analyzes the received video or audio to identify a communication partner as a target of the emotion data;
The communication server according to claim 6.

Images