JP2009267623A - Communication device and voice communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for notifying a speaker whether a listener is actually listening to conversation in voice communication among a plurality of communication terminals. <P>SOLUTION: A terminal 10 decodes and mixes received voice data for outputting to a voice processing section 15, when receiving the voice data from another terminal 10. When the listener depresses a button B1, an operation section 18 outputs an operation signal corresponding to the operation content. A packet compositing section 17 transmits the voice data indicating voice collected by a microphone MC and notification data according to the operation signal output from the operation section 18 to another terminal 10. The voice processing section 15 processes the voice data so that the longer elapsed time after the last depression of the button B1 is, the larger the degree of deterioration in sound quality becomes. The voice data processed by the voice processing section 15 are supplied to a speaker SP for outputting as sound. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a communication device and a voice communication system.

In recent years, a remote conference system that performs a conference using a plurality of communication terminals connected via a communication network has become widespread. In such a teleconference system, since the speaker and listener are not directly facing each other, it is difficult for the speaker to feel the listener's reaction, and he / she feels uneasy about whether his / her voice reaches the other party. There is a case. Therefore, Patent Document 1 proposes a technique for displaying in real time on the transmission side what kind of state the transmitted voice / image data has reached on the reception side in order to eliminate the anxiety of the speaker. . Patent Document 2 discloses an input voice transmitted from another base (own base) connected via a network in an audio conference system that transmits and receives audio data between a plurality of bases connected via a network. A technique has been proposed in which an echo audio signal is sent back to another site, so that it can be confirmed at the other site that is sent back, whether it is being reproduced with the clear voice quality expected at the other site.
JP 2005-269498 A JP 2007-274176 A

However, with the techniques described in Patent Documents 1 and 2 described above, it is possible to confirm in what state the voice on the speaker side has reached the counterpart terminal, but is the listener really listening to the story? It is not possible to confirm whether or not. Even if the sound is surely delivered to the other party's terminal, there is a possibility that the listener is not listening to the voice played on the other party's terminal. In such a case, the speaker cannot confirm it. It was.
The present invention has been made in view of the above-described background. When performing voice communication between a plurality of communication terminals, it is possible to notify the speaker whether the listener is actually listening to the speech. The purpose is to provide technology.

  In order to solve the above-described problems, the present invention provides a reception unit that receives audio data transmitted from another terminal connected via a communication network, and an output unit that is output in advance from an operation unit that is operated by an operator. A signal detection unit for detecting a predetermined signal, a notification data transmission unit for transmitting notification data indicating that to the other terminal when the signal detection unit detects the signal, and the reception Audio data processing means for processing the audio data received by the means, wherein the audio data is processed so that the degree of processing becomes higher as the elapsed time from the detection of the signal by the signal detection means becomes longer. Provided is a communication device comprising: voice data processing means; and output means for outputting voice data processed by the voice data processing means to a sound emitting means. .

  In a preferred aspect of the present invention, the audio data processing means may process the audio data so that the degree of deterioration in sound quality increases as the elapsed time from the detection of the signal by the signal detection means increases. Good.

  In another preferable aspect of the present invention, the sound data processing means processes the sound data so that the sound pressure decreases as the elapsed time from the detection of the signal by the signal detection means increases. Also good.

  Further, in a further preferred aspect of the present invention, the operating means includes a plurality of operating elements, outputs a signal corresponding to the operating element operated by an operator, and the signal detecting means is provided to the operating element of the operating means. A corresponding signal is detected, and the notification data transmission means transmits notification data corresponding to an operator corresponding to the detected signal to the other terminal when the detection means detects the signal. May be.

  Moreover, in a further preferred aspect of the present invention, the receiving means receives voice data transmitted from a plurality of other terminals connected via a communication network for each of the other terminals, and is received by the receiving means. A sound pressure detecting means for detecting the sound pressure of the recorded voice data for each of the other terminals, and a terminal specification for specifying a terminal from which the sound pressure is detected by the sound pressure detecting means among the plurality of other terminals And when the signal detection means detects the signal, the notification data transmission means may send notification data indicating the fact to the terminal specified by the terminal specification means. Good.

  The present invention is also an audio communication system in which a first terminal and a second terminal are connected via a communication network, and the first terminal transmits audio transmitted from the second terminal. Audio data receiving means for receiving data; signal detecting means for detecting a predetermined signal output from an operating means operated by an operator; and when the signal detecting means detects the signal, to that effect Notification data transmitting means for transmitting notification data indicating to the other terminal, and voice data processing means for processing voice data received by the receiving means, wherein the signal detecting means detects the signal The sound data processing means for processing the sound data so that the degree of processing becomes higher as the elapsed time from the time elapses, and the sound data processed by the sound data processing means is emitted Voice data output means for outputting to the stage, and the second terminal transmits voice data representing the voice collected by the sound collection means to the first terminal; And a notification data receiving means for receiving the notification data transmitted by the notification data transmitting means, and a notifying means for notifying the contents indicated by the communication data received by the notification data receiving means. A communication system is provided.

  ADVANTAGE OF THE INVENTION According to this invention, when performing voice communication between several communication terminals, it can alert | report to a speaker whether a listener is actually listening to a talk.

<Configuration>
FIG. 1 is a block diagram showing a configuration of a remote conference system 1 according to an embodiment of the present invention. The remote conference system 1 is configured by connecting a plurality of terminals 10a, 10b, 10c... Installed in various places to a communication network 20 such as the Internet. In the following description, for convenience of description, when it is not necessary to distinguish the terminals 10a, 10b, 10c..., These will be referred to as “terminal 10”. A remote conference is realized by a voice conference performed by a participant of the remote conference using the terminal 10.

  FIG. 2 is a block diagram illustrating an example of the configuration of the terminal 10. In the figure, the microphone MC is a sound collecting means for collecting a voice uttered by a person participating in the conference (hereinafter referred to as “participant”) and outputting a voice signal (analog signal) representing the collected voice. is there. The audio signal output from the microphone MC is output to the CODEC 16. The CODEC 16 converts an audio signal output from the microphone MC into digital data. The operation unit 18 outputs a signal corresponding to an operation by a conference participant. The operation unit 18 is provided with a dedicated button B <b> 1 for inputting that the conference participants are listening to other participants properly. In the following description, for convenience of description, a signal output from the operation unit 18 when the button B1 is pressed will be referred to as “operation signal S1”. The operation signal S1 output from the operation unit 18 is output to the packet synthesizing unit 17 and also output to the voice processing unit 15. The packet combining unit 17 detects the operation signal S1 output from the operation unit 18, and generates notification data according to the operation signal S1 detected at the timing when the operation signal S1 is detected. The packet synthesizing unit 17 packetizes the audio data output from the CODEC 16 and the notification data corresponding to the operation signal S1 output from the operation unit 18, and transmits the packetized data to other terminals 10. This notification data is data generated when the participant actively presses the dedicated button B1, and thus can be used as data indicating whether the conference participant is listening to the speaker. .

  The terminal 10 is provided with a data receiving unit 22 for each of the other terminals 10 that perform voice communication. The terminal 10 individually receives data received from each of the other terminals 10. The packet separator 11 receives data transmitted from another terminal 10 connected via the communication network 20. The data transmitted from the other terminal 10 includes the voice data representing the sound collected by the microphone MC of the other terminal 10 and the listening status of the other participants (the participant listens to the speaker properly). Notification data indicating whether or not) is included. The packet separation unit 11 separates the voice data and the notification data from the received data, outputs the voice data to the CODEC 12, and outputs the notification data to the continuity determination unit 13. The CODEC 12 decodes the received audio data and outputs it to the audio mixer 14. The audio mixer 14 mixes audio signals output from the CODECs 12, 12,... Corresponding to each of the plurality of other terminals 10 and outputs the mixed audio signals to the speaker SP. The speaker SP emits sound according to the supplied audio signal.

  The continuity determination unit 13 determines whether or not the listener is listening to the speaker properly according to the notification data of the other terminal 10 supplied from the packet separation unit 11, and displays data indicating the determination result on the display unit 19. Output to. The display unit 19 displays an image indicating the determination result according to the data supplied from the continuity determination unit 13. As a display mode, for example, a message or an image indicating that the listener is listening to the speaker's story may be displayed, and the content corresponding to the received notification data is notified to the speaker. Anything can be used. Further, in the present embodiment, the speaker is notified by displaying an image representing the content of the notification data on the display unit 19. However, the notification mode is not limited to this, and for example, a voice message corresponding to the notification data is displayed on the speaker. The sound may be emitted from the SP, and any may be used as long as the content corresponding to the notification data is notified to the speaker.

  The audio processing unit 15 processes the audio data output from the audio mixer 14 and outputs the processed audio data to the speaker SP. The voice processing unit 15 detects the operation signal S1 output from the operation unit 18, and the longer the elapsed time after the operation signal S1 is detected (that is, after the button B1 is pressed by the participant), the greater the degree of processing. The audio data is processed so that becomes higher. In this embodiment, the audio processing unit 15 includes a gain adjustment unit (not shown) that adjusts the output gain of audio data, and the sound pressure increases as the elapsed time from the operation signal S1 output from the operation unit 18 increases. Adjust the output gain to make it smaller.

  FIG. 3 is a diagram illustrating an example of a change in gain of audio data output from the audio processing unit 15. In the figure, the horizontal axis indicates time, and the vertical axis indicates the level of output gain. In the example shown in FIG. 3, the voice processing unit 15 changes the output gain from the reference level (hereinafter referred to as “reference level”) from the time t1 when a predetermined time has elapsed since the start of voice communication. Gradually lowers over time. When the button B1 is pressed by the listener at time t2, the operation unit 18 outputs an operation signal S1 corresponding to the operated content to the sound processing unit 15. When the operation signal S1 is supplied from the operation unit 18, the sound processing unit 15 returns the output gain to the reference level (see time t2 in FIG. 3). When a predetermined time elapses from time t2, the voice processing unit 15 starts a process of gradually reducing the output gain of the voice data (see time t3 in FIG. 3). Thereafter, the voice processing unit 15 continues the process of gradually reducing the output gain until the button B1 is pressed again (see time t4 in FIG. 3).

  As described above, in the present embodiment, the sound pressure emitted becomes lower as time elapses after the button B1 is pressed, and the emitted sound becomes difficult to hear. Then, when the listener presses the button B1, the gain returns to the reference level. As a result, the listener must periodically press the button B1, and as a result, the speaker can check the listener's listening status based on the notification data periodically transmitted from the terminal 10 of the listener. Can be recognized.

<Operation>
Next, the operation of this embodiment will be described. First, the terminals 10 perform voice communication with each other. The terminal 10 transmits the audio data representing the sound collected by the microphone MC to the other terminal 10, receives the audio data received from the other terminal 10 via the communication network 20, and receives the received audio data Is emitted as a sound from the speaker SP. Thereby, a remote conference is realized.

  The user of the terminal 10 operates the operation unit 18 and listens to the speaker's story while pressing the dedicated button B1 every predetermined time length. Here, when the button B1 is pressed, the operation unit 18 outputs an operation signal S1 according to the operated content. The sound collected by the microphone MC is encoded by the CODEC 16 and output to the packet synthesizer 17 as sound data. The packet synthesizing unit 17 packetizes the audio data output from the CODEC 16 and the notification data corresponding to the operation signal S1 output from the operation unit 18, and transmits the packetized data to another terminal 10.

  On the other hand, the packet separation unit 11 separates the voice data and the notification data from the received data, outputs the voice data to the CODEC 12, and outputs the notification data to the continuity determination unit 13. The CODEC 12 decodes the supplied audio data and outputs it to the audio mixer 14. The audio mixer 14 mixes the audio data output from the plurality of CODECs 12 and outputs it to the audio processing unit 15.

  The voice processing unit 15 adjusts the output gain so that the sound pressure becomes lower as the elapsed time from when the operation signal S1 is output from the operation unit 18 is longer. The audio data whose gain has been adjusted by the audio processing unit 15 is output to the speaker SP. From the speaker SP, the voice represented by the voice data received from the other terminal 10 is emitted. At this time, the longer the elapsed time from when the button B1 is pressed, the lower the sound pressure of the emitted sound.

  The plurality of notification data separated by the packet separation unit 11 is supplied to the continuity determination unit 13. The continuity determination unit 13 determines whether the listener is listening properly according to the supplied notification data, and outputs data indicating the determination result to the display unit 19. The display unit 19 displays an image indicating the determination result of the continuity determination unit 13 according to the data supplied from the continuity determination unit 13.

  As described above, in the present embodiment, the sound pressure emitted becomes lower as time elapses after the button B1 is pressed, and the emitted sound becomes difficult to hear. Since the terminal 10 processes and outputs the sound such that the sound deteriorates as the time elapsed since the button B1 was pressed, this facilitates the listener to actively press the button B1. be able to. Further, the speaker can recognize that the call has been established by the notification data periodically transmitted from the listener's terminal 10.

<Modification>
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. An example is shown below. In addition, you may combine each following aspect suitably.
(1) In the above-described embodiment, the case where a remote conference is performed using the terminal according to the present invention has been described. However, the present invention is not limited to this, and for example, when a lecture or lecture is performed via a communication network. The present invention can be applied.

(2) In the above-described embodiment, the voice processing unit 15 processes the voice data so that the sound pressure becomes lower as the elapsed time after the button B1 is pressed. For example, the audio processing unit 15 may process the audio data so that the degree of deterioration in sound quality increases as the elapsed time from the time when the button B1 is pressed is longer. In this case, specifically, for example, the voice processing unit 15 may process the voice data so that the proportion of noise included increases as the elapsed time from the pressing of the button B1 increases. Good. Further, for example, the sound quality may be deteriorated by increasing the sound pressure level in a predetermined frequency band as the elapsed time after the button B1 is pressed is longer. As described above, the sound processing unit 15 may gradually reduce the sound pressure of the sound, may be processed so that the sound quality gradually deteriorates, or may be combined. In short, the voice processing unit 15 may process the voice data so that the longer the elapsed time since the button B1 was last pressed, the higher the degree of processing. In short, the voice processing unit 15 processes and outputs the voice data so that the longer the elapsed time since the button B1 was last pressed, the harder it is to hear the voice emitted from the speaker SP. That's fine.

(3) In the above-described embodiment, the notification data indicating that the dedicated button B1 is pressed by the user is used. However, the notification data is not limited to this, and the sound collected by the microphone MC of the terminal 10 is used. Notification data may be generated by analyzing voice data to be represented. The configuration in this case will be described with reference to FIG. In FIG. 4, the packet separation unit 11, the CODEC 12, the continuity determination unit 13, the audio mixer 14, the CODEC 16, the packet synthesis unit 17, and the audio processing unit 15 are the same as the respective units shown in FIG. Detailed description thereof is omitted.

  In FIG. 4, the voice analysis unit 21 analyzes the voice data output from the CODEC 16, and generates notification data according to the analysis result. Specifically, for example, in a predetermined memory, data for matching, which expresses the characteristics of a nodding voice, a nodding voice (hereinafter referred to as “nodding voice”) and a nodding voice, is stored in advance. The voice data output from the CODEC 16 is collated with the data for collation stored in the memory, and voice data satisfying a predetermined collation result (hereinafter “nodding voice data”) is extracted. Notification data may be generated when a message is detected. In this case, the voice processing unit 15 processes the voice data so that the degree of processing increases as the elapsed time from the last detection of the nodding voice increases. As a result, the longer the elapsed time since the listener uttered the nodding sound, the more difficult it is to hear the sound. By doing in this way, a listener's listening situation can be alert | reported to a speaker by a listener's nodding voice, and it can accelerate | stimulate that a listener emits a nodding voice.

  Further, not only the voice data is collated with the collation data, for example, the voice analysis unit 21 detects the sound pressure of the voice data output from the CODEC 16 and generates notification data according to the detected sound pressure. You may do it. Specifically, for example, the voice analysis unit 21 may generate notification data when a sound pressure equal to or higher than a predetermined threshold is detected. In this case, when a sound pressure equal to or higher than a predetermined threshold is not continuously detected (when it is too quiet), the longer the duration is, the harder it is to hear the emitted sound. On the other hand, when sound pressure equal to or higher than a predetermined threshold is detected for a predetermined period of time (when it is too noisy), it is assumed that the speaker is not listening and the notification data is not generated. Also good.

  In the above-described embodiment, the nodding operation may be detected by image analysis. In this case, a configuration is provided in which photographing means for photographing the user of the terminal 10 is provided, image data output from the photographing means is subjected to image analysis, nodding motion is detected according to the analysis result, and nodding motion is detected Notification data may be generated.

(4) In the above-described embodiment, the terminal 10 notifies the listener of the listening situation by displaying an image representing the content indicated by the notification data on the display unit 19, but the notification mode is not limited to this. For example, notification may be made by emitting a predetermined notification sound, or notification may be made by outputting a voice message, for example. In short, any notification data may be output so that a message or information can be transmitted to the speaker by some means.

(5) In the above-described embodiment, the time during which the reference level is maintained (the time from when the output gain reaches the reference level until the gain reduction starts: time 0 to t1, time t2 to t3 in FIG. 3) is constant. However, the time for which the reference level is maintained does not have to be constant, and may be random.

(6) In the above-described embodiment, after the terminal 10 transmits the notification data, the terminal 10 may not transmit the next notification data until a predetermined time has elapsed. If the notification sound is emitted too frequently, the speaker may feel annoying. Therefore, for example, after sending notification data once, control may be performed such that at least 20 seconds are left until the next transmission.

(7) In the above-described embodiment, the terminal 10 generates the notification data in response to the operation signal S1 output from the operation unit 18 when the button B1 is pressed. Etc.) may be used. In this case, the terminal 10 may detect a signal corresponding to each operation element of the operation unit, and transmit notification data corresponding to the detected signal to another terminal 10. Specifically, for example, a button indicating “OK”, a button indicating “NG”, or the like is provided in the operation unit, and the terminal 10 receives notification data corresponding to the pressed button (notification data indicating “OK” or “ Notification data indicating “NG” or the like) may be transmitted to another terminal 10.

(8) In the above-described embodiment, the notification data may be transmitted to the terminal 10 of the participant who is speaking without transmitting the notification data to all the other terminals 10 that are performing voice communication. . In this case, the terminal 10 detects the sound pressure of the received sound pressure data for each of the other terminals 10 and identifies the terminal 10 from which the sound pressure is detected from among the plurality of other terminals 10. Notification data may be transmitted to the terminal 10.

(9) Each unit of the terminal 10 in the above-described embodiment may be configured as hardware, and a control unit such as a CPU (Central Processing Unit) executes a computer program stored in a storage unit such as hardware. By doing so, it may be realized as software. In this case, the program executed by the control unit is a computer-readable recording medium such as a magnetic recording medium (magnetic tape, magnetic disk, etc.), an optical recording medium (optical disk, etc.), a magneto-optical recording medium, and a semiconductor memory. Can be provided in a recorded state. It is also possible to download to the terminal 10 via a network such as the Internet.

It is a block diagram which shows an example of a structure of a remote conference system. It is a block diagram which shows an example of a structure of a terminal. It is a figure which shows an example of the content of the process of an audio processing part. It is a block diagram which shows an example of a structure of a terminal.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Remote conference system, 10 ... Terminal, 11 ... Packet separation part, 12 ... CODEC, 13 ... Continuity determination part, 14 ... Audio mixer, 15 ... Audio processing part, 16 ... CODEC, 17 ... Packet synthesis part, 18 ... Operation , 19 ... display unit, 20 ... communication network, 21 ... voice analysis unit, 22 ... data reception unit.

Claims (6)

Receiving means for receiving audio data transmitted from another terminal connected via a communication network;
Signal detection means for detecting a predetermined signal output from an operation means operated by an operator;
Notification data transmitting means for transmitting notification data indicating that to the other terminal when the signal detecting means detects the signal;
Audio data processing means for processing the audio data received by the receiving means, wherein the audio data is processed so that the degree of processing increases as the elapsed time from the detection of the signal by the signal detection means increases. Audio data processing means for processing;
Output means for outputting the sound data processed by the sound data processing means to the sound emitting means. The sound data processing means processes the sound data so that the degree of deterioration of sound quality becomes higher as the elapsed time from the detection of the signal by the signal detection means becomes longer. The communication device described. 2. The communication according to claim 1, wherein the voice data processing means processes the voice data such that a sound pressure becomes smaller as an elapsed time after the signal detection means detects the signal. apparatus. The operation means includes a plurality of operators, and outputs a signal corresponding to the operator operated by the operator,
The signal detection means detects a signal corresponding to the operation element of the operation means,
The notification data transmission unit transmits notification data corresponding to an operation element corresponding to the detected signal to the other terminal when the detection unit detects the signal. Item 4. The communication device according to any one of Items 1 to 3. The receiving means receives voice data transmitted from a plurality of other terminals connected via a communication network for each of the other terminals,
A sound pressure detecting means for detecting the sound pressure of the voice data received by the receiving means for each of the other terminals;
Terminal specifying means for specifying a terminal whose sound pressure is detected by the sound pressure detecting means from among the plurality of other terminals, and
2. The notification data transmitting unit, when the signal detecting unit detects the signal, transmits notification data indicating the fact to the terminal specified by the terminal specifying unit. 5. The communication device according to any one of items 4 to 4. A voice communication system in which a first terminal and a second terminal are connected via a communication network,
The first terminal is
Voice data receiving means for receiving voice data transmitted from the second terminal;
Signal detection means for detecting a predetermined signal output from an operation means operated by an operator;
Notification data transmitting means for transmitting notification data indicating that to the other terminal when the signal detecting means detects the signal;
Audio data processing means for processing the audio data received by the receiving means, wherein the audio data is processed so that the degree of processing increases as the elapsed time from the detection of the signal by the signal detection means increases. Audio data processing means for processing;
Voice data output means for outputting the voice data processed by the voice data processing means to the sound emitting means,
The second terminal is
Audio data transmitting means for transmitting audio data representing the sound collected by the sound collecting means to the first terminal;
Notification data receiving means for receiving notification data transmitted by the notification data transmitting means;
A voice communication system, comprising: notification means for notifying the contents indicated by the communication data received by the notification data receiving means.

Images