JP2002374297A - Voice communication terminal and voice communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice communication system that enables a user to be able to grasp conversation timing with the opposite party, even if voice transfer delay occurs and enables the user to be able to make communication with the opposite party by taking into account the voice transfer delay. SOLUTION: A voice communication terminal 1A is provided with a VOCOD ER 101A that encodes voice data XA, a data multiplexer 102A that multiplexes the data XA with feedback voice data FXB to generate multiplexed data MDA, a transmission line modulator 103A that modulates the data MDA and transmits the modulated data, a transmission line demodulator 109A that receives and demodulates multiplexed data MDB, a data distributor 108A that separates the data MDB into voice data XB and feedback voice data FXA, a voice decoder 107A that decodes the data XB, a feedback vocoder 104A, that re-encodes the data XB as the data FXB, a feedback voice decoder 105A that decodes the data FXA, and a gain controller 106A that mixes the data XB and the data FXA in accordance with a set ratio, and the voice communication terminal 1B, having the same configuration as that of the terminal 1A, is connected to the terminal 1A via a transmission line 2.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a VOIP (Voice).
And a voice communication terminal device constituting the system, wherein the voice communication system and the voice communication terminal device are provided with a voice transmission delay countermeasure (a voice delay countermeasure system and a voice delay countermeasure terminal, respectively). Device).

[0002]

2. Description of the Related Art A VOIP system is not a general exchange network (GSTN) used for conventional telephones and the like.
This is a technique for transmitting voice data using a computer network protocol (IP) using the Internet that is becoming widespread as a computer network system. In this VOIP system, voice is digitally encoded, and compressed voice data is transmitted in packets.
The use of a technique such as not transmitting when there is no voice has advantages such as better use of the transmission path compared to a normal telephone.

[0003]

In the above-mentioned VOIP system, unlike the conventional general exchange network, the voice is digitally encoded, transmitted as a packet to the other party using the Internet network, decoded, and decoded. However, since the Internet network was originally developed as a network that does not require real-time properties, and the user side cannot select the routing, packet transmission delays occur in addition to code decoding delays. A voice transfer delay (a delay until the spoken voice reaches the other party's ear) occurs. Also, depending on the transmission path, fluctuations in packet transmission delay may occur, which may cause fluctuations in the voice transfer delay. Because of this voice transfer delay, it is not possible to hear the other party's voice instantly as in a normal conversation, and it is not possible to grasp the timing at which the other party listens to their own voice. There was a problem that it was difficult to have a conversation, or the conversation was not established. Thus VOIP
The system was a voice communication system that had problems in usage.

The present invention has been made to solve such a conventional problem. Even when a voice transfer delay occurs, the timing of a conversation with a partner can be grasped, and the voice transfer is considered in consideration of the voice transfer delay. It is an object of the present invention to provide a voice communication terminal device and a voice communication system that can perform the communication.

[0005]

According to a first aspect of the present invention, there is provided a voice communication terminal device for generating first multiplexed data including input first voice data. Encoding means for encoding the first audio data in a communication terminal device for transmitting and transmitting the second audio data and receiving the second multiplexed data including from the other party and outputting the second audio data; Multiplexing the multiplexed first audio data and the second feedback audio data, which is the second audio data re-encoded for feedback to the other party, to generate first multiplexed data Means,
A modulating means for modulating and transmitting the first multiplexed data according to the transmission specification, a demodulating means for receiving and demodulating the modulated second multiplexed data, and a demodulating means for demodulating the second multiplexed data. Distributing means for separating encoded second audio data and encoded first feedback audio data; decoding means for decoding the separated second audio data; Feedback encoding means for re-encoding the audio data as second feedback audio data, feedback decoding means for decoding the separated first feedback audio data, and the decoded second audio data and the decoded second audio data Mixing means for mixing the first feedback voice data with the set feedback ratio.

According to a second aspect of the present invention, the voice communication terminal device generates and transmits first multiplexed data including the input first voice data, and includes the second multiplexed data transmitted from the other party. In a communication terminal device that receives data and outputs second audio data, an encoding unit that encodes the input first audio data, and feeds back the encoded first audio data and the other party. Multiplexing means for multiplexing the second feedback voice data, which is the coded second voice data delayed for generating the first multiplexed data, and transmitting the first multiplexed data to the transmission specification. And demodulation means for receiving and demodulating the modulated second multiplexed data, and second audio data obtained by encoding the demodulated second multiplexed data. The first coded Distribution means for separating the separated feedback audio data; decoding means for decoding the separated second audio data; feedback decoding means for decoding the separated first feedback audio data; Of the second audio data in the decoding means.
Delay means for delaying by a total delay amount of the decoding delay amount of the audio data and the encoding delay amount of the first audio data in the encoding means, and outputting as the second feedback audio data; Mixing means for mixing the second audio data and the decoded first feedback audio data according to a set ratio.

The voice communication system according to claim 3 is connected to each other via a transmission line, and the voice communication terminals are connected to each other via a transmission line. A plurality of voice communication terminal devices, or one or more voice communication terminal devices according to claim 1 and one or more voice communication terminal devices connected to each other via a transmission path. , Wherein the first feedback voice data is first voice data decoded and re-encoded by the other party and fed back from the other party.

According to a fourth aspect of the present invention, the voice communication terminal device generates and transmits first multiplexed data including the input first voice data, and includes the second multiplexed data transmitted from the other party. A communication terminal device for receiving data and outputting second audio data, encoding means for encoding the first audio data and assigning first index data to the first audio data; The first
Multiplexing means for multiplexing the first audio data, the first index data, and the second feedback index data delayed for feedback to the other party, and generating first multiplexed data; A modulating means for modulating and transmitting the data according to the transmission specification, a demodulating means for receiving and demodulating the modulated second multiplexed data, and a demodulating means for modulating the demodulated second multiplexed data. Distributing means for separating the second audio data, the second index data, and the first feedback index data, decoding means for decoding the separated second audio data, and Delay by a total delay amount of the decoding delay amount of the second audio data in the decoding unit and the encoding delay amount of the first audio data in the encoding unit. Delay means for outputting as second feedback index data, feedback delay means for delaying the separated first feedback index data by a decoding delay amount of the second audio data in the decoding means, The voice data and the first index data allocated to the first voice data are held, and the first feedback index data is output at the timing when the delayed first feedback index data is output from the feedback delay unit. A buffer for outputting first audio data to which first index data corresponding to the index data is assigned; a second decoded audio data and a first audio data output from the buffer; Mixing means for mixing according to the specified ratio. Characterized in that was.

According to a fifth aspect of the present invention, there is provided a voice communication system comprising a plurality of voice communication terminals connected to each other via a transmission path, wherein the first feedback index data is transmitted to the other side on the other side. 1 is the first index data which is delayed by the total delay amount of the decoding delay amount of the audio data and the encoding delay amount of the second audio data and fed back from the other party, and the second index data is , Which is assigned to the second voice data on the other side.

According to a sixth aspect of the present invention, the voice communication terminal device generates and transmits first multiplexed data including the input first voice data, and includes the second multiplexed data transmitted from the other party. Encoding means for encoding the first audio data and assigning the first index data to the encoded first audio data in a communication terminal device for receiving the data and outputting the second audio data; Multiplexing means for multiplexing the encoded first audio data, the first index data, and the second feedback index data delayed for feedback to the other party to generate first multiplexed data Modulation means for modulating and transmitting the first multiplexed data according to the transmission specification,
Demodulation means for receiving and demodulating the modulated second multiplexed data; second audio data, second index data, and first feedback index obtained by encoding the demodulated second multiplexed data; Distribution means for separating the data into data, decoding means for decoding the separated second audio data, and a decoding delay amount of the second audio data in the decoding means for converting the separated second index data into the second index data Delay means for delaying by a total delay amount of the encoding delay amount of the first audio data in the encoding means, and outputting the result as second feedback index data; The first index data allocated to the encoded first audio data is held, and the first feedback index data is stored in the distribution unit. At a timing et output, first corresponding to the first feedback index data
Buffer means for outputting the encoded first audio data to which the index data is assigned, feedback decoding means for decoding the first audio data output from the buffer means, and the decoded second audio data. Mixing means for mixing the audio data and the decoded first audio data in accordance with a set ratio.

According to a seventh aspect of the present invention, there is provided a voice communication system comprising a plurality of voice communication terminals connected to each other via a transmission path, wherein the first feedback index data is transmitted to a third party on the other end. 1 is the first index data which is delayed by the total delay amount of the decoding delay amount of the audio data and the encoding delay amount of the second audio data and fed back from the other party, and the second index data is , Which is assigned to the encoded second audio data on the other side.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment FIG. 1 is a block diagram showing a voice delay countermeasure system according to a first embodiment of the present invention. In FIG. 1, the terminal device 1
A and the terminal device 1B are connected via the transmission line 2,
The terminal device 1A, the terminal device 1B, and the transmission line 2 constitute the above-described audio delay countermeasure system.

The terminal device 1A includes a speech encoder 101A.
, A data multiplexer 102A, and a transmission line modulator 103A.
, Feedback speech encoder 104A, feedback speech decoder 105A, and gain controller 1
06A, the speech decoder 107A, and the data distributor 10A.
8A, a transmission line demodulator 109A, an audio data input terminal IN1A, a gain setting data input terminal IN2A, and an audio data output terminal OUTA.

The terminal device 1B includes a speech encoder 10
1B, the data multiplexer 102B, and the transmission line modulator 10
3B, a feedback speech encoder 104B, a feedback speech decoder 105B, a gain controller 106B, a speech decoder 107B, a data distributor 108B, a transmission path demodulator 109B, and a speech data input terminal IN1B. , Gain setting data input terminal IN2B
And an audio data output terminal OUTB.

The configuration and operation of the terminal device 1A shown in FIG. 1 will be described below. The terminal device 1A and the terminal device 1
B has the same configuration. The speech encoder 101A of the terminal device 1A and the speech encoder 101B of the terminal device 1B, the data multiplexer 102A of the terminal device 1A and the data multiplexer 102B of the terminal device 1B have the same configuration. In the following description of the configuration and operation of the terminal device 1A, those in which the reference numerals are replaced with those in parentheses are descriptions of the configuration and operation of the terminal device 1B in FIG.

The audio data XA (XB) is input to the input terminal IN1A (IN1B), and the audio data XA (XB)
B) is input to the speech encoder 101A (101A).

The audio encoder 101A (101B) encodes the input audio data XA (XB) and outputs the encoded data to the data multiplexer 102A (102B). The above-described encoding includes processing such as audio compression for efficiently using the transmission path 2.

The data multiplexer 102A (102B)
The coded audio data XA (XB) and the feedback audio data FXB (FXA) from the feedback audio encoder 104A (104B) are multiplexed and output to the transmission line modulator 103A (103B).

The transmission line modulator 103A (103B) converts the format of the multiplexed data MDA (MDB) and sends it out to the transmission line 2. The above format conversion is
It is converted into a format used in the transmission path 2 represented by TCP / IP, and includes packetization and the like.

The transmission line demodulator 109A (109B) receives the multiplexed data MDB (MDA) transmitted to the transmission line 2 from the terminal device 1B (1A), and
The data distributor 10B performs an inverse conversion of the format conversion performed by the transmission line modulator 103B (103A) in (1A).
8A (108B).

The data distributor 108A (108B) separates the multiplexed data MDB (MDA) into coded voice data XB (XA) and coded feedback voice data FXA (FXB). , Audio data X
B (XA) is output to audio decoder 107A (107B), and feedback audio data FXA (FXB) is output to feedback audio decoder 105A (105B).

The audio decoder 107A decodes the encoded audio data XB (XB), and
06A (106B) and the feedback speech encoder 104A (104B).

The feedback speech encoder 104A (1
04B) encodes the audio data XB (XA) again,
The data is output to the data multiplexer 102A (102B) as feedback voice data FXB (FXA).

The feedback speech decoder 105A (1
05B) is the encoded feedback audio data F
XA (FXB) is decoded and gain controller 106A is decoded.
(106B).

The gain setting data rA (rB) is input to the input terminal IN2A (IN2B), and the gain setting data rA (rB) is input to the gain controller 106A (10
6A).

The gain controller 106A receives the audio data XB transmitted from the terminal device 1B,
Feedback audio data FXA, which is audio data of the terminal device 1A fed back from B, is mixed at a ratio according to the gain setting data rA, and for example, output audio data rA.XB + {(1-rA) FXA} is output to the output terminal OU.
Output from TA. However, 0 ≦ rA ≦ 1. The gain controller 106B of the terminal device 1B converts the audio data XA transmitted from the terminal device 1A and the feedback audio data FXB, which is the audio data of the terminal device 1B fed back from the terminal device 1A, according to the gain setting data rB. The output audio data rB.XA + {(1-rB) FXB} is output from the output terminal OUTB. However, 0 ≦ rB ≦ 1.

FIG. 2 is a diagram showing an example of the configuration of the gain controller 106A. The gain controller 106B of the terminal device 1B has the same internal configuration as the gain controller 106A of the terminal device 1A. Each of the reference numerals in FIG. 2 and the following description of FIG. 2 in which each reference numeral is replaced with a reference in parentheses is an example of the configuration of the gain controller 106B.

In FIG. 2, the gain controller 106
A (106B) includes multipliers 201 and 202 and an adder 2
03, input terminals in1 and in2, and an output terminal out. The input terminal in1 is connected to the audio decoder 107 in FIG.
A (107B), and the input terminal in2 is connected to the feedback speech encoder 104A (104B) of FIG.
The input terminal in3 is connected to the gain setting data input terminal IN2A (IN2B) in FIG. 1, and the output terminal out is connected to the audio data output terminal OUTA (OU) in FIG.
TB).

The multiplier 201 multiplies the audio data XB from the terminal device 1B input to the terminal in1 by the gain setting data rA input to the terminal in3, and obtains rA · XB
Is output to the adder 203. The multiplier 201 of the gain controller 106B multiplies the audio data XA from the terminal device 1A input to the terminal in1 by the gain setting data rB input to the terminal in3, and adds rB · XA to the adder 203. Output.

The multiplier 202 multiplies the feedback voice data FXA input to the terminal in2 by (1-rA) and outputs (1-rA) FXA to the adder 203. The multiplier 202 of the gain controller 106B
Multiplies the feedback voice data FXB input to the terminal in2 by (1-rB), and obtains (1-rB) FX
B is output to the adder 203.

The adder 203 calculates the above rA · XB and (1
−rA) FXA and add rA · XB + (1−rA)
FXA is output from the terminal out. This rA · XB +
(1-rA) FXA is audio data output from the audio data output terminal OUTA of the terminal device 1A. The output audio data rA · XB + (1−rA) FXA is rA = 1
Is set to only the audio data XB transmitted from the terminal device 1B, and when rA = 0, the terminal device 1B is fed back from the terminal device 1B.
Feedback audio data FXA which is audio data of A
Only. When 0 <rA <1 is set,
The audio data X at a rate corresponding to the gain setting data rA
B and the feedback audio data FXA are mixed. Note that the adder 203 of the gain controller 106B has r
B · XA and (1-rB) FXB are added, and rB · XA
+ (1-rB) FXB is output from the terminal out.

The flow of audio data XA and XB in the audio delay countermeasure system of FIG. 1 will be described below.

The audio data XA input to the terminal device 1A
Are an encoder 101A, a data multiplexer 102A, a transmission line modulator 103A, a transmission line 2, a transmission line demodulator 109B,
The data is input to the gain controller 106B and the feedback speech encoder 104B via the data distributor 108B and the speech decoder 107B. The audio data XA input to the gain controller 106B is mixed with the feedback audio data FXB and output from the output terminal OUTB.

Also, the feedback speech encoder 104
B is input to the data multiplexer 10.
2B, the transmission path modulator 103B, and the transmission path 2 are fed back to the terminal device 1A. The feedback audio data FXA is input to the gain controller 106A via the transmission path demodulator 109A, the data distributor 108A, and the feedback audio decoder 105A, mixed with the audio data XB, and output from the output terminal OUTA.

Similarly, the audio data XB input to the terminal device 1B is encoded by the encoder 101B and the data multiplexer 1B.
02B, transmission path modulator 103B, transmission path 2, transmission path demodulator 109A, data distributor 108A, speech decoder 10
The signal is input to gain controller 106A and feedback speech encoder 104A via 7A. The audio data XB input to the gain controller 106A is
It is mixed with the feedback audio data FXA and output from the output terminal OUTA.

The feedback speech coder 104
The audio data XB input to A is transmitted to the data multiplexer 10.
The signal is fed back to the terminal device 1B via the transmission line modulator 103A and the transmission line modulator 2A. The feedback audio data FXB is input to the gain controller 106B via the transmission path demodulator 109B, the data distributor 108B, and the feedback audio decoder 105B, mixed with the audio data XA, and output from the output terminal OUTB.

Here, the speech encoders 101A and 101A
B, the data input / output delay time of the feedback speech encoders 104A and 104B is Ta, the total data delay time of the data multiplexer 102A and the transmission line modulator 103A, and the data delay time of the data multiplexer 102B and the transmission line modulation The total data delay time of the device 103B is Tb,
The data transmission delay time of the transmission line 2 is defined as Tc, the total data delay time of the transmission line demodulator 109A and the data distributor 108A, and the transmission line demodulator 109B and the data distributor 10B.
The total data delay time of 8B is Td, and the data input / output delay times of the audio decoders 107A and 107B and the feedback audio decoders 105A and 105 are Te. Also, Ta + Tb + Tc + Td + Te =
Let it be T.

The voice data XA [0] input to the terminal device 1A at time t = 0 is input to the gain controller 106B and the feedback voice encoder 104B of the terminal device 1B at time t = T, and is output from the terminal device 1B. Is output.

The audio data XA [0] input to the feedback audio encoder 104B is input to the data multiplexer 102B as feedback audio data FXA [0] at time t = T + Ta. The audio data XB [T] input to the terminal device 1B at time t = T also
= T + Ta is input to the data multiplexer 102B.

The feedback voice data FXA [0] and the voice data XB [T] are both at time t = T + Ta
+ Tb from the terminal device 1B to the transmission line 2 at time t
= 2T, the signal is input to the gain controller 106A of the terminal device 1A and output from the terminal device 1A.

Similarly, voice data XB [0] input to terminal device 1B at time t = 0 is input to gain controller 106A and feedback voice encoder 104A of terminal device 1A at time t = T, Terminal device 1A
Output from

The voice data XB [0] input to the feedback voice encoder 104A is input to the data multiplexer 102A as feedback voice data FXB [0] at time t = T + Ta. Also, the audio data XA [T] input to the terminal device 1A at time t = T also
= T + Ta is input to the data multiplexer 102A.

Feedback voice data FXB [0] and voice data XA [T] are both at time t = T + Ta
+ Tb is transmitted from the terminal device 1A to the transmission path 2 at time t
= 2T is input to the gain controller 106B of the terminal device 1B and output from the terminal device 1B.

As described above, the audio data XA [0] transmitted from the terminal device 1A (1B) to the terminal device 1B (1A)
(XA [0]) is transmitted from the terminal device 1B (1A) to the terminal device 1
A (1B) is fed back, and the feedback voice data FXA [0] (FIXB [0]) is input to the gain controller 106A (106B), and the voice data XB [T] transmitted from the terminal device 3B (3A).
(XA [T]) and output from the terminal device 3A (3B).

Feedback voice data FXA [0]
(FIXB [0]) is the gain controller 106A (1
06B) is input to the terminal device 1A (1
Audio data XA [0] (XB [0]) input to B)
Code delay and transmission line delay in the terminal device 1A (1B) and the terminal device 1B (1A), and the audio data XB [0] input to the terminal device 1B (1A).
Considering the voice code decoding delay in the terminal device 1B (1A) and the terminal device 1A (1B) of (XA [0]), the timing is based on the total delay amount of these. As a result, the audio data XA [0] (XB [0]) is converted from the audio data XA [0] (XB [0]) to the terminal device 1B (1).
When output from A), it is mixed with audio data XB [T] (XA [T]) input to the terminal device 1B (1A) and output from the terminal device 1A (1B).

As described above, according to the first embodiment, the input sound of the terminal device 1A (1B) is
A (1B) and the terminal device 1B (1A) are mixed with the transmission voice from the terminal device 1B (1A) on the other side, including the code decoding delay and the transmission line delay in the terminal device 1B (1A). Can be heard at Accordingly, the user of the terminal device 1A (1B) can grasp the transfer delay of his / her voice even if the voice transfer delay and its fluctuation occur, so that the user can grasp the timing of the conversation with the other party. It becomes possible to talk while measuring the timing.

The mixing ratio between the input voice of the terminal device 1A (1B) and the voice transmitted from the other party is determined by the terminal device 1A (1B)
B), and the mixing ratio of the input sound can be set to 0. Therefore, when the user's own sound is annoying, such as when the delay is very small,
It is also possible to delete the user's own voice.

Note that the terminal device 1 of the first embodiment is
In A (1B), the feedback speech coder 10
4A (104B) instead of the data distributor 108A (1
08B) is separated from the audio data XB (XA) in the audio decoder 107A (107B).
(XA) decoding delay and speech encoder 101A (10
1B), the data multiplexer 102A delays by the total delay amount of the code delay of the audio data XA (XB).
A configuration may be adopted in which a delay means such as a data buffer or a delay unit for outputting to (102B) is provided.

Second Embodiment FIG. 3 is a block diagram showing an audio delay countermeasure system according to a second embodiment of the present invention. In FIG. 3, the same components as those in FIG. 1 are denoted by the same reference numerals. In FIG. 3, the terminal device 3A and the terminal device 3B are connected via the transmission line 2, and the terminal device 3A
The terminal device 3B and the transmission path 2 constitute the above-described audio delay countermeasure system.

The terminal device 3A includes a speech encoder 301A
, A data multiplexer 302A, and a transmission line modulator 303A.
, A frame index delay unit 304A, a feedback frame index delay unit 305A, a gain controller 306A, an audio decoder 307A, a data distributor 308A, a transmission line demodulator 309A, and an audio data buffer (FIFO buffer) 310A. And an audio data input terminal IN1A, a gain setting data input terminal IN2A, and an audio data output terminal OUTA.

Further, the terminal device 3 B
1B, a data multiplexer 302B, and a transmission line modulator 30.
3B, a frame index delay unit 304B, a feedback frame index delay unit 305B, a gain controller 306B, and a speech decoder 307B.
, A data distributor 308B, and a transmission path demodulator 309B.
And audio data buffer (FIFO buffer) 310B
And an audio data input terminal IN1B, a gain setting data input terminal IN2B, and an audio data output terminal OUTB.

The configuration and operation of the terminal device 3A shown in FIG. 3 will be described below. The following terminal devices 3A and 3B have the same configuration. The speech encoder 301A of the terminal device 3A and the speech encoder 301 of the terminal device 3B
B, the data multiplexer 302A of the terminal device 3A and the data multiplexer 302B of the terminal device 3B have the same configuration.
In the following description of the configuration and operation of the terminal device 3A,
Replace each code with the code in parentheses,
4 is an illustration of the configuration and operation of the terminal device 3B of FIG.

The audio data XA (XB) is input to the input terminal IN1A (IN1B).
B) is input to the audio encoder 301A (301B) and the audio data buffer 310A (310B).

The speech encoder 301A (301B) encodes the inputted speech data XA (XB) and outputs it to the data multiplexer 302A (302B). The above-described encoding includes processing such as audio compression for efficiently using the transmission path 2. Further, the speech encoder 301A (301
B) allocates an audio frame index IXA (IXB) to the audio data XA (XB), and stores the audio frame index data IXA (IXB) in the audio data buffer 310A (310B) and the data multiplexer 302.
A (302B).

The data multiplexer 302A (302B)
The encoded audio data XA (XB), the audio frame index data IXA (IXB) allocated to the audio data XA (XB), and the feedback audio frame index data FIXB (FIXB) from the frame index delay unit 304A (304B). FIXA) and outputs the result to the transmission path modulator 303A (303B).

The transmission line modulator 303A (303B) converts the format of the multiplexed data MDIA (MDIB) and sends out the data to the transmission line 2. The above-mentioned format conversion is a conversion to a format used in the transmission path 2 represented by TCP / IP, and includes packetization and the like.

The transmission line demodulator 309A (309B) receives the multiplexed data MDIB (MDIA) transmitted to the transmission line 2 from the terminal device 3B (3A), and
B (3A) performs an inverse conversion of the format conversion performed by the transmission line modulator 303B (303A), and the data distributor 3
08A (308B).

The data distributor 308A (308B) converts the multiplexed data MDIB (MDIA) into encoded audio data XB (XA) and the encoded audio data XB (XA).
Audio frame index data I allocated to A)
XB (IXA) and feedback audio frame index data FIXA (FIXB), and the audio data XB (XA) is decoded by the audio decoder 307A (307B).
To the audio frame index data IXB (I
XA) to the frame index delay unit 304A (304).
B), and outputs the feedback voice frame index data FIXA (FIXB) to the feedback frame index delay unit 305A (305A). The feedback voice frame index data FIXA (FIXB) is voice frame index data transmitted from the terminal device 1A to the terminal device 1B on the other side and returned from the terminal device 1B.
Is assigned to the audio data XA transmitted to the user. Also, the above-mentioned audio frame index data IX
B (IXA) is the voice frame index data transmitted from the terminal device 1B on the partner side,
Is assigned to the audio data XB transmitted from the.

The frame index delay unit 304A (3
04B) is the audio frame index data IXB
(IXA) is calculated based on the data input / output delay amount of the audio decoding device 307A (307B) and the audio encoding device 301A (3
01B) is delayed by the same amount as the total amount of data input / output delay amounts, and is used as feedback voice frame index data FIXB (FIXA).
2A (302B).

The audio decoder 307A (307B) decodes the encoded audio data XB (XA) and outputs it to the gain controller 306A (306B).

The feedback frame index delay unit 305A (305B) converts the fed back voice frame index data FIXA (FIXB) into
It delays by the same amount as the delay amount of the audio data XB (XA) in the audio decoder 307A (307B) and outputs it to the audio data buffer 310A (310B).

Audio data buffer 310A (310B)
Stores input audio data XA (XB) and audio frame index data IXA (IXB) allocated to the audio data XA (XB), and stores feedback audio frame index data FIXA (FIX
B) is a feedback frame index delay unit 30
5A (305B), the feedback voice frame index data FIXA (FIX
B) Voice frame index data IXA that matches
(IXB) is assigned to the audio data XA (XB),
Output to the gain controller 306A (306B).

That is, the audio data buffer 310A (3
10B) gains the audio data XA (XB) to which the input feedback audio frame index data FIXA (FIXB) is allocated by the gain controller 306A.
(306B), the input audio data XA (XB) of the terminal device 3A is output from the terminal device 3A at a timing in consideration of the code decoding delay and the transmission path delay. Voice frame index data IXA (IX
B), together with the input audio data XA (XB), the audio data buffer 310A to identify the encoded audio data XA (XB) to which the feedback audio frame index data FIXA (FIXB) is allocated.
(310B).

The gain setting data rA (rB) is input to the input terminal IN2A (IN2B), and the gain setting data rA (rB) is input to the gain controller 306A (30).
6B).

Gain controller 306A (306B)
Is the gain controller 106 according to the first embodiment.
A (106B) has the same internal configuration. The gain controller 306A mixes the audio data XB transmitted from the terminal device 3B and the input audio data XA of the terminal device 3A delayed by the audio data buffer 310A at a ratio according to the gain setting data rA, and outputs the output audio data. rA
XB + {(1-rA) XA} is output from the output terminal OUTA. The gain controller 30 of the terminal device 3B
6B is the audio data XA transmitted from the terminal device 3A,
The input audio data XB of the terminal device 3B delayed by the audio data buffer 310B and the gain setting data rB
And the output audio data rB · XA +
{(1-rB) XB} is output from the output terminal OUTB.

The flow of audio data XA and XB and audio frame index data IXA and IXB in the audio delay countermeasure system of FIG. 3 will be described below.

The audio data XA input to the terminal device 3A
Is an encoder 301A and an audio data buffer 310
A is input to A. The audio data XA input to the encoder 301A is transmitted to the data multiplexer 302A and the transmission path modulator 3
03A, the transmission path 2, the transmission path demodulator 309B, the data distributor 308B, and the speech decoder 307B.
B is mixed with the input audio data XB of the output terminal OUTB.
Output from

The audio data XA input to the audio data buffer 310A is temporarily stored in the audio data buffer 310A, and the audio data XB transmitted from the terminal device 3B.
Is output to the gain controller 306A at the timing synchronized with the audio data XA, mixed with the audio data XA from the terminal device 3B, and output from the output terminal OUTB.

The input audio data XA of the terminal device 3A
Is input from the audio encoder 301A to the audio data buffer 310A and the data multiplexer 302A. The audio frame index IXA input to the audio data buffer 310A is stored in the audio data buffer 310A together with the input audio data XA to which the audio frame index IXA is allocated.

The voice frame index IXA input to the data multiplexer 302 A is
A, transmission path 2, transmission path demodulator 309B, data distributor 3
08B, the frame index delay unit 304B, the data multiplexer 302B, the transmission line modulator 303B, and the feedback to the terminal device 3A via the transmission line 2. This feedback voice frame index data FIXA
Is input to the audio data buffer 310A via the transmission line demodulator 309A, the data distributor 308A, and the feedback frame index delay unit 305A.

The audio data buffer 310A outputs the audio data XA to which the feedback audio frame index data FIXA is allocated to the gain controller 306A at the timing when the feedback audio frame index data FIXA is input.

Similarly, audio data XB input to terminal device 3B is input to encoder 301B and audio data buffer 310B. The audio data XB input to the encoder 301B is transmitted to a data multiplexer 302B, a transmission path modulator 303B, a transmission path 2, and a transmission path demodulator 309.
A, via the data distributor 308A and the audio decoder 307A, input to the gain controller 306A, mixed with the input audio data XA of the terminal device 3A, and output terminal OU
Output from TA.

The audio data XB input to the audio data buffer 310B is temporarily stored in the audio data buffer 310B, and the audio data XA transmitted from the terminal device 3A.
Is output to the gain controller 306B at the timing synchronized with the audio data XA, mixed with the audio data XA from the terminal device 3A, and output from the output terminal OUTB.

The input voice data XB of the terminal device 3B
Is assigned to the audio frame index IXB.
From the audio encoder 301B to the audio data buffer 310B
And input to the data multiplexer 302B. The audio frame index IXB input to the audio data buffer 310B is stored in the audio data buffer 310A together with the input audio data XA to which the audio frame index IXA is allocated.

The voice frame index IXB input to the data multiplexer 302 B is
B, transmission path 2, transmission path demodulator 309A, data distributor 3
08A, the frame index delay unit 304A, the data multiplexer 302A, the transmission line modulator 303A, and the feedback to the terminal device 3B via the transmission line 2. This feedback voice frame index data FIXB
Is input to the audio data buffer 310B via the transmission line demodulator 309B, the data distributor 308B, and the feedback frame index delay unit 305B.

The audio data buffer 310B outputs the audio data XB to which the feedback audio frame index data FIXB is allocated to the gain controller 306B at the timing when the feedback audio frame index data FIXB is input.

Here, the speech encoders 301A and 301B
, And the total data delay time of the data multiplexer 302A and the transmission path modulator 303A, and the total data delay time of the data multiplexer 302B and the transmission path modulator 303B. Is also Tb, the data transmission delay time of the transmission line 2 is Tc, the total data delay time of the transmission line demodulator 309A and the data distributor 308A, and the total data delay time of the transmission line demodulator 309B and the data distributor 308B. Are Td,
Let Te be the data input / output delay time of the audio decoders 307A and 307B and the feedback frame index delay units 305A and 305B. In addition, the data input / output delay time of the frame index delay units 304A and 304B is both Te + Ta. Also, Ta + Tb
+ Tc + Td + Te = T.

The audio data XA [0] input to the terminal device 3A at the time t = 0 is represented by the time t = Ta + Tb + Tc + T
d is input to the audio decoder 307B of the terminal device 3B,
At time t = T, it is input to the gain controller 306B,
Output from the terminal device 3B.

The audio frame index data IXA [0] allocated to the audio data XA [0] is input to the frame index delay unit 304B of the terminal device 3B at time t = Ta + Tb + Tc + Td, and the time t = T
+ Ta is the feedback voice index data FIX
A [0] is input to the data multiplexer 302B.
The audio data XB [T] input to the terminal device 3B at time t = T and the audio frame index data IXB [T] allocated to the audio data XB [T] are also:
At time t = T + Ta, data is input to data multiplexer 302B.

The feedback voice frame index data FIXA [0] and the voice data XB [T] are
Both are transmitted from the terminal device 3B to the transmission line 2 at time t = T + Ta + Tb, and time t = T + Ta + Tb + Tc + Td.
Is output from the data distributor 308A of the terminal device 3A.

The audio data X transmitted from the terminal device 3B
B [T] indicates that the gain controller 306 is at time t = 2T.
A and is output from the terminal device 3A. Also, the feedback voice frame index data FIXA
[0] indicates that the audio data buffer 310A at time t = 2T
Is input to Thereby, the audio data buffer 310
The audio data XA [0] stored in A is at time t =
The signal is input to the gain controller 306A at 2T, and is output from the terminal device 3A together with the audio data XB [T].

Similarly, voice data XB [0] input to terminal device 3B at time t = 0 is obtained at time t = Ta + T
The audio decoder 307A of the terminal device 3A is added to b + Tc + Td.
And at time t = T, the gain controller 306A
And output from the terminal device 3A.

The voice frame index data IXB [0] allocated to the voice data XB [0] is input to the frame index delay unit 304A of the terminal device 3A at time t = Ta + Tb + Tc + Td, and the time t = T
+ Ta is the feedback voice index data FIX
The data is input to the data multiplexer 302A as B [0].
The audio data XA [T] input to the terminal device 3A at time t = T and the audio frame index data IXA [T] allocated to the audio data XA [T] are also:
At time t = T + Ta, data is input to data multiplexer 302A.

The feedback voice frame index data FIXB [0] and the voice data XA [T] are
Both are transmitted from the terminal device 3A to the transmission line 2 at the time t = T + Ta + Tb, and the time t = T + Ta + Tb + Tc + Td.
Is output from the data distributor 308B of the terminal device 3B.

Voice data X transmitted from terminal device 3A
A [T] indicates that the gain controller 306 is at time t = 2T.
B and output from the terminal device 3B. Also, the feedback voice frame index data FIXB
[0] indicates that the audio data buffer 310B at time t = 2T
Is input to Thereby, the audio data buffer 310
The audio data XB [0] stored in B at time t =
The signal is input to the gain controller 306B at 2T, and output from the terminal device 3B together with the audio data XA [T].

As described above, the audio data XA [0] (XB
[0]) and transmitted from the terminal device 3A (3B) to the terminal device 3B (3A), the voice frame index data IXA [0] (IXB [0]) is transmitted to the terminal device 3B.
(3A) is fed back to the terminal device 3A (3B), and when the feedback voice frame index data FIXA [0] (FIXB [0]) is input to the voice data buffer 310A (310B),
The audio data XA [0] (XB [0]) is supplied from the audio data buffer 310A (310B) to the gain controller 3A.
06A (306B) and the terminal device 3B (3A)
Is mixed with the audio data XB [T] (XA [T]) transmitted from the terminal device 3A (3B).

The above timing is determined by the terminal device 3A (3
Audio data XA [0] (XB [0]) input to B)
Code decoding delay and transmission path delay in the terminal device 3A (3B) and the terminal device 3B (3A), and the audio data XB [0] input to the terminal device 3B (3A).
Considering the voice code decoding delay in the terminal device 3B (3A) and the terminal device 3A (3B) of (XA [0]), the timing is based on the total delay amount of these. Thus, the audio data XA [0] (XB [0]) is converted from the audio data XA [0] (XB [0]) to the terminal device 3B (3
The audio data XB [T] (XA [T]) input to the terminal device 3B (3A) when output from the terminal device 3A is mixed and output from the terminal device 3A (3B).

As described above, according to the second embodiment, the input sound of the terminal device 3A (3B) is
A (3B) and the terminal device 3B (3A), including the code decoding delay and the transmission line delay in the terminal device 3B (3A), are mixed with the transmission sound from the partner terminal device 3B (3A). Can be heard at Thereby, the user of the terminal device 3A (3B) can grasp the delay amount of his / her voice even if the voice transfer delay and its fluctuation occur, so that the user can grasp the timing of the conversation with the other party. It becomes possible to talk while measuring the timing.

The mixing ratio of the input voice of the terminal device 3A (3B) and the voice transmitted from the other party is determined by the terminal device 3A (3B).
B), and the mixing ratio of the input sound can be set to 0. Therefore, when the user's own sound is annoying, such as when the delay is very small,
It is also possible to delete the user's own voice.

Further, in the second embodiment, not the audio data but the audio frame index data assigned to the audio data is transmitted to the other party and fed back from the other party. Since the amount is generally smaller than the data amount of the encoded audio data, the amount of transmission data can be made smaller than that of the first embodiment.

Third Embodiment FIG. 4 is a block diagram showing a voice delay countermeasure system according to a third embodiment of the present invention. In FIG. 4, the same components as those in FIG. 1 are denoted by the same reference numerals. In FIG. 4, the terminal device 4A and the terminal device 4B are connected via the transmission line 2, and the terminal device 4A
The terminal device 4B and the transmission path 2 constitute the above-described audio delay countermeasure system.

[0092] The terminal device 4A includes a speech coder 401A.
, A data multiplexer 402A, and a transmission line modulator 403A.
, A frame index delay unit 404A, a feedback speech decoder 405A, and a gain controller 4
06A, a speech decoder 407A, and a data distributor 40
8A, a transmission line demodulator 409A, an audio data buffer (FIFO buffer) 410A, an audio data input terminal IN1A, a gain setting data input terminal IN2A, and an audio data output terminal OUTA.

[0093] The terminal device 4B includes a speech coder 40.
1B, the data multiplexer 402B, and the transmission line modulator 40
3B, a frame index delay unit 404B, a feedback audio decoder 405B, a gain controller 406B, an audio decoder 407B, a data distributor 408B, a transmission path demodulator 409B, and an audio data buffer (FIFO buffer). 410B, an audio data input terminal IN1B, and a gain setting data input terminal IN2B
And an audio data output terminal OUTB.

The terminal devices 4A and 4B of the third embodiment store the encoded audio data in the audio data buffer in the terminal devices 3A and 3B of the second embodiment (see FIG. 2). It is like that.

The configuration and operation of the terminal device 4A shown in FIG. 4 will be described below. The following terminal devices 4A and 4B have the same configuration. Also, the speech encoder 401A of the terminal device 4A and the speech encoder 401 of the terminal device 4B.
B, the data multiplexer 402A of the terminal device 4A and the data multiplexer 402B of the terminal device 4B have the same configuration.
In the following description of the configuration and operation of the terminal device 4A,
Replace each code with the code in parentheses,
5 is a description of the configuration and operation of the terminal device 4B of FIG.

The audio data XA (XB) is input to the input terminal IN1A (IN1B).
B) is input to the speech encoder 401A (401B).

The speech encoder 401A (401B) encodes the inputted speech data XA (XB) and outputs it to the speech data buffer 410A (410B) and the data multiplexer 402A (402B). In the above encoding,
Processing such as voice compression for efficiently using the transmission path 2 is included. Further, the speech encoder 401A (401B)
Allocates an audio frame index IXA (IXB) to audio data XA (XB), and stores the audio frame index data IXA (IXB) in an audio data buffer 410A (410B) and a data multiplexer 402A.
(402B).

The data multiplexer 402A (402B)
The encoded audio data XA (XB), the audio frame index data IXA (IXB) allocated to the audio data XA (XB), and the feedback audio frame index data FIXB (FIXB) from the frame index delay unit 404A (404B). FIXA) and outputs the result to the transmission line modulator 403A (403B).

The transmission line modulator 403A (403B) converts the format of the multiplexed data MDIA (MDIB) and sends out the data to the transmission line 2. The above-mentioned format conversion is a conversion to a format used in the transmission path 2 represented by TCP / IP, and includes packetization and the like.

The transmission line demodulator 409A (409B) receives the multiplexed data MDIB (MDIA) sent to the transmission line 2 from the terminal device 4B (4A),
B (4A) performs inverse conversion of the format conversion performed by the transmission line modulator 403B (403A), and
08A (408B).

The data distributor 408A (408B) converts the input data MDIB (MDIA) into encoded audio data XB (XA) and the encoded audio data XB (XA).
Audio frame index data I allocated to A)
XB (IXA) and feedback audio frame index data FIXA (FIXB), and the audio data XB (XA) is decoded by the audio decoder 407A (407B).
To the audio frame index data IXB (I
XA) to the frame index delay unit 404A (404
B) and outputs the feedback voice frame index data FIXA (FIXB) to the voice data buffer 4.
Output to 10A (410B).

The frame index delay unit 404A (4
04B) is the audio frame index data IXB
(IXA) is calculated based on the data input / output delay amount of the audio decoding device 407A (407B) and the audio encoding device 401A (4
01B) is delayed by the same amount as the total delay amount of the data input / output amounts of the data multiplexer / demultiplexer 40 as feedback voice frame index data FIXB (FIXA).
2A (402B).

[0103] The audio decoder 407A (407B) decodes the encoded audio data XB (XA) and outputs it to the gain controller 406A (406B).

Audio data buffer 410A (410B)
Stores encoded input audio data XA (XB) and audio frame index data IXA (IXB) allocated to the audio data XA (XB),
Feedback voice frame index data FIX
When A (FIXB) is input from the data distributor 408A (408B), the encoded audio data XA to which the audio frame index data IXA (IXB) that matches the feedback audio frame index data FIXA (FIXB) is allocated. (XB) is output to the feedback speech decoder 405A (405B).

That is, the audio data buffer 410A (4
10B) outputs the encoded audio data XA (XB) to which the input feedback audio frame index data FIXA (FIXB) is allocated, to the feedback audio decoder 405A (405B), so that the terminal device 3A The input audio data XA (XB) is output from the terminal device 4A (4B) at a timing in consideration of the code decoding delay and the transmission path delay.

The feedback speech decoder 405A (4
05B) is an audio data buffer 410A (410B)
, Decodes the encoded audio data XA (XB) input from the controller and outputs it to the gain controller 406A (406B).

The gain setting data rA (rB) is input to the input terminal IN2A (IN2B), and the gain setting data rA (rB) is input to the gain controller 406A (40).
6B).

Gain controller 406A (406B)
Is the gain controller 106 according to the first embodiment.
A (106B) has the same internal configuration. The gain controller 406A mixes the audio data XB transmitted from the terminal device 4B and the input audio data XA of the terminal device 4A delayed by the audio data buffer 410A at a ratio according to the gain setting data rA, and outputs the output audio data. rA
XB + {(1-rA) XA} is output from the output terminal OUTA. The gain controller 40 of the terminal device 4B
6B is the audio data XA transmitted from the terminal device 4A,
The input audio data XB of the terminal device 4B delayed by the audio data buffer 410B and the gain setting data rB
And the output audio data rB · XA +
{(1-rB) XB} is output from the output terminal OUTB.

The audio data XA and the audio frame index data I in the audio delay countermeasure system of FIG.
The flow of XA will be described below. The following audio data XA and audio frame index data IX
In the description of the flow of A, the description of the flow of the audio data XB and the audio frame index data IXB is obtained by replacing the respective symbols with those in parentheses.

The audio data XA (XB) input to the terminal device 4A (4B) is encoded by the encoder 401A (301B), and the audio data buffer 410A (410B)
And input to the data multiplexer 402A (402B). The audio data XA (XB) input to the data multiplexer 402A (402B) is transmitted to the transmission line modulator 303A (3
03B), transmission path 2, transmission path demodulator 309B (309
A), via the data distributor 308B (308A) and the audio decoder 307B (307B), input to the gain controller 406B (406A), and delayed input audio data XB (XA) of the terminal device 4B (4A). And output from the output terminal OUTB (OUTA).

Audio data buffer 410A (410B)
Is temporarily stored in the audio data buffer 410A (410B), and the feedback audio decoder is synchronized with the audio data XB (XA) transmitted from the terminal device 4B (4A). 40
5A (405B), is decoded by the feedback speech decoder 405A (405B), and is inputted to the gain controller 406A (406B).
Terminal device 3B (4A) decoded at 07A (407B)
Is mixed with the audio data XB (XA) from the
Output from UTB (OUTA).

The audio frame index IXA (IXB) allocated to the input audio data XA (XB) of the terminal device 4A (4B) is stored in the audio encoder 401A (40).
1B) is input to the audio data buffer 410A (410B) and the data multiplexer 402A (402B). The audio frame index IXA (IXB) input to the audio data buffer 410A (410B) is together with the encoded input audio data XA (XB) to which the audio frame index IXA (IXB) is allocated. ).

Speech frame index IXA (IXB) input to data multiplexer 402A (402B)
Are transmission path modulators 403A (403B), transmission path 2, transmission path demodulator 409B (409A), data distributor 408
B (408A), frame index delay unit 404B
(404A), the data multiplexer 402B (402A),
The signal is fed back to the terminal device 4A (4B) via the transmission path modulator 403B (403A) and the transmission path 2. This feedback voice frame index data FIX
A (FIXB) is a transmission path demodulator 409A (409
B), and is input to the audio data buffer 410A (410B) via the data distributor 408A (408B).

Audio data buffer 410A (410B)
Is the feedback voice frame index data F
At the timing when IXA (FIXB) is input, the feedback voice frame index data FIXA
(FIXB) assigned encoded audio data X
A (XB) is converted to a feedback speech decoder 405A (4
05B).

Here, the speech encoders 401A and 401B
, The total data delay time of the data multiplexer 402A and the transmission line modulator 403A, and the total data delay time of the data multiplexer 402B and the transmission line modulator 403B. Is also Tb, the data transmission delay time of the transmission line 2 is Tc, the total data delay time of the transmission line demodulator 409A and the data distributor 408A, and the total data delay time of the transmission line demodulator 409B and the data distributor 408B. Are Td,
Let Te be the data input / output delay time of the audio decoders 407A and 407B and the feedback audio decoders 405A and 405B. Further, the data input / output delay time of the frame index delay units 404A and 404B is Te + Ta. Also, Ta + Tb + Tc + Td +
Let Te = T.

At time t = 0, audio data XA [0] (XA [0]) input to terminal device 4A (4B)
= T, the gain controller 30 of the terminal device 3B (3A)
6B (306A) and output from the terminal device 4B.

The above audio data XA [0] (XB
The audio frame index data IXA [0] (IXB [0]) allocated to [0]) is at time t = T + Ta
Feedback audio index data FIXA
Data multiplexer 402 as [0] (FIXB [0])
B (402A). The audio data XB [T] input to the terminal device 4B (4A) at time t = T
(XA [T]), and audio data XB [T] (XA
The voice frame index data IXB [T] (IXA [T]) allocated to [T]) is also at time t = T + Ta.
Is input to the data multiplexer 402B (402A).

Feedback audio frame index data FIXA [0] (FIXB [0]) and audio data XB [T] (XA [T]) are both at time t = T +
The data is output from the data distributor 408A (408B) of the terminal device 4A (4B) at Ta + Tb + Tc + Td.

The audio data XB [T] (XA [T]) transmitted from the terminal device 4B (4A) is input to the gain controller 406A (406B) at time t = 2T, and is output from the terminal device 4A (4B). Is done. Also, the feedback voice frame index data FIXA [0]
(FIXB [0]) is input to the audio data buffer 410A (410B) at time t = 2T. This allows
The audio data XA [0] (XB [0]) stored in the audio data buffer 410A (410B) is at time t =
The signal is input to the gain controller 406A (406B) at 2T, and output from the terminal device 4A (4B) together with the audio data XB [T] (XA [T]).

As described above, also in the third embodiment, similarly to the second embodiment, the terminal devices 4A (4B)
Of the audio data XA [0] (XB [0]) input to the terminal device 4A (4B) and the terminal device 4B (4A), and the input to the terminal device 4B (4A). Audio data XB [0] (XA
[0]) Terminal device 4B (4A) and terminal device 4A
At the timing according to the audio code decoding delay in (4B), the audio data XA [0] (XB [0]) is output to the gain controller 406A (406B) and transmitted from the terminal device 4B (4A). Data XB [T]
(XB [T]) and output from the terminal device 3A (3B). Thereby, the audio data XA [0] (XB
[0]) is the audio data XA [0] (XB [0])
Is output from the terminal device 4B (4A), the audio data XB [T] (XA) input to the terminal device 4B (4A).
[T]) and output from the terminal device 4A (4B).

As described above, according to the third embodiment, the input sound of the terminal device 4A (4B) is
A (4B) and the terminal device 4B (4A), including the code decoding delay and the transmission line delay in the terminal device 4B (4A), are mixed with the transmission voice from the terminal device 3B (3A) on the other side, and the terminal device 4A (4B) Can be heard at Thereby, the user of the terminal device 4A (4B) can grasp the delay amount of his / her voice even if the voice transfer delay and its fluctuation occur, so that the user can grasp the timing of the conversation with the other party. It becomes possible to talk while measuring the timing.

Further, the mixing ratio of the input voice of the terminal device 4A (4B) and the voice transmitted from the other party is determined by the terminal device 4A (4B).
B), and the mixing ratio of the input sound can be set to 0. Therefore, when the user's own sound is annoying, such as when the delay is very small,
It is also possible to delete the user's own voice.

Further, in the third embodiment, the encoded audio data XA (XB) is held in the audio data buffer 410A (410B), and the audio data XA (XB) is stored in the audio data buffer 410A (410B).
(XB) is converted to the feedback speech decoder 405A (40
5B) and the gain controller 406A (406
B), and the data amount of the encoded audio data XA (XB) is compressed more than before encoding, so that the audio data buffer 410A (4
The data capacity of 10B) can be smaller than that of the second embodiment. Audio data buffer 410A (410B)
And the feedback speech decoder 405A (405
If B) can be made more compact than the feedback frame index delay unit 305A (305B) and the audio data buffer 310A (310B) of the second embodiment, the third embodiment is adopted. By adopting the second embodiment, the size of the apparatus can be reduced.

In the first to third embodiments, an example has been described in which the terminal device is configured by hardware. However, the terminal device may be configured by software, or by hardware and software. May be configured.

[0125]

As described above, according to the present invention, the user can grasp the amount of delay of his / her voice even if the voice transfer delay and its fluctuation occur, so that the user can grasp the timing of conversation with the other party. The effect is that the conversation can be performed while measuring the timing.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an audio delay countermeasure system according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a configuration of a gain controller in FIG. 1;

FIG. 3 is a block diagram showing an audio delay countermeasure system according to a second embodiment of the present invention.

FIG. 4 is a block diagram showing an audio delay countermeasure system according to a third embodiment of the present invention.

[Explanation of symbols]

1A, 1B, 3A, 3B, 4A, 4B terminal device, 2
Transmission line, 101A, 101B, 301A, 301
B, 401A, 401B speech encoder, 102A,
102B, 302A, 302B, 402A, 402B
Data multiplexer, 103A, 103B, 303A, 30
3B, 403A, 403B Transmission line modulator, 104
A, 104B feedback speech encoder, 105
A, 105B, 405A, 405B Feedback speech decoder, 106A, 106B, 306A, 306
B, 406A, 406B Gain controller, 10
7A, 107B, 307A, 307B, 407A, 40
7B audio decoder, 108A, 108B, 308
A, 308B, 408A, 408B Data distributor, 1
09A, 109B, 309A, 309B, 409A, 4
09B transmission line demodulator, 304A, 304B, 404
A, 404B Frame index delay unit, 305
A, 305B Feedback frame index delay unit, 310A, 310B, 410A, 410B audio data buffer, IN1A, IN1B audio data input terminal, IN2A, IN2B gain setting data input terminal, OUTA, OUTB audio data output terminal.

Claims (7)

[Claims] The first audio data including the input first audio data.
A communication terminal device that generates and transmits multiplexed data of the first audio data and receives the second multiplexed data that is transmitted from the other party and outputs the second audio data. And multiplexing the encoded first audio data and the second feedback audio data, which is the second audio data re-encoded for feedback to the other party, Multiplexing means for generating one multiplexed data, modulating means for modulating and transmitting the first multiplexed data according to transmission specifications, and demodulation for receiving and demodulating the modulated second multiplexed data. Means for separating the demodulated second multiplexed data into coded second voice data and coded first feedback voice data; and distributing the separated second voice data. Decrypt Encoding means; feedback encoding means for re-encoding the decoded second audio data as second feedback audio data; feedback decoding means for decoding the separated first feedback audio data; Mixing means for mixing the decoded second audio data and the decoded first feedback audio data in accordance with a set ratio. 2. The method according to claim 1, wherein the first audio data includes first audio data.
A communication terminal device that generates and transmits multiplexed data of the first audio data and receives the second multiplexed data that is transmitted from the other party and outputs the second audio data. And multiplexing the encoded first audio data and the second feedback audio data, which is the encoded second audio data delayed for feedback to the other party. ,
Multiplexing means for generating first multiplexed data, modulating means for modulating and transmitting the first multiplexed data according to transmission specifications, and receiving and demodulating the modulated second multiplexed data. Demodulation means; distribution means for separating the demodulated second multiplexed data into coded second voice data and coded first feedback voice data; separated second voice data Decoding means for decoding the separated first feedback sound data; and decoding delay of the second sound data in the decoding means for decoding the separated second sound data. Delay means for delaying by a total delay amount of the first audio data and the encoding delay amount of the first audio data in the encoding means, and outputting as second feedback audio data; Mixing means for mixing the data and the decoded first feedback voice data in accordance with a set ratio. 3. The plurality of voice communication terminal apparatuses according to claim 1, which are connected to each other via a transmission path, or the plurality of voice communication terminal apparatuses according to claim 2, which are connected to each other via a transmission path. Alternatively, the first feedback voice data includes one or more voice communication terminal devices according to claim 1 and one or more voice communication terminal devices according to claim 2, which are connected to each other via a transmission path. Is the first voice data decoded and re-encoded by the other party and fed back from the other party. 4. The first audio data including the input first audio data.
A communication terminal device that generates and transmits multiplexed data of the first audio data and receives the second multiplexed data including the second multiplexed data transmitted from the other party and outputs the second audio data. Encoding means for assigning the first index data to the first audio data, and a delay for feeding back the encoded first audio data, the first index data and the other party to the other party Multiplexed with the second feedback index data,
Multiplexing means for generating first multiplexed data, modulating means for modulating and transmitting the first multiplexed data according to transmission specifications, and receiving and demodulating the modulated second multiplexed data. Demodulating means; distributing means for separating the demodulated second multiplexed data into coded second audio data, second index data, and first feedback index data; A decoding unit for decoding the audio data, a decoding delay amount of the second audio data in the decoding unit, and an encoding of the first audio data in the encoding unit. Delay means for delaying by a total delay amount and outputting as second feedback index data; and decoding means for separating the separated first feedback index data. Delay means for delaying the second audio data by the decoding delay amount, and holding the first audio data and the first index data assigned to the first audio data, and When the first feedback index data is output from the feedback delay means, the first feedback index data is allocated to the first index data corresponding to the first feedback index data.
Buffer means for outputting the second audio data, and mixing means for mixing the decoded second audio data and the first audio data output from the buffer means in accordance with a set ratio. Voice communication terminal device. 5. The apparatus according to claim 4, further comprising a plurality of voice communication terminals connected to each other via a transmission path, wherein the first feedback index data is a decoding delay of the first voice data on the other side. Quantity and second
Is the first index data which is delayed by the total delay amount of the encoding delay amount of the audio data and fed back from the other party, and the second index data is the second index data at the other party.
An audio communication system characterized in that the audio data is allocated to the audio data. 6. A first audio data including the input first audio data.
A communication terminal device that generates and transmits multiplexed data of the first audio data and receives the second multiplexed data including the second multiplexed data transmitted from the other party and outputs the second audio data. Encoding means for allocating the first index data to the encoded first audio data; and a delay for feeding back the encoded first audio data, the first index data, and the other party. Multiplexed with the second feedback index data,
Multiplexing means for generating first multiplexed data, modulating means for modulating and transmitting the first multiplexed data according to transmission specifications, and receiving and demodulating the modulated second multiplexed data. Demodulating means; distributing means for separating the demodulated second multiplexed data into coded second audio data, second index data, and first feedback index data; A decoding unit for decoding the audio data, a decoding delay amount of the second audio data in the decoding unit, and an encoding of the first audio data in the encoding unit. Delay means for delaying by a total delay amount and outputting it as second feedback index data; encoded first audio data and encoded first audio data; And the first index data corresponding to the first feedback index data is allocated at the timing when the first feedback index data is output from the distribution unit. Buffer means for outputting the encoded first audio data, feedback decoding means for decoding the first audio data output from the buffer means, and the decoded second audio data and the decoded second audio data. Mixing means for mixing the first audio data with the first audio data in accordance with a set ratio. 7. A plurality of voice communication terminal devices according to claim 6, which are connected to each other via a transmission line, wherein the first feedback index data is a decoding delay of the first voice data on the other side. Quantity and second
Is the first index data that is delayed by the total delay amount of the encoding delay amount of the audio data and fed back from the other party, and the second index data is the second audio data encoded by the other party. A voice communication system characterized by being assigned to data.