JP4117301B2 - Audio data interpolation apparatus and audio data interpolation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice data interpolator and interpolation method, in which the content of conversation can be transmitted accurately by concealing sound break due to delay, even at the occurrence of long delays which cannot be interpolated by an interpolation waveform during conversation. <P>SOLUTION: A voice packet or voice data during talk spurt period is stored temporarily, up to the moment when a delay occurs and is reproduced, immediately prior to the reproduction of the voice in a voice packet being received with a lag, after the delay has been eliminated. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to a voice data interpolating apparatus and a voice data interpolation method for concealing voice interruption due to interruption of continuous reception of voice packets, and more particularly, to a relatively long sound interruption caused by a spike delay of a network. The present invention relates to an audio data interpolating apparatus and an audio data interpolating method for continuing a conversation without recognizing sound interruption even if it occurs.

  In recent years, a VoIP (Voice over IP) in which a router transfers a voice packet to a destination according to an IP address via an IP network such as the Internet can form a speech path inexpensively without using an existing exchange. It is used for a telephone communication system such as an Internet telephone or a cellular phone which connects an IP network and a telephone exchange network with a gateway to form a communication path.

  Since this IP network is a best-effort type communication with no bandwidth guarantee, a voice packet delay or a part of voice packet loss occurs depending on the network line condition. This phenomenon of delay and packet loss is roughly divided into three types: base delay, fluctuation delay, and spike delay.

  The base delay is a steady delay caused by passing through the IP network line, and is a slight delay that can be ignored in a call. The fluctuation delay increases or decreases within a certain range of delay time due to network line traffic. A playout buffer is provided on the receiving device side to absorb the fluctuation delay, and continuous audio is reproduced from the audio packet.

  Although the spike delay is infrequent, reception of voice packets is interrupted without delay, and after 400 msec to 1 sec, voice packets that should have been received during this time are collectively received, and voice packets may be lost. Possible causes of this delay include regular maintenance of the router when the network line traffic is concentrated beyond the allowable value of the router, the router takes time to form a new route for some reason, and the like.

  If an attempt is made to absorb the delay time of the spike delay by the playout buffer, a delay that causes a hindrance to the conversation in bidirectional audio reproduction occurs, so there is a limit to the response by the playout buffer.

Therefore, conventionally, in order to prevent sound interruptions from becoming noticeable even if a delay occurs, previous and subsequent voice packets are added to the part where the voice packets are interrupted by the delay, or an interpolation waveform is generated based on the voice data of the previous and next voice packets. A method has been proposed.
(For example, refer to Patent Document 1).

  FIG. 13 is a block diagram showing a conventional audio data interpolating apparatus 100 described in Patent Document 1. An instantaneous interruption due to a delay of a digital audio signal input from the input means 101 is detected by the instantaneous interruption detection means 102. Detected. When the instantaneous interruption detecting unit 102 does not detect the instantaneous interruption, the digital audio signal input from the input unit 101 is output to the ring buffer unit 104 by the digital audio signal selecting unit 103 and stored. The digital audio signal stored in the ring buffer means 104 is encoded by the encoding means 105 and output via the output means 106.

  When the instantaneous interruption detecting unit 102 detects an instantaneous interruption, the attenuation increasing unit 107 reads the digital audio signal immediately before the instantaneous interruption stored in the ring buffer unit 104 and generates an interpolation signal. The attenuation increasing means 107 includes an attenuating means, an increasing means, and a silence signal generating means. The interpolation signal generated at this time is an attenuation signal that is gradually attenuated by multiplying the digital audio signal immediately before the momentary interruption by an attenuation coefficient. is there.

  The digital audio signal selection unit 103 stores the attenuation signal generated by the attenuation increase unit 107 in the ring buffer unit 104 in succession to the digital audio signal immediately before the momentary interruption, thereby causing the instantaneous interruption after the momentary interruption from the output unit 106. A digital audio signal attenuated by the post-attenuation coefficient is continuously output. When the instantaneous interruption detecting unit 102 detects the instantaneous interruption for a certain time or more, the attenuation increasing unit 107 generates a silence signal following the attenuation signal and outputs it to the ring buffer unit 104 in the same manner as the attenuation signal. Therefore, the output means 106 outputs a silence signal continuously to the digital audio signal that attenuates after a momentary interruption.

  When the instantaneous interruption detecting unit 102 detects a digital audio signal input after the instantaneous interruption, the attenuation increasing unit 107 initially attenuates the digital audio signal input after the instantaneous interruption. As the time elapses, the value is gradually increased by multiplying by a certain constant, and output to the ring buffer means 104. As a result, when the instantaneous interruption is recovered after the silence signal is output from the output means 106, a digital audio signal having a gradually increased signal level is output.

  As described above, according to the conventional audio data interpolating apparatus 100, even if the input of the digital audio signal is momentarily interrupted due to the delay of the network, the sound is smoothly changed from the sound state to the soundless state and from the soundless state to the sound state. Since the transition occurs, the call is interrupted due to a momentary interruption.

  Also, it determines whether the voice packet is a packet containing voiced voice or a packet without voice from the characteristics included in the received voice packet, and generates a pseudo voice signal from the voice packet containing voiced voice when a delay occurs An audio data interpolation device that reproduces a pseudo audio signal as an interpolated signal that is reproduced during a delay is disclosed in Patent Document 2 and Patent Document 3.

Japanese Patent Laid-Open No. 10-282959 (Items 0064 to 0068 of the specification, FIG. 5) Japanese Patent Application Laid-Open No. 6-202696 (Item 0008 of FIG. 1, FIG. 1) JP-A-8-123497 (Items 0011 to 0018 of the specification, FIG. 1)

  However, in each of the above-described conventional audio data interpolating devices, it is possible to conceal the interruption due to a short delay of up to about 20 msec so as not to make the sound out of the call inconspicuous, but it is longer than 400 msec as a spike delay. When a delay occurs during a call, at least one phoneme is included during this time, so it is difficult to interpolate waveforms from the previous and next voice packets, and the blockage due to the delay cannot be concealed. There was a problem that was not transmitted correctly.

  For example, in a conversation “I am tomorrow, I ’m in love”, “Juku” is transmitted by a spike delay between voice packets that convey “Shin” and “Juku” continuously. Assuming that the voice packet is delayed by 400 msec or more, if the voice packet received with a delay is discarded, “I am tomorrow, I'm stuck.” Is played back. Even if processing that does not destroy is performed, it is replayed as “I am tomorrow, Shin-Jukueki.” In both cases, not only is the blockage due to delay noticeable, but “Shinjuku-eki” ”Was broken, and the conversation was not accurately communicated.

  The present invention has been made in consideration of such conventional problems, and even if a spike delay of 400 msec or more occurs during a conversation, sound interruption due to the delay is concealed and the contents of the conversation can be accurately transmitted. An object of the present invention is to provide an audio data interpolation device and an audio data interpolation method.

In order to achieve the above-described object, the voice data interpolating apparatus according to claim 1 receives the voice packets, to which the packet sequence numbers are assigned in the order of transmission by the transmitting apparatus, received by the network connecting section and the network connecting section. A playout buffer that temporarily stores voice packets in the order of transmission from the packet sequence number, a voice decoding circuit that reads a voice packet stored at the head of the playout buffer and decodes it into voice data, and is output from the voice decoding circuit From the period in which voiced voice data continues, the voiced judgment unit that judges voiced voice data by comparing the feature value obtained by quantifying the feature when the voice data contains voice with a predetermined threshold A talk spurt detector for determining a talk spurt period to be estimated as a period, and a talk spurt A talk spurt holding buffer that continuously resets the voice data of the talk spurt period that is reset each time the start of the talk spurt period is detected at the outgoing part and is newly output from the voice decoding circuit, and the voice from the voice data. From the audio playback unit to be played back, the output of the audio decoding circuit, the output of the talk spurt holding buffer, the switching control unit that selectively switches to the input of the audio playback unit, and the storage state of the audio packet in the playout buffer, A delay detection unit that detects a spike delay of a voice packet due to a network failure,
When the delay detection unit detects a spike delay during the talk spurt period, before the voice packet received delayed by the spike delay is decoded into the audio data and output to the audio reproduction unit, the switching control unit While the audio data of the talk spurt period temporarily stored in the holding buffer is output, the input of the audio reproducing unit switched to the output of the audio decoding circuit is switched to the output of the talk spurt holding buffer, and the audio reproducing unit The voice is reproduced from the voice data of the voice packet received with a delay due to the spike delay in succession to the voice data of the talk spurt period temporarily stored in the talk spurt holding buffer.

  When the delay detector detects a spike delay during the talk spurt period, the talk spurt holding buffer stores the decoded audio data of the talk spurt period until the spike delay occurs. Before an audio packet received with a delay due to a spike delay is decoded into audio data and output to the audio playback unit, before being divided by the delay by switching the input of the audio playback unit to the output of the talk spurt holding buffer The voice data in the talk spurt period and the voice data in the talk spurt period after the division are reproduced as continuous voices in one talk spurt period.

  Since the talk spurt period is estimated as a unit of the conversation period divided by the silent part, the voice is continuously reproduced in a conversation period longer than at least the word unit.

  The audio data interpolating apparatus according to claim 2, wherein the switching control unit outputs the audio data of the talk spurt period temporarily stored in the talk spurt holding buffer when the expected delay time due to the spike delay elapses. The output of the decoding circuit and the output of the talk spurt holding buffer are switched to the input of the audio reproduction unit.

  The delay time due to the spike delay is a fixed time that can be predicted to some extent depending on the network environment. Therefore, if all the voice data of the talk spurt period temporarily stored in the talk spurt holding buffer is output when the expected delay time elapses, then the voice data of the voice packet received delayed by the spike delay is output, Played as continuous audio.

  In the speech data interpolating apparatus according to claim 3, the switching control unit is switched to the output of the speech decoding circuit on condition that the speech packet received delayed by the spike delay is temporarily stored in the playout buffer. The input of the audio playback unit is switched to the output of the talk spurt retention buffer, and all the audio data in the talk spurt period temporarily stored in the talk spurt retention buffer is output to the audio playback unit, and then output to the talk spurt retention buffer. The input of the switched audio reproduction unit is switched to the output of the audio decoding circuit, and audio data decoded from the audio packet temporarily stored in the playout buffer is output to the audio reproduction unit. To do.

  Even if the delay time due to the spike delay cannot be predicted, the audio data of the talk spurt period temporarily stored in the talk spurt holding buffer is provided on the condition that the audio packet received late is temporarily stored in the playout buffer. Therefore, it is possible to output the voice data of the voice packet that is received later with a delay due to the spike delay, and to reproduce it as a continuous voice.

According to another aspect of the present invention, there is provided a voice data interpolating device that receives a voice packet having a packet sequence number assigned in the order of transmission by a transmitting device via a network, and transmits a voice packet received by the network connecting portion from the packet sequence number. A playout buffer that temporarily stores data in sequence, a voice decoding circuit that decodes voice packets into voice data, and a feature value obtained by quantifying features when voice data output from the voice decoding circuit includes voice, A voice determination unit that determines voice data of voice compared with a threshold; a talk spurt detection unit that determines a talk spurt period to be estimated as a conversation period from a period in which the voice data of voice is continuous; and voice decoding An audio playback unit that plays back audio from the audio data output from the circuit, and a talk spurt detection unit A talk spurt holding buffer that continuously and temporarily stores audio packets in the talk spurt period that is reset each time the start of the talk spurt period is detected and is newly read from the head of the play out buffer, the output of the play out buffer, A switching control unit that selectively switches the output of the part holding buffer to the input of the audio decoding circuit; and a delay detection unit that detects a spike delay of the audio packet due to a network failure from the storage state of the audio packet in the playout buffer; With
When the delay detection unit detects a spike delay during the talk spurt period, the switching control unit temporarily stores in the talk spurt holding buffer before the voice packet received delayed by the spike delay is output to the voice decoding circuit. While the stored voice packet of the talk spurt period is output, the input of the voice decoding circuit switched to the output of the playout buffer is switched to the output of the talk spurt holding buffer. A voice packet received after a delay due to a spike delay is decoded into voice data in succession to the voice packet in the talk spurt period output from the holding buffer, and voice is reproduced from the voice data by the voice reproduction unit. .

  When the delay detector detects a spike delay during the talk spurt period, the talk spurt holding buffer stores voice packets during the talk spurt period until the spike delay occurs. Before the voice packet received delayed by the spike delay is output to the voice decoding circuit, the talk spurt before being divided by the delay is switched by switching the input of the voice decoding circuit to the output of the talk spurt holding buffer. The audio data of the period and the audio data of the talk spurt period after being divided are reproduced as continuous audio in one talk spurt period.

  Since the talk spurt period is estimated as a unit of the conversation period divided by the silent part, the voice is continuously reproduced in a conversation period longer than at least the word unit.

  The audio data interpolating apparatus according to claim 5, wherein the switching control unit outputs all the audio packets in the talk spurt period temporarily stored in the talk spurt holding buffer when the expected delay time due to the spike delay elapses. The output of the out buffer and the output of the talk spurt holding buffer are switched to the input of the audio decoding circuit.

  The delay time due to the spike delay is a fixed time that can be predicted to some extent depending on the network environment. Therefore, if all audio packets in the talk spurt period temporarily stored in the talk spurt holding buffer are output when the expected delay time elapses, then the audio packets received delayed by the spike delay are output, and continuous audio is output. As played.

  According to another aspect of the audio data interpolating apparatus of the present invention, the switching control unit is switched to the output of the playout buffer on condition that an audio packet received with a delay due to a spike delay is temporarily stored in the playout buffer. The input of the voice decoding circuit is switched to the output of the talk spurt holding buffer, and after all the voice buckets in the talk spurt period temporarily stored in the talk spurt holding buffer are decoded by the voice decoding circuit, the output is output to the talk spurt holding buffer. The input of the switched voice decoding circuit is switched to the output of the playout buffer, and the voice packet temporarily stored in the playout buffer is decoded by the voice decoding circuit.

  Even if the delay time due to the spike delay cannot be predicted, the voice packet in the talk spurt period temporarily stored in the talk spurt holding buffer is provided on the condition that the voice packet received late is temporarily stored in the playout buffer. Therefore, it is possible to output a voice packet that is received later with a delay due to a spike delay, and to reproduce it as a continuous voice.

According to a seventh aspect of the present invention, the voice data interpolation method decodes voice packets received from the network into voice data in the transmission order of the transmission device, reproduces voice from the decoded voice data, and starts from a period in which voice data including voice is continuous. The talk spurt period to be estimated as the conversation period is determined, and from the start of the talk spurt period, voice packets newly received during the talk spurt period or voice data decoded from the voice packets are continuously stored in the talk spurt holding buffer . Temporarily memorize, reset every time the talk spurt period starts and repeat the temporary memorization, and when a voice packet spike delay occurs during the talk spurt period, the voice packet of the talk spurt period temporarily stored in the talk spurt holding buffer or After playing audio from audio data Characterized by continuously reproducing audio from audio data of the voice packets received with a delay by the spike delay.

  When a voice packet spike delay occurs during the talk spurt period, the talk spurt holding buffer stores the voice packet in the talk spurt period until the spike delay occurs or voice data decoded from the voice packet. . If voice is played back from the voice packet or voice data temporarily stored in the talk spurt holding buffer immediately before playing back the voice data from the voice packet received delayed by the spike delay, both are played back continuously. The

  Since the talk spurt period is estimated to be a unit of a conversation period in which voiced speech continues, even if the reproduction of the talk spurt period is divided due to the spike delay, the voices of the talk spurt period before and after the division are continuous, at least Voice is played continuously in a conversation period longer than the word unit.

  The voice data interpolation method according to an eighth aspect is characterized in that the voice packet received after the spike delay is reproduced by performing the speech speed conversion in the transmission order of the transmission device, thereby eliminating the spike delay.

  Speaking speed conversion shortens the silence period during a call or absorbs delay by increasing the audio playback speed to an inconspicuous level, so the call continues without the caller being aware and eliminates the delay caused by the spike delay Is done.

  The voice data interpolation method according to claim 9 temporarily stores voice data decoded from a voice period or a voice packet outside a talk spurt period, detects a spike delay, and then temporarily stores the voice data in a talk spurt period. Before reproducing the voice from the packet or the voice data, the ambient noise reproduced from the voice packet or the voice data in the silent period is reproduced.

  When determining the talk spurt period, it is possible to temporarily store a voice packet in a voiceless period or voice data decoded from a voice packet from the voiceless period in between. Voice data decoded from a voice packet or a voice packet during a non-voice period is generated from a breathing during a call or between conversations, and thus includes noise around the caller. Therefore, the ambient noise of the caller is reproduced from these voice packets or voice data.

  According to the first, fourth, and seventh aspects of the present invention, even if the voice in the talk spurt period is divided by the spike delay, the voice in the talk spurt period until the occurrence of the delay and the voice packet received with a delay are obtained. The voice of the talk spurt period to be played back is played back continuously, and played back in units of conversation periods separated by silence. Therefore, even if a spike delay occurs, it is played back in units of a conversation and is not divided at least during the playback of a word, so that the content of the conversation is accurately transmitted.

  The voice in the talk spurt period until the spike delay occurs is interrupted by the spike delay after being played once, and is output from the talk spurt holding buffer before playing back the voice packet received late. The sound is reproduced again from the beginning of the talk spurt period, but the sound that is repeatedly reproduced within the spike delay time of 400 msec to 1 sec seems to be rephrased during the conversation. Rephrasing is a frequent occurrence in everyday conversations and is familiar to human ears, so it is recognized as a natural conversation and conceals the occurrence of sound interruption due to spike delay.

  According to the second and fifth aspects of the present invention, since the voice packet or voice data during the talk spurt period temporarily stored in the talk spurt holding buffer is reproduced as voice during the delay time due to the spike delay, Therefore, there is no delay in the reproduction time, and the time during which the sound is interrupted during the delay time is reduced.

  According to the third and sixth aspects of the present invention, even when a voice packet delayed by a spike delay is not received, only the voice packet or voice data in the talk spurt period temporarily stored in the talk spurt holding buffer is stored. Without being repeatedly played, continuous sound can be played reliably in the talk spurt period.

  According to the invention of claim 8, when a spike delay occurs, the caller can simply hear that the conversation is restated, and thereafter, the delay is resolved by the conversion of the speech speed so that the delay is not noticed by the caller. The failure can be resolved during natural conversation without making the caller aware of the failure caused by the spike delay.

  According to the ninth aspect of the invention, the interpolated signal generated from the characteristics of the voice packet including voiced sound and the reproduced sound reproduced from the silent signal are artificial sounds, whereas the voice packet or voice packet in the silent period is used. The ambient noise that is reproduced from the audio data decoded from the sound can be heard as if the calling party is breathing, or speaking between the conversations. Therefore, by playing ambient noise from a voice packet or voice data during a non-voice period during a sound break period caused by a spike delay before playing the voice from the talk spurt holding buffer, the voice breaks without the caller being aware. Can be eliminated.

  Hereinafter, an audio data interpolation apparatus 1 and an audio data interpolation method according to the present invention will be described with reference to FIGS. FIG. 1 is a block diagram of the voice data interpolating apparatus 1, and among the functions for making a two-way call via an IP network (communication network for communication using the Internet protocol), a receiving function for mainly reproducing the other party's voice. The structure regarding is shown.

  The voice data interpolating apparatus 1 is connected to the IP network 3 which is the Internet here in the network connecting part 2 having the LAN interface part, and the voice is compressed by the voice codec encoder on the transmitting apparatus side (not shown) for a certain communication time. A voice packet in which voice data (for example, 10 msec to 80 msec) is collected as one lump can be received via the Internet 3.

  The transmission device attaches an IP header including the IP address of the transmission device that is the transmission source of the voice packet, the IP address of the voice data interpolation device 1 that is the transmission destination, and a packet sequence number that indicates the transmission sequence of the voice packet to the voice packet. Then, each router of the Internet 3 forwards the voice packet to the network connection unit 2 based on the destination IP address.

  The voice packet received by the network connection unit 2 is input to the playout buffer 4 connected to the output side after the IP address is separated. The playout buffer 4 basically has a FIFO structure, and a voice packet input first from the network connection unit 2 is output first. Accordingly, the packets are normally output in the order of the packet sequence numbers. However, if they are input without being received in the order of the sequence numbers due to the network conditions or the like, they are sorted in the order of the sequence numbers, that is, in the order of transmission from the transmission device. It also has a function.

  Further, the storage capacity of the playout buffer 4, that is, the number of temporarily stored voice packets is preferably an amount that absorbs the fluctuation delay if the fluctuation delay of the network is within a short time. If you try to temporarily store all the voice packets until the time, the number of voice packets to be temporarily stored will increase, and the delay of the entire playback will increase, so the number of voice packets to be temporarily stored will follow the change in the delay amount of fluctuation delay. Alternatively, a playout buffer that discards voice packets received with a delay at a constant rate may be used. Such a playout buffer is published in “Low Delay Real-Time Voice Communication System Using Load Adaptive Control in LAN Environment” published by the Information Processing Society of Japan published in July 1999.

  An audio codec decoder 5 is connected to the output side of the playout buffer 4. The audio codec decoder 5 reads and decodes the audio packet stored at the head of the playout buffer 4 when the audio data stored in the D / A output buffer 7 to be described later is almost exhausted. The audio data for the actual audio time compressed into A voice packet is read from the playout buffer 4 in units of one voice packet every time the voice data stored in the D / A output buffer 7 is almost exhausted unless the voice packets stored in the playout buffer 4 are exhausted. Is output from the audio codec decoder 5 as audio data continuous with the audio data decoded from the audio packet read immediately before.

  The playout buffer 4 is connected to a packet loss detection unit 6 that monitors the packet sequence number of the voice packet stored at the head of the playout buffer 4. The packet loss detection unit 6 detects the packet sequence that is lost when the packet sequence number of the immediately preceding voice packet read by the voice codec decoder 5 and the packet sequence number newly stored at the head are not consecutive in the transmission order. The voice packet lost from the number is notified to the interpolation control unit 8.

  The output of the audio codec decoder 5 is connected to the interpolation waveform generator 9 and the first switching terminal 10 </ b> A of the first changeover switch 10. The interpolated waveform generation unit 9 generates an interpolated waveform that is similar to the audio waveform from the characteristics of the audio data output from the audio codec decoder 5 and is similar to the audio waveform reproduced from the audio data. This compensates for the instantaneous interruption of the reproduced sound due to a short delay of about 40 msec. For example, the pitch of the speech waveform immediately before the interpolation is detected, and an interpolation waveform having the same pitch as that of the speech waveform and having the phase matched so as to be smoothly continuous with the speech waveform immediately before the interpolation is generated. As will be described later, when the waveform after the output of the interpolation waveform is known, an interpolation waveform that is smoothly continuous with the waveform immediately after the interpolation is generated in accordance with the pitch and phase of the subsequent waveform. The output of the interpolated waveform generation unit 9 is connected to the second switch terminal 10B of the first switch 10 that is switch-controlled by the interpolation control unit 8, and when the voice data of the interpolated waveform is output, the first switch 10 The common output terminal 10C is switched to the second switching terminal 10B side for output.

  The common output terminal 10 </ b> C of the first changeover switch 10 from which the audio data is output is ambient noise via the first changeover terminal 11 </ b> A of the second changeover switch 11, the sound determination unit 12, the talk spurt holding buffer 13, and the input switch 14. Each holding buffer 15 is connected.

  The voice determination unit 12 represents the feature when the voice data includes voice as a numerical feature, and compares the voice data output from the voice codec decoder 5 with a predetermined threshold value to determine whether voice data includes voiced voice. Determine if. Here, it is assumed that the signal level rises when speech is included compared to no speech, and as shown in FIG. 2, the signal level of the speech data is obtained from the mean square value between them as signal power every 5 msec. Compared to the threshold level set to a predetermined value, when the threshold level is equal to or higher than the threshold level, voiced voice data is obtained, and when the threshold level is lower than the threshold, voiceless voice data including only ambient noise is used. .

  Since the signal power changes as the ambient noise level increases or decreases, the threshold level is also made to follow the change in signal power to be compared. Here, as shown in FIG. 3, the threshold level is set to a value obtained by adding a constant value α to the minimum value of the signal power obtained every 5 msec, and the minimum value of the signal power is not updated, for example, 30 msec (FIG. 3). T2 after 30 msec from middle t1) The minimum value is raised and the threshold level is made to follow the signal power so that voice can be determined even if the ambient noise level increases.

The determination result of the sound determination unit 12 is output to the talk spurt detection unit 20. As shown in FIG. 4, the talk spurt detection unit 20 determines a period in which voiced voice data determined by the voice determination unit 12 is continuous as a talk spurt period. However, even during voiced conversations, instantaneous silence time (T _S ) occurs between phonemes, so even during periods when voiced voice data is continuous, up to the maximum silence period (MaxT _S ). The silent period is ignored and the talk spurt period. The maximum silence period (MaxT _S ) is set to 100 msec as a standard, but this also varies depending on the conversation characteristics of the caller. Therefore, the operation of the talk spurt detection unit 20 is monitored for a certain period of time. If the start of is not detected, the maximum silence period (MaxT _S ) is shortened.

Therefore, even if silent audio data continues during the talk spurt period, it is not immediately determined that the talk spurt period ends until the maximum silence period (MaxT _S ) elapses, and the maximum silence period (MaxT _S ). It is determined that the talk spurt period has ended when no voice data continues beyond

On the other hand, a period other than the talk spurt period, that is, a period in which silent voice data continues is set as a silent period. It is determined that the period starts. This is because there is no problem because the noise voice data stored at the start of the talk spurt period by the actual conversation is reset even if the voice data of the repeated talk spurt described later is started due to noise. In addition, the determination of the start of the talk spurt period by actual conversation is made promptly without determining whether it is noise or not.

  The talk spurt detection unit 20 outputs the talk spurt period signal to the interpolation control unit 8 and the input switch 14 during the duration of the talk spurt period, and also determines the talk spurt start signal when determining the talk spurt period from the silent period. Output to the talk spurt holding buffer 13.

  Since the talk spurt holding buffer 13 is connected to the common output terminal 10C of the first changeover switch 10 from which audio data is output, the talk spurt holding buffer 13 always stores audio data to be reproduced as audio. All the audio data stored in the talk spurt holding buffer 13 is erased every time the talk spurt period starts using the talk spurt start signal input from the talk spurt detection unit 20 as a reset signal. Therefore, in the talk spurt holding buffer 13, audio data corresponding to the sound from the beginning of the talk spurt period being reproduced (hereinafter referred to as repetitive talk spurt audio data) is stored in order from the start address. Will be. The output of the talk spurt holding buffer 13 is connected to the second switching terminal 11B of the second switching switch 11 that is switched to and from the first switching terminal 11A under the control of the interpolation control unit 8, whereby the control of the interpolation control unit 8 is performed. The audio data of the repeated talk spurt stored in the talk spurt holding buffer 13 is output to the common output terminal 11C at the timing described later.

  The input switch 14 connects the common output terminal 10 </ b> C of the first changeover switch 10 to which the audio data is output to the ambient noise holding buffer 15 while the talk spurt period signal is not input from the talk spurt detection unit 20, that is, during the silent period. . Therefore, the ambient noise holding buffer 15 is inputted with silent voice data during the interruption of the voice among voice data reproduced as voice. The ambient noise holding buffer 15 has a ring buffer structure, and rewrites the oldest stored voice data into silent voice data to be newly input. Ambient noise that can be heard when a conversation is interrupted does not have a function to transmit a specific language, and is a complex waveform. Therefore, it is relatively difficult to generate a waveform that resembles natural noise, and for a long time. It is relatively difficult to generate an interpolated waveform that can be played back. If an interpolated waveform that is generated when there is a sound break of 1 second or more that occurs due to a spike delay or the like is used, an interpolated waveform with a short period is reproduced repeatedly. Plays as annoying noise like a buzzer.

  In the present embodiment, in the process of detecting the talk spurt period, it is possible to easily detect a voiceless period including only ambient noise other than the talk spurt period, and the voice data of the voiceless period is sent to the ambient noise holding buffer 15. By temporarily storing, without generating a pseudo waveform, the ambient noise is reproduced and interpolated during a long sound interruption period, and the sound interruption is concealed as if the other party's conversation was interrupted. Is.

  The voice data of the non-voice period stored in the ambient noise holding buffer 15 is output at a timing described later under the control of the interpolation control unit 8, and the output voice data is output from the interpolation control unit 8 with a predetermined increase rate or attenuation rate. Is added to the audio data output from the common output terminal 11C of the second change-over switch 11.

  The adder 16 that adds the outputs of the common output terminal 11C and the ambient noise holding buffer 15 and the D / A output buffer 7 on the output side thereof are either directly connected or the speech speed acceleration unit 17 is connected. A speech speed conversion switch 18 for selecting whether to connect via is arranged. The switching of the speech speed conversion switch 18 is also controlled by the interpolation control unit 8 and is normally switched to the direct connection via the speech speed acceleration unit 17 during a period to be described later when the delay needs to be eliminated.

  The D / A output buffer 7 has a FIFO structure, and the D / A converter 19 connected to the output side continuously reads out audio data temporarily stored in the D / A output buffer 7 in the order of input. The audio data is D / A converted at a constant sampling rate in the D / A converter 19 and reproduced as audio from a speaker (not shown).

  The interpolation control unit 8 controls the operation of each circuit unit described above while monitoring the storage state of the voice packet stored in the playout buffer 4. Hereinafter, the interpolation control unit 8 responds to the delay state occurring in the Internet 3. The operation of the different audio data interpolating apparatus 1 will be described focusing on the operation of the interpolation control unit 8. In each of the examples described below, there is a conversation of the other party called “Shinjuku de te teki”, and this conversation is transmitted as voice packets from packet 0 to packet 10 from the other party's transmission device via the Internet 3. Assume that the data is transmitted to the audio data interpolating apparatus 1. That is, packet 0 is voiceless voice data, packets 1 to 3 are voice data including voiced “shin”, and packets 4 to 6 are voice data including voiced voice “juku-eki”. Packets 7 and 8 are voice-free voice data that is generated from a conversation that separates “Shinjuku-eki” and “Mate”. Packets 9 and 10 are voices that include “Mate” voiced voice. Assume that each data is encoded.

(1) First example (example in which no delay occurs, see FIG. 5)
First, assuming that a voice packet transmitted from the transmission device is normally received without delay on the Internet 3, the voice packet received via the network connection unit 2 is transmitted in the order of transmission from the transmission device, that is, packet 0. Are stored in the playout buffer 4 and sequentially decoded by the audio codec decoder 5 into audio data.

  Normally, while the voice packet is received without delay, the common output terminal 10C of the first changeover switch 10 and the common terminal 11C of the second changeover switch 11 are both switched to the first changeover terminals 10A and 11A side. Since the speed conversion switch 18 is switched to the side where the adder 16 and the D / A output buffer 7 are directly connected, the decoded audio data is temporarily stored in the D / A output buffer 7 as it is, and the D / A After being D / A converted by the A converter 19, it is reproduced with the voice waveform shown in FIG. Hereinafter, the output path through which the audio packet stored in the playout buffer 4 is reproduced as normal audio is the same as in the first example. Therefore, unless otherwise specified, the first changeover switch 10, the second changeover switch 11, and the talk Since the switching position of the speed conversion switch 18 is the same as that in the first example, the description thereof is omitted.

  At this time, the audio data decoded by the audio codec decoder 5 output from the common output terminal 10C of the first changeover switch 10 is also input to the audio determination unit 12, and it is determined whether or not it is voiced audio data. The result is output to the talk spurt detection unit 20. Since the audio data decoded from the packet 0 and the packet 7 is non-audio data, it is a non-audio period while these audio data are output to the D / A output buffer 7 and reproduced, and the talk spurt detection unit 20 does not output a talk spurt period signal. Accordingly, the input switch 14 is closed, and the audio data decoded from the packet 0 and the packet 7 output from the common output terminal 10C is stored in the ambient noise holding buffer 15. However, in this example, since a relatively long delay does not occur, the audio data stored in the ambient noise holding buffer 15 is not output.

On the other hand, since the audio data decoded from the packets 1 to 6 is voiced audio data, while these audio data are output to the D / A output buffer 7 and played back, the talk spurt detection unit 20 outputs the audio data. A period signal is output to the interpolation control unit 8. Further, when it is determined from the no-voice period to the talk spurt period, that is, when audio data decoded from the packet 1 is output, a talk spurt start signal is output to the talk spurt holding buffer 13. The talk spurt holding buffer 13 is reset when a talk spurt start signal is input and starts storing new audio data output from the common output terminal 10C. Is stored. However, since the voice data stored in the talk spurt holding buffer 13 does not generate a relatively long spike delay in this example, the voice data of the packet 9 in the next talk spurt period is output without being output. Reset to.

(2) Second example (example in which a delay occurs during a non-voice period, see FIG. 6)
A description will be given assuming that a voice packet equal to or less than the packet 8 is delayed and a delay occurs between the packet 7 and the packet 8 which are silent periods. The delay of the voice packet is detected by the interpolation control unit 8 by monitoring the storage state of the voice packet stored in the playout buffer 4, and the voice data stored in the D / A output buffer 7 is likely to be exhausted. When the voice packet stored at the head of the playout buffer 4 is read out, the voice packet to be read out is not stored, so that the occurrence of delay is detected.

  From packet 0 to packet 7, it is assumed that audio is reproduced without delay as in the first example. When audio data obtained by decoding packet 7 is output from D / A output buffer 7, reception of packet 8 is delayed. Therefore, it is not stored in the playout buffer 4 yet. When detecting the delay, the interpolation control unit 8 determines whether the delay occurs during the talk spurt period or the silent period depending on whether a talk spurt period signal is input from the talk spurt detection unit 20 at the time of detection. To do. In this example, since the audio data output from the common output terminal 10C immediately before the occurrence of the delay is silent audio data obtained by decoding the packet 7, the talk spurt period signal is not output and it is determined that the audio is not in use. Is done.

  If the interpolation control unit 8 determines that the delay has occurred during the silent period, the silent voice data stored in the ambient noise holding buffer 15 from the previous silent period (for example, packet 0 in the first example) The audio data decoded from the packet 7) is output to the adder 16, and the audio data decoded from the packet 7 is output to the D / A output buffer 7 continuously. At this time, the speech speed conversion switch 18 is switched to the side where the adder 16 and the D / A output buffer 7 are directly connected. Since the ambient noise holding buffer 15 has a ring buffer structure, this output is repeated until the packet 8 received late in the playout buffer 4 is stored. Therefore, the ambient noise is repeatedly reproduced while the sound breaks due to the delay, and the ambient noise is interpolated during the silent period of the conversation “Shinjukueki” and the conversation “Mate”. So, the caller can hear it during the conversation and not even notice a relatively long delay.

  Audio packets of packet 8 or less received after delay and stored in the playout buffer 4 are immediately decoded into audio data by the audio codec decoder 5 in the order of packet sequence numbers, and output to the D / A output buffer 7. At this time, since the playback time is delayed for the time when the voice data of the ambient noise is played back, the speech speed conversion switch 18 is switched to the side via the speech speed acceleration section 17 and the packet 8 or less delayed. The audio data is output to the D / A output buffer 7 through the speech speed acceleration unit 17. The speech speed accelerating unit 17 converts the input voice data so that the input voice data is reproduced at an unnoticeable speed. For example, the voice data obtained by decoding the packet 8 which is silent voice data is deleted. The voiced audio data decoded from the packet 9 and below is converted into audio data with a compressed reproduction time. The speech speed conversion switch 18 is switched to the side through the speech speed acceleration section 17 until it is determined from the packet sequence number of the voice packet newly stored in the playout buffer 4 that the reproduction delay has been eliminated.

(3) Third example (example in which a short delay occurs in the talk spurt period, see FIG. 7)
A description will be given assuming that a voice packet equal to or less than packet 4 is delayed and a short delay of up to 40 msec occurs between packet 3 and packet 4, which is the talk spurt period. From packet 0 to packet 3, audio is reproduced without delay as in the first example. However, when audio data obtained by decoding packet 3 is output from D / A output buffer 7, reception of packet 4 is delayed. Therefore, it is not stored in the playout buffer 4. Accordingly, since the audio data output from the common output terminal 10C immediately before the occurrence of the delay is voiced audio data obtained by decoding the packet 3, a talk spurt period signal is output, and the interpolation control unit 8 performs the detection of the delay. Since a talk spurt period signal is input, it is determined that a delay has occurred during the talk spurt period.

  When the interpolation control unit 8 determines that the delay has occurred during the talk spurt period, the interpolation control unit 8 first switches the common terminal 10C of the first changeover switch 10 from the first changeover terminal 10A to the second changeover terminal 10B, and the interpolation waveform generation unit. The audio data of the interpolated waveform generated in 9 is output. Therefore, the interpolated waveform audio data output to the common terminal 10C is directly output to the D / A output buffer 7 and reproduced with the interpolated waveform audio.

  As described above, the interpolation waveform is continuous with the pitch and phase of the waveform of the audio data output from the audio codec decoder 5 just before, that is, the audio data obtained by decoding the packet 3, and is similar to the waveform of the audio data. Is generated as follows. Since there is a limit to the time to interpolate with the interpolation waveform to the extent that the caller does not notice, the maximum waveform interpolation possible time (WCT) is set to 40 msec, and the time for reproducing the interpolated waveform audio data is the maximum waveform interpolation. Within possible time (WCT).

  While reproducing the voice data of the interpolated waveform, that is, within the maximum waveform interpolable time (WCT) after the interpolation control unit 8 detects the delay, the delayed packet 4 is received in the playout buffer 4. When a predetermined number of voice packets are stored in the order of bucket sequence numbers starting with packet 4, interpolation control unit 8 determines that the short delay has been eliminated. If it is determined that the short delay is eliminated, the audio data of the interpolated waveform should be reproduced at the time when the interpolated waveform audio data is reproduced without being read out and output from the audio packet 4 received with a delay immediately. The voice packet (packet 4 in this example) was deleted from the playout buffer 4 and read from the subsequent voice packet 5. This is because if the voice is played back as it is from the delayed voice packet, the same voice as that of the interpolated waveform will be played back repeatedly, and the playback voice will be delayed, and there will be a delay in the playback of the interpolated waveform playback time. It is because it is doing.

  Therefore, the packet 5 after being deleted in the order of the bucket sequence number is read by the audio codec decoder 5 and decoded into audio data. Since the audio data decoded from the packet 5 is continuous with the audio data of the interpolated waveform that is generated by the interpolated waveform generation unit 9 and is being reproduced, the back of the interpolated waveform approximates the audio waveform of the packet 5, and the pitch and A waveform with a continuous phase. As a result, a phoneme that approximates “ju” is reproduced from the waveforms of the preceding and following packets 3 and 5 from the speech data of the interpolated waveform.

The decoded audio data from the packet 5 is temporarily stored in the D / A output buffer 7 in the order of the packet sequence number by switching the common output terminal 10C of the first changeover switch 10 to the first changeover terminal 10A and reproduced as audio. Is done. At this time, since the bucket 4 is deleted, there is no reproduction delay, and therefore the speech speed increasing unit 17 is not routed. In this way, even if a delay occurs during the talk spurt period , if the delay is short, as shown in FIG. Played.

(4) Fourth example (example in which a long delay occurs in the talk spurt period and a voice packet delayed within the expected delay time (SPT) due to spike delay is received, see FIG. 8)
A description will be given assuming that a voice packet of packet 4 or less is delayed and a long delay of 40 msec or more, which is the maximum waveform interpolation possible time (WCT), occurs between packet 3 and packet 4 which are the talk spurt period. The audio data of the interpolated waveform generated by the interpolated waveform generating unit 9 is output and reproduced after detecting the delay until the maximum waveform interpolable time (WCT) is the same as in the third example.

If a delayed voice packet is not received even after the maximum waveform interpolation possible time (WCT) is exceeded, the following delay processing sequence is executed. As shown in FIG. 8, the time required for the delay processing sequence is such that all delay processing sequences including the maximum possible waveform interpolation time (WCT) are completed within the expected delay time (SPT) due to the spike delay. Although it is set, if the repeated talk spurt reproduction time is longer than expected, the expected delay time (SPT) may be slightly exceeded. The expected delay time (SPT) due to the spike delay may cause a packet loss due to the spike delay, so a retransmission request for the lost voice packet is made and all the delayed voice packets are stored in the playout buffer 4 in the order of the packet sequence numbers. It is assumed that the time is until 1 second, and here is 1 second. The repetitive talk spurt reproduction time included in the delay processing sequence is a fixed time obtained from the amount of audio data stored in the talk spurt holding buffer 13 at least during execution of the delay processing sequence. The time required for the delay processing sequence is adjusted by the interpolation time.

  After the delay is detected, when the maximum waveform interpolation possible time (WCT) elapses, the interpolated waveform audio data output from the interpolated waveform generation unit 9 is replaced with new interpolated waveform audio data so that the interpolated waveform gradually attenuates. Is generated by performing a predetermined addition process so that the ambient noise waveform gradually increases from 0, and outputs both of them as an adder. When the maximum waveform interpolation possible time (WCT) elapses as shown in FIG. 8, the output of the adder 16 that is only the interpolated waveform audio data is added to the output of the adder 16 as shown in FIG. Thus, only voiceless voice data is output.

  In order to compensate for the interpolated waveform attenuated in this way and the attenuation of the ambient noise component included in the interpolated waveform, the sound by the ambient noise waveform is gradually increased from 0 to increase the abrupt gain. Avoid fluctuations and reduce unnatural feeling when switching from interpolated waveform to ambient noise waveform.

  After the feedout time has elapsed, as described above, the remaining time obtained by subtracting the maximum waveform interpolable time (WCT), feedout time, and repeated talk spurt playback time from the expected delay time (SPT) of the spike delay The ambient noise audio data stored in the buffer structure ambient noise holding buffer 15 is repeatedly output to reproduce the ambient noise.

  Subsequently, the common output terminal 11C of the second changeover switch 11 is switched from the first changeover terminal 11A to the second changeover terminal 11B under the control of the interpolation control unit 8, and the repetitive talkspart stored in the talkspart holding buffer 13 is switched. Audio data is output. The audio data of the repeated talk spurt output at this time is audio data decoded from the packets 1 to 3, and is audio data for reproducing the phoneme “shin”.

  In this fourth example, it is assumed that a voice packet delayed within the expected delay time (SPT) is received, so that a voice packet received with a delay is played in the playback time of ambient noise or the subsequent repeated talk spurt playback time. Stored in the out buffer 4. When the packet loss detection unit 6 detects the loss of the voice packet due to the spike delay from the packet sequence number of the voice packet stored at the head of the playout buffer 4 with a delay, the interpolation control unit 8 detects that the network connection unit It is assumed that the spike delay has been eliminated when a packet requesting retransmission of the lost packet is sent from 2 to the transmission device on the transmission side, and the voice packet requested for retransmission is received and stored in the playout buffer 4. In this example, as shown in FIG. 11, after the spike delay occurs, the packet sequence number of the voice packet first stored at the head of the playout buffer 4 is the packet 7, so that the packets 4, 5, and 6 These packets are assumed to have been lost, and a retransmission request is made. However, since the audio packet requested to be retransmitted is usually received around 100 msec after the retransmission request, all the delayed audio is received within the expected spike delay time (SPT). A packet is obtained.

  Therefore, immediately after the estimated spike delay time (SPT) elapses, that is, as shown in FIG. The audio data stored and sequentially decoded by the audio codec decoder 5 is output and reproduced as audio. That is, in this example, after playing “shin”, the interpolated waveform and ambient noise are played during the spike delay, and then “shinjuku” is played. If you hear a long delay while playing the word “Shinjuku”, the meaning of the word is accurately communicated, and the caller has a long delay. I do not notice that occurred.

  While the spike delay is occurring, the delay processing sequence is executed and the playback is delayed. Therefore, the voice data obtained by decoding the delayed voice packet (packet 4 or less) is converted to the speech speed. The switch 18 is switched to the side via the speech speed acceleration unit 17 and the speech speed conversion is performed using the speech speed acceleration unit 17 as shown in FIG. 11 until it is determined that the reproduction delay has been eliminated. Reproduce.

(5) Fifth example (example in which a long delay occurs in the talk spurt period and a voice packet delayed after the expected delay time (SPT) due to spike delay is received, see FIG. 9)
In the fourth example, all voice packets delayed within the expected delay time (SPT) due to the spike delay are received, but in the fifth example, the voice packets delayed within the expected delay time (SPT) are not received. is there.

  Therefore, a series of delay processing sequences is executed until the repeated talk spurt reproduction time elapses, and therefore the description is omitted because it is the same as the fourth example. When the expected delay time (SPT) elapses, the interpolated waveform generation unit 9 generates interpolated waveform audio data in accordance with the pitch and phase of the last audio of the repeated talk spurt audio data so that the interpolated waveform gradually attenuates. New interpolated waveform voice data is generated, and a predetermined addition process is performed on the voiceless voice data stored in the ambient noise holding buffer 15 from the voiceless period so that the ambient noise waveform gradually increases from zero. 9 and adding them by the adder 16, the output of the adder 16 which is only the audio data of the interpolated waveform when the expected delay time (SPT) elapses as shown in FIG. When the out time elapses, only voice data without sound is output. This process is the same as the process described in the fourth example for transferring waveform interpolation voice data to ambient noise, and naturally transfers the voice data of the interpolation waveform to ambient noise. After the feed-out time has elapsed, only the ambient noise audio data is repeatedly output in the same manner as in the fourth example to reproduce the ambient noise.

  In the fifth example, the delayed talk packet is reproduced on condition that the delayed voice packet is received so that the delayed voice packet is not received and only the voice of the repeated talk part is not reproduced again. Accordingly, the ambient noise interpolation time for reproducing the ambient noise shown in FIG. 9 is undecided, and when reception of a voice packet delayed during this time is detected, that is, when the voice packet is stored in the playout buffer 4, The common output terminal 11C of the second changeover switch 11 is switched from the first changeover terminal 11A to the second changeover terminal 11B, and the audio data of repeated talk spurts stored in the talk spurt holding buffer 13 is output.

  At this time, the voice packet stored at the head of the playout buffer 4 does not necessarily have a state where all the voice packets after the delay are received when there is a packet loss, but one of the voice packets is received after the delay. Therefore, it can be estimated that the spike delay has been eliminated. Therefore, by performing a lost packet retransmission request in the same manner as in the fourth example, all delayed voice packets including the lost packet are stored in the playout buffer 4 during the repeated talk spurt reproduction period.

  If the voice data of the delayed voice packet is to be repeated more reliably and repeatedly with the voice data of the talk spurt, it is repeated after detecting that all the delayed voice packets have been received by making a loss packet retransmission request. You may make it output the audio | voice data of talk spurt.

  In any case, the same processing as described in the fourth example is performed, and the voice data of the delayed voice packet is output continuously from the repeated talk spurt voice data, and both can be reproduced continuously. . In this fifth example, even if the other party's caller is heard as if he / she spoke “shin, Shin--dedicated”, a long delay occurs while the word “shinjuku” is played back. , The meaning of the word is communicated accurately.

(5) Sixth example (example in which a long delay occurs in the talk spurt period and a delayed voice packet is not received, see FIG. 10)
The fifth example is a case where a delayed voice packet is received after the expected delay time (SPT) has elapsed, but the sixth example is due to a network failure or the like, and is delayed even after waiting while reproducing ambient noise. This is an example in which a received voice packet is not received.

  Therefore, as shown in FIG. 10, after executing the series of delay processing sequences shown in the fourth example, until the ambient noise audio data shown in the fifth example is reproduced, it is the same as the fourth example and the fifth example. There will be no explanation in between.

  The interpolation control unit 8 counts the elapsed time from the time when the delay is detected as an off time, and when the off time exceeds the allowable standby time, the common terminal 10C of the first changeover switch 10 is changed to the second changeover terminal 10B. The silent audio data generated by the switching / interpolation waveform generation unit 9 is output, and the audio data stored in the talk spurt holding buffer 13 is deleted. The allowable waiting time is set for the purpose of notifying the caller of the occurrence of a failure that cannot be easily recovered, such as a device failure, as distinguished from a normal delay. Here, at least after repeatedly playing the talk spurt, The time is longer than the time until playback starts, and 5 seconds. Therefore, when a delayed voice packet is not received and the off time has elapsed for 5 seconds, the ambient noise becomes silent, and the operator can know a failure other than the delay.

  In the fourth example described above, after the expected delay time (SPT) has elapsed, the voice data of the talk spurt is repeatedly output and reproduced without waiting for the reception of the delayed voice packet. As described in the fifth example, the voice data of the talk spurt may be repeatedly output and reproduced on condition that the delayed voice packet is received.

  In the above-described embodiment, the audio data obtained by decoding the audio packet is stored in the talk spurt holding buffer 13 and the ambient noise holding buffer 15, respectively, but either or both of the audio data before decoding are stored. You may memorize | store in the state of a packet.

  FIG. 12 is a block diagram of an audio data interpolating apparatus 30 according to another embodiment in which audio packets are stored in a talk spurt holding buffer and an ambient noise holding buffer. Components that are the same as or similar to those in the audio data interpolating apparatus 1 according to the first embodiment are given the same numbers, and descriptions thereof are omitted.

  In the audio data interpolating device 30, the common terminal 31D of the third changeover switch 31 controlled to be switched by the interpolation control unit 8 is connected to the input of the audio codec decoder 5, as shown in the figure, and the first changeover terminal 31A and the second changeover terminal 31A are connected. By connecting the switching terminal 31B and the third switching terminal 31C to the outputs of the ambient noise holding buffer 33, the talk spurt holding buffer 32 and the playout buffer 4, respectively, the voice codec output from each buffer is sent to the voice codec decoder 5. Input is made selectively.

  Since the talk spurt holding buffer 32 is connected to the output of the playout buffer 4, it always stores the audio buckets output from the playout buffer 4 to the audio codec decoder 5. The voice packets stored in the talk spurt holding buffer 32 are all erased every time the talk spurt period starts using the talk spurt start signal input from the talk spurt detection unit 20 as a reset signal. Therefore, in the talk spurt holding buffer 32, during the talk spurt period, voice packets output from the start part of the talk spurt period (hereinafter referred to as repeated talk spurt voice packets) are stored in order from the head address. Will be. The voice packet of the repeated talk spurt is decoded by the voice codec decoder 5 by switching the common terminal 31D to the second switching terminal 31B at the same timing as the voice data of the repeated talk spurt is output in the first embodiment. And output as repeated talk spurt audio data.

  Out of the audio packets output from the playout buffer 4, the audio packets output during the silent period during which the conversation is interrupted are input to the ambient noise holding buffer 33. The ambient noise holding buffer 33 also has a ring buffer structure, and rewrites the oldest voice packet stored therein into a voice packet of a voiceless period that is newly input. The ambient noise holding buffer 33 corresponds to the ambient noise holding buffer 15 of the first embodiment, and voiceless voice data stored in the ambient noise holding buffer 15 in the first embodiment is output. At the same timing, the common terminal 31D is switched to the first switching terminal 31A, decoded to voiceless voice data by the voice codec decoder 5, and reproduced as ambient noise.

  The operation of the audio data interpolating device 30 is the same as the operation described for the audio data interpolating device 1 by replacing the audio packet of the repeated talk spurt as the audio data of the repeated talk spurt and replacing the audio packet of the no audio period with audio data of no audio. Similarly, in the audio data interpolating apparatus 30 according to the other embodiment, the audio packet that is received after being repeatedly delayed with respect to the audio packet of the repeated talk spurt is output to the audio codec decoder 5, and the continuous audio Is played.

  In each of the embodiments described above, the Internet is used as an example of an IP network. However, a network such as a LAN may be used as long as a voice packet is transferred by a router based on an IP address. In addition, the present invention can be applied to a communication system in which another telephone such as a mobile phone is used as a communication device as long as it is a call system for making a call with a transmission device on the other side via an IP network.

  INDUSTRIAL APPLICABILITY The present invention is suitable for a call system that performs a call over a network in which a delay that cannot be interpolated with an interpolation waveform is expected.

1 is a block diagram of an audio data interpolation device 1. FIG. It is explanatory drawing explaining the determination method of voiced and unvoiced. It is a wave form diagram which shows the change of a threshold level. It is explanatory drawing which determines a talk spurt period and a silent period. FIG. 6 is a waveform diagram of a first example in which no delay occurs. It is a wave form diagram of the 2nd example in which delay occurred in a silent period. FIG. 10 is a waveform diagram of a third example in which a short delay occurs in the talk spurt period. FIG. 10 is a waveform diagram of a fourth example in which a long delay occurs in a talk spurt period and a voice packet delayed within an expected delay time (SPT) due to a spike delay is received. FIG. 14 is a waveform diagram of a fifth example in which a long delay occurs in the talk spurt period and a voice packet delayed after the expected delay time (SPT) due to spike delay is received. FIG. 16 is a waveform diagram of a sixth example in which a long delay occurs in the talk spurt period and a voice packet that is delayed as it is is not received. It is explanatory drawing explaining the process in the case of accompanying packet loss in a 4th example. It is a block diagram of the audio | voice data interpolation apparatus 30 which concerns on other embodiment of this invention. It is a block diagram which shows the conventional audio | voice data interpolation apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 30 Voice data interpolation apparatus 2 Network connection part 3 IP network 4 Playout buffer 5 Voice decoding circuit 8 Delay detection part (interpolation control part)
11 Change control unit (second changeover switch)
12 Voice determination unit 13, 32 Talk spurt holding buffer 19 Audio playback unit (D / A converter)
20 Talk spurt detection part 31 Change control part (3rd change-over switch)

Claims (9)

A network connection unit that receives, via the network, voice packets to which packet sequence numbers are assigned in the order of transmission by the transmission device;
A playout buffer for temporarily storing voice packets received by the network connection unit in order of transmission from the packet sequence number;
An audio decoding circuit that reads out audio packets stored at the top of the playout buffer and decodes them into audio data;
A voice determination unit that compares voice characteristics output when voice data output from the voice decoding circuit includes voice with a predetermined threshold and determines voice data of voice,
A talk spurt detection unit for determining a talk spurt period to be estimated as a conversation period from a period in which voiced voice data is continuous;
A talk spurt holding buffer that is reset every time the start of the talk spurt period is detected by the talk spurt detection unit and temporarily stores the voice data of the talk spurt period newly output from the voice decoding circuit;
A sound playback unit for playing back sound from the sound data;
A switching control unit that selectively switches the output of the audio decoding circuit and the output of the talk spurt holding buffer to the input of the audio reproduction unit;
A delay detection unit that detects a spike delay of the voice packet due to a network failure from a storage state of the voice packet in the playout buffer;
When the delay detection unit detects a spike delay during the talk spurt period, before the voice packet received delayed by the spike delay is decoded into the audio data and output to the audio reproduction unit, the switching control unit While the audio data of the talk spurt period temporarily stored in the holding buffer is output, the input of the audio reproduction unit switched to the output of the audio decoding circuit is switched to the output of the talk spurt holding buffer,
An audio reproducing unit reproduces audio from audio data of an audio packet received delayed by a spike delay in succession to audio data of a talk spurt period temporarily stored in a talk spurt holding buffer. Data interpolation device. The switching control unit outputs the speech decoding circuit and the talk spurt so that all the speech data of the talk spurt period temporarily stored in the talk spurt retention buffer is output when the expected delay time due to the spike delay elapses. The audio data interpolating apparatus according to claim 1, wherein the output of the buffer is switched to an input of an audio reproducing unit. The switching control unit, on the condition that the voice packet received delayed by the spike delay is temporarily stored in the playout buffer.
The input of the audio playback unit that is switched to the output of the audio decoding circuit is switched to the output of the talk spurt holding buffer, and all audio data for the talk spurt period temporarily stored in the talk spurt holding buffer is output to the audio reproducing unit. After
The input of the audio playback unit switched to the output of the talk spurt holding buffer is switched to the output of the audio decoding circuit, and the audio data decoded from the audio packet temporarily stored in the playout buffer is sent to the audio playback unit. The audio data interpolating apparatus according to claim 1, wherein the audio data interpolating apparatus is output. A network connection unit that receives, via the network, voice packets to which packet sequence numbers are assigned in the order of transmission by the transmission device;
A playout buffer for temporarily storing voice packets received by the network connection unit in order of transmission from the packet sequence number;
A voice decoding circuit for decoding voice packets into voice data;
A voice determination unit that compares voice characteristics output when voice data output from the voice decoding circuit includes voice with a predetermined threshold and determines voice data of voice,
A talk spurt detection unit for determining a talk spurt period to be estimated as a conversation period from a period in which voiced voice data is continuous;
An audio reproduction unit for reproducing audio from audio data output from the audio decoding circuit;
A talk spurt holding buffer which is reset every time the start of the talk spurt period is detected by the talk spurt detection unit and temporarily stores voice packets of the talk spurt period newly read from the head of the playout buffer, and
A switching control unit that selectively switches the output of the playout buffer and the output of the talk spurt holding buffer to the input of the audio decoding circuit;
A delay detection unit that detects a spike delay of the voice packet due to a network failure from a storage state of the voice packet in the playout buffer;
When the delay detection unit detects a spike delay during the talk spurt period, the switching control unit temporarily stores in the talk spurt holding buffer before the voice packet received delayed by the spike delay is output to the voice decoding circuit. While the stored voice packet of the talk spurt period is output, the input of the voice decoding circuit switched to the output of the playout buffer is switched to the output of the talk spurt holding buffer,
The voice decoding circuit decodes the voice packet received after the talk spurt period output from the talk spurt holding buffer, delayed by the spike delay, into voice data, and the voice playback unit decodes the voice data from the voice data. An audio data interpolating device characterized by reproducing the sound. The switching control unit outputs the playout buffer and the talk spurt holding buffer so that all the voice packets in the talk spurt period temporarily stored in the talk spurt holding buffer are output when the expected delay time due to the spike delay elapses. The voice data interpolating apparatus according to claim 4, wherein the output is switched to an input of a voice decoding circuit. The switching control unit, on the condition that the voice packet received delayed by the spike delay is temporarily stored in the playout buffer.
The input of the audio decoding circuit switched to the playout buffer output is switched to the output of the talk spurt holding buffer, and all the audio buckets in the talk spurt period temporarily stored in the talk spurt holding buffer are decoded by the audio decoding circuit. After
The input of the speech decoding circuit switched to the output of the talk spurt holding buffer is switched to the output of the playout buffer, and the speech packet temporarily stored in the playout buffer is decoded by the speech decoding circuit. The voice data interpolating device according to claim 4. Decode voice packets received from the network into voice data in the transmission order of the transmitting device,
While playing back the audio from the decoded audio data, determine the talk spurt period to be estimated as the conversation period from the period in which the audio data including the voice is continuous,
From the start of the talk spurt period, voice packets newly received during the talk spurt period or voice data decoded from the voice packets are continuously temporarily stored in the talk spurt holding buffer and reset at the start of the talk spurt period. Repeat the temporary memory,
When spikes voice packet delay occurs during a talk spurt period, after playing a sound from the audio packet or audio data of the talk spurt period temporarily stored in the talk spurt holding buffer, the voice packets received with a delay by the spike delay A voice data interpolation method, wherein voice is continuously reproduced from the voice data. 8. The voice data interpolation method according to claim 7, wherein the voice packet received after the spike delay is reproduced by performing the speech speed conversion in the transmission order of the transmission device to eliminate the spike delay. After temporarily storing voice packets or voice data decoded from voice packets outside the talk spurt period and detecting spike delay, before playing back voice from voice packets or voice data temporarily stored during the talk spurt period 9. The audio data interpolation method according to claim 7, wherein ambient noise is reproduced from an audio packet or audio data in a silent period.

Images