KR100591544B1 - METHOD AND APPARATUS FOR FRAME LOSS CONCEALMENT FOR VoIP SYSTEMS - Google Patents

Abstract

The present invention relates to a method and apparatus for concealing frame loss for reducing sound quality degradation due to packet loss that may occur when voice data is transmitted through a packet network.

The frame loss concealment method for a VoIP system according to the present invention uses frame data closest to the lost frame among frame data correctly received before the lost frame and frame data in a playout buffer when the frame is lost. To restore the lost frame data.

In this way, there is an effect that can contribute to the improvement of the sound quality of the speech coder in a packet loss environment without increasing the amount of computation.

Playout buffer, decoder, frame loss concealment module, VoIP system

Description

Frame loss concealment method and apparatus for BIPIP system {METHOD AND APPARATUS FOR FRAME LOSS CONCEALMENT FOR VoIP SYSTEMS}

1 is a diagram illustrating a frame loss concealment method based on repetition used in the conventional G.729.

2 is a diagram illustrating a proposed frame loss concealment method based on the characteristics of a conventional VoIP system.

3 is a block diagram illustrating an apparatus for transmitting and receiving a voice signal showing a process of transmitting a voice signal through a packet network for explaining the present invention.

4 is a block diagram of a frame loss concealment apparatus for a VoIP system according to an embodiment of the present invention.

5 is a flowchart illustrating an operation of a frame loss concealment apparatus for a VoIP system according to an embodiment of the present invention.

6 is a conceptual diagram illustrating a frame loss concealment method for a VoIP system according to an embodiment of the present invention.

The present invention relates to a method and apparatus for concealing a frame (packet) loss for reducing sound quality degradation due to packet loss that may occur when transmitting a voice signal through a packet network.

With the widespread use of the Internet, Voice over Internet Protocol (VoIP) technology for transmitting voice signals over the Internet has attracted much attention.

In general, in order to transmit a voice signal through a packet network, the transmitting end compresses the voice signal into a voice coder in units of frames and then packs and transmits the packet in packet units, and the receiving end uses packet data transmitted from the transmitting end again. Play the original audio signal. The voice coder, which is widely used to compress voice signals in VoIP systems, is a CELP type coder that processes voice signals of 20msec at once. This unit is called a frame. Generally, one packet may consist of one frame data or may consist of several frame data. That is, one packet may consist of one frame or several frames. However, since the speech coder operates in units of frames, the packet loss concealment algorithm also operates in units of frames. Therefore, the term frame loss concealment is sometimes used interchangeably with the term packet loss concealment.

When transmitting a voice signal through a packet network, the time taken to deliver each packet may not be constant because not all packets are transmitted through the same path but each packet may be transmitted through a different path. Therefore, a packet may be lost due to a network load, a packet may not be received within a predetermined time, or a packet may be received out of order.

If no real time operation is required, all packets can be reliably received by requesting retransmission of lost packets and increasing the buffer at the receiving end. However, when real-time operation such as voice call is required, retransmission cannot be requested, resulting in a drop in sound quality due to packet loss due to various causes. Packet loss is a major cause of direct impact on sound quality.

1 is a diagram illustrating a frame loss concealment method based on repetition used in the conventional G.729.

In G.729, as shown in FIG. 1, in order to recover parameters of a lost frame, information of frame data received correctly without loss immediately before the loss is used is used. That is, in the prior art, the LSF (Line Spectrum Frequency) of the lost current frame data uses the LSF coefficient of the previous frame as it is, and the pitch is increased by increasing the pitch of the previous subframe by one. In addition, the gain of the pitch and the fixed codebook is used by reducing the gain of the previous frame by using the attenuation constant.

In order to improve the performance of the repetition based frame loss concealment method used in G. 729, US Patent No. 5,732,389 (1998.3.24) discloses "Voiced / unvoiced classification of speech for excitation codebook selection in CELP speech decoding during frame erasures. In the prior art, basically, the data of the lost frame is used as it is or scaled by the data of the previous frame, and the lost frame is classified into voiced and unvoiced, and in the case of voiced sound, the excitation signal is reproduced using only the pitch information. Reproduces the excitation signal only with fixed codebook information.

Meanwhile, FIG. 2 is a diagram illustrating a proposed frame loss concealment method based on characteristics of a conventional VoIP system.

In general, a VoIP system uses a playout buffer at a receiving end to reduce sound degradation caused by delay jitter. Rather than restoring and outputting the audio signal as soon as the packet is received, it has a buffer that can act as a buffer and starts restoring the audio signal when the data is full in the buffer. Therefore, if the size of the playout buffer is increased, the sound quality deterioration due to packet loss can be reduced. However, if the size of the playout buffer is increased, the overall call delay is increased, so that reverberation occurs and natural conversation becomes difficult. have.

Unlike a wireless network, a VoIP system can use frame data in a playout buffer to predict lost frame information. If the frame data after the lost frame is stored in the playout buffer, the future frame data may be linearly interpolated together with the past frame data information to predict the lost frame information. Such a method is disclosed in the paper "improved frame erasure concealment for CELP-based coders" published in ICASSP2000.

In this paper, when one frame data is lost, the current frame data is predicted by linear interpolation of the immediately previous frame and the subsequent frame data correctly received without loss.

At this time, pitch gain and pitch lag without using a predictor in quantization are predicted through linear interpolation, and the remaining LSF and fixed codebook parameters are frame loss concealment based on repetition mainly used in existing codecs. Predict using Experimental results for the case of one frame per packet and no packet loss occur continuously show that the frame loss concealment method using linear interpolation is better than the repetition based frame loss concealment method.

On the other hand, the voice signal has a property that its characteristics slowly change with time. Therefore, there is a correlation between neighboring frame data and a very high correlation between adjacent frame data. And as the distance between frames increases, the correlation decreases. Therefore, most frame loss concealment methods utilize the correlation between these frame data. However, the VoIP system can use the received frame data after the loss of the frame data in the playout buffer, so that frames lost between the received frame data and the frame data in the playout buffer before frame loss occurs without additional delay increases. The lost frame data can be predicted more effectively by using the frame data having the highest correlation with.

 The present invention aims to solve the above problems, and an object of the present invention is to store lost data stored in a playout buffer and frame data correctly received before a lost frame in order to recover a voice signal of a lost frame in a VoIP system. Disclosed is a method and apparatus for concealing a frame loss for a VoIP system which can more accurately recover a lost audio signal of a lost frame using the frame data closest to the lost frame after the lost frame.

In order to achieve the above object, the present invention provides a method that can effectively predict the frame data lost due to packet loss.

Frame loss concealment method for a VoIP system according to an aspect of the present invention comprises the steps of: a) detecting the lost frame; b) when frame loss is detected in step a), determining one frame data closest to the lost frame from the correct frame data received before the lost frame and the frame data in a playout buffer; And c) reconstructing a voice signal by predicting the data of the lost frame using the determined frame data.

When restoring lost frame data by using the previously received frame data without loss, constraints may be given when restoring parameters based on the frame data in the play-out buffer. When restoring lost frame data, constraints may be given when restoring parameters based on frame data previously received without loss when restoring the lost frame data.

Further, when the frame loss occurs continuously, each lost frame is successively using frame data closest to the lost frame among the correct frame data received before the lost frame and the frame data in the playout buffer. The lost frame data can be sequentially restored.

The present invention provides a frame loss concealment apparatus for recovering a lost voice signal when a loss occurs in packet data of a voice signal transmitted from a transmitter.

A frame loss concealment apparatus for a VoIP system according to an aspect of the present invention stores the frame data in a playout buffer, delivers the frame data to the decoder at predetermined time intervals, and in a playout buffer when a frame is lost. A playout buffer for transmitting the frame data closest to the lost frame and the time information to a frame loss concealment module; In the case of restoring a voice signal differently according to correct frame data and lost frame data among the frame data transferred from the playout buffer, and restoring lost frame data, play with the transmitted frame data without loss before the lost frame. A frame loss concealment apparatus for predicting lost frame data using frame data closest to the lost frame among frame data transferred from an outbuffer; And a Bad Frame Indicator (BFI) instructing between the playout buffer and the decoder to switch based on the correct frame and the lost frame.

And the decoder comprises: a decoding module for recovering correct frame data; And received frame data closest to the lost frame from the play out buffer, and received frame data received without loss before the lost frame from the decoding module to select the frame closest to the lost frame and lost the frame data. It includes a frame loss concealment module for predicting the reconstructed speech signal.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention. Like parts are designated by like reference numerals throughout the specification.

Hereinafter, with reference to the accompanying drawings will be described in detail the present invention.

3 is a block diagram illustrating an apparatus for transmitting and receiving a voice signal showing a voice signal through a packet network for explaining the present invention.

As illustrated in FIG. 3, the voice signal transmitter includes an A / D converter 20, an encoder 30, and a packet protocol module 40.

The A / D converter 20 converts an analog voice signal of a sender input through a microphone into a digital voice signal.

The encoder 30 compresses and encodes a digital speech signal.

The packet protocol module 40 converts the compressed and encoded digital voice data into a form suitable for transmission through a packet network by processing it according to the Internet Protocol (IP;) and outputs it in the form of a voice packet.

In addition, the voice signal receiver includes a packet protocol module 40, a playout buffer 50, a decoder 60, and a D / A converter 70.

In the voice signal receiver, the packet protocol module 40 receives and unpacks a voice packet transmitted through a packet network, and stores the voice data in a frame unit in the playout buffer 50 in a time sequence.

The playout buffer 50 stores the converted voice data in frame units, and transmits the frame data to the decoder 60 at predetermined times.

The decoder 60 restores the voice signal from the received frame data.

The D / A converter 70 converts digital voice data reconstructed into a voice signal into an analog signal. The converted analog voice signal is output through the speaker.

In the apparatus for transmitting and receiving a voice signal configured as described above, most of the encoders except the PCM encoder operate on a frame basis. Data is output by compressing a voice signal in units of frames, and one frame of data is packed into one packet, or two or three frames of data are packed into one packet and transmitted. The receiving end arranges the received packet data in time order, unpacks the data, and converts the received packet data into frame data and stores the data in the playout buffer 50. The size of the playout buffer 50 may vary depending on the size of the transmission delay, may have a fixed size, and deliver the frame data to the decoder 60 at frame time intervals. The digital voice signal is reproduced using the frame data, converted into an analog signal through a D / A converter, and output.

As described above, when packet data is transmitted through a packet network, packets that are not received at a predetermined time are generally regarded as losses because packets are lost due to network load or the time required to transmit each packet is not constant. As a result, the sound quality deteriorates. When the lost voice signal is effectively restored, the sound quality deterioration can be reduced.

4 is a block diagram illustrating a frame loss concealment apparatus for a VoIP system according to an embodiment of the present invention.

As shown in FIG. 4, the apparatus for concealing frame loss according to an embodiment of the present invention includes a playout buffer 50, a decoder 60, and a BFI 80. The decoder 60 also includes a frame loss concealment module 62 and a decoding module 64.

The playout buffer 50 stores the converted data in units of frames, transfers the frame data at frame time intervals, and when a loss of a frame occurs, the playout buffer 50 closest to the lost frame among the frame data in the playout buffer 50. The data of the frame is passed to the frame loss concealment module 62. The playout buffer 50 may have a fixed size, but the size of the buffer may change according to network conditions. In addition, the playout buffer 50 transmits the data of the frame closest to the lost frame to the frame loss concealment module, while also transmitting time information. Here, in the embodiment of the present invention according to FIG. 4, it is assumed that the size of the playout buffer (or jitter buffer) 50 is 4, and an error (loss) occurs at the Nth time. The frame data for this is designated as F (N).

The decoder 60 includes a frame loss concealment module 62 to reduce sound quality degradation due to frame loss. In this case, a module operating when frame data is normally received without loss is called a decoding module. A module for reducing sound quality degradation due to lost frame data is called a frame loss concealment module. Here, the frame data includes a pitch period, an LSP, a fixed codebook index, a pitch, and a gain of the fixed codebook in the case of a CELP type encoder / decoder.

The frame loss concealment module 62 uses the frame data closest to the lost frame data among the frame data received without loss and the frame data stored in the playout buffer 50 before the frame loss occurs, and the voice of the lost frame is lost. Restore the signal. In addition, when the frame loss concealment module 62 restores a lost frame using the data of the frame closest to the lost frame among the received frame data without loss before and after the loss occurs, the frame loss concealment module 62 is received without loss of the other one. Frame data can be used as a constraint when predicting lost frame parameters.

The decoding module 64 recovers the audio signal of the received frame without loss.

The BFI (Bad Frame Indicator) 80 is a flag indicating whether or not a frame loss has occurred, and the data of a frame received without loss causes the decoding module 64 to restore a voice signal, and when a frame loss occurs. The frame loss concealment module 62 enables switching to recover the speech signal for the lost frame. For example, if frame loss occurs and there is no frame data to be delivered to the decoder in the playout buffer 50, the BFI 80 becomes 1 to instruct the switch to be connected to the frame loss concealment module 62, If there is frame data in time order in the playout buffer 50, the BFI 80 goes to zero indicating that the switch is coupled to the decoding module 64.

The BFI 80 may be configured as a separate module or may be provided with a separate module for the BFI 80 in the playout buffer 50 or the decoder 60.

The operation of the frame loss concealment apparatus for the VoIP system configured as described above will be described in detail with reference to FIG. In this case, it is assumed that an error (loss) occurs in the Nth time as shown in FIG. 4, and the data of the frame for the Nth time is denoted as F (N). When restoring the frame data based on the time information, the operation as shown in FIG. 5 is continuously repeated.

As shown in FIG. 5, the decoder 60 receives a packet according to time information (N-1 to N + 2) (S60), restores it, and reproduces a voice signal. At this time, the frame data generated by unpacking the received packet is stored in the playout buffer 50 (S61). When the frame data is completely filled in the playout buffer 50, the audio signal is transmitted to the decoder 60 from that time.

First, it is determined whether a loss of a frame has occurred. To this end, the playout buffer 50 has a frame data F (N-1) corresponding to the N-1th time according to the time order, so that the BFI 80 decodes the decoding module. (64). Then, the decoding module 64 receives F (N-1) and restores the voice signal (S64-S65).

Next, since the playout buffer 50 does not have F (N) in the playout buffer 50 at the point in time when the frame data F (N) corresponding to the Nth time should be transmitted to the decoder 60, the BFI 80 ) Is connected to the frame loss concealment module 62 (S60-S62). The playout buffer 50 then transfers the data F (N + 1) of the frame corresponding to the N + 1th time to the Nth time to the frame loss concealment module 62. The frame loss concealment module 62 reconstructs the speech signal by predicting data of the lost frame using the frame closest to the lost frame among the previous frame and the subsequent frame of the lost frame (S63). At this time, when predicting the lost frame data using the frame which is transmitted before the lost frame and received without loss, or the frame transmitted after the lost frame without loss and the closest to the lost frame, the remaining one is estimated. Frame data is used as a constraint when predicting lost frame data.

Next, when the frame data F (N + 1) corresponding to the N + 1th time needs to be restored, F (N + 1) exists in the playout buffer 50, so that F (N +) to the decoding module 64. 1) is transmitted to restore F (N + 1) to reproduce the audio signal (S64-65). The frame data corresponding to the N + 2th time operates in the same manner as the method of restoring F (N + 1).

In the embodiment of the present invention, the case of one frame per packet has been described. However, when the number of frames per packet increases, several frames may be continuously lost even if one packet is lost. When a plurality of frames are continuously lost in this manner, the frame loss concealment module 62 uses the closest frame data among the received data without loss before or after the lost frame for each lost frame to provide time information. Therefore, the lost audio signal is sequentially restored.

As described above, the frame loss concealment method according to an embodiment of the present invention mainly uses a CELP (Code-Excited Linear Predictive) speech coder.

The CELP coder uses a kind of hybrid coding technique that produces high quality speech signals even at bit rates lower than 16 kbps. In general, a CELP speech coder extracts and transmits a fixed codebook index and a pitch and a gain of an LSF, a fixed codebook, and a pitch from a speech signal of one frame, and restores the speech signal from these data.

6 illustrates a frame loss concealment method according to an embodiment of the present invention. Hereinafter, a method of recovering data of lost frames in the CELP speech encoder will be described with reference to FIG. 6.

As illustrated in FIG. 6, the CELP voice encoder stores frame data in the playout buffer 20, and the playout buffer 50 delivers the frame data to the decoder at predetermined times. When the frame loss occurs, the playout buffer 50 transmits the data of the frame closest to the frame lost in the playout buffer 50 to the frame loss concealment module 62. The frame loss concealment module 62 selects the frame closest to the lost frame by comparing the frame received from the playout buffer 50 with the received frame before frame loss occurs. The lost frame data is recovered using the selected frame data. In the case of the CELP-based speech coder, the LSP, pitch information, and fixed codebook information of the lost frame are recovered. In this case, data other than the frame selected to recover the lost frame is used as a constraint for prediction. Can be used

Table 1 shows the perceptual evaluation of the sound quality according to the conventional frame loss concealment method according to Figures 1 and 2 and the frame loss concealment method according to an embodiment of the present invention. Here, the frame loss concealment method based on repetition according to FIG. 1 is called F-PLC, and the frame loss concealment method based on linear interpolation according to FIG. 2 is called MI-PLC, and the frame loss concealment method according to an embodiment of the present invention is described. Is called FB-PLC.

Packet size PLC Type % Frame loss One 3 5 7 10 15 10 F-PLC 3.67 3.38 3.24 3.09 2.91 2.72 MI-PLC 3.69 3.46 3.32 3.20 3.04 2.84 FB-PLC 3.71 3.50 3.39 3.28 3.13 2.92 20 F-PLC 3.62 3.27 3.09 2.93 2.70 2.52 MI-PLC 3.65 3.35 3.17 3.03 2.81 2.63 FB-PLC 3.68 3.42 3.30 3.16 3.02 2.78 30 F-PLC 3.62 3.30 2.99 2.81 2.60 2.32 MI-PLC 3.65 3.42 3.13 2.97 2.80 2.52 FB-PLC 3.70 3.47 3.21 3.09 2.93 2.74

In Table 1, when the frame loss rate is 5% in Table 1, Preference test results of MI-PLC and FB-PLC are shown.

Speaker Preference Score (%) MI-PLC FB-PLC female 33.33 66.67 male 30.56 69.44

Referring to Table 1 and Table 2, it can be seen that the frame loss concealment method according to the embodiment of the present invention is more effective than the conventional frame loss concealment method.

In addition, as described above, the lost audio signal restoration method according to the present invention may be stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) recording a program implemented as a program.

The above embodiments are intended to illustrate the present invention, the scope of the present invention is not limited to the embodiments, it can be modified or modified by those skilled in the art within the scope of the invention defined by the appended claims. .

According to the present invention, the lost frame data is recovered using the frame data closest to the lost frame among the correct frame data before the lost frame and the frame data after the lost frame stored in the playout buffer. By doing so, it is possible to more efficiently and more accurately recover the lost audio signal of the lost frame due to the correlation between the adjacent audio signals of the adjacent frames, and conceal the frame loss concealment method based on the conventional repetition or the linear interpolation Compared to the method, it can contribute to the improvement of voice call quality in packet network with transmission error without increasing the amount of computation.

Claims (6)

In the frame loss concealment method for reducing the deterioration of the quality of the restored speech signal generated by the loss of the frame data of the speech signal received from the transmitting device, a) detecting whether the received frame data is lost; b) when frame loss is detected in step a), determining a frame closest to the lost frame among the frames received in the playout buffer and the frames received without loss before the lost frame; And c) restoring a speech signal by predicting data of the lost frame using the determined frame data; Frame loss concealment method for a VoIP system comprising a. The method of claim 1, When predicting lost frame data by using the data of the frame received without loss before, VoIP system characterized in that it is used as a constraint when predicting the lost frame data using the frame data in the playout buffer Method for concealing frame loss. The method of claim 1, When restoring lost frame data by using the frame data in the playout buffer, the start of the lost frame data is a constraint when predicting the data of the lost frame using the previously received frame data without loss. A frame loss concealment method for a VoIP system, characterized in that the use. The method of claim 1, When the loss of the frame data occurs continuously, each lost frame uses data of the frame closest to the lost frame among the correct frame data received before the lost frame and the frame data in the playout buffer. And sequentially recovering lost frame data sequentially. In the frame loss concealment apparatus for reducing the deterioration of the quality of the restored speech signal generated by the loss of the frame data of the speech signal received from the transmitting apparatus, A playout buffer that stores frame data, transfers the frame data to the decoder at predetermined times, and transfers frame data and the time information closest to the lost frame when a frame loss occurs; When the frame data received from the playout buffer and the frame data lost without loss are restored in a different manner, and when the frame data is lost, the frame data transferred without loss before the lost frame and A decoder for predicting lost frame data by using frame data closest to the lost frame among the frame data transferred from the playout buffer; And A bad frame indicator (BFI) instructing between the playout buffer and the decoder to switch based on the received frame and the lost frame without the loss Frame loss concealment device for a VoIP system comprising a. The method of claim 5, wherein the decoder, A decoding module for recovering received frame data without loss; And Frame data lost from the play out buffer is received closest to the lost frame, frame data received without loss before the lost frame is received from the decoding module, and the frame closest to the lost frame is lost. Frame loss concealment module to predict speech signal Frame loss concealment device for a VoIP system comprising a.

Images