KR20040090567A - Method for concealing Packet Loss using Information of Packets before and after Packet Loss - Google Patents

Abstract

PURPOSE: A PLC(Packet Loss Concealment) method using the packet information before/after loss is provided to compare both phonemes by using the pitch information obtained from two packets and the spectral distance between the packets before/after packet loss, and to compensate for a loss section with front/back phoneme information, thereby supplying more natural synthetic voice signals. CONSTITUTION: A receiving-side confirms that packet loss occurred during a VoIP(Voice over Internet Protocol) transmission(601). To conceal the packet loss, the receiving-side detects pitches from two packets before and after the loss(602). The receiving-side confirms whether the two packets are all the same as voiced sounds or voiceless sounds(603). If so, the receiving-side calculates the spectral distance between voice signals of the two packets(604), and compares the calculated spectral distance with the spectral distance between adjacent packets to check whether the different between the two spectral distance is big(605). If not, the receiving-side judges that the phonemes of the two packets are the same(606), and compensates for the entire packet loss section with the packet before the packet loss(607). If the different is big, the receiving-side judges the phonemes of the two packets are different(608), and compensates for two-thirds of the packet loss section with the phoneme of the packet before the packet loss and the rest with the phoneme of the packet after the loss(609).

Description

Method for concealing Packet Loss using Information of Packets before and after Packet Loss}

The present invention relates to a packet loss concealment method using packet information before and after loss, and a computer readable recording medium recording a program for realizing the method. More specifically, it is Voice over Internet Protocol (VoIP). The present invention relates to a packet loss concealment method using pre- and post-loss packet information for improving voice quality degradation due to packet loss in an audio transmission system, and a computer-readable recording medium recording a program for realizing the method.

Conventionally, techniques commonly used to compensate for packet loss in VoIP include FEC (Forward Error Correction) and Packet Loss Concealment (PLC). The FEC has a problem that additional processing of the source is required at the transmitting side and the receiving side, and its efficiency depends on the loss rate and the loss distribution. In addition, the FEC method requires a wide bandwidth and a delay due to additional processing.

Packet loss concealment, on the other hand, generates a synthetic speech signal that can replace the lost data on the network for the received bit stream. Since the speech signal is usually partially stabilized, it is possible to estimate the speech signal of the lost packet from the previous speech signal. So far, packet loss concealment methods mainly applied to VoIP codecs are replaced or silently processed the information of previous packet in the loss interval. Various algorithms based on pattern matching or voice information modeling are studied and application cases are presented. have.

However, the loss concealment method up to now can be effectively applied for the low packet loss rate (<15%) of a single frame (4-40ms), and it is difficult to compensate for the packet loss over a relatively long period due to continuous packet loss. There was a problem.

The problem of the prior art will be described once again.

On a real network, packet loss often occurs consecutively. Therefore, the loss compensation of the speech signal should provide a robust result even for packet loss in a relatively long period. However, it is true that the loss compensation using only the previous frame information is limited in terms of the transition | transition of the audio signal or the loss to which all one phoneme information is lost.

For example, by unduly repeating pitch information, unnatural artifacts are generated. This phenomenon cannot be said that the audio signal is compensated for loss in audio transmission correctly. Of course, the 'ITU-T' recommendation or the conventional patent application "Method and apparatus for performing packet loss or frame erasure concealment (Korean Application No. 10-2000-7014272)" also gradually reduces the generated signal to prevent this. Presenting. However, this has a problem that, as described above, it is not enough to compensate for a relatively long period of packet loss.

The present invention has been made to solve the above problems, and the packet loss in the audio transmission system using a method such as VoIP is lost by comparing both phonemes using the pitch information before and after the loss and the spectral distance. The packet loss concealment method using pre / post loss packet information to provide a more natural synthesized speech signal using the front / rear phoneme information of the section and a computer-readable recording medium recording a program for implementing the method. The purpose is to provide.

1 is an exemplary configuration diagram of an audio transmission system to which the present invention is applied.

2 is an exemplary view of packet loss interval compensation in a packet loss concealment method using before and after loss packet information according to the present invention.

3 is an exemplary diagram of an OLA for reducing discontinuity in a packet loss concealment method using before and after loss packet information according to the present invention.

Figure 4 is an exemplary view for controlling the size of the loss interval synthesized voice in the packet loss concealment method using before and after loss packet information according to the present invention.

5 is a diagram illustrating an embodiment of a packet loss concealment method using before and after loss packet information according to the present invention;

6A and 6B are flowcharts illustrating an embodiment of a packet loss concealment method using before and after loss packet information according to the present invention.

* Explanation of symbols for the main parts of the drawings

110: encoder 115: lost frame detector

120: decoder 130: packet loss concealment module

In order to achieve the above object, the present invention provides a packet loss concealment method applied to an audio transmission system, wherein a loss of a received voice packet is checked and packets before and after the loss interval are different from each other. Determining whether the phoneme is a phoneme; A second step of concealing packet loss by compensating the entire loss section by using the voice information of the pre-lost packet if it is determined that the packets before and after the loss are the same phoneme; And as a result of the determination in the first step, if it is determined that the packets before and after the loss are different phonemes from each other, the predetermined portion of the loss interval is compensated by using the voice information of the packet before the loss and the predetermined portion of the loss interval is excluded. And a third step of concealing packet loss by compensating the interval by using the voice information of the packet after the loss.

In addition, the present invention, the audio transmission system having a processor, the first function of checking the loss of the received voice packet and comparing the packets before and after the loss interval to determine whether the packets before and after the loss are different phonemes ; A second function of concealing packet loss by compensating for the entire loss section by using the voice information of the pre-lost packet if it is determined that the packets before and after the loss are the same phonemes as a result of the determination of the first function; And as a result of the determination of the first function, if it is determined that the packets before and after the loss are different phonemes from each other, the predetermined portion of the loss interval is compensated using the voice information of the packet before the loss, and the other portions except the predetermined portion of the loss interval are excluded. A computer readable recording medium having recorded thereon a program for realizing a third function for concealing packet loss by compensating the section using the voice information of the packet after the loss is provided.

In order to improve the voice quality of VoIP, the present invention compares the pitch information and the spectral distance of the phoneme before and after the loss of the voice packet loss interval to check whether the same phoneme is used, and accordingly the loss interval using only the phoneme information before packet loss. To compensate for this loss or to use additional phoneme information after loss to compensate for the loss. Here, the compensation of the loss section is to produce a synthesized speech signal to replace the missing data.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an exemplary configuration diagram of an audio transmission system to which the present invention is applied.

The audio transmission system to which the present invention is applied includes an encoder 110, a lost frame detector 115, a decoder 120, and a packet loss concealment module 130 as shown in the figure.

In FIG. 1, the encoder 110 receives an input audio frame and transmits a coded bit-stream. The transmitted bit-stream is passed to the lost frame detector 115 at the receiving side which determines whether the frame has been lost. If the lost frame detector 115 determines that the frame is lost, the lost frame detector 115 notifies the packet loss concealment module 130 to reconstruct the lost frame.

The packet loss concealment module 130 compensates for the lost frame according to a given packet loss concealment scheme so that an intact audio frame can be output through the decoder 120.

The audio transmission system as described above may be an example of a system to which the present invention is applied.

In the packet loss concealment method according to the present invention, pitch detection is performed on the packet before the loss and the packet after the loss, and when the pitch is detected, the packet loss concealment is divided into voiced sound and voiceless sound. If both voiced or voiced sounds are equal, the spectral distance between the voice signals of the two packets is calculated. If the spectral distance is relatively different from the neighboring spectral distance, it is determined to be different phonemes from each other, otherwise it is determined to be the same phoneme.

If the phonemes at both ends of the loss section are the same, the phoneme information of the previous packet is used to compensate for the entire loss section. Otherwise, two-thirds of the loss section is compensated with the previous phoneme information, and the remaining third part is the phoneme of the later packet. Reward with information In this case, when the phoneme is a voiced sound, the pitch section voice information is repeated, and in the case of the unvoiced sound, all voice information of the packet is used. Here, the repetition of the pitch section speech information performed in the case of voiced sound is briefly shown through FIG. 2.

When the loss section is compensated with the front / rear voice information, a discontinuous part appears at the end or the middle of the loss section. This part can reduce the discontinuity by applying an overlap addition (OLA: Overlap Add) using a triangular window by additionally compensating for the overlapping section D (5 msec) as shown in FIG.

As the duration of the loss becomes longer, the synthesized sound to be reproduced becomes unnatural if the same voice signal is repeatedly compensated for a long time. In this case, as illustrated in FIG. 4, the control unit adjusts the size of the voice signal in the loss section 2/3 by 20% per 10 msec, and gradually increases the remaining 1/3 to match the size of the back frame voice signal. If the front and rear of the loss section are the same phoneme, the amplitude of the compensated voice signal is linearly adjusted in consideration of the peak points of the front and rear voice information.

2 is an exemplary view of packet loss interval compensation in a packet loss concealment method using before and after loss packet information according to the present invention.

2 shows the characteristics of a packet loss concealment method using before and after loss packet information according to the present invention. That is, it shows that the pitch section speech information before the loss section and the pitch section speech information after the loss section are applied to the loss section for the packet loss section. In addition, it indicates that the OLA is applied to reduce the discontinuity between the packets before and after the loss section.

3 is an exemplary diagram of an OLA for reducing discontinuity in a packet loss concealment method using before and after loss packet information according to the present invention.

According to the present invention, the OLA for reducing the discontinuity of the boundary portion generated by using the voice information before and after loss in compensating the loss interval synthesized voice is shown. At this time, the OLA is made in the form of a triangular window as shown in FIG. 3.

4 is an exemplary view illustrating the control of the size of the loss interval synthesized voice in a packet loss concealment method using before and after loss packet information according to the present invention.

In FIG. 4, since the loss interval is long as described above, the size may be unnatural. The method shown in FIG. 4 is a case where different phonemes are before and after loss, and when the same phonemes are used, the compensation voice signal is linearly adjusted in consideration of peak points of the voice information before and after.

5 is a diagram illustrating an embodiment of a packet loss concealment method using before and after loss packet information according to the present invention.

If a packet loss interval occurs between the detected speech packet and the speech packet, pitch detection is performed and the loss interval is measured by comparing the spectral distance calculation result through Fast Fourier Transform (FFT) with the spectral distance between basic neighboring packets. To compensate. Also, adjust the size of the speech waveform to fit the length of the loss section.

6A and 6B are flowcharts illustrating an embodiment of a packet loss concealment method using before and after loss packet information according to the present invention.

The receiver confirms that packet loss has occurred on the VoIP transmission (601).

To conceal packet loss, the receiving side first performs pitch detection on the packet before loss and the packet after loss (602). As a result of pitch detection, if a pitch is detected, it is discriminated as a voiced sound or an unvoiced sound.

Next, it is checked whether the packets before and after the loss are the same with voiced or unvoiced sounds (603). As a result, if the packets before and after the loss are the same as the voiced or unvoiced sound, the spectral distance between the voice signals of the two packets is calculated and identified (604). In operation 605, it is checked whether the identified spectral distance is different from the spectral distance between neighboring packets.

As a result of the check, if there is not much difference, it is determined as the same phoneme (606), and the entire packet loss section is compensated with the packet before loss (607).

As a result of the check, if the differences are relatively large, the packets before and after the loss are determined to be different phonemes (608). In addition, as a result of confirming whether the packets before and after the loss are the same as the voiced sound or the unvoiced sound, one is voiced sound and the other is unvoiced sound.

If the packets before and after the loss are determined to be different phonemes, compensation is made by two-thirds of the loss section being the phonemes of the pre-loss packet and the remaining 1/3 being the phonemes of the post-loss packet after the determination (609).

When the phoneme is voiced in the compensation of the lost section, the pitch section voice information is repeated as shown in FIG. 1, and in the case of the unvoiced sound, all voice information of the packet is used.

Next, it is examined whether discontinuous portions have occurred according to the loss interval compensation (610). As a result, if a discontinuous surface is generated, an additional compensating voice signal is made for the corresponding overlapping section (D of FIG. 3), and the discontinuity is reduced by applying an OLA using a triangular window (611). In other words, if the phoneme before and after the loss section is considered to be different phonemes, discontinuities may occur in the middle part compensated by the voice information before and after the loss section. By compensating for the loss section, discontinuities can occur at the end of the loss section, and OLA is applied when such discontinuities occur.

After applying the OLA to the portion where the discontinuous face does not occur or the discontinuous face caused by the loss interval compensation, it is checked whether the loss interval is longer than the reference value (612). The longer the loss interval is, the longer the repeated audio signal is, so that the synthesized sound is unnatural. Therefore, by checking whether the loss section is longer than the given reference value, if the loss section is longer than the given reference value, the amplitude is linearly adjusted for the speech signal of the loss section to be compensated (613). That is, if the phoneme before and after the loss section is different from each other, adjust the audio signal size of 2/3 section of the loss section by 20% per 10msec, and adjust the remaining 1/3 to match the size of the back frame audio signal. If it is determined that the phonemes are gradually increased, the amplitude of the compensating voice signal is linearly adjusted in consideration of the peak points of the front and rear voice information.

The method of the present invention as described above may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes are possible in the art without departing from the technical spirit of the present invention. It will be clear to those of ordinary knowledge.

As described above, the present invention compares the two phonemes using the pitch information before and after the loss and the spectral distance, and compensates the loss section with the phoneme information before and after the loss. There is an effect that the natural speech signal can be compensated for a relatively long loss period due to continuous packet loss.

Claims (8)

In the packet loss concealment method applied to an audio transmission system, Checking a loss of the received voice packet and comparing packets before and after the loss interval to determine whether the packets before and after the loss are different phonemes; A second step of concealing packet loss by compensating the entire loss section by using the voice information of the pre-lost packet if it is determined that the packets before and after the loss are the same phoneme; And As a result of the determination in the first step, when it is determined that the packets before and after the loss are different phonemes, a portion of the loss section is compensated using the voice information of the packet before the loss, and the section except the predetermined portion of the loss section is excluded. A third step of concealing packet loss by compensating for the loss using the voice information of the packet after the loss Packet loss concealment method using before / after loss packet information comprising a. The method of claim 1, A fourth to reduce discontinuity by applying an overlap addition (OLA: OverLap Add) over a predetermined overlap period to the discontinuous portion generated in the loss section by compensating the loss section using the voice information of the before and after loss packet. step Packet loss concealment method using before / after loss packet information further comprising. The method of claim 1, The first step is, A fifth step of the audio transmission system confirming a packet loss with respect to the received voice packet; A sixth step of determining whether packets before and after the loss are phonemes different from each other based on whether a pitch is detected for the packets before and after the loss; And In the case where the pitch detection result is the same, the spectral distance between the voice signals of the pre- and post-loss packets is obtained to determine whether the pre- and post-loss packets are different phonemes based on whether there is a difference between the spectral distances between neighboring packets. 7 steps Packet loss concealment method using before / after loss packet information comprising a. The method according to any one of claims 1 to 3, An eighth step of linearly adjusting a magnitude of a compensated voice signal reproduced based on voice information of the pre / post loss packet for the loss section in which the synthesized sound is unnatural due to a length longer than a predetermined reference value; Packet loss concealment method using before / after loss packet information further comprising. The method of claim 4, wherein The eighth step, A ninth step of checking whether packets before and after the loss are determined to be different phonemes for the loss intervals in which the synthesized sound is unnatural due to a length longer than a predetermined reference value; As a result of the checking in the ninth step, if it is determined that the pre- and post-loss packets are different phonemes, the magnitude of the voice signal for the predetermined portion of the first half of the loss section compensated with the voice information of the pre-loss packet is based on this. A tenth step of adjusting to reduce the ratio at a predetermined rate and gradually increasing the rest to match the size of the packet voice signal after the loss; And As a result of the checking in the ninth step, if it is determined that the packets before and after the loss are the same phonemes, the amplitude of the compensated speech signal is linearly determined in consideration of the peak points of the voice information of the packets before and after the loss. 11th step to adjust Packet loss concealment method using before / after loss packet information comprising a. In an audio transmission system having a processor, A first function of checking a loss of a received voice packet and comparing packets before and after a loss interval to determine whether the packets before and after the loss are different phonemes; A second function of concealing packet loss by compensating for the entire loss section by using the voice information of the pre-lost packet if it is determined that the packets before and after the loss are the same phonemes as a result of the determination of the first function; And As a result of the determination of the first function, if it is determined that the packets before and after the loss are different phonemes from each other, the predetermined portion of the loss interval is compensated using the voice information of the packet before the loss, and the interval except the predetermined portion of the loss interval is excluded. A third function for concealing packet loss by compensating for the loss after the loss by using the voice information of the packet A computer-readable recording medium having recorded thereon a program for realizing this. The method of claim 6, A fourth method for reducing discontinuity by applying an overlap addition (OLA: OverLap Add) over a predetermined overlapping section for the discontinuous portion generated in the loss section as compensation for the loss section using the voice information of the before and after loss packet. function A computer-readable recording medium that records a program for further realization. The method according to claim 6 or 7, A fifth function of linearly adjusting a magnitude of a compensated voice signal reproduced based on voice information of the pre- and post-loss packet for the loss section in which the synthesized sound may be unnatural because it is longer than a predetermined reference value. A computer-readable recording medium that records a program for further realization.