JP5074749B2 - Voice signal receiving apparatus, voice packet loss compensation method used therefor, program for implementing the method, and recording medium recording the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily perform the processing of speech packet loss compensation. <P>SOLUTION: Speech packets stored in a receiving buffer (12) are read out in the increasing order of numbers, decoded, and speech signals as many as a predetermined number of feames are put in a delay buffer unit (19). A speech signal having the smallest frame number is output from the delay buffer unit, and put in an output buffer (14). When a frame output from the delay buffer unit has packet loss, a speech waveform interpolation processing unit (17) cuts waveforms of a pitch length out of a frame waveform from the output buffer (14) and a frame waveform from the delay buffer unit (19), arranges them one after the other to generate two frame waveforms, and generates an interpolated speech signal from them. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention provides an audio signal with stable quality on the receiving side when transmitting an acoustic signal (hereinafter collectively referred to as an audio signal) such as digitized voice or music via a packet communication network such as the Internet. The present invention relates to an audio signal receiving apparatus for reproducing an audio signal, an audio packet loss compensation method used therefor, a program for implementing the method, and a recording medium on which the program is recorded.

  Services that transmit voice signals using Voice over IP technology are becoming popular. As shown in FIG. 1, an input voice signal is divided into voice packets by the voice signal transmitting apparatus 20 every predetermined time called a frame (for example, 10 milliseconds to 20 milliseconds) and converted into voice packets. When transmitting to the receiving apparatus 10 in real time, depending on the state of the communication network, packet loss occurs in the middle of the communication path, which causes a problem of quality degradation such that the reproduced voice is interrupted. In particular, in the case of a communication service called “best effort” such as the Internet, this problem is particularly remarkable when the communication network is congested.

  Therefore, when communicating voice signals over a packet communication network, using a technique called packet loss compensation, if the packet does not reach the receiving side within the time limit due to loss or delay in the communication path. In the case of packet loss (hereinafter collectively referred to as packet loss), a method of estimating and compensating for an audio signal in a section corresponding to a packet that has not arrived (hereinafter referred to as lost packet) is used. Non-Patent Document 1 is known as a typical method of packet loss compensation processing.

  FIG. 2 is a configuration example of the audio signal receiving apparatus 10 used in Non-Patent Document 1. In the following, for the sake of simplicity, it is assumed that the audio signal transmitting apparatus inserts audio signal code data for each frame into one packet and transmits it. A series of voice packets is sequentially received by the packet receiving unit 11, also called a fluctuation absorbing buffer, and stored in the reception buffer 12.

  Under the control of the control unit 16, the voice packets accumulated from the reception buffer 12 are extracted in ascending order of frame numbers. The extracted voice packet is sent to the voice waveform decoding unit 13, decoded into a digital voice signal, and output. The output digital audio signal is stored in the output audio buffer 14 for a predetermined time (number of frames). In the following description, the audio signal decoded and output by the audio waveform decoding unit is a digital audio signal including the case of the present invention. Therefore, in the following description, the audio signal is simply an audio signal regardless of the digital signal. Called a signal.

When the voice packet having the frame number to be taken out is not accumulated at the stage of taking out the voice packet from the reception buffer 12, the packet loss determination unit 15
Determines that a packet loss has occurred in the frame and informs the control unit 16 of the packet loss.
When the control unit 16 receives that the packet loss has occurred, the control unit 16 sets the switch SW to the B side. The voice waveform interpolation processing unit 17 uses the voice signal in the output voice buffer 14 to generate a voice signal of a frame in which packet loss has occurred by interpolation processing, and outputs it through the switch SW.

FIG. 3 conceptually shows an interpolation process for generating an output audio signal in the audio waveform interpolation processing unit 17. A waveform 3A that is traced back from the last sample point S _E of the audio signal in the output audio buffer 14 by a length corresponding to the basic period of the audio, which is called a pitch length, is copied, and the waveform is a diagram in the audio waveform interpolation processing unit 17. A waveform 3B, 3C, and 3D are sequentially arranged and pasted on a loss frame buffer not shown. The synthesized waveform audio signal is output as an audio signal of the current frame (loss frame) through the switch SW. In order to prevent the waveform connecting portion from being discontinuous, a method may be used in which the waveform 3A is slightly longer than one pitch length and copied while overlapping a part of the waveform.

The problem of the method of Non-Patent Document 1 (FIG. 3) is that a transient part where the characteristics of the sound fluctuate because waveform interpolation processing (also referred to as unidirectional extrapolation) is performed using only past output signals. For example, when a packet loss occurs in a change portion from a silent (background noise) interval to a voice interval or a change portion from an unvoiced sound to a voiced sound, the sound quality deteriorates.
As a method for solving this problem, the method of Non-Patent Document 2 has been proposed. FIG. 4 shows an example of the configuration of an audio signal receiving apparatus used in Non-Patent Document 2 with the same reference numerals assigned to the same components as those in FIG.

  The audio signal receiving apparatus 10 shown in FIG. 4 has a configuration in which a buffer search unit 18 is added to the apparatus shown in FIG. In the apparatus of FIG. 4, when a packet loss occurs, the buffer search unit 18 searches the reception buffer 12, and a packet having a frame number after the frame in which the packet loss has occurred has arrived in the reception buffer 12. In this case, the packet is decoded, and an audio signal (hereinafter referred to as a prefetch waveform) obtained by decoding is sent to the audio waveform interpolation processing unit 17.

FIG. 5 conceptually shows an interpolation process for generating a speech waveform by the interpolation process by the speech waveform interpolation processing unit 17 in FIG. Similar to the method of Non-Patent Document 1 (FIG. 3), the speech waveform interpolation processing unit 17 copies the waveform 3A of 1 pitch length from the speech signal in the output speech buffer 14 and fills one frame with the waveforms 3B and 3C. , 3D, and the process of copying and pasting the waveform 4A having a pitch length of 1 pitch from the beginning of the pre-read waveform, and arranging and pasting it as waveforms 4B, 4C, and 4D so as to fill one frame backward. The frame waveform 31 and the backward interpolated frame waveform 41 are generated, and one frame waveform is generated by the weighted sum of the two frame waveforms. Such processing is also referred to as longitudinal interpolation. Similar to the description of FIG. 3, it is also possible to use a method in which the waveform 3A or 4A is slightly longer than one pitch length and copied while overlapping a part of the waveform so that the waveform connecting portion does not become discontinuous. Good.
ITU-T Rec. G.711 Appendix I (1999) Omuro, Mori, Hiwasaki, Kurihara, Kataoka, "Improvement of burst packet loss tolerance by parallel transmission of voice features", IEICE Technical Report SP2004-77 (2004)

  The method of Non-Patent Document 2 (FIG. 5) is an excellent method for suppressing the quality deterioration of the transient part when packet loss occurs in the transient part of the audio signal. In general packet voice communication terminals, the reception buffer and the packet loss compensation processing are often implemented in different processors. However, on the hardware in which the reception buffer and the packet loss compensation are implemented in different processors in FIG. 4, the reception buffer processing and the packet loss compensation processing, particularly the search processing by the reception buffer search unit 18, are densely performed for the same reception buffer 12. The realization of the method of Non-Patent Document 2 in conjunction with the above requires a very complicated mounting and is substantially difficult to realize.

  An object of the present invention is to provide a voice signal receiving apparatus, a voice packet loss compensation method, a program for the method, and a recording medium on which the program is recorded.

The audio signal receiving apparatus according to the present invention is
A reception buffer for temporarily storing received voice packets;
Decoding means for extracting voice packets from the reception buffer in ascending order of frame numbers and decoding voice codes in the voice packets to obtain voice signals;
Packet loss determination means for determining whether or not a voice packet to be extracted is accumulated in the reception buffer, and generating a determination result as a packet loss flag indicating whether or not a packet loss has occurred;
Delay buffer means for accumulating the decoded audio signal up to a specified number of frames and outputting the audio signal in ascending order of frame number;
Output audio buffer means for storing output audio signals for a predetermined time or number of frames;
When a frame to be output from the delay buffer means is a packet loss, a first interpolated waveform is generated by copying the waveform from the output audio signal in the output audio buffer and searched from within the delay buffer means A second interpolation waveform is generated by a waveform copied from the audio signal having a frame number after the frame of the packet loss, and the interpolated audio signal is generated using the first interpolation waveform and the second interpolation waveform. Voice waveform interpolation processing means for generating an output voice corresponding to a packet loss frame, and
If the frame to be output from the delay buffer means is not a packet loss, the audio signal from the delay buffer means is output as an output audio signal, and if it is a packet loss, the interpolated audio signal from the audio waveform interpolation processing means is output. Output control means for outputting as an audio signal;
Are included.

Also, the voice packet loss compensation method according to the present invention provides:
(a) temporarily storing received voice packets in a reception buffer;
(b) extracting voice packets from the reception buffer in ascending order of frame numbers and decoding voice codes in the voice packets to obtain voice signals;
(c) determining whether a voice packet to be extracted is stored in the reception buffer, and generating a determination result as a packet loss flag indicating whether a packet loss has occurred;
(d) storing the decoded audio signal in the delay buffer means up to a specified number of frames, and outputting the audio signals from the delay buffer means in ascending order of frame numbers;
(e) accumulating an output audio signal for a predetermined time or number of frames in the output audio buffer means;
(f) When a frame to be output from the delay buffer means is a packet loss, a first interpolation waveform is generated by copying a waveform from the output audio signal in the output audio buffer, and the delay buffer means A second interpolation waveform is generated from a waveform copied from an audio signal having a frame number later than the frame of the packet loss searched from within, and the first interpolation waveform and the second interpolation waveform are used. Generating an interpolated audio signal and making it an output audio corresponding to a packet loss frame;
(g) If the frame to be output from the delay buffer means is not a packet loss, the audio signal from the delay buffer means is output as an output audio signal, and if it is a packet loss, the interpolated audio from the audio waveform interpolation processing means is output. Outputting a signal as an output audio signal;
Including.

According to the present invention, the decoded audio signal is held and output for the number of frames designated in the delay buffer means, and for the packet loss frame, the audio signal after the loss frame in the delay buffer means is pre-read to interpolate the waveform. Therefore, even when the reception buffer and the packet loss compensation are implemented in different processors, it is possible to realize the packet loss compensation with little deterioration in sound quality in the transitional part with the minimum increase in delay. As a result, a packet voice communication terminal with low cost and high call quality can be easily realized.
The present invention can be executed as a computer main body and a computer program, or can be realized by being mounted on a digital signal processor or a dedicated LSI.

[Example 1]
FIG. 6 is a configuration example of an audio signal receiving apparatus according to the present invention. Components common to the apparatus in FIGS. 2 and 4 are given the same reference numerals. The voice signal receiving apparatus according to the present invention includes a packet receiver 11, a reception buffer 12, a voice waveform decoder 13, an output voice buffer 14, a packet loss determiner 15, a controller 16, a voice waveform interpolation processor 17, and a buffer searcher 18. Is provided in the same manner as in FIG. In the embodiment of FIG. 6, a delay buffer unit 19 and an output control unit 21 are further added. The output audio buffer 14, the audio waveform interpolation processing unit 17, the buffer search unit 18, the delay buffer unit 19, and the switch SW constitute a packet loss compensation processing unit 100.

  As described in FIG. 2, the voice packet is received by the packet receiver 11 and sent to the reception buffer 12. Under the control of the control unit 16, voice packets stored in ascending order of frame numbers are extracted from the reception buffer 12. The extracted voice packet is sent to the voice waveform decoding unit 13, decoded into a voice signal, and sent to the delay buffer unit 19 in the present invention. Here, the decoded speech signal includes not only a speech waveform signal in a narrow sense, that is, a speech waveform signal typified by the PCM format, but also speech parameters (for example, pitch, power, frame number information, etc.) obtained at the time of speech waveform decoding. May also be included.

The packet loss determination unit 15 determines whether or not a voice packet having a frame number to be extracted is accumulated at the stage where the voice packet is extracted from the reception buffer, and the determination result indicates whether or not a packet loss has occurred in the frame. _Is given to the control unit 16 as a packet loss flag _FPL . For example, when F _PL = 1, it is assumed that packet loss has occurred.

When receiving the packet loss flag _FPL , the control unit 16 _{notifies the} delay buffer unit 19 of it. Alternatively, the packet loss flag _FPL may be directly supplied from the packet loss determination unit 15 to the delay buffer unit 19. Each time the statistical value calculation unit 16A of the control unit 16 receives the packet loss flag _FPL from the packet loss determination unit 15, the statistical value calculation unit 16A statistically obtains the frequency and pattern of packet loss as the statistical value of packet loss, and stores it in the delay buffer unit 19. The number of frames N _F to be held is determined and supplied to the delay buffer unit 19. Regarding the relationship between the frequency and pattern of packet loss and the number of required frames N _F, a rule (correspondence table) is created in advance and stored in the table memory 16B, and the number of required frames N _F is determined by referring to it. To do.

The packet loss rate is used as the frequency of packet loss. For example, the statistical value calculation unit 16A calculates the packet loss rate e (k) in the frame k for each frame.
e (k) = e (k-1) x 0.99 + 0.01, packet loss (F _PL = 1) (1)
e (k) = e (k-1) x 0.99, no packet loss (F _PL = 0) (2)
Calculate according to k is a frame number k = 1, 2, 3,..., and an initial value of e (k) is set to e (0) = 0. Alternatively, the packet loss flag F _PL is accumulated in a buffer (not shown) in the statistical value calculation unit 16A for a predetermined time, and the packet loss rate e (k) is stored in the buffer from the packet loss flag F _PL in the buffer for each frame. may be calculated as the ratio of F _PL = 1 the number and F _PL of the total number (number of frames).

To express the packet loss pattern as a numerical value, the continuous packet loss rate is calculated. The n consecutive packet loss rate (n is an integer of 1 or more) is defined as a rate of packet loss continuously for n frames or more, and the n continuous packet loss rate in the kth frame is expressed as en (k). Here, the consecutive packet loss rates of ei (k), i = 1, 2, ..., n are calculated, and based on them, the number of required frames _NF is determined by referring to the table as described later. To do. A method of calculating ei (k) will be described below with reference to the flowchart of FIG.

First, the initial value of the continuous number parameter r is set to 0 (step S1). Next, it is determined whether the packet loss is caused by whether the packet loss flag _FPL is 1 or 0 (step S2). If it is a packet loss, the value of r is incremented by 1 in step S3, and the process returns to step S2. If it is determined in step S2 that there is no packet loss, ei (k), i = 1, 2,..., N is calculated as follows in step S4.
For ei (k) of r <i
ei (k) ＝ ei (k-1) × 0.99 (3)
Is calculated r + 1 times.
For ei (k) of r ≧ i
ei (k) = ei (k-1) x 0.99 + 0.01 (4)
Is calculated r times, and
ei (k) ＝ ei (k-1) × 0.99 (5)
Is calculated once. Note that e1 (k) in the case of i = 1 is the same as e (k). Similarly to the calculation of e (k), the calculation of ei (k) accumulates the packet loss flag F _PL in the buffer for a certain period of time, and continuously packet loss from the packet loss flag F _PL in the buffer for each frame. The rate may be calculated. The constants 0.99 and 0.01 in the formulas (1) to (5) are determined experimentally as appropriate, and may be combinations of 0.95 and 0.05, 0.995 and 0.005, respectively, for example. Also, a constant set of 0.95 and 0.05 is used while the frame number k is small, that is, shortly after the start of voice packet reception, and a set of 0.99 and 0.01 is used after a certain period of time. In addition, the constant may be changed on the way.

FIG. 8 and FIG. 9A show examples of rules for determining the number of required frames N _F by e (k) and ei (k), i = 2, 3, and 4, respectively. These tables are stored in advance in the table memory 16B of the control unit 16. The number of required frames N _F may be determined based only on e (k) shown in FIG. 8, or may be determined based on e (k) in FIG. 8 and ei (k) in FIG. 9A. In the latter case, when there are a plurality of corresponding items in FIG. 8 and FIG. 9A, the largest value among the corresponding items is set as the number of required holding frames N _F. 8 and 9A may be changed to other values, and FIG. 9B may be used instead of FIG. 9A. Generally, if the threshold value is set large, the number of required frames is set small, and the call delay due to the delay buffer unit 19 is reduced. However, quality degradation at the time of packet loss increases, and conversely, if the threshold value is set small, the number of required frames required Although it is set large and quality deterioration at the time of packet loss is suppressed, since the call delay by the delay buffer unit 19 becomes large, there is a trade-off relationship. Therefore, the threshold value should be adjusted in an environment where packet loss actually occurs so that it becomes as large as possible within a range where quality degradation is not noticeable.

The delay buffer unit 19 is a shift buffer or a ring buffer that internally holds decoded audio signals for the number of required frames N _F specified by the control unit 16. The delay buffer unit 19 holds the audio signal received from the audio waveform decoding unit 13 and outputs the audio signal having the smallest frame number among the held audio signals. However, when a packet loss has occurred from the control unit 16 (F _PL = 1), instead of receiving the decoded audio signal from the audio waveform decoding unit 13 and storing it in the delay buffer unit 19, the frame is a packet loss. Stores information to that effect. As the packet loss information, for example, a packet loss flag _FPL and frame number information are used. When the number of required frames N _F specified by the control unit 16 is 0, since the audio signal is received from the audio waveform decoding unit 13 and output at the same time, no delay occurs.

  The audio signal output from the delay buffer unit 19 is sent to the A terminal of the switch SW. When the output control unit 21 outputs the audio signal having the smallest frame number from the delay buffer unit 19, if the frame is not a packet loss, the output control unit 21 sets the switch SW to the A side, that is, the delay buffer unit side. When information indicating that the frame output from the delay buffer unit 19 is a packet loss instead of the decoded audio signal is stored in the frame, the switch SW is set to the B side, that is, the audio waveform interpolation processing unit 17 side. . The audio signal is output through the switch SW and written to the output audio buffer 14.

  The output audio buffer 14 accumulates the output audio signal sent from the switch SW for a predetermined time (number of frames), and sends the accumulated audio signal to the audio waveform interpolation unit 17. The buffer search unit 18 searches the delay buffer unit 19 when the frame output from the delay buffer unit 19 is a packet loss, and a frame number (large frame number) after the frame in which the packet loss has occurred. Are stored in the delay buffer unit 19, the waveform (prefetch waveform) obtained by the search is sent to the speech waveform interpolation processing unit 17.

  Similar to the method described with reference to FIG. 5, the speech waveform interpolation processing unit 17 performs processing to cut out and arrange one pitch waveform from the speech signal in the output speech buffer 14, and 1 from the head of the prefetched waveform from the delay buffer unit 19. Two frame waveforms are generated by cutting out the pitch waveform and arranging the waveforms backward, and one frame waveform is generated as an interpolated speech signal by the weighted sum of them. Similarly to the description of FIG. 5, a method may be used in which a waveform to be cut out is slightly longer than one pitch length and copied while overlapping a part of the waveform so that the waveform connecting portion does not become discontinuous. The generated interpolated audio signal is output as an output audio signal from the B terminal of the switch SW and written to the output audio buffer 14.

The voice packet loss compensation method implemented in the above-described voice signal receiving apparatus of FIG. 6 will be described with reference to the flowchart of FIG.
Step S1: It is determined whether a voice packet to be taken out is stored in the reception buffer 12, and a packet loss flag _FPL is generated.
Step S2: The control unit 16 based on the packet loss flag F _PL consecutive packet loss rate ei (k), i = 1 , 2, ..., to calculate the n.
Step S3: The control unit 16 determines the number of required frames N _F from the continuous packet loss rate with reference to the table of the table memory 16B (FIGS. 8 and 9A).

Step S4: It is determined whether the packet loss flag _FPL indicates a packet loss.
Step S5: If it is determined in step S4 that there is no packet loss, the voice packet extracted from the reception buffer 12 is decoded to obtain a voice signal, and the decoded voice signal is delayed under the condition of the number of required frames N _F Write to the buffer. Whether or not the decoded audio signal is actually written to the delay buffer unit 19 under the restriction of N _F will be described in the embodiments described later.
Step S6: If it is determined in step S4 that there is a packet loss, packet loss information is written in the delay buffer unit 19.
Step S7: It is determined whether the frame with the smallest number in the delay buffer unit 19 is a packet loss.

Step S8: If it is determined in step S7 that there is no packet loss, the read audio signal is output from the audio signal receiving device, written to the output audio buffer 14, and the process returns to step S1.
Step S9: If it is determined in step S7 that there is a packet loss, the buffer search unit 18 determines whether a frame with a number greater than the frame number of the packet loss in the delay buffer unit 19 is a packet loss. The determination is repeated in ascending order of frame number until a frame is found.
Step S10: Using the audio signal of the frame obtained by the search in step S9 and the audio signal of the previous frame held in the output audio buffer 14, the audio waveform interpolation processing unit 17 converts the interpolated audio signal by the interpolation processing described above. It is generated and output via the switch SW and written to the output audio buffer 14, and the process returns to step S1.

As described above, in the present invention, the delay buffer unit 19 that holds the decoded audio signal is provided, and the pre-read waveform for use in the interpolation processing is obtained by searching the delay buffer unit 19. It is not necessary to consider the timing of the 12 writing and reading processes, and the packet loss compensation process by the interpolation process can be easily performed.
[Example 2]

In the configuration of FIG. 6, the delay buffer unit 19 retains the decoded speech signal of a main holding frame number N _F content designated by the control unit 16 therein, a main holding frame number N _F designated by the control unit 16 Is changed in the middle, the delay buffer unit 19 transitions the internal state of the delay buffer unit 19 so as to hold the updated decoded speech signal of the required number of frames N _F inside. For example, when the number of required frames N _F decreases, the decoded speech signal in the silence period or the non-speech period is prohibited from being taken into the delay buffer unit 19 and discarded, thereby being held in the delay buffer unit 19. Reduce the number of frames. When the number of required frames N _F increases, the number of frames to be held in the delay buffer unit 19 is obtained by taking the same decoded audio signal into the delay buffer unit 19 a plurality of times during a silence period or a non-speech period. increase.

A configuration example of such a delay buffer unit 19 is shown in FIG. 11, and an operation flow thereof is shown in FIG. The delay buffer unit 19 includes a frame buffer 19A, a comparison unit 19B, a silent period detection unit 19C, and a write control unit 19D. For simplicity in the following, the initial value N _F0 of a main holding frame number N _F is the predetermined value, until the number of frames in the frame buffer 19A m is m = N _F0 in packet loss from the decoding start does not occur It is assumed that the decoded audio signal _SD is captured.
The comparison unit 19B receives the number of required frames N _F from the control unit 16 (step S1), compares it with the number m of frames in the frame buffer 19A given from the write control unit 19D, and gives the comparison result to the write control unit 19D. (Step S2).

The silent section detector 19C determines whether or not the decoded speech signal _SD given from the speech waveform decoder 13 is a silent section signal (steps S3 and S6), and gives the determination result to the write controller 19D. . If m <N _F and the decoded audio signal _SD is in the silent section, the write control unit 19D takes the decoded audio signal _SD into the frame buffer 19A and increases the value of m by 1 (step S4). ), The process proceeds to step S5. If it is determined in step S3 that the decoded speech signal _SD is not in the silent section, the process proceeds to step S5 as it is. In the case of m = N _F in step S2 proceeds to a step S5.

In step S5, the writing control unit 19D takes the supplied decoded audio signal _SD into the frame buffer 19A, and returns to step S1. Since the frame buffer 19A outputs the audio signal of the frame with the smallest number when the audio signal for one frame is taken in, there is no increase / decrease in the value of m by the process of step S5.

If m> N _F , if it is determined in step S6 that it is not a silent section, step S5 is executed. If it is determined in step S6 that there is a silent section, the writing control unit 19D prohibits and discards the given decoded speech signal _{SD in} that silent section, and the frame with the smallest number from the frame buffer 19A. Therefore, the number m of frames in the frame buffer 19A is decreased by 1, so the value of m is decreased by 1 (step S7) and the process returns to step S1.
Thus, by performing the processing in FIG. 12 for each frame, m = N _F and so as to gradually frame number m is changed.

As a method for detecting a silence interval by the silence interval detector 19C, for example, when the power, which is one of the audio parameters included in the decoded audio signal, is smaller than a predetermined threshold value, the silence interval is determined. When steps S4 and S5 are executed, the same decoded audio signal _SD that has been given is fetched twice into the frame buffer 19A. Instead of fetching the decoded speech signal _SD into the frame buffer 19A in step S4, for example, a predetermined speech waveform of a silent period or a non-speech period is stored in the waveform memory 19E as shown by a broken line in FIG. Then, the audio signal having the audio waveform may be taken into the frame buffer 19A.
[Example 3]

In FIG. 13, as described above, the decoded speech signal _SD of the k-th frame is a narrowly defined speech waveform signal, that is, a speech waveform signal S _PCM typified by the PCM format, and a relative position j that is a frame number difference. This is an example in which the audio parameter P _kj of the kth frame is set. That is, on the voice signal transmitting apparatus (see FIG. 1) side, the code of the PCM format voice waveform signal S _PCM of the kth frame is set to the code of the voice parameter P _kj of the kjth frame before j frames, and the voice packet is set. Insert and send.

If only the kth frame speech waveform signal code and the kth frame speech parameter code are inserted into the kth frame packet, if the kth frame packet cannot be received, the kth frame speech information is received. Therefore, even if an audio signal is generated from an audio signal of an adjacent frame by interpolation processing, a high-quality audio signal cannot be obtained. On the other hand, as shown in FIG. 13, if the speech parameter P _kj of the kjth frame is added to the speech waveform signal S _PCM of the kth frame, the output control unit 21 tries to output from the delay buffer unit 19. when the k-th frame is detected to be a packet loss, in the speech parameter P _k of the k-th frame included in the audio signal S _D of the k + j frame held in the delay buffer 19 Using the pitch length, for example, a 1-pitch length waveform is cut out from the previous and next frame waveforms described in FIG. 5, and the waveform is generated by interpolation processing. If necessary, the power in the audio parameter P _k is used for interpolation. This is because by correcting the power of the waveform generated by the processing, it is possible to obtain a relatively high-quality audio output even if a packet loss occurs in the kth frame.

Number main holding frame designated by the control unit 16 to the delay buffer unit 19 N _F is changed in the middle, the audio signal of the main holding frame number N _F of the delay buffer 19 is updated by the processing described with reference to FIG. 12 described above When the frame waveform is discarded / inserted in order to hold the signal inside, if the target is only the PCM speech waveform signal S _PCM , the processing necessary for the discard / insertion does not occur. as such, the PCM audio waveform signal S _PCM k th frame, when the speech parameters P _kj frame number kj is in sets, the discard / insertion process of the frame waveform, PCM audio waveform signal S _PCM and voice The frame relative positional relationship of the parameter P _kj will shift. For such an assembly, the delay buffer unit 19 needs to perform a process of correcting the relative positional relationship of the audio parameters together with the discarding / inserting process of the frame waveform.

FIG. 14 shows the configuration of the delay buffer unit 19 for that purpose, and FIG. 15 shows the processing flow. The delay buffer unit 19 shown in FIG. 14 has a configuration in which a relative position correcting unit 19D1 is provided in the write control unit 19D in the delay buffer unit 19 shown in FIG. When inserting or deleting the decoded audio signal _SD from the audio waveform decoding unit 13, the relative position j included in the audio signal of j frames or j-1 frames following the decoded audio signal by the relative position correcting unit 19D1. Are corrected to j + 1 or j-1, respectively.

Processing flow shown in FIG. 15, the correction processing of the relative position j in step S4 and S7 in the process flow shown in FIG. 12 has been added. For m <N _F, the decoded speech signal is determined to be silent section at step S3, the relative position j respectively included in the audio signal of the j frame following the decoded speech signal Te step S4 smell Then, the decoded audio signal is taken into the frame buffer 19A and m is increased by 1. In step S7, writing of the decoded audio signal is prohibited, m is subtracted by 1, and the relative position j included in each of j −1 frames following the decoded audio signal is corrected to j−1. Other processes are the same as those in FIG.

In the description of FIGS. 13, 14, and 15, the relative position j has been described as meaning that the sound parameter of the kth frame is stored in the k + j frame after the jth frame. Can be defined as being stored in the kj frame before j frames. In this case, for example, when an audio signal is inserted in step S4, j is corrected to j + 1 for the latest j frames in the frame buffer 19A, and writing is prohibited in step S7. The j may be corrected to j-1 for the latest j -1 frames in the frame buffer 19A .
[Example 4]

  FIG. 16 shows a configuration example in which the speech signal receiving apparatus of the present invention is applied to band division coding represented by wideband speech coding. The packet loss compensation processing unit 100 shown in FIG. 6 has two low-band packet loss compensation processing units 100L and a high-band packet loss compensation processing unit 100H. These components are indicated by adding a symbol L to the low frequency side and a symbol H to the high frequency side, respectively.

The speech waveform decoding unit 13 decodes the speech packet extracted from the reception buffer 12 and outputs a high frequency audio signal _SDH and a low frequency audio signal _SDL , respectively. The high frequency audio signal S _DH is sent to the delay buffer unit 19H of the high frequency packet loss compensation processing unit 100H. Low-band speech signal S _DL is sent to the delay buffer portion 19L of the low-band packet loss compensation processing unit 100L.

  The processes of the high frequency packet loss compensation processing unit 100H and the low frequency packet loss compensation processing unit 100L are the same as those in FIGS. However, the interpolation processing of the high frequency speech waveform interpolation processing unit 17H may be a method in which a frame length waveform is simply regarded as a pitch unit waveform and used for interpolation as it is, instead of a method of cutting out and arranging in units of pitch. The audio signals output from the high frequency packet loss compensation processing unit 100H and the low frequency packet loss compensation processing unit 100L are subjected to band synthesis by the band synthesis unit 22 and output as output audio.

When the control unit 16 receives from the packet loss determination unit 15 that a packet loss has occurred (that is, the packet loss flag F _PL = 1), the control unit 16 notifies the delay buffer units 19H and 19L to that effect. Similarly to the case of FIG. 6, the control unit 16 statistically obtains the frequency and pattern of packet loss from the packet loss information (packet loss flag F _PL ) received from the packet loss determination unit 15, and the delay buffer unit 19 H , to determine the frame number N _F to be held in a 19L, instructs the main holding frame number N _F highband packet loss compensation processing unit 100H and a low packet loss compensation processing section 100L of the delay buffer unit 19H, the 19L .

The delay buffer units 19H and 19L hold the decoded audio signal having the number of required frames N _F designated by the control unit 16 inside. When the required number of required frames N _F specified by the control unit 16 changes halfway, the delay buffers 19H and 19L delay so that the updated decoded speech signal of the required number of required frames N _F is held inside. Buffer unit 19H. The frame holding state in 19L is transitioned. That is, similar to the configuration of FIG. 6, for example, in the case when the reduced the main holding frame number N _F is that discards the frame waveform silence section or non-speech section, have increased main holding frame number N _F is A frame waveform corresponding to silence may be inserted in a silent section or a non-voice section.
[Example 5]

As understood from the description of FIG. 12, in the configuration of FIG. 16, from the instruction to change the number of required frames N _F by the control unit 16, both of the instructed number of frames N _F are sent to both delay buffer units 19H and 19L. A time lag often occurs until the audio signal is actually accumulated. This is because, in the method of deleting and inserting a waveform in a silent section, it is necessary to wait until the silent section is reached after receiving an instruction. Even in such a case, increase / decrease of the actual delay amount of both the delay buffer units 19H and 19L of the high frequency packet loss compensation processing unit 100H and the low frequency packet loss compensation processing unit 100L, that is, increase / decrease of the number of retained frames must be synchronized. Control.

For example, in the low frequency region, after the control unit 16 instructs the delay buffer unit 19L to change the number of required frames N _F , the time until the low frequency frame count _ML actually changes to the command value N _F is longer. In many cases, the control unit 16 first instructs the low-frequency delay buffer unit 19L about the number N _F of necessary holding frames, and the actual holding frame number m _L of the low-frequency delay buffer unit 19L is shown in FIG. As shown by the broken line, it is transmitted to the high-frequency delay buffer unit 19H, and the high-frequency decoded speech signal _SDH is deleted or captured multiple times regardless of whether or not the high-frequency decoded speech signal _SDH is in a silent period. the frame number m _H holding the delay buffer portion 19H of the frequency forcibly may be matched to the current frame number m _L of the low band.
[Example 6]

  FIG. 17 shows a configuration example of the speech waveform interpolation processing unit 17 (17H, 17L) in FIG. 6 (and FIG. 16). In this example, the speech waveform interpolation processing unit 17 includes a pitch extraction unit 17A, a forward waveform generation unit 17B, a backward waveform generation unit 17C, and a weighting addition unit 17D. The pitch extraction unit 17A analyzes the audio signal in the output audio buffer 14 (14H, 14L), determines the pitch length corresponding to the basic period of the audio, and sets the pitch length to the forward waveform generation unit 17B and the backward waveform generation unit. Send to 17C. The pitch length is usually expressed by the number of samples, and in the case of 8 kHz sampling, it is often a value of about 20 to 140. As described with reference to FIG. 5, the forward waveform generation unit 17B copies the waveform having the pitch length from the last sample point of the audio signal in the output audio buffer 14 and pastes the waveforms in order in the loss frame buffer.

As described with reference to FIG. 5, the backward waveform generation unit 17 </ b> C also performs a process of copying the 1-pitch waveform from the beginning of the prefetch waveform and arranging the waveforms in the backward direction in the current frame buffer. The weighting addition unit 17D creates an interpolation waveform for one frame by weighting and adding the waveform generated by the forward waveform generation unit 17B and the waveform generated by the backward waveform generation unit 17C, Output as an interpolated audio signal.
[Modification]

In FIG. 17, the audio signal held in the output audio buffer 14 is analyzed to extract the pitch length, and the audio waveform is copied and pasted with the pitch length to generate the interpolated audio signal. In addition, when the decoded speech signal _{SD of} each frame is a speech waveform signal in a narrow sense, that is, a speech waveform signal typified by the PCM format, and a speech parameter, the pitch included in the speech parameter Parameters may be used. Here, unlike the example of FIG. 13, as shown in FIG. 18, when the audio signal _{SD of} each frame is k as the current frame number, the audio waveform signal S _PCM of the kth frame and It is assumed that the audio parameter P _k is set.

  FIG. 19 is a configuration example of the speech waveform interpolation processing unit 17 (17H, 17L) when the speech signal of FIG. 18 is used. In this case, audio parameters are set for both the audio signal of each frame in the delay buffer unit 19 and the audio signal in the output audio buffer 14. The pitch length is acquired by the pitch parameter acquisition unit 17E from the audio parameters set in the audio signal of the pre-read waveform read by the search in the delay buffer unit 19, and the obtained pitch length is sent to the backward waveform generation unit 17C. It is done. The pitch length is acquired by the pitch parameter acquisition unit 17F from the audio parameters set in the audio signal in the output audio buffer 14, and the obtained pitch length is sent to the forward waveform generation unit 17B. Other processes are the same as those in FIG.

  When the method of FIG. 17 is compared with the method of FIG. 19, an interpolated waveform with less quality degradation is obtained in FIG. Further, FIG. 19 has an advantage that the processing amount on the receiving side is small.

The speech waveform interpolation processing unit of FIG. 19 shows a configuration example in the case of using a speech signal composed of a speech waveform signal of the same frame number and a speech parameter set as shown in FIG. 18, but as shown in FIG. It is also possible to use a decoded speech signal _SD whose speech waveform signal frame number and speech parameter frame number are different. In that case, in FIG. 19, the pitch parameter acquisition unit 17F is not provided, and when the k-th frame to be output from the delay buffer unit 19 is a loss frame, the pitch parameter acquisition unit 17E is held in the delay buffer unit 19. The pitch length is acquired from the speech parameter P _k of the kth frame included in the decoded speech signal of the k + j frame and is given to the backward waveform generation unit 17C, and the same pitch length is forwarded as indicated by a broken line. This is also given to the waveform generator 17B.

  The audio signal receiving apparatus according to the present invention described above may be configured to execute the processing by executing a program on a computer. Further, the voice packet loss compensation method according to the present invention may be stored in a recording medium as a computer-executable program, and the program of the recording medium may be executed by the computer.

  Usage forms for performing voice communication on a packet communication network have become widespread, and by applying the present invention, voice communication with low cost and high reliability can be realized.

The figure which shows the example of the system which packetizes an audio | voice signal and communicates. The block diagram which shows the structural example of the conventional audio | voice signal receiver. FIG. 3 is a waveform diagram conceptually showing speech waveform interpolation processing in FIG. 2. The block diagram which shows the other structural example of the conventional audio | voice signal receiver. FIG. 5 is a waveform diagram conceptually showing speech waveform interpolation processing in FIG. 4. The block diagram which shows the structural example of the audio | voice signal receiver by this invention. The processing flow figure which performs calculation of n continuous packet loss rate. The table | surface which shows the example of the number of required frames with respect to a packet loss rate. A is an example of a table indicating the number of required frames for the continuous packet loss rate, and B is another example of the table. The processing flow figure of voice packet loss compensation. The block diagram which shows the structural example of the delay buffer part which performs frame waveform deletion / insertion. FIG. 12 is a flowchart for performing frame waveform deletion / insertion processing in FIG. 11. The figure which shows the example of the audio | voice signal by which the audio | voice parameter was set to the PCM audio | voice waveform signal. The block diagram which shows the structural example of the delay buffer part which corrects the flame | frame relative position of an audio | voice parameter. FIG. 15 is a process flow diagram for correcting the frame relative position of the audio parameter in FIG. 14. The block diagram which shows the other structural example of the audio | voice signal receiver by this invention. The block diagram which shows the structural example of a speech waveform interpolation process part. The figure which shows the other example of the audio | voice signal by which the audio | voice parameter was set to the PCM audio | voice waveform signal. The block diagram which shows the other structural example of a speech waveform interpolation process part.

Claims (15)

A reception buffer for temporarily storing received voice packets;
A voice packet is extracted from the reception buffer in ascending order of the frame number, the voice code in the voice packet is decoded, and the PCM format voice waveform signal S _{PCM of} the k-th frame (k is the frame number) is the frame number difference. Decoding means for obtaining an audio signal in which the relative position j and the audio parameter P _kj of the kth frame are set ;
Packet loss determination means for determining whether or not a voice packet to be extracted is accumulated in the reception buffer, and generating a determination result as a packet loss flag indicating whether or not a packet loss has occurred ;
Determining the number of frames to be held in the delay buffer means (hereinafter referred to as the number of required frames), and determining the number of required frames to be given to the delay buffer means;
If the number of frames held in the delay buffer means is larger than the number of required frames, the decoded audio signal is discarded and the audio signals stored in the frame buffer are output in ascending order of frame numbers. When the number of frames held in the delay buffer means is smaller than the number of required frames, the decoded audio signal is accumulated twice in the frame buffer and the audio signal accumulated in the frame buffer is framed. If the number of frames output in ascending order and the number of frames held in the delay buffer means is equal to the number of required frames, the decoded audio signal is stored in the frame buffer and stored in the frame buffer. a delay buffer unit which output the audio signals in ascending order of frame numbers,
An output speech buffer means for accumulation by a predetermined time or number of frames the output audio signal,
When the frame to be output from the delay buffer means is a packet loss, the audio waveform signal of the output audio signal in the output audio buffer and the frame after the frame that is the packet loss in the delay buffer means Voice waveform interpolation processing means for generating an interpolated voice signal using a voice waveform signal and a voice parameter corresponding to the voice waveform signal of the frame that is the packet loss in the delay buffer means ;
If the frame to be output from the delay buffer means is not a packet loss, the audio signal from the delay buffer means is output as an output audio signal, and if it is a packet loss, the interpolated audio signal from the audio waveform interpolation processing means is output. An audio signal receiving device including output control means for outputting as an audio signal,
The delay buffer means includes
If the number of frames held in the delay buffer means is larger than the number of required frames, the value of the relative position j included in the j-1 audio signals following the discarded frame is corrected to j-1. If the number of frames held in the delay buffer means is smaller than the number of required frames, the value of the relative position j included in the audio signal of j frames following the frame is set to j + 1 Relative position correcting means for correcting to
Audio signal receiving apparatus which comprises and. The audio signal receiving device according to claim 1 ,
Given above SL packet loss flag, further comprising a packet loss statistics calculation hand stage for obtaining the statistical values of packet loss,
The required frame number determining means includes:
Based on the packet loss statistics, the delay buffer means the number of frames to be retained within the determined (hereinafter main holding frame number), the audio signal receiving apparatus, wherein the Turkey applied to the delay buffer means. The audio signal receiving apparatus according to claim 2, further comprising a memory means for storing rules defining the number of main support frame for statistics pre Me packet loss,
The audio signal receiving apparatus according to claim 1, wherein the number of required frames determining means determines the number of required frames from the packet loss statistical value with reference to the rules of the memory means. 4. The audio signal receiving device according to claim 2, wherein the delay buffer means includes:
Silent section detecting means for determining whether the decoded speech signal is a speech section of a silent section or a non-speech section ,
Discard or twice accumulation No. sound Koeshin to the audio signal receiving apparatus, wherein the audio signal der Turkey of silence interval or non-speech section in said write control means. In the audio signal receiving apparatus according to claim 1乃Itaru 4,
The decoding means decodes the high frequency audio code and the low frequency audio code obtained from each audio packet, respectively, and outputs a high frequency decoded audio signal and a low frequency decoded audio signal,
The combination of the delay buffer means, the output buffer means, and the speech waveform interpolation processing means is a high-frequency set for processing the high-frequency decoded speech signal given as a decoded speech signal to generate a high-frequency output speech signal. And two sets of low-frequency sets for processing the low-frequency decoded audio signal given as the decoded audio signal to generate a low-frequency output audio signal, and
An audio signal receiving apparatus comprising band synthesizing means for generating an output audio signal by synthesizing the high frequency output audio signal and the low frequency output audio signal. The audio signal receiving apparatus according to claim 5 Symbol mounting, the delay buffer means for the set of the delay buffer means and said low frequency for the high band, the number of frames of the main holding the same value is given, each high frequency decoding Means are provided for controlling so that the actual number of frames held in the high frequency delay buffer means and the low frequency delay buffer means are always synchronized by deleting or inserting the audio signal and the low frequency decoded audio signal. An audio signal receiving apparatus characterized by comprising: The audio signal receiving apparatus according to claim 5 Symbol placement, delay buffer means for the low range by the deletion or insertion of the low band decoding voice signal according to the delay buffer means main holding frame number is given for the low band, different After transitioning to the state of the number of retained frames, the number of retained frames is given to the high frequency delay buffer means, and the high frequency delay buffer means forcibly matches the number of retained frames with the high frequency decoded audio signal. An audio signal receiving apparatus that performs deletion or insertion of In the audio signal receiving apparatus according to claim 1乃optimum 4, the speech waveform interpolation processing means,
A first pitch length is acquired from an audio parameter included in an output audio signal in the output audio buffer means, and a waveform corresponding to the first pitch length is copied from the output audio signal to generate the first interpolation waveform. Forward waveform generating means for
A second pitch length is obtained from a speech parameter included in the speech signal searched from within the delay buffer means, a waveform corresponding to the second pitch length is copied from the searched speech signal, and the second interpolation is performed. A backward waveform generating means for generating a waveform;
Weighted addition means for generating the interpolated audio signal by weighted addition of the first interpolation waveform and the second interpolation waveform;
An audio signal receiving device comprising: In the audio signal receiving apparatus according to claim 1乃optimum 4, the speech waveform interpolation processing means,
The pitch length is acquired from the audio parameter included in the audio signal after j frames specified in advance from the packet loss frame in the delay buffer means, and the waveform corresponding to the pitch length is copied from the searched audio signal. Backward waveform generation means for generating the second interpolation waveform;
Forward waveform generation means for copying the waveform corresponding to the pitch length from the output audio signal in the output audio buffer means to generate the first interpolation waveform;
Weighted addition means for generating the interpolated audio signal by weighted addition of the first interpolation waveform and the second interpolation waveform;
An audio signal receiving device comprising: Voice packet loss compensation method,
(a) temporarily storing received voice packets in a reception buffer;
(b) Voice packets are extracted from the reception buffer in ascending order of frame numbers, the voice codes in the voice packets are decoded , the PCM format voice waveform signal S _{PCM of} the kth frame (k is the frame number), and the frame number Obtaining a sound signal in which a relative position j that is a difference and a sound parameter P _kj of the kth frame are set ;
(c) determining whether a voice packet to be extracted is stored in the reception buffer, and generating a determination result as a packet loss flag indicating whether a packet loss has occurred;
(d) determining the number of frames to be held in the delay buffer means (hereinafter referred to as the number of frames to be held) and giving the delay buffer means;
(e) When the number of frames held in the delay buffer means is larger than the number of required frames, the decoded audio signal is discarded and the audio signal stored in the frame buffer is reduced in frame number When the number of frames output in order and held in the delay buffer means is smaller than the number of required frames, the decoded audio signal is stored twice in the frame buffer and the audio stored in the frame buffer When the signals are output in ascending order of the frame number and the number of frames held in the delay buffer means is equal to the number of required frames, the decoded audio signal is stored in the frame buffer and stored in the frame buffer. and outputting from the delay buffer means the stored audio signals in ascending order of frame numbers,
(f) a step of storing a predetermined time or number of frames by output speech buffer means output audio signal,
(g) When the frame to be output from the delay buffer means is a packet loss, the audio waveform signal of the output audio signal in the output audio buffer and the frame that is the packet loss in the delay buffer means Generating an interpolated speech signal using the speech waveform signal of the frame and a speech parameter corresponding to the speech waveform signal of the frame that is the packet loss in the delay buffer means ;
(h) If the frame to be output from the delay buffer means is not a packet loss, the audio signal from the delay buffer means is output as an output audio signal, and if it is a packet loss, the interpolated audio from the audio waveform interpolation processing means is output. Outputting a signal as an output audio signal;
A voice packet loss compensation method including:
Step (e) above is
If the number of frames held in the delay buffer means is larger than the number of required frames, the value of the relative position j included in the j-1 audio signals following the discarded frame is corrected to j-1. If the number of frames held in the delay buffer means is smaller than the number of required frames, the value of the relative position j included in the audio signal of j frames following the frame is set to j + 1 Steps to fix,
Voice packet loss compensation method which comprises and. Furthermore according to claim 10 Symbol mounting method,
(i) includes steps for obtaining the statistical values of packet loss from the packet loss flag,
Step (d) above is
Based on SL packet loss statistics, the delay buffer means the number of frames to be retained within the determined (hereinafter main holding frame number), the voice packet loss compensation, wherein the Turkey applied to the delay buffer means Method. In claim 11 Symbol mounting method, to determine the number of the main holding frame from step (d) of the packet loss statistics with reference to the rules that govern the number of main support frame for statistics in advance packet loss A characteristic voice packet loss compensation method. 11. was or in 12 Symbol mounting method, the step (e) is
(e- 1) decoded the audio signal comprises steps determines whether the audio signal silence section or non-speech section,
Discard or twice No. sound Koeshin accumulating voice packets loss compensation and wherein the audio signal der Turkey of silence interval or non-speech section in said write control means. A program for implementing the voice packet loss method according to any one of claims 10 to 13 by a computer. It was recorded claim 14 Symbol mounting program, the computer-readable recording medium.

Images