JP5694745B2 - Concealment signal generation apparatus, concealment signal generation method, and concealment signal generation program - Google Patents

Description

  The present invention relates to error concealment when transmitting a voice packet via an IP network or a mobile communication network, and more particularly, a concealment signal generation apparatus and concealment signal generation method for generating a concealment signal for error concealment. And a concealment signal generation program.

  When voice / acoustic signals (hereinafter collectively referred to as “voice signals”) are transmitted in an IP network or mobile communication, the voice signals are encoded and expressed in a small number of bits and divided into voice packets. Is transmitted via the communication network. A voice packet received through the communication network is decoded by a receiving server, MCU, terminal, etc., and a decoded voice signal is obtained.

  When a voice packet is transmitted through a communication network, a phenomenon in which a part of the voice packet is lost or an error occurs in a part of information written in the voice packet due to a congestion state of the communication network (so-called Packet loss) can occur. In such a case, since a voice packet cannot be correctly decoded on the receiving side, a desired decoded voice signal cannot be obtained. Also, since the decoded voice signal corresponding to the voice packet in which the packet loss has occurred is perceived as noise, the subjective quality given to the person to be listened to is significantly impaired.

  As a packet loss concealment method in the frequency domain, there is the following Patent Document 1 regarding “an improved error concealment technique in the frequency domain”. This is because the decoded speech expressed in the frequency domain (Fourier series) included in the packet normally received in the past is accumulated in the buffer, and when packet loss is detected, the optimum is obtained from the decoded speech accumulated in the buffer. This is a technique for restoring the signal corresponding to the packet loss by estimating the gain and multiplying the decoded speech included in the most recently received packet by the optimum gain.

  Further, ITU-T G.711 Appendix I (Non-patent Document 1) is known as an error concealment technique for interpolating a voice / acoustic signal of a portion lost due to packet loss. This is because a part of the decoded speech / acoustic signal received normally is stored in the buffer, and if a packet loss occurs, the waveform is taken out from the buffer in units of pitch and repeated, so that the lost part is restored. Synthesize the corresponding signal.

  As a more advanced error concealment technique, there is the following Patent Document 2 regarding “a concealment signal generation device, a concealment signal generation method, and a concealment signal generation program”. According to the present invention, signal continuity is constantly monitored for signals obtained by decoding packets that have been normally received in the past, and signals in which continuity is recognized (hereinafter referred to as “stationary signals”) are recorded in a buffer. When an error occurs, it is determined whether or not the signal immediately before the error occurs is a stationary signal. If the signal is a stationary signal, the signal copy range of the stationary signal in the buffer indicates the continuity of the signal. This is a method of making a decision using parameters and copying to a lost part. Usually, when packet loss continues, the same waveform is repeated to generate noise like a beat. However, if the present invention is used, the number of repetitions of the same waveform can be reduced, thus reducing the noise described above. can do.

Japanese Patent No. 3999807 JP 2008-203783 A

ITU-T G.711 Appendix I

  However, the conventional packet loss concealment in the frequency domain including the technique of Patent Document 1 generates a concealment signal by repeating a decoded signal that has been normally received in the past, but the unit of repetition is a signal of one frame. When signals having different properties such as vowels and consonants are mixed in the extracted frame, signals having different properties are also mixed in the concealed signal, and there is a problem that a sufficient concealing effect cannot be exhibited.

  In addition, the technique of Patent Document 2 that generates a concealment signal as a technique that can be flexibly selected without limiting the repetition unit to one frame can avoid to some extent the mixing of signals having different properties. The signal generation reference is limited to the signal continuity and pitch period, and it is difficult to perform flexible processing such as determining a repetitive waveform using a change in power or spectrum. In addition, since the present invention relates to a packet loss concealment method in the time domain, in order to combine with a speech coding / decoding device in the frequency domain or the time frequency domain, it is decoded and converted to a time domain signal. This is necessary and is not realistic from the viewpoint of computational complexity.

  As described above, when generating a concealment signal using decoded speech stored in the buffer, it is possible to generate a concealment signal by flexibly obtaining a repetitive unit according to a change in power or a property change of a power spectrum. Have difficulty. In particular, when the decoded speech is expressed in the frequency domain, the repetition unit cannot be made shorter than one frame, so it is difficult to avoid mixing signals with different properties in the concealment signal.

  An object of the present invention is to solve the above-described problems and prevent deterioration in sound quality of a concealment signal for packet loss concealment.

The concealment signal generation device according to the present invention receives a packet error or packet loss detection result in a received packet including a voice code, and a decoded signal obtained by decoding the voice code from the outside, and takes the packet loss part. A concealment signal generator for concealing a packet loss for a corresponding decoded signal, and a decoded signal accumulating unit for accumulating a decoded signal obtained from a speech code included in the packet whose detection result is normal; When the detection result is abnormal, it detects the power change of the decoded signal that is a decoded signal expressed in the time-frequency domain and is stored by the decoded signal storage unit and / or one of the property change of the power spectrum, and as the signal identification information representing a detection result, both the information related to information and end of the change relative to the start of a change or a signal outputting one First concealment signal generation for generating a concealment signal for interpolating the decoded signal corresponding to the packet loss part based on the separate unit, the signal identification information, and the decoded signal stored by the decoded signal storage unit And a section.

The first concealment signal generation unit may generate, as a concealment signal, a signal obtained by repeating the decoded signal in a range specified by using the signal identification information in the decoded signal storage unit.

The first concealment signal generation unit generates a concealment signal as a signal obtained by adjusting the power after repeating the decoded signal in the range specified using the signal identification information in the decoded signal storage unit. May be.

  By the way, the invention related to the concealment signal generation device described above can be regarded as an invention related to the concealment signal generation method and an invention related to the concealment signal generation program, and can be described as follows.

The concealment signal generation method according to the present invention receives a packet error or packet loss detection result in a received packet including a voice code, and a decoded signal obtained by decoding the voice code from the outside, A concealment signal generation method executed by a concealment signal generation apparatus for concealing packet loss for a corresponding decoded signal, the decoding obtained from a speech code included in a packet whose detection result is normal A decoded signal accumulating step for accumulating the signal in the decoded signal accumulating unit; and when the detection result is abnormal, a power change of the decoded signal which is a decoded signal expressed in a time-frequency domain and accumulated by the decoded signal accumulating unit ; detecting one or both of the property change of the power spectrum, as a signal identification information indicating the detection result, to start changing A signal identification step of outputting both or one of the information about the termination of the distribution and the change, the signal identification information, based on the decoded signals stored by said decoded signal storage step, the decoded signal corresponding to the parts packet loss And a first concealment signal generation step for generating a concealment signal for interpolating.

The concealment signal generation program according to the present invention allows a computer to perform decoding obtained from a speech code included in a packet in which a packet error or packet loss detection result in a received packet including the speech code is normal. Decoding signal accumulation unit for accumulating a signal, and when the detection result is abnormal, a change in power of the decoded signal that is a time-frequency domain expressed decoding signal and accumulated in the decoding signal accumulation unit and a property change in the power spectrum The signal identification unit that detects both or one of them and outputs both or one of information about the start of change and information about the end of change as signal identification information representing the detection result, the signal identification information, and the decoded signal Based on the decoded signal stored by the storage unit, the hidden signal for interpolating the decoded signal corresponding to the packet loss portion The first concealment signal generator that generates a signal, which is concealment signal generation program for functioning as a.

  According to the present invention as described above, the signal repetition unit when generating the concealment signal can be made shorter than the frequency domain signal using the conventional MDCT or FFT, so that signals having different properties are mixed in the signal output for concealment. Can be prevented, and deterioration of the sound quality of the packet loss concealment signal can be prevented.

  According to the present invention, it is possible to prevent deterioration in sound quality of a concealment signal for packet loss concealment.

It is a figure which shows the system environment in one Embodiment of invention. It is a block diagram of a decoding part. It is a block diagram of the signal identification part in 1st Embodiment. It is a flowchart which shows operation | movement of the 1st concealment signal generation part in 1st Embodiment. It is a block diagram of the signal identification part in 2nd Embodiment. It is a flowchart which shows operation | movement of the signal identification part in 2nd Embodiment. It is a flowchart which shows operation | movement of the 1st concealment signal generation part in 2nd Embodiment. It is a block diagram of the signal identification part in 3rd Embodiment. It is a flowchart which shows operation | movement of the signal identification part in 3rd Embodiment. It is a block diagram of the signal identification part in 4th Embodiment. It is a flowchart which shows operation | movement of the 1st concealment signal generation part in 4th Embodiment. It is a block diagram of the signal identification part in 5th Embodiment. It is a figure which shows the relationship of the auxiliary information in 5th Embodiment. It is a hardware block diagram of a computer. It is an external view of a computer. It is a figure which shows the structure of a concealment signal generation program.

  Hereinafter, various embodiments according to the present invention will be described with reference to the drawings.

[First Embodiment]
First, a system environment assumed by the present invention will be described with reference to FIG. As shown in FIG. 1, an audio signal obtained through a sensor such as a microphone is expressed in a digital format and input to the encoding unit 1.

  The encoding unit 1 encodes the digital signal in the buffer every time a predetermined number of audio signals of a predetermined number of samples are accumulated in the built-in buffer. The predetermined amount, that is, the number of accumulated samples is called a frame length, and a set of digital signals to be encoded is called a frame. For example, when a frame length of 20 ms is used when collecting sound at a sampling frequency of 32 kHz, a digital signal of 640 samples is stored in the buffer. The buffer may store extra digital signals for prefetching. As the timing of encoding, encoding may be performed in units of frame length, or encoding may be performed with an overlap of a certain length between frames. Any encoding method may be used for encoding.

  The packet construction unit 2 adds information necessary for communication such as an RTP header to the speech code obtained by the encoding unit 1 to generate a speech packet. The generated voice packet is sent to the receiving side through the network.

  The packet separation unit 3 separates the voice packet received through the network into an RTP header and a voice code, generates a bit stream obtained by adding an error flag indicating an error state of the voice packet to the voice code, Is output to the decoding unit 4.

  As illustrated in FIG. 2, the decoding unit 4 includes an error / loss detection unit 41, a speech decoding unit 42, and a concealment signal generation unit 43. The decoding unit 4 detects an abnormality (packet error or packet loss) in the voice packet by identifying the error flag state in the error / loss detection unit 41, and in the case of normal (no abnormality), the voice decoding unit 42 The voice code is decoded at, and a decoded signal is output. On the other hand, when an abnormality (packet error or packet loss) is detected, a concealment signal generation unit 43 generates a concealment signal and outputs the concealment signal as a decoded signal. The decoding unit 4 outputs decoded speech for each frame. The decoded sound is sent to an audio buffer or the like and reproduced through a speaker or the like, or stored in a recording medium such as a memory or a hard disk.

  Hereinafter, the operation of the decoding unit 4 will be described. The error / loss detection unit 41 detects an abnormality (packet error or packet loss) in the voice packet by identifying the state of the error flag included in the bitstream.

  Here, when a value indicating that the voice packet is normal is set in the error flag, the error / loss detection unit 41 displays the error flag in the voice decoding unit 42 and the concealment signal generation unit 43 (specifically, a decoded signal accumulation described later). And the voice code is sent to the voice decoding unit 42. Then, the speech decoding unit 42 generates a decoded signal by decoding the speech code and outputs it as decoded speech. At this time, the voice decoding unit 42 also sends the decoded signal to the concealment signal generation unit 43.

  On the other hand, when a value indicating voice packet abnormality is set in the error flag, the error / loss detection unit 41 displays the error flag as a concealment signal generation unit 43 (specifically, a decoded signal accumulation unit 431 and a signal identification unit described later). 434). The concealment signal generation unit 43 generates a concealment signal from the decoded signal corresponding to the voice packet normally received in the past. Details of the operation of the concealment signal generator 43 will be described later.

  The overall configuration of FIG. 1 described above and the operations of the error / loss detection unit 41 and the speech decoding unit 42 of the decoding unit 4 of FIG. 2 are the same in the second to fifth embodiments described later. In the fifth embodiment, a duplicate description is omitted.

  Hereinafter, the configuration and operation of the concealment signal generation unit 43 will be described in detail. In 1st Embodiment, the concealment signal generation part 43 shows the example which uses the sudden change of the power in the signal of a time domain as signal identification information.

  As shown in FIG. 2, the concealment signal generation unit 43 includes a decoded signal accumulation unit 431, a signal identification unit 434, and a first concealment signal generation unit 433.

  The decoded signal accumulation unit 431 accumulates the decoded signal input from the audio decoding unit 42 when a value indicating normal audio packet is set in the error flag. The number of samples of the decoded signal to be stored is preferably the past several frames (d frames) (here, x (0),..., X (dL). Note that the length of one frame is L. )

  As shown in FIG. 3, the signal identification unit 434 includes a decoded signal accumulation unit 4340, a time envelope calculation unit 4341, and a signal identification information generation unit 4342. Among these, the decoded signal storage unit 4340 performs the same operation as the decoded signal storage unit 431 when a value indicating normal voice packet is set in the error flag.

  When a value indicating voice packet abnormality is set in the error flag, the time envelope calculation unit 4341 reads the accumulated decoded signal (hereinafter referred to as “accumulated decoded signal”) from the decoded signal accumulation unit 4340, and accumulates the decoded signal. Time envelope information, which is information about each power, is calculated. As a modification, instead of providing the decoded signal storage unit 4340, the time envelope calculation unit 4341 may read the stored decoded signal from the decoded signal storage unit 431 instead.

x(k)は、ｋ番目のサンプルの値を表す。ここで、ｋ^l _startはｌ番目の小区間の開始位置を示し、ｋ^l _endはｌ番目の小区間の終了位置を示す。また、ここではｋ^l _start＝ｋ^l-1 _end＋１としたが、小区間同士でオーバーラップを持たせるようにしてもよい。 As a method for calculating the time envelope information here, there are various methods such as a method for calculating time envelope information using a maximum value of amplitude for each of a plurality of small sections and a method for calculating time envelope information using variance. Though conceivable, for example, time envelope information is calculated according to the following equation. Here, it is assumed that time envelope information is calculated for K small sections.

x (k) represents the value of the kth sample. Here, k ^l _start indicates the start position of the l-th subsection, and k ^l _end indicates the end position of the l-th subsection. In addition, here, k ^l _start = k ^l−1 _end +1, but it is also possible to provide overlap between the small sections.

The signal identification information generation unit 4342 detects a sudden change in power and outputs signal identification information according to the result. Specifically, various methods such as calculating a power dispersion value and detecting a sudden change in power by comparing the dispersion value and a threshold value can be used. A sudden change in power is detected as follows.
Step 1: An envelope Penv (l) obtained by smoothing Env (l) is calculated by the following equation. However, α is a constant that satisfies 0 <α <1.
Penv (l) = α ・ Penv (l−1) + (1−α) ・ Env (l)
Step 2: Using Env (l) and Penv (l), a rapid change in power is detected by comparing Env (l) and (β · Penv (l)). Where β is a constant. That is, when Env (l)> β · Penv (l), it is determined that the power changes abruptly in subsample l.

  The method described above is a simple example of signal change detection based on power change, and signal change detection may be performed by another more complicated method. When a sudden change in power is detected as a result of the above processing, the index lstart of the subsample at which the change starts is output as signal identification information. When a signal whose power changes rapidly is not detected, a value obtained by subtracting the number of samples for one frame from the end of the buffer may be used as the index lstart. Note that a simple method such as setting lstart as the top index of the buffer or the last index of the buffer may be used, or a value obtained by calculating the pitch period and subtracting the pitch period from the end of the buffer may be set as lstart. .

  The first concealment signal generation unit 433 generates a concealment signal using the signal identification information and the stored decoded signal. Specifically, the concealment signal is generated by the following procedure. The operation of the first concealment signal generation unit 433 is shown in FIG.

  In step S11 of FIG. 4, the first concealment signal generation unit 433 obtains the index lstart by referring to the signal identification information, and sets the index of the last subsample of the accumulated decoded signal stored in the buffer as lend. Here, the value of lend-lstart is set in the variable L ′.

In step S12, the first concealment signal generation unit 433 copies the accumulated decoded signal from the decoded signal accumulation unit 431. When copying, samples from lstart to lend are repeatedly copied until the number N of samples included in one frame is satisfied. For example, first, the variable i is reset to 0 (step S121), and the stored decoded signal stored in the decoded signal storage unit 431 is copied as the concealment signal v (i) corresponding to the packet loss part according to the following equation (step S121). Step S122).
v (i) = b (lstart + i% L ')
Here, b (i) means an accumulated decoded signal accumulated in the decoded signal accumulation unit 431, and (i% L ′) represents a remainder obtained by dividing i by L ′.

  If the variable i is less than the sample number N (YES in step S124), the variable i is incremented by one (step S123), and the process of step S122 is performed on the counted variable i. Thereafter, steps S122 and S123 are repeated until the variable i is equal to the number of samples N (NO in step S124). As a result, samples from lstart to lend can be copied until the number N of samples included in one frame is satisfied.

Next, in step S13, the first concealment signal generation unit 433 calculates the mean square amplitude for each subsample and normalizes the copied accumulated decoded signal, and then attenuates it to the mean square amplitude of the subsample immediately before the packet loss. A concealment signal is generated by multiplying the power of the coefficient. For example, first, the variable i is reset to 0 (step S131), and the concealment signal v corresponding to the packet loss part is generated according to the following equation (step S132).
v (iL '+ k) = v (iL' + k) / 10 ^{(Env (i) / 2)}・ 10 ^{(Env (K-1) / 2)}・ γ ⁱ
Here, Env (i) represents the time envelope of the i-th subsection (K is the number of subsections), v (i) represents the concealment signal corresponding to the packet loss portion, and γ represents the attenuation constant.

  If the variable i is less than the sample number N (YES in step S134), the variable i is incremented by one (step S133), and the process of step S132 is performed on the counted variable i. Thereafter, steps S132 and S133 are repeated until the variable i becomes equal to the number of samples N (NO in step S134). Thereby, a concealment signal is generated.

  In step S14, the first concealment signal generation unit 433 outputs the generated concealment signal.

In addition to the above, a concealment signal may be generated by prediction. Specifically, the following method may be used.
Step 1: The index lstart is obtained by referring to the signal identification information. Also, let the index of the last subsample of the accumulated decoded signal stored in the buffer be lend.
Step 2: The stored decoded signal in the decoded signal storage unit 431 is copied from lstart to lend and subjected to linear prediction analysis.
Step 3: The residual signal obtained in Step 2 is repeated until the number N of samples included in one frame is satisfied.
Step 4: The signal obtained in Step 3 is inverse filtered with the linear prediction coefficient obtained in Step 2, and then a predetermined attenuation coefficient is multiplied for each sample. The signal thus obtained is used as a concealment signal.

  As described above, in the first embodiment, the concealment signal generation unit 43 can generate and output signal identification information using a rapid change in power in a time domain signal.

[Second Embodiment]
In the first embodiment, signal identification information is output using an abrupt change in power. In the second embodiment, an example in which signal identification information is generated using an abrupt change in power spectrum will be described.

  In this embodiment, a time-domain signal is assumed as a decoded signal. However, when a decoded signal is obtained as a frequency-domain signal (for example, a QMF coefficient), the decoded signal storage unit remains represented in the frequency domain. It is also possible to store the decoded signal in the configuration and omit the time frequency conversion unit.

  Hereinafter, the operation of the concealment signal generation unit 43 will be described.

  The operation of the decoded signal storage unit 431 shown in FIG. 2 is the same as that of the first embodiment.

  As shown in FIG. 5, the signal identification unit 434 includes a decoded signal accumulation unit 4340, a time frequency conversion unit 4343, a conversion gain calculation unit 4344, and a signal identification information generation unit 4342. Among these, the operation of the decoded signal storage unit 4340 is the same as that of the first embodiment.

  A time-frequency conversion unit 4343, a conversion gain calculation unit 4344, and a signal identification information generation unit 4342 that operate when a value indicating voice packet abnormality is set in the error flag will be described below with reference to FIG.

  The time frequency conversion unit 4343 reads out the stored decoded signal from the decoded signal storage unit 4340 and outputs the stored frequency signal converted into the frequency domain (step S21 in FIG. 6). For the conversion, FFT (Fast Fourier Transform) may be used, MDCT (Modified Discrete Transform) may be used, or QMF may be used. Further, as a modification, a configuration may be adopted in which the stored decoded signal is read from the decoded signal storage unit 431 without providing the decoded signal storage unit 4340.

ここで、例えばV(k,l)のサブサンプルl₀と言う場合には、

なる周波数ビンの集合を現すものとする。Eは時間方向のサブサンプル数を表し、Kは周波数ビンの数を表す。ｋは周波数ビンのインデックスであり（0≦k≦K-1）、lはサブサンプルのインデックス（0≦l≦L-1）である。また、ｐ_A(n)は分析に用いる窓関数を表す。 In the present embodiment, an example is shown in which time-frequency conversion is performed using, for example, QMF. Here, the stored decoded signal is assumed to be x (n).

Here, for example, when it is referred to as subsample l _{0 of} V (k, l),

Let be a set of frequency bins. E represents the number of subsamples in the time direction, and K represents the number of frequency bins. k is the frequency bin index (0 ≦ k ≦ K−1), and l is the subsample index (0 ≦ l ≦ L−1). P _A (n) represents a window function used for analysis.

なお、本実施例では全周波数ビンを用いてU(l)を算出したが、一部の周波数ビンのみを用いて相加平均／相乗平均U(l)を算出してもよい。 The conversion gain calculation unit 4344 refers to the (arithmetic mean / geometric mean) value (hereinafter referred to as “U (l)”) of the accumulated decoded signal subjected to time-frequency conversion. l) ”(step S22 in FIG. 6).

In this embodiment, U (l) is calculated using all frequency bins, but the arithmetic average / geometric mean U (l) may be calculated using only some of the frequency bins.

  The signal identification information generation unit 4342 detects, as the signal identification information lstart, the subsample l that exceeds the threshold Th with the arithmetic mean / geometric mean U (l) (step S23 in FIG. 6). For example, first, the subsample index l is reset to 0 (step S231), and it is determined whether the arithmetic mean / geometric mean U (l) exceeds the threshold Th (step S232). If the arithmetic mean / geometric mean U (l) does not exceed the threshold Th, the process proceeds to step S235, and if the arithmetic mean / geometric mean U (l) exceeds the threshold Th, the index l at that time is used as the signal identification information. Set to lstart (step S233), and proceed to step S235.

  If the index l is less than L in step S235 (YES in step S235), the index l is incremented by one (step S234), and the processes in steps S232 and S233 are performed on the counted index l. Thereafter, steps S232 to S234 are repeated until the index l becomes equal to L (NO in step S235). As a result, the subsample l exceeding the threshold Th with the arithmetic mean / geometric mean U (l) can be detected as the signal identification information lstart.

  Then, the signal identification information generation unit 4342 outputs the signal identification information lstart obtained in step S23 (step S24 in FIG. 6).

  The first concealment signal generation unit 433 generates a concealment signal using the signal identification information and the stored decoded signal. Specifically, the concealment signal is generated by the following procedure. The operation of the first concealment signal generation unit 433 is shown in FIG.

  In step S25 of FIG. 7, the first concealment signal generation unit 433 obtains the index lstart by referring to the signal identification information, and sets the index of the last subsample of the accumulated decoded signal stored in the buffer as lend. Here, the value of lend-lstart is set in the variable L ′.

In step S26, the first concealment signal generation unit 433 copies the accumulated decoded signal from the decoded signal accumulation unit 431. When copying, samples from lstart to lend are copied until the number of samples N included in one frame is satisfied. For example, first, the variable i is reset to 0 (step S261), and the stored decoded signal stored in the decoded signal storage unit 431 is copied as the concealment signal V (k, i) corresponding to the packet loss part according to the following equation. (Step S262).
V (k, i) = B (k, lstart + i% L ')
Here, B (k, i) is a signal obtained by time-frequency conversion of the stored decoded signal stored in the decoded signal storage unit 431, V (k, i) is a concealment signal corresponding to the packet loss part, and (i% L ′ ) Represents the remainder obtained by dividing i by L ′.

  If the variable i is less than the sample number N (YES in step S264), the variable i is incremented by one (step S263), and the process of step S262 is performed on the counted variable i. Thereafter, steps S262 and S263 are repeated until the variable i becomes equal to the number of samples N (NO in step S264). As a result, samples from lstart to lend can be copied until the number N of samples included in one frame is satisfied.

ステップS28で第一隠蔽信号生成部４３３は、コピーした蓄積復号信号をサブサンプル毎に平均二乗振幅を算出して正規化した上で、パケットロス直前のサブサンプルの平均二乗振幅に減衰係数のべき乗を乗算することで隠蔽信号を生成する。例えば、まず変数iを０にリセットし（ステップS281）、パケットロス部分に対応する隠蔽信号Vを、以下の式に従い生成する（ステップS282）。
V(k,i) = V(k,i)／10^(Env(i)/2)・10^(Env(L-1)/2)・γⁱ
ここでEnv(i)はi番目の小区間の時間包絡(Ｋは小区間の数)、V(k,i)はパケットロス部分に対応する隠蔽信号、γは減衰定数をそれぞれ表す。 In step S27, the first concealment signal generation unit 433 calculates the sub-sample power Env (l).

In step S28, the first concealment signal generation unit 433 calculates the mean square amplitude for each subsample and normalizes the copied accumulated decoded signal, and then adds the power of the attenuation coefficient to the mean square amplitude of the subsample immediately before the packet loss. Is used to generate a concealment signal. For example, first, the variable i is reset to 0 (step S281), and the concealment signal V corresponding to the packet loss part is generated according to the following equation (step S282).
V (k, i) = V (k, i) / 10 ^{(Env (i) / 2)} · 10 ^{(Env (L-1) / 2)} · γ ⁱ
Here, Env (i) represents the time envelope of the i-th subsection (K is the number of subsections), V (k, i) represents the concealment signal corresponding to the packet loss part, and γ represents the attenuation constant.

  If the variable i is less than the sample number N (YES in step S284), the variable i is incremented by one (step S283), and the process of step S282 is performed on the counted variable i. Thereafter, steps S282 and S283 are repeated until the variable i becomes equal to the number of samples N (NO in step S284). Thereby, a concealment signal is generated.

ここで、i（0≦i<L）は時間領域の信号のインデックスであり、k（0≦k<K-1）はサブフレームのインデックスである。 In step S29, the first concealment signal generation unit 433 generates a time domain concealment signal y (kL + i) by inversely transforming the concealment signal V (k, i) corresponding to the packet loss part using the combined QMF. Output.

Here, i (0 ≦ i <L) is a time domain signal index, and k (0 ≦ k <K−1) is a subframe index.

  As described above, in the second embodiment, signal identification information can be generated and output using a sudden change in the power spectrum.

[Third Embodiment]
In the first embodiment, signal identification information is calculated using a sudden change in power, and in the second embodiment, signal identification information is calculated using a sudden change in power spectrum. An example of calculating signal identification information using both will be described.

  In this embodiment, a time-domain signal is assumed as a decoded signal. However, when a decoded signal is obtained as a frequency-domain signal (for example, a QMF coefficient), the decoded signal storage unit remains represented in the frequency domain. It is also possible to store the decoded signal in the configuration and omit the time frequency conversion unit.

  Hereinafter, the operation of the concealment signal generation unit 43 will be described.

  The operation of the decoded signal storage unit 431 shown in FIG. 2 is the same as that of the first embodiment.

  As shown in FIG. 8, the signal identification unit 434 includes a decoded signal accumulation unit 4340, a time envelope calculation unit 4341, a time frequency conversion unit 4343, a conversion gain calculation unit 4344, and a signal identification information generation unit 4342. Among these, the decoded signal storage unit 4340 operates in the same manner as the decoded signal storage unit 431 when a value indicating voice packet abnormality is set in the error flag.

  Hereinafter, with respect to the time-frequency conversion unit 4343, the time envelope calculation unit 4341, the conversion gain calculation unit 4344, and the signal identification information generation unit 4342, which operate when a value indicating voice packet abnormality is set in the error flag, FIG. This will be described below.

ここで、例えばV(k,l)のサブサンプルl₀と言う場合には、

なる周波数ビンの集合を現すものとする。Eは時間方向のサブサンプル数を表し、Kは周波数ビンの数を表す。ｋは周波数ビンのインデックスであり（0≦k≦K-1）、lはサブサンプルのインデックス（0≦l≦L-1）である。また、ｐ_A(n)は分析に用いる窓関数を表す。 The time frequency conversion unit 4343 reads out the stored decoded signal from the decoded signal storage unit 4340 and outputs the stored frequency signal converted into the frequency domain (step S31 in FIG. 9). As a modification, instead of providing the decoded signal storage unit 4340, a configuration in which the stored decoded signal is read from the decoded signal storage unit 431 may be employed. Here, the stored decoded signal is x (n).

Here, for example, when it is referred to as subsample l _{0 of} V (k, l),

Let be a set of frequency bins. E represents the number of subsamples in the time direction, and K represents the number of frequency bins. k is the frequency bin index (0 ≦ k ≦ K−1), and l is the subsample index (0 ≦ l ≦ L−1). P _A (n) represents a window function used for analysis.

変換利得算出部４３４４は、蓄積周波数信号について、相加平均／相乗平均U(l)を算出する（図９のステップS33）。

なお、本実施例では全周波数ビンを用いてU(l)を算出したが、一部の周波数ビンのみを用いて相加平均／相乗平均U(l)を算出してもよい。 The time envelope calculation unit 4341 calculates a time envelope according to the following equation, for example (step S32 in FIG. 9).

The conversion gain calculation unit 4344 calculates the arithmetic mean / geometric mean U (l) for the accumulated frequency signal (step S33 in FIG. 9).

In this embodiment, U (l) is calculated using all frequency bins, but the arithmetic average / geometric mean U (l) may be calculated using only some of the frequency bins.

  The signal identification information generation unit 4342 detects the subsample l exceeding the threshold value Th as the arithmetic mean / geometric mean U (I) as the signal identification information lstart, and then determines Env (l) and (β · Penv (l) ) Is detected, and a rapid change in power is detected, and if necessary, lstart is updated to generate signal identification information lstart (step S34 in FIG. 9). Where β is a constant. When Env (l)> β · Penv (l), lstart may be calculated by determining that the power changes rapidly in subsample l. Note that the conversion gain and the time envelope may be weighted, and the signal identification information may be generated by combining the conversion gain and the time envelope.

  Specifically, in step S34, as shown in FIG. 9, the signal identification information generation unit 4342 first resets the subsample index l to 0 (step S341), and the arithmetic mean / geometric mean U (l) is a threshold value. It is determined whether or not Th is exceeded (step S342). If the arithmetic mean / geometric mean U (l) does not exceed the threshold Th, the process proceeds to step S345. If the arithmetic mean / geometric mean U (l) exceeds the threshold Th, the index l at that time is used as the signal identification information. Set to lstart (step S343), and proceed to step S345.

  If the index l is less than L in step S345 (YES in step S345), the index l is incremented by one (step S344), and the processes in steps S342 and S343 are performed on the counted index l. Thereafter, steps S342 to S344 are repeated until the index l becomes equal to L (NO in step S345). As a result, the subsample l exceeding the threshold Th with the arithmetic mean / geometric mean U (l) can be detected as the signal identification information lstart.

  Next, the index l is reset to 0 (step S346), Env (l) is compared with (β · Penv (l)) (step S347), and Env (l) becomes (β · Penv (l) ), The process proceeds to step S34A. If Env (l) exceeds (β · Penv (l)), the index l at that time is set in the signal identification information lstart (step S348), and step S34A. Proceed to

  If the index l is less than L in step S34A (YES in step S34A), the index l is incremented by one (step S349), and the processes of steps S347 and S348 are performed on the counted index l. Thereafter, steps S347 to S349 are repeated until the index l becomes equal to L (NO in step S34A). As a result, it is possible to detect the subsample l whose power changes rapidly as the signal identification information lstart.

  Then, the signal identification information generation unit 4342 outputs the signal identification information lstart obtained in step S34 (step S35 in FIG. 9).

  In addition, the 1st concealment signal generation part 433 in 3rd Embodiment performs the operation | movement similar to 2nd Embodiment.

  As described above, in the third embodiment, signal identification information can be generated and output using both a rapid change in power and a rapid change in power spectrum.

[Fourth Embodiment]
In the fourth embodiment, unlike the first to third embodiments, a processing example in the case of acquiring signal identification information from the outside will be described. As an input method of signal identification information, for example, there is a method of using a parameter obtained in an auxiliary manner in the decoding process.

  Hereinafter, an example in which packet loss concealment is performed using the parameters obtained at the time of decoding as signal identification information when TS26.401 (enhanced aacPlus) is used as an encoding method will be described.

  TS26.401 encodes a high frequency signal with a small bit amount by SBR (Spectral Band Replication). In decoding by SBR, a high frequency signal is generated using auxiliary information sent from the SBR encoding side and a low frequency decoded signal. The auxiliary information includes time boundary information in the frame. Specifically, it is a tE parameter defined in TS26.404 section 3.2. The time boundary in the frame is inserted by the SBR encoder when the power suddenly increases or when the power spectrum property changes.

  In the present embodiment, a case will be described in which the concealment signal generation unit 43 in FIG. 2 generates a concealment signal using the time boundary tE in the frame. The operation of the decoded signal storage unit 431 is the same as that in the first embodiment.

  As shown in FIG. 10, the signal identification unit 434 includes a signal identification information storage unit 4345. The signal identification information accumulation unit 4345 accumulates the signal identification information input from the audio decoding unit 42 when the error flag is set to a value indicating normal voice packet. The signal identification information is the time boundary tE in the frame described above. When a value indicating voice packet abnormality (for example, packet error or packet loss) is set in the error flag, the accumulated signal identification information (hereinafter referred to as “accumulated signal identification information”) is sent from the signal identification information accumulation unit 4345 to the first. It is output to the concealment signal generation unit 433.

  The first concealment signal generation unit 433 uses the accumulated signal identification information and the accumulated decoded signal to generate a concealment signal according to the following procedure. The operation of the first concealment signal generation unit 433 is shown in FIG.

ステップS42で第一隠蔽信号生成部４３３は、復号信号蓄積部４３１から蓄積復号信号をコピーする。コピーする際には、lstartからlendまでのサブサンプルを１フレームに含まれるサブサンプル数を満たすまでコピーする。例えば、まず変数iを０にリセットし（ステップS421）、以下の式に従い、パケットロス部分に対応する隠蔽信号V(k,i)として、復号信号蓄積部４３１に蓄積された蓄積復号信号をコピーする（ステップS422）。
V(k,i)=B(k,lstart+i%L’)
ここで、B(k,i)は復号信号蓄積部４３１に蓄積された蓄積復号信号を時間周波数変換した信号、V(k,i)はパケットロス部分に対応する隠蔽信号、(i%L’)はiをL’で割った余りをそれぞれ表す。 In step S41 of FIG. 11, the first concealment signal generation unit 433 refers to the accumulated signal identification information and obtains the index lstart and the index lend according to the following expressions. Here, tE represents the signal identification information stored in the signal identification information storage unit and included in the most recently received packet. L ′ is the number of sections separated by time boundaries in the frame.

In step S42, the first concealment signal generation unit 433 copies the accumulated decoded signal from the decoded signal accumulation unit 431. When copying, the subsamples from lstart to lend are copied until the number of subsamples included in one frame is satisfied. For example, first, the variable i is reset to 0 (step S421), and the stored decoded signal stored in the decoded signal storage unit 431 is copied as the concealment signal V (k, i) corresponding to the packet loss part according to the following equation. (Step S422).
V (k, i) = B (k, lstart + i% L ')
Here, B (k, i) is a signal obtained by time-frequency conversion of the stored decoded signal stored in the decoded signal storage unit 431, V (k, i) is a concealment signal corresponding to the packet loss part, and (i% L ′ ) Represents the remainder obtained by dividing i by L ′.

  If the variable i is less than the sample number N (YES in step S424), the variable i is incremented by one (step S423), and the process of step S422 is performed on the counted variable i. Thereafter, steps S422 and S423 are repeated until the variable i becomes equal to the number of samples N (NO in step S424). Thereby, the subsamples from lstart to lend can be copied until the number of subsamples included in one frame is satisfied.

ステップS44で第一隠蔽信号生成部４３３は、コピーした蓄積復号信号をサブサンプル毎に平均二乗振幅を算出して正規化した上で、パケットロス直前のサブサンプルの平均二乗振幅に減衰係数のべき乗を乗算することで隠蔽信号を生成する。例えば、まず変数iを０にリセットし（ステップS441）、パケットロス部分に対応する隠蔽信号Vを、以下の式に従い生成する（ステップS442）。
V(k,i) = V(k,i)／10^(Env(i)/2)・10^(Env(L-1)/2)・γⁱ
ここでEnv(i)はi番目の小区間の時間包絡(Ｋは小区間の数)、V(k,i)はパケットロス部分に対応する隠蔽信号、γは減衰定数をそれぞれ表す。 In step S43, the first concealment signal generation unit 433 calculates the sub-sample power Env (l).

In step S44, the first concealment signal generation unit 433 calculates the mean square amplitude for each subsample and normalizes the copied accumulated decoded signal, and then adds the power of the attenuation coefficient to the mean square amplitude of the subsample immediately before the packet loss. Is used to generate a concealment signal. For example, first, the variable i is reset to 0 (step S441), and the concealment signal V corresponding to the packet loss part is generated according to the following equation (step S442).
V (k, i) = V (k, i) / 10 ^{(Env (i) / 2)} · 10 ^{(Env (L-1) / 2)} · γ ⁱ
Here, Env (i) represents the time envelope of the i-th subsection (K is the number of subsections), V (k, i) represents the concealment signal corresponding to the packet loss part, and γ represents the attenuation constant.

  If the variable i is less than the number of samples N (YES in step S444), the variable i is incremented by one (step S443), and the process of step S442 is performed on the counted variable i. Thereafter, steps S442 and S443 are repeated until the variable i becomes equal to the number of samples N (NO in step S444). Thereby, a concealment signal is generated.

ここで、i（0≦i<L）は時間領域の信号のインデックスであり、k（0≦k<K-1）はサブフレームのインデックスである。 Then, in step S45, the first concealment signal generation unit 433 generates a time domain concealment signal y (kL + i) by inversely transforming the concealment signal V (k, i) corresponding to the packet loss part by the combined QMF. Output.

Here, i (0 ≦ i <L) is a time domain signal index, and k (0 ≦ k <K−1) is a subframe index.

  As described above, in the fourth embodiment, a concealment signal can be generated and output even when signal identification information is acquired from the outside.

[Fifth Embodiment]
In the fifth embodiment, unlike the fourth embodiment, an example will be described in which signal identification information is calculated from parameters obtained in an auxiliary manner in the decoding process.

  In the present embodiment, for example, a scale factor when AAC is used for encoding, a time-frequency domain representation of a decoded signal in another encoding method, or the like can be used. For example, the following example can be used by substituting these parameters into V (k, l) in the example described below. Similar processing may be performed using parameters that can be calculated from these time-frequency domain expressions.

  In the present embodiment, hereinafter, an example will be described in which signal identification information is calculated from parameters obtained at the time of decoding and packet loss concealment is performed when TS26.401 (enhanced aacPlus) is used as an encoding method.

In the present embodiment, the signal identification information is calculated from the power spectrum envelope information included in the auxiliary information of TS26.404. Specifically, the auxiliary information of TS26.404 represents a parameter representing the power spectrum envelope power defined as E _orig defined in section 3.2 of TS26.404, and the band boundary of the power spectrum envelope defined as f _TableHigh. It is a parameter. FIG. 13 shows an example of the relationship between these parameters. The parameters are calculated in the decoding process.

  Therefore, in this embodiment, a case where a concealment signal is generated using the above will be described. Note that the operation of the decoded signal storage unit 431 in the concealment signal generation unit 43 in FIG. 2 is the same as that in the first embodiment.

  As shown in FIG. 12, the signal identification unit 434 includes an auxiliary information accumulation unit 4346, a power spectrum envelope calculation unit 4347, a time envelope calculation unit 4341, a conversion gain calculation unit 4344, and a signal identification information generation unit 4342. Prepare.

Among these, the auxiliary information accumulating unit 4346, when the error flag is set to a value indicating normal voice packet, the power spectrum envelope power of E _orig and the power spectrum envelope band of f _TableHigh inputted from the voice decoding unit 42 Accumulate boundaries.

ここで、全周波数帯域をn_high個の帯域に分割したとする。 When a value indicating voice packet abnormality is set in the error flag, the power spectrum envelope calculation unit 4347 reads the power of the power spectrum envelope and the band boundary of the power spectrum envelope from the auxiliary information storage unit 4346, and signals as follows: The identification information is calculated. That is, the power spectrum envelope V (k, l) is calculated from E _orig and f _TableHigh as follows.

Here, it is assumed that the entire frequency band is divided into n _high bands.

  The operations of the time envelope calculation unit 4341, the conversion gain calculation unit 4344, and the signal identification information generation unit 4342 are the same as those in the third embodiment. The operation of the first concealment signal generation unit 433 is the same as that in the third embodiment.

In the above, the example of calculating the signal identification information using the power spectrum envelope power of E _orig obtained in the decoding process and the band boundary of the power spectrum envelope of f _TableHigh has been described. Any method of calculating signal identification information using parameters and generating a concealment signal based on the obtained signal identification information may be used.

  As described above, in the fifth embodiment, signal identification information can be calculated from parameters obtained in an auxiliary manner in the decoding process, and a concealment signal can be generated and output based on the signal identification information.

[Hidden signal generation program]
First, a concealment signal generation program that causes a computer to operate as a concealment signal generation apparatus according to the present invention will be described.

  FIG. 16 is a diagram illustrating a configuration of a concealment signal generation program according to an embodiment. FIG. 14 is a hardware configuration diagram of a computer according to an embodiment. FIG. 15 is an external view of a computer according to an embodiment. The concealment signal generation program P43 illustrated in FIG. 16 can cause the computer C10 illustrated in FIGS. 14 and 15 to operate as the concealment signal generation unit 43. Note that the program described in this specification is not limited to the computer illustrated in FIGS. 14 and 15, and any information processing device such as a mobile phone, a portable information terminal, or a portable personal computer is operated according to the program. be able to.

  The concealment signal generation program P43 can be provided by being stored in the recording medium M. The recording medium M is exemplified by a recording medium such as a flexible disk, CD-ROM, DVD, or ROM, or a semiconductor memory.

  As shown in FIG. 14, the computer C10 stores programs stored in a reading device C12 such as a flexible disk drive device, a CD-ROM drive device, and a DVD drive device, a working memory (RAM) C14, and a recording medium M. A memory C16 to be stored, a display C18, a mouse C20 and a keyboard C22 as input devices, a communication device C24 for transmitting and receiving data and the like, and a central processing unit (CPU) C26 for controlling execution of a program. .

  When the recording medium M is inserted into the reading device C12, the computer C10 can access the concealment signal generation program P43 stored in the recording medium M from the reading device C12. It becomes possible to operate as a concealment signal generation device.

  As shown in FIG. 15, the concealment signal generation program P43 may be provided via a network as a computer data signal W superimposed on a carrier wave. In this case, the computer C10 can store the concealment signal generation program P43 received by the communication device C24 in the memory C16 and execute the concealment signal generation program P43.

  As shown in FIG. 16, the concealment signal generation program P43 includes a decoded signal accumulation module P431, a signal identification module P434, and a first concealment signal generation module P433. These decoded signal storage module P431, signal identification module P434, and first concealment signal generation module P433 have the same functions as the above-described decoded signal storage unit 431, signal identification unit 434, and first concealment signal generation unit 433, respectively. The computer C10 is executed. According to the concealment signal generation program P43, the computer C10 can operate as the concealment signal generation apparatus according to the present invention.

  According to the various embodiments described above, when obtaining a waveform as a repetition unit from a signal in a buffer for the purpose of generating a concealment signal, a time change of power or a time change of power spectrum is used. Further, a concealment signal is generated using a frequency domain signal obtained by time-frequency conversion using QMF (Quadrature Mirror Filter). In these cases, the repetition unit of the signal at the time of generating the concealment signal can be made shorter than the frequency domain signal using the conventional MDCT or FFT, thus preventing signals having different properties from being mixed in the signal output for concealment. Therefore, it is possible to prevent deterioration of the sound quality of the packet loss concealment signal.

  DESCRIPTION OF SYMBOLS 1 ... Coding part, 2 ... Packet structure part, 3 ... Packet separation part, 4 ... Decoding part, 41 ... Error / loss detection part, 42 ... Speech decoding part, 43 ... Concealment signal generation part, 431 ... Decoding signal storage part 433: First concealment signal generation unit, 434: Signal identification unit, 4340: Decoded signal storage unit, 4341 ... Time envelope calculation unit, 4342 ... Signal identification information generation unit, 4343 ... Time frequency conversion unit, 4344 ... Conversion gain calculation Unit, 4345 ... signal identification information storage unit, 4346 ... auxiliary information storage unit, 4347 ... power spectrum envelope calculation unit, C10 ... computer, C12 ... reading device, C14 ... working memory, C16 ... memory, C18 ... display, C20 ... Mouse, C22 ... Keyboard, C24 ... Communication device, C26 ... CPU, M ... Recording medium, W ... Computer data signal, P43 ... Audio code Program, P431 ... decoded signal storage module, P433 ... first concealment signal generation module, P434 ... signal identification module.

Claims (5)

The packet error concealment of the decoded signal corresponding to the packet loss part is received from the outside by receiving the packet error or packet loss detection result in the received packet including the voice code and the decoded signal obtained by decoding the voice code. A concealment signal generation device that performs
A decoded signal accumulating unit for accumulating a decoded signal obtained from a speech code included in a packet in which the detection result is normal;
When the detection result is abnormal, the detection is performed by detecting the power change and / or the property change of the power spectrum of the decoded signal that is expressed in the time-frequency domain and is stored by the decoded signal storage unit. As signal identification information representing the result, a signal identification unit that outputs information about the start of change and information about the end of change, or one of them,
A first concealment signal generation unit that generates a concealment signal for interpolating a decoded signal corresponding to a packet loss part based on the signal identification information and the decoded signal stored by the decoded signal storage unit;
A concealment signal generation device comprising: Wherein the first concealment signal generator according to claim, characterized in that the signal obtained by repeating the decoded signal of the range specified by using the signal identification information in the decoded signal storage unit, generates a concealment signal 1 concealment signal generator according to. The first concealment signal generation unit generates, as a concealment signal, a signal obtained by adjusting power after repeating a decoded signal in a range specified by using the signal identification information in the decoded signal storage unit The concealment signal generation device according to claim 1 . The packet error concealment of the decoded signal corresponding to the packet loss part is received from the outside by receiving the packet error or packet loss detection result in the received packet including the voice code and the decoded signal obtained by decoding the voice code. A concealment signal generation method executed by a concealment signal generation apparatus,
A decoded signal accumulating step for accumulating a decoded signal obtained from a speech code included in a packet whose detection result is normal, in a decoded signal accumulating unit ;
When the detection result is abnormal, the detection is performed by detecting the power change and / or the property change of the power spectrum of the decoded signal that is expressed in the time-frequency domain and is stored by the decoded signal storage unit. A signal identification step for outputting information on the start of change and / or information on the end of change as signal identification information representing the result,
A first concealment signal generating step for generating a concealment signal for interpolating a decoded signal corresponding to a packet loss part based on the signal identification information and the decoded signal stored by the decoded signal storage step;
A concealment signal generation method comprising: Computer
A decoded signal accumulation unit for accumulating a decoded signal obtained by decoding from a voice code included in a packet in which a packet error or packet loss detection result in a received packet including a voice code is normal; and
When the detection result is abnormal, the detection is performed by detecting the power change and / or the property change of the power spectrum of the decoded signal that is expressed in the time-frequency domain and is stored by the decoded signal storage unit. As signal identification information representing the result, a signal identification unit that outputs information about the start of change and information about the end of change, or one of them,
A first concealment signal generation unit that generates a concealment signal for interpolating a decoded signal corresponding to a packet loss part based on the signal identification information and the decoded signal accumulated by the decoded signal accumulation unit;
A concealment signal generation program for functioning as

Images