WO2017084545A1 - 一种语音丢包补偿的方法及系统 - Google Patents

一种语音丢包补偿的方法及系统 Download PDF

Info

Publication number
WO2017084545A1
WO2017084545A1 PCT/CN2016/105684 CN2016105684W WO2017084545A1 WO 2017084545 A1 WO2017084545 A1 WO 2017084545A1 CN 2016105684 W CN2016105684 W CN 2016105684W WO 2017084545 A1 WO2017084545 A1 WO 2017084545A1
Authority
WO
WIPO (PCT)
Prior art keywords
sequence
speech
voice
speech sequence
vec1
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/105684
Other languages
English (en)
French (fr)
Inventor
邹莹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Academy of Telecommunications Technology CATT
Original Assignee
China Academy of Telecommunications Technology CATT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Academy of Telecommunications Technology CATT filed Critical China Academy of Telecommunications Technology CATT
Publication of WO2017084545A1 publication Critical patent/WO2017084545A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0056—Systems characterized by the type of code used
    • H04L1/0061—Error detection codes
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M11/00—Telephonic communication systems specially adapted for combination with other electrical systems
    • H04M11/06—Simultaneous speech and data transmission, e.g. telegraphic transmission over the same conductors

Definitions

  • the present disclosure relates to the field of communications technologies, and in particular, to a method and system for voice packet loss compensation.
  • VoIP Voice-over-IP
  • VoIP a technology service that carries voice over IP networks
  • VoIP uses IP networks to transport packetized voice data. Due to the wide distribution of IP networks and the low cost of VoIP telephony, multimedia communication, voice and data integration can be realized. Therefore, VoIP has developed rapidly in recent years and is gradually replacing traditional circuit-switched networks (such as Public Switched Telephone Network, PSTN-Public Service Telephone). The trend of the network).
  • IP networks are mainly used for transmitting data services, they use a best-effort connectionless transmission technology, and thus there is no corresponding quality of service guarantee.
  • packet loss compensation technology is generally used to reconstruct the lost voice packet, and the anti-network delay jitter technology is applied to eliminate the adverse effects caused by the delay jitter.
  • the purpose of the PLC (Packet Loss Concealment) algorithm is to generate synthesized speech that can replace the lost speech.
  • the ideal synthesized speech should have the same timbre and spectral characteristics as the lost speech. Therefore, the most common method is to use the historical data before losing the voice data to generate a reasonable approximate speech. If the lost data length is not too long, and the lost speech segment is not located in the area where the speech changes rapidly, the speech compensated by the PLC algorithm sounds quite natural.
  • Receiver-based PLC technologies proposed in some existing standards include ANSI TI.521 (Annex B), ITU-T Rec. G.711 (Appendix I), ITU-T Rec. G.722 (Appendix III), etc. .
  • the commonality of several PLC algorithms in the above mentioned standards is the two parts of pitch detection and synthesized speech (as shown in Figure 1), in which the pitch period detection is only performed when the first packet loss compensation after packet loss is found, if When the packet loss compensation is continuously performed, the pitch period detected for the first time is used. Since the calculation of the pitch period is complicated, it is generally implemented in two steps: first, the downsampling historical data (original signal or residual signal) is subjected to a rough estimation of the pitch period, and then in the rough estimated pitch period. Nearly finer to find a more accurate pitch period.
  • a frame of speech approximated to the original speech is synthesized, and a smooth transition is required between the real speech data and the synthesized speech data or between the synthesized speech and the synthesized speech to avoid sharp noise and continuous packet loss.
  • the decrement of energy is used to reduce the correlation between the back frame speech and the previous frame speech.
  • the existing PLC methods only use the historical data before the lost packet to compensate for the lost voice packet data, so that the compensation effect is not optimal, especially in the transition from the pitch to the pitch, the compensation effect is not ideal.
  • the present disclosure provides a method and system for voice packet loss compensation, which solves the problem that the compensation effect of voice packet loss compensation in the related art is not satisfactory.
  • a method for voice packet loss compensation comprising: when a lost voice packet is first discovered (ie, when packet loss compensation is performed for the first time), according to a history before a lost voice packet Voice data, generating a first voice sequence vec1; if the next packet voice data after the lost voice packet can be acquired, generating a second voice sequence vec2 according to the next packet voice data after the lost voice packet; according to the first voice
  • the sequence vec1 and the second speech sequence vec2 perform packet loss compensation.
  • the method further includes: performing packet loss compensation according to the first voice sequence vec1.
  • the generating, according to the historical voice data before the lost voice packet, the first voice sequence specifically: performing a pitch estimation on the historical voice data to obtain a pitch rough estimation result; and performing a pitch sound according to the pitch rough estimation result Searching, determining a first pitch period Lag1; generating the first speech sequence vec1 from the historical voice data according to the first pitch period Lag1.
  • the finding the second voice sequence vec2 that is in phase alignment with the first voice sequence vec1 from the next packet of voice data after the lost voice packet includes: according to the first pitch period Lag1, Performing a fine pitch search in the next packet voice data, determining a second pitch period Lag2 of the next packet voice data; finding the first voice sequence in the next packet voice data The most relevant position of the column vec1;
  • the second speech sequence vec2 is determined from the next packet of speech data based on the most correlated position and the second pitch period Lag2.
  • the method before performing packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2, the method further includes: calculating the first speech sequence vec1 and the second speech sequence vec2 Normalized cross-correlation value; comparing the normalized cross-correlation value with a set threshold; if the normalized cross-correlation value is greater than the set threshold, then entering according to the Step of performing packet loss compensation on a speech sequence vec1 and the second speech sequence vec2; if the normalized cross-correlation value is less than or equal to the set threshold value, entering according to the first speech sequence vec1 The step of performing packet loss compensation.
  • the performing packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2 further includes: calculating a length of padding with data of a numPitch pitch length of avgLag and an actual padding length a ratio of the difference to the average pitch period length avgLag; determining whether there is a pitch doubling according to the ratio; if so, regenerating the second speech sequence vec2 according to the corrected pitch length; if not, according to the padded pitch data Number, which calculates the length of each pitch data to be filled.
  • the smoothing first voice sequence is used according to the length of each of the pitch data
  • the column vec1 and the second speech sequence vec2 are combined into a speech sequence of a pitch period length, and specifically include: determining a synthesis time according to a distance relationship between the positions of the to-be synthesized speech and the first speech sequence vec1 and the second speech sequence vec2 a weight w1 of the first speech sequence vec1 and a weight w2 of the second speech sequence vec2; utilizing smoothing according to the first speech sequence vec1 and its weight w1, and the second speech sequence vec2 and its weight w2
  • the following speech sequences vec1 and vec2 are combined to obtain a speech sequence with a pitch period length.
  • Vec1 and the second speech sequence vec2 are sequentially synthesized into other speech sequences. Each time a data with a long pitch period is synthesized, it will check whether the lost data packet arrives, and if so, perform a smooth connection, if otherwise continue to compensate by the above method.
  • a system for voice packet loss compensation comprising: a first voice sequence module, configured to: when a lost voice packet is first discovered, according to a history before a lost voice packet The voice data is generated to generate a first voice sequence vec1, and the second voice sequence module is configured to generate a second voice sequence according to the next packet voice data after the lost voice packet if the next packet voice data after the lost voice packet can be acquired a compensation module, configured to perform packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2.
  • the compensation module is further configured to perform packet loss compensation according to the first voice sequence vec1 if the next packet voice data cannot be obtained.
  • the first speech sequence module is specifically configured to perform a pitch estimation on the historical voice data to obtain a pitch rough estimation result; and according to the pitch rough estimation result, perform a pitch fine search to determine a first pitch period Lag1;
  • the first pitch period Lag1 generates the first speech sequence vec1 from the historical voice data.
  • the second voice sequence module is specifically configured to: find a second voice sequence vec2 that is phase-aligned with the first voice sequence vec1 from the next packet voice data after the lost voice packet.
  • the second voice sequence module specifically includes:
  • a first unit configured to perform a fine pitch search in the next voice data according to the first pitch period Lag1, and determine a second pitch period Lag2 of the next voice data;
  • a second unit configured to find, in the next packet voice data, a location that is most correlated with the first voice sequence vec1;
  • a third unit configured to determine the second speech sequence vec2 from the next packet voice data according to the most relevant position and the second pitch period Lag2.
  • the second voice sequence module further includes:
  • a fourth unit configured to calculate a normalized cross-correlation value of the first speech sequence vec1 and the second speech sequence vec2;
  • a fifth unit configured to compare the normalized cross-correlation value with a set threshold; if the normalized cross-correlation value is greater than the set threshold, triggering the compensation module according to the The first speech sequence vec1 and the second speech sequence vec2 perform packet loss compensation; if the normalized cross-correlation value is less than or equal to the set threshold value, triggering the compensation module according to the first The speech sequence vec1 performs packet loss compensation.
  • the compensation module specifically includes:
  • a sixth unit configured to calculate, according to the first speech sequence vec1 and the second speech sequence vec2, the number of pitch data to be filled;
  • a seventh unit configured to calculate a length of each pitch data to be filled according to the number of the padded pitch data
  • An eighth unit configured to perform smoothing processing on the first speech sequence vec1 and the second speech sequence vec2, respectively;
  • an eighth unit configured to synthesize a speech sequence of a pitch period length by using the smoothed first speech sequence vec1 and the second speech sequence vec2 according to the length of each of the pitch data.
  • the compensation module further includes:
  • a tenth unit configured to calculate a ratio of a difference between a length of a padded numPitch pitch period of avgLag and an actual padding length and an average pitch period length avgLag; determining whether a pitch doubling exists according to the ratio; And triggering the second speech sequence module to regenerate the second speech sequence vec2 according to the corrected pitch; if not, triggering the seventh unit according to the The number of padded pitch data is calculated, and the length of each pitch data to be filled is calculated.
  • the ninth unit is further configured to: determine, according to a distance relationship between the positions of the to-be-synthesized speech and the first speech sequence vec1 and the second speech sequence vec2, the first speech sequence vec1 during synthesis Weight w1 and weight w2 of the second speech sequence vec2; according to the first speech sequence vec1 and its weight w1, and the second speech sequence vec2 and its weight w2, the smoothed first speech sequence vec1 Synthesizing with the second speech sequence vec2 to obtain a speech sequence of a pitch period length, wherein if the synthesized speech period length speech sequence is filled, and the lost speech data packet has not arrived, only the smoothed first speech needs to be used again.
  • the sequence vec1 and the second speech sequence vec2 are sequentially synthesized into other speech sequences, and each time a data with a long pitch period is synthesized, it is checked whether the lost data packet arrives, and if so, smoothing is performed, if otherwise, the above method is used for compensation.
  • a system for voice packet loss compensation comprising: a memory, and a processor for reading a program in the memory, performing the following process: when the first lost voice packet is found Generating a first speech sequence vec1 according to historical voice data before the lost voice packet; if acquiring the next packet voice data after the lost voice packet, generating a second speech sequence according to the next packet of voice data after the lost voice packet Vec2; performing packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2.
  • the beneficial effects of the present disclosure are: in the case of discovering the current data packet loss in the embodiment of the present disclosure, if the next voice packet of the lost packet cannot be obtained, the packet loss compensation is performed by using the historical data before the lost packet; When the next voice packet of the lost packet can be obtained, the packet loss compensation is performed by using the data before and after (bidirectional) of the lost packet to improve the quality of the synthesized voice. And at the same time of packet loss compensation, the pitch double phenomenon can be detected and corrected at a small cost. In the case of discovering packet loss, each time the method is called, only one pitch period length data is compensated, so that the fine compensation length can be more reasonable to use the packet data when receiving the corresponding delay packet in the process. Do packet loss compensation.
  • FIG. 1 is a schematic flow chart of a packet loss compensation algorithm in the related art
  • FIG. 2 is a flowchart of a method for voice packet loss compensation in some embodiments of the present disclosure
  • FIG. 3 is a flowchart of a method for voice packet loss compensation in some embodiments of the present disclosure
  • FIG. 4 is a schematic diagram of a compensation method in some embodiments of the present disclosure.
  • 5 is a schematic diagram of smoothing vec1 in some embodiments of the present disclosure.
  • FIG. 6 is a schematic diagram of a cosine window function waveform in some embodiments of the present disclosure.
  • FIG. 7 is a schematic diagram of speech synthesis in some embodiments of the present disclosure, wherein the left side is stretched and the right side is compressed;
  • Figure 8 is a schematic diagram of a waveform with no compensation for missing 20ms data
  • Figure 9 is a schematic diagram of a waveform in which only 20ms of data is lost using historical data compensation
  • Figure 10 is a schematic diagram of a waveform compensated by bidirectional data for missing 20ms data
  • Figure 11 is a schematic diagram of the original waveform without packet loss
  • 13 is a system block diagram of voice packet loss compensation in some embodiments of the present disclosure.
  • FIG. 2 a method for voice packet loss compensation in some embodiments is shown. The specific steps are as follows:
  • Step S201 When the lost voice packet is found for the first time, the first voice sequence vec1 is generated according to the historical voice data before the lost voice packet, and then proceeds to step S203.
  • step S201 when the lost voice packet is found for the first time (that is, when the packet loss compensation is performed for the first time), the historical voice data is subjected to a pitch rough estimation to obtain a pitch rough estimation result; according to the pitch rough estimation result. Performing a pitch search to determine a first pitch period Lag1; generating the first speech sequence vec1 from the historical voice data according to the first pitch period Lag1.
  • Step S203 If the next packet of voice data after the lost voice packet can be acquired, the second voice sequence vec2 is generated according to the next packet voice data after the lost voice packet, and then proceeds to step S205.
  • step S203 finding a second speech sequence vec2 that is phase-aligned with the first speech sequence vec1 from the next packet of speech data after the lost speech packet, specifically includes: according to the first pitch period Lag1, performing a fine pitch search in the next packet of voice data, determining a second pitch period Lag2 of the next packet of voice data; finding and the first in the next packet of voice data The position where the speech sequence vec1 is most strongly correlated; the second speech sequence vec2 is determined from the next packet speech data according to the most relevant position and the second pitch period Lag2.
  • the second packet loss compensation data is output according to the first voice sequence vec1, and packet loss compensation is performed.
  • Step S205 Perform packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2.
  • step S205 the number of pitch data to be filled is calculated according to the first speech sequence vec1 and the second speech sequence vec2; and the length of each pitch data to be filled is calculated according to the number of padded pitch data; Performing smoothing processing on the first speech sequence vec1 and the second speech sequence vec2 respectively; synthesizing a pitch of a pitch period length by using the smoothed first speech sequence vec1 and the second speech sequence vec2 according to the length of each pitch data sequence.
  • the smoothed first speech sequence vec1 and the second speech sequence vec2 are used to form a speech sequence of a pitch period length by using waveform interpolation. It can be understood that the specific interpolation algorithm is not limited in this embodiment.
  • the number of pitch data to be filled is calculated, which specifically includes: according to the length d of the lost voice packet, the first speech sequence.
  • a speech sequence of a pitch period length is synthesized, specifically including: a distance from the first speech sequence vec1 and the second speech sequence vec2 according to the position of the speech to be synthesized.
  • the current packet loss is found, if the next voice packet of the lost packet cannot be obtained, the historical data before the lost packet is used for packet loss compensation; if the next packet of the lost packet can be obtained.
  • packet loss compensation is performed using data before and after (two-way) loss of packets to improve the quality of synthesized speech. And at the same time of packet loss compensation, the pitch double phenomenon can be detected and corrected at a small cost.
  • each time the method is called only one pitch period length data is compensated, so that the fine compensation length can be more reasonable to use the packet data when receiving the corresponding delay packet in the process. Do packet loss compensation.
  • FIG. 3 a flow chart of using a two-way data packet loss compensation algorithm is shown, and the specific steps are as follows:
  • the pitch data is firstly estimated by using the downsampled historical data.
  • the purpose of the downsampling is to reduce the amount of calculation, for example, the autocorrelation method is used for downsampling, wherein the data can be reduced.
  • Sampling to 4K the autocorrelation function is calculated as follows:
  • N is the length of the sequence x(i) and p is the number of autocorrelation values to be calculated.
  • a speech sequence (vec1, generated by traversing Lag1 samples from the latest sample in the historical data) is generated from the historical data based on the found pitch period to synthesize the final output speech.
  • the speech sequence synthesized by the previous step is directly used to synthesize the output speech sequence; if the next packet data of the lost packet exists, the pitch period fine search is performed.
  • the target of the new pitch cycle search is locked near Lag1, the search is centered on Lag1, and the search range is positive and negative Lag1/2.
  • the maximum autocorrelation value is found in this range by autocorrelation method to determine the new pitch period: Lag2 .
  • a speech sequence vec2 aligned with vec1 is found from the next packet data of the lost packet (hereinafter referred to as: new data).
  • new data the next packet data of the lost packet
  • vec1 the latest data in the speech buffer
  • the Lag1 samples are traced forward, and the length is Lag1; alternatively, the cross-correlation method is used (the calculation method of the cross-correlation function is shown in Equation 4-2)
  • the new data Next packet data in Figure 4) Find the position with the strongest correlation with vec1 (the position k corresponding to the maximum value in corr(k)), and take out the Lag2 long data, and record the sequence as vec2.
  • the purpose of finding the starting point of vec2 according to the strongest correlation principle is to align the two sequences in phase.
  • the normalized cross-correlation values of vec1 and vec2 are calculated (the calculation method of the normalized cross-correlation function is shown in Equation 4-3), and compared with a set threshold, the correlation between vec1 and vec2 is compared. In case of strong (greater than the threshold), the two-way data is used for packet loss compensation, otherwise only the historical data is used for packet loss compensation.
  • the length rd determined by the phase alignment, Lag1 and Lag2 calculate the number of pitches to be filled, and judge whether the length of the previously determined pitch period is reasonable according to the number of fillings.
  • step 1 If the ratio falls between 0.4 and 0.6, then it is proved that the previously determined pitch period length is likely to be a doubled pitch period.
  • step 2 To determine this possibility, in the three candidate pitch period values mentioned in step 1. Look for the value to fall in [avgLag/2-5, avgLag/2+5], update Lag1 with this value if any, and the program jumps to step 2 to re-execute; if it does not exist, That means there is a problem with the pitch period search or correlation detection. In order to avoid the problem from continuing to deteriorate, only the historical data is used for packet loss compensation. If the ratio falls within [0.2, 0.4] or [0.6, 0.8], the pitch period is explained. There is a problem with the search or correlation detection. In order to avoid the problem from continuing to deteriorate, only the historical data is used for packet loss compensation. If the ratio falls within [0, 0.2] or [0.8, 1], the pitch period search is correct, continue below. A step of.
  • step 6 the length of each pitch to be inserted is calculated. Since the length of the pitch period is gradual, the length of the pitch period before and after the packet loss is likely to be different. In order to be closer to the change of the period length in practice, we also need to change the length of the pitch period of the padding, and the length after the change. And Rd is equal. The specific practices are as follows:
  • len is greater than or equal to rd, it must be handled in two cases:
  • Lag2 is greater than or equal to Lag1. Decrease the length of the compensated first pitch by one by one. If the last pitch is not reduced and len is equal to rd, repeat the above process from the first pitch until len is equal to rd.
  • Lag2 is smaller than Lag1.
  • the length is decremented by one by one. If the first pitch is reduced and len is not equal to rd, then the above process is repeated from the last pitch until len is equal to rd.
  • len is less than rd, it must be handled in two cases.
  • Lag2 is greater than or equal to Lag1.
  • the length of the last pitch is incremented by one. If the first pitch is added, the len is equal to rd, and then the above process is repeated from the last pitch until len is equal to rd.
  • Lag2 is smaller than Lag1. Add 1 to their length from the first pitch of the compensation. If the last pitch cannot be added to let len be equal to rd, repeat the above process from the first pitch until len is equal to rd.
  • step 7 smoothing vec1 and vec2. Because the pitch data to be inserted later is interpolated with vec1 and vec2. If vec1 or vec2 are not smoothed between the beginning and the end, then the discontinuity will occur between the interpolated pitch data. If it is serious, it will be sharp. The noise appears. Let's take vec1 as an example to explain how to do smoothing.
  • step 6 data with a long pitch period is generated.
  • new data is synthesized in turn according to the length of each pitch calculated in step 6.
  • the method of synthesis can be selected by waveform interpolation, as follows:
  • the original length be L and the target length be N.
  • L is equal to N
  • the cosine window function is as follows:
  • Stretching processing (when L is less than N): Generate a cosine window of length 2 ⁇ L, and the left/right boundary of the left/right half cosine window is aligned with the left/right boundary of the original sequence, added to the original sequence, and the data p1 is obtained. /p2.
  • the left boundary of p1 is aligned with the left boundary of the target sequence (Fig. 7 left a)
  • the right boundary of p2 is aligned with the right boundary of the target sequence (Fig. 7 left b)
  • the corresponding points of p1 and p2 are added (Fig. 7 left c) to get the final synthesized speech.
  • Compression processing (when L is greater than N): generates a cosine window of length 2*N, and the left/right boundary of the left/right half cosine window is aligned with the left/right boundary of the original sequence, added to the original sequence, and the data is obtained. P1/p2.
  • the left boundary of p1 is aligned with the left boundary of the target sequence (Fig. 7 right a)
  • the right boundary of p2 is aligned with the right boundary of the target sequence (Fig. 7 right b)
  • the corresponding points of p1 and p2 are added (Fig. 7 right c) to get the final synthesized speech.
  • the respective weights w1 and w2 are calculated.
  • the calculation of the weight is based on the distance relationship between the position of the speech to be synthesized and vec1 and vec2. The principle is which sequence of synthesized speech is closer to which sequence its waveform should be closer.
  • the experimental data of packet loss compensation under the condition of strong data correlation before and after the lost packet is shown, and the experimental condition is that the packet length is 20 ms and the packet loss rate is 5%. As shown in Figure 8, the middle part is the case of losing one packet.
  • the algorithm finds that there is a packet loss situation, it will perform packet loss compensation as follows:
  • Equation 4-1 First take 30 milliseconds of data from the buffer BUFFER, downsample the data to 4K, and find 50 autocorrelation values for the downsampled data using Equation 4-1, where N takes 60 samples (60 samples) Point At the 4K sampling rate, corresponding to the maximum pitch period length), and find the largest three of them, the corresponding k value corresponds to the number of samples corresponding to the rough estimated pitch period. Then map the three k values to the corresponding position k' under the original sampling rate, and in the vicinity of k' (within the range of 0.5ms left and right), optionally calculate the autocorrelation value using Equation 4-1, and calculate the range.
  • the position corresponding to the maximum value is set to k", and the maximum value among the three candidate k" is the final gene cycle, which is set to Lag1. And trace the Lag1 samples from the latest sample in the buffer BUFFER, and take out the speech sequence vec1.
  • step 3 phase alignment of vec1 and vec2, vec1 moves Lag2 times point by point in the new data, and calculates the cross-correlation value of vec1 and corresponding vec2 every time it moves, according to formula 4-2 Calculate Laco2 cross-correlation values, where N is MIN (Lag1, the length of the data packet LEN-Lag2), and finally find the largest one from the Lag2 cross-correlation values, and take Lag2 sample data from the corresponding position of the maximum value as The final sequence vec2.
  • N is MIN (Lag1, the length of the data packet LEN-Lag2)
  • step 7 Smooth the sequence according to the method described in step 7, smoothing the length of the region: 5*samplerate/8000 samples, where samplerate is the sampling rate of the data.
  • a system block diagram of voice packet loss compensation in a third embodiment the system 13 comprising:
  • the first voice sequence module 131 is configured to generate a first voice sequence vec1 according to historical voice data before the lost voice packet when the lost voice packet is first found;
  • the second voice sequence module 133 is configured to: if the next packet voice data after the lost voice packet can be acquired, generate a second voice sequence vec2 according to the next packet voice data after the lost voice packet;
  • the compensation module 135 is configured to perform packet loss compensation according to the first speech sequence vec1 and the second speech sequence vec2.
  • the compensation module 135 is further configured to perform packet loss compensation according to the first voice sequence vec1 if the next packet voice data cannot be obtained.
  • the first speech sequence module 131 is specifically configured to perform a pitch estimation on the historical speech data to obtain a pitch rough estimation result; and according to the pitch rough estimation result, perform a fine pitch search to determine a first pitch period Lag1;
  • the first pitch period Lag1 generates the first speech sequence vec1 from the historical voice data.
  • the second speech sequence module 133 is specifically configured to: find a second speech sequence vec2 that is in phase alignment with the first speech sequence vec1 from the next packet of speech data after the lost speech packet.
  • the second speech sequence module 133 specifically includes: a first unit, configured to perform a fine pitch search in the next voice data according to the first pitch period Lag1, and determine the next voice data. a second pitch period Lag2; a second unit, configured to find a position that is most correlated with the first voice sequence vec1 in the next packet voice data; and a third unit, configured to use the correlation according to the correlation The strongest position and the second pitch period Lag2 determine the second speech sequence vec2 from the next packet of speech data.
  • the second speech sequence module further includes: a fourth unit, configured to calculate a normalized cross-correlation value of the first speech sequence vec1 and the second speech sequence vec2; Comparing the normalized cross-correlation value with a set threshold; if the normalized cross-correlation value is greater than the set threshold, triggering the compensation module according to the first speech sequence vec1 Performing packet loss compensation with the second voice sequence vec2; if the normalized cross-correlation value is less than or equal to the set threshold, triggering the compensation module to perform packet loss according to the first voice sequence vec1 make up.
  • a fourth unit configured to calculate a normalized cross-correlation value of the first speech sequence vec1 and the second speech sequence vec2; Comparing the normalized cross-correlation value with a set threshold; if the normalized cross-correlation value is greater than the set threshold, triggering the compensation module according to the first speech sequence vec1 Performing packet loss compensation with the second voice sequence vec
  • the compensation module 135 specifically includes: a sixth unit, configured to calculate, according to the first speech sequence vec1 and the second speech sequence vec2, the number of pitch data to be filled; the seventh unit, Calculating a length of each of the pitch data to be filled according to the number of the padded pitch data; an eighth unit, configured to the first speech sequence vec1 and the second speech sequence vec2 Performing a smoothing process separately; a ninth unit, configured to separately synthesize a pitch period length of the smoothed first speech sequence vec1 and the second speech sequence vec2 (eg, using waveform interpolation) according to the length of each of the pitch data Speech sequence.
  • a sixth unit configured to calculate, according to the first speech sequence vec1 and the second speech sequence vec2, the number of pitch data to be filled
  • the seventh unit Calculating a length of each of the pitch data to be filled according to the number of the padded pitch data
  • an eighth unit configured to the first speech sequence vec1 and the second speech sequence
  • the seventh unit is specifically configured to: determine, according to the length d of the lost voice packet, the length rd determined by the phase alignment of the first voice sequence vec1 and the second voice sequence vec2, the first pitch
  • the compensation module 135 further includes: a tenth unit, configured to calculate a ratio of a difference between the length of the padding with numPitch pitch period avgLag and the actual padding length and an average pitch period length avgLag; Determining whether there is a pitch doubling according to the ratio; if yes, triggering the second speech sequence module to regenerate the second speech sequence vec2; if not, triggering the seventh unit to calculate the filling according to the number of the filled pitch data The length of each pitch data.
  • the ninth unit is further configured to: determine, according to a distance relationship between the positions of the to-be-synthesized speech and the first speech sequence vec1 and the second speech sequence vec2, the first speech sequence vec1 during synthesis
  • the weight w1 and the weight w2 of the second speech sequence vec2 are combined to obtain a speech sequence of a pitch period length according to the first speech sequence vec1 and its weight w1, and the second speech sequence vec2 and its weight w2.
  • a smoothed first speech sequence vec1 and a second speech sequence vec2 are combined to obtain a pitch sequence length speech sequence, wherein if the synthesized pitch period length speech sequence is filled, If the lost voice data packet has not arrived yet, it is only necessary to use the smoothed first speech sequence vec1 and the second speech sequence vec2 (for example, using waveform interpolation) to sequentially synthesize other speech sequences, each time synthesizing a pitch period. After the long data, it will check whether the lost data packet arrives, and if so, smooth the connection, if otherwise continue to compensate with the above method.
  • the packet loss compensation system is applied in a play thread, and the play thread performs 10 milliseconds of voice data playback every 10 milliseconds.
  • the new The data is related to the historical data (including the compensated data) for the strongest position determination, and then the smooth connection is made at the most relevant position to ensure a smooth transition at the boundary, and then the processed data is copied to the play buffer. Play. If the last operation is not packet loss compensation, copy the decoded data to the play buffer for playback; when WantedTS is smaller than CurrTS, it indicates that the currently needed data packet does not exist in the voice packet buffer. It may be really lost, or it may be because the delay has not arrived at the specified time.
  • the packet loss compensation algorithm is called to perform the replenishment operation, and one gene period is compensated once.
  • Data after the compensation is completed, judge whether the data in the voice buffer is enough for 10 milliseconds. If it is enough, play it. Otherwise, the program jumps to the beginning and the voice packet needs to be detected according to the time stamp. If it arrives, it will be used as described above. The delay packet performs a smooth transition, and if it does not arrive, the packet loss compensation continues until the data in the voice buffer is played for 10 milliseconds.
  • the disclosed method and apparatus may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • each functional unit in various embodiments of the present disclosure may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium.
  • the above software functional units are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform part of the steps of the methods described in various embodiments of the present disclosure.
  • the foregoing storage medium includes: a USB flash drive, A removable medium such as a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本公开提供了一种语音丢包补偿的方法及系统，该方法包括：当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2，根据第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。

Description

一种语音丢包补偿的方法及系统
相关申请的交叉引用
本申请主张在2015年11月19日在中国提交的中国专利申请号No.201510802586.8的优先权，其全部内容通过引用包含于此。
技术领域
本公开涉及通信技术领域，尤其涉及一种语音丢包补偿的方法及系统。
背景技术
Voice-over-IP(VoIP，一种由IP网络传送话音的技术服务)使用IP网络对打包后的语音数据进行传输。由于IP网络分布广泛，VoIP电话费用低廉，可实现多媒体通信、语音与数据的整合，因此近些年来VoIP发展迅猛，正有逐步替代传统电路交换网络(如Public Switched Telephone Network，PSTN-公共服务电话网络)的趋势。鉴于目前IP网络主要用于传输数据业务，其采用一种尽力而为的无连接传输技术，由此没有相应的服务质量保证。当网络出现拥塞时，会导致在接收端有包丢失和延迟抖动的问题出现，这将严重影响接收端的语音质量。因此在VoIP系统的接收端，一般采用丢包补偿技术来重建丢失的语音包，应用抗网络延迟抖动技术消除延迟抖动带来的不良影响。
PLC(Packet Loss Concealment，丢包补偿)算法的目的是为了产生能够替代丢失的语音的合成语音，理想的合成语音应该具有与丢失语音相同的音色和频谱特性。因此最为常用的方法就是利用丢失语音数据前的历史数据产生合理的近似语音。如果丢失的数据长度不是太长，并且丢失语音段不位于语音快速变化的区域，经过PLC算法补偿后的语音听起来还是比较自然的。现有的一些标准中提出的基于接收端的PLC技术有ANSI TI.521(Annex B)，ITU-T Rec.G.711(Appendix I)，ITU-T Rec.G.722(Appendix III)等等。
上面提及的标准中几种PLC算法的共性就是基音检测和合成语音两个部分(如图1所示)，其中基音周期检测只是在发现丢包后的第一次丢包补偿时进行，如果连续做丢包补偿时则用第一次检测出的基音周期。由于检测基音周期的运算复杂度较高，因此一般都会分两步实现：首先对降采样后的历史数据(原始信号或残差信号)进行基音周期粗估，再在粗估出的基音周期附 近细搜出更加精确的基音周期。之后按此长度合成出与原始语音近似的一帧语音，并且要在真实语音数据与合成的语音数据之间或合成语音与合成语音之间要做平滑过渡以免产生尖锐的噪声，同时在连续丢包的时候要通过能量递减的方式来减少后帧语音与前帧语音的相关性。
现有的PLC方法均仅利用丢失包前的历史数据来补偿丢失语音包数据，这样不能达到补偿效果的最优，尤其是在基音到基音的转折处，补偿的效果不太理想。
发明内容
鉴于上述技术问题，本公开提供一种语音丢包补偿的方法及系统，解决相关技术中语音丢包补偿的补偿效果不理想的问题。
依据本公开的一个方面，提供了一种语音丢包补偿的方法，所述方法包括：第一次发现丢失语音包时(即第一次做丢包补偿时)，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
可选地，若不能获取所述下一包语音数据，所述方法还包括：根据所述第一语音序列vec1进行丢包补偿。
可选地，所述根据丢失语音包之前的历史语音数据，生成第一语音序列，具体包括：对所述历史语音数据进行基音粗估，得到基音粗估结果；根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；根据所述第一基音周期Lag1从所述历史语音数据生成所述第一语音序列vec1。
可选地，所述根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2，具体为：从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2。
可选地，所述从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2，具体包括：根据所述第一基音周期Lag1，在所述下一包语音数据中进行基音细搜索，确定出下一包语音数据的第二基音周期Lag2；在所述下一包语音数据中找出与所述第一语音序 列vec1相关性最强的位置；
根据所述相关性最强的位置和第二基音周期Lag2，从所述下一包语音数据中确定出所述第二语音序列vec2。
可选地，在根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿之前，所述方法还包括：计算出所述第一语音序列vec1和所述第二语音序列vec2的归一化互相关值；比较所述归一化互相关值与设定的门限值；若所述归一化互相关值大于所述设定的门限值，则进入根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿的步骤；若所述归一化互相关值小于或等于所述设定的门限值，则进入根据所述第一语音序列vec1进行丢包补偿的步骤。
可选地，所述根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿，具体包括：根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数；根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度；对所述第一语音序列vec1和所述第二语音序列vec2分别进行平滑处理；根据所述各个基音数据的长度，对平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列。
可选地，所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数，具体包括：根据丢失语音包的长度d、所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
可选地，所述根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿，还包括：计算用numPitch个基音长度为avgLag的数据进行填充的长度与实际要填充长度的差值与平均基音周期长度avgLag的比值；根据所述比值判断是否存在基音加倍；若是，则根据校正的基音长度重新生成第二语音序列vec2；若否，则根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度。
可选地，所述根据所述各个基音数据的长度，利用平滑后的第一语音序 列vec1和第二语音序列vec2合成一个基音周期长度的语音序列，具体包括：根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，利用平滑后的语音序列vec1和vec2合成得到一个基音周期长度的语音序列，若填充此合成的一个基音周期长度语音序列后，丢失的数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则进行平滑衔接，若否则继续用上述方法补偿。
依据本公开的另一个方面，还提供了一种语音丢包补偿的系统，所述系统包括：第一语音序列模块，用于当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；第二语音序列模块，用于若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；补偿模块，用于根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
可选地，所述补偿模块，还用于若不能获取所述下一包语音数据，根据所述第一语音序列vec1进行丢包补偿。
可选地，所述第一语音序列模块具体用于对所述历史语音数据进行基音粗估，得到基音粗估结果；根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；根据所述第一基音周期Lag1从所述历史语音数据生成所述第一语音序列vec1。
可选地，所述第二语音序列模块具体用于：从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2。
可选地，所述第二语音序列模块具体包括：
第一单元，用于根据所述第一基音周期Lag1，在所述下一个语音数据中进行基音细搜索，确定出所述下一个语音数据的第二基音周期Lag2；
第二单元，用于在所述下一包语音数据中找出与所述第一语音序列vec1相关性最强的位置；
第三单元，用于根据所述相关性最强的位置和第二基音周期Lag2，从所述下一包语音数据中确定出所述第二语音序列vec2。
可选地，所述第二语音序列模块还包括：
第四单元，用于计算出所述第一语音序列vec1和所述第二语音序列vec2的归一化互相关值；
第五单元，用于比较所述归一化互相关值与设定的门限值；若所述归一化互相关值大于所述设定的门限值，则触发所述补偿模块根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿；若所述归一化互相关值小于或等于所述设定的门限值，触发所述补偿模块根据所述第一语音序列vec1进行丢包补偿。
可选地，所述补偿模块具体包括：
第六单元，用于根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数；
第七单元，用于根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度；
第八单元，用于对所述第一语音序列vec1和所述第二语音序列vec2分别进行平滑处理；
第八单元，用于根据所述各个基音数据的长度，利用平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列。
可选地，所述第六单元具体用于：根据丢失语音包的长度d、所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
可选地，所述补偿模块还包括：
第十单元，用于计算用numPitch个基音周期长度为avgLag的数据进行填充后的长度与实际要填充长度的差值与平均基音周期长度avgLag的比值；根据所述比值判断是否存在基音加倍；若是，触发第二语音序列模块根据校正过的基音重新生成第二语音序列vec2；若否，则触发所述第七单元根据所 述填充的基音数据的个数，计算要填充的各个基音数据的长度。
可选地，所述第九单元进一步用于：根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，对平滑后的第一语音序列vec1和第二语音序列vec2合成得到一个基音周期长度的语音序列，其中若填充此合成的一个基音周期长度语音序列后，丢失的语音数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则进行平滑衔接，若否则继续用上述方法补偿。
依据本公开的另一个方面，还提供了一种语音丢包补偿的系统，包括：存储器，以及处理器，用于读取存储器中的程序，执行下列过程：当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
本公开的有益效果是：在本公开的实施例中在发现当前数据包丢失情况下，如果不能获取丢失包的下一个语音包时，则利用该丢失包前的历史数据进行丢包补偿；如果能获取丢失包的下一个语音包时，则利用丢失包前后(双向)的数据进行丢包补偿，以提高合成语音的质量。并且在丢包补偿的同时，能以较小的代价对基音加倍现象进行检测和纠正。该在发现丢包情况下，每次调用该方法时只补偿出一个基音周期长度的数据，这样精细的补偿长度，为在该过程中收到相应的延迟包时能利用此包数据更合理地做丢包补偿。
附图说明
图1为相关技术中丢包补偿算法流程简图；
图2为本公开的一些实施例中语音丢包补偿的方法流程图；
图3为本公开的一些实施例中语音丢包补偿的方法流程图；
图4为本公开的一些实施例中补偿方法示意图；
图5为本公开的一些实施例中对vec1做平滑处理的示意图；
图6为本公开的一些实施例中余弦窗函数波形的示意图；
图7为本公开的一些实施例中语音合成示意图，其中左侧围伸展情况，右侧围压缩情况；
图8为丢失20ms数据没有补偿的波形的示意图；
图9为丢失20ms数据只利用历史数据补偿的波形的示意图；
图10为丢失20ms数据用双向数据补偿的波形的示意图；
图11为没有丢包的原始波形的示意图；
图12为本公开的语音丢包补偿的系统进行语音丢包补偿的流程图；
图13为本公开的一些实施例中语音丢包补偿的系统框图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
参见图2，图中示出了一些实施例中语音丢包补偿的方法，具体步骤如下：
步骤S201、当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1，然后进入步骤S203。
具体地，在步骤S201中，当第一次发现丢失语音包时(即第一次做丢包补偿时)，对所述历史语音数据进行基音粗估，得到基音粗估结果；根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；根据所述第一基音周期Lag1从所述历史语音数据生成所述第一语音序列vec1。
步骤S203、若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2，然后进入步骤S205。
具体地，在步骤S203中，从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2，具体包括：根据所述第一基音周期Lag1，在下一包语音数据中进行基音细搜索，确定出下一包语音数据的第二基音周期Lag2；在所述下一包语音数据中找出与所述第一 语音序列vec1相关性最强的位置；根据相关性最强的位置和第二基音周期Lag2，从所述下一包语音数据中确定出所述第二语音序列vec2。
可选地，若不能获取下一包语音数据，根据第一语音序列vec1输出第二丢包补偿数据，并进行丢包补偿。
步骤S205、根据第一语音序列vec1和第二语音序列vec2进行丢包补偿。
具体地，在步骤S205中，根据第一语音序列vec1和第二语音序列vec2，计算要填充的基音数据的个数；根据填充的基音数据的个数，计算要填充的各个基音数据的长度；对所述第一语音序列vec1和所述第二语音序列vec2分别进行平滑处理；根据各个基音数据的长度，利用平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列。可选的，利用平滑后的第一语音序列vec1和第二语音序列vec2采用波形内插法合成一个基音周期长度的语音序列。当然可以理解的是，在本实施例中并不限定其具体插值算法。
需要说明的是，上述根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数，具体包括：根据丢失语音包的长度d、所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
需要说明的是，上述根据各个基音数据的长度，合成一个基音周期长度的语音序列，具体包括：根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，对平滑后的第一语音序列vec1和第二语音序列vec2(例如采用波形内插法)合成得到一个基音周期长度的语音序列，其中若填充此合成的一个基音周期长度语音序列后，丢失的语音数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2(例如采用波形内插法)依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则 进行平滑衔接，若否则继续用上述方法补偿。
在本公开的实施例中在发现当前数据包丢失情况下，如果不能获取丢失包的下一个语音包时，则利用该丢失包前的历史数据进行丢包补偿；如果能获取丢失包的下一个语音包时，则利用丢失包前后(双向)的数据进行丢包补偿，以提高合成语音的质量。并且在丢包补偿的同时，能以较小的代价对基音加倍现象进行检测和纠正。该在发现丢包情况下，每次调用该方法时只补偿出一个基音周期长度的数据，这样精细的补偿长度，为在该过程中收到相应的延迟包时能利用此包数据更合理地做丢包补偿。
在一实施例中，参见图3，图中示出了利用双向数据丢包补偿算法流程图，具体步骤如下：
第1步，当第一次进行丢包补偿时，先利用降采样后的历史数据进行基音粗估，降采样的目的是降低运算量，例如采用自相关法进行降采样处理，其中可以对数据降采样到4K，自相关函数的计算方法如下：
Figure PCTCN2016105684-appb-000001
其中，N为序列x(i)的长度，p为要计算的自相关值的个数。
从p个自相关值中找出三个最大值，它们分别对应一个待选基音周期。接着在三个待选的基因周期附近利用原始数据进行基因细搜索(细搜索的方法同样采用自相关法)，并确定出最终的基音周期：Lag1。根据找到的基音周期从历史数据产生出一个语音序列(vec1，产生方法为从历史数据中的最新样点向前追溯Lag1个样点)用来合成最终输出的语音。
第2步，如果丢失包的下一包数据不存在，那么直接利用上一步生成的语音序列合成输出的语音序列；如果丢失包的下一包数据存在，则对其做基音周期细搜索。新的基音周期搜索的目标就锁定在Lag1附近，搜索以Lag1为中心，搜索范围为正负Lag1/2，通过自相关法在此范围内找出最大自相关值来确定新的基音周期：Lag2。
第3步，从丢失包的下一包数据(下称：新数据)中找出一个与vec1相位对齐的语音序列vec2。设历史数据存放在speech buffer中，那么vec1为speech buffer中最新数据向前追溯Lag1个样点，长度为Lag1；可选的，用互相关法(互相关函数的计算方法见式4-2)在新的数据(图4中的Next packet data) 中找出与vec1相关性最强的位置(corr(k)中最大值对应的位置k)，并取出Lag2长数据，记此序列为vec2。根据相关性最强原则找出vec2的起始点的目的是为了让两个序列相位对齐。
Figure PCTCN2016105684-appb-000002
第4步，计算出vec1与vec2的归一化互相关值(归一化互相关函数的计算方法见式4-3)，并与一个设定的门限做比较，在vec1与vec2相关性较强(大于门限)的情况下才利用双向的数据进行丢包补偿，否则只利用历史数据进行丢包补偿。
Figure PCTCN2016105684-appb-000003
第5步，根据丢失数据的长度d(如图3)，相位对齐后确定的长度rd，Lag1和Lag2计算出要填充的基音个数，并根据填充个数判断前面确定的基音周期长度是否合理。方法如下：
1.计算平均基音周期长度avgLag＝(Lag1+Lag2)/2；
2.计算要填充的基音个数numPitch＝(rd+(avgLag/2))/avgLag；
3.计算用用numPitch个长为avgLag的数据进行填充后与实际要填充长度的差值与avgLag的比值：abs(numPitch×avgLag-rd)/avgLag。
如果这个比值落在0.4到0.6之间，那么证明前面确定的基音周期长度很可能是加倍了的基音周期，为了确定这个可能性，在第1步提到的三个待选的基音周期值中寻找是否有值落在[avgLag/2-5，avgLag/2+5]之中，如果有的话就用这个值更新Lag1，并且程序跳转到第2步重新执行；如果不存在这个值，那就说明基音周期搜索或相关性检测出现问题，为了避免问题继续恶化，只利用历史数据进行丢包补偿；如果这个比值落在[0.2，0.4]或[0.6，0.8]之中，说明基音周期搜索或相关性检测出现问题，为了避免问题继续恶化，只利用历史数据进行丢包补偿；如果这个比值落在[0，0.2]或[0.8，1]之中，说明基音周期搜索正确，继续下面的步骤。
第6步，计算要插入的各个基音的长度。因为基音周期长度是渐变的，所以丢包前后的基音周期长度很可能不同，为了与实际中的周期长度的变化更接近，我们对填充的基音周期长度也要做变化，并且变化后的长度之和与 rd相等。具体做法如下：
●将Lag1作为numPitch个要填充的基音的初始值长度，统计一下按目前这种填充方式填充后的总长度len，如果len与rd相等，说明目前的填充方式最优，继续进行第7步。否则进行下面的步骤。
●如果len大于等于rd，要分两种情况处理：
Figure PCTCN2016105684-appb-000004
情况1，Lag2大于等于Lag1。从补偿的第一个基音开始逐个对其长度减1，如果减到最后一个基音还不能让len等于rd，那就再从第一个基音开始重复上面的过程直到len等于rd。
Figure PCTCN2016105684-appb-000005
情况2，Lag2小于Lag1。从补偿的最后一个基音开始倒着向前逐个对其长度减1，如果减到第一个基音还不能让len等于rd，那就再从最后一个基音开始重复上面的过程直到len等于rd。
如果len小于rd，要分两种情况处理
Figure PCTCN2016105684-appb-000006
情况1，Lag2大于等于Lag1。从补偿的最后一个基音开始倒着向前逐个对其长度加1，如果加到第一个基音还不能让len等于rd，那就再从最后一个基音开始重复上面的过程直到len等于rd。
Figure PCTCN2016105684-appb-000007
情况2，Lag2小于Lag1。从补偿的第一个基音开始逐个对其长度加1，如果加到最后一个基音还不能让len等于rd，那就再从第一个基音开始重复上面的过程直到len等于rd。
第7步，对vec1，vec2做平滑处理。因为之后要插入的基音数据都是用vec1和vec2插值得到的，如果vec1或vec2各自首尾之间不做下平滑的话那么插值出的基音数据之间就会出现不连续现象，严重的话会有尖锐的噪声出现。下面以vec1为例，说明一下如何做平滑处理。
如图5所示，取vec1前面一段数据，称为o1，取vec1最新的一段数据，称为o2，o2与o1长度相等。对o1施加三角窗的左半部，对o2施加三角窗的右半部，然后将两段信号重叠相加，结果更新o2区域。
第8步，产生出基音周期长的数据。每次调用PLC算法时，根据第6步计算得到的各个基音的长度依次合成新的数据。合成的方法可选采用波形内插法，具体如下：
●根据目标长度N(要合成的基音的长度)调整vec1和vec2的长度。 调整的方法如下：
设原始长度为L，目标长度为N。当L等于N时不用调整，不相等的时候不能通过变采样的方式进行调整，否则会破坏原始语音的音色，要通过加余弦窗的方式做伸展或压缩处理。参见图6，余弦窗函数如下：
Figure PCTCN2016105684-appb-000008
Figure PCTCN2016105684-appb-000009
伸展处理(当L小于N时)：生成一个长为2×L的余弦窗，左/右半余弦窗左/右边界与原序列的左/右边界对齐，加到原序列上，得到数据p1/p2。合成时把p1的左边界与目标序列的左边界对齐(图7左a)，p2的右边界与目标序列的右边界对齐(图7左b)，然后将p1与p2对应点相加(图7左c)得到最终合成的语音。
Figure PCTCN2016105684-appb-000010
压缩处理(当L大于N时)：生成一个长为2*N的余弦窗，左/右半余弦窗的左/右边界与原序列的左/右边界对齐，加到原序列上，得到数据p1/p2。合成时把p1的左边界与目标序列的左边界对齐(图7右a)，p2的右边界与目标序列的右边界对齐(图7右b)，然后将p1与p2对应点相加(图7右c)得到最终合成的语音。
●按上述方法将vec1和vec2按目标长度调整之后，要计算各自的权重w1和w2。权重的计算根据待合成语音的位置与vec1和vec2的距离关系。原理是合成语音离哪个序列近其波形应该更接近哪个序列。步骤6中得到要插入的基音个数为numPitch，要合成的第i(i＝1，2，...，numPitch)个基音的数据时，相应权重为w2＝i/(numPitch+1)，w1＝1-w2。
●合成一个基音周期长度的语音。vec1乘以权重w1，vec2乘以权重w2，二者相加就得到了最终的合成语音。在连续做丢包补偿时，只要重复执行第8步，依次生成numPitch段合成语音即可。
参见图8～图11，图中示出了丢失包前后数据相关性较强情况下的丢包补偿的实验数据，实验条件为包长20ms，5％的丢包率。如图8所示，中间部分为丢失一包的情况，当算法发现有丢包情况发生时，便按如下步骤进行丢包补偿：
(1)先从缓存BUFFER中取出30毫秒数据，将数据降采样到4K，对降采样后的数据用公式4-1求出50个自相关值，其中N取60个样点(60个样点 在4K采样率下对应最大基音周期长度)，并找出其中最大的三个，相应的k值对应粗估出的基音周期对应的样点数。再把三个k值映射到原采样率下的对应位置k′，在k′附近(左右0.5ms范围内)时，可选的再用公式4-1计算自相关值，此范围内算出的最大值对应的位置设为k″，3个待选的k″中的最大值为最终的基因周期，设为Lag1。并从缓存BUFFER中最新的样本向前追溯Lag1个样点，取出语音序列vec1。
(2)在能获取丢失包下一个包的情况下，我们对拿到的数据包中的数据按步骤2所述方法找出新的基音周期Lag2，同样可选的用公式4-1求自相关值，N取Lag1。
(3)按上述第3步对vec1和vec2进行相位对齐，vec1在新的数据中逐点移动Lag2次，每移动一次都计算vec1和与之对应的vec2的互相关值，按公式4-2计算Lag2个互相关值，其中N取MIN(Lag1，数据包的长度LEN-Lag2)，最后从Lag2个互相关值中找出最大的一个，从此最大值对应位置处取出Lag2个样点数据作为最终的序列vec2。
(4)按上述第4步计算vec1和vec2的归一化互相关值bestCorr。其中N取MIN(Lag1，数据包的长度LEN-Lag2)，k取0。根据测试经验将比较门限取为0.6，当bestCorr大于0.6时认为两个序列相关性强，反之相关性弱。
(5)按上述第5步计算要填充的基音个数，并判断出前面确定的基音周期长度是否合理。
(6)按上述第6步计算要插入的各个基音的长度。
(7)按步骤7所述方法对序列进行平滑，平滑区域长度：5*samplerate/8000个样点，其中samplerate为数据的采样率。
(8)在每次需要合成数据的时候，按上述第8步进行基音周期长的数据的合成。按如上步骤合成出的数据如图10所示，可以看出用本算法补偿出的波形相对于图9更接近于原始波形。
在有一些实施例中，参见图13，图中示出了第三实施例中语音丢包补偿的系统框图，该系统13包括：
第一语音序列模块131，用于当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；
第二语音序列模块133，用于若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；
补偿模块135，用于根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
需要说明的是，补偿模块135，还用于若不能获取所述下一包语音数据，根据所述第一语音序列vec1进行丢包补偿。
需要说明的是，第一语音序列模块131具体用于对所述历史语音数据进行基音粗估，得到基音粗估结果；根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；根据所述第一基音周期Lag1从所述历史语音数据生成所述第一语音序列vec1。
需要说明的是，第二语音序列模块133具体用于：从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2。
需要说明的是，第二语音序列模块133具体包括：第一单元，用于根据所述第一基音周期Lag1，在所述下一个语音数据中进行基音细搜索，确定出所述下一个语音数据的第二基音周期Lag2；第二单元，用于在所述下一包语音数据中找出与所述第一语音序列vec1相关性最强的位置；第三单元，用于根据所述相关性最强的位置和第二基音周期Lag2，从所述下一包语音数据中确定出所述第二语音序列vec2。
需要说明的是，第二语音序列模块还包括：第四单元，用于计算出所述第一语音序列vec1和所述第二语音序列vec2的归一化互相关值；第五单元，用于比较所述归一化互相关值与设定的门限值；若所述归一化互相关值大于所述设定的门限值，则触发所述补偿模块根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿；若所述归一化互相关值小于或等于所述设定的门限值，触发所述补偿模块根据所述第一语音序列vec1进行丢包补偿。
需要说明的是，补偿模块135具体包括：第六单元，用于根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数；第七单元，用于根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度；第八单元，用于对所述第一语音序列vec1和所述第二语音序列vec2 分别进行平滑处理；第九单元，用于根据所述各个基音数据的长度，对平滑过的第一语音序列vec1和第二语音序列vec2(例如采用波形内插法)分别合成一个基音周期长度的语音序列。需要说明的是，所述第七单元具体用于：根据丢失语音包的长度d、所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
需要说明的是，补偿模块135还包括：第十单元，用于计算用numPitch个基音周期长度为avgLag的数据进行填充后的长度与实际要填充长度的差值与平均基音周期长度avgLag的比值；根据所述比值判断是否存在基音加倍；若是，触发第二语音序列模块重新生成第二语音序列vec2；若否，则触发所述第七单元根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度。
需要说明的是，第九单元进一步用于：根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，合成得到一个基音周期长度的语音序列。具体地，对平滑后的第一语音序列vec1和第二语音序列vec2(例如采用波形内插法)合成得到一个基音周期长度的语音序列，其中若填充此合成的一个基音周期长度语音序列后，丢失的语音数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2(例如采用波形内插法)依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则进行平滑衔接，若否则继续用上述方法补偿。
结合图12，本丢包补偿系统应用在播放线程中，播放线程每隔10毫秒进行10毫秒的语音数据播放。本系统中有一个语音包缓冲区用来缓存未被播放的接收到的语音包。通过时间标的关系来判断当前包是否丢失，WantedTS表示当前播放需要的数据包的时间标，CurrTS表示语音包缓存区中的存在的 有效的最小的时间标。当WantedTS与CurrTS相等时，表示当前需要的语音包存在，则对此包数据进行解码并更新WantedTS，这种情况下还要判断上一次的操作是否为丢包补偿，如果是，还要对新的数据与历史数据(包含补偿出的数据)进行相关最强的位置确定，然后在相关性最强的位置处做平滑衔接保证边界处的平滑过渡，再将处理后的数据拷贝到播放缓冲进行播放。如果上一次操作不是丢包补偿，则将解码后的数据拷贝到播放缓冲区中进行播放就可以了；当WantedTS小于CurrTS时，表示当前需要的数据包不存在于语音包缓冲区中，这包可能是真的丢失了，也可能是由于延迟在指定的时间还没到达，不管何种原因，由于播放的实时性都要调用丢包补偿算法进行补包操作，一次补偿出一个基因周期长的数据，补偿完之后判断一下语音缓冲区中的数据是不是够10毫秒，够了则进行播放，否则程序再跳到开始处根据时间标检测需要的语音包到达否，如果到了则按上述方法利用延迟包进行平滑过渡，如果没到则继续进行丢包补偿，直到语音缓冲区中的数据够10毫秒播放为止。
在本申请所提供的几个实施例中，应该理解到，所揭露方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理包括，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、 移动硬盘、只读存储器(Read-Only Memory，简称ROM)、随机存取存储器(Random Access Memory，简称RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述的是本公开的可选实施方式，应当指出对于本技术领域的普通人员来说，在不脱离本公开所述的原理前提下还可以做出若干改进和润饰，这些改进和润饰也在本公开的保护范围内。

Claims (21)

  1. 一种语音丢包补偿的方法，包括：
    当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；
    若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
  2. 根据权利要求1所述的方法，其中，若不能获取所述下一包语音数据，所述方法还包括：
    根据所述第一语音序列vec1进行丢包补偿。
  3. 根据权利要求1所述的方法，其中，所述根据丢失语音包之前的历史语音数据，生成第一语音序列，具体包括：
    对所述历史语音数据进行基音粗估，得到基音粗估结果；
    根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；
    根据所述第一基音周期Lag1从所述历史语音数据生成所述第一语音序列vec1。
  4. 根据权利要求1所述的方法，其中，所述根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2，具体为：
    从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2。
  5. 根据权利要求4所述的方法，其中，所述从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2，具体包括：
    根据所述第一基音周期Lag1，在所述下一包语音数据中进行基音细搜索，确定出所述下一包语音数据的第二基音周期Lag2；
    在所述下一包语音数据中找出与所述第一语音序列vec1相关性最强的位置；
    根据所述相关性最强的位置和第二基音周期Lag2，从所述下一包语音数 据中确定出所述第二语音序列vec2。
  6. 根据权利要求5所述的方法，其中，在根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿之前，所述方法还包括：
    计算出所述第一语音序列vec1和所述第二语音序列vec2的归一化互相关值；
    比较所述归一化互相关值与设定的门限值；
    若所述归一化互相关值大于所述设定的门限值，则进入根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿的步骤；
    若所述归一化互相关值小于或等于所述设定的门限值，则进入根据所述第一语音序列vec1进行丢包补偿的步骤。
  7. 根据权利要求6所述的方法，其中，所述根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿，具体包括：
    根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数；
    根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度；
    对所述第一语音序列vec1和所述第二语音序列vec2分别进行平滑处理；
    根据所述各个基音数据的长度，利用平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列。
  8. 根据权利要求7所述的方法，其中，所述根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数，具体包括：
    根据丢失语音包的长度d、在所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；
    numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
  9. 根据权利要求8所述的方法，其中，所述根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿，还包括：
    计算用numPitch个基音周期长度为avgLag的数据进行填充后的长度与实际要填充长度的差值与平均基音周期长度avgLag的比值；
    根据所述比值判断是否存在基音加倍；
    若是，则要根据校正过的基音重新生成第二语音序列vec2；
    若否，则根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度。
  10. 根据权利要求8所述的方法，其中，所述根据所述各个基音数据的长度，利用平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列，具体包括：
    根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；
    根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，对平滑后的第一语音序列vec1和第二语音序列vec2合成得到一个基音周期长度的语音序列，其中若填充此合成的一个基音周期长度语音序列后，丢失的语音数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则进行平滑衔接，若否则继续用上述方法补偿。
  11. 一种语音丢包补偿的系统，包括：
    第一语音序列模块，用于当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；
    第二语音序列模块，用于若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；
    补偿模块，用于根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
  12. 根据权利要求11所述的系统，其中，所述补偿模块，还用于若不能获取所述下一包语音数据，根据所述第一语音序列vec1进行丢包补偿。
  13. 根据权利要求11所述的系统，其中，所述第一语音序列模块具体用于对所述历史语音数据进行基音粗估，得到基音粗估结果；根据所述基音粗估结果，进行基音细搜索，确定出第一基音周期Lag1；根据所述第一基音周 期Lag1从所述历史语音数据生成所述第一语音序列vec1。
  14. 根据权利要求11所述的系统，其中，所述第二语音序列模块具体用于：从丢失语音包之后的下一包语音数据中找出一个与所述第一语音序列vec1相位对齐的第二语音序列vec2。
  15. 根据权利要求14所述的系统，其中，所述第二语音序列模块具体包括：
    第一单元，用于根据所述第一基音周期Lag1，在所述下一个语音数据中进行基音细搜索，确定出所述下一个语音数据的第二基音周期Lag2；
    第二单元，用于在所述下一包语音数据中找出与所述第一语音序列vec1相关性最强的位置；
    第三单元，用于根据所述相关性最强的位置和第二基音周期Lag2，从所述下一包语音数据中确定出所述第二语音序列vec2。
  16. 根据权利要求15所述的系统，其中，所述第二语音序列模块还包括：
    第四单元，用于计算出所述第一语音序列vec1和所述第二语音序列vec2的归一化互相关值；
    第五单元，用于比较所述归一化互相关值与设定的门限值；若所述归一化互相关值大于所述设定的门限值，则触发所述补偿模块根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿；若所述归一化互相关值小于或等于所述设定的门限值，触发所述补偿模块根据所述第一语音序列vec1进行丢包补偿。
  17. 根据权利要求16所述的系统，其中，所述补偿模块具体包括：
    第六单元，用于根据所述第一语音序列vec1和所述第二语音序列vec2，计算要填充的基音数据的个数；
    第七单元，用于根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度；
    第八单元，用于对所述第一语音序列vec1和所述第二语音序列vec2分别进行平滑处理；
    第九单元，用于根据所述各个基音数据的长度，利用平滑后的第一语音序列vec1和第二语音序列vec2合成一个基音周期长度的语音序列。
  18. 根据权利要求17所述的系统，其中，所述第六单元具体用于：根据丢失语音包的长度d、所述第一语音序列vec1和所述第二语音序列vec2相位对齐后确定的长度rd、所述第一基音周期Lag1和第二基音周期Lag2，采用如下公式计算要填充的基音数据的个数numPitch；numPitch＝(rd+(avgLag/2))/avgLag，其中avgLag表示平均基音周期长度，avgLag＝(Lag1+Lag2)/2。
  19. 根据权利要求18所述的系统，其中，所述补偿模块还包括：
    第十单元，用于计算用numPitch个基音周期长度为avgLag的数据进行填充后的长度与实际要填充长度的差值与平均基音周期长度avgLag的比值；根据所述比值判断是否存在基音加倍；若是，触发第二语音序列模块根据校正过的基音重新生成第二语音序列vec2；若否，则触发所述第七单元根据所述填充的基音数据的个数，计算要填充的各个基音数据的长度。
  20. 根据权利要求18所述的系统，其中，所述第九单元进一步用于：根据待合成语音的位置分别与所述第一语音序列vec1和所述第二语音序列vec2的距离关系，确定合成时所述第一语音序列vec1的权重w1和所述第二语音序列vec2的权重w2；根据所述第一语音序列vec1和其权重w1，以及所述第二语音序列vec2和其权重w2，对平滑后的第一语音序列vec1和第二语音序列vec2合成得到一个基音周期长度的语音序列，其中若填充此合成的一个基音周期长度语音序列后，丢失的语音数据包还未到达，则只需要再次利用平滑后的第一语音序列vec1和第二语音序列vec2依次合成得其它的语音序列，每次合成一个基音周期长的数据后都会去检查丢失的数据包是否到达，若是则进行平滑衔接，若否则继续用上述方法补偿。
  21. 一种语音丢包补偿的系统，包括：
    存储器，以及
    处理器，用于读取存储器中的程序，执行下列过程：
    当第一次发现丢失语音包时，根据丢失语音包之前的历史语音数据，生成第一语音序列vec1；
    若能够获取丢失语音包之后的下一包语音数据，则根据丢失语音包之后的下一包语音数据，生成第二语音序列vec2；
    根据所述第一语音序列vec1和所述第二语音序列vec2进行丢包补偿。
PCT/CN2016/105684 2015-11-19 2016-11-14 一种语音丢包补偿的方法及系统 Ceased WO2017084545A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510802586.8A CN106788876B (zh) 2015-11-19 2015-11-19 一种语音丢包补偿的方法及系统
CN201510802586.8 2015-11-19

Publications (1)

Publication Number Publication Date
WO2017084545A1 true WO2017084545A1 (zh) 2017-05-26

Family

ID=58717321

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105684 Ceased WO2017084545A1 (zh) 2015-11-19 2016-11-14 一种语音丢包补偿的方法及系统

Country Status (2)

Country Link
CN (1) CN106788876B (zh)
WO (1) WO2017084545A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114387989A (zh) * 2022-03-23 2022-04-22 北京汇金春华科技有限公司 语音信号处理方法、装置、系统及存储介质
US20220239414A1 (en) * 2019-10-14 2022-07-28 Huawei Technologies Co., Ltd. Data processing method and related apparatus
CN115954008A (zh) * 2022-12-09 2023-04-11 成都华曜芯科技股份有限公司 一种丢包隐藏中基音周期的计算方法、计算装置及可读介质
CN120783770A (zh) * 2025-09-10 2025-10-14 东南大学 基于改进离散余弦变换域生成对抗模型的语音丢包补偿方法及设备

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110288999B (zh) * 2019-07-02 2020-12-11 腾讯科技（深圳）有限公司 语音识别方法、装置、计算机设备及存储介质
CN110661716B (zh) * 2019-09-16 2022-07-19 锐捷网络股份有限公司 网络丢包的通知方法、监控装置、交换机和存储介质
CN111326166B (zh) * 2020-02-25 2023-04-14 网易（杭州）网络有限公司 语音处理方法及装置、计算机可读存储介质、电子设备
CN111554309B (zh) 2020-05-15 2024-11-22 腾讯科技（深圳）有限公司 一种语音处理方法、装置、设备及存储介质
TWI748594B (zh) * 2020-08-10 2021-12-01 盛微先進科技股份有限公司 可補償斷音的無線接收裝置及其資訊處理方法
CN112634912B (zh) * 2020-12-18 2024-04-09 北京猿力未来科技有限公司 丢包补偿方法及装置
CN113763974B (zh) * 2021-08-31 2023-05-16 易兆微电子(杭州)股份有限公司 一种丢包补偿方法、装置、电子设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1984203A (zh) * 2006-04-18 2007-06-20 华为技术有限公司 对丢失的语音业务数据帧进行补偿的方法
CN103489448A (zh) * 2013-09-03 2014-01-01 广州日滨科技发展有限公司 语音数据处理方法及系统
CN104978966A (zh) * 2014-04-04 2015-10-14 腾讯科技（深圳）有限公司 音频流中的丢帧补偿实现方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101364854B (zh) * 2007-08-10 2011-01-26 北京理工大学 一种基于边信息的语音包丢失恢复方法
WO2012158159A1 (en) * 2011-05-16 2012-11-22 Google Inc. Packet loss concealment for audio codec
CN104751851B (zh) * 2013-12-30 2018-04-27 联芯科技有限公司 一种基于前后向联合估计的丢帧差错隐藏方法及系统

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1984203A (zh) * 2006-04-18 2007-06-20 华为技术有限公司 对丢失的语音业务数据帧进行补偿的方法
CN103489448A (zh) * 2013-09-03 2014-01-01 广州日滨科技发展有限公司 语音数据处理方法及系统
CN104978966A (zh) * 2014-04-04 2015-10-14 腾讯科技（深圳）有限公司 音频流中的丢帧补偿实现方法和装置

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220239414A1 (en) * 2019-10-14 2022-07-28 Huawei Technologies Co., Ltd. Data processing method and related apparatus
US11736235B2 (en) * 2019-10-14 2023-08-22 Huawei Technologies Co., Ltd. Data processing method and related apparatus
CN114387989A (zh) * 2022-03-23 2022-04-22 北京汇金春华科技有限公司 语音信号处理方法、装置、系统及存储介质
CN114387989B (zh) * 2022-03-23 2022-07-01 北京汇金春华科技有限公司 语音信号处理方法、装置、系统及存储介质
CN115954008A (zh) * 2022-12-09 2023-04-11 成都华曜芯科技股份有限公司 一种丢包隐藏中基音周期的计算方法、计算装置及可读介质
CN120783770A (zh) * 2025-09-10 2025-10-14 东南大学 基于改进离散余弦变换域生成对抗模型的语音丢包补偿方法及设备

Also Published As

Publication number Publication date
CN106788876B (zh) 2020-01-21
CN106788876A (zh) 2017-05-31

Similar Documents

Publication Publication Date Title
WO2017084545A1 (zh) 一种语音丢包补偿的方法及系统
CN113612808B (zh) 音频处理方法、相关设备、存储介质及程序产品
JP5284477B2 (ja) 音声データの伝送にエラーがある際のエラー隠蔽方法
CN101325631B (zh) 一种估计基音周期的方法和装置
US10643624B2 (en) Apparatus and method for improved concealment of the adaptive codebook in ACELP-like concealment employing improved pulse resynchronization
CN103854649B (zh) 一种变换域的丢帧补偿方法及装置
US8185384B2 (en) Signal pitch period estimation
CN105741843B (zh) 一种基于延时抖动的丢包补偿方法及系统
WO2009065348A1 (en) Packet loss concealment method and system thereof
KR101648290B1 (ko) 컴포트 노이즈의 생성
JP2014507689A (ja) ピッチ検出方法及び装置
US9773510B1 (en) Correcting clock drift via embedded sine waves
TWI711033B (zh) 用以判定一估計音調滯後之裝置及方法、用以重建包括語音信號之訊框的系統、以及相關電腦程式
WO2008067763A1 (en) A decoding method and device
CN112530450A (zh) 频域中的样本精度延迟识别
WO2010127489A1 (zh) 检测信号延迟的方法、检测装置及编码器
KR20180049182A (ko) 누락 데이터에 대한 사인곡선 보간
WO2010108445A1 (zh) 通道间延迟估计的方法及其装置和编码器
RU2742739C1 (ru) Выбор задержки основного тона
WO2020135610A1 (zh) 音频数据恢复方法、装置及蓝牙设备
CN115064180B (zh) 一种基于挑选峰的连续语音共振峰提取方法
CN101506873B (zh) 开环基音跟踪平滑
CN105788601B (zh) VoLTE的抖动隐藏方法和装置
CN1385014A (zh) 语音同步
Sun et al. Efficient three-stage pitch estimation for packet loss concealment.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16865722

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16865722

Country of ref document: EP

Kind code of ref document: A1