WO2008074249A1 - Procédé, système et appareils utilisés pour supprimer la perte de trame - Google Patents

Procédé, système et appareils utilisés pour supprimer la perte de trame Download PDF

Info

Publication number
WO2008074249A1
WO2008074249A1 PCT/CN2007/071129 CN2007071129W WO2008074249A1 WO 2008074249 A1 WO2008074249 A1 WO 2008074249A1 CN 2007071129 W CN2007071129 W CN 2007071129W WO 2008074249 A1 WO2008074249 A1 WO 2008074249A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
mlt
current
time domain
information
Prior art date
Application number
PCT/CN2007/071129
Other languages
English (en)
Chinese (zh)
Inventor
Zhe Wang
Jun Zhang
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Publication of WO2008074249A1 publication Critical patent/WO2008074249A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0045—Arrangements at the receiver end
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm

Definitions

  • the present invention relates to audio data transmission techniques, and in particular, to a frame loss concealment method, system, encoding apparatus and decoding apparatus for an MLT-based audio codec scheme. Background of the invention
  • the frame loss concealment technique is a technique for concealing or even recovering lost/missed frames, which can alleviate the discomfort caused to the listener due to dropped frames or wrong frames.
  • MLT Modulated Lapped Transform
  • MLT is a coding method that avoids the discontinuity of the boundary of the transform interval and is less sensitive to quantization error. Therefore, more and more applications are applied in audio and video coding schemes.
  • MLT is a linear transform process. The input of each MLT transform is 2N sample points of two adjacent frames in the time domain, and the output is N MLT coefficients. The process of MLT transformation can be described as follows:
  • Step 1 Perform an overlap and add operation on the time domain waveform signal, and use a window function as the weighting coefficient.
  • v(n) w[(N / 2— 1)— n]x[(N /2 - l) - n] + w(N 12 + n)x(N 12 + n) 0 ⁇ n ⁇ N/ 2
  • v(n + N/2) w[(N - 1) - n]x(N + n) - w(n)x[(2N - l) - n] 0 ⁇ n ⁇ N/2
  • w ( n ) ( ( ⁇ + 0 ⁇ 5 ) /2 ⁇ ) is a window function, 0 ⁇ n ⁇ N.
  • ⁇ ( ⁇ ) is the input audio signal.
  • v(n) is the first half of the overlap-added time-domain frame obtained by the overlap-add operation of a certain time-domain frame
  • v(n + N/2) is the overlap obtained by the overlap-add operation of the same time-domain frame. Add the last half of the time domain frame.
  • Step 2 Perform a type IV discrete cosine transform (DCT, Discrete Cosine Transform) on ") to obtain the final MLT frame.
  • DCT Discrete Cosine Transform
  • n is the ordinal number of each data in the overlap-added time domain frame participating in the DCT transform.
  • FIG. 1 is a schematic diagram of the principle of the prior art MLT overlap addition operation mechanism, that is, the calculation of the above step 1.
  • 1 , 2 , 3 , 4 , 5 , 6 represent input time domain audio signals.
  • 1 and 2, 3 and 4, 5 and 6 respectively represent the first frame, the second frame and the third frame of three consecutive frames, each frame has a frame length of N.
  • Each frame is further divided into two subframes of equal length, and the subframe length is N/2.
  • a, b, c, d, e, f respectively represent overlapping and added time domain frames obtained by overlapping and adding the time domain audio signals through the above step 1.
  • a, c, and e are the first half of each overlap-added time domain frame
  • b, d, and f are the second half of each overlap-added time-domain frame.
  • the transmitting end transmits the MLT frame based on the MLT audio encoding to the receiving end via the packet communication network, and the receiving end uses the reverse MLT transform to restore the received MLT frame to the time domain frame, thereby obtaining an actual audio frame currently transmitted by the transmitting end. data.
  • IMLT Inverse Modulated Lapped Transform
  • IDCT inverse discrete cosine transform
  • IDCT Inverse Discrete Cosine Transform
  • u(n) is the overlap-added time domain frame obtained by IDCT.
  • Step 2 The first half of the current MLT frame IDCT transform result is overlapped and added with the last half of the previous MLT frame IDCT transform result, and the window function is used as the weighting coefficient.
  • the formula is as follows:
  • y(n + N/2) w(N/2 + n)u(n)-w[(N/2-l)-n ⁇ i_old[(N/2-l)-n]
  • w(n) is the same window function as the MLT transform
  • y(n) is the output time domain signal
  • u(n) is the overlap-added time domain frame of the current frame IDCT transform
  • u_old(n) is the overlap-added time domain frame of the previous frame IDCT transform.
  • FIG. 2 is a schematic diagram of the principle of the prior art IMLT.
  • a, b, c, d, e, and f are overlapping and added time domain frames obtained by IDCT transform, corresponding to u(n) in step 2.
  • 1, 2, 3, 4, 5, 6 are the output time domain frames, corresponding to y(n) in step 2.
  • a, b, c, d, e, f is known to be 1 2, 3, 4, 5, 6, 3 is calculated by weighting the b and c respectively through the window function and then adding and subtracting, as shown in the figure. 2 solid arrows indicate.
  • the G.722.1 audio coding standard of the International Telecommunication Union Telecommunication Standardization Group defines a frame loss concealment strategy for MLT-based audio coding schemes.
  • Frame loss defined by the standard
  • the hidden strategy is very simple and can be described as follows:
  • FIG. 3 is a diagram showing the effect of the prior art on the lost frame loss of a single frame in the MLT audio coding scheme.
  • the upper waveform is the waveform of the original audio signal
  • the lower is the waveform hidden by the lost frame.
  • the missing frame segment is located at a lower energy center of the waveform. It can be seen that the waveform of the recovered lost frame is very similar to the waveform of the previous frame of the frame, and is far from the actual waveform.
  • FIG. 4 is a diagram showing the effect of the prior art on the frame loss of continuous frame loss in the MLT audio coding scheme.
  • Figure 4 (a) shows the recovered signal and its original audio signal waveform obtained by the G.722.1 hiding strategy when two consecutive frames are dropped.
  • Figure 4 (b) shows that when three consecutive frames are dropped, it is hidden by G722.1.
  • the recovery signal obtained by the strategy and its original audio signal waveform; wherein the original audio signal waveform is above the respective graphs, and below is the recovered waveform after the lost frame is hidden. It can be seen that the MLT frame of the dropped frame is set to 0, so that the recovered signal has a 0 signal point or a 0 signal segment.
  • one of the disadvantages of the frame loss concealment strategy provided by G.722.1 is that it has better concealment effect in the more stable signal segment when dealing with single frame loss, and in the signal segment with large changes, such as signal transition. Segment, its hidden effect is poor.
  • Another disadvantage of the frame loss concealment strategy is that when the continuous frame loss situation is processed, the MLT coefficient of all frame drops is set to 0, and the recovery signal shows a 0 signal, which causes a significant interruption when playing the restored audio. Causes discomfort.
  • the frame loss hiding strategy provided by G.722.1 belongs to the hidden technology based on the decoder.
  • the decoding technology based on the decoding side is a passive hiding, and all the information used to recover the lost frames is obtained by the decoding end itself.
  • the hiding technology based on the encoding end is an active hiding. Some bits containing important information in the frame are generated and transmitted to the decoding end at the encoding end, and the decoding end can be based on the hidden technology based on the decoding end. The better way to recover lost frames is to increase the bandwidth to some extent.
  • the audio coding standard such as G.729.1, involves the active frame dropping technique, that is, the auxiliary information generated by the encoding end of G.729.1 and used for assisting the decoding end to perform frame loss recovery, the auxiliary information includes frame type information, frame energy. Information and frame phase information.
  • the frame type information is a type of five types of audio frames represented by 2 bits, namely: a non-sounding class, a non-sounding transition class, a vocal transition class, a utterance class, and a start class.
  • the hiding algorithm uses the corresponding hidden strategy to hide.
  • the frame dropping technology provided by G.729.1 has better recovery ability for frames with longer lengths, but has no ability to recover short note frames. None of the five categories can identify the presence of a short note frame. In the actual transmission process, short note frames in the audio signal may be lost, and thus have a large influence on the received audio quality.
  • the G729.1 frame loss concealment algorithm is not specifically designed based on the MLT audio coding scheme, so the characteristics of the MLT transform itself are not fully utilized, and the algorithm efficiency and hidden quality are not optimal.
  • the first object of the embodiments of the present invention is to provide a frame dropping method, which can improve the accuracy of recovering errors/drop frames.
  • a second object of the embodiments of the present invention is to provide a frame loss concealment coding apparatus, which can improve the accuracy of recovering a wrong/lost frame.
  • a third object of the embodiments of the present invention is to provide a frame loss concealment decoding apparatus, which can improve the accuracy of recovering a wrong/lost frame.
  • a fourth object of the embodiments of the present invention is to provide a frame loss concealment system, which can improve the accuracy of recovering errors/drop frames.
  • the present invention provides a frame dropping method for storing a historical time domain waveform at a decoding end; the method includes:
  • the encoding end sends the obtained related frame information and the current MLT frame to the decoding end; the decoding end performs error/drop frame detection on the received current MLT frame, and generates error/lost frame information; according to the error/drop frame Information, the related frame information and the historical time domain waveform recover error/drop frame.
  • an embodiment of the present invention provides a frame loss concealment coding apparatus, where the coding apparatus includes an MLT coding unit, a history buffer unit, and an associated frame information generation unit;
  • the MLT encoding unit is configured to perform MLT transformation according to a subsequent frame acquired by the encoding device and a current frame acquired from the history buffer unit, generate an MLT frame, and send the MLT frame, and send the data out to the history buffer unit. a current frame and the preamble frame;
  • the related frame information generating unit is configured to generate related frame information according to a subsequent frame acquired by the encoding device, and the current frame and the pre-frame obtained from the history unit buffer, and send the related frame information.
  • an embodiment of the present invention provides a frame loss concealment decoding apparatus, where the decoding apparatus includes a frame error detector, a frame loss concealing unit, and a decoding waveform history buffer.
  • the decoded waveform history buffer is used to store a historical time domain waveform
  • the frame error detector is configured to perform error/drop frame detection on the current MLT frame, generate error/lost frame information, and send the error/drop frame information to the frame loss concealment unit;
  • the frame loss concealing unit is configured to recover an error according to the error/lost frame information received from the frame error detector, related frame information received from an encoding device, and a historical time domain waveform in the decoded waveform history buffer. / Drop frame.
  • the embodiment of the present invention provides a frame dropping frame.
  • Hidden system the system comprising an encoding device and a decoding device:
  • the encoding device is configured to send the obtained related frame information and the current MLT frame to the decoding device;
  • the decoding device is configured to perform error/drop frame detection on the received current MLT frame, and generate error/drop frame information; according to the error/drop frame information, the related frame information and the historical time domain waveform recovery error / Drop frame.
  • the frame dropping method, system, encoding device and decoding device provided by the embodiments of the present invention provide an active frame dropping technique, which generates relevant frame information at the encoding end and is encoded by the MLT.
  • the MLT frame is sent to the decoder. If the decoder receives the error/drop frame, the original waveform of the error/drop frame can be recovered according to the relevant frame information.
  • FIG. 1 is a schematic diagram showing the principle of a prior art MLT overlap addition operation mechanism.
  • FIG. 2 is a schematic diagram of the principle of the prior art IMLT overlap addition operation.
  • FIG. 3 is a diagram showing the effect of the prior art on the lost frame loss of a single frame in the MLT audio coding scheme.
  • Figure 4 (a) shows the effect of the prior art on the frame loss of two consecutive dropped frames in the MLT audio coding scheme.
  • Figure 4 (b) shows the effect of the prior art on the frame loss of three consecutive dropped frames in the MLT audio coding scheme.
  • FIG. 5 is a flowchart of a method for dropping a frame hiding method according to an embodiment of the present invention.
  • FIG. 6 is a flowchart of a method for deleting a frame hiding method according to a first preferred embodiment of the present invention.
  • FIG. 7 is a flowchart of a method for selecting a hiding policy by a decoding end according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a principle of a hidden policy according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of a method for selecting a hidden frame of a continuous frame loss/error frame according to an embodiment of the present invention.
  • FIG. 10 is a block diagram of a frame dropping system according to an embodiment of the present invention.
  • FIG. 11 is a block diagram showing a first preferred embodiment of a frame dropping concealment coding apparatus according to an embodiment of the present invention.
  • FIG. 12 is a block diagram showing a first preferred embodiment of a frame loss concealment decoding apparatus according to an embodiment of the present invention.
  • FIG. 13 is a flowchart of a method for deleting a frame hiding method according to a second preferred embodiment of the present invention.
  • FIG. 14 is a block diagram showing a second preferred embodiment of a frame dropping concealment coding apparatus according to an embodiment of the present invention.
  • FIG. 15 is a flowchart of a method for deleting a frame hiding method according to a third preferred embodiment of the present invention. Mode for carrying out the invention
  • the core idea of the embodiment of the present invention is to provide an active frame dropping prevention technology, where the encoding end sends the obtained related frame information and the current MLT frame to the decoding end.
  • the decoding end performs error/drop frame detection on the received current MLT frame, and generates error/lost frame information.
  • the decoding end recovers errors according to the error/lost frame information, the related frame information, and the historical time domain waveform/ Drop frames, thus completing frame loss hiding.
  • Embodiments of the present invention are directed to a frame loss concealment scheme for an MLT-based codec scheme.
  • FIG. 5 is a flowchart of a method for dropping a frame hiding method according to an embodiment of the present invention. The specific steps of the method are as follows:
  • Step 501 The encoding end sends the obtained related frame information and the current MLT frame to the decoding end.
  • Step 502 The decoding end performs error/drop frame detection on the current MLT frame, and generates a wrong/lost frame signal.
  • Step 503 When performing frame loss concealment processing, the decoding end recovers the error/drop frame according to the error/lost frame information, combined with the related frame information and the historical time domain waveform.
  • the MLT frame and its associated frame information can be sent in packets to avoid loss of related frame information for frame loss recovery when the frame is lost.
  • the related frame information can be embedded in the preceding frame immediately adjacent to the current frame. Or transmitting in subsequent frames, the relevant frame information is not lost when the current frame is lost; the related frame information may also be transmitted in the enhancement layer in a layered coding system.
  • the specific method of recovering the wrong/lost frame according to the error/drop frame information combined with the relevant frame information and the historical time domain waveform may be pre-stored in the decoding end in a table or logical inference manner, differently.
  • the error/drop frame condition and related frame information correspond to different hidden strategies.
  • the decoding end first selects a corresponding concealment strategy according to the error/lost frame information, combines the relevant frame information, and recovers the error/drop frame according to the concealment strategy and the historical time domain waveform.
  • FIG. 6 is a flowchart of a method for deleting a frame hiding method according to a first preferred embodiment of the present invention.
  • the method adopts the form of packetization, and separately packages and sends the MLT frame and its related frame information to the decoding end.
  • the specific steps of the method are:
  • Step 601 The encoding end performs MLT transformation according to the received subsequent frame and the stored current frame to generate an MLT frame.
  • the current processing frame is saved in the history buffer of the encoding side, and the current processing frame is called the current frame.
  • the subsequent frame is actually the frame received at the current time, which is called the subsequent frame.
  • the current frame is MLT transformed with the subsequent frames to generate MLT encoded data of the current frame, that is, the current MLT frame.
  • the MLT transform is the same as the prior art MLT transform.
  • Step 602 The encoding end generates related frame information of the current MLT frame.
  • the encoding end performs intra-frame correlation detection according to the current frame to generate intra-frame correlation. Sex information; and inter-frame correlation detection according to the current frame and the pre-frame, and generating inter-frame correlation information.
  • the preamble frame is the time domain waveform of the previous frame of the current frame.
  • the preamble frame is also saved in the history buffer of the encoding side. Therefore, the time domain waveform of two frames is saved in the history buffer.
  • the related frame information includes intra-frame correlation information and inter-frame correlation information.
  • the intra-frame correlation refers to the stability of the intra-frame signal. The more stable the pitch is, the stronger the pitch period is, and the stronger the correlation is.
  • Inter-frame correlation consists of two parts, one is the correlation between the first half of the frame and the end of its preamble or preamble, and the other is the intraframe correlation in the first half of the frame. When the pitch of the first half of the frame is very similar to the pitch of the preamble or at least the end of the preamble, and the intraframe correlation of the first half of the frame is also strong, the interframe correlation is considered strong.
  • the intra-frame correlation of the first half of the frame is strong, it is necessary to determine the inter-frame correlation between the first half of the frame and its pre-frame. If the inter-frame correlation between the first half of the frame and its pre-frame is strong, the inter-frame correlation of the frame is strong, otherwise it is weak. If the intra-frame correlation of the first half of the frame is already weak, no subsequent judgment is made, and the inter-frame correlation of the frame is directly considered to be weak.
  • the intra-frame correlation and the inter-frame correlation can be judged by the decision threshold. When the correlation is higher than the threshold, it is judged to be strong, otherwise it is weak.
  • the intra-correlation as the relevant frame information refers to the intra-frame correlation of the present frame.
  • the most straightforward method for generating intra-frame correlation and inter-frame correlation is to rely on autocorrelation and cross-correlation functions to determine intra-frame and inter-frame correlation.
  • the intra-correlation calculation here is applied to the intra-correlation calculation of the current frame and the intra-correlation calculation of the first half of the frame.
  • the auto-correlation of this frame is sought.
  • the autocorrelation function is defined as:
  • Inter-frame correlation can be obtained using a cross-correlation function.
  • the intra-frame correlation of the first half of the frame is obtained by using the above formula, where x(n) is the time domain signal of the first half of the frame. If it is strong, the partial correlation signal of the first half of the frame is cross-correlated with the signal of the same length at the end of the preamble to obtain inter-frame correlation.
  • the definition of the normalized cross-correlation function is:
  • x(n), y(n) respectively represent the partial signal of the first half of the frame and the signal of the same length at the end of the preceding frame. Correlation is considered strong when the self/cross-correlation calculation is above a predefined threshold, and vice versa.
  • the inter-frame correlation can also be calculated as follows: Calculate the pitch at the beginning of the first half of the frame and at the end of the first half of the frame. If they match, the correlation in the first half of the frame is considered strong, otherwise it is weak. When the intra-frame intra-frame correlation is strong, it is compared with the pitch at the end of the pre-frame. If it is consistent, the inter-frame correlation of the frame is strong, and vice versa.
  • the relevant frame information generated in this step may be represented by an intra-frame correlation identifier and an inter-frame correlation identifier, and each flag is represented by a 1-bit binary number.
  • the relevant frame information is composed of 2 bits identified by the intra-frame and inter-frame correlation. These 2 bits represent 4 different combinations. The four combinations are shown in Table 1:
  • Step 603 The encoding end sends the MLT frame and its related frame information to the decoding end.
  • the information sent to the decoding end is an MLT frame and corresponding intra-frame correlation information and inter-frame correlation information of the frame.
  • the MLT frame and related frame information are sent in packets.
  • Step 604 The decoding end performs error/drop frame detection on the received MLT frame, generates error/drop frame information, and determines whether the current MLT frame or the pre-MLT frame is a wrong frame or a dropped frame. If the current MLT frame is an error/drop frame, or the pre-MLT frame is a wrong/drop frame, or the current MLT frame and the pre-MLT frame are both lost frames, step 605 is performed; otherwise, steps 610 and 611 are performed, The current MLT frame is normally decoded; the overlap-added time domain frame obtained by IDCT transformation of the current MLT frame (such as a, b, c... in FIG. 2) and the final time domain frame obtained by IMLT transformation ( Save as 1 , 2, 3... in Figure 2.
  • the purpose of the error/drop frame detection is to check whether the current MLT frame has a frame loss or a wrong frame.
  • the detection result is used to determine the subsequent operation on the current MLT frame.
  • the detection result is also saved to the next MLT frame, as a basis for determining whether the pre-MLT frame is a wrong/lost frame.
  • the frame loss detection function is implemented by Real-Time Transport Protocol (RTP).
  • RTP Real-Time Transport Protocol
  • the voice/audio frames are packed and transmitted.
  • Each RTP packet contains a serial number in the header, and the serial number of the continuously transmitted packets is continuous.
  • the decoder receives a packet, it checks whether the serial number in the packet header is continuous with the sequence number of the previous packet. If there is an interval in the middle, it indicates that the packet is lost. At this time, it is determined that the frame is a dropped frame.
  • the error frame detection is to detect whether there is an error in the frame.
  • This function can be implemented by FEC (Forward Error Correction) technology.
  • FEC Forward Error Correction
  • FEC can be applied in the transport protocol or integrated in the encoding algorithm. After the MLT frame and its associated frame information transmitted by the encoding end to the decoding end, some error correction bits are added. The error correction bits are specially designed and reflected in the error correction bits when an error occurs in the currently transmitted frame. By checking the error correction bits, it can be determined whether the current MLT frame is a wrong frame.
  • Step 605 The decoding end selects a hidden policy according to the error/lost frame information and the related frame information. slightly.
  • the error/drop frame detection result may be represented by a wrong/lost frame error identifier.
  • the error/drop frame error identifier includes a current frame error identifier and a pre-frame error identifier. Both the current frame error identifier and the pre-frame error identifier can be represented by 1 or 0. Therefore, there are several combinations of error/lost frame error identifiers: 01, that is, the pre-MLT frame is the normal frame, and the current MLT frame is the wrong/drop frame; That is, the pre-MLT frame is the wrong/lost frame, and the current MLT frame is the normal frame; 11 , that is, the pre-MLT frame and the current MLT frame are both wrong/lost frames.
  • each time the decoder selects a hidden strategy it generates and updates a previous frame reliability identifier. In order to continuously drop frames, determine the hidden strategy.
  • the previous frame reliability indicator indicates whether the error/drop frame recovered according to the hidden policy selected this time is reliable.
  • the previous frame reliability indicator can also be determined each time the error/drop frame is recovered.
  • the hidden policy is determined according to the previous frame reliability identifier and the current MLT frame related frame information. Determine the way in which hidden strategies use logical judgment.
  • the hidden strategy table preset on the encoding end is checked to determine the hidden policy.
  • the hidden policy table stores hidden policies corresponding to various related frame information.
  • the hidden strategy table preset at the encoding end is checked to determine the hidden policy.
  • Step 606 The decoding end recovers the error/drop frame according to the selected hidden policy and the historical time domain waveform information saved in the decoded waveform history buffer.
  • Step 607 The decoding end stores the recovered error/lost frame into the decoding waveform history buffer.
  • Step 605 The decoding end selects the hidden strategy according to the error/drop frame detection result and the related frame information, which is an important step implemented by the foregoing method. This step is described in detail below.
  • FIG. 7 is a flowchart of a method for selecting a hiding policy by a decoding end according to an embodiment of the present invention. Method package Including the following steps:
  • Step 6051 determining whether the pre-MLT frame and the current MLT frame are both wrong/lost frames; if yes, executing step 6055, determining a hidden policy for the current frame according to the previous frame reliability identifier and the associated frame information of the current MLT frame; Otherwise, step 6052 is performed.
  • step 6055 it is determined whether the pre-MLT frame is an error/drop frame according to the pre-frame error identifier, and whether the current MLT frame is a wrong/lost frame according to the current frame error identifier.
  • step 6055 is performed, and the logical reasoning method is adopted according to the previous frame reliability identifier and the current frame-related frame information. , Select a hidden policy for the current MLT frame. Step 606 in Figure 6 is then performed.
  • Step 6052 Determine whether the current MLT frame is faulty. If yes, go to step 6056 to check the hidden strategy table preset on the encoding end according to the current MLT frame related frame information, and determine the hidden policy. Otherwise, go to step 6053.
  • the hidden policy determining method of step 6056 belongs to the case where the pre-MLT frame is a normal frame but the current MLT frame is a wrong/lost frame.
  • the pre-MLT frame is a lost/error frame but the current MLT frame is a normal frame.
  • the current MLT is a normal frame
  • the IMLT transform uses the overlap-added time domain frame information of the pre-MLT frame, it is also necessary to select a hidden strategy for the current MLT frame.
  • Step 6053 The current MLT frame is subjected to IDCT transformation to obtain an overlap-added time domain frame of the current MLT frame.
  • the overlap-added time domain frame of the current MLT frame obtained in this step is used as part of the historical time domain waveform to recover the error/drop frame based on the hidden strategy and the historical time domain waveform.
  • Step 6054 According to the current MLT frame related frame information and the pre-MLT frame related frame information, check a hidden policy table preset at the encoding end to determine a hidden policy.
  • the hidden strategy is pre-designed and stored on the decoder side.
  • the order of judging various situations in the hidden policy selection process is not limited to the order described above, and it is only necessary to distinguish the various cases without affecting the judgment result.
  • the following examples and attached tables are described.
  • FIG. 8 is a schematic diagram of a principle of a hidden policy according to an embodiment of the present invention
  • the overlap-added time-domain waveform is a time-domain waveform obtained by IDCT transformation of the MLT frame by the decoding end, and the final time-domain waveform is the time-domain waveform after the overlap-added time-domain waveform is inversely overlapped.
  • the current frame and subsequent frames of the final time domain waveform are affected.
  • the second frame shown in Figure 8 is lost, that is, (, d frame. Since the calculation of 3, 4 depends on b, c; the calculation of 4, 5 depends on d, e, so the final time domain waveform In the 3, 4, 5, and 6 will be affected by (d, wrong / lost.
  • Table 2 is a frame loss concealment strategy table corresponding to all possible combinations of intra-frame and inter-frame correlation.
  • ⁇ means that the pitch information is immediately copied
  • ⁇ means that it is copied by the immediately adjacent pitch information
  • V means that the time domain frame is overlapped (ie, a, b, c, d, e .. . ) Calculated
  • x means unrecoverable recovery, and can only be hidden by repeating the pitch of adjacent signals and attenuating them.
  • the pitch used for copying is selected from the front or back according to "-" or " ⁇ ".
  • Sequence frame 3 and 4 phase frame 5 and 6 phase recovered frame recovered frame off frame information off frame information 3, and 4, 5, and 6,
  • the pre-MLT frame is a normal frame, and the current MLT frame is a wrong/lost frame.
  • frames a and b are normal frames, and frames c and d are wrong/drop frames.
  • frames 3, and 4 are recovered based on the associated frame information of frames 3 and 4.
  • the hidden strategy utilizes the algebraic relationship of the time domain signal of the audio frame with the signal added by the MLT overlap, and estimates the first half of the final time domain frame of the current MLT frame. To find the rest of the time domain audio signal.
  • the specific implementation process is:
  • v(n + N/2) w[(N - 1) - n]x(N + n) _ w(n)x[(2N - l) - n] 0 ⁇ n ⁇ N/2
  • x[(2N-l)-n] is 4
  • x(N+n) is 3
  • v(n+N/2) is b. It can be seen that 3, 4 and b have a unique relationship.
  • the pitch at the end of repetition 2 is obtained by 3; due to the unique deterministic relationship of 3, 4, and b, 3 is used instead of 3, and 3 is calculated from 3, and b.
  • the estimate of 4, 4 can be roughly restored.
  • the same phase as the phase at the end of 2 should be selected from 2 as the starting point of the repeated pitch. This makes 3, the starting phase is the same as the phase at the end of 2, and the phase of 2 and 3 is continuous.
  • a possible pitch selection method is: Select the length of a pitch from the end of 2 and use this segment as a repeated pitch. The principle of selecting a pitch can be adopted for the forward copying pitch shown in "-," in Table 1, but is not limited thereto.
  • 3 and b can be used to calculate 4 .
  • 4 is needed to increase the amount of calculation. From the perspective of reducing the cost of the binning algorithm, it is not necessary to increase the additional calculation. Therefore, when there is strong correlation between intra frames, 3, 4, and 2 repetitions are possible, as in Table 2, the hidden strategies with numbers 1, 2, 3, and 5; and the correlation between frames is strong. When the internal correlation is weak, the calculation method is used to obtain 4, 3. The method of repeating 2 is still adopted, such as the hidden strategies of the numbers 4, 6, 7, and 8 in Table 2.
  • the first half frame and the second half frame of the final time domain frame of the current MLT frame are repeated by repeating the pitch of the second half of the final time domain frame of the pre-MLT frame and performing full frame attenuation.
  • the attenuation intensity gradually increases from the first half frame to the second half frame.
  • the attenuation method is to multiply the pre-attenuated frame by the attenuation factor.
  • the attenuation factor represents the intensity of the attenuation, and the larger the attenuation factor, the greater the attenuation intensity.
  • the pre-MLT frame is a wrong/lost frame
  • the current MLT frame is a normal frame.
  • frames c and d are wrong/drop frames
  • frames e and f are normal frames.
  • frames 5, and 6, are recovered based on the associated frame information of frames 3 and 4, and the associated frame information of frames 5 and 6.
  • a possible pitch selection method is: taking the length of a pitch starting from the front end of 6, and using this segment as a repeated pitch. The principle of selecting a pitch can be adopted for the backward reproduction of the pitch shown by " ⁇ " in Table 1, but is not limited thereto.
  • the pre-MLT frame has error/lost frame, and the current MLT frame is wrong/drop frame.
  • the beginning of the frame loss sequence can be recovered according to the 1) hidden method.
  • For the intermediate frame use the following hidden method to recover.
  • the front frame reliability indicator is updated each time the hidden policy is selected. In general, it is considered unreliable to use a waveform that repeats a certain frame and attenuated, and the front frame reliability flag is set to be unreliable; it is calculated by a formula, or a waveform of a certain frame is repeated. And the recovery frame that is not attenuated is considered to be reliable, and the front frame reliability flag is set to be reliable at this time. Restoring consecutive frame loss/error frames requires the use of the previous frame reliability indicator, the pre-MLT frame final time domain frame, and the associated frame information of the current MLT frame.
  • FIG. 9 is a flowchart of a method for selecting a hidden frame of a continuous frame loss/error frame according to an embodiment of the present invention. The specific steps of the method are as follows:
  • Step 801 Determine whether the pre-frame is reliable. If yes, go to step 802; otherwise, go to step 811.
  • step 811 it is determined whether the final time domain frame of the pre- MLT frame is reliable according to the previous frame reliability identifier, and if not, step 811 is performed, and the final time domain frame of the current MLT frame repeats the final time domain frame of the pre-frame, and The final time domain frame of the current MLT frame is subjected to full frame attenuation.
  • frames 5 and 6 frames 5, and 6, from repeated frames 3, and 4, when the full frame is attenuated, the attenuation strength gradually increases from the front end of 5 to the end of 6. And the reliability information of the obtained frames 5, and 6, is expressed as unreliable. Since the pre-frame is an unreliable frame, the 3, and 4 waveforms can be copied as they are 5, and 6.
  • Step 802 Determine whether the inter-frame correlation is strong; if yes, go to step 803, otherwise go to step 812.
  • step 812 is performed regardless of the intra-frame correlation, and the final time-domain frame of the pre-existing MLT frame is repeated as the final time-domain frame of the current MLT frame, and the current current obtained is obtained.
  • the frame MLT final time domain frame performs full frame attenuation, and the attenuation intensity gradually increases from front to back. And the reliability information of the final time domain frame of the current MLT frame obtained is not reliable. Since the inter-frame correlation is weak, the current frame and frame 3, and 4 are not similar, so 3, and 4, the waveform can be copied as the 5, and 6, respectively.
  • Step 803 Determine whether the intra-frame correlation is strong; if yes, go to step 804; otherwise, go to step 813.
  • Step 813 is performed to repeat the second half of the final time domain frame of the preamble frame as the first half frame and the second half frame of the current MLT frame, and obtain the current MLT frame before the final time domain frame.
  • the half frame is not attenuated, and the latter half frame is attenuated, and the attenuation intensity is gradually increased from the front end of the second half frame to the end of the second half frame.
  • the reliability information of the final time domain frame of the current MLT frame obtained is identified as unreliable.
  • Step 804 repeating the last half frame of the final time domain frame of the pre-MLT frame as the first half frame and the second half frame of the current MLT frame, without attenuation. And the reliability information of the final time domain frame of the current MLT frame obtained is identified as reliable.
  • the embodiment of the present invention provides a frame dropping hidden system.
  • FIG. 10 is a block diagram of a frame dropping system according to an embodiment of the present invention, the system comprising an encoding device and a decoding device.
  • the encoding device transmits the obtained related frame information and the current MLT frame to the decoding device.
  • the decoding device performs error/drop frame detection on the current MLT frame to generate error/drop frame information.
  • the decoding device recovers the error/loss according to the error/lost frame information, the associated frame information and the historical time domain waveform. frame.
  • the coding apparatus in this embodiment adopts the first preferred embodiment of the frame loss concealment coding apparatus provided in the embodiment of the present invention shown in FIG.
  • the encoding apparatus includes an MLT encoding unit 110, a history buffer unit 120, and a correlation frame information generating unit 130.
  • the history buffer unit 120 is configured to buffer the time domain waveform of the current frame and the time domain waveform of the preamble frame.
  • the history buffer unit 120 includes a current frame history buffer 121 and a pre-frame history buffer 122.
  • the current frame history buffer 121 is used to buffer the time domain waveform of the current frame. After the encoding apparatus completes the MLT encoding and outputs, the current frame history buffer 121 updates the pre-frame history buffer 122 with its saved current frame, and saves the subsequent frame in the current frame history buffer 121.
  • the pre-frame history buffer 122 is used to buffer the time domain waveform of the pre-frame.
  • the MLT encoding unit 110 is configured to perform an MLT transformation according to a subsequent frame acquired by the encoding device and a current frame acquired from the history buffer unit 120, generate an MLT frame, and send the MLT frame.
  • the related frame information generating unit 130 is configured to generate related frame information according to the subsequent frame acquired by the encoding device and the current frame and the pre-frame obtained from the history unit buffer, and send the related frame information.
  • the related frame information generating unit 130 includes an intra correlation detector 131 and an inter correlation detector 132.
  • the intra-frame correlation detector 131 is configured to generate intra-frame correlation information according to the current frame acquired from the current frame history buffer 121, and send the information.
  • the inter-frame correlation detector 132 is configured to generate inter-frame correlation information according to the current frame acquired from the current frame history buffer 121 and the pre-frame obtained from the pre-frame history buffer 122, and send the inter-frame correlation information.
  • each relevant frame information is sent to a multiplexer and sent out.
  • the MLT frame and its associated frame information need to be sent in packets, or the related frame information may be embedded in the pre-frame or the subsequent frame of the current frame, or
  • the associated frame information is transmitted in the enhancement layer in a layered coding scheme, and the MLT frame is transmitted at the core layer.
  • an MLT frame buffer unit needs to be added to buffer the MLT frame generated by the MLT coding unit, and the delay is sent out. Therefore, the current MLT frame transmission carries the relevant frame information of its subsequent frames.
  • an associated frame information buffer unit When embedding the subsequent frame transmission, an associated frame information buffer unit is needed to buffer the relevant frame information generated by the related frame information generating unit, and the delay is sent out. Therefore, the current MLT frame transmission carries the relevant frame information of its preamble frame.
  • the decoding apparatus in this embodiment uses the first preferred embodiment of the frame loss concealment decoding apparatus provided in the embodiment of the present invention shown in FIG. As shown in FIG. 12, the decoding apparatus includes a frame error detector 201, an IMLT decoding module 202, a frame loss concealing unit 220, and a history of decoding waveforms. Rush 210.
  • the frame error detector 201 is configured to perform error/drop frame detection on the MLT frame acquired from the encoding device, generate an error/drop frame error identifier, and send the error/drop frame error identifier to the frame loss concealing unit 220. Since the current MLT frame is normal but the pre-MLT frame is a wrong/drop frame, the IMLT decoding module 202 is required to assist in IDCT transformation of the current MLT frame.
  • the frame error detector 201 further determines according to the error/drop frame error flag: if it is determined that the current MLT frame is normal and the pre-MLT frame is normal, the frame error detector 201 sends the current MLT frame to the IMLT decoding module 202; If it is determined that the current MLT frame is correct, but the pre-MLT frame is an error/drop frame, the error/drop frame error identifier is sent to the IMLT decoding module 202 together with the current MLT frame, and the frame loss concealing unit 220 is sent an error/ The frame loss identifier is sent; otherwise, the error/drop frame error identifier needs to be sent to the frame loss concealment unit 220.
  • the IMLT decoding module 202 performs IMLT transformation on the current MLT frame when it receives the current MLT frame.
  • the transformed MLT frame overlap-added time domain frame and final time domain frame are stored in the decoded waveform history buffer 210.
  • the current MLT frame and its error/lost frame identifier are received, only the current MLT frame is subjected to IDCT transformation, and the obtained current MLT frame overlap-added time domain frame is stored in the decoded waveform history buffer 210, and the final time domain frame is The recovery frame determined by the lost frame hiding unit 220 is saved.
  • the frame error detector 201 may also send the error/drop frame error identifier together with the current MLT frame to the IMLT decoding module 202 when the current MLT frame is a normal frame, and the IMLT decoding module 202 may be based on the error/loss.
  • the frame error flag identifies subsequent operations.
  • the IMLT decoding module 202 decodes the current MLT frame based on the error/drop frame error flag sent by the frame error detector 201.
  • the error/drop frame error flag indicates that the current MLT frame and the pre-MLT frame are both normal frames
  • the decoded current MLT frame overlap-added time domain frame and the final time domain frame are stored in the decoded waveform history buffer 210.
  • the error/drop frame error flag indicates that the current MLT frame is normal but the pre-MLT frame is wrong/drop frame
  • only the current MLT frame is IDCT.
  • the obtained MLT frame overlap-added time domain frame is stored in the decoded waveform history buffer 210, and the final time domain frame stores the restored frame determined by the lost frame hiding unit 220.
  • the frame loss concealing unit 220 is configured to determine a frame loss concealment strategy according to the received error/drop frame error identifier sent by the frame error detector 201, combined with the received current MLT frame related frame information and the buffered pre-MLT frame related frame information; The error/drop frame is recovered according to the overlap-and-add time domain frame and the final time domain frame saved in the frame loss concealment strategy and the decoded waveform history buffer 210.
  • the frame loss concealing unit 220 includes an associated frame information history buffer 221, a frame loss concealment strategy selection module 222, and a frame loss recovery module 223.
  • the related frame information history buffer 221 is configured to buffer the current MLT frame related frame information and the pre-MLT frame related frame information.
  • the frame loss hiding policy selection module 222 is configured to store a frame loss hiding policy.
  • the frame loss concealment strategy is determined based on the error/drop frame error flag sent by the frame error detector 201 and the associated frame information in the associated frame information history buffer 221.
  • the frame loss concealment strategy is pre-designed and stored in the frame loss concealment strategy selection module 222.
  • the frame loss can be used in the form of a table for easy searching and use.
  • the frame loss concealment policy selection module 222 also needs to generate a pre-frame reliability identifier, which is used to select a frame loss concealment strategy when consecutive frame drops occur.
  • the frame loss recovery module 223 is configured to recover the error/drop frame according to the frame loss concealment strategy determined by the frame loss concealment policy selection module 222 and the overlap addition time domain frame and the final time domain frame saved in the decoded waveform history buffer 210.
  • the recovered error/drop frame is also sent to the decoded waveform history buffer as the final time domain frame of the current MLT.
  • the decoded waveform history buffer 210 is configured to store the overlap-added time domain frame and the final time domain frame.
  • the module is further subdivided into an overlap-added time domain frame history buffer 211 and a final time domain frame history buffer 212.
  • the final time domain frame history buffer 212 is configured to buffer the final time domain frame of the current MLT frame decoded by the IMLT decoding module 202, or the error/drop frame recovered by the buffered frame loss recovery module 223.
  • FIG. 13 is a flow chart of a method of a second preferred embodiment of a method for dropping frames according to an embodiment of the present invention.
  • This embodiment uses a first preferred embodiment of the encoding device and a first preferred embodiment of the decoding device to form a frame loss concealment system, and implements a frame dropping method.
  • the specific steps of the method are as follows: Step 301: The MLT encoding unit 110 of the encoding apparatus performs MLT transformation according to the received subsequent frame and the current frame stored in the current frame history buffer 121 to generate an MLT frame.
  • Step 302 The intra-frame correlation detector 131 detects the current frame to generate intra-frame correlation information.
  • the inter-frame correlation detector 132 detects the current frame and the pre-frame to generate inter-frame correlation information.
  • Step 303 The encoding device sends the MLT frame and the intra-frame and inter-frame correlation information to the decoding device.
  • Step 304 The frame error detector 201 of the decoding device receives the MLT frame, and the related frame information history buffer 221 buffers the intra-frame and inter-frame correlation information.
  • Step 305 the frame error detector 201 performs error/drop frame detection on the current MLT frame, and determines whether the current MLT frame or the pre-MLT frame is an error/drop frame; if yes, step 306 is performed; otherwise, steps 320 and 321 are performed.
  • the error detector 201 sends the current MLT frame to the IMLT decoding module 202, which performs normal MLT decoding on the current MLT frame.
  • the current MLT frame is subjected to IDCT transformation to obtain an overlap-added time domain frame, and is stored in the overlap-added time domain frame history buffer 211, and the decoded final time domain waveform is saved in the final time domain frame history buffer 212. End this process.
  • Step 306 the frame error detector 201 further determines whether the current MLT is a normal frame, and if yes, performs steps 307 and 308; otherwise, directly executes 308.
  • Step 307 the frame error detector 201 sends the error/drop frame error identifier and the current MLT frame to The IMLT decoding module 202 performs IDCT transformation on the current MLT frame; then, the current MLT frame is subjected to IDCT transformation to obtain an overlap-added time domain frame, and is stored in the overlap-added time domain frame history buffer 211.
  • Step 308 The frame error detector 201 sends the error/drop frame error identifier to the frame loss concealment strategy selection module 222.
  • the error/lost frame error identifier includes a 1-bit pre-frame error identifier and a 1-bit current frame error identifier.
  • steps 309, 312-314 are performed; if the error/drop frame error flag is 10, steps 310, 312-314 are performed; if the error/drop frame error flag is 11 Then, steps 311, 312-314 are performed.
  • Step 309 the frame loss concealment policy selection module 222 selects a frame loss concealment strategy according to intra-frame and inter-frame correlation information of the current MLT frame acquired from the related frame information history buffer 221 .
  • Step 310 The frame loss concealment policy selection module 222 selects a frame loss concealment strategy according to the intra-frame and inter-frame correlation information of the pre-MLT frame and the current MLT frame acquired from the related frame information history buffer 221.
  • Step 311 The frame loss concealment policy selection module 222 selects a frame loss concealment strategy according to the pre-frame reliability identifier and the intra-frame and inter-frame correlation information of the current MLT frame acquired from the related frame information history buffer 221 .
  • Step 311 The hidden policy selected is shown in Figure 9.
  • Step 312 The frame loss concealment policy selection module 222 updates the pre-frame reliability identifier according to the selected concealment policy. This step of the service can be performed after step 313.
  • Step 313 the frame loss recovery module 223 recovers the error/drop frame according to the selected hidden policy, the overlap-added time domain frame in the overlap-added time domain frame history buffer 211, and the final time domain frame in the final time domain frame history buffer 212.
  • Step 314 the frame loss recovery module 223 outputs the recovered error/drop frame and stores it in the final time domain frame history buffer 212.
  • the present invention performs short note detection on the current frame using the pre-frame and subsequent frames at the encoding end, and outputs short note information.
  • the short note information is included as part of the associated frame information, including short note identification or including short note identification and short note feature parameters.
  • the decoding end recovers the error/drop frame based on the short note information.
  • short note identifiers in short note information can be represented by a 1-bit binary number, 1 for short notes and 0 for non-short notes.
  • the short note characteristic parameters can be in various forms, such as a set of parameters representing a short note pitch, an amplitude envelope, or the like, or a coarse version of the frame encoded by another low precision encoder.
  • a short note frame has a strong intra-frame correlation, but the correlation with the adjacent frames is weak. Therefore, in the short note detection, the intra frame correlation is first judged for the current frame. If the intra-frame correlation is strong, the correlation with the pre-frame and subsequent frames is further checked separately. If the correlation with them is weak, it is determined that the current frame is a short note frame. Correlation calculations can be performed using autocorrelation, cross-correlation functions, as previously described.
  • the current frame is a short note frame
  • the short note characteristic parameter of the short note frame is determined.
  • the short note characteristic parameter is a short note pitch and a short note amplitude envelope
  • the short note pitch represents the smallest period of a periodic strong audio signal.
  • the short note amplitude envelope represents the shape of this frame waveform.
  • the recovery method is: repeating the short note pitch in the entire frame, and then adjusting the shape of the frame envelope according to the amplitude envelope information.
  • the encoding end has Two encoders, high-precision encoders encode each frame, and low-precision encoders are only used to encode short note frames.
  • high-precision encoded short note frame is lost, the frame of low-precision encoding is decoded.
  • the decoder also has two decoders.
  • FIG. 14 is a block diagram showing a second preferred embodiment of a frame dropping concealment coding apparatus according to an embodiment of the present invention.
  • the encoding apparatus includes an MLT encoding unit 110, a history buffer unit 120, and a correlation frame information generating unit 730.
  • the related frame information generating unit 730 includes not only the intra-correlation detector 731 and the inter-frame correlation detector 732.
  • a short note detector 733 for generating short note information based on a subsequent frame received by the encoding device, a current frame acquired from the current frame history buffer 121, and a preamble frame acquired from the preamble history buffer 122. Specifically, a short note identifier is generated according to the three frames of the pre-frame, the current frame, and the subsequent frame, and the short note feature parameter is generated according to the current frame.
  • the information transmitted by the encoding device to the decoding device through the packet communication network includes the current MLT frame, intra-frame correlation information, inter-frame correlation information, and short note information.
  • the decoding apparatus can employ the constituent structure of the decoding apparatus shown in Fig. 12 as the second preferred embodiment of the decoding apparatus.
  • the difference from the first preferred embodiment of the decoding apparatus is that, in the embodiment, the related frame information history buffer 221 also holds short note information.
  • the frame loss concealment policy selection module 222 receives the error/drop frame error identifier sent by the frame error detector 201, and finds the current frame short note information buffered by the associated frame information history buffer 211, the current error/lost frame is short. For note frames, you only need to recover lost frames based on short note information.
  • FIG. 15 is a flowchart of a method for deleting a frame hiding method according to a third preferred embodiment of the present invention.
  • the method uses a second preferred embodiment of the encoding device and a second preferred embodiment of the decoding device to form a frame loss hiding system, and implements a frame dropping method.
  • the method includes the following steps:
  • Step 401 The MLT encoding unit 110 of the encoding apparatus performs MLT transformation according to the received subsequent frame and the current frame stored in the history buffer 121 to generate an MLT frame.
  • Step 402 The intra-frame correlation detector 731 detects the current frame to generate intra-frame correlation information.
  • the inter-frame correlation detector 732 detects the current frame and the pre-frame to generate inter-frame correlation information.
  • the short note correlation detector 733 detects the current frame based on the pre-frame and the subsequent frame to generate short note information.
  • the short note information includes a short note identifier and a short note feature parameter. If the short note correlation detector detects that the current frame is a short note frame, set the short note identification to
  • the short note correlation detector detects that the current frame is not a short note frame, the short note identification is set to 0 and no short note feature parameters are generated.
  • Step 403 The encoding device sends the MLT frame and its intra and inter-frame correlation information and short note information to the decoding device.
  • Step 404 the frame error detector 201 of the decoding device receives the MLT frame, and the related frame information history buffer 221 receives the intra and interframe correlation information, and the short note information.
  • Step 405 The frame error detector 201 performs error/drop frame detection on the current MLT frame, determines whether the current MLT frame or the pre-MLT frame has an error/drop frame, and generates error/lost frame information; if yes, step 406; otherwise Steps 420 and 421 are performed. Steps 420 and 421 are the same as steps 320 and 321 described above.
  • Step 406 the frame error detector 201 further determines whether the current MLT is a normal frame, and if yes, performs steps 407 and 408; otherwise, directly executes 408.
  • Step 407 is the same as step 307 described above.
  • Step 408 the framing error detector 201 sends the error/lost frame error identifier to the frame loss concealment strategy selection module 222.
  • Step 409 the frame loss concealment policy selection module 222 determines whether the current error/lost frame is a short note based on the short note information obtained from the related frame information history buffer 221. If the current MLT frame is a wrong/drop frame and is a short note, step 430 is executed, the frame loss concealment policy selection module 222 sends the short note information to the frame loss recovery module 223, and the frame loss recovery module 223 is based on the short tone. Symbol information, recover lost frames. And the recovery frame is considered unreliable, and then step 413 is performed. Otherwise, step 410 is performed.
  • the short note identifier in the short note information can be used for judging.
  • Step 410 The frame loss concealment policy selection module 222 selects a concealment strategy according to the error/lost frame error detection identifier, combined with the related frame information and the previous frame reliability identifier. See steps 309, 310, and 311 of Figure 13 for specific selection methods.
  • Step 411 The frame loss hiding policy selection module 222 updates the previous frame reliability identifier according to the selected hiding policy.
  • Step 412 the frame loss recovery module 223 recovers the error/drop frame according to the selected hidden policy, the overlap-added time domain frame in the overlap-added time domain frame history buffer 211, and the final time domain frame in the final time domain frame history buffer 212. .
  • Step 413 the frame loss recovery module 223 outputs the recovered error/drop frame and stores it in the final time domain frame history buffer 212.
  • the correlation frame information includes inter-frame and intra-frame correlation information and short note information.
  • more related frame information types can be generated and transmitted together with the MLT frame.
  • These new related frame information can describe intra- and inter-frame correlation in more detail, such as the correlation between the pre-frame and subsequent frames, etc.
  • the new related frame information can also be a set of audio frame feature parameters.
  • the method, system, encoding device and decoding device for frame loss concealment provided by the embodiments of the present invention can select different hidden policies and recover errors/drop frames.
  • the The hidden strategy utilizes the algebraic relationship between the time domain signal of the audio frame and the signal added by the MLT overlap, and obtains the remaining part of the time domain audio signal by estimating the partial time domain audio signal; and also utilizes the MLT overlap addition process.
  • a recovery frame with a wrong/drop frame is obtained by using a part of the restored frame to hold a better frame. Therefore, the embodiment of the present invention can better recover errors/drop frames, improve the frame hiding capability, and avoid the confusion of time domain signals.
  • short note information when generating related frame information, short note information is also generated, and the current identifier is identified.
  • the decoding device can recover the short note error/drop frame when the frame is lost.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

Un procédé de suppression de la perte de trame se déroule comme suit: une forme d'onde d'historique du domaine temporel est stockée dans un décodeur et un codeur envoie au décodeur (501) une trame MLT (transformée lissée modulée) donnée et les informations de corrélation de trame obtenues; le décodeur effectue la détection d'erreur/perte de trame sur la trame MLT donnée et génère des informations d'erreur/perte de trame (502); en fonction des informations d'erreur/perte de trame, des informations de corrélation de trame et de la forme d'onde d'historique du domaine temporel, le décodeur récupère l'erreur/perte de trame (503). La présente invention concerne également un système de suppression de la perte de trame, un codeur et un décodeur. Cette invention permet d'améliorer la capacité de suppression de la perte de trame du plan de codage-décodage MLT et de récupérer de manière préférentielle les erreurs/pertes de trames.
PCT/CN2007/071129 2006-12-19 2007-11-26 Procédé, système et appareils utilisés pour supprimer la perte de trame WO2008074249A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN 200610161747 CN101207468B (zh) 2006-12-19 2006-12-19 丢帧隐藏方法、系统和装置
CN200610161747.0 2006-12-19

Publications (1)

Publication Number Publication Date
WO2008074249A1 true WO2008074249A1 (fr) 2008-06-26

Family

ID=39536001

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2007/071129 WO2008074249A1 (fr) 2006-12-19 2007-11-26 Procédé, système et appareils utilisés pour supprimer la perte de trame

Country Status (2)

Country Link
CN (1) CN101207468B (fr)
WO (1) WO2008074249A1 (fr)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9514755B2 (en) 2012-09-28 2016-12-06 Dolby Laboratories Licensing Corporation Position-dependent hybrid domain packet loss concealment
US10249310B2 (en) 2013-10-31 2019-04-02 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10262662B2 (en) 2013-10-31 2019-04-16 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10997982B2 (en) 2018-05-31 2021-05-04 Shure Acquisition Holdings, Inc. Systems and methods for intelligent voice activation for auto-mixing
CN113488068A (zh) * 2021-07-19 2021-10-08 歌尔光学科技有限公司 音频异常检测方法、装置及计算机可读存储介质
US11297423B2 (en) 2018-06-15 2022-04-05 Shure Acquisition Holdings, Inc. Endfire linear array microphone
US11297426B2 (en) 2019-08-23 2022-04-05 Shure Acquisition Holdings, Inc. One-dimensional array microphone with improved directivity
US11302347B2 (en) 2019-05-31 2022-04-12 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
US11303981B2 (en) 2019-03-21 2022-04-12 Shure Acquisition Holdings, Inc. Housings and associated design features for ceiling array microphones
US11310592B2 (en) 2015-04-30 2022-04-19 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US11310596B2 (en) 2018-09-20 2022-04-19 Shure Acquisition Holdings, Inc. Adjustable lobe shape for array microphones
US11438691B2 (en) 2019-03-21 2022-09-06 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition functionality
US11445294B2 (en) 2019-05-23 2022-09-13 Shure Acquisition Holdings, Inc. Steerable speaker array, system, and method for the same
US11477327B2 (en) 2017-01-13 2022-10-18 Shure Acquisition Holdings, Inc. Post-mixing acoustic echo cancellation systems and methods
US11523212B2 (en) 2018-06-01 2022-12-06 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
US11552611B2 (en) 2020-02-07 2023-01-10 Shure Acquisition Holdings, Inc. System and method for automatic adjustment of reference gain
US11558693B2 (en) 2019-03-21 2023-01-17 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality
US11678109B2 (en) 2015-04-30 2023-06-13 Shure Acquisition Holdings, Inc. Offset cartridge microphones
US11706562B2 (en) 2020-05-29 2023-07-18 Shure Acquisition Holdings, Inc. Transducer steering and configuration systems and methods using a local positioning system
US11785380B2 (en) 2021-01-28 2023-10-10 Shure Acquisition Holdings, Inc. Hybrid audio beamforming system
US12028678B2 (en) 2019-11-01 2024-07-02 Shure Acquisition Holdings, Inc. Proximity microphone

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2903004A4 (fr) * 2012-09-24 2016-11-16 Samsung Electronics Co Ltd Procédé et appareil permettant de masquer des erreurs de trame, et procédé et appareil permettant de décoder des données audio
CN104751851B (zh) * 2013-12-30 2018-04-27 联芯科技有限公司 一种基于前后向联合估计的丢帧差错隐藏方法及系统
CN104767544B (zh) 2014-01-02 2018-08-24 深圳市中兴微电子技术有限公司 一种实现解扰解扩的方法和矢量运算器
US10043523B1 (en) * 2017-06-16 2018-08-07 Cypress Semiconductor Corporation Advanced packet-based sample audio concealment
JP7178506B2 (ja) * 2019-02-21 2022-11-25 テレフオンアクチーボラゲット エルエム エリクソン（パブル） 位相ｅｃｕ ｆ０補間スプリットのための方法および関係するコントローラ
CN113542795B (zh) * 2020-04-21 2023-04-18 腾讯科技（深圳）有限公司 视频处理方法、装置、电子设备及计算机可读存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002221994A (ja) * 2001-01-26 2002-08-09 Nippon Telegr & Teleph Corp <Ntt> 音声信号の符号列のパケット組立方法、装置及びパケット分解方法、装置並びにこれらの方法を実行するプログラム、プログラムを記録する記録媒体
US6775649B1 (en) * 1999-09-01 2004-08-10 Texas Instruments Incorporated Concealment of frame erasures for speech transmission and storage system and method
US20060173687A1 (en) * 2005-01-31 2006-08-03 Spindola Serafin D Frame erasure concealment in voice communications

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6775649B1 (en) * 1999-09-01 2004-08-10 Texas Instruments Incorporated Concealment of frame erasures for speech transmission and storage system and method
JP2002221994A (ja) * 2001-01-26 2002-08-09 Nippon Telegr & Teleph Corp <Ntt> 音声信号の符号列のパケット組立方法、装置及びパケット分解方法、装置並びにこれらの方法を実行するプログラム、プログラムを記録する記録媒体
US20060173687A1 (en) * 2005-01-31 2006-08-03 Spindola Serafin D Frame erasure concealment in voice communications

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SANG-UK RYU ET AL.: "Encoder assisted frame loss concealment for MPEG-AAC decoder", INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, TOULOUSE, FRANCE, 14 May 2006 (2006-05-14) - 19 May 2006 (2006-05-19), XP002423159 *

Cited By (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9514755B2 (en) 2012-09-28 2016-12-06 Dolby Laboratories Licensing Corporation Position-dependent hybrid domain packet loss concealment
US9881621B2 (en) 2012-09-28 2018-01-30 Dolby Laboratories Licensing Corporation Position-dependent hybrid domain packet loss concealment
US10373621B2 (en) 2013-10-31 2019-08-06 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10339946B2 (en) 2013-10-31 2019-07-02 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10262662B2 (en) 2013-10-31 2019-04-16 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10262667B2 (en) 2013-10-31 2019-04-16 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10269359B2 (en) 2013-10-31 2019-04-23 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10269358B2 (en) 2013-10-31 2019-04-23 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung, E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10276176B2 (en) 2013-10-31 2019-04-30 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung, E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10283124B2 (en) 2013-10-31 2019-05-07 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung, E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10290308B2 (en) 2013-10-31 2019-05-14 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10249310B2 (en) 2013-10-31 2019-04-02 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10249309B2 (en) 2013-10-31 2019-04-02 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US10381012B2 (en) 2013-10-31 2019-08-13 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment based on a time domain excitation signal
US10964334B2 (en) 2013-10-31 2021-03-30 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio decoder and method for providing a decoded audio information using an error concealment modifying a time domain excitation signal
US11832053B2 (en) 2015-04-30 2023-11-28 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US11678109B2 (en) 2015-04-30 2023-06-13 Shure Acquisition Holdings, Inc. Offset cartridge microphones
US11310592B2 (en) 2015-04-30 2022-04-19 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US11477327B2 (en) 2017-01-13 2022-10-18 Shure Acquisition Holdings, Inc. Post-mixing acoustic echo cancellation systems and methods
US10997982B2 (en) 2018-05-31 2021-05-04 Shure Acquisition Holdings, Inc. Systems and methods for intelligent voice activation for auto-mixing
US11798575B2 (en) 2018-05-31 2023-10-24 Shure Acquisition Holdings, Inc. Systems and methods for intelligent voice activation for auto-mixing
US11800281B2 (en) 2018-06-01 2023-10-24 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
US11523212B2 (en) 2018-06-01 2022-12-06 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
US11297423B2 (en) 2018-06-15 2022-04-05 Shure Acquisition Holdings, Inc. Endfire linear array microphone
US11770650B2 (en) 2018-06-15 2023-09-26 Shure Acquisition Holdings, Inc. Endfire linear array microphone
US11310596B2 (en) 2018-09-20 2022-04-19 Shure Acquisition Holdings, Inc. Adjustable lobe shape for array microphones
US11303981B2 (en) 2019-03-21 2022-04-12 Shure Acquisition Holdings, Inc. Housings and associated design features for ceiling array microphones
US11438691B2 (en) 2019-03-21 2022-09-06 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition functionality
US11558693B2 (en) 2019-03-21 2023-01-17 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality
US11778368B2 (en) 2019-03-21 2023-10-03 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition functionality
US11445294B2 (en) 2019-05-23 2022-09-13 Shure Acquisition Holdings, Inc. Steerable speaker array, system, and method for the same
US11800280B2 (en) 2019-05-23 2023-10-24 Shure Acquisition Holdings, Inc. Steerable speaker array, system and method for the same
US11688418B2 (en) 2019-05-31 2023-06-27 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
US11302347B2 (en) 2019-05-31 2022-04-12 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
US11750972B2 (en) 2019-08-23 2023-09-05 Shure Acquisition Holdings, Inc. One-dimensional array microphone with improved directivity
US11297426B2 (en) 2019-08-23 2022-04-05 Shure Acquisition Holdings, Inc. One-dimensional array microphone with improved directivity
US12028678B2 (en) 2019-11-01 2024-07-02 Shure Acquisition Holdings, Inc. Proximity microphone
US11552611B2 (en) 2020-02-07 2023-01-10 Shure Acquisition Holdings, Inc. System and method for automatic adjustment of reference gain
US11706562B2 (en) 2020-05-29 2023-07-18 Shure Acquisition Holdings, Inc. Transducer steering and configuration systems and methods using a local positioning system
US11785380B2 (en) 2021-01-28 2023-10-10 Shure Acquisition Holdings, Inc. Hybrid audio beamforming system
CN113488068A (zh) * 2021-07-19 2021-10-08 歌尔光学科技有限公司 音频异常检测方法、装置及计算机可读存储介质
CN113488068B (zh) * 2021-07-19 2024-03-08 歌尔科技有限公司 音频异常检测方法、装置及计算机可读存储介质

Also Published As

Publication number Publication date
CN101207468B (zh) 2010-07-21
CN101207468A (zh) 2008-06-25

Similar Documents

Publication Publication Date Title
WO2008074249A1 (fr) Procédé, système et appareils utilisés pour supprimer la perte de trame
US10424306B2 (en) Frame erasure concealment for a multi-rate speech and audio codec
JP5587405B2 (ja) スピーチフレーム内の情報のロスを防ぐためのシステムおよび方法
TWI390503B (zh) Dual channel voice transmission system, broadcast scheduling design module, packet coding and missing sound quality damage estimation algorithm
KR101290425B1 (ko) 소거된 스피치 프레임을 복원하는 시스템 및 방법
JP4473869B2 (ja) 音響信号のパケット通信方法、送信方法、受信方法、これらの装置およびプログラム
CN112786060B (zh) 用于对音频内容进行编码和解码的编码器、解码器和方法
EP2200018B1 (fr) Procédé et dispositif pour effectuer une dissimulation de perte de paquets
CN100545908C (zh) 用于隐蔽压缩域分组丢失的方法和装置
WO2008154852A1 (fr) Procédé et dispositif pour le masquage de trames perdues
WO2008040250A1 (fr) Procédé, dispositif et système destinés au masquage d&#39;erreurs d&#39;un flux de données audio
WO2013060223A1 (fr) Procédé et appareil de compensation de perte de trames pour signal à trames de parole
JP2004361731A (ja) オーディオ復号装置及びオーディオ復号方法
JPWO2002071389A1 (ja) オーディオデータ補間装置および方法、オーディオデータ関連情報作成装置および方法、オーディオデータ補間情報送信装置および方法、ならびにそれらのプログラムおよび記録媒体
WO2008067763A1 (fr) Procédé et dispositif de décodage
KR100792209B1 (ko) 디지털 오디오 패킷 손실을 복구하기 위한 방법 및 장치
KR100462024B1 (ko) 부가 음성 데이터를 이용한 패킷 손실 복구 방법 및 이를이용한 송수신기
KR102717379B1 (ko) 멀티 채널 오디오 프레임에서 예측적인 코딩에서 에러 복구를 위한 방법 및 장치
Lecomte et al. Packet loss and concealment
US20100185441A1 (en) Error Concealment
MX2009000054A (es) Dispositivo de descodificacion de datos de audio.
JP2005039674A (ja) データ送信装置、データ受信装置及びデータ送受信装置
JP2005107283A (ja) ＶｏＩＰ音声通信におけるパケット損失隠蔽方法、装置およびプログラム
JP2004023191A (ja) 信号符号化方法及び信号復号化方法及び信号符号化装置及び信号復号化装置及び信号符号化プログラム及び信号復号化プログラム
EP1713194A1 (fr) Procédé de mise à jour du registre d&#39;état d&#39;un codeur

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07817319

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07817319

Country of ref document: EP

Kind code of ref document: A1