TWI789577B - Method and system for recovering audio information - Google Patents

Abstract

A method and a system for recovering audio information are provided. The method includes steps of: transmitting audio packages sequentially to a receiver from a transmitter; performing a Fourier transform operation on the audio package of previous frames of a frame in which the audio package delays or misses by the receiver; calculating a threshold according to the audio package of the previous frame by the receiver; performing a linear prediction on the frequency segments each having an amplitude that is higher than the threshold; and performing a Fourier transform inverse operation on the extrapolated frequency segments to extrapolate the audio package that delays or misses.

Description

Audio data reconstruction method and system

The present invention relates to audio, in particular to an audio data reconstruction method and system.

Digital audio data is usually encoded in a frame and then transmitted to the receiving end through a wired or wireless network for decoding and playback. During the transmission process, due to interference or network congestion, audio packets are lost or delayed, causing the receiver to interrupt playback due to buffer exhaustion. A simple method is to increase the buffer size and request to resend lost packets before the buffer is exhausted, but this will increase the playback delay, which is not suitable for some applications that require low latency.

Another method is to use the audio data in the intact packets before and after the lost packet to do interpolation (interpolation), or use the audio data in the intact packet before the lost packet to do extrapolation (extrapolation) operation, and use the calculated audio data Replace the audio data with lost packets, so that the playback can be kept without interruption, and there is no need to increase the buffer.

At present, a variety of methods for reconstructing lost audio data have been proposed. First, the audio data is converted from the time domain to the frequency domain, and the sound is represented by a sinusoidal model. The sine wave of the intact packet is interpolated to calculate the sine wave frequency (frequency), amplitude (amplitude) and phase (phase) of the lost packet, and finally the frequency domain to time domain conversion is performed to obtain the reconstructed audio data.

The technical problem to be solved by the present invention is to provide a method for reconstructing audio data in view of the deficiencies in the prior art, which includes the following steps: the audio data is divided into multiple audio packets by the sending end and sent to the receiving end sequentially; Fast Fourier transform operation is performed on the audio packets of the first few frames of the delayed delivery audio packet to convert the audio packet from the time domain to the frequency domain; Calculate the threshold value of a frame of audio packets; use the receiving end to compare the amplitude and threshold value of each frequency segment of each frame before the frame of the lost or delayed delivery of the audio packet to filter; use the receiving end to Perform linear prediction on the amplitude and phase of each frequency segment whose amplitude is greater than the threshold value, and extrapolate the lost or delayed audio packets; The obtained audio packets are extrapolated and converted from the frequency domain back to the time domain.

In an implementation aspect, the audio data reconstruction method further includes the following steps: using the receiving end to calculate the amplitude and phase of the extrapolated audio packets; using the receiving end, in the extrapolated audio packets, the The real and imaginary parts of the frequency segment are filled with noise; and the receiving end is used to perform an inverse fast Fourier transform operation to convert the audio packet obtained after extrapolation and noise filling from the frequency domain back to the time domain .

In an implementation aspect, the audio data reconstruction method further includes the following steps: using the receiving end to calculate and filter out the amplitude of each frequency segment of the previous frame after the fast Fourier transform operation; using the receiving end to sum the previous Amplitudes of all frequency segments of the frame to obtain a total amplitude; and using the receiving end to calculate a threshold based on the total amplitude.

In an implementation aspect, the audio data reconstruction method further includes the following step: using the receiving end to divide the total amplitude by the signal-to-noise ratio to obtain the threshold value.

In addition, the present invention provides an audio reconstruction system, including a sending end and a receiving end. The sending end is configured to divide an audio data into multiple audio packets, and send the multiple audio packets sequentially. The receiving end includes an audio receiving module, an audio conversion module, an audio filtering module and an extrapolation calculation module. The audio receiving module is connected to the sending end and is configured to sequentially receive multiple audio packets sent by the sending end. The audio conversion module is connected to the audio receiving module and is configured to perform fast Fourier transform operation on the audio packets of the first few frames of the frame where the audio packet is lost or delayed, so as to convert the audio packet from the time domain to the frequency domain. The audio filtering module is connected to the audio converting module. The audio screening module is configured to calculate a threshold value according to the audio packet of the frame preceding the frame of the lost or delayed audio packet after the fast Fourier transform operation. The audio filtering module is configured to compare the amplitude of each frequency segment of each frame preceding the frame in which the audio packet is lost or delayed to a threshold for filtering. The extrapolation calculation module is connected with the audio filtering module and the audio conversion module. The extrapolation operation module performs linear prediction on all the amplitudes filtered out for fast Fourier transform operation to extrapolate lost or delayed audio packets. The frequency segment introduced outside the audio conversion module is subjected to inverse fast Fourier transform operation, so as to convert the audio packet obtained through extrapolation from the frequency domain back to the time domain.

In an implementation aspect, the receiving end further includes a phase calculation module connected to the audio filtering module. The phase calculation module is configured to calculate the amplitude and phase of the extrapolated audio packets.

In one embodiment, the receiving end further includes a noise filling module connected to the phase calculation module, the extrapolation operation module and the audio conversion module. The noise filling module is configured to fill the real part and the imaginary part of the frequency segment whose amplitude is not greater than the threshold value into the noise in the extrapolated audio packet. The audio conversion module performs an inverse fast Fourier transform operation to convert the audio packets obtained after extrapolation and noise filling from the frequency domain back to the time domain.

In one embodiment, the audio screening module is configured to calculate and filter out the amplitude of each frequency segment of the previous frame after performing the fast Fourier transform operation, and add up the amplitudes of all frequency segments of the previous frame to obtain the total amplitude , and based on the total amplitude, the threshold value is calculated.

In one embodiment, the audio screening module is configured to divide the total amplitude by the signal-to-noise ratio to obtain the threshold value.

As mentioned above, the present invention provides a method and method for reconstructing audio data, which reconstructs lost or late sound waves. The main advantages are as follows: 1. Computation is only performed at the receiving end, and no special coding or calculation is required at the sending end; 2. Use pure PCM data for calculation, which has nothing to do with the compression method of audio data; 3. The operation is simple, the amount of calculation is low, and it is suitable for devices with low power consumption and low computing power; 4. There is no need for the intact packet after the lost or late packet, but only the intact packet before the lost or late packet for calculation, which is suitable for low-latency playback devices.

In order to further understand the features and technical content of the present invention, please refer to the following detailed description and drawings related to the present invention. However, the provided drawings are only for reference and description, and are not intended to limit the present invention.

The implementation of the present invention is described below through specific specific examples, and those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various modifications and changes can be made to the details in this specification based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only for simple illustration, and are not drawn according to the actual size, which is stated in advance. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the protection scope of the present invention. In addition, the term "or" used herein may include any one or a combination of more of the associated listed items depending on the actual situation.

[first embodiment]

Please refer to FIG. 1 , which is a flow chart of the steps of the audio data reconstruction method according to the first embodiment of the present invention. The audio data reconstruction method of this embodiment may include steps S101 - S107 as shown in FIG. 1 , which are described in detail as follows.

In step S101, the audio data is divided into multiple audio packets by the sending end and sent to the receiving end sequentially. During transmission of multiple audio packets from the sender to the receiver in sequence, audio packets may be lost or delayed. When this happens, use the receiving end to perform Fast Fourier Transform (FFT) operations on the frame of the lost or delayed delivery of the audio packet or the audio packets of the first few frames called frames (frame) to convert Audio packets are converted from the time domain to the frequency domain.

In step S103, use the receiving end to calculate the threshold value according to the audio packet of the previous frame of the frame of the lost or delayed delivery of the audio packet, and compare each of the (multiple frames) before the frame of the lost or delayed delivery of the audio packet The amplitude and the threshold value of each frequency bin (FFT bin) among the multiple frequency bins in one frame are used to screen out amplitudes higher than the threshold value, that is, to screen out high-energy frequency bins.

In step S105, the receiving end performs linear prediction on the amplitudes of all frequency segments screened out to extrapolate lost or delayed audio packets.

In step S107, the receiving end performs an inverse fast Fourier transform operation to convert the audio packet obtained through extrapolation from the frequency domain back to the time domain to obtain a pulse code modulation (PCM) audio packet.

[Second embodiment]

Please refer to FIG. 2 to FIG. 5, wherein FIG. 2 is a flow chart of the steps of the audio data reconstruction method according to the second embodiment of the present invention; FIG. A schematic diagram of calculating the threshold value in the previous frame; FIG. 4 is a spectrogram after the fast Fourier transform operation is performed in the previous frame of the frame of the lost or delayed delivery audio packet according to the second embodiment of the present invention; FIG. 5 is the spectrum diagram of the first frame of the present invention Spectrogram of the reconstructed frame of the second embodiment.

During the process of sequentially sending multiple audio packets with multiple frames to the receiving end, environmental factors or network congestion may interfere, causing some audio packets such as the last frame of the audio packet to be lost or delayed in delivery to the receiving end. , resulting in the receiving end not receiving the last frame of audio packets (that is, missing audio packets), or not receiving the last frame of audio packets within a predetermined time (that is, delaying delivery of audio packets). As a result, after the receiver receives and plays multiple frames in sequence, a staccato occurs. For example, as shown in Figure 3, after the receiver receives and plays multiple frames FR1~FR7 in sequence, a staccato region Gap occurs. In order to solve the problem of staccato, the method for reconstructing audio data in this embodiment may include steps S201-S211 as shown in FIG. 2 , which are described in detail as follows.

In step S201, during the transmission process in which the sending end sequentially sends a plurality of audio packets to the receiving end, the receiving end uses the information of the first few frames (for example, the frame FR7 shown in FIG. 3 ) of the frame in which the audio packet is lost or delayed The audio packet undergoes a Fast Fourier Transform (FFT) operation to convert the audio packet from the time domain to the frequency domain. As shown in Figure 4, the frame before the frame of the lost or delayed delivery of the audio packet is executed fast Spectrum waveform WAVE1 after Fourier transform operation.

In step S203, the receiver calculates the threshold value based on the audio packet of the previous frame (such as the frame FR7 shown in FIG. 3 ) of the frame (such as the Gap shown in FIG. 3 ) in which the audio packet is lost or delayed. . Then, use the receiving end to compare the amplitude and threshold value of each frequency segment in each of the multiple frames (for example, each frame FR1~FR7 shown in Figure 3) before the frame of the lost or delayed delivery of the audio packet (such as the threshold value TH shown in Figure 4 such as the noise-to-noise ratio) to filter out frequency amplitudes higher than the threshold value, for example, filter out all frequency amplitudes higher than the threshold value TH shown in Figure 4, and those exceeding the threshold value TH Only the frequency segment needs to do amplitude linear prediction and phase calculation.

In step S205, the receiver performs linear prediction on all the filtered amplitudes to extrapolate lost or delayed audio packets, as shown in FIG. 5 is the reconstructed frame spectrum waveform WAVE2.

； 其中magnitude代表頻率的振幅，real代表頻率的實部，image代表頻率的虛部。Specifically, the amplitude of each frequency segment of the previous frame (for example, the frame FR7 before the staccato Gap as shown in Figure 3 ) is calculated and screened out by the receiving end, expressed by the following equation:

; Among them, magnitude represents the amplitude of the frequency, real represents the real part of the frequency, and image represents the imaginary part of the frequency.

Next, the receiving end sums up the amplitudes of all the frequency segments of the previous frame to obtain the total amplitude. Use the receiving end to calculate the threshold value based on the total amplitude, which is expressed by the following equation: TM=M1+M2+M3+...+Mn, where TM represents the total amplitude, M1~Mn represents the amplitude of multiple frequency segments, and n represents the amplitude used for calculation The number of frequency amplitudes of the threshold value, n=FFT size/2, where the FFT size represents the number of frequency amplitudes that can perform fast Fourier transform operations.

， 其中S代表門檻值，TM代表總振幅，L代表訊雜比，此訊雜比可為任意適當值，例如1000。在利用接收端篩選出高能量的頻率片段後，剩餘篩掉的低能量雜訊視為雜訊，不作為後續步驟中外推丟失或延遲送達的音訊封包的依據。For example, using the receiving end to divide the total amplitude by the signal-to-noise ratio to obtain the threshold value, expressed by the following equation:

, where S represents the threshold value, TM represents the total amplitude, and L represents the signal-to-noise ratio, which can be any appropriate value, such as 1000. After the high-energy frequency segment is screened out by the receiving end, the remaining low-energy noise screened out is regarded as noise, and is not used as a basis for extrapolating lost or delayed audio packets in subsequent steps.

； 其中Phase代表相位，image代表頻率片段的虛部，real代表頻率片段的實部。In step S207, the phase of the extrapolated audio packet is calculated by the receiving end, for example, obtained by the following equation:

; Among them, Phase represents the phase, image represents the imaginary part of the frequency segment, and real represents the real part of the frequency segment.

For example, use the receiving end to calculate the phase of the previous frame (frame N) of the extrapolated frame (that is, the frame where the audio packet is lost, frame N+1), and calculate the phase of the previous frame (frame N) The phase difference between the phase of the previous frame (frame N-1) and the phase difference between the calculated phase difference and the phase of the previous frame (frame N) to obtain the phase of the extrapolated frame, Expressed in the following equation: Phase[N+1] = Phase[N]+(Phase[N] – Phase[N-1]); Among them, Phase[N+1] represents the phase of the extrapolated frame, indicating that the audio packet of the extrapolated frame is arranged in the N+1th transmission among the multiple audio packets sent by the sender, and N can be any appropriate integer greater than 1 Value, and Phase[N] represents the phase of the previous frame of the extrapolated frame, and Phase[N-1] represents the phase of the previous frame of the extrapolated frame.

其中real代表頻率片段的實部，image代表頻率片段的虛部，magnitude代表頻率片段的振幅，Phase代表頻率片段的相位。The real part and the imaginary part of the frequency segment of the sound wave of the audio packet can be expressed according to the following equation:

Where real represents the real part of the frequency segment, image represents the imaginary part of the frequency segment, magnitude represents the amplitude of the frequency segment, and Phase represents the phase of the frequency segment.

In step S209 , in the sound wave of the audio packet of the extrapolated frame, the real part and the imaginary part of the frequency segment whose amplitude is not greater than the threshold value are filled with noise. In detail, in the sound wave of the extrapolated frame, the real part and the imaginary part of the frequency segment not greater than the threshold value are filled with the noise value less than the threshold value.

In step S211 , the receiving end performs an inverse fast Fourier transform operation to convert the audio packet obtained through extrapolation from the frequency domain back to the time domain to obtain the PCM audio packet.

It should be understood that the present invention is not limited to the examples of the embodiments herein, and the execution sequence and content of the steps in the methods of the embodiments herein can be appropriately adjusted according to actual needs, or the procedure of adding or subtracting steps can be appropriately repeated if necessary One or more steps exemplified in the embodiments herein.

[Third embodiment]

Please refer to FIG. 6 and FIG. 7, wherein FIG. 6 is a block diagram of the audio data reconstruction system of the third embodiment of the present invention, and FIG. 7 is the use of the mobile phone to send the audio source data to the earphone adopting the audio data reconstruction system of the third embodiment schematic diagram.

The audio data reconstruction system of this embodiment may include a transmitting end TX and a receiving end RX as shown in FIG. 6 . Wherein, the receiving end RX may include an audio receiving module 10 , an audio conversion module 20 , an audio filtering module 30 and an extrapolation operation module 40 , which can be used to execute the above steps S101 ~ S109 , S201 ~ S205 , S211 . For example, as shown in FIG. 7 , the transmitting end TX can be a mobile phone, and the receiving end RX can be an earphone. This is only an example and the present invention is not limited thereto.

The audio receiving module 10 is connected to the sending end TX and the audio receiving module 10 . After the sending end TX cuts the audio data AU into multiple audio packets, it sends the multiple audio packets to the receiving end RX. The audio receiving module 10 of the receiving end RX sequentially receives the audio packets sent by the sending end TX via wired or wireless means (such as but not limited to Bluetooth wireless transmission technology).

During the process that the transmitting end TX continues to send multiple audio packets to the audio receiving module 10 of the receiving end RX in sequence, the audio conversion module 20 judges that any audio packet is lost or delayed in delivery, and the audio packet that is lost or delayed in delivery The audio packets of the first few frames of a frame are subjected to fast Fourier transform operations to convert the audio packets from the time domain to the frequency domain.

The audio filtering module 30 is connected to the audio conversion module 20 . After performing the fast Fourier transform operation, the audio screening module 30 calculates the threshold value according to the coefficients or parameters of the audio packet of the frame preceding the frame where the audio packet is lost or delayed. The audio filtering module 30 then compares the amplitude of each frequency segment of each frame preceding the frame of the lost or delayed audio packet with the threshold value for screening.

The extrapolation operation module 40 is connected to the audio screening module 30 and the audio conversion module 20, configured to linearly predict the amplitude of the filtered frequency segments, and extrapolate lost or delayed audio packets. Finally, the audio conversion module 20 performs an inverse fast Fourier transform operation to convert the audio packets obtained through extrapolation from the frequency domain back to the time domain.

If necessary, the receiving end RX may further include a phase calculation module 50 and a noise filling module 60, which can be used to perform the above steps S207 and S209 respectively. The phase calculation module 50 is connected to the audio screening module 30 . The noise filling module 60 is connected to the phase calculation module 50 , the extrapolation operation module 40 and the audio conversion module 20 .

After the extrapolation operation module 40 extrapolates the lost or delayed audio packets, the phase calculation module 50 calculates the phase of the frequency segment of the extrapolated lost or delayed audio packets. Next, the noise filling module 60 fills the real part and the imaginary part of the frequency segment whose amplitude is not greater than the threshold value into the noise in the exported audio packet.

Finally, the audio conversion module 20 performs an inverse fast Fourier transform operation to convert the audio packet obtained after extrapolation and noise filling from the frequency domain back to the time domain. After the audio playback module (not shown) of the RX at the receiving end plays the punctually arrived audio packet, it can then play the extrapolated audio packet with added noise (replacing the lost or delayed audio packet).

Please refer to FIG. 8 and FIG. 9 , wherein FIG. 8 is a waveform diagram of staccato; FIG. 9 is a waveform diagram of audio data reconstructed by the audio data reconstruction system and method according to an embodiment of the present invention after missing or late sound waves are reconstructed.

During the process of sequentially sending multiple audio packets from the sending end to the receiving end, after the receiving end receives the sound wave W1 of an audio packet as shown in Figure 8, the next audio packet is lost or lost due to environmental interference or other factors. Arriving late, and the buffer is about to be exhausted. In this case, the audio data reconstruction operation must be started to perform audio compensation, so as to avoid staccato sound after playing the sound wave W1 at the receiving end, as shown in the staccato region GDP in FIG. 8 .

Therefore, the audio data reconstruction system and method of the above-mentioned embodiments of the present invention are adopted. After receiving the sound wave W1, the receiving end RX can execute the above steps S201-S213 sequentially when it does not receive the next sound wave, so as to extrapolate the sound wave W2 shown in FIG. 9 according to the sound wave W1 and reconstruct it after the sound wave W1. In this way, after playing the sound wave W1, the receiving end RX can then play the reconstructed sound wave W2, and then play the received sound wave W3, so as to avoid sound interruption during the playing process.

For example, the complete sound wave W1 shown in FIG. 8 received by the receiving end RX from the sending end TX within a predetermined time may include sound waves of frames FR1 to FR7 as shown in FIG. 3 , while the broken sound waves shown in FIG. 8 The sound area GDP may correspond to the staccato area Gap as shown in FIG. 3 .

[Advantageous Effects of Embodiment]

One of the beneficial effects of the present invention is that the audio data reconstruction method and method provided by the present invention can reconstruct lost or late sound waves, and the main advantages are as follows: 1. Computation is only performed at the receiving end, and no special coding or calculation is required at the sending end; 2. Use pure PCM data for calculation, which has nothing to do with the compression method of audio data; 3. The operation is simple, the amount of calculation is low, and it is suitable for devices with low power consumption and low computing power; 4. There is no need for the intact packet after the lost or late packet, but only the intact packet before the lost or late packet for calculation, which is suitable for low-latency playback devices.

The content disclosed above is only a preferred feasible embodiment of the present invention, and does not therefore limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made by using the description and drawings of the present invention are included in the application of the present invention. within the scope of the patent.

S101~S107, S201~S211: steps FR1~FR7: frame Gap, GDP: staccato area TH: Threshold value TX: sender AU: audio data RX: Receiver 10: Audio receiving module 20:Audio conversion module 30: Audio Filter Module 40: Extrapolation operation module 50: Phase calculation module 60: Noise filling module W1, W2, W3: Sound waves WAVE1, WAVE2: spectrum waveform

FIG. 1 is a flow chart of the steps of the audio data reconstruction method according to the first embodiment of the present invention.

FIG. 2 is a flow chart of the steps of the audio data reconstruction method according to the second embodiment of the present invention.

FIG. 3 is a schematic diagram of calculating the threshold value based on the previous frame of the lost or delayed audio packet according to the second embodiment of the present invention.

FIG. 4 is a spectrum diagram of the frame before the frame of the lost or delayed audio packet after the fast Fourier transform operation is performed according to the second embodiment of the present invention.

FIG. 5 is a spectrum diagram of a reconstructed frame according to the second embodiment of the present invention.

FIG. 6 is a block diagram of an audio data reconstruction system according to a third embodiment of the present invention.

FIG. 7 is a schematic diagram of the mobile phone sending audio data to the earphone using the audio data reconstruction system of the third embodiment.

Figure 8 is a waveform diagram of a staccato.

FIG. 9 is a waveform diagram of audio data reconstructing lost or late sound waves through the audio data reconstruction system and method according to an embodiment of the present invention.

S201~S211: steps

Claims (9)

A method for reconstructing audio data, comprising the following steps: An audio data is divided into multiple audio packets by a sending end and sent to a receiving end in sequence; Using the receiving end, performing a fast Fourier transform operation on the audio packets of the preceding frames of the frame in which the audio packet is lost or delayed, so as to convert the audio packet from the time domain to the frequency domain; Using the receiving end to calculate a threshold value according to the audio packet of the frame preceding the frame of the audio packet lost or delayed; Using the receiving end, comparing the amplitude of each frequency segment of each frame preceding the frame in which the audio packet is lost or delayed with the threshold value for screening; Using the receiving end, performing linear prediction on all the amplitudes screened out for fast Fourier transform operation, to extrapolate the lost or delayed delivery of the audio packets; and The receiving end is used to perform an inverse fast Fourier transform operation, and the extrapolated frequency segment is subjected to an inverse fast Fourier transform operation, so as to convert the audio packet obtained through the extrapolation from the frequency domain back to the time domain. The audio data reconstruction method as described in claim 1 further includes the following steps: using the receiving end to calculate the extrapolated amplitude and phase of the audio packet; Using the receiving end, in the extrapolated audio packet, the real part and the imaginary part of the frequency segment whose amplitude is not greater than the threshold value are filled with noise; and Using the receiving end, an inverse fast Fourier transform operation is performed to transform the audio packet obtained after extrapolation and noise filling from the frequency domain back to the time domain. The audio data reconstruction method as described in claim 1 further includes the following steps: Using the receiving end, calculate and filter out the amplitude of each frequency segment of the previous frame after performing the fast Fourier transform operation; using the receiving end, summing up the amplitudes of all the frequency segments of the previous frame to obtain a total amplitude; and Using the receiver, the threshold value is calculated based on the total amplitude. The audio data reconstruction method as described in claim 3 further includes the following steps: Using the receiving end, dividing the total amplitude by a signal-to-noise ratio to obtain the threshold value. An audio reconstruction system comprising: A sending end, configured to divide an audio data into a plurality of audio packets, and send the plurality of audio packets in sequence; and A receiver, including: An audio receiving module, connected to the sending end, configured to sequentially receive the plurality of audio packets sent by the sending end; An audio conversion module, connected to the audio receiving module, configured to perform fast Fourier transform operations on the audio packets of the first few frames of the frame that was lost or delayed in delivering the audio packet, so as to convert the audio packet from the time domain to the frequency domain; An audio screening module, connected to the audio conversion module, configured to calculate a threshold value based on the audio packet of the previous frame of the frame that was lost or delayed in delivery of the audio packet after the fast Fourier transform operation, and compare the amplitude and the threshold value of each frequency segment of each frame preceding the frame in which the audio packet was lost or delayed for screening; and an extrapolation operation module, connected to the audio screening module and the audio conversion module, configured to perform linear prediction on all the amplitudes screened out for fast Fourier transform operation, extrapolate the lost or delayed delivery of the audio packets, and then the Inverse Fast Fourier Transform operation is performed on the frequency segment derived from the audio conversion module, so as to convert the audio packet obtained through extrapolation from the frequency domain back to the time domain. The audio reconstruction system as claimed in claim 5, wherein the receiving end further includes a phase calculation module connected to the audio screening module and configured to calculate the extrapolated amplitude and phase of the audio packet. The audio reconstruction system as described in claim 6, wherein the receiving end further includes a noise filling module, connected to the phase calculation module, the extrapolation calculation module and the audio conversion module, configured to extrapolate the audio In the packet, the real and imaginary parts of the frequency segment whose amplitude is not greater than the threshold are filled with noise; The audio conversion module performs an inverse fast Fourier transform operation to convert the audio packet obtained after extrapolation and noise filling from the frequency domain back to the time domain. The audio reconstruction system as described in claim 5, wherein the audio screening module is configured to calculate and filter out the amplitude of each frequency segment of the previous frame after performing the fast Fourier transform operation, and sum up the amplitude of all frequency segments of the previous frame amplitude to obtain a total amplitude, and based on the total amplitude, the threshold value is calculated. The audio reconstruction system as claimed in claim 8, wherein the audio screening module is configured to divide the total amplitude by a signal-to-noise ratio to obtain the threshold value.

Images