CN1581292A - A Nonlinear Overlap Method for Time Series Transformation - Google Patents

Abstract

The present invention provides a method for converting S₁ [n]And S₂ [n]Synthesis into S₃ [n]The non-linear overlapped time sequence conversion method of (1), wherein S₁ [n]Containing N₁A signal, S₂ [n]Containing N₂A signal, the method comprising the steps of: will S₂ [n]Delaying a predetermined number to form S₅ [n](b) establishing S₁ [n]And S₅ [n]And (c) correlating S with S₃ [n]The setting is as follows: s₁ [n]When 0 < n < (the predetermined number + the maximum index value corresponding to the maximum correlation value in the correlation table + the first threshold value); s₁ [n]Weighted synthesis S₄ [n]When (the predetermined number + the maximum index value + the first threshold value) < ≦ N < (N)₁-a second critical value); s₄[ n- (the predetermined number + the maximum index value)]When ₁-the second critical value N₂+ the predetermined number + the maximum index value; wherein the first and second threshold values are not zero at the same time, and S₄ [n]Is S₅ [n]The maximum index value is delayed.

Description

A Nonlinear Overlap Method for Time Series Transformation

technical field

The present invention relates to providing a signal synthesis method, in particular to a nonlinear overlap method applied to time scaling.

Background technique

With the advancement of technology, some audio-visual playback devices such as karaoke can provide more and more functions, such as audio clean-up, dream sound field (dream), and timing conversion ( time scaling) and other functions. The so-called timing conversion (also known as time stretching, time compression/expansion or time correction) is to change the length of the audio signal without affecting the pitch (pitch), that is, to change the playback rate (tempo) of the audio signal.

At present, most audio-visual devices on the market use the following three methods to complete timing conversion, one is Phase Vocoder, the other is MPEX (Minimum Perceived Loss Time Expansion/Compression), and the other is Time Domain Harmonic Scaling (TDHS) . Phase vocoder first uses STFT (Short Time Fourier Transform) to convert an audio signal into a Fourier-type frequency domain signal (complex Fourier representation), and then uses the inner difference and iSTFT (inverse) to convert the frequency domain signal into a corresponding The audio signal is time scaled with respect to the timing of the audio signal. MPEX was recently developed by Prosoniq. MPEX is a method of simulating the characteristics of human hearing, similar to artificial neural network (artificial neural network). MPEX is based on the audio signal recorded in a specific time sequence, and then "learns" various characteristics of the audio signal in the specific time period, in order to try to extend or shorten the audio signal. TDHS is a more common method for time series conversion. It first calculates each correlation value (magnitudes of a autocorrelogram function) in the correlation table (autocorrelogram) of the first audio signal, and then calculates each correlation value (magnitudes of a autocorrelation function) according to the maximum The maximum index value corresponding to the correlation value delays the first audio signal to generate a second audio signal, and then copies the first audio signal to the second audio signal in a synchronized overlap-add (SOLA) manner to generate a third audio signal longer than the first audio signal.

Generally speaking, the above-mentioned correlation table is established through a digital signal processor (DSP), and the DSP is specially used for processing complex mathematical operations such as convolution calculation (convolution), fast Fourier transform (fast Fouriertransform, FFT), etc. . Even so, the process of DSP overlapping and synthesizing all parts of the first audio signal overlapping the second audio signal with the second audio signal to form the third audio signal is not only tedious, but also to some extent no need.

Contents of the invention

Therefore, the main object of the present invention is to provide a non-linear overlapping method for time sequence conversion, which rapidly synthesizes the first audio signal and the second audio signal into the third audio signal without The quality of the third audio signal is significantly affected.

According to the claims of the present invention, the present invention discloses a nonlinear overlapping timing conversion method for synthesizing S ₁ [n] and S ₂ [n] into S ₃ [n], wherein S ₁ [n] comprising N ₁ signals, and S ₂ [n] comprising N ₂ signals, the method comprises the following steps: (a) delaying S ₂ [n] by a predetermined number to form S ₅ [n], (b) establishing S A correlation table of ₁ [n] and S ₅ [n], and (c) setting S ₃ [n] as:

S ₁ [n], when 0<=n<(the predetermined number+the maximum index value corresponding to the maximum correlation value in the correlation table+the first critical value);

S ₁ [n] is weighted and synthesized in S ₄ [n], when (the predetermined number+the maximum index value+the first critical value)<=n<(N ₁ -the second critical value);

S ₄ [n-(the predetermined number+the maximum index value)], when (N ₁ -the second critical value)<=n<=N ₂ +the predetermined number+the maximum index value;

The first and second critical values are not zero at the same time, and S ₄ [n] is the maximum index value of S ₅ [n] delay.

In the method of the present invention, only a part of the part of the first audio signal that overlaps the second audio signal is weighted and synthesized with the second audio signal to generate the third audio signal. Therefore, it can be used to process timing conversion The operating performance of the computer where the DSP is located.

Description of drawings

Fig. 1 is the flowchart of the method of the present invention.

Fig. 2 is a schematic diagram of synthesizing S ₁ [n] and S ₂ [n] into S ₃ [n] by the method of the present invention.

Fig. 3 is a schematic diagram of increasing an audio signal by the method of the present invention.

Fig. 4 is a schematic diagram of shortening an audio signal by the method of the present invention.

Graphical Symbol Description

Δ predetermined number τ _max maximum index value

th ₁ first threshold th ₂ second threshold

Detailed ways

After establishing the correlation table corresponding to the first audio signal and the second audio signal (or the audio signal delayed in the second audio signal), the method 100 in the preferred embodiment of the present invention is based on the maximum The maximum index value corresponding to the correlation value, the first critical value, the second critical value and the first audio signal and the second audio signal to calculate the third audio signal. Specifically, in order to save the calculation time of the DSP for synthesizing the first audio signal and the second audio signal to generate the third audio signal, the method 100 calculates the maximum index value and delays the second audio signal After the maximum index value, instead of weighting and synthesizing all parts of the first audio signal overlapping the second audio signal, only the first audio signal overlapping the second audio signal A part of the signal parts (ie, the overlapping part between the first threshold and the second threshold in the overlapping part) is weighted and combined with the second audio signal to generate the third audio signal.

Please refer to FIG. 1 , which is a flowchart of a method 100 in a preferred embodiment of the present invention. Method 100 includes the following steps:

Step 102: start;

(S ₁ [n] and S ₂ [n] will be synthesized into S ₃ [n], assuming S ₁ [n] and S ₂ [n] contain N ₁ and N ₂ signals respectively)

Step 104: Delay S ₂ [n] by a predetermined number Δ to form S ₅ [n];

(In order to avoid the phenomenon that the optical pickup head (pickuphead) in the audio-visual playback device reads S ₃ [n], the phenomenon of insufficient read data (run-in), so the method 100 of the present invention is to first convert S ₂ [n ] After delaying the predetermined number Δ, the maximum index value τ _max required for synthesizing S ₁ [n] and S ₅ [n] is calculated. In a preferred embodiment of the present invention, the predetermined number Δ is equal to [N ₁ /3] )

Step 106: Establish a correlation table (crosscorrelogram) of S ₁ [n] and S ₅ [n] and delay S ₅ [n] according to the maximum index value τ _max corresponding to the maximum correlation value in the correlation table to form S ₄ [ n];

(The correlation table contains multiple correlation values (magnitudes of a crosscorrelation function), and each correlation value corresponds to an index value)

Step 108: Synthesize S ₁ [n] and S ₄ [n] into S ₃ [n];

(S ₃ [n] is set to:

S ₁ [n], when 0<=n<(predetermined number Δ+maximum index value τ _max +first critical value th ₁ );

S ₁ [n] is weighted and synthesized on S ₄ [n], when (predetermined number Δ+maximum index value τ _max +first critical value th ₁ )<=n<(N ₁ -second critical value th ₂ );

S ₄ [n-(predetermined number Δ+maximum index value τ _max )], when (N ₁ -second critical value th ₂ )<=n<=N ₂ +predetermined number Δ+maximum index value τ _max ;

Wherein the first critical value _th1 and the second critical value _th2 are not zero at the same time)

Step 110: end.

Please refer to FIG. 2 . FIG. 2 is a schematic diagram of synthesizing S ₁ [n] and S ₂ [n] into S ₃ [n] in a preferred embodiment of the present invention. The first part 401 in Fig. 4 is S ₁ [n] and S ₂ [n] in the step 102 of the display method 100, and the second part 402 is S ₁ [n] and S ₅ [ in the step 104 of the display method 100 n], the third part 403 is the calculated τ _max and S ₄ [n] in the step 106 of the display method 100, and the fourth part 404 and the fifth part 405 are displayed in the step 108 of the method 100 by S ₁ [ n] and S ₃ [n] synthesized by S ₄ [n].

S ₃ [n] shown in the fourth part 404 of FIG. 2 is at (predetermined number Δ+maximum index value τ _max +first critical value th ₁ )<=n<(N ₁ -second critical value th ₂ ) is equal to:

$\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; th</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

And S ₃ [n] shown in the fifth part 405 of FIG. 2 is (predetermined number Δ+maximum index value τ _max +first critical value th ₁ )<=n<(N ₁ -second critical value th ₂ ) is equal to:

$\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span>}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&tau;</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

If the above S ₁ [n] is completely equal to S ₂ [n], that is, both S ₁ [n] and S ₂ [n] are separated from the same position of S[n], as shown in Figure 3, then the method 100 is the growth S ₁ [n]. Conversely, if S ₁ [n] and S ₂ [n] are not equal, that is, S ₁ [n] and S ₂ [n] are separated from different positions of S[n], as shown in Figure 4, then Method 100 shortens S ₁ [n], S ₆ [n] (discarded), and S ₂ [n] into S ₃ [n].

Compared with the known TDHS, the method of the present invention is based on the maximum index value corresponding to the maximum correlation value in the correlation table and the first two used to reduce the overlap of S ₁ [n] and S ₂ [n] and the second critical value to calculate S ₃ [n] synthesized in S ₁ [n] and S ₂ [n]. Since the present invention does not need to calculate all the values of S ₁ [n] overlapping S ₂ [n] one by one after calculating the maximum index value, that is, it only needs to calculate the values between the first and the second in S ₃ [n]. Part of the value between the two critical values, thus saving the time required for the DSP calculation of S 3 [n] based on S ₁ [n] and S ₂ [n] to synthesize _{S 3 [} n], and jointly, also Increase the operating performance of the computer where the DSP is located.

The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the claims of the present invention shall fall within the scope of the patent of the present invention.

Claims (19)

1. a non-linear method of superposition that is used for the sequential conversion is used for S ₁[n] and S ₂[n] synthesizes S ₃[n], S ₁[n] comprises N ₁Individual signal, and S ₂[n] comprises N ₂Individual signal, this method comprises the following step:

(a) with S ₂[n] postpones a predetermined number to form S ₅[n];

(b) set up S ₁[n] and S ₅The correlation table of [n] comprises a plurality of correlations in this correlation table, all corresponding index value of each correlation; And

(c) according to the pairing maximum index value of the maximum related value in this correlation table, with S ₃[n] sets for:

S ₁[n] is when 0＜=n＜(this predetermined number+this maximum index value+first critical value);

S ₁[n] weighting is synthesized in S ₄[n] is as (this predetermined number+this maximum index value+this first critical value)＜=(N ₁-the second critical value) time;

S ₄[n-(this predetermined number+this maximum index value)] is as (N ₁-this second critical value)＜=n＜=N ₂+ this predetermined number+this maximum index value;

Wherein this first, second critical value is not zero simultaneously, and S ₄[n] is S ₅[n] postpones this maximum index value.

2. the method for claim 1 is wherein worked as (this predetermined number+this maximum index value+this first critical value)＜=n＜(N ₁-the second critical value) time, S ₃[n] equals (N ₁-this second critical value-n)/(N ₁-(this predetermined number+this maximum index value+this first critical value+this second critical value)) * S ₁[n]+(n-(this predetermined number+this maximum index value+this first critical value))/(N ₁-(this predetermined number+this maximum index value+this first critical value+this second critical value)) * S ₄[n-(this predetermined number+this maximum index value)].

3. the method for claim 1 is wherein worked as (this predetermined number+this maximum index value+this first critical value)＜=n＜(N ₁-the second critical value) time, S ₃[n] equals (N ₁-n)/(N ₁-(this predetermined number+this maximum index value)) * S ₁[n]+(n-(this predetermined number+this maximum index value))/(N ₁-(this predetermined number+this maximum index value)) * S ₄[n-(this predetermined number+this maximum index value)].

4. the method for claim 1, wherein S ₁[n] and S ₂[n] takes a sample from S respectively ₁(t) and S ₂(t).

5. method as claimed in claim 3, wherein S ₁(t) and S ₂(t) be to separate from an original signal.

6. method as claimed in claim 5, wherein this original signal is a sound signal.

7. method as claimed in claim 5, wherein this original signal is a vision signal.

8. method as claimed in claim 4, wherein S ₁(t) be to equal S ₂(t).

9. method as claimed in claim 4, wherein S ₁(t) be to be not equal to S ₂(t).

10. the method for claim 1, wherein this predetermined number is to equal [N ₁/ 3].

11. a non-linear method of superposition that is used for the sequential conversion is used for S ₁[n] and S ₂[n] synthesizes S ₃[n], S ₁[n] comprises N ₁Individual signal, and S ₂[n] comprises N ₂Individual signal, this method comprises the following step:

(a) set up S ₁[n] and S ₂The correlation table of [n] comprises a plurality of correlations in this correlation table, all corresponding index value of each correlation; And

(b) according to the pairing maximum index value of the maximum related value in this correlation table, with S ₃[n] sets for:

S ₁[n] is when 0＜=n＜(this maximum index value+first critical value);

S ₁[n] weighting is synthesized in S ₄[n] is as (this maximum index value+this first critical value)＜=n＜(N ₁-the second critical value) time;

S ₄[this maximum index value of n-]], as (N ₁-this second critical value)＜=n＜=(N ₂+ this maximum index value);

Wherein this first, second critical value is not zero simultaneously, and S ₄[n] is S ₂[n] postpones this maximum index value.

12. method as claimed in claim 11 is wherein as (this maximum index value+this first critical value)＜=n＜(N ₁-the second critical value) time, S ₃[n] equals (N ₁-this second critical value-n)/(N ₁-(this maximum index value+this first critical value+this second critical value)) * S ₁[n]+(n-(this maximum index value+this first critical value))/(N ₁-(this maximum index value+this first critical value+this second critical value)) * S ₄[n-(this maximum index value)].

13. method as claimed in claim 11 is wherein as (this predetermined number+this maximum index value+this first critical value)＜=n＜(N ₁-the second critical value) time, S ₃[n] equals (N ₁-n)/(N ₁-(this predetermined number+this maximum index value)) * S ₁[n]+(n-(this predetermined number+this maximum index value))/(N ₁-(this predetermined number+this maximum index value)) * S ₄[n-(this predetermined number+this maximum index value)].

14. method as claimed in claim 11, wherein S ₁[n] and S ₂[n] takes a sample from S respectively ₁(t) and S ₂(t).

15. method as claimed in claim 14, wherein S ₁(t) and S ₂(t) be to separate from an original signal.

16. method as claimed in claim 15, wherein this original signal is a sound signal.

17. method as claimed in claim 15, wherein this original signal is a vision signal.

18. method as claimed in claim 14, wherein S ₁(t) be to equal S ₂(t).

19. method as claimed in claim 14, wherein S ₁(t) be to be not equal to S ₂(t).

Images