KR101336137B1 - Method of fast normalized cross-correlation computations for speech time-scale modification - Google Patents

Abstract

The present invention can reduce the normalized cross-correlation calculation amount for speech time-base conversion while producing a synthesized sound result without sound deterioration that is exactly the same as existing SOLA and WSOLA by eliminating redundancy of normalized cross-correlation function calculation using a sum table. In order to provide a method, the method further comprises the steps of: setting an arbitrary reproduction speed for the stored voice signal, and applying the sum table only to the denominator in the formula of the normal cross-correlation when the set speed is a high speed. When the double speed is low, the step of applying the sum table to the numerator and the denominator in the formula of the normal cross-correlation, respectively, and calculates the optimum moving value by calculating the fast normal cross-correlation using the generated sum table Determining the degree of similarity according to the calculated optimal shift value and removing the duplication of the cross-correlation function; Following the last frame, makin comprises the steps of synthesizing and outputting the output signal.

Description

Method of fast normalized cross-correlation computations for speech time-scale modification

The present invention relates to a time-base conversion method of speech, and more particularly, to a method of reducing the amount of calculation of an overlap-add technique based on waveform similarity (WSOLA).

Various methods have been proposed for converting the speech signal to the speech time base. Among them, the conversion methods in the time domain are widely used because they have a small calculation amount and excellent performance. In other words, simply increasing or decreasing the overall time axis of a speech signal distorts the spectral information and sounds like modulated speech. The time-base conversion technique is used to change only the speech rate while minimizing the distortion of such spectral information.

The sound velocity conversion methods in the time domain basically used simple OLA (overlap and add) method, which is simple to implement hardware, but this is insufficient to minimize distortion of spectral information, and then synchronized overlap and add (SOLA) and WSOLA (an Overlap-Add technique based on Waveform Similarity, and Pitch-Synchronous Overlap-Add (PSOLA). SOLA and WSOLA are similar ways to synchronize waveforms to improve the quality of the output voice.

As such, SOLA and WSOLA are typical methods for realizing digital signal processing techniques for changing audio signal length, and SOLA and WSOLA can generate high quality time scaled output signals.

Currently, research is being conducted to reduce the amount of computation of WSOLA, a time-base transformation technology. However, the method of reducing the search interval by using the periodic characteristics of the conventional speech reduces the computational amount, while the synthesis result of the time-base transformation slightly degrades the sound quality of the original time-base method.

Recently, mobile devices are in the spotlight, and mobile devices are currently lacking in performance for SOLA and WSOLA-based voice timebase conversion. Therefore, in order to apply SOLA and WSOLA in such insufficient devices, SOLA and WSOLA-based The normalized cross-correlation for the speech time-base conversion of NF needs to solve the problem of sound quality reduction as well as the reduction of the computational amount.

Accordingly, the present invention has been made to solve the above problems, by eliminating the redundancy of the normalized cross-correlation function calculation using the sum table, while producing a speech result without sound deterioration that is exactly the same as the existing SOLA and WSOLA. Its purpose is to provide a way to reduce the amount of normalized cross-correlation for transforms.

Another object of the present invention is that when calculating cross-correlation, there is a redundancy of calculation only for the denominator portion of the normalized cross-correlation function at high speed, and at both the denominator and the numerator of the normalized cross-correlation function at low speed. In consideration of the redundancy of calculations, it is possible to reduce the amount of calculation by applying the sum table of the numerator and denominator to the calculation of the normal cross-correlation of the speech signal according to the degree of duplication and the time-base transformation speed of the speech.

에 대해서 임의의 재생 배속(

)을 설정하는 단계와, (B) 상기 설정된 배속에 따라, 최적의 이동값

를 구하기 위해서 음성 신호와 출력 신호를 평가하는 기준으로 사용되는 정규상호상관도의 수식에 합 테이블을 분모에만 적용할 것인지, 또는 분자와 분모에 모두 적용할 것인지를 설정하는 단계와, (C) 상기 (B) 단계에서 합 테이블을 분모에만 사용하는 경우, 해당 프레임의 분모에만 합 테이블을 생성하고, 상기 (B) 단계에서 합 테이블을 분모에만 사용하는 경우, 해당 프레임의 분자와 분모에 각각 합 테이블을 생성하는 단계와, (D) 상기 생성된 합 테이블을 이용하여 중복이 제거된 정규상호상관도 계산으로 유사도를 판정하여 이를 최대화하는 값을 최적 이동값으로 산출하는 단계와, (E) 상기 유사도 판정에 따라 동기화된 위치에 중첩을 가산하는 단계와, (F) 상기 (D) 단계의 중복 제거를 마지막 프레임까지 수행한 후, 합성하여 출력 신호로 출력하는 단계를 포함하여 이루어지는데 있다.A feature of the fast normalized cross-correlation calculation method for speech time-base transformation according to the present invention for achieving the above object is (A) the stored speech signal

Any playback speed for

) And (B) the optimum moving value according to the set double speed

(C) setting whether to apply the sum table only to the denominator or to both the numerator and the denominator in the equation of the normal cross-correlation that is used as a criterion for evaluating the speech signal and the output signal in order to obtain. When the sum table is used only for the denominator in step (B), the sum table is generated only for the denominator of the corresponding frame, and when the sum table is used only for the denominator in step (B), the sum table is added to the numerator and denominator of the frame, respectively. (D) determining similarity by calculating normal cross-correlation with duplicates removed using the generated sum table, and calculating a value for maximizing it as an optimal shift value, and (E) the similarity. Adding overlap to the synchronized position according to the determination; and (F) performing deduplication of the step (D) to the last frame, and then synthesizing and outputting the output signal. It comprises a step.

)은 시간축 변환 비율로써 1을 기준으로 그보다 클 경우 고배속을, 1보다 작을 경우 저배속을 나타내는 것을 특징으로 한다.Preferably the regeneration speed (

) Is a time-base conversion ratio, which is characterized by high speed when the value is greater than 1 and low speed when the value is less than 1.

Preferably, if the regeneration speed set in step (B) is high, the sum table is applied only to the denominator, and if the set regeneration speed is low, the sum table is applied to both the numerator and the denominator.

이며, 이때,

는 k번째 프레임의 기준 신호를 의미하고,

는 k번째 프레임의 유사도 판별을 할 신호 즉, 이동값

에 의해 이동하는 비교 신호를 나타낸 것이며, L은 중첩 길이에 해당하는 값이고, k는 합성 프레임의 인덱스를 나타내는 것을 특징으로 한다.Preferably the formula of the normal cross-correlation

Lt; / RTI >

Means the reference signal of the k-th frame,

Is a signal to determine the similarity of the kth frame, that is, a moving value

The comparison signal is moved by, L is a value corresponding to the overlap length, k is characterized in that the index of the composite frame.

Preferably, the optimum shift value is a value that maximizes the expression of the normal correlation.

와 같이 회귀식의 형태로 생성되는 것을 특징으로 하는 특징으로 한다.Preferably, the sum table of the denominators generated in the step (C) is

It is characterized in that it is generated in the form of a regression equation.

와 같이 생성되는 것을 특징으로 한다.Preferably, the sum table of the denominators generated in step (C) is

It is characterized in that it is generated as.

Preferably, step (E) is characterized by using the method used in WSOLA, which is a time-base conversion method of speech.

Preferably, the denominator portion is the energy of the comparison signal, and the molecular portion is a cross-correlation function of the comparison signal.

A feature of the fast normalized cross-correlation calculation method for speech timebase conversion according to the present invention for achieving the above object is the step of setting an arbitrary reproduction speed for the stored speech signal, Applying a sum table only to the denominator in the formula of the normal cross-correlation for high speed, and applying the sum table to the numerator and the denominator in the formula for normal cross-correlation, if the set double speed is low. Calculating an optimal moving value by calculating a fast normal cross-correlation using the calculated sum table, determining similarity according to the calculated optimum moving value, and eliminating duplication of the cross-correlation function; And performing the removal to the last frame, then synthesizing and outputting the output signal.

Preferably, the reproduction double speed is a time-base conversion ratio, and is characterized by high speed when it is larger than 1 and low speed when it is smaller than 1.

As described above, the fast normalized cross-correlation calculation method for speech time-base transformation according to the present invention has a high quality through additional reduction of the normalized cross-correlation calculation amount for WSOLA-based speech time-base transformation even in a device lacking its performance. Can generate a time scaled output signal. This can reduce the amount of computation without degrading sound quality for SOLA and WSOLA among the methods of time-base transformation.

Accordingly, it is possible to perform high speed reproduction and search without distorting the tone of the voice data, and to reduce the load on the system as the calculation amount of the voice decoding decreases.

1 is a flowchart illustrating a fast normalized cross-correlation calculation method for WSOLA-based speech timebase conversion according to an embodiment of the present invention.

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

A preferred embodiment of a fast normalized cross-correlation calculation method for speech time-base transformation according to the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to let you know. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

1 is a flowchart illustrating a fast normalized cross-correlation calculation method for WSOLA-based speech timebase conversion according to an embodiment of the present invention.

에 대해서 재생 배속(

)을 설정한다(S10). Referring to Figure 1, first, the pre-stored voice signal

Regeneration double speed for (

(S10).

에 대해서 출력 신호

는 다음 수학식 1과 같이 주어진다.Typically the speech signal input in the calculation for speech timebase conversion

Output signal

Is given by Equation 1 below.

는 k번째 프레임에서의 최적 이동값을 나타낸다. 또한

는 시간축 변환 비율로써 1을 기준으로 그보다 클 경우 빠른 속도(고배속)로의 변환이고, 1보다 작을 경우 느린 속도(저배속)로의 변환이 된다. In this case, in Equation 1, L is a value corresponding to the overlap length, and the window length is N = 2L since the window is always half overlapped. K denotes the index of the composite frame,

Denotes an optimal shift value in the k th frame. Also

When the ratio is larger than 1, the speed is converted to a high speed (high speed), and when it is less than 1, the speed is converted to a slow speed (low speed).

를 구하기 위해서 음성 신호와 출력 신호를 평가하는 기준으로 정규상호상관도를 사용하며, 다음 수학식 2와 같이 나타낼 수 있다.Therefore, the optimum moving value

In order to obtain, the normal cross-correlation is used as a criterion for evaluating the speech signal and the output signal, and it can be expressed as Equation 2 below.

The two signals compared in Equation 2 may be defined as in Equation 3 below.

는 k번째 프레임의 기준 신호를 의미하고,

는 k번째 프레임의 유사도 판별을 할 신호 즉, 이동값

에 의해 이동하는 비교 신호를 나타낸 것이다. In Equations 2 and 3

Means the reference signal of the k-th frame,

Is a signal to determine the similarity of the kth frame, that is, a moving value

This shows a comparison signal moving by.

는 상기 수학식 2를 최대화하는 값으로, 다음 수학식 4와 같이 구할 수 있다.Therefore, the optimum moving value

Is a value maximizing Equation 2, and can be obtained as Equation 4 below.

에 의해 정해지고,

이 클수록 계산량이 증가한다.At this time, the range for comparison is

Determined by

The larger the value, the greater the amount of calculation.

가 이동하는 구간,

만큼 반복해서 계산하게 된다. 따라서 수학식 2의 계산량을 줄이는 것이 WSOLA의 계산량 감소로 직결된다. 상기 수학식 2를 살펴보면 근접한 구간 내에서 중복성을 가짐을 알 수 있고, 이러한 중복에 대해서 계산을 피하는 방법인 합 테이블(sum table) 방법을 사용하면 계산 횟수를 줄일 수 있다.Like this,

Is the interval to move,

It will be calculated repeatedly. Therefore, reducing the calculation amount of Equation 2 directly leads to the reduction of the calculation amount of the WSOLA. Looking at Equation 2, it can be seen that there is redundancy in the adjacent section, and the number of calculations can be reduced by using a sum table method, which is a method of avoiding calculation for such overlap.

Accordingly, the sum table determines whether to apply only to the denominator of Equation 2 or both to the numerator and the denominator (S20). In the case of the denominator, duplicate calculations occur in the calculation of the proximity signal, and the sum table can be applied to remove them. In the case of numerators, there is no room for redundancy because the calculations are performed on new signals each time at high-speed transformations. However, at slower speeds, redundancy occurs, and using sum tables reduces computation.

Therefore, it is determined according to the double speed set in step S10, and one of the method of using the sum table of the denominator and the method of using both the sum table of the denominator and the numerator is selected.

As a result of the selection (S20), when using the sum table of only the denominator, the sum table is generated only on the denominator of the corresponding frame (S30).

인 최적 이동값

를 구하는 과정에 있어서, 이동값

와 상관없이 계산되는 기준 신호

의 에너지는 상수 성분으로 볼 수 있다. 따라서 수학식 2에서는 계산할 필요가 없다.The denominator of Equation 2 is the energy of each signal. By the way, in Equation 2 to maximize the moving value

Optimal shift

In the process of finding,

Reference signal calculated regardless of

The energy of can be seen as a constant component. Therefore, it is not necessary to calculate in Equation 2.

의 에너지는 인접 구간에 대해서 중복 계산을 한다. 이 경우에 합 테이블 방법을 쓰면 인접 구간 에너지에 대해서 중복 계산을 제거할 수 있다. 상기 합 테이블 방법은 먼저 에너지 계산을 하기 전에 다음 수학식 5와 같이 합 테이블을 만드는데 회귀식의 형태로 계산한다.In addition, the comparison signal in Equation 2

The energy of is duplicated for adjacent sections. In this case, the sum table method can eliminate duplicate calculations for adjacent section energy. The sum table method first generates a sum table as shown in Equation 5 before calculating the energy, and calculates it in the form of a regression equation.

Using the sum table thus generated, a high speed normal cross-correlation is calculated to calculate an optimum moving value to determine an optimum similarity (S40).

의 에너지는 다음 수학식 6과 같이 정리될 수 있다.That is, using the table of Equation 5

The energy of can be summarized as in Equation 6 below.

를 산출하고 있다. 따라서 최적의 이동값을 산출하게 되면 이를 통해 최적의 유사도를 판정할 수 있게 된다.In order to use the proposed signal period estimation for the purpose of reducing the conventional computations, we assume that the speech signal is periodic and utilize the relationship between the window size and the period to obtain the optimal shift value for optimal similarity.

Is calculating. Therefore, when the optimal movement value is calculated, the optimal similarity can be determined.

을 산출하여 최적의 유사도를 판정하게 된다. 이 테이블을 활용하면 인접한 구간에 대해 에너지를 구하는 정규상호상관도의 계산량은 대략 50% 감소된다.As such, the value of maximizing the equation (2), the optimum moving value

The optimum similarity is determined by calculating Using this table, the computation of normal cross-correlation for energy for adjacent intervals is reduced by approximately 50%.

The overlap is added to the synchronized position according to the similarity determination to remove the overlap of the cross-correlation function (S50). The method of eliminating duplication of the cross-correlation function by adding the overlap to the synchronized position uses the method used in the WSOLA, which is a time-base transformation method of the existing speech.

Deduplication of the cross-correlation function is performed until the last frame (S60), and then synthesized and output (S70).

In addition, when using the sum table for the selection result (S20), the denominator and the denominator, a sum table is generated for the denominator and the numerator of the corresponding frame (S30).

Then, using the sum table generated in the denominator and the numerator, the high-speed normal cross-correlation of which duplicates are removed is determined to determine similarity, and a value maximizing it is calculated as an optimal shift value (S90). In this case, since the denominator is performed in the same manner as described above, description according to the denominator is omitted, and only the case of the molecule is described.

The molecular portion of Equation 2 is a cross-correlation function. That is, in the low-speed transform, there is an overlap calculation portion of the cross-correlation function for adjacent frames. Therefore, in order to remove duplicate calculations only in the case of a low-speed time-base transformation, a sum table is formed as shown in Equation 7 for the molecular part of the normal correlation.

By using the sum table generated as described above, a high speed normal cross-correlation of which duplicates are removed is determined to determine the similarity, and an optimal shift value is calculated (S90).

Then, the overlap is added to the synchronized position according to the similarity determination (S100). At this time, as mentioned above, in the low-speed conversion, the sum table of the current frame may have a duplicate of the sum table of the previous frame or the next frame. As shown in the figure above, if you have a duplicate table value that is equal to the size of the duplicate, you can avoid the calculation as much as you get.

과

의 중첩 길이를 구하면 수학식 3에 의해

로 계산되며, 이 값이 음수일 경우 중첩 구간이 없는 것이므로 실제 중첩 길이

는 다음 수학식 8과 같이 표현된다.Therefore, in order to find the overlapping interval of the sum table, first, the reference signal

and

If we find the overlap length of

If this value is negative, there is no overlapping interval, so the actual overlap length

Is expressed by Equation 8 below.

에 대해서 이전 프레임의

와 중첩 구간을 구한다. Then another component of the cross-correlation product

Of previous frame

Find the overlapping interval with.

과

사이의 시간 차이는 의

구간을 가지고

과

가

의 구간을 갖는다. 따라서 중첩이 있는 경우를 고려한 구간

는 다음 수학식 9와 같이 나타난다.By Equation 4

and

The time difference between

With segment

and

end

Has a section of. Therefore, the interval considering the case of overlap

Is expressed as in Equation 9 below.

과

이 중첩되는 구간을 수학식 8에서

와

이 중첩되는 구간을 수학식 9에서 구했다.At this time,

and

This overlapping interval is expressed in Equation 8

Wow

This overlapping section was obtained from equation (9).

의 합 테이블 값을

에 가져다 쓸 수 있다.As such, the intervals overlapped by the equations (8) and (9) are the same cross-correlation function product and thus do not need to be recalculated.

Sum of table values

Can be used to

는 다음 식으로 표현되며 수학식 8과 수학식 9의 곱의 전체 프레임 합에 해당하는데, 저배속에서 배속이 낮을수록

과

의 중복이 많아지고, 또한 최적 이동값

과

의 차이가 적을수록 중복이 많아지고 분자의 전체 프레임의 계산량 감소는 다음 수학식 10과 같다.As such, the normal cross-correlation is the total overlap of the molecular components.

Is expressed by the following equation and corresponds to the total frame sum of the product of Equation 8 and Equation 9.

and

Of duplicates, and also the optimal shift

and

The smaller the difference is, the more overlap and the decrease in the calculation amount of the entire frame of the molecule is expressed by Equation 10 below.

Deduplication of the cross-correlation function is performed until the last frame (S110), and then synthesized and output (S70).

Although the technical spirit of the present invention described above has been described in detail in a preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (7)

(A) Stored voice signal

Any playback speed for

),
(B) the optimum moving value according to the set double speed

Setting whether to apply the sum table only to the denominator, or to both the numerator and the denominator, in the formula of the normal cross-correlation that is used as a criterion for evaluating the speech signal and the output signal to obtain
(C) When the sum table is used only for the denominator in step (B), if the sum table is generated only for the denominator of the frame, and the sum table is used only for the denominator in step (B), the numerator and denominator of the frame Generating a sum table for each;
(D) calculating similarity by calculating normal mutual correlation with duplicates removed using the generated sum table and calculating a value that maximizes this as an optimum moving value;
(E) adding the overlap to the synchronized position according to the similarity determination,
(F) performing a deduplication of the step (D) to the last frame, and then synthesizing and outputting the result as an output signal. The method of claim 1,
The regeneration speed

) Is a time-base conversion ratio of 1 to a higher value than 1, and a high-speed normalized cross-correlation calculation method for speech time-base conversion, characterized in that less than one. 3. The method of claim 2,
If the reproduction speed set in the step (B) is high, the sum table is applied only to the denominator. If the reproduction speed is low, the sum table is applied to both the numerator and the denominator. Normalized cross-correlation calculation method. The method of claim 1,
The formula of the normal mutual correlation

Lt;
At this time,

Means the reference signal of the k-th frame,

Is a signal to determine the similarity of the kth frame, that is, a moving value

A method of calculating a normalized cross-correlation for speech time-base transformation, wherein the comparison signal is moved by L, L is a value corresponding to the overlap length, and k is an index of the synthesized frame. The method of claim 1,
The sum table of the denominators generated in the step (C) is

A fast normalized cross-correlation calculation method for speech time-base transformation, characterized in that it is generated in the form of a regression equation. The method of claim 1,
The sum table of the denominators generated in step (C) is

A fast normalized cross-correlation calculation method for speech time-base transformation, characterized in that it is generated as follows. The method of claim 1,
The step (E) is a fast normalized cross-correlation calculation method for speech time-base conversion, characterized in that using the method used in WSOLA which is the time-base conversion method of speech.

Images