JP2012159540A - Speaking speed conversion magnification determination device, speaking speed conversion device, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine a speaking speed conversion magnification which is stable and adaptive even when a background sound and a voice are mixed.SOLUTION: A speaking speed conversion magnification determination device 1a comprises: a physical index calculation section 2 having a sound/soundless determination part 102 for determining a sound section and a soundless section of an input signal, a fundamental frequency calculation part 104 for calculating the fundamental frequency of the input signal and determining a stable section and an unstable section in the sound section, a frequency smoothing part 106 for smoothing the time change of the fundamental frequency in the stable section, a pseudo fundamental frequency calculation part 108 for calculating a pseudo fundamental frequency in which the fundamental frequency is interpolated in the unstable section and in the soundless section, and a fundamental frequency contour connection part 110 for connecting the smoothed fundamental frequency and the pseudo fundamental frequency to obtain a sample value of the fundamental frequency, and outputting the sample value of the fundamental frequency as a physical index; and a speaking speed conversion magnification specification part 120 for calculating a speaking speed conversion magnification based on the physical index.

Description

  The present invention relates to a speech speed conversion magnification determination device, a speech speed conversion device, a program, and a recording medium that determine an adaptive conversion magnification of a speech speed (speaking speed) of an input signal.

  In the conventional technology for adaptively converting the speech speed, when an arbitrary playback speed such as 1 × speed (playback in real time) or 2 × speed (playback in half the real time) is given, the entire input signal is applied. Rather than changing the speed at a uniform magnification α, the speed is changed at a magnification that is larger or smaller than the magnification α according to each part, so that the entire time is the same as when the speech speed is converted at a uniform magnification α. The purpose is to generate speech speed converted speech that is “slowly easier to hear” than the case where the speech speed is converted at a uniform magnification α for the listener who adjusts the bottom of the book so that it is played back.

  To that end, (1) slow the speech speed at a high fundamental frequency, speed up the speech speed at a low fundamental frequency, and (2) slow the speech speed at the start of the speech utterance as a unit. The speed of speech is gradually increased toward the end point according to the change of the fundamental frequency, and (3) the silent section between the sections uttered at a breath is shortened within a range in which there is no sense of incongruity. A technique is known (see, for example, Patent Document 1).

  In addition, a silent section longer than a certain length is set as a pause section, and in a voice section sandwiched between pause sections, the speech speed is reduced at the start point, and the speech speed is increased based on a predetermined decrease function over a predetermined time T. A technique is known in which after the time T has elapsed, the rate at which the speech speed is relaxed is changed in consideration of the magnitude relationship of the maximum fundamental frequency in each voiced section (see, for example, Patent Document 2).

  Further, in the speech speed control disclosed in Patent Document 1 and Patent Document 2, it is allowed to shorten a short silent section within a speech section sandwiched between pause sections within a range in which there is no sense of incongruity in hearing. If each part of the fast-converted speech matches the expected time when the speech speed is converted at a uniform magnification α, or if it is not very late, the subsequent speech speed is set to be as slow as possible. The greater the degree of delay with respect to the time assumed when the speech speed is converted with a large magnification α, the more the degree of slowing down the subsequent speech speed is set, so that each part of the speech speed converted speech is There is known a technique for reducing the deviation as much as possible with respect to the time assumed when the speech speed is converted at a uniform magnification α (see, for example, Patent Document 3).

  Furthermore, the input signal is divided into a voice interval and a silence interval, and the speech speed of the voice interval is slowed down and the silence interval is shortened. However, the input signal per unit time is reduced by reducing the speech speed of the voice interval. Since the output voice length is longer than the length, it is necessary to temporarily store the voice after speech speed conversion in the memory, but since there is an upper limit on the amount of memory, depending on the remaining amount of memory, Techniques for gradually increasing the speech speed or increasing the amount of silent section deletion are known (see, for example, Patent Documents 4 and 5).

  In addition, the coefficient is such that the increase / decrease in the magnitude (power) or height (fundamental frequency) of the input signal is inversely proportional to the speech speed, or the coefficient that the speech speed is inversely proportional to the nth power of the magnitude or height of the input signal. A technique for determining the speech speed of each part is also known (see, for example, Patent Document 6).

Japanese Patent No. 3249567 Japanese Patent No. 3219892 Japanese Patent No. 3220043 Japanese Patent No. 3357742 Japanese Patent No. 3373933 Japanese Patent No. 3619946

  What is common to the techniques described in Patent Documents 1 to 5 is that the input signal is divided into a voice section with voice and a silent section without voice. In the voice section, the duration is divided into parts based on some information. The silence period is shortened and the overall voice time length is adjusted comprehensively. However, when the input signal is only a human voice, there is no problem with these methods. However, when the background sound and sound are mixed, such as in a broadcast program, the section with only the background sound without sound is “silent. Whether it is determined to be “section” or “voice section” is not guaranteed, and if an erroneous determination occurs, correct operation cannot be expected, and the speech speed converted speech may be difficult to hear. .

  With respect to Patent Document 6, the magnitude (power) of the input voice can be obtained in all sections of the input voice, but the height (fundamental frequency) of the input voice is where there is a voice and the vocal cords vibrate. It can be obtained correctly only in the “voiced sound section”. Therefore, also in Patent Document 6, when the background sound and the sound are mixed, since the power is large and the fundamental frequency cannot be obtained correctly in the section of only the background sound without the sound, the speech speed is originally a section that is not a sound. Although it is necessary to speed up, it is possible that the speed of the speech will be relaxed because of the high power.

  As described above, in the conventional speech speed conversion method, when background sound and voice are mixed, if the determination of the voice section with voice and the silent section without voice is not performed correctly, There was a problem that typical speech speed conversion did not work as expected.

  In order to solve the above problem, an object of the present invention is to provide a speech speed conversion magnification determination device and a speech speed conversion device capable of stably determining an adaptive speech speed conversion magnification even when background sound and sound are mixed. It is to provide a program and a recording medium.

  In order to solve the above problems, a speech speed conversion magnification determination device according to the present invention is a speech speed conversion magnification determination device that determines an adaptive conversion magnification of a speech speed of an input signal, and is a voiced interval of an input signal. A stable section in which the fundamental frequency of the input signal is calculated at regular time intervals in the voiced section, and the value of the fundamental frequency changes within a predetermined variation range And a fundamental frequency calculation unit that determines an unstable interval in which the value of the fundamental frequency changes beyond a predetermined change width, and a frequency smoothing unit that smoothes a temporal change in the fundamental frequency in the stable interval; In the unstable section and the silent section, a pseudo fundamental frequency calculator that calculates a pseudo fundamental frequency by interpolating a fundamental frequency with reference to a value of a fundamental frequency in the smoothed stable section, and the smoothed A basic frequency outline connecting unit that connects the fundamental frequency and the pseudo fundamental frequency and obtains a sample value of a series of basic frequencies, and using the sample value of the outline of the basic frequency as a physical index A physical index calculating unit for outputting; and a speech speed conversion magnification specifying unit for calculating a speech speed conversion magnification to be specified for an input signal based on the physical index.

  Furthermore, in the speech speed conversion magnification determination apparatus according to the present invention, the physical index calculation unit includes a power calculation unit that calculates the power of the input signal at a constant time interval, and smoothes the temporal change of the power so as to approximate the power. A power smoothing unit that obtains a sample value of the shape, and outputs the approximate sample value of the fundamental frequency and the approximate sample value of the power as the physical index.

  Further, in the speech rate conversion magnification determination apparatus according to the present invention, the physical index calculation unit includes a voicing degree calculation unit that obtains voicing degree from an input signal waveform, and an outline of voicing degree by smoothing a temporal change of the voicing degree A voiciness smoothing unit for obtaining a sample value of the basic frequency, and outputting the sample value of the approximate shape of the fundamental frequency, the sample value of the approximate shape of the power, and the sample value of the approximate shape of the voicing degree as the physical index It is characterized by doing.

  Further, in the speech speed conversion magnification determination apparatus according to the present invention, the physical index calculation unit calculates a rough degree of irregularities of the basic frequency representing a change tendency of the rough shapes of the fundamental frequency. And a sample value of the approximate shape of the fundamental frequency, a sample value of the approximate shape of the power, and a degree of unevenness of the approximate shape of the fundamental frequency are output as the physical index.

  Furthermore, in the speech rate conversion magnification determination apparatus according to the present invention, the physical index calculation unit includes a power unevenness degree calculation unit that calculates a degree of unevenness of the power outline, which represents a change tendency of the power outline, The rough sampling value of the fundamental frequency, the rough sampling value of the power, and the roughness of the rough shape of the power are output as the physical index.

  Furthermore, in the speech rate conversion magnification determination apparatus according to the present invention, the physical index calculation unit calculates a power spectrum of the input signal, and the normalized power in the first frequency band and the first power higher than the first frequency band. A frequency band division / power calculation unit that calculates normalized power in two frequency bands; and a divided band power ratio calculation unit that calculates a ratio of normalized power in the first frequency band and the second frequency band. Output the approximate sample value of the fundamental frequency, the approximate sample value of the power, and the ratio of normalized power in the first frequency band and the second frequency band as the physical index, To do.

  Furthermore, in the speech speed conversion magnification determination apparatus according to the present invention, the speech speed conversion magnification designation unit calculates the speech speed conversion magnification based on the physical index and a contribution rate of each physical index to the speech speed. It is characterized by that.

  Furthermore, in the speech rate conversion magnification determination apparatus according to the present invention, when the time length to be reproduced for each part of the entire input signal or the divided input signal is given, the time length of the whole input signal or each part is determined. It further comprises a speech speed conversion magnification fine adjustment unit that finely adjusts the speech speed conversion magnification to match the time length to be reproduced to determine the final speech speed conversion magnification.

  In order to solve the above problem, a speech speed conversion device according to the present invention is a speech speed conversion magnification determination device that performs adaptive speech speed conversion of an input signal, and includes the above-described speech speed conversion magnification determination device and A speech speed conversion unit that converts the input signal according to the speech speed conversion magnification, and the speech speed conversion unit is a time length to be reproduced for each part of the input signal as a whole or in the divided input signal. Is given at a certain time interval, the target signal to be output when it is expanded or contracted at a uniform magnification, and the converted signal obtained by converting the input signal by the speech speed conversion magnification on the time series of the signal. And the speech speed conversion magnification fine adjustment unit re-adjusts the subsequent speech speed conversion magnification according to the temporal displacement.

  In order to solve the above problems, the present invention provides a computer configured as a speech speed conversion magnification determination device that determines an adaptive conversion magnification of a speech speed of an input signal, and includes a voiced section and a silent section of the input signal. And calculating a fundamental frequency of the input signal at a constant time interval in the sounded section, a stable section in which the value of the fundamental frequency changes within a predetermined change width, and the value of the fundamental frequency is Determining an unstable section that changes beyond a predetermined change width; smoothing a temporal change in the fundamental frequency in the stable section; and smoothing in the unstable section and the silent section. A step of calculating a pseudo fundamental frequency by interpolating the frequency with reference to a value of the fundamental frequency in the stable interval, and the smoothed fundamental frequency and the pseudo fundamental frequency Subsequently, obtaining a sample value of the approximate shape of the fundamental frequency, and calculating a speech rate conversion magnification to be specified for the input signal according to the sample value of the approximate shape of the fundamental frequency, It is also characterized as a program for execution and a recording medium on which the program is recorded.

  According to the adaptive speech speed conversion performed based on physical features such as the fundamental frequency and power of the input signal proposed by the present invention, conventionally, when background sound and sound are mixed, “speech interval” and “ If the “silence interval” is not correctly judged, the disadvantage that adaptive speech speed conversion does not work as expected can be avoided, and even when background sound and audio are mixed, a stable and slow feeling can be avoided. This makes it possible to perform adaptive speech speed conversion with a high sounding effect and natural sound.

It is a block diagram which shows the structure of the speech-speed conversion magnification determination apparatus of Example 1 by this invention. It is a figure which shows the example of calculation of the rough form of a fundamental frequency, and determination of a temporary expansion / contraction rate. It is a flowchart which shows operation | movement of the speech-speed conversion magnification determination apparatus of Example 1 by this invention. It is a block diagram which shows the structure of the speech-speed converter of Example 1 by this invention. It is a block diagram which shows the structure of the speech-speed conversion magnification determination apparatus of Example 2 by this invention. It is a figure which shows the example of calculation of the outline of power, and determination of the temporary expansion-contraction rate. It is a flowchart which shows operation | movement of the speech-speed conversion magnification determination apparatus of Example 2 by this invention. It is a block diagram which shows the structure of Example 3 by this invention. It is a figure explaining calculation of an autocorrelation function. It is a flowchart which shows operation | movement of the speech-speed conversion magnification determination apparatus of Example 3 by this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the speech speed conversion magnification determining apparatus according to the first embodiment of the present invention. The speech speed conversion magnification determination apparatus 1a of the present embodiment includes a physical index calculation unit 2 and a speech speed conversion magnification determination unit 3, thereby performing adaptive speech speed conversion of an input signal. The physical index calculation unit 2 calculates a physical index of the input signal. The speech speed conversion magnification determination unit 3 determines the speech speed conversion magnification α _n to be specified for each segment (section) of the input signal based on the physical index input from the physical index calculation unit 2. Here, the subscript n is an integer value indicating the position of the input signal when the input signal is divided every unit time (fixed time interval, for example, 5 ms) from the beginning. Hereinafter, as an example of the segment per unit time, this separation interval will be described as 5 ms.

  The physical index calculation unit 2 includes a basic frequency outline calculation unit 100. The basic frequency outline calculation unit 100 includes a sound / silence determination unit 102, a basic frequency calculation unit 104, a smoothing unit 106, and a pseudo basic. The frequency calculation unit 108 and the basic frequency outline connecting unit 110 are included. The speech speed conversion magnification determination unit 3 includes a first speech speed conversion magnification designation unit (speech speed conversion magnification designation unit a) 120 and a speech speed conversion magnification fine adjustment unit 140.

Speech speed conversion magnification determining device 1a of this embodiment, the comprehensive, with F _n representing the envelope of the variation of the fundamental frequency and pseudo fundamental frequency of each unit in the input signal time (5 ms) as "physical index" Thus, the speech speed conversion magnification α _n to be specified for each segment of the input signal is determined.

Hereinafter, the determination of the speech speed conversion ratio for each of the intervals for the input signal based on a physical index F _n will be described in order. Here, the speech rate conversion magnification refers to the conversion rate of the reproduction speed of the input signal, and corresponds to the reciprocal of the temporal expansion / contraction rate for the signal interval per unit time.

(Calculation of physical indicators _{F n)}
First, it will be described with reference to FIGS. 1 and 2 for the calculation of physical indicators F _n. FIG. 2 is a diagram illustrating an example of calculation of a rough shape of a fundamental frequency and determination of a temporary expansion / contraction rate.

  The voice / silence determination unit 102 calculates the amplitude and power from the input signal and, based on the magnitude, “voice” and / or “background sound” (music and noise), or a portion where both exist simultaneously. A certain “sound section” is distinguished from a “silent section” without sound. For example, when the amplitude or power of the input signal exceeds a predetermined threshold, it is determined as a voiced section, and when it is less than the predetermined threshold, it is determined as a silent section.

  Here, a simple specific example using the power threshold will be described. The input signal x (k) is cut out by aligning the center of the nth segment and the center of the Hamming window h (k) corresponding to the window width of 20 ms, the number of sampling points is K, and the quantization accuracy of the input signal is 16 bits. If so, the power of the segment is defined by the following equation (1).

  Then, the sound / silence determination unit 102 outputs the signal of the sound period to the fundamental frequency calculation unit 104 and outputs the signal of the sound period to the pseudo fundamental frequency calculation unit 108. FIG. 2A is a diagram illustrating an example of an input signal waveform that is determined as a sound section by the sound / silence determination unit 102.

  The fundamental frequency calculation unit 104 calculates a fundamental frequency for each unit time (a constant time interval, for example, 5 ms) for the input signal that is input from the sound / silence determination unit 102 and is determined to be a sound section. , The section where the calculated fundamental frequency value is stable and almost continuously changing within a predetermined change width is determined as the “stable section”, and the calculated fundamental frequency value is not stable and changes discontinuously. A severe section is determined as an “unstable section”. Then, the fundamental frequency calculation unit 104 identifies the fundamental frequency in each stable interval, outputs the identified fundamental frequency in each stable interval to the smoothing unit 106, and outputs the signal in the unstable interval to the pseudo fundamental frequency calculation unit 108. Output to. The fundamental frequency calculation unit 104 rejects all fundamental frequency values in the “unstable section”. Note that the fundamental frequency for each unit time can be calculated using any known technique (for example, see Japanese Patent No. 3219868). FIG. 2B shows the fundamental frequency for each unit time of the input signal shown in FIG. FIG. 2B shows a “stable section” surrounded by a square frame, and the other sections are “unstable sections”.

  The smoothing unit 106 smoothes the trajectory composed of the fundamental frequency of each stable section so that the fundamental frequency of each stable section input from the fundamental frequency calculation unit 104 becomes a smoother trajectory. For this smoothing, it is preferable to use a low-pass filter having a cutoff frequency of about 3 to 6 Hz. Further, the smoothing unit 106 outputs the value of the fundamental frequency of the smoothed locus in the stable section to the pseudo fundamental frequency calculation unit 108 and the fundamental frequency outline connecting unit 110. FIG. 2B shows the locus of the smoothed fundamental frequency with a thick line.

  The pseudo fundamental frequency calculation unit 108 uses each value of the fundamental frequency of the smoothed trajectory of the stable section supplied from the smoothing unit 106, and interpolates with an interpolation function (for example, a spline function), thereby generating a silent section and a non-interval. A pseudo fundamental frequency (pseudo fundamental frequency) in the stable section is calculated, and the calculated pseudo fundamental frequency is output to the fundamental frequency outline connecting unit 110. FIG. 2B shows the fundamental frequency of the pseudo fundamental frequency with a thin line.

The fundamental frequency outline linking unit 110 includes the value of the fundamental frequency of the smoothed trajectory of the stable section supplied from the smoothing unit 106, and the pseudo fundamental of the silent section and the unstable section supplied from the pseudo fundamental frequency calculation unit 108. A continuous trajectory composed of the fundamental frequency and the pseudo fundamental frequency (hereinafter referred to as “general shape of the fundamental frequency”) over the entire interval (unit time) of the input signal to be processed by connecting the frequency value. A value obtained by sampling the outline of the fundamental frequency per unit time (hereinafter referred to as “sample value of the outline of the fundamental frequency”) _Fn is the first speech speed conversion magnification of the speech speed conversion magnification determining unit 3 The data is output to the designation unit (speech speed conversion magnification designation unit a) 120.

(Determination of speech rate conversion magnification)
Next, determination of the speech speed conversion magnification will be described with reference to FIGS. The first speech speed conversion magnification designation unit (speech speed conversion magnification designation unit a) 120 has a speech speed conversion magnification for each unit time (hereinafter simply referred to as “speech speed conversion magnification designation unit a”) in a portion where the sample value F _n of the approximate shape of the fundamental frequency is large. referred to as "speech speed conversion ratio") of .alpha.a _n relatively small and (slowly speech speed), the sample value F _n value is less part of the envelope of the fundamental frequency, relative to the speech speed conversion ratio .alpha.a _n Basically, it should be large (fast speaking speed). That is, relatively small speech speed conversion ratio .alpha.a _n is portion with high voice (fundamental frequency), the lower part of the voice relatively large speech speed conversion ratio .alpha.a _n. This is because the high voice part is emphasized in meaning and may be an important content in the sentence. It seems to work in favor of understanding.

Further, as described above, since there is a high possibility that the silent section and the unstable section are not voices, it is considered that there is little adverse effect on the understanding of the content even if the speech speed is relatively increased. In the pseudo fundamental frequency calculation unit 108, the pseudo fundamental frequency of the stable section before and after the pseudo fundamental frequency is calculated by spline interpolation or the like, but as a physical characteristic of general human speech, 2B, the change in the fundamental frequency rises to the right at the beginning of the talk from time 150ms, and the change in the fundamental frequency falls to the right immediately before the pause, that is, in the vicinity of time 1500ms in FIG. Therefore, although not shown in FIG. 2, the pseudo fundamental frequency of a certain pause section (including a section of only background sound) is often interpolated in a downwardly convex valley shape. That is, the value of envelope sample value F _n of the fundamental frequency of that portion becomes relatively small, resulting in the speech speed conversion ratio .alpha.a _n becomes high speech rate operates to quickened.

Next, a specific method of determining the speech speed ratio using the sample value F _n of envelope of the fundamental frequency, illustrating a few examples. When the number of sample values F _n of the approximate shape of the fundamental frequency is finite, the first speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit a) 120 normalizes the whole using the median value. . For example, when the median value is regarded as 1.0 and the difference between the maximum value and the median value is larger between the maximum value and the minimum value, the maximum value is regarded as 2.0 and all the fundamental frequencies are A value between 0 and 2 is newly assigned to the approximate sample value F _{n by} proportional distribution, and the value is determined as a temporary expansion / contraction rate F ′ _n per unit time (5 ms). When the difference between the minimum value and the median value is larger among the maximum value and the minimum value, the same operation is performed with the minimum value regarded as 0.0. Or by performing the same operations after obtaining the logarithm logF _n sample values F _n of envelope of the fundamental frequency. Further, instead of the median, the mean value and the sample value F _n of envelope of all of the fundamental frequency may be used an average value of the maximum and minimum values. FIG. 2C shows a temporary expansion / contraction rate F ′ _n corresponding to the approximate sample value F _n of the fundamental frequency shown in FIG. In this example, since the frequency (vertical axis) is a logarithmic scale, F ′ _n is calculated based on the outline of the fundamental frequency by logF _n .

Speech speed conversion-magnification determining device 1a operates in real time, when the sequentially input signals it is necessary to convert the speech speed is not determined that the number of sampled values F _n of envelope of the fundamental frequency. Therefore, for example, it holds the value of the sample value F _n of envelope of the fundamental frequency within the past 3 seconds, the maximum, minimum, or by using a central value, etc., envelope of the fundamental frequency of the current time It is also possible to normalize the value of the sample value F _n and to set it as the provisional expansion / contraction rate F ′ _n . However, in this case, in the physical index calculation unit 2, the smoothing unit 106 performs smoothing calculation using only the calculation results of the past and current fundamental frequencies. The pseudo fundamental frequency calculation unit 108 also calculates an interpolation value using a spline function or the like using the output of the past smoothing unit 106. However, as described above, since the change in the fundamental frequency decreases to the right at the end of the talk, if the pseudo fundamental frequency after that is interpolated using only the output of the past smoothing unit 106, the value decreases more and more. since going to deal with such provision lower limit to the pseudo fundamental frequency (e.g., 1/2 of the average value of sampled values F _n of envelope of the fundamental frequency within the past 3 seconds).

Next, describing calculation of speech speed conversion ratio .alpha.a _n corresponding to the value of the scaling factor F _'n provisional. As described above, the value of the scaling factor F _'n tentative are normalized between 0 to 2, the first speech speed conversion magnification specifying unit (speech speed conversion ratio specifying section a) 120, for example following formula (2) and (3), calculates the speech speed conversion ratio .alpha.a _n.

Finally, the operation in the speech speed conversion magnification fine adjustment unit 140 will be described. The speech speed α _n given to the n-th counting unit time (5 ms) from the beginning of the input signal is obtained by equations (2) and (3).

When a speech speed conversion magnification α (double speed α) (hereinafter referred to as “reproduction speed conversion magnification”) is given to the entire input signal, this is finely adjusted by the following procedure. As the reproduction speed conversion magnification α, an arbitrary value such as 0.5 to 5.0 can be set. When the reproduction speed conversion magnification α is given, if the length of the entire input signal is L (unit is second), the length of the entire converted signal must be L / α. Therefore, first, the speech speed conversion magnification fine adjustment unit 140 performs speech speed conversion on all input signal sections, and first calculates the length L ₀ of the converted speech after connection as a whole.

Then, the following equation (4), the speech speed conversion ratio αa _{by n} a fine adjustment to determine the final speech speed conversion ratio alpha _n, the time length to be reproduced the entire length of the converted signal Can be adapted to
_{α n = αa n × L 0} / (L / α) (4)

When adjusting to the same timing as the voice converted uniformly at the playback speed conversion magnification α as frequently as possible, not the length L of the entire input signal but the length of the voice divided into shorter units. Thus, the speech speed conversion magnification α _n is corrected so as to perform fine adjustment. For _example, if it can be divided into M as _{L = L 1 + L 2 +} ··· + L M _is, L _1, L 2, · · ·, divides the input waveform on each section of the _{L M,} respectively in divided section, the m th interval, first the speech speed conversion ratio .alpha.a _n of the section of this m-th section by converting speech rate using the partial length Lm ₀ of connected converter voice calculated above In the equation (4), Lm is set in place of L and Lm ₀ is set in place of L ₀ to obtain each speech speed conversion magnification α _n and fine adjustment is performed by converting the speech speed again.

Various methods of speech speed conversion (waveform expansion / contraction) for realizing the speech speed conversion magnification α _n have already been proposed. However, as a method for maintaining voice pitch, PICOLA (Pointer Interval Controlled OverLap) and Add) method, TDHS (Time Domain Harmonic Scaling) method, PSOLA (Pitch Synchronous OverLap Add) method, and the like. Other than this, it is disclosed in Japanese Patent No. 2612868, Japanese Patent No. 3083830, Japanese Patent No. 2955247, etc. There is a waveform expansion / contraction method, and any waveform expansion / contraction method may be used.

  FIG. 3 is a flowchart illustrating the operation of the speech speed conversion magnification determination apparatus 1a according to the first embodiment. The speech speed conversion magnification determination apparatus 1a inputs a signal for speech speed conversion (step S101). When a signal for speech speed conversion is input, the speech speed conversion magnification determination apparatus 1a determines a sound section and a silence section of the input signal by the sound / silence determination unit 102 (step S102). When it is determined in step S102 that the voice section is present, the speech rate conversion magnification determination apparatus 1a calculates the fundamental frequency for each unit time by the fundamental frequency calculation unit 104 (step S103), and determines the degree of change in the fundamental frequency. Based on this, it is determined whether it is a stable section or an unstable section (step S104). If it is determined in step S104 that it is a stable section, the speech rate conversion magnification determination apparatus 1a smoothes the trajectory composed of the fundamental frequency of each stable section by the smoothing unit 106 (step S105).

  On the other hand, if it is determined in step S102 that it is a silent section, or if it is determined in step S104 that it is an unstable section, the speech speed conversion magnification determination apparatus 1a uses the pseudo fundamental frequency calculation unit 108 to determine the stable section. By using each value of the smoothed fundamental frequency of the trajectory and interpolating with an interpolation function, a pseudo fundamental frequency in a silent section and an unstable section is calculated (step S106). In general, since the fundamental frequency cannot be stably calculated only for the background sound such as noise and music, the pseudo fundamental frequency is calculated. In addition, if there is a part where no noise or background sound exists in the input signal, the fundamental frequency is not calculated in the “silent period” detected in that part, and the value of the period in which the fundamental frequency is stably obtained is calculated. Interpolate with reference to determine the pseudo fundamental frequency.

Then, the speech speed conversion magnification determination device 1a uses the fundamental frequency outline linking unit 110 to calculate the fundamental frequency value of the stable section trajectory smoothed in step S105, the silent section and the non-interval calculated in step S106. The value of the pseudo fundamental frequency in the stable interval is connected to derive a sample value F _n of the basic shape of the fundamental frequency (step S107). Then, the speech speed conversion ratio determination apparatus 1a, the first speech speed conversion magnification specifying unit (speech speed conversion ratio specifying section a) 120, speech rate based on the value of the sampled values F _n of envelope of the fundamental frequency calculating the conversion ratio .alpha.a _n (step S108). The envelope sample value F _n value is large portion of the fundamental frequency loosen the speech speed according to the degree that, in the partial value is smaller accelerate speech speed depending on the degree thereof. As a result, even when noise and background sound are mixed in the input signal, adaptive speech speed conversion can be stably performed while adjusting to the target time length as a whole. Finally, when the playback speed conversion magnification α is given by the speech speed conversion magnification fine adjustment unit 140, the speech speed conversion magnification determination device 1a determines the final speech speed conversion magnification α _n (step S109).

Thus, according to the speech speed conversion magnification determination device 1a of the present embodiment, adaptive speech speed conversion is possible even when background sound and sound are mixed. Further, by providing the speech speed conversion magnification fine adjustment unit 140, an arbitrary reproduction speed conversion magnification α such as 1 × speed (reproduction with the original time length) or 2 × speed (reproduction with a time length that is half the real time) is given. When changing the speed at a magnification larger or smaller than the playback speed conversion magnification α depending on each part, the whole book is played back at the same time as the speech speed conversion at the uniform playback speed conversion magnification α. As a result, it is possible to finely adjust the speech speed so that the speech speed is converted, and as a result, the speech speed converted speech can be generated with the same time length as when the speech speed is converted with the uniform playback speed conversion magnification α. . When a predetermined time length is set for each of the parts divided into N parts according to a predetermined rule, the divided parts are given to W ₁ , W ₂ , W ₃ ,..., W _N. uniform reproduction speed conversion ratio alpha ₁ for adapting to the length of time _{is, α 2, α 3, ···} , to adjust the balance out to play at the same time as that speech speed conversion alpha _N Fine-tune the speaking speed sequentially.

(Speaking speed converter)
Next, the speech speed conversion device will be described with reference to FIG. FIG. 4 is a block diagram showing the configuration of the speech speed conversion apparatus according to the first embodiment of the present invention. The speech speed conversion device 10 a includes the above-described speech speed conversion magnification determination device 1 a and the speech speed conversion unit 4. The speech speed conversion unit 4 converts the input signal according to the speech speed conversion magnification determined by the speech speed conversion magnification determination device 1a.

  The speech speed conversion unit 4 operates in real time, and when it is necessary to sequentially convert and output the input signal, the speech speed conversion unit 4 applies to the entire input signal or each part of the divided input signal. Given the length of time to be played back, the target signal to be output when it is expanded or contracted at a uniform magnification at a certain time interval, and the converted signal obtained by converting the input signal by the speech speed conversion magnification, Are compared with each other on the time series, and the information on the temporal shift is returned to the speech speed conversion magnification determination device 1a. In the speech speed conversion magnification fine adjustment unit 140 of the speech speed conversion magnification determination device 1a, according to the shift amount, Readjust the subsequent speech speed conversion magnification.

That is, in the speech speed conversion unit 4, a signal to be output when any part of the past input signal is expanded / contracted at a uniform magnification by the reproduction speed conversion magnification α, and the speech speed conversion magnification determination device. The signal output after the speech speed conversion is performed at an adaptive speech speed conversion magnification according to α _n output from 1a is compared at a time interval of Lm on the signal time series. Then, when the output signal of the adaptive speech speed conversion at that time is the audio content temporally before the output signal of the virtual uniform expansion / contraction speech speed conversion (reproduction speed conversion magnification α is 1). Is smaller), the information on the amount of temporal deviation is returned to the speech speed conversion magnification fine adjustment unit 140 of the speech speed conversion magnification determination device 1a, and the speech speed conversion magnification fine adjustment unit 140 responds to the amount of deviation. Thus, a fine adjustment for slightly shifting the speech speed conversion magnification α _n given to each subsequent speech section to the high speed side is added.

In addition, a signal that is converted by the speech speed conversion unit 4 at an adaptive speech speed conversion magnification according to α _n that is actually output from the speech speed conversion magnification determination device 1a is output as a virtual one. When the audio content is temporally later than the output signal of the speech rate conversion of the expansion and contraction (the playback speed conversion magnification α may be smaller or larger than 1), information on the temporal deviation amount is obtained. The speech speed conversion magnification fine adjustment unit 140 of the speech speed conversion magnification determination apparatus 1a returns to the speech speed conversion magnification fine adjustment unit 140. The speech speed conversion magnification fine adjustment unit 140 reduces the speech speed conversion magnification α _n to be given to each subsequent speech interval according to the deviation amount. A fine adjustment that shifts slightly to the speed side is added.

  In this way, the amount of temporal deviation between the signal that is output after being converted at the adaptive speed conversion magnification and the voice that is virtually converted at the playback speed conversion magnification α is obtained as follows: Try to keep it as small as possible. As a result, the sequential signal input / output relationship in the real-time operation of the speech speed conversion magnification determination device 1a and the speech speed conversion unit 4 can be maintained. Therefore, when it is necessary to immediately output a signal obtained by converting the speech speed with respect to a signal sequentially input to the speech speed converter 10a, the speech speed converter can be configured as a real-time system.

  Here, in order to function as the speech speed conversion magnification determination device 1a or the speech speed conversion device 10a, a computer can be preferably used, and such a computer realizes each function of the speech speed conversion magnification determination device 1a. It can be realized by storing a program describing the processing contents in a storage unit of the computer, and reading and executing the program by a central processing unit (CPU) of the computer.

  Thereby, the speech speed conversion magnification determination device 1a or the speech speed conversion device 10a can be operated as a program for a personal computer or an application that operates on a mobile device such as a portable music player or a smartphone.

  In addition, the program describing the processing contents can be recorded on a computer-readable recording medium such as a DVD or CD-ROM, and the recording medium can be distributed by selling, transferring, renting, etc. For example, the program can be distributed by storing the program in a storage unit of a server on a network such as an IP and transferring the program from the server to another computer via the network.

  In addition, a computer that executes such a program can temporarily store, for example, a program recorded in a recording medium or a program transferred from a server in its storage unit. As another embodiment of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and each time the program is transferred from the server to the computer. In addition, the processing according to the received program may be executed sequentially.

  Next, the speech speed conversion magnification determining apparatus according to the second embodiment of the present invention will be described. In addition, the same reference number is attached | subjected to the same component as Example 1, and description is abbreviate | omitted.

FIG. 5 is a block diagram showing the configuration of the speech speed conversion magnification determining apparatus according to the second embodiment of the present invention. Similar to the speech speed conversion magnification determining apparatus 1a of the first embodiment, the speech speed conversion magnification determining apparatus 1b of the present embodiment calculates the physical index of the input signal for each segment obtained by dividing the input signal for each unit time. The index calculation unit 2 and the speech speed conversion magnification determination unit 3 that determines the speech speed conversion magnification α _n to be specified for each segment of the input signal based on the physical index input from the physical index calculation unit 2.

  Compared with the speech speed conversion magnification determination device 1a (see FIG. 1) of the first embodiment, the physical index calculation unit 2 further includes a power outline calculation unit 200 in the speech speed conversion magnification determination device 1b of the second embodiment. The speech speed conversion magnification determination unit 3 further includes a second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b) 220. The power outline calculating unit 200 includes a power calculating unit 202 and a smoothing unit 204.

Speech speed conversion magnification determining device 1b of this embodiment, the comprehensive, and F _n representing the envelope of the fundamental frequency per unit time in the input signal, approximate the change in the value of power per unit time in the input signal Using two “physical indices” of P _n representing the shape, the speech speed conversion magnification α _n to be specified for each segment of the input signal is determined and the speech speed conversion is performed to generate an output signal subjected to the speech speed conversion. Output.

Since the speech speed conversion magnification determination apparatus 1b according to the second embodiment uses two physical indexes, the first speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit a) 120 according to the second embodiment provides an overview of the basic frequency. considering contribution to the speech speed in the form of sampled values F _n, for example, by the following equation (5) to (7), calculates a speech speed conversion ratio .alpha.a _n.

Here, Ra is a contribution rate to the speech speed specified by the sample value F _n of the basic shape of the fundamental frequency, and 0 ≦ Ra ≦ 1. Also, K is, together with the normalized value of the temporary scaling factor F _'n, is a constant for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

(Calculation of physical index _Pn )
Next, the calculation of the physical index _Pn will be described with reference to FIGS. FIG. 5 is a diagram illustrating an example of calculation of a power outline and determination of a provisional expansion / contraction rate.

  The power calculation unit 202 calculates the power of the input signal every unit time (5 ms) and outputs it to the smoothing unit 204. As a general method, the power can be obtained by the sum of squares of each sample value after weighting the input signal waveform by a window function such as a Hamming window having a time width of about 20 ms. As an example of a specific calculation method, the method described using Equation (1) can be given. FIG. 6A is a diagram illustrating an example of an input signal waveform. FIG. 6B shows the power for each unit time of the input signal shown in FIG.

The smoothing unit 204 smoothes the power trajectory calculated for each unit time so that the power input from the power calculation unit 202 becomes a smoother trajectory. The sampled value P _n (hereinafter referred to as “sample value of power outline”) is calculated and output to the second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b) 220. For this smoothing, it is preferable to use a low-pass filter having a cutoff frequency of about 3 to 6 Hz.

(Determination of speech rate conversion magnification)
Next, determination of the speech speed conversion magnification will be described with reference to FIGS. The second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b) 220 relatively reduces the speech speed conversion magnification (slowly speaking) in the portion where the sample value P _n of the power outline is large. In the portion where the sample value _Pn of the outline of the power is small, the speech speed conversion magnification is basically increased (to make the speech speed faster). That is, the relative speech speed conversion magnification decreases in a portion where the voice (power) is high, and the relative speech speed conversion magnification increases in a portion where the voice is low. This is because the meaning of the loud part of the voice is emphasized and it may be important content in the sentence. It can be expected to work in favor of understanding. In addition, it is considered that the silent section has little adverse effect on the understanding of the content even if the speech speed is relatively increased.

Next, some examples of a specific method for determining the speech rate magnification using the sample value P _n of the power outline will be described. When the number of sample values _Pn of the outline of power is finite, the whole is normalized using the median value. For example, if the median value is assumed to be 1.0 and the difference between the maximum value and the maximum value is greater than the median value, the maximum value is assumed to be 2.0, and all powers are approximated. For the sample value P _n of the shape, a value between 0 and 2 is newly assigned by proportional distribution, and the value is determined as a temporary expansion / contraction rate P ′ _n per unit time (5 ms). When the difference between the minimum value and the median value is larger among the maximum value and the minimum value, the same operation is performed with the minimum value regarded as 0.0. Or by performing the same operations after obtaining the logarithm logP _n relative sampled value P _n envelope of all power. Furthermore, instead of the median value, an average value of the sample values _Pn of the outlines of all powers or an average value of the maximum value and the minimum value may be used. FIG. 6C shows a provisional expansion / contraction rate P ′ _n corresponding to the approximate sample value P _n of the power shown in FIG. 6B. In this example, since the power (vertical axis) is decibeled, P ′ _n is calculated based on the outline of power by log P _n .

When the speech speed conversion magnification determination device 1b operates in real time and needs to sequentially convert and output the input signal, the number of sample values P _n of the power outline is not determined. . Therefore, for example, a sample value P _n of the approximate shape of the power within the past 3 seconds is held, and a sample of the approximate shape of the power at the current time is used by using the maximum value, minimum value, median value, etc. The value P _n may be normalized, and this may be used as the temporary expansion / contraction rate P ′ _n . However, in this case, in the physical index calculation unit 2, the smoothing unit 204 performs smoothing calculation using only the past and current power calculation results.

Next, calculation of the speech rate conversion magnification αb _n according to the value of the temporary expansion / contraction rate P ′ _n will be described. As described above, since the value of the provisional expansion / contraction rate P ′ _n is normalized between 0 and 2, the second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b) 220 is, for example, The speech speed conversion magnification αb _n is calculated from the equations (8) to (10).

Here, Rb is a contribution rate to the speech speed specified by the sample value P _n of the power outline, and 0 ≦ Rb ≦ 1. Also, K is, together with the normalized value of the temporary scaling factor P _'n, is a constant for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

  For example, when the input signal is broadcast, each value of the contribution rate Ra of the equations (5) to (7) and the contribution rate Rb of the equations (8) to (10) is the program genre (news, documentary, drama, variety, If you know rakugo, comic story, etc.), adaptive speech speed conversion with higher ease of hearing and higher naturalness is possible by optimizing the value distribution ratio accordingly. For example, Ra = 0.7, Rb = 0.3 for news, Ra = 0.5, Rb = 0.5 for documentary and drama, Ra = 0.3, Rb = 0.7 or the like. Also, by adjusting the values of the contribution ratios Ra and Rb depending on the language used for the speech speed conversion, it is possible to obtain converted speech with a more natural way of listening in each language.

Finally, an example of the operation in the speech speed conversion magnification fine adjustment unit 140 will be described. The speech speed conversion magnification α _n given to the nth counting from the head of the input signal every unit time (5 ms) basically uses the equations (5), (6), (8), (9). and α _n = αa _n × αb _n in the formula (7), is set to α _n = αa _n + αb _n the case of using (10). However, when the reproduction speed conversion magnification α is given, this is finely adjusted according to the following procedure. As the reproduction speed conversion magnification α, an arbitrary value such as 0.5 to 5.0 can be set.

When the reproduction speed conversion magnification α is given, if the length of the entire input signal is L (unit is second), the length of the entire converted signal must be L / α. First, the speech speed conversion ratio fine adjustment section 140, based on speech speed conversion ratio .alpha.a _n and .alpha.b _n, calculates the speech speed conversion ratio αab _n. Equation (5), (6), in the case of using the (8), (9), and _{αab n = αa n × αb n} , equation _{(7), αab n = αa} n in the case of using the (10) As + αb _n , speech speed conversion is performed for all input signal sections, and the length L ₀ of the converted speech after connection is calculated first as a whole.

Then, by the following equation (11), the speech speed conversion ratio αab _{by n} a fine adjustment to determine the final speech speed conversion ratio alpha _n, the time length to be reproduced the entire length of the converted signal Can be adapted to
α _n = αab _n × L ₀ / (L / α) (11)

If you want to synchronize with the same timing as the voice converted uniformly with the playback speed conversion magnification α as often as possible, as in the first embodiment, divide this into shorter units instead of the length L of the entire input signal. It is only necessary to correct α _n so as to finely adjust the length of the voice. For _example, if it can be divided into M as _{L = L 1 + L 2 +} ··· + L M _is, L _1, L 2, · · ·, divides the input signal waveform in each section of the _{L M,} respectively of the divided section, the m th interval, first the interval unit of time (5 ms) of each portion of each speech speed conversion ratio αab _{n (αa n} × _{αb n} or αa _n + αb _n) the m-th using Convert the speech speed of the section, calculate the partial length Lm ₀ of the connected converted speech first, set Lm instead of L in Equation (11), and set Lm ₀ instead of L ₀ to speak speed conversion Fine adjustment is performed by obtaining the magnification α _n and converting the speech speed again. Note that the speech speed conversion (waveform expansion / contraction) method for realizing the speech speed conversion magnification α _n can be the same as in the first embodiment.

FIG. 7 is a flowchart illustrating the operation of the speech speed conversion magnification determination apparatus 1b according to the second embodiment. Steps S201 to S208 are the same as steps S101 to S108 of the operation of the speech speed conversion magnification determination apparatus 1a according to the first embodiment illustrated in FIG. When a signal for speech speed conversion is input, the speech speed conversion magnification determination apparatus 1b calculates the power of the input signal by the power calculation unit 202 (step S209). Then, the speech speed conversion magnification determining apparatus 1b smoothes the calculated power trajectory by the smoothing unit 204, and derives a sample value P _n of the approximate power shape (step S210). Next, the speech speed conversion magnification determination apparatus 1b uses the second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b) 220 to convert the speech speed based on the value of the sample value P _n of the power outline. The magnification αb _n is calculated (step S211). Finally, the speech speed conversion ratio determination device 1b, the speech speed conversion ratio fine adjustment section 140, obtains the speech speed conversion ratio alpha _n from the speech speed conversion ratio .alpha.a _n and .alpha.b _n. When the reproduction speed conversion magnification α is given, α _n is finely adjusted as the final speech speed conversion magnification (step S212).

Thus, according to the speech speed conversion magnification determination device 1b of the present embodiment, by determining the speech speed conversion magnification α _n based on the fundamental frequency and power, for example, only in the background sound (such as BGM), Even when the power is high, it is possible to determine that the speech speed is increased in a portion where the pseudo fundamental frequency shows a small value.

  Furthermore, the following advantages can be obtained by adding the power value to the speech speed control. Usually, there is a positive correlation between the pitch and loudness of the voice, and the power is high at the part where the fundamental frequency is high. Such a part is often a vowel part, and the fundamental frequency is stably calculated in the vowel part. Therefore, if the speech speed is relaxed at a large fundamental frequency and power value, the probability of slowing the speech speed around the vowel part is high. It is known that the vowel part expands and contracts mainly when the actual speech rate is slow and fast (for example, 148th Meeting of Acoustical Society of America, 4pSC3. (See Journal of the Acoustical Society of America, Vol.116, No.4, Pt.2 of 2. p2628.) Therefore, this method enables adaptive speech speed conversion with natural sound. Become.

  Another advantage is as follows. Japanese and Chinese are “pitch accents”, and tend to control differences in homonyms and emphasis on meaning by changing the pitch of the voice. Western languages, on the other hand, are “stress / accent”, and it is said that the rhythm and meaning of each word are controlled by changing the volume of the voice. Therefore, in adaptive speech speed control, it is possible to optimize to various languages by taking into account both voice pitch and loudness values.

  Also, if the target voice to be converted to speech speed is broadcast voice, if the genre of the program (news, documentary, drama, variety, rakugo / comic) is given as meta information that has been developed in recent years, Further, by optimizing the distribution rate of the multiplication value and the exponent value (contribution rate) to the speech rate conversion magnification, adaptive speech rate conversion with higher ease of listening and higher naturalness becomes possible.

  Here, similarly to the speech speed conversion device 10a of the first embodiment, the speech speed conversion device 10b of the second embodiment is a speech determined by the above-described speech speed conversion magnification determination device 1b and the speech speed conversion magnification determination device 1b. And a speech speed conversion unit 4 for converting the speech speed of the input signal according to the speed conversion magnification. The operation when the speech speed conversion device 4 needs to operate in real time is the same as that of the first embodiment.

  Similarly to the first embodiment, a computer can be suitably used to function as the speech speed conversion magnification determination device 1b or the speech speed conversion device 10b. Such a computer can be used as the speech speed conversion magnification determination device 1b. It can be realized by storing a program describing the processing contents for realizing each function in the storage unit of the computer, and reading and executing the program by a central processing unit (CPU) of the computer. .

  In addition, the program describing the processing contents can be recorded on a computer-readable recording medium such as a DVD or CD-ROM, and the recording medium can be distributed by selling, transferring, renting, etc. For example, the program can be distributed by storing the program in a storage unit of a server on a network such as an IP and transferring the program from the server to another computer via the network.

  In addition, a computer that executes such a program can temporarily store, for example, a program recorded in a recording medium or a program transferred from a server in its storage unit. As another embodiment of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and each time the program is transferred from the server to the computer. In addition, the processing according to the received program may be executed sequentially.

  Next, a speech speed conversion magnification determination apparatus according to a third embodiment to which auxiliary means for further stabilizing the effect of adaptive speech speed conversion in the present invention is described. In addition, the same reference number is attached | subjected to the same component as Example 2, and description is abbreviate | omitted.

FIG. 8 is a block diagram showing the configuration of the speech speed conversion magnification determination apparatus according to the third embodiment of the present invention. The speech speed conversion magnification determination device 1c according to the present embodiment is similar to the speech speed conversion magnification determination device 1a according to the first embodiment and the speech speed conversion magnification determination device 1b according to the second embodiment. The physical index calculation unit 2 that calculates the physical index of the input signal for the segment, and the speech speed conversion magnification α _n to be specified for each segment of the input signal are determined based on the physical index input from the physical index calculation unit 2. And a speech speed conversion magnification determination unit 3.

  Compared to the speech speed conversion magnification determination device 1b (see FIG. 5) of the second embodiment, the physical index calculation unit 2 further includes an auxiliary physical index calculation unit in the speech speed conversion magnification determination device 1c of the third embodiment. The voiced outline calculation unit 300, the fundamental frequency outline calculation unit 400, the unevenness calculation unit 410, the power outline calculation unit 500, the unevenness calculation unit 510, and the frequency band division / power calculation unit. 600 and a divided band power ratio calculation unit 610, and the speech speed conversion magnification determination unit 3 is a speech speed conversion magnification specification unit (speech speed conversion magnification specification unit based on an auxiliary physical index). Speaking speed conversion magnification designation unit c) 320, fourth speech speed conversion magnification designation unit (speech speed conversion magnification designation unit d) 420, and fifth speech speed conversion magnification designation unit (speech speed conversion magnification designation unit e). 520 and a sixth speech speed conversion magnification designation section (speech speed conversion magnification designation section f) 620 That point is different. The power outline calculating unit 200 includes a power calculating unit 202 and a smoothing unit 204. The voiced outline calculation unit 300 includes a voiced calculation unit 302 and a smoothing unit 304. The frequency band division / power calculation unit 600 includes a spectrum calculation unit 602, a band division unit 604, and a power calculation unit 606. The internal configuration of the basic frequency outline calculator 400 is the same as that of the basic frequency outline calculator 100, and the internal configuration of the power outline calculator 500 is the same as that of the power outline calculator 200.

(Subsidiary speed conversion magnification control using voicedness)
The voicing degree calculation unit 302 obtains an autocorrelation function R (τ) from the input signal waveform including background sound such as broadcast sound, and obtains the voicing degree using the autocorrelation function R (τ). The autocorrelation function R (τ) is derived from the following equation (12), and the voicing degree u is derived from the following equation (13).

u = W (τ) · R (τ) _max / R (0) (13)
Here, R (τ) _max is the maximum value at τ> 0, as shown in FIG. 9B. τ is a time delay amount, and W (τ) is a weight according to the value of τ giving R (τ) _max . Alternatively, as another calculation method, the number of zero crossings of the input signal waveform per unit time (5 ms) can be counted, and the reciprocal of this count value can be obtained.

The voicing degree u is reliably calculated every unit time (5 ms) in any part of the input signal, but the value does not always change smoothly. Therefore, the smoothing unit 304 calculates U _n (hereinafter referred to as “sample value of outline of voicedness”) obtained by smoothing the locus of voicedness for each unit time input from the voicedness calculating unit 302. The third speech speed conversion magnification designation unit (speech speed conversion magnification designation unit c) 320 outputs the result. For this smoothing, it is preferable to use a low-pass filter having a cutoff frequency of about 3 to 6 Hz.

Third speech speed conversion magnification specification unit (speech speed conversion ratio specifying section c) 320, depending on the value of the sample value U _n of the envelope of Yukoedo calculates the speech speed conversion ratio .alpha.c _n. A case where an autocorrelation function is used will be described. Generally, the sample value _{U n} of the envelope of Yukoedo takes a value in the range of about -0.2～1.2. Therefore, if the sample value _{U n} of the envelope of Yukoedo is greater than 0.5 loosen the speech speed _{(αc n <1.0), U} n is accelerating the speech speed if 0.5 or less (.alpha.c _n > 1.0) and that, for example, calculates the speech speed conversion ratio .alpha.c _n by the following equation (14) to (16).

However, in Formula (14), when U _n <−0.2, the calculation is performed as U _n = −0.2. Here, Rc is the contribution rate to the speech rate conversion magnification specified by the outline of the voicedness, and 0 ≦ Rc ≦ 1. Also, K is, together with the value of the sample value U _n of the envelope of Yukoedo is a constant for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

(Auxiliary speech rate conversion magnification control using the rough shape of the fundamental frequency)
Next, an example of an operation using the rough shape of the fundamental frequency will be described. The basic frequency outline calculation unit 400 operates in the same manner as the basic frequency outline calculation unit 100 described in the first embodiment, and outputs a sample value F _n of the basic frequency outline for each unit time.

Uneven calculator (fundamental frequency unevenness degree calculation unit) 410, asperity representing a change trend of the sampled values F _n envelope of the fundamental frequency (hereinafter, referred to as "irregularity of the envelope of the fundamental frequency") S _n Is calculated. For example, with respect to the sample value _{F n} envelope of the fundamental frequency, by using the value Fb _n front 30ms, respectively, and a value Fa _n of rear _30ms, and _{(F n -Fb n) (F} n -Fa The average value of _n ) is defined as the rough shape S _n of the rough shape of the fundamental frequency, and the degree of maximum or minimum is calculated. In this case, in a section where the trajectory is flat or monotonously increases or decreases, the maximum or minimum is a value close to zero. It should be noted that out of the irregularities S _n of the rough shape of the fundamental frequency, the values having the largest absolute values are used to normalize the irregularities S _n of the rough shapes of all the fundamental frequencies. Therefore, the value of irregularity level S _n of envelope of the fundamental frequency of the order of maximum or minimum becomes -1 to 1.

This method is equivalent to calculating the second-order difference of the sample value F _n of the approximate shape of the fundamental frequency if a value of 5 ms (1 sample) before and after the sample value F _n of the approximate shape of the fundamental frequency is used. It is. That is, first, a second-order difference F ″ _n = (F _n −F _n−1 ) − (F _n−1 −F _n−2 ) is calculated for the entire sample value F _n of the basic frequency. then use the largest value that is an absolute value, all F _"n normalized, the asperity S _n of envelope of the fundamental frequency by reversing the sign at the same time. As a result, the value of irregularity level S _n of envelope of the fundamental frequency also becomes -1 to 1. As is well known, the second-order differentiation of a function takes a positive value when the function is a minimum and takes a negative value when the function is a maximum. In addition, the larger the absolute value, the stronger the maximum / minimum degree (the unevenness is sharper). Since the second difference for any continuous curve is considered to second derivative equivalent, you can treat this S _n as envelope of the unevenness of the fundamental frequency.

Fourth speech speed conversion magnification specification unit (speech speed conversion ratio specifying section d) 420, depending on the value of the irregularity degree S _n of envelope of the fundamental frequency of each unit time (5 ms), the value of S _n is a positive the loosening the speech speed when, when the negative and to speed up the speech rate, for example, by the following equation (17) to (19), calculates the speech speed conversion ratio .alpha.d _n.

Here, Rd is a contribution rate to the speech rate conversion magnification specified by the rough shape of the fundamental frequency, and 0 ≦ Rd ≦ 1. Also, K is, together with the value of the unevenness of S _n of envelope of the fundamental frequency is a constant for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

(Auxiliary speech speed conversion magnification control using unevenness of power outline)
Next, an example of the operation using the rough shape of the power shape will be described. The basic method is the same as the case of using the rough shape of the basic frequency. With respect to the input signal, the unevenness calculation unit 510 calculates the unevenness of the peaks and valleys for the output of the power outline calculation unit 500. The basic frequency outline calculation unit 500 operates in the same manner as the power outline calculation unit 200 described above, and a power outline sample value P _n is output every unit time (5 ms).

The unevenness degree calculation unit (power unevenness degree calculation unit) 510 calculates an unevenness degree (hereinafter referred to as “power outline shape unevenness degree”) Q _n representing a change tendency of the sample value P _n of the power outline. . For example, with respect to the sample value _{P n} envelope of the power, with the value Pb _n front 30ms, respectively, the value Pa _n of rear _30ms, (P n -Pb _n) and _(P n -Pa _n ) Is the degree of unevenness Q _n of the approximate shape of power, and the degree of maximum or minimum is calculated. In this case, in a section where the trajectory is flat or monotonously increases or decreases, the maximum or minimum is a value close to zero. Among the asperity Q _n of envelope power, the absolute value with the largest value, normalized by dividing the degree of unevenness Q _n of envelope of all power. Therefore, the value of irregularity level S _n of envelope power of the order of maximum or minimum becomes -1 to 1.

This method is equivalent to calculating the second-order difference of the sample value P _n of the power outline if a value of 5 ms (1 sample) before and after the sample value P _n of the outline of power is used. This is the same as the approximate sample value of the fundamental frequency. That is, the second-order difference P ″ _n = (P _n −P _n−1 ) − (P _n−1 −P _n−2 ) is calculated for the entire sample value P _n of the power outline, Then, using the value whose absolute value is the largest, all P ″ _n are normalized, and at the same time, the sign is inverted to obtain the rough shape of the power Q _n . As a result, the value of irregularity level Q _n of envelope power is also an -1 to 1.

The fifth speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit e) 520 has a positive Q _n value according to the rough roughness Q _n value of the power shape per unit time (5 ms). time loosening the speech speed, when the negative and to speed up the speech rate, for example, by the following equation (20) to (22), calculates the speech speed conversion ratio .alpha.e _n.

Here, Re is a contribution rate to the speech rate conversion magnification specified by the rough shape of the power, and 0 ≦ Re ≦ 1. Also, K is, together with the value of the unevenness of Q _n of envelope power, is a constant for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

(Subsidiary speed conversion magnification control using power ratio of divided frequency bands)
Next, an example of an operation using the power ratio of the divided frequency band will be described. The frequency band division / power calculation unit 600 calculates the power spectrum of the input signal, and calculates the normalized power in the first frequency band and the normalized power in the frequency band higher than the first frequency band.

  The spectrum calculation unit 602 converts a time-domain waveform into a frequency domain by a fast Fourier transform (FFT) or the like every unit time (5 ms) with respect to an input signal, and a logarithmic power spectrum ( The unit is dB).

  The band dividing unit 604 divides the power spectrum input from the spectrum calculating unit 602 into a plurality of frequency bands. For example, the frequency band B1: 0 to 300 Hz, the frequency band B2: 300 to 1500 Hz, the frequency band B3: 1500 to 3000 Hz, the frequency band B4: 3000 to 8000 Hz, and the frequency band B5: 8000 Hz or more are divided.

  The power calculation unit 606 calculates normalized power for the lower frequency band and the higher frequency band, respectively. For example, here, the frequency band B2 is selected as the lower frequency band, and the frequency band B4 is selected as the higher frequency band. The normalized power is calculated by summing each value by the number of power spectra included in each frequency band and then dividing by the number. The power calculation unit 606 outputs the calculated normalized power of the frequency band B2 and the frequency band B4 to the divided band power ratio calculation unit 610.

The divided band power ratio calculation unit 610 has a logarithmized difference between the normalized power on the low frequency side and the normalized power on the high frequency side input from the power calculation unit 606. As a difference, The normalized power on the high frequency side is subtracted from the normalized power (that is, the ratio of the normalized power is obtained). Usually, the difference is about 10 dB to 40 dB. Then, the locus of the calculated the value for each unit time (5 ms) smoothes the ratio of the normalized power of the divided frequency bands (hereinafter, referred to as split band power ratio) is calculated E _n, the sixth The data is output to the speech speed conversion magnification designation unit (speech speed conversion magnification designation unit f) 620. For this smoothing, it is preferable to use a low-pass filter having a cutoff frequency of about 3 to 6 Hz.

Sixth speech speed conversion magnification specification unit (speech speed conversion ratio specifying section f) 620, if the subband power ratio E _n is greater than 25dB will loosen the speech speed, if 25dB less split-band power ratio E _n For example, the speech speed conversion magnification αf _n is calculated by the following equations (23) to (25).

However, in Formula (23), when E _n <10 (the unit is dB), calculation is performed with E _n = 10. Here, Rf is a contribution rate to the speech rate conversion magnification specified by the divided band power ratio, and 0 ≦ Rf ≦ 1. Also, K is, together with the values of the divided band power ratio E _n, are constants for adjusting the width or faster loosen speech rate, for example, K = 1.4 to 2.0.

  Each value of Rc in Formulas (14) to (16), Rd in Formulas (17) to (19), Re in Formulas (20) to (22), and Rf in Formulas (23) to (25) The values are adjusted in the same row as Ra of 5) to (7) and Rb of formulas (8) to (10). For example, when the input signal is broadcast, if the genre of the program (news, documentary, drama, variety, rakugo / comic) is known, the distribution ratio of the values is optimized accordingly, making it easier to hear and more natural. Highly adaptive adaptive speech speed conversion is possible. For example, for news, Ra = 0.3, Rb = 0.1, Rc = 0.1, Rd = 0.3, Re = 0.1, Rf = 0.1, for documentary, drama, Ra = 0.2, Rb = 0.2, Rc = 0.1, Rd = 0.2, Re = 0.2, Rf = 0.1, if rakugo / comic, Ra = 0.1, Rb = 0. 1, Rc = 0.3, Rd = 0.2, Re = 0.2, Rf = 0.1, and the like.

  Also, by adjusting the values of Ra, Rb, Rc, Rd, Re, and Rf according to the language used for speech speed conversion, it is possible to obtain a converted speech that sounds more natural in each language. .

(Fine adjustment of speech speed conversion magnification)
Finally, an example of the operation in the speech speed conversion magnification fine adjustment unit 140 will be described. The speech speed conversion magnification α _n given to the n-th counting unit time (5 ms) from the beginning of the input signal is basically the expressions (5), (6), (8), (9), (14 ), (15), (17), (18), (20), (21), (23), (in the case of using a _{24) α n = αa n ×} αb n × αc n × αd n × αe and _n × .alpha.f _n, equation (7), (10), (16), (19), (22), in the case of using _{(25) α n = αa n} + αb n + αc n + αd n + αe n + αf n And However, when the reproduction speed conversion magnification α is given, this is finely adjusted according to the following procedure. As the reproduction speed conversion magnification α, an arbitrary value such as 0.5 to 5.0 can be set.

When the reproduction speed conversion magnification α is given, if the length of the entire input signal is L (unit is second), the length of the entire converted signal must be L / α. First, the speech speed conversion magnification fine adjustment unit 140 performs the expressions (5), (6), (8), (9), (14), (15), (17), (18), (20), ( 21), (23), (all input signal segment to convert the speech speed as _{αaf n = αa n × αb n} × αc n × αd n × αe n × αf n in the case of using the 24), equation (7 ), (10), (16), (19), (22), speech rate all input signal segment as _{αaf n = αa n + αb n} + αc n + αd n + αe n + αf n in the case of using the (25) As a result of the conversion, the length L ₀ of the converted speech after connection is calculated first as a whole.

Next, by the following equation (26), the speech speed conversion magnification αaf _n of each part is finely adjusted to determine the final speech speed conversion magnification α _n , so that the length of the entire converted signal is to be reproduced. Can be adjusted to the length.
α _n = αaf _n × L ₀ / (L / α) (26)

When adjusting to the same timing as the voice converted uniformly at the playback speed conversion magnification α as frequently as possible, not the length L of the entire input signal but the length of the voice divided into shorter units. Then, α _n may be corrected so that fine adjustment is performed. For _example, if it can be divided into M as _{L = L 1 + L 2 +} ··· + L M _is, L _1, L 2, · · ·, divides the input waveform on each section of the _{L M,} respectively in divided section, m-th in the intervals is first speech speed conversion ratio of each part per unit time of the interval _{(5ms) αaf n (αa n} × αb n × αc n × αd n × αe n × αf n or .alpha.a _n + αb _n + αc _n + αd _n + αe _n + αf _n ) is used to perform speech speed conversion of the m-th interval, and the partial length Lm ₀ of the connected converted speech is first calculated. the Lm instead, by setting Lm ₀ instead of L in L ₀ seek the speech speed conversion ratio alpha _n, a fine adjustment by converting again speech speed. Note that the speech speed conversion (waveform expansion / contraction) method for realizing the speech speed conversion magnification α _n can be the same as in the first embodiment.

FIG. 10 is a flowchart illustrating the operation of the speech speed conversion magnification determination apparatus 1c according to the third embodiment. The speech speed conversion magnification determination apparatus 1c inputs a signal for speech speed conversion (step S301). When the signal to be converted speech speed is input, the speech speed conversion ratio determination device 1c, the fundamental frequency envelope calculator 100 derives a sample value F _n of envelope of the fundamental frequency (step S302), the power envelope the calculation unit 200 derives a sample value _{P n} of the envelope power (step S303), the Yukoedo envelope calculator 300 derives a sample value _{U n} of Yukoedo outline (step S304), the fundamental frequency the envelope calculation section 400 and the uneven degree calculation unit 410 derives the degree of unevenness _{S n} of envelope of the fundamental frequency (step S305), the power envelope calculation unit 500 and the uneven calculator 510, the outline of the power derives asperity _{Q n} (step S306), the frequency band dividing power calculating unit 600 and the divided band power ratio calculation section 610 derives a split band power ratio _{E n} (step S307)

When the approximate sample value F _n of the fundamental frequency is derived in step S302, the speech speed conversion magnification determining apparatus 1c is operated by the first speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit a) 120. calculating a speech speed conversion ratio .alpha.a _n (step S308). When the sample value P _n of the outline of power is derived in step S303, the speech speed conversion magnification determining apparatus 1c uses the second speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit b) 220 to speak. The speed conversion magnification αb _n is calculated (step S309). If sample value U _n of the chromatic Koedo envelope is derived in step S304, speech speed conversion-magnification determining device 1c, the third speech speed conversion magnification specification unit (speech speed conversion ratio specifying section c) 320, Talk calculating the speed conversion ratio .alpha.c _n (step S310). When asperity S _n of envelope of the fundamental frequency is derived in step S305, speech speed conversion-magnification determining device 1c, the fourth speech speed conversion magnification specification unit (speech speed conversion ratio specifying section d) 420, calculating a speech speed conversion ratio .alpha.d _n (step S311). When the irregularity degree Q _n of the approximate shape of the power is derived in step S306, the speech speed conversion magnification determining apparatus 1c uses the fifth speech speed conversion magnification specifying unit (speech speed conversion magnification specifying unit e) 520 to speak. calculating the speed conversion ratio .alpha.e _n (step S312). When split-band power ratio E _n at step S307 is derived, the speech speed conversion ratio determination device 1c, the sixth speech speed conversion magnification specification unit (speech speed conversion ratio specifying section f) 620, the speech speed conversion ratio αf _n is calculated (step S313). And, finally, speech speed conversion-magnification determining device 1c, the speech speed conversion ratio fine adjustment section 140, obtains the speech speed conversion ratio alpha _n from the speech speed conversion ratio αa _n ~αf _n. When the reproduction speed conversion magnification α is given, α _n is finely adjusted as the final speech speed conversion magnification (step S314).

Here, although an example of using the speed conversion factor αa _n ~αf _n all talk, for speech speed conversion ratio αc _n ~αf _n may be configured to use at least one.

When the sampled value U _n (speech rate conversion magnification αc _n ) of the outline of voicedness is used in combination, there are the following advantages. As described above, this physical index can be obtained at all positions of the input signal. It can also be calculated even when background sounds (music and noise) are mixed. Usually, the vowel part is highly voiced. In addition, a completely silent part or a background sound such as music or noise in which various sound frequency components are generally mixed has low voicedness. Therefore, by slowing the speech speed at high voiced levels and increasing the speech speed at low voiced levels, even when background sounds are mixed, the speech speed is reduced at the vowel part, which is an important part of speech. , the portion of only complete silence and background sound voice speed is accelerated, in addition to the sample value F _n of envelope of the fundamental frequency, voicing rate also by adding, as an entire input signal, more stable effective Adaptive speech speed conversion becomes possible.

Further, when the rough shape S _n (speech rate conversion magnification αd _n ) of the rough shape of the fundamental frequency is used together, there are the following advantages. This is different from Patent Document 1 in which “the speaking speed is slowed down when the fundamental frequency is high and the speaking speed is increased when the fundamental frequency is low”. For example, when thinking about comics such as a combination of men and women, the voices of men and women change violently with almost no pause. With respect to such an input signal, the technique described in Patent Document 1 “slows the speech speed when the fundamental frequency is high, and accelerates the speech speed when the fundamental frequency is low”. There is a tendency for male voices to always be faster because they are low. Compared with the technique of Patent Document 1 in which a voiced section and a voiceless section have to be correctly distinguished, in the speech speed conversion magnification determination apparatuses 1a and 1b according to the first and second embodiments, noise and background sound are mixed. Although there is an advantage that it operates stably in terms of using an outline of the continuous fundamental frequency including the voice, when the voices of men and women are mixed, the speech rate is proportional to the approximate value of the fundamental frequency. Operation may become unstable in terms of setting the conversion magnification.

Thus, by a combination of uneven degree S _n of the envelope of the fundamental frequency, in the approximate shape of the fundamental frequency, in association such as the words of the accent, even in parts of the female voice, even in parts of the voice of the man, always uneven Therefore, for both men and women, the convex part can slow down the speech speed, and the concave part can speed up the speech speed, and it is possible to adaptively control the speech speed with fair distribution to both men and women.

Further, when the unevenness Q _n (speaking speed conversion magnification αe _n ) of the rough shape of the power is used together, there are the following advantages. For example, when considering a drama or a lecture, it is often the case that one sentence is uttered with a loud voice and the next one sentence is suddenly uttered with a small voice. With respect to such an input signal, in the speech speed conversion magnification determination apparatus 1b according to the second embodiment, there is a tendency that a sentence with a loud voice relatively slows down and a sentence with a low voice relatively accelerates. .

Therefore, by using the rough shape Q _n of the power outline together, even in a sentence with a loud voice or a sentence with a low voice, an irregularity always accompanies the accent of each word. Can slow the speech speed, and the concave portion can speed up the speech speed, and adaptive control of the speech speed is possible with a fair distribution regardless of the size of the voice.

Further, when the divided band power ratio E _n (speech rate conversion magnification αf _n ) is used in combination, the following advantages are obtained. In Patent Documents 4 and 5, by comparing the powers of a plurality of bands of the frequency spectrum in the steady state and the corresponding bands of the frequency spectrum of the input signal, it is determined whether the input signal is a “voice section” or “silent section”. Although it is stated that the frequency spectrum is divided into a plurality of bands, the “power ratio between the lower side and the higher side of two bands when the frequency spectrum is divided into a plurality of bands” of the present invention is compared with the power of the spectrum in the steady state. Instead, only the frequency spectrum at a certain moment of the input signal is subjected to band division, and the power ratio between the lower side and the higher side of two of those bands is obtained. It is a physical quantity that is essentially different from technology. When the technology described in Patent Document 4 or 5 is used to determine whether it is a “voice section” or “silent section”, as described above, music with a certain level of volume is mixed as background sound. Therefore, it is difficult to correctly distinguish between “voice section” and “silent section”, and therefore adaptive speech speed conversion cannot be performed correctly.

Therefore, the combined use of the subband power ratio E _n, targeting only the frequency spectrum of a certain moment the input signal in order to determine the speech rate on the basis of the two bands of the low side and high side power ratio of certain of its There is essentially no judgment error, and the speech speed can be controlled stably. For example, it is possible to perform control such as slowing the speech speed when the power on the high side is small relative to the power on the low side, and increasing the speech speed when the power on the high side is large relative to the power on the low side. This “power ratio between the low side and the high side of the two bands” varies depending on the type of the input signal, such as voice interval, music, noise, and silence. As a result, the speech speed can be reduced in the voice section, and the speech speed can be increased in the music, noise, and silence sections.

  Here, similarly to the speech speed conversion device 10a of the first embodiment, the speech speed conversion device 10c of the third embodiment is a speech determined by the above-described speech speed conversion magnification determination device 1c and the speech speed conversion magnification determination device 1c. And a speech speed conversion unit 4 for converting the speech speed of the input signal according to the speed conversion magnification. The operation when the speech speed conversion device 4 needs to operate in real time is the same as that of the first embodiment.

  Similarly to the first embodiment, a computer can be suitably used to function as the speech speed conversion magnification determination device 1c or the speech speed conversion device 10c. Such a computer can be used as the speech speed conversion magnification determination device 1c. It can be realized by storing a program describing the processing contents for realizing each function in the storage unit of the computer, and reading and executing the program by a central processing unit (CPU) of the computer. .

  In addition, the program describing the processing contents can be recorded on a computer-readable recording medium such as a DVD or CD-ROM, and the recording medium can be distributed by selling, transferring, renting, etc. For example, the program can be distributed by storing the program in a storage unit of a server on a network such as an IP and transferring the program from the server to another computer via the network.

  In addition, a computer that executes such a program can temporarily store, for example, a program recorded in a recording medium or a program transferred from a server in its storage unit. Further, as another embodiment of the program, the computer may read the program directly from the recording medium and execute processing according to the program, and each time the program is transferred from the server to the computer, The processing according to the received program may be executed sequentially.

  Each of the above embodiments has been described as a representative example, but it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims.

  The present invention is useful for any application that requires speech speed conversion. For example, it is possible to listen to television or radio sound slowly in real time, or to record it once on a hard disk recorder or the like for viewing slowly or quickly. In addition, there is a demand from a visually impaired person to efficiently listen to audio information. By using the present invention, it is possible to reproduce and listen to a recording book for a visually impaired person at high speed. Furthermore, in a language learning or utterance training system, it can be used when creating the teaching material, or it can be used for learning by converting the speech speed of the voice according to the learner's progress.

DESCRIPTION OF SYMBOLS 1a, 1b, 1c Speech speed conversion magnification determination apparatus 2 Physical index calculation part 3 Speech speed conversion magnification determination part 4 Speech speed conversion part 10a, 10b, 10c Speech speed conversion apparatus 100 Fundamental frequency outline calculation part 102 Sound / silence determination Unit 104 Fundamental frequency calculation unit 106 Smoothing unit 108 Pseudo fundamental frequency calculation unit 110 Fundamental frequency outline concatenation unit 120 First speech speed conversion magnification designation unit (speech speed conversion magnification designation unit a)
140 Spoken speed conversion magnification fine adjustment unit 200 Power outline calculation unit 202 Power calculation unit 204 Smoothing unit 220 Second speech speed conversion magnification designation unit (speech speed conversion magnification designation unit b)
300 voiced outline calculation unit 302 voiced calculation unit 304 smoothing unit 320 third speech speed conversion magnification designation unit (speech speed conversion magnification designation unit c)
400 Fundamental frequency outline calculation unit 410 Concavity and convexity calculation unit 420 Fourth speech speed conversion magnification designation unit (speech speed conversion magnification designation unit d)
500 Power outline calculation unit 510 Concavity and convexity calculation unit 520 Fifth speech speed conversion magnification designation unit (speech speed conversion magnification designation unit e)
600 Frequency band division / power calculation unit 602 Spectrum calculation unit 604 Band division unit 606 Power calculation unit 610 Division band power ratio calculation unit 620 Sixth speech rate conversion magnification designation unit (speech rate conversion magnification designation unit f)

Claims (11)

A speech speed conversion magnification determination device for determining an adaptive conversion ratio of a speech speed of an input signal,
A sound / silence determination unit for determining a sound section and a silence section of the input signal;
The fundamental frequency of the input signal is calculated at a certain time interval in the sound interval, the stable interval in which the fundamental frequency value changes within a predetermined variation range, and the fundamental frequency value exceeds the predetermined variation range. A fundamental frequency calculation unit for determining an unstable interval that changes with
A frequency smoothing unit that smoothes a temporal change in the fundamental frequency in the stable section;
In the unstable section and the silent section, a pseudo fundamental frequency calculation unit that calculates a pseudo fundamental frequency by interpolating the fundamental frequency with reference to a value of the fundamental frequency in the smoothed stable section;
A fundamental frequency outline linking unit that connects the smoothed fundamental frequency and the pseudo fundamental frequency to obtain a sample value of a series of fundamental frequencies;
A physical index calculation unit for outputting a sample value of a rough shape of the fundamental frequency as a physical index;
Based on the physical index, a speech speed conversion magnification specifying unit that calculates a speech speed conversion ratio to be specified in the input signal;
A speech speed conversion magnification determination apparatus comprising: The physical index calculation unit includes a power calculation unit that calculates the power of an input signal at a constant time interval, and a power smoothing unit that smoothes the time change of the power and obtains a sample value of an outline of the power. ,
The speech rate conversion magnification determination apparatus according to claim 1, wherein the approximate sample value of the fundamental frequency and the approximate sample value of the power are output as the physical index. The physical index calculation unit includes: a voicing degree calculation unit that obtains voicing degree from an input signal waveform; and a voicing degree smoothing unit that obtains a sample value of the voicing degree by smoothing a temporal change of the voicing degree. ,
The speech rate according to claim 2, wherein the sample value of the approximate shape of the fundamental frequency, the sample value of the approximate shape of the power, and the sample value of the approximate shape of the voicedness are output as the physical index. Conversion magnification determination device. The physical index calculation unit includes a fundamental frequency unevenness degree calculation unit that calculates an approximate shape unevenness degree of the fundamental frequency, which represents a change tendency of the approximate shape of the fundamental frequency,
The speech rate according to claim 2, wherein the sample value of the approximate shape of the fundamental frequency, the sample value of the approximate shape of the power, and the degree of irregularity of the approximate shape of the fundamental frequency are output as the physical index. Conversion magnification determination device. The physical index calculation unit includes a power irregularity degree calculating unit that calculates a degree of irregularity of the power outline, which represents a change tendency of the power outline.
The speech rate conversion according to claim 2, wherein the sample value of the rough shape of the fundamental frequency, the sample value of the rough shape of the power, and the degree of unevenness of the rough shape of the power are output as the physical index. Magnification determination device. The physical index calculation unit calculates a power spectrum of an input signal, and calculates a normalized power in a first frequency band and a normalized power in a second frequency band higher than the first frequency band. A power calculation unit and a divided band power ratio calculation unit that calculates a ratio of normalized power in the first frequency band and the second frequency band;
The approximate sample value of the fundamental frequency, the approximate sample value of the power, and the ratio of the normalized power in the first frequency band and the second frequency band are output as the physical index. The speech speed conversion magnification determining apparatus according to claim 2.   The speech rate conversion magnification specifying unit calculates the speech rate conversion rate based on the physical index and a contribution rate of each physical index to the speech rate. The speech speed conversion magnification determination apparatus according to one item.   Given the length of time to be reproduced for each part of the input signal or the divided input signal, the speech speed is adjusted so that the time length of the whole input signal or each part matches the time length to be reproduced. The speech speed conversion magnification determination apparatus according to claim 7, further comprising a speech speed conversion magnification fine adjustment unit that finely adjusts the conversion magnification to determine a final speech speed conversion magnification. A speech speed conversion device that performs adaptive speech speed conversion of an input signal,
A speech speed conversion magnification determination device according to claim 8, and a speech speed conversion unit that converts an input signal according to the speech speed conversion magnification.
The speech speed conversion unit is output when the entire input signal or the length of time to be reproduced for each part of the divided input signal is given and expanded at a uniform magnification at regular time intervals. The target signal to be calculated and the converted signal obtained by converting the input signal by the speech speed conversion magnification are compared on the time series of the signal to calculate a temporal shift amount,
The speech speed conversion magnification fine adjustment unit re-adjusts the subsequent speech speed conversion magnification according to the temporal shift amount. In a computer configured as a speech speed conversion magnification determination device that determines an adaptive conversion magnification of the speech speed of an input signal,
Discriminating between a voiced section and a silent section of the input signal;
The fundamental frequency of the input signal is calculated at a certain time interval in the sound interval, the stable interval in which the fundamental frequency value changes within a predetermined variation range, and the fundamental frequency value exceeds the predetermined variation range. Determining an unstable interval that varies with
Smoothing a time change of the fundamental frequency in the stable interval;
In the unstable section and the silent section, referring to the value of the fundamental frequency in the smoothed stable section, calculating a pseudo fundamental frequency by interpolating the fundamental frequency;
Connecting the smoothed fundamental frequency and the pseudo fundamental frequency to obtain a sample value of a series of fundamental frequencies;
Calculating a speech rate conversion magnification to be specified in the input signal according to a sample value of the outline of the fundamental frequency;
A program for running   A recording medium on which the program according to claim 10 is recorded.