JP2018116096A - Voice processing program, voice processing method and voice processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly calculate speech speed in voice data.SOLUTION: A voice processing program causes a computer to perform following first to fifth processing. The first processing is processing to detect a plurality of vowel sections included in a speech section in input voice data. The second processing is processing to calculate time lengths of the plurality of detected vowel sections. The third processing is processing to calculate frequency distribution on the time lengths of the plurality of vowel sections and to specify a minimum value in the time lengths of the plurality of vowel sections. The fourth processing is processing to calculate a mora number corresponding to the time lengths of the plurality of vowel sections on the basis of the specified minimum value of the time lengths and the frequency distribution. The fifth processing is processing to control an output signal corresponding to the speech section in the input voice data in accordance with the calculated mora number.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a voice processing program, a voice processing method, and a voice processing apparatus.

  As one of audio processes for audio data obtained by collecting sound with a microphone, a process for calculating a speech speed in the audio data is known. When calculating speech speed in voice data, a change in vowels included in the voice data is detected based on a change in vocal tract characteristics such as formant frequency, and the number of vowels (number of mora) per unit time is calculated. (For example, see Patent Documents 1 and 2.)

JP 7-295588 A Japanese Patent Laid-Open No. 10-70790

  When a person utters, long vowels in which the same vowel continues may be included. When the same vowel is continuously generated, the vocal tract characteristic hardly changes at the vowel break. For this reason, when detecting the change of the vowel included in the voice data based on the change of the vocal tract characteristic, the long vowel is detected as one vowel. That is, when detecting a change in vowels included in voice data based on a change in vocal tract characteristics, a vowel section that is actually 2 mora becomes a vowel section of 1 mora, and the correct speech speed in the voice data is calculated. It is difficult.

  In one aspect, an object of the present invention is to correctly calculate the speech speed in voice data.

  A sound processing program according to one aspect causes a computer to execute the following first to fifth processes. The first process is a process for detecting a plurality of vowel sections included in the utterance section in the input voice data. The second process is a process for calculating the time length of each of the detected plurality of vowel segments. The third process is a process of calculating the frequency distribution for the time lengths of the plurality of vowel segments and specifying the minimum value of the time lengths of the plurality of vowel segments. The fourth process is a process of calculating the number of mora corresponding to the time length of each of the plurality of vowel intervals based on the specified minimum value of the time length and the frequency distribution. The fifth process is a process for controlling the output signal corresponding to the speech section in the input voice data according to the calculated number of mora.

  According to the above-described aspect, it is possible to correctly calculate the speech speed in the voice data.

It is a figure which shows the functional structure of the speech-speed estimation apparatus which concerns on 1st Embodiment. It is a flowchart explaining the process which the speech-speed estimation apparatus which concerns on 1st Embodiment performs. It is a flowchart explaining the content of the process which detects a vowel section. It is a figure explaining the relationship between the single vowel and long vowel in audio | voice data. It is a figure which shows the example of the frequency distribution of a vowel section. It is a figure explaining the utterance content of an utterance area, and the calculation result of the number of mora of each vowel area. It is a figure which shows the example of an output of the calculation result of speech speed. It is a figure which shows another example of the calculation method of the number of mora. It is a figure which shows the 1st application example of a speech-speed estimation apparatus. It is a figure which shows the 2nd application example of a speech-speed estimation apparatus. It is a figure which shows the functional structure of the speech-speed estimation apparatus which concerns on 2nd Embodiment. It is a flowchart explaining the content of the process which detects the vowel area which concerns on 2nd Embodiment. It is a flowchart explaining the modification of the process which detects the vowel area which concerns on 2nd Embodiment. It is a figure which shows the system configuration | structure of the telephone call system which concerns on 3rd Embodiment. It is a figure which shows the functional structure of the speech-speed adjustment part which concerns on 3rd Embodiment. It is a figure which shows the functional structure of a speech speed control part. It is a flowchart explaining the process which the speech speed adjustment part which concerns on 3rd Embodiment performs. It is a flowchart explaining the content of the process which controls the speech speed of audio | voice data. It is a figure explaining the detection method of a fundamental period. It is a figure explaining the method of superimposing a speech waveform. It is a figure explaining the weighting method at the time of superimposing a speech waveform. It is a figure which shows the hardware constitutions of a computer.

[First Embodiment]
FIG. 1 is a diagram illustrating a functional configuration of the speech speed estimation apparatus according to the first embodiment.

  As shown in FIG. 1, the speech speed estimation apparatus 1 according to the present embodiment includes a speech acquisition unit 110, a speech segment detection unit 120, a vowel segment detection unit 130, a mora number determination unit 140, and a speech speed calculation unit. 150 and an output unit 160. Although omitted in FIG. 1, the speech speed estimation apparatus 1 includes a storage unit that stores various types of information including voice data.

The audio acquisition unit 110 acquires an audio signal (audio data) from the sound collection device 2.
The utterance section detection unit 120 detects the utterance section in the voice data. The utterance section is a section including a voice uttered by a person who is a target of speech speed estimation.

The vowel section detector 130 detects a vowel section included in the speech section of the voice data.
The mora number determination unit 140 calculates the number of mora in each vowel section based on the length (time length) of each of the plurality of vowel sections detected from the speech data. First, the mora number determination unit 140 of the speech speed estimation apparatus 1 according to the present embodiment calculates a time length of each of a plurality of vowel segments and calculates a frequency distribution of the time lengths for the plurality of vowel segments. The minimum value of the time length of each vowel section is specified. Thereafter, the mora number determination unit 140 calculates (determines) the number of mora corresponding to the time length of each of the plurality of vowel intervals based on the specified minimum value of the time length and the frequency distribution.

  The speech speed calculation unit 150 calculates the speech speed (speech speed) of the speech section in the speech data based on the time length of the speech section and the number of mora of the vowel section included in the speech section.

  The output unit 160 generates display data for visualizing the calculated speech speed and outputs the display data to the display device 3.

  Further, the mora number determination unit 140 in the speech speed estimation apparatus 1 of the present embodiment includes a frequency distribution calculation unit 141, a peak detection unit 142, and a mora number calculation unit 143, as illustrated in FIG.

  The frequency distribution calculating unit 141 calculates a frequency distribution indicating the appearance frequency for each time length of the vowel section in the speech data, based on the time length of the plurality of vowel sections.

  The peak detection unit 142 detects the time length of a vowel section where the appearance frequency is a peak (maximum value) in the frequency distribution of the time length of the vowel section.

  The mora number calculation unit 143 calculates the number of mora in each vowel section based on the time length of the vowel section in which the appearance frequency is a maximum value. The mora number calculation unit 143 according to the present embodiment uses the shortest time length of the time lengths of the vowel section where the appearance frequency is the maximum value as the reference time length (time length corresponding to 1 mora), and the time of the vowel section Based on the ratio between the length and the reference time length, the number of mora in each vowel section is calculated.

  When the operation is started, the speech speed estimation apparatus 1 of the present embodiment acquires voice data from the sound collection apparatus 2, and processes to estimate the speech speed in the acquired voice data and output the estimation result to the display apparatus 3. I do. The voice acquisition unit 110 of the speech speed estimation apparatus 1 performs the process of acquiring the voice data.

  On the other hand, the process of estimating the speech speed in the speech data and outputting the estimation result to the display device 3 includes the speech segment detection unit 120, the vowel segment detection unit 130, the mora number determination unit 140, and the speech speed calculation unit of the speech speed estimation device 1. 150 and the output unit 160. The speech speed estimation device 1 performs the processing shown in FIG. 2 as processing for estimating speech speed in speech data and outputting the estimation result to the display device 3.

  FIG. 2 is a flowchart for explaining processing performed by the speech speed estimation apparatus according to the first embodiment.

  In the process of estimating the speech speed in the speech data and outputting the estimation result to the display device 3, the speech speed estimation device 1 first detects an utterance section included in the speech data (step S1). The processing of step S1 is performed by the utterance section detection unit 120. The utterance section detection unit 120 detects an utterance section (in other words, a section including speech uttered by a person who is a target of speech speed estimation) included in the sound data according to a known detection method. For example, the utterance section detection unit 120 detects the utterance section by Voice Activity Detection (VAD).

  Next, the speech speed estimation device 1 detects a vowel section included in the utterance section (step S2). The process of step S2 is performed by the vowel section detection unit 130. The vowel section detection unit 130 detects a vowel section included in the speech section of the voice data according to a known detection method. For example, the vowel section detection unit 130 detects, as one vowel section, one section in which the signal-to-noise ratio is continuous at a predetermined threshold or more based on the time change of the signal-to-noise ratio in the utterance section.

  Next, the speech speed estimation apparatus 1 performs processing (steps S3 to S5) for determining the number of mora (number of vowels) in the detected vowel section. The processing of steps S3 to S5 for determining the number of mora in the vowel section is performed by the mora number determination unit 140 of the speech speed estimation apparatus 1.

  The mora number determination unit 140 first calculates a frequency distribution for the time length of the vowel section based on the detected time lengths of the plurality of vowel sections (step S3). The processing in step S3 is performed by the frequency distribution calculation unit 141 of the mora number determination unit 140. The frequency distribution calculating unit 141 calculates the time length of each vowel section based on the start time and end time of each detected vowel section. Further, the frequency distribution calculation unit 141 calculates the frequency distribution by counting the appearance frequency of the vowel section for each time length based on the time length of each vowel section. At this time, the frequency distribution calculation unit 141 calculates the frequency distribution by excluding, for example, the last vowel section in one utterance section. In addition, the frequency distribution calculation unit 141 may calculate the frequency distribution by extracting, for example, a vowel section whose time length is within a predetermined range from all vowel sections included in one utterance section. . The predetermined range of the time length is set based on, for example, the time length of a single vowel and a long vowel at a general speaking speed.

  Next, the mora number determination unit 140 detects a time length at which the appearance frequency reaches a peak (maximum value) in the frequency distribution calculated in step S3 (step S4). The processing in step S4 is performed by the peak detection unit 142 of the mora number determination unit 140. For example, the peak detection unit 142 compares the appearance frequency of the time length that is the determination target with the appearance frequency of the time length before and after the appearance frequency in order from the appearance frequency of the shortest time length in the frequency distribution, and the appearance frequency becomes the maximum value. Detect time length.

  Next, the mora number determination unit 140 calculates the number of mora in each vowel section based on the minimum value of the time lengths detected from the frequency distribution and the time length of the vowel sections (step S5). The process of step S5 is performed by the mora number calculation unit 143 of the mora number determination unit 140. The mora number calculation unit 143 calculates a value obtained by dividing the time length of the vowel section by the reference time length, with the minimum value among the plurality of time lengths peaking in the frequency distribution as the reference time length. The reference time length corresponds to a time length of 1 mora (single vowel) in the speech section of the voice data. For this reason, the mora number calculation unit 143 sets an integer value close to a value obtained by dividing the time length of the vowel section by the reference time length as the number of mora of the vowel section. For example, when the time length of a certain vowel section is about twice the reference time length, the mora number calculation unit 143 sets the number of mora of the vowel section to two.

  When the processing of steps S3 to S5 by the mora number determination unit 140 is completed, the speech speed estimation apparatus 1 next utters based on the mora number of each vowel section calculated in step S5 and the time length of the utterance section. The speech speed of the section is calculated (step S6). The speech speed calculation unit 150 performs the process of step S6. The speech speed calculation unit 150 calculates, as the speech speed, a value (mora / second) obtained by dividing the total number of mora for the vowel section included in the speech section by the time length of the speech section.

  Next, the speech speed estimation apparatus 1 outputs the calculated speech speed (step S7). The output unit 160 performs the process in step S7. For example, the output unit 160 generates display data for visualizing the speech speed calculated in step S <b> 6 and outputs the display data to the display device 3.

  The speech speed estimation apparatus 1 repeatedly performs the processes of steps S1 to S7 described above. The speech speed estimation apparatus 1 may perform processing (step S1) for detecting the next utterance interval after completing the processing of steps S2 to S7 for one utterance interval detected in step S1. The whole or part of the processing of S7 may be performed in a pipeline.

FIG. 3 is a flowchart for explaining the contents of processing for detecting a vowel section.
The speech speed estimation apparatus 1 according to the present embodiment performs, for example, the process illustrated in FIG. 3 as the process of the above step S2 (process of detecting a vowel segment). The processing shown in FIG. 3 is performed by the vowel section detection unit 130 of the speech speed estimation device 1.

First, the vowel section detection unit 130 calculates the signal-to-noise ratio SNR (t _m ) at the time t _m (m = 1, 2,..., M) in the speech section of the voice data (step S201). The time interval (t _m −t _m−1 ) at time t _m in the utterance section is, for example, the time length of the processing unit (frame) when calculating the power and noise power of the voice data. In the following description, a frame associated with time t _m in the audio data is referred to as a frame at time t _m .

In the process of step S201, the vowel section detection unit 130 calculates the signal-to-noise ratio SNR (t _m ) in the frame at time t _m by the following equation (1).

SNR (t _m ) = P (t _m ) + N (t _m ) (1)

P (t _m ) in Equation (1) is the power in the frame at time t _{m in} the audio data. N (t _m ) in equation (1) is the noise power in the frame at time t _{m in} the audio data. The vowel section detector 130 calculates the power P (t _m ) and noise power N (t _m ) in the frame at time t _m according to a known calculation method. For example, the noise power N (t _m ) at time t _m is calculated by the equation (2).

N1 (t _m-1 ) in Expression (2) is a noise power that is updated based on the difference between the power P (t _m-1 ) and the noise power N (t _m-2 ) in the audio data. Further, in the equation (2), TH _P is determined threshold, COF is a forgetting factor. The value of the determination threshold value TH _P and the forgetting factor COF, respectively, may be set as appropriate.

When the processing of step S201 is completed, the vowel section detection unit 130 then sets the variable m, the variable i, and the signal-to-noise ratio SNR (t ₀ ) as m = 1, i = 1, and SNR (t ₀ ) = 0 is set (step S202). The variable i is a value that identifies a vowel section.

Next, the vowel section detection unit 130 has a signal-to-noise ratio SNR (t _m-1 ) at time t _m-1 smaller than the threshold TH _SNR and a signal-to-noise ratio SNR (t _m ) at time t _m. It is determined whether or not the threshold value is TH _SNR or more (step S203). That is, in step S203, it is determined whether SNR (t _m-1 ) <TH _SNR and SNR (t _m ) ≧ TH _SNR .

The threshold value TH _SNR is a value for determining whether the voice in the utterance section is a vowel or a non-vowel (consonant, etc.). The signal-to-noise ratio of the vowel section included in the speech section of the speech data is a larger value than the signal-to-noise ratio of the non-vowel section (such as consonant section) in the speech section. For this reason, in this embodiment, a section in which the signal-to-noise ratio in the utterance section is equal to or greater than the threshold TH _SNR is set as a vowel section. The threshold TH _SNR is set based on, for example, a statistical value of the signal-to-noise ratio in the vowel section and a statistical value of the signal-to-noise ratio in the non-vowel section.

When SNR (t _m−1 ) <TH _SNR and SNR (t _m ) ≧ TH _SNR , the signal-to-noise ratio is from a value smaller than the threshold TH _SNR between time t _m−1 and t _m. It changes to a value greater than TH _SNR . That is, when SNR (t _m−1 ) <TH _SNR and SNR (t _m ) ≧ TH _SNR , time t _m−1 is included in the non-vowel section and time t _m is included in the vowel section. Therefore, when SNR (t _m−1 ) <TH _SNR and SNR (t _m ) ≧ TH _SNR (step S203; YES), the vowel section detection unit 130 next starts the i-th vowel section start time. Ts (i) is set at time t _m (step S204). When the processing of step S204 is completed, the vowel section detection unit 130 next determines whether or not the variable m is equal to or greater than the last value M in the utterance section (step S207).

On the other hand, when SNR (t _m−1 ) ≧ TH _SNR or SNR (t _m ) <TH _SNR (step S203; NO), the vowel section detection unit 130 then selects SNR (t _m−1 ) ≧ TH _SNR, and determines whether the _{SNR (t m) <TH SNR} ( step S205).

When SNR (t _m−1 ) ≧ TH _SNR and SNR (t _m ) <TH _SNR (step S205; YES), time t _m−1 is included in the vowel interval, and time t _m is in the non-vowel interval. included. Therefore, when SNR (t _m−1 ) ≧ TH _SNR and SNR (t _m ) <TH _SNR (step S205; YES), the vowel section detection unit 130 then ends the ith vowel section end time. Te (i) is set at time t _m−1 and the variable i is updated to i + 1 (step S206). In step S206, the vowel section detection unit 130 sets the end time Te (i) of the vowel section to the time t _m−1 and then updates the variable i to i + 1. When the process of step S206 is completed, the vowel section detection unit 130 next performs the determination of step S207.

On the other hand, when SNR (t _m-1 ) ≧ TH _SNR and SNR (t _m ) <TH _SNR are not satisfied, the signal-to-noise ratio at time t _m−1 and time t _m both has a threshold value TH. whether there are _SNR than, or both smaller than the threshold value _{TH SNR.} That is, when SNR (t _m−1 ) ≧ TH _SNR and SNR (t _m ) <TH _SNR are not satisfied, both times t _m−1 and t _m are included in the non-vowel interval, or both Both are included in the vowel section. Therefore, when SNR (t _m−1 ) <TH _SNR or SNR (t _m ) ≧ TH _SNR (step S205; NO), the vowel section detection unit 130 skips step S206, and then proceeds to step S207. Judgment is made.

  In step S207, the vowel section detection unit 130 determines whether the current value of the variable m is equal to or greater than the maximum value M in the utterance section as described above. When m <M (step S207; NO), the vowel section detection unit 130 updates the variable m to m + 1 (step S208), and performs the processing after step S203.

  On the other hand, if m ≧ M (step S207; YES), the vowel section detection unit 130 outputs information [Ts (i), Te (i)] indicating the vowel section (step S209), one utterance The process of detecting a vowel section included in the section is terminated.

  The vowel section detection unit 130 according to the present embodiment performs the processes of steps S201 to S209 in FIG. 3 for each of the utterance sections detected by the utterance section detection unit 120.

  After the vowel section detecting unit 130 detects a plurality of vowel sections included in the utterance section, the speech speed estimation apparatus 1 determines the number of mora (number of vowels) in each detected vowel section by the processes of steps S3 to S5. . The process of determining the number of mora in the vowel section is performed by the mora number determination unit 140 of the speech speed estimation apparatus 1. As described above, the mora number determination unit 140 according to the present embodiment determines the number of mora in each vowel section based on the time length L (i) = {Te (i) −Ts (i)} of each vowel section. decide.

FIG. 4 is a diagram for explaining the relationship between single vowels and long vowels in voice data.
FIG. 4A shows the length of time for the utterance section 401 of the word “Haru” in the voice data as an example of a single vowel. For example, when “Haru” is the season “Spring”, the utterance time of “Ha” is substantially the same as the utterance time of “Ru”. That is, the time length ΔT1 (= t _m2 −t ₁ ) of “ha” shown in FIG. 4A and the time length ΔT2 (= t _m3 −t _m2 ) of “ru” are substantially the same (ΔT1≈ ΔT2).

On the other hand, FIG. 4B shows the length of time for the utterance section 403 of the word “Kyo” in the speech data as an example of a long vowel. For example, when “Kyo” is “Today” on the date, the utterance time of “Kyo” is substantially the same as the utterance time of “U”. That is, the time length ΔT3 (= t _m5 −t ₁ ) of “Kyo” and the time length ΔT4 (= t _m6 −t _m5 ) of “U” shown in FIG. ΔT3≈ΔT4). Furthermore, when the speaker is the same person and speaks at a constant speaking speed, the time length ΔT3 of “Kyo” and the time length ΔT4 of “U” are the time periods “ha” shown in FIG. The length ΔT1 and the time length ΔT2 of “ru” are substantially the same.

When a person utters “Haru”, the vowel changes to a consonant (non-vowel) when uttering “ru” of the second syllable, as in the phonetic notation 402 shown in FIG. To do. For this reason, in the utterance section 401 of “Haru” in the voice data, the time t _m1 between the time t ₁ and the time t _m2 uttering “ha” is the start time Ts (1) of the vowel section, The time t _m2 when the utterance of “ha” ends becomes the end time Te (1) of the vowel section. Further, in the speech section 401 of “Haru” in the voice data, the start time Ts (2) of the vowel section exists between the time t _m2 and the time t _{m3 of} “Ru”, and “R”. The time t _{m3 at} which the utterance of vowel ends is the end time Te (2) of the vowel section. Therefore, there are two vowel segments detected from the speech segment 401 “Haru” in the voice data.

On the other hand, when a person utters “Kyo”, he often utters “KYOO” as in the phonetic notation 404 shown in FIG. That is, when a person utters “Kyo”, instead of pronouncing the second syllable “U (U)”, the vowel “O” of “KYO” is uttered like a long vowel. There are many cases. For this reason, in the speech section 403 of “Kyo” in the voice data, the time t _m4 between the time t ₁ and the time t _m5 uttering “Kyo” is the start time Ts of the vowel section, and “U” The time t _m6 when the utterance of ends is the end time Te of the vowel section. Therefore, there is one vowel section detected from the speech section 403 “Kyo” in the voice data.

  As described above, the utterance section 401 “Haru” and the utterance section 403 “Kyo” in the speech data are both two syllables and the duration of the utterance section is substantially the same, but are detected vowel sections. The number of is different. “Haru” has two vowel segments, whereas “Kyo” has one vowel segment. For this reason, when the speech speed is calculated by dividing the number of vowels by the duration of the utterance section, the speech speed of the utterance section 403 “Kyo” in the speech data is about the speech speed of the utterance section 401 “Haru”. It becomes twice. The speech speed estimation apparatus 1 according to the present embodiment, in order to prevent such an error in speech speed due to the lengthening of the long vowels, is based on the frequency distribution of the time length of the vowel sections and the number of vowels (number of mora) in each vowel section. To decide.

  In the syllable including consonants such as “ha (HA)” and “ru (RU)” in the speech data as in the phonetic notation 402 of FIG. 4A, the time length of the vowel section is the time length of the entire syllable. It accounts for over 70%. For example, in the speech section of “ha” in the speech data, the time length L (1) = Te (1) −Ts (1) of the vowel section occupies 70% or more of the time length ΔT1 of the speech section. Similarly, in the speech section of “ru” in the speech data, the time length L (2) = Te (2) −Ts (2) of the vowel section occupies 70% or more of the time length ΔT2 of the speech section.

  On the other hand, when the vowel is turned into a long vowel in a section in which a syllable including a consonant and a syllable of only a vowel are continuous as in the phonetic notation 404 of FIG. 4B, the time length of the vowel section is Become. First, the time length of the vowel section in the syllable of “Kyo” including consonants occupies 70% or more of the time length ΔT3 of the syllable. Further, the time length of the vowel section in the “U” syllable that is only a vowel is the time length ΔT4 of the syllable. Therefore, the time length L of the vowel section included in the utterance section 403 “Kyo” is {(0.7 × ΔT3) + ΔT4} ≦ L <(ΔT3 + ΔT4). Therefore, the time length L of the vowel section included in the utterance section 403 “Kyo” is about twice the time length L (1) and L (2) of the single vowel.

  In this way, the time length of the vowel section that is converted into a long vowel is a value close to an integral multiple of the time length of a single vowel. For this reason, a characteristic corresponding to the time length of a single vowel appears in the frequency distribution of the time lengths of a plurality of vowel intervals included in the utterance interval.

FIG. 5 is a diagram illustrating an example of a frequency distribution in a vowel section.
The frequency distribution calculated by counting the frequency of the vowel section for each time length based on the time length in each of the plurality of vowel sections included in one utterance section is, for example, four as shown in FIG. Peaks P1, P2, P3, and P4. In the frequency distribution of FIG. 5, the time length L is divided into a plurality of sections every 0.02 seconds, and the number of vowel sections whose time length is included in the section is calculated as the frequency for each section. For example, the frequency of a vowel section with a time length L = 0.2 seconds is such that the time length (Te−Ts) of a plurality of vowel sections in the utterance section is 0.19 ≦ (Te−Ts) <0.21. Indicates the number of vowel intervals.

  The time length of a vowel section that is a long vowel or a vowel section in which a plurality of types of single vowels are continuous becomes longer than the time length of a vowel section of a single vowel. In addition, since Japanese generally has the property of uttering a single mora (single vowel) that is being uttered with approximately the same length (mora isochronism), as described above, a single mora within a single utterance interval is used. The time length of the vowel section of the vowel is substantially constant. For this reason, it can be said that the peak P1 having the shortest time length among the four peaks P1 to P4 in the frequency distribution indicates the time length of the vowel section of the single vowel. Further, the time length of the vowel section that is converted into a long vowel is a value close to an integral multiple of the time length of the vowel section of the single vowel. For this reason, the time length at which the peaks P2 to P4 appear in the frequency distribution is a value close to an integral multiple of the time length TP1 of the peak P1. Therefore, in this embodiment, the time length TP1 of the peak P1 with the shortest time length in the frequency distribution is set as the reference time length, and the number of vowels (number of mora) VN (i) in each vowel section is calculated by the following equation (3). To do.

  L (i) in Equation (3) is the time length of the i-th vowel section.

  As described above, the speech speed estimation apparatus 1 according to the present embodiment estimates the reference time length (single vowel time length) TP1 of the vowel section based on the frequency distribution regarding the time length of the vowel section, and the time of each vowel section. The number of vowels in each vowel section is determined from the ratio between the length and the reference time length TP1. For this reason, in the speech speed estimation apparatus 1 according to the present embodiment, it is possible to prevent an error in the number of mora (vowel number) in a vowel section that has been made into a long vowel.

  FIG. 6 is a diagram for explaining the utterance content in the utterance section and the calculation result of the number of mora in each vowel section.

The character string 411 in FIG. 6 shows the utterance content in the utterance section of the voice data as text. Also, the phonetic notation 412 in FIG. 6 shows each syllable in the character string 411 as a vowel and a consonant. When the word “you are good today” shown in the character string 411 is spoken naturally, the vowels are long in the parts “Kyo”, “Good”, and “Yo” as shown in the pronunciation notation 412. Often vowelized. For this reason, when processing for detecting a vowel section included in a speech section of speech data is performed, seven vowel sections are detected as shown below the phonetic notation 412. At this time, the vowel section detected from the “haoi” portion of the character string 411 is a section in which the section where the vowel is “A” and the section where the vowel is “I” are continuous. Therefore, when the number of vowels in a vowel section in which the vowel is a long vowel is 1, the number of vowels in the vowel section detected from the “yes” part is 2, and the number of vowels in the other vowel sections is 1. Become. Therefore, when the number of vowels in a vowel section in which a vowel is a long vowel is 1, the number of mora (vowel number) in the speech section (time t _{1 to} t _M ) of the voice data is 8, and the actual number of mora Different). That is, since there is a difference between the number of mora used for calculating the speech speed and the actual number of mora in the speech data, it is difficult to estimate the speech speed with high accuracy.

  In contrast, in the method for calculating the number of mora according to the present embodiment, first, the reference time length TP1 is calculated based on the frequency distribution for the time length of each vowel section. The reference time length TP1 in the example shown in FIG. 6 is the time length L (4), L (5), L (6) of the vowel section detected from the partial section of “Ki-N” in the character string 411, and The time length is approximately the same as each of L (7). When the calculated reference time length TP1 is used to calculate the number of mora VN (i) of each vowel section by Equation (3), the results shown in FIG. 6 are obtained. The time length L (1) of the first vowel section and the time length L (3) of the third vowel section are each about twice the reference time length TP1. Therefore, the mora number VN (1) of the first vowel section and the mora number VN (3) of the third vowel section are 2, respectively. Further, the time length L (2) of the second vowel section is about three times the reference time length TP1. Therefore, the mora number VN (2) of the second vowel section is 3. Further, the mora numbers VN (4) to VN (7) of the remaining vowel sections are each 1. Therefore, in the method for calculating the number of mora according to the present embodiment, the number of mora included in the utterance section of the character string 411 is 11. That is, according to this embodiment, it is possible to correctly calculate the number of mora in each vowel section. Therefore, according to the present embodiment, it is possible to correctly estimate the speech speed of the utterance section.

  The speaking speed SR of the utterance section is calculated by the following equation (4), for example.

  STs and STe in Equation (4) are the start time and end time of the utterance section, respectively.

  If the time length (STe-STs) of the utterance section is 10 seconds, the speech speed calculated based on the number of vowel sections included in the utterance section is about 0.7 (mora / second). On the other hand, the speech rate SR when the number of mora VN (i) of each vowel section is determined based on the frequency distribution of the time length of the vowel section is 1.1 (mora / second). As described above, according to the present embodiment, the number of mora (number of vowels) in the utterance section can be correctly calculated, and the speech speed in the utterance section can be accurately calculated.

  The speech speed estimation apparatus 1 according to the present embodiment calculates the speech speed, generates display data for visualizing the calculated speech speed, and outputs the display data to the display apparatus 3.

FIG. 7 is a diagram illustrating an output example of the speech speed calculation result.
FIG. 7 shows a graph 421 that presents the temporal change of the speech speed calculated for each speech section as an output example of the speech speed calculation result. On the left end of the graph 421, the value of the speech speed and information indicating whether or not the speech speed is appropriate are displayed. In the example shown in FIG. 7, the case where the speech speed is 6.0 to 8.0 mora / second is set as an appropriate speech speed. The horizontal axis in the graph 421 is the conversation time. In the graph 421, a curve 501 indicating the speech speed SR in each utterance section detected from the speech data is displayed. The curve 501 in the graph 421 displayed on the display device 3 is updated every time the speech speed estimation device 1 detects the speech section and the vowel section and calculates the speech speed. The curve 501 in the graph 421 may indicate, for example, the speaking speed of all the utterance sections during the conversation, or may indicate only the speaking speed within the latest predetermined time during the conversation.

  Note that the graph 421 in FIG. 7 is merely an example of a method for visualizing and displaying the calculation result of the speech speed. The calculation result of the speech speed is not limited to the method showing the temporal change of the speech speed as in the graph 421, and may be displayed by another method. For example, the display form of the speech speed calculation result may be a display form such as a level meter, and only the latest (most recent) speech speed calculated by the speech speed estimation apparatus 1 may be displayed. In addition, when the calculation result of the speech speed is displayed on the display device 3, for example, a display that alerts the speaker only when the calculated speech speed is equal to or higher than a predetermined threshold value may be displayed.

  As described above, in the speech speed estimation apparatus 1 according to the present embodiment, the time length of a single vowel detected based on the frequency distribution of the time length in each of a plurality of vowel sections included in the utterance section, and the time of the vowel section Based on the ratio to the length, the number of mora (number of vowels) in each vowel section is calculated. For this reason, in the speech speed estimation apparatus 1 according to the present embodiment, it is possible to prevent an error in the number of mora for a long vowel vowel section and to calculate a correct speech speed of the utterance section. Therefore, for example, the speech speed estimation apparatus 1 can present the time change of the correct speech speed to the user who is the target of speech speed estimation and guide the user to speak at an appropriate speech speed. It becomes possible.

  Note that the flowcharts of FIGS. 2 and 3 are merely examples of processing performed by the utterance estimation apparatus 1 according to the present embodiment. The processing performed by the utterance estimation apparatus 1 according to the present embodiment is not limited to the contents shown in FIGS. 2 and 3 and can be changed as appropriate. For example, in the process of detecting the vowel section included in the utterance section, the vowel section may be detected based on the waveform autocorrelation of the speech data or the formant frequency instead of the signal-to-noise ratio.

  Further, when calculating the frequency distribution for the time length of the vowel section, for example, only the vowel section whose time length is within a predetermined range among the detected plurality of vowel sections may be extracted and calculated. Good. The predetermined range is set based on, for example, the time length of a single vowel obtained by performing statistical processing in advance.

  In addition, the reference time length used when calculating the number of mora in each vowel section is not limited to the time length TP1 corresponding to the peak having the shortest time length among the plurality of peaks in the frequency distribution. It is good also as an average value of the time interval of an adjacent peak.

FIG. 8 is a diagram illustrating another example of a method for calculating the number of mora.
FIG. 8 shows an example of the frequency distribution for the time lengths of a plurality of vowel sections included in the utterance section. The frequency distribution in FIG. 8 is calculated by the same calculation method as the frequency distribution in FIG. That is, the frequency distribution of FIG. 8 is obtained by dividing the horizontal axis (time length L) into a plurality of sections every 0.02 seconds and calculating the number of vowel sections whose time length is included in each section as the frequency. doing. For example, the frequency of a vowel section with a time length L = 0.2 seconds is such that the time length (Te−Ts) of a plurality of vowel sections in the utterance section is 0.19 ≦ (Te−Ts) <0.21. Indicates the number of vowel intervals.

  From the frequency distribution of FIG. 8, the first peak P1, the second peak P2, the third peak P3, and the fourth peak P4 are detected in order from the peak with the shortest time length. The time length TP1 of the first peak P1 substantially coincides with the time length of a vowel section in which the number of mora is 1 (that is, the time length of a single vowel). In addition, the time length TP2 of the second peak P2, the time length TP3 of the third peak P3, and the time length TP4 of the fourth peak P4 are the time lengths of the vowel sections having the mora numbers 2, 3, and 4, respectively. Is approximately the same.

  As described above, when four peaks are detected from the frequency distribution of the time length of the vowel section, the reference time length used when calculating the number of mora of each vowel section is calculated by the following equation (5), for example. The average time length d_ave may be used.

  In Equation (5), dj (j = 1, 2, 3) is a time difference between the time length TPj + 1 of the j + 1-th peak Pj + 1 and the time length TPj of the j-th peak Pj.

  When the average time length d_ave calculated by the equation (5) is used as the reference time length, the mora number VN (i) of each vowel section is calculated by the following equation (6).

  The time length TP1 of the first peak P1 in the frequency distribution is determined by the time length of a vowel section in one syllable containing only vowels and the time length of a vowel section in one syllable including consonants as shown in FIG. The time length of a vowel section in one syllable including consonants occupies 70% or more of the time length of one syllable as described above, but is shorter than the time length of one syllable. Therefore, as shown in FIG. 4B, when the vowel section of the syllable including the consonant is made longer by one syllable, the time length of the vowel section becomes shorter than the time length of two syllables.

  On the other hand, the time differences d1, d2, and d3 in the equation (5) substantially coincide with the time length corresponding to one syllable. Therefore, compared to the case where the mora number VN (i) of each vowel section is calculated based only on the time length TP1 of the first peak P1, for example, the mora number VN (i) at the boundary portion of the determination condition is correctly set. It is possible to calculate.

  The speech speed estimation apparatus 1 according to the present embodiment can be applied to estimation of a speaker's speech speed in a call system using a network such as a telephone network, for example.

FIG. 9 is a diagram illustrating a first application example of the speech speed estimation apparatus.
As shown in FIG. 9, the call system 10 is used for a call between a first speaker 9A and a second speaker 9B. The first speaker 9A is a communication device (telephone) obtained by connecting the sound collection device 2, the display device 3, and the receiver 13 to the information processing device 12 including the speech speed estimation device 1 and the call processing device 11. Used as

  When the first speaker 9A makes a call using the call device (information processing device 12), the call processing device 11 is connected to the call device (the call device used by the call partner (via the first switch 14A and the network 15)). Connected to the telephone). When the communication partner of the first speaker 9A is the second speaker 9B, the call processing device 11 transmits the second speaker via the first switch 14A, the network 15, and the second switch 14B. 9B is connected to the telephone set 16 used.

  The call processing device 11 transmits the voice data including the voice of the first speaker 9 </ b> A acquired from the sound collection device 2 to the telephone set 16, and outputs the voice data received from the telephone set 16 to the receiver 13. And do. On the other hand, the call processing unit 1610 of the telephone set 16 transmits the voice data including the voice of the second speaker 9B acquired from the sound collection device 1620 to the call processing device 11 (information processing device 12), The audio data received from the processing device 11 is output to the receiver 1630.

  While the first speaker 9A and the second speaker 9B are talking, the speech speed estimation device 1 included in the information processing device 12 includes the sound of the first speaker 9A from the sound collection device 2. Voice data is acquired, and the speaking speed of the first speaker 9A is calculated according to the flowcharts of FIGS. Further, the speech speed estimation device 1 generates display data for visualizing the calculated speech speed and outputs the display data to the display device 3. Therefore, the first speaker 9A can grasp whether or not his / her speaking speed is appropriate during a call with the second speaker 9B, and can adjust the speaking speed. Thereby, for example, the first speaker 9A can be guided to an utterance at a speaking speed at which the second speaker 9B can easily hear the utterance content.

  The information processing device 12 is not limited to a device that includes two devices, the speech speed estimation device 1 and the call processing device 11, but a first processing unit that performs processing performed by the speech speed estimation device 1, It may be one device including a second processing unit responsible for processing performed by the call processing device 11.

  Further, the call system 10 may be, for example, a telephone conference system using the network 15 or a video conference system.

FIG. 10 is a diagram illustrating a second application example of the speech speed estimation apparatus.
The call system 10 shown in FIG. 10 is used for a call between the first speaker 9A and the second speaker 9B. The first speaker 9 </ b> A uses, as a call device (telephone), a sound collecting device 2 and a receiver 13 connected to an information processing device 12 including a call processing unit 21 corresponding to the call processing device 11. At this time, the speech speed estimation apparatus 1 is connected to the information processing apparatus 12, and the display apparatus 3 is connected to the speech speed estimation apparatus 1.

  When the first speaker 9A makes a call using the call device, the call processing unit 21 of the information processing device 12 uses the first switch 14A and the network 15 to call the call device (telephone) used by the other party. ). When the communication partner of the first speaker 9A is the second speaker 9B, the call processing unit 21 of the information processing apparatus 12 passes through the first switch 14A, the network 15, and the second switch 14B. It is connected to the telephone 16 used by the second speaker 9B.

  The call processing unit 21 of the information processing device 12 receives the voice data including the voice of the first speaker 9A acquired from the sound pickup device 2 to the telephone set 16 and the voice data received from the telephone set 16 as a receiver. 13 is output. Further, the call processing unit 21 of the information processing device 12 outputs the voice data acquired from the sound collection device 2 to the speech speed estimation device 1. On the other hand, the telephone call processing unit 1610 of the telephone set 16 receives the voice data including the voice of the second speaker 9B acquired from the sound pickup device 1620 toward the information processing device 12, and the information received from the information processing device 12. Audio data is output to the receiver 1630.

  While the first speaker 9A and the second speaker 9B are talking, the speech speed estimation device 1 acquires voice data including the voice of the first speaker 9A via the information processing device 12. The speech speed of the first speaker 9A is calculated according to the flowcharts of FIGS. Further, the speech speed estimation device 1 generates display data for visualizing the calculated speech speed and outputs the display data to the display device 3. Thus, the first speaker 9A can grasp whether or not his / her speaking speed is appropriate during a call with the second speaker 9B and can adjust the speaking speed.

  9 and 10 is not limited to the vicinity of the first speaker 9A, for example, the display device 3 is installed in a place different from the first speaker 9A. It is also possible to install in the vicinity. Furthermore, the display device 3 in the call system 10 may be plural.

  In addition, the speech speed of the first speaker 9A calculated (estimated) by the speech speed estimation device 1 may be stored in, for example, a storage unit of the speech speed estimation device 1 (not shown) or another device.

  Note that the speech speed estimation device 1 described in the present embodiment is a speech processing device that controls input speech data and an output signal corresponding to each of a plurality of vowel intervals determined by the number of moras determination unit 140. It is only an example. That is, the mora number of each of the plurality of vowel intervals determined by the mora number determination unit 140 according to the present embodiment controls not only the display data (output signal) indicating the speech speed of the input voice data but also the input data and It can also be used to control other corresponding output signals.

Furthermore, the functional configuration of the speech speed estimation apparatus 1 according to the present embodiment is not limited to the configuration illustrated in FIG. 1, and can be changed as appropriate according to the content of processing performed by the speech speed estimation apparatus 1. For example, the processes in steps S3, S4, and S5 in FIG. 2 can be replaced with steps S3 ′, S4 ′, and S5 ′, respectively.
(Step S3 ′) A process of calculating the time length of each of the plurality of vowel sections.
(Step S4 ′) A process of calculating the frequency distribution for the time lengths of the plurality of vowel sections and specifying the minimum value of the time lengths of the plurality of vowel sections.
(Step S5 ′) A process of calculating the number of mora corresponding to the time length of each of the plurality of vowel sections based on the specified minimum value of the time length and the frequency distribution.

  When the processing of steps S3 ′ to S5 ′ is performed in the speech speed processing device 1, the mora number determination unit 140 of the speech speed estimation device 1 includes a time length calculation unit, a minimum value specifying unit, and a mora number calculation unit. It may be included. In this case, the time length calculation unit performs the process of step S3 ', and the minimum value specifying unit performs the process of step S4'. Further, the mora number calculation unit performs the process of step S5 '.

[Second Embodiment]
FIG. 11 is a diagram illustrating a functional configuration of the speech speed estimation apparatus according to the second embodiment.

  As shown in FIG. 11, the speech speed estimation apparatus 1 according to the present embodiment includes a speech acquisition unit 110, a speech segment detection unit 120, a vowel segment detection unit 130, a mora number determination unit 140, and a speech speed calculation unit. 150 and an output unit 160. Although omitted in FIG. 11, the speech speed estimation apparatus 1 includes a storage unit that stores various types of information including voice data.

The sound acquisition unit 110 acquires sound data from the sound collection device 2.
The utterance section detection unit 120 detects the utterance section in the voice data. The utterance section is a section including a voice uttered by a person who is a target of speech speed estimation.

  The vowel section detector 130 detects a vowel section included in the speech section of the voice data. In addition, the vowel section detection unit 130 in the speech speed estimation apparatus 1 according to the present embodiment detects a vowel section from the entire speech data, not from the utterance section detected by the utterance section detection unit 120.

  The mora number determination unit 140 calculates the number of mora in each vowel section based on the time length in each of the plurality of vowel sections detected from the speech data. A frequency distribution calculating unit 141, a peak detecting unit 142, and a mora number calculating unit 143 are included.

  The speech speed calculation unit 150 calculates the speech speed in the speech data based on the time length of the speech section and the number of mora in the vowel section included in the speech section.

  The output unit 160 generates display data for visualizing the calculated speech speed and outputs the display data to the display device 3.

  When the operation is started, the speech speed estimation apparatus 1 of the present embodiment acquires voice data from the sound collection apparatus 2, and processes to estimate the speech speed in the acquired voice data and output the estimation result to the display apparatus 3. I do. The voice acquisition unit 110 of the speech speed estimation apparatus 1 performs the process of acquiring the voice data.

  On the other hand, the process of estimating the speech speed in the speech data and outputting the estimation result to the display device 3 includes the speech segment detection unit 120, the vowel segment detection unit 130, the mora number determination unit 140, and the speech speed calculation unit of the speech speed estimation device 1. 150 and the output unit 160. The speech speed estimation apparatus 1 according to the present embodiment performs the processes of steps S1 to S7 in FIG. 2 as a process of estimating the speech speed in voice data and outputting the estimation result to the display device 3. In addition, the vowel section detection unit 130 in the speech speed estimation apparatus 1 of the present embodiment detects a vowel section from the entire speech data without referring to the detection result of the utterance section by the utterance section detection unit 120. That is, the vowel section detection unit 130 in the speech speed estimation apparatus 1 of the present embodiment performs, for example, the process shown in FIG. 12 as the process of detecting the vowel section in step S2.

  FIG. 12 is a flowchart for explaining the contents of processing for detecting a vowel section according to the second embodiment.

The vowel section detection unit 130 in the speech speed estimation apparatus 1 of the present embodiment divides the speech data into a plurality of sections, and performs the processing of steps S211 to S219 shown in FIG. 12 for each section. First, the vowel section detection unit 130 performs waveform autocorrelation AC (t _m ) of the voice data at each time t _m (m = 1, 2,..., M) in the processing target section of the voice data. Is calculated (step S211). The time interval (t _m −t _m−1 ) at time t _m in the utterance section is, for example, the time length of a processing unit (frame) when processing voice data. In the following description, a frame associated with time t _m in the audio data is referred to as a frame at time t _m .

In the process of step S211, the vowel section detection unit 130 calculates the waveform autocorrelation AC (t) at time t (each time t _m ) according to the following equation (7).

  N in Expression (7) is a calculation width (number of samples) of the waveform autocorrelation, and for example, N = 500. SL in Expression (7) is a lower limit value (number of samples) of the search range of the waveform autocorrelation, for example, SL = 20. SH in Expression (7) is an upper limit value (number of samples) of the search range of the waveform autocorrelation. For example, SH = 120.

When the processing of step S211 is completed, the vowel section detection unit 130 next sets the variable m, the variable i, and the waveform autocorrelation AC (t ₀ ) to m = 1, i = 1, and AC (t ₀ , respectively. ) = 0 (step S212). The variable i is a value that identifies a vowel section.

Next, the vowel section detection unit 130 has a waveform autocorrelation AC (t _m−1 ) at time t _m−1 smaller than the threshold TH _AC and a self waveform correlation AC (t _m ) at time t _m has a threshold TH. It is determined whether or not it is _AC or more (step S213). That is, in step S213, it is determined whether AC (t _m−1 ) <TH _AC and AC (t _m ) ≧ TH _AC .

The threshold TH _AC is a value for determining whether the voice included in the voice data is a vowel or a non-vowel (consonant, etc.). When the waveform autocorrelation is calculated by the equation (7), the waveform autocorrelation of the vowel section included in the speech data is a larger value than the waveform autocorrelation of the non-vowel section (consonant section etc.). For this reason, in this embodiment, the section in which the waveform autocorrelation in the voice data is equal to or greater than the threshold TH _AC is set as the vowel section. The threshold TH _AC is set based on, for example, the statistical value of the waveform autocorrelation in the vowel section and the statistical value of the waveform autocorrelation in the non-vowel section.

When AC (t _m−1 ) <TH _AC and AC (t _m ) ≧ TH _AC , the waveform autocorrelation is between a value smaller than the threshold TH _{AC and a} threshold TH between time t _m−1 and t _m. The value changes to _AC or higher. That is, when AC (t _m−1 ) <TH _AC and AC (t _m ) ≧ TH _AC , the frame at time t _m−1 is included in the non-vowel section, and the frame at time t _m is included in the vowel section. included. Therefore, when AC (t _m-1 ) <TH _AC and AC (t _m ) ≧ TH _AC (step S213; YES), the vowel section detection unit 130 then starts the i-th vowel section start time. Ts (i) is set at time t _m (step S214). When the processing of step S214 is completed, the vowel section detection unit 130 next determines whether or not the variable m is equal to or greater than the last value M in the section of waveform data in which the waveform autocorrelation is calculated (step S217). ).

On the other hand, when AC (t _m−1 ) ≧ TH _AC or AC (t _m ) <TH _AC (step S213; NO), the vowel section detection unit 130 then determines that AC (t _m−1 ) ≧ TH _AC, and _AC (t m) <determines whether the _{TH AC} (step S215).

When AC (t _m−1 ) ≧ TH _AC and AC (t _m ) <TH _AC (step S215; YES), the frame at time t _m−1 is included in the vowel section, and the frame at time t _m is Included in non-vowel section. Therefore, when AC (t _m−1 ) ≧ TH _AC and AC (t _m ) <TH _AC (step S215; YES), the vowel section detection unit 130 next ends the end time of the i-th vowel section. Te (i) is set at time t _m−1 and the variable i is updated to i + 1 (step S216). In step S216, the vowel section detection unit 130 sets the end time Te (i) of the vowel section to the time t _m−1 and then updates the variable i to i + 1. When the process of step S216 is completed, the vowel section detection unit 130 next performs the determination of step S217.

On the other hand, when AC (t _m-1 ) ≧ TH _AC and AC (t _m ) <TH _AC is not satisfied (step S215; NO), the waveform autocorrelation at time t _m-1 and time t _m is either both the threshold value _{TH AC} above, or both smaller than the threshold value _{TH AC.} That is, if AC (t _m−1 ) ≧ TH _AC and AC (t _m ) <TH _AC are not satisfied, the frames at time t _m−1 and t _m are both included in the non-vowel interval, Alternatively, both are included in the vowel section. Therefore, when AC (t _m-1 ) ≧ TH _AC and AC (t _m ) <TH _AC are not satisfied (step S215; NO), the vowel section detection unit 130 skips the process of step S216, and next In step S217, the determination is made.

  In step S217, as described above, the vowel section detection unit 130 determines whether or not the variable m is equal to or greater than the last value M in the section that is the current processing target in the speech data. When m <M (step S217; NO), the vowel section detection unit 130 updates the variable m to m + 1 (step S218), and performs the processing after step S213.

  On the other hand, if m ≧ M (step S217; YES), the vowel section detection unit 130 outputs information [Ts (i), Te (i)] indicating the vowel section (step S219), and 1 in the voice data The process of detecting a vowel section included in each processing target section is terminated.

  The vowel section detection unit 130 according to the present embodiment performs the processes of steps S211 to S219 for each processing target section in the voice data.

  The speech speed estimation apparatus 1 according to the present embodiment is based on the utterance interval detected by the utterance interval detection unit 120 and the vowel interval detected by the vowel interval detection unit 130 by the processing of steps S211 to S219. The processing from S3 to S6 is performed to calculate the speech speed. That is, the speech speed estimation apparatus 1 according to the present embodiment detects the time length of a vowel section of 1 mora (single vowel) based on the frequency distribution of the time length of the vowel section included in the speech data. Thereafter, the speech speed estimation apparatus 1 determines the number of mora in each vowel section based on the time length of the vowel section of one mora and the time length of the vowel section. For this reason, in the speech speed estimation apparatus 1 according to the present embodiment, it is possible to prevent an error in the number of mora for a long vowel vowel section and to calculate a correct speech speed of the utterance section. Therefore, for example, the speech speed estimation apparatus 1 can present the time change of the correct speech speed to the user who is the target of speech speed estimation and guide the user to speak at an appropriate speech speed. It becomes possible.

  Moreover, the speech speed estimation apparatus 1 of the present embodiment generates display data for visualizing the calculated speech speed and outputs the display data to the display apparatus 3, for example. At this time, the speech speed estimation apparatus 1 generates display data such as a graph 421 shown in FIG. As a result, it is possible to present the speaking speed during the conversation to the person whose speech speed is to be estimated, and to lead to an appropriate speaking speed.

  Note that the flowchart of FIG. 12 is merely an example of processing for detecting a vowel section. In the process of detecting a vowel section according to the present embodiment, the vowel section may be detected based not only on the time change of the waveform autocorrelation but also on the time change of other characteristics in the speech data. For example, in the process of detecting the vowel section, the vowel section may be detected based on the time change of the signal-to-noise ratio described in the first embodiment. Further, for example, in the process of detecting the vowel section, the vowel section may be detected based on the time change of the formant frequency as shown in FIG.

  FIG. 13 is a flowchart for explaining a modification of the process for detecting a vowel section according to the second embodiment.

When detecting a vowel section based on the formant frequency, the vowel section detection unit 130 firstly at each of the times t _m (m = 1, 2,..., M) in the processing target section of the speech data. The formant frequency is calculated (step S221). The time interval (t _m −t _m−1 ) at time t _m in the utterance section is, for example, the time length of a processing unit (frame) when processing voice data. In the following description, a frame associated with time t _m in the audio data is referred to as a frame at time t _m .

In the process of step S211, the vowel section detection unit 130 calculates, for example, the formant frequency FM (t _m , k) in the frame at time t _m of the audio data according to a known calculation method. For example, the vowel section detection unit 130 calculates a first formant frequency FM (t _m , 1), a second formant frequency FM (t _m , 2), and a third formant frequency FM (t _m , 3).

When the process of step S221 is completed, the vowel section detection unit 130 then sets each value of the variable m, the variable i, the formant frequency, and the time-varying average of the formant frequency as initial values (step S222). In step S222, the vowel section detection unit 130 sets the variables m and i to m = 1 and i = 1, respectively. The variable i is a value that identifies a vowel section. Further, the vowel section detection unit 130 sets the initial value of the formant frequency to, for example, FM (t ₀ , 1) = FM (t ₀ , 2) = FM (t ₀ , 3) = 0. Furthermore, the vowel section detection unit 130 sets the initial value ΔFM (t ₀ ) of the time change average of the formant frequency to a value that satisfies ΔFM (t ₀ ) ≧ TH _FM . The threshold value TH _FM is a value for determining whether the voice included in the voice data is a vowel or a non-vowel (consonant).

Next, the vowel section detection unit 130 calculates the time change average ΔFM (t _m ) of the formant frequency from the time t _m−1 to t _m (step S223). The vowel section detection unit 130 calculates, for example, the time change average ΔFM (t _m ) of the formant frequency by the following equation (8).

Next, the vowel section detection unit 130 has a time change average ΔFM (t _m−1 ) at time t _m−1 that is equal to or greater than a threshold TH _FM and a time change average ΔFM (t _m ) at time t _m. It is determined whether it is smaller than the threshold value TH _FM (step S224). The threshold value TH _FM is a value for determining whether the voice included in the voice data is a vowel or a non-vowel (consonant). The characteristics of the vowel section in the speech data are mainly determined by the distribution of the first formant frequency and the second formant frequency. In other words, the time change average ΔFM of the formant frequency in one vowel section of the speech data is a substantially constant value, and is smaller than the time change average ΔFM in the non-vowel section. For this reason, it is possible to regard a section in which continuous frames having a time change average ΔFM of formant frequency smaller than the threshold value TH _FM in the voice data as vowel sections. The threshold value TH _FM is set based on, for example, the statistical value of the time change average of the formant frequency in the vowel interval and the statistical value of the time change average of the formant frequency in the non-vowel interval obtained by statistical processing.

If ΔFM (t _m−1 ) ≧ TH _FM and ΔFM (t _m ) <TH _FM (step S224; YES), the vowel section detection unit 130 then starts the i-th vowel section start time Ts ( i) is set to the time t _m−1 (step S225). When the process of step S225 is completed, the vowel section detection unit 130 next determines whether or not the variable m is equal to or greater than the last value M in the section in which the formant frequency of the speech data is calculated (step S228). .

On the other hand, if ΔFM (t _m−1 ) <TH _FM or ΔFM (t _m ) ≧ TH _FM (step S224; NO), the vowel section detection unit 130 then sets ΔFM (t _m−1 ) < TH _FM, and determines whether the _{ΔFM (t m) ≧ TH FM} ( step S226).

When ΔFM (t _m−1 ) <TH _FM and ΔFM (t _m ) ≧ TH _FM (step S226; YES), the vowel section detection unit 130 then ends the ith vowel section end time Te ( i) is set at time t _m−1 and the variable i is updated to i + 1 (step S227). When the process of step S227 is completed, the vowel section detection unit 130 next performs the determination of step S228.

On the other hand, when ΔFM (t _m−1 ) <TH _FM and ΔFM (t _m ) ≧ TH _FM are not satisfied, the time change averages of the formant frequencies at time t _m−1 and time t _m are both It is greater than or equal to the threshold TH _FM , or both are smaller than the threshold TH _FM . That is, when ΔFM (t _m−1 ) <TH _FM and ΔFM (t _m ) ≧ TH _FM are not satisfied, both frames at time t _m−1 and t _m in the speech data are included in the non-vowel section. Or both are included in the vowel section. Therefore, when ΔFM (t _m−1 ) <TH _FM and ΔFM (t _m ) ≧ TH _FM are not satisfied (step S226; NO), the vowel section detection unit 130 skips the process of step S227, Next, determination of step S228 is performed.

  As described above, in step S228, the vowel section detection unit 130 determines whether or not the variable m is equal to or greater than the last value M in the section in which the formant frequency is calculated in the speech data. When m <M (step S228; NO), the vowel section detection unit 130 updates the variable m to m + 1 (step S229), and performs the processing after step S223.

  On the other hand, if m ≧ M (step S228; YES), the vowel section detection unit 130 outputs information [Ts (i), Te (i)] indicating the vowel section (step S230), and 1 in the speech data The process of detecting a vowel section for each processing target section is terminated.

  As described above, the vowel section in the voice data can be detected based on the time-change average of the formant frequency including the first formant frequency and the second formant frequency. Even when a vowel section is detected based on the time-change average of the formant frequency, a vowel section that is actually 2 mora, for example, may be detected as one vowel section by converting the vowel into a long vowel. . However, by determining the number of vowels in each vowel section based on the frequency distribution of the time length of the vowel section included in the speech data, the number of mora (number of vowels) in the speech section in the speech data is correctly calculated, and the speech speed Can be calculated with high accuracy.

  Note that the speech speed estimation device 1 described in the present embodiment is a speech processing device that controls input speech data and an output signal corresponding to each of a plurality of vowel intervals determined by the number of moras determination unit 140. It is only an example. That is, the mora number of each of the plurality of vowel intervals determined by the mora number determination unit 140 according to the present embodiment controls not only the display data (output signal) indicating the speech speed of the input voice data but also the input data and It can also be used to control other corresponding output signals.

  Further, the mora number determination unit 140 of the speech speed estimation apparatus 1 according to the present embodiment includes a time length calculation unit that performs the process of step S3 ′ described above, a minimum value specification unit that performs a process of step S4 ′, and step S5. It may include a mora number determination unit that performs the process'.

[Third Embodiment]
FIG. 14 is a diagram illustrating a system configuration of a call system according to the third embodiment.

  As shown in FIG. 14, the call system 10 according to this embodiment is used for a call between a first speaker 9A and a second speaker 9B. The first speaker 9A uses the mobile phone terminal 25 as a call device (telephone). The mobile phone terminal 25 includes a call processing unit 21, a sound collection device 2, a receiver 13, and a speech speed adjustment unit 26.

  When the first speaker 9 </ b> A makes a call using the mobile phone terminal 25, the call processing unit 21 is connected to a call device (telephone) used by the call partner via the base station 30 and the network 15. . The mobile phone terminal 25 (call processing unit 21) and the base station 30 are connected by wireless communication in accordance with a predetermined wireless communication standard. When the other party of the first speaker 9A is the second speaker 9B, the call processing unit 21 uses the telephone set used by the second speaker 9B via the base station 30, the network 15, and the exchange 14B. 16 is connected.

  The cellular phone terminal 25 adjusts the speech speed of the voice data including the voice of the first speaker 9 </ b> A acquired from the sound collection device 2 and transmits the voice data to the telephone set 16, and the voice data received from the telephone set 16. Processing to output to the receiver 13 is performed. The speech speed adjustment unit 26 performs processing for adjusting the speech speed of the voice data in the mobile phone terminal 25. In the cellular phone terminal 25, the call processing unit 21 performs processing for transmitting voice data with adjusted speech speed to the telephone set 16 and processing for outputting voice data received from the telephone set 16 to the receiver 13.

  On the other hand, the telephone set 16 used by the second speaker 9B includes a call processing unit 1601, a sound collection device 1620, and a receiver 1630. The telephone call processing unit 1610 of the telephone 16 transmits the voice data including the voice of the second speaker 9B acquired from the sound pickup device 1620 to the mobile phone terminal 25, and the voice data received from the mobile phone terminal 25. Are output to the receiver 1630.

  As described above, the cellular phone terminal 25 in the call system according to the present embodiment adjusts the speech speed of the speech data including the speech of the first speaker 9 </ b> A acquired from the sound collection device 2 in the speech speed adjustment unit 26. Perform the process. The speech speed adjustment unit 26 calculates the speech speed of the first speaker 9A in the speech data, and then based on the speech speed calculation result, the speech data is adjusted so that the speech speed in the speech data becomes an appropriate speech speed. adjust. In the present embodiment, the speech speed adjustment unit 26 decompresses the speech data to reduce the speech speed when the speech speed in the speech data is equal to or higher than the threshold value.

FIG. 15 is a diagram illustrating a functional configuration of the speech speed adjustment unit according to the third embodiment.
As shown in FIG. 15, the speech speed adjustment unit 26 in the mobile phone terminal 25 of the present embodiment includes a voice acquisition unit 110, a speech segment detection unit 120, a vowel segment detection unit 130, a mora number determination unit 140, A speech speed calculation unit 150, a speech speed control unit 170, and an output unit 162 are provided. The speech speed adjustment unit 26 includes a storage unit 190 that stores various types of information including the audio data 191 and the target expansion rate 192. The target expansion rate 192 is a target value for the expansion rate of the audio data when the speech speed in the audio data is reduced.

The sound acquisition unit 110 acquires sound data from the sound collection device 2.
The utterance section detection unit 120 detects the utterance section in the voice data. The utterance section is a section including a voice uttered by a person who is a target of speech speed estimation.

  The vowel section detector 130 detects a vowel section included in the speech section of the voice data. Note that the vowel segment detection unit 130 in the speech speed adjustment unit 26 of the present embodiment detects a vowel segment from the entire speech data, not from the utterance segment detected by the utterance segment detection unit 120.

  The mora number determination unit 140 calculates the mora number of each vowel section based on the time lengths of the plurality of vowel sections detected from the speech data. The mora number determination unit 140 calculates the mora number of each vowel section by the method described in the first embodiment. The mora number determination unit 140 includes a frequency distribution calculation unit 141, a peak detection unit 142, and a mora number calculation unit 143.

  The speech speed calculation unit 150 calculates the speech speed (speech speed) in the speech data based on the time length of the speech section and the number of mora of the vowel section included in the speech section.

  The speech speed control unit 170 controls the speech speed of the voice data based on the speech speed calculated by the speech speed calculation unit 150. When the calculated speech speed is faster than the appropriate speech speed, the speech speed control unit 170 according to the present embodiment refers to the target expansion rate 192 so that the speech speed of the speech data becomes the appropriate speech speed. Decompress data.

  The output unit 162 outputs the voice data whose speech speed is controlled by the speech speed control unit 170 to the call processing unit 21.

FIG. 16 is a diagram illustrating a functional configuration of the speech speed control unit.
As shown in FIG. 16, the speech rate control unit 170 according to the present embodiment includes a basic period detection unit 171, a waveform processing unit 172, and an extension control unit 173.

  The fundamental period detector 171 detects the fundamental period of the vowel section that is the processing target in the speech data.

The waveform processing unit 172 extends the audio data by superimposing the waveform of the basic period on the audio data.
The decompression control unit 173 controls whether or not the speech data is decompressed, in other words, whether or not the waveform of the basic period is overlapped. First, the expansion control unit 173 determines whether or not to expand the voice data based on the speech speed calculated by the speech speed calculation unit 150. Further, when expanding the audio data, the expansion control unit 173 further overlaps the waveform of the basic period based on the actual expansion rate of the audio data obtained by superimposing the waveform of the basic period by the waveform processing unit 172 and the target expansion rate. Control whether or not.

  The speech speed adjustment unit 26 in the mobile phone terminal 25 of the present embodiment performs a process of acquiring voice data from the sound collection device 2 and a process of controlling the speech speed in the acquired voice data and outputting it to the call processing unit 21. Do. The voice acquisition unit 110 of the speech speed adjustment unit 26 performs the process of acquiring the voice data.

  On the other hand, the process of controlling and outputting the speech speed in the speech data includes the speech interval detection unit 120, the vowel interval detection unit 130, the mora number determination unit 140, the speech speed calculation unit 150, and the speech speed control unit of the speech speed adjustment unit 26. 170 and the output unit 162. The speech speed adjustment unit 26 performs the process shown in FIG. 17 as a process of controlling and outputting the speech speed in the voice data.

  FIG. 17 is a flowchart illustrating processing performed by the speech speed adjustment unit according to the third embodiment.

  In the process of controlling and outputting the speech speed in the speech data, the speech speed adjusting unit 26 first detects a speech section included in the speech data (step S1). The processing of step S1 is performed by the utterance section detection unit 120. The utterance section detection unit 120 detects an utterance section (in other words, a section including speech uttered by a person who is a target of speech speed estimation) included in the sound data according to a known detection method. For example, the utterance section detection unit 120 detects the utterance section by VAD.

  Next, the speech speed adjustment unit 26 detects a vowel section included in the utterance section (step S2). The process of step S2 is performed by the vowel section detection unit 130. The vowel section detection unit 130 detects a vowel section in the voice data according to a known detection method. For example, the vowel section detection unit 130 detects, as one vowel section, one section in which the signal-to-noise ratio is continuous at a predetermined threshold or more based on the time change of the signal-to-noise ratio in the speech data.

  Next, the speech speed adjustment unit 26 performs processing (steps S3 to S5) for determining the number of mora (number of vowels) in the detected vowel section. The process of steps S3 to S5 for determining the number of mora in the vowel section is performed by the mora number determining unit 140 of the speech speed adjusting unit 26.

  The mora number determination unit 140 first calculates a frequency distribution for the time length of the vowel section based on the detected time lengths of the plurality of vowel sections (step S3). The processing in step S3 is performed by the frequency distribution calculation unit 141 of the mora number determination unit 140. The frequency distribution calculation unit 141 calculates the time length of the vowel section based on the start time and end time of each of the detected plurality of vowel sections. Further, the frequency distribution calculation unit 141 calculates the frequency distribution by counting the appearance frequency of the vowel section for each time length based on the time length of each vowel section. At this time, the frequency distribution calculation unit 141 calculates the frequency distribution by excluding, for example, the last vowel section in one utterance section.

  Next, the mora number determination unit 140 detects a time length at which the appearance frequency becomes a maximum value (peak) in the frequency distribution calculated in step S3 (step S4). The processing in step S4 is performed by the peak detection unit 142 of the mora number determination unit 140. For example, the peak detection unit 142 compares the appearance frequency of the time length that is the determination target with the appearance frequency of the time length before and after the appearance frequency in order from the appearance frequency of the shortest time length in the frequency distribution, and the appearance frequency becomes the maximum value. Detect time length.

  Next, the mora number determination unit 140 calculates the number of mora in each vowel section based on the minimum value of the time lengths detected from the frequency distribution and the time length of the vowel sections (step S5). The process of step S5 is performed by the mora number calculation unit 143 of the mora number determination unit 140. The mora number calculation unit 143 sets a minimum value among a plurality of time lengths peaking in the frequency distribution as a reference time length, and first calculates a value obtained by dividing the time length of the vowel section by the reference time length. Thereafter, the mora number calculation unit 143 sets an integer value close to a value obtained by dividing the time length of the vowel section by the reference time length as the mora number of the vowel section.

  When the processing of steps S3 to S5 by the mora number determination unit 140 is completed, the speech speed adjustment unit 26 next utters based on the mora number of each vowel section calculated in step S5 and the time length of the utterance section. The speech speed of the section is calculated (step S6). The speech speed calculation unit 150 performs the process of step S6. The speech speed calculation unit 150 calculates, as the speech speed, a value (mora / second) obtained by dividing the time length of the utterance section by the total number of mora for the vowel section included in the utterance section.

  Next, the speech speed adjustment unit 26 performs processing (step S8) for controlling the speech speed of the voice data based on the calculated speech speed. The speech speed control unit 170 performs the process in step S8. The speech speed control unit 170 performs a process of extending the speech data by superimposing the waveform of the basic period in the corresponding section on a section in which the calculated speech speed is faster than the appropriate speech speed.

  Next, the speech speed adjustment unit 26 sets the speech data whose speech speed has been adjusted by the speech speed control unit 170 to the call processing unit 21 (step S9). The output unit 162 performs the process of step S9.

  The speech speed adjustment unit 26 repeatedly performs the processes of steps S1 to S6, S8, and S9 described above during a call. At this time, the speech speed adjustment unit 26 may perform the processing for the next processing target section after finishing the processes of steps S1 to SS6, S8, and S9 for one processing target section in the voice data. The step processing may be performed in a pipeline.

  As described above, the speech speed adjustment unit 26 in the mobile terminal device 1 according to the present embodiment is based on the frequency distribution of the time length of the vowel section in the speech data, and the vowel section corresponding to 1 mora (single vowel). The time length is calculated and the number of vowels in each vowel section is determined. For this reason, the speech speed adjusting unit 26 can prevent an error in the speech speed due to a long vowel and can calculate (estimate) the speech speed with high accuracy, and adjust the speech speed in the speech data to an appropriate speech speed. It becomes possible.

  The speech speed control unit 170 in the speech speed adjustment unit 26 of the present embodiment performs, for example, the process shown in FIG. 18 as a process (step S8) for controlling the speech speed of the voice data.

FIG. 18 is a flowchart for explaining the contents of processing for controlling the speech speed of voice data.
The speech speed control unit 170 performs the processing of steps S801 to S807 in FIG. 18 for each section (for example, speech section) where the speech speed in the voice data is calculated. The speech speed control unit 170 first determines whether or not the speech speed of the processing target section is equal to or higher than a threshold based on the speech speed calculated by the speech speed calculation unit 150 (step S801). The determination in step S801 is performed by, for example, the extension control unit 173 of the speech speed control unit 170. When the speaking speed is smaller (slower) than the threshold (step S801; NO), the speaking speed control unit 170 ends the process of controlling the speaking speed for the processing target section. The threshold of the speech speed is, for example, 8 mora / second.

  When the speech speed of the processing target section is equal to or higher than the threshold (step S801; YES), the speech speed control unit 170 then initializes the actual expansion rate for the processing target section (step S802). The extension control unit 173 of the speech speed control unit 170 performs the process in step S802.

  Thereafter, the speech speed control unit 170 performs the processes of steps S803 to S807 for expanding the processing target section in the voice data. For example, the processing in steps S803 to S807 is frame processing in which one frame period is 20 milliseconds.

  In the process of extending the processing target section in the voice data, the speech speed control unit 170 first detects the basic period of the vowel section in the processing target section (step S803). The basic cycle detection unit 171 of the speech speed control unit 170 performs the process of step S803. The basic period detection unit 171 calculates the autocorrelation for the speech waveform in the processing target section according to a known detection method, and calculates a period corresponding to the shift amount at which the autocorrelation is maximized for the first time in the section where the shift amount is greater than zero. And calculated as a basic period.

  Next, the speech speed control unit 170 calculates a time change rate of the pitch based on the calculated basic period (step S804). For example, the extension control unit 173 of the speech speed control unit 170 performs the process in step S804. The extension control unit 173 calculates the time change rate of the pitch according to a known calculation method.

  Next, the speech speed control unit 170 determines whether or not the actual expansion rate is smaller than the target expansion rate (step S805). The determination process in step S805 is performed by, for example, the expansion control unit 173 of the speech speed control unit 170. The expansion control unit 173 determines whether or not the current actual expansion rate is smaller than the target expansion rate 192 with reference to the target expansion rate 192 of the storage unit 190.

  When the actual expansion rate is smaller than the target expansion rate (step S805; YES), the speech speed control unit 170 superimposes the speech waveform of the basic period on the processing target section of the audio data (step S806). The processing in step S806 is performed by the waveform processing unit 172 of the speech speed control unit 170. The waveform processing unit 172 superimposes the speech waveform of the basic period on the processing target section of the speech data according to a known method.

  After the process of step S806, the speech speed control unit 170 updates the actual expansion rate (step S807), and performs the processes after step S803. The process of step S807 is performed by, for example, the extension control unit 173. For example, the expansion control unit 173 calculates the actual expansion rate rate_result (n) by the following equation (9).

  In Equation (9), s and n are the start frame and the current frame of the processing target section, respectively. M in Equation (9) is the number of samples in one frame, and for example, M = 160. In the equation (9), add (i) is the number of samples added in the i-th frame processing.

  On the other hand, in the determination process of step S805, when the actual expansion rate is equal to or higher than the target expansion rate (step S805; NO), the speech speed control unit 170 outputs the processing target section in the voice data and performs the processing for the processing target section. The process for controlling the speech speed is terminated. When the speech speed control unit 170 finishes the process for one process target section, the speech speed control unit 170 performs the processes after step S801 for the next process target section.

  In the flowchart of FIG. 18, the process is repeated until the actual expansion rate becomes equal to or higher than the target expansion rate. However, the process for controlling the speech speed is not limited to this. An upper limit value may be provided. That is, the processing for controlling the speech speed may be terminated even when the actual expansion rate is smaller than the target expansion rate when the loop processing after step S803 is performed a predetermined number of times. Thereby, for example, the processing becomes longer, and it becomes possible to prevent the deterioration of the call quality due to the delay of the voice data.

FIG. 19 is a diagram for explaining a basic period detection method.
In the process of detecting the basic period in step S803, the basic period is detected based on the autocorrelation of the speech waveform in the processing target section as described above. The autocorrelation for the speech waveform, for example, as shown by a curve 511 shown in FIG. 19, becomes a minimum value at a shift amount that gradually decreases as the shift amount increases from 0, and then becomes a maximum at the shift amount Sm. It becomes. The basic cycle detection unit 171 of the speech speed control unit 170 according to the present embodiment detects, as a basic cycle, a cycle corresponding to the shift amount Sm in which the autocorrelation becomes the maximum P for the first time in a section where the shift amount is greater than 0 ( calculate).

  FIG. 20 is a diagram for explaining a method of superimposing speech waveforms. FIG. 21 is a diagram for explaining a weighting method used when superimposing speech waveforms.

  In the process of superimposing the speech waveforms in step S806, the speech waveform of the basic period K in the process target section of the speech data is superimposed as described above. For example, it is assumed that the waveform x (t) in the processing target section is a waveform including speech waveforms 521, 522, and 523 having a basic period K as shown in FIG. When superimposing the speech waveform of the basic period K on this waveform x (t), the speech rate control unit 170 extracts the speech waveform 522 and between the speech waveform 522 and the speech waveform 523 in the waveform x (t). A waveform y (t) into which the speech waveform 522 is inserted is generated. As a result, the time length of the processing target section is increased by the basic period K, and the time length of the utterance section including the processing target section is increased by the basic period K. Therefore, a value (speech speed) obtained by dividing the number of vowels (number of mora) included in the utterance section by the time length of the utterance section becomes small.

  In addition, when superimposing speech waveforms, it is preferable to perform weighted addition in order to prevent deterioration in sound quality due to discontinuity of samples.

  For example, when superimposing the speech waveform 522 of the basic period K in the waveform x (t) of FIG. 21A, a first range 531 including a part of the speech waveform 523 behind the speech waveform 522, A second range 532 including a part of the voice waveform 521 is set in front of the voice waveform 522. Then, the waveform of the audio data is expanded by shifting the second range 532 in the positive time direction by the basic period K and combining the first range 531 and the second range 532.

  When combining the first range 531 and the second range 532 shifted by the basic period K, the end portion of the first range 531 and the start portion of the second range 532 overlap. To do. A portion where the first range 531 and the second range 532 overlap is weighted and added to the waveform (amplitude) in the first range 531 and the waveform in the second range 532.

  For example, as shown in FIG. 21B, the weighting factor w1 (t) for the first range 531 is set to w1 (t) = 1 in a section from time t to t1 that does not overlap with the second range 532. A section from time t1 to t2 that overlaps the second range 532 is set to w1 (t) = f (t). Here, the function f (t) is a function that monotonously decreases from f (t1) = 1 to f (t2) = 0 in the interval from time t1 to t2.

  In addition, the weighting factor w2 (t) for the second range 532 is set to w1 (t) = g (t) in the interval between times t1 and t2 that overlaps the first range 531 and does not overlap the first range 531. The section after t2 is set to w2 (t) = 1. Here, the function g (t) is a function that monotonically increases from g (t1) = 0 to g (t2) = 1 and satisfies f (t) + g (t) = 1 in the interval from time t1 to t2. To do.

  Waveform y (t) obtained by superimposing speech waveform 522 of basic period K using weighting factors w1 (t) and w2 (t) to waveform x (t) of speech data shown in FIG. Is calculated by the following equation (10).

    y (t) = w1 (t) .x (t) + w2 (t) .x (t + K) (10)

  As described above, the waveform (sample) becomes discontinuous at the boundary portion of the voice data obtained by extending the vowel interval by weighting and adding the waveforms at the portion that becomes the boundary when superimposing the speech waveforms of the basic period K. Can be prevented from deteriorating.

  As described above, the mobile phone terminal 25 of the present embodiment estimates the speech speed of the input voice data input from the sound collection device 2 of the own apparatus during a call, and sets the speech speed of the input voice data to an appropriate speech. It is possible to transmit to the other party's telephone with speed control. Further, in the speech speed adjustment unit 26 in the mobile phone terminal 25 of the present embodiment, as in the speech speed estimation device 1 described in the first embodiment, when estimating the speech speed, a vowel section included in the input speech data is used. The number of vowels in each vowel section is determined based on the frequency distribution for the time length of. For this reason, according to the present embodiment, it is possible to accurately estimate the speech speed in speech data, and it is possible to easily control (decelerate) the speech speed of speech data to an appropriate speech speed. . Further, when superimposing speech waveforms of the basic period K, it is possible to prevent deterioration of sound quality in speech data obtained by extending a vowel section by weighting and adding waveforms at the boundary portions.

  Note that the process of superimposing the speech waveforms of the basic period K is not limited to the method described with reference to FIGS. 20 and 21 and may be performed according to another method.

Moreover, the detection method used when the speech speed adjustment unit 26 detects the speech segment and the vowel segment may be any known detection method. For example, when detecting an utterance section and a vowel section, detection may be performed based on an autocorrelation coefficient AC (t _m ) in speech data or a time-change average ΔFM (t _m ) of a formant frequency.

  Furthermore, the processing performed by the speech speed adjustment unit 26 of the mobile phone terminal 25 according to the present embodiment is not limited to the flowchart of FIG. 17 and can be changed as appropriate. For example, the speech speed adjustment unit 26 may perform the above-described steps S3 'to S5' instead of the steps S3 to S5. When the speech speed adjustment unit 26 performs the processes of steps S3 ′ to S5 ′, the mora number determination unit 140 includes a time length calculation unit that performs the process of step S3 ′, and a minimum value identification unit that performs the process of step S4 ′. And a mora number determination unit that performs the process of step S5 ′.

  The speech speed estimation device 1 mentioned in the first embodiment and the second embodiment and the mobile phone terminal 25 mentioned in the third embodiment can be realized by a computer and a program executed by the computer. It is. Hereinafter, the speech speed estimation apparatus 1 realized by a computer and a program will be described with reference to FIG.

FIG. 22 is a diagram illustrating a hardware configuration of a computer.
As shown in FIG. 22, the computer 8 includes a processor 801, a main storage device 802, an auxiliary storage device 803, an input device 804, an output device 805, an input / output interface 806, a communication control device 807, and a medium. A driving device 808. These elements 801 to 808 in the computer 8 are connected to each other via a bus 810 so that data can be exchanged between the elements.

  The processor 801 is a central processing unit (CPU), a micro processing unit (MPU), or the like. The processor 801 controls the overall operation of the computer 8 by executing various programs including an operating system. For example, the processor 801 executes a sound processing program including the processes of steps S1 to S6 in FIG.

  The main storage device 802 includes a read only memory (ROM) and a random access memory (RAM) not shown. In the ROM of the main storage device 802, for example, a predetermined basic control program read by the processor 801 when the computer 8 is started is recorded in advance. On the other hand, the RAM of the main storage device 802 is used as a working storage area as needed when the processor 801 executes various programs. The RAM of the main storage device 802 can be used, for example, for holding (storing) a part of voice data, information indicating a speech segment and a vowel segment, a frequency distribution of time length of the vowel segment, a calculated speech speed, and the like.

  The auxiliary storage device 803 is a storage device having a larger capacity than the RAM of the main storage device 802, and includes, for example, a hard disk drive (HDD) and a non-volatile memory (Solid State Drive (SSD)) such as a flash memory. ) Etc. The auxiliary storage device 803 can be used for storing various programs executed by the processor 801 and various data. The auxiliary storage device 803 can be used, for example, for storing a voice processing program including the processes of steps S1 to S6 in FIG. In addition, the auxiliary storage device 803 can be used for holding (storing) voice data, information indicating speech sections and vowel sections included in the voice data, frequency distribution of time length of vowel sections, calculated speech speed, and the like. It is.

  The input device 804 is, for example, a keyboard device or a touch panel device. When an operator (user) of the computer 8 performs a predetermined operation on the input device 804, the input device 804 transmits input information associated with the operation content to the processor 801. The input device 804 can be used, for example, for inputting a command for starting a process for estimating speech speed, inputting various setting values, and the like.

  The output device 805 is, for example, a display device such as a liquid crystal display device or a printing device such as a printer. The output device 805 can be used to output the speech speed of the utterance section calculated in step S6 of FIG.

  The input / output interface 806 connects the computer 8 and other electronic devices. The input / output interface 806 includes, for example, a phone jack, a universal serial bus (USB) standard connector, and the like. The input / output interface 806 can be used, for example, for connection between the computer 8 and the sound collection device 2.

  The communication control device 807 is a device that connects the computer 8 to a network such as the Internet and controls various communications between the computer 8 and other communication devices via the network. The communication control device 807 can be used for transmission / reception of audio data between the computer 8 and the telephone set 16, for example.

  The medium driving device 808 reads a program or data recorded in the portable storage medium 890 and writes data stored in the auxiliary storage device 803 to the portable storage medium 890. For the medium driving device 808, for example, a memory card reader / writer corresponding to one type or a plurality of types of standards can be used. When a memory card reader / writer is used as the medium driving device 808, the portable storage medium 890 is a memory card (flash memory) conforming to a standard supported by the memory card reader / writer, for example, the Secure Digital (SD) standard. ) Etc. can be used. Further, as the portable recording medium 890, for example, a flash memory having a USB standard connector can be used. Further, when the computer 8 is equipped with an optical disk drive that can be used as the medium driving device 808, various optical disks that can be recognized by the optical disk drive can be used as the portable recording medium 890. Examples of the optical disc that can be used as the portable recording medium 890 include a Compact Disc (CD), a Digital Versatile Disc (DVD), and a Blu-ray Disc (Blu-ray is a registered trademark). The portable recording medium 890 can be used, for example, for storing a voice processing program including the processes in steps S1 to S6 in FIG. The portable recording medium 890 is used to hold (store), for example, voice data, information indicating the utterance section and vowel section included in the voice data, frequency distribution of time length of the vowel section, and calculated speech speed. Is possible.

  For example, when the operator inputs a command to start processing for estimating speech speed using the input device 804 or the like to the computer 8, the voice stored by the processor 801 in a non-temporary recording medium such as the auxiliary storage device 803. Read and execute the processing program. When the voice processing program is a program including the processes of steps S1 to S7 in FIG. 2, the computer 8 calculates the speech speed from the input voice data and outputs the calculated speech speed to the output device 805 such as a display device. repeat. While executing the speech processing program, the processor 801 includes the speech acquisition unit 110, the speech segment detection unit 120, the vowel segment detection unit 130, the mora number determination unit 140, the speech rate calculation unit 150, and the speech speed estimation apparatus 1. Functions (operates) as the output unit 160. While the processor 801 is executing the speech processing program, the RAM of the main storage device 802, the auxiliary storage device 803, and the like function as a storage unit (not shown) of the speech speed estimation device 1. That is, the RAM of the main storage device 802, the auxiliary storage device 803, and the like store voice data, information indicating the utterance section and vowel section included in the voice data, frequency distribution of time length of the vowel section, calculated speech speed, and the like. Functions as a storage unit.

  Note that the computer 8 operated as the speech speed estimation apparatus 1 does not need to include all the elements 801 to 808 shown in FIG. 22, and some elements can be omitted depending on the application and conditions. For example, the computer 8 may be one in which the communication control device 807 and the medium driving device 808 are omitted.

  The voice processing program to be executed by the computer 8 includes a process of controlling the voice speed of the voice data based on the calculated voice speed and outputting the voice data with the voice speed controlled as shown in the flowchart of FIG. It may be.

  Further, the computer 8 can be operated as a telephone such as the mobile phone terminal 25. For example, a call processing program for transmitting and receiving voice data between the computer 8 and another telephone in parallel with the voice processing program for causing the computer 8 to perform the processing described in each embodiment. Is executed. In this case, the program to be executed by the computer 8 may be a program that performs processing for controlling the speech speed of the voice data based on the speech speed calculated (estimated) by the processing in steps S1 to S6.

  In addition, the voice processing program to be executed by the computer 8 may be a program in which the processes in steps S3 to S5 in the flowcharts of FIGS. 2 and 17 are replaced with the processes in steps S3 'to S5' described above.

The following additional notes are disclosed for each of the embodiments described above.
(Appendix 1)
Detect a plurality of vowel sections included in the utterance section in the input voice data,
Calculating a time length of each of the detected vowel intervals;
Calculating a frequency distribution for the time lengths of the plurality of vowel intervals, and specifying a minimum value of the time lengths of the plurality of vowel intervals,
Based on the identified minimum value of the time length and the frequency distribution, calculate the number of mora corresponding to the time length of each of the plurality of vowel intervals,
Controlling an output signal corresponding to the utterance interval in the input voice data according to the calculated number of mora;
A voice processing program for causing a computer to execute processing.
(Appendix 2)
In the process of specifying the minimum value of the time length,
In the frequency distribution, specify a minimum value among a plurality of time lengths at which the frequency peaks.
The speech processing program according to appendix 1, wherein
(Appendix 3)
In the process of specifying the minimum value of the time length,
Identify a plurality of time lengths in which the frequency peaks in the frequency distribution;
Based on the plurality of time lengths, calculating a time length between the adjacent peaks in the frequency distribution,
The average value of the time length between the calculated peaks is specified as the minimum value of the time length.
The speech processing program according to appendix 1, wherein
(Appendix 4)
In the process of calculating the frequency distribution,
Of all the vowel sections included in the utterance section, calculate the frequency distribution by excluding the last vowel section in the utterance section.
The speech processing program according to appendix 1, wherein
(Appendix 5)
In the process of calculating the frequency distribution,
Of all the vowel sections included in the utterance section, the vowel section whose time length is within a predetermined range is extracted to calculate the frequency distribution.
The speech processing program according to appendix 1, wherein
(Appendix 6)
In the process of calculating the number of mora,
An integer value closest to a value obtained by dividing the time length of the vowel section by the minimum value of the time length is set as the number of mora of the vowel section.
The speech processing program according to appendix 1, wherein
(Appendix 7)
In the process of detecting the vowel section,
Calculating a signal-to-noise ratio in the input voice data;
Detecting a section in which the calculated signal-to-noise ratio is continuous at a threshold value or more as the vowel section,
The speech processing program according to supplementary note 1, wherein
(Appendix 8)
In the process of detecting the vowel section,
Calculating a waveform autocorrelation in the input voice data;
Detecting the section in which the calculated waveform autocorrelation is equal to or greater than a threshold as the vowel section,
The speech processing program according to supplementary note 1, wherein
(Appendix 9)
In the process of detecting the vowel section,
Calculate the formant frequency in the input voice data,
Including a process of detecting a time at which the calculated time change amount of the formant frequency is equal to or greater than a threshold as a boundary between the vowel section and the non-vowel section.
The speech processing program according to supplementary note 1, wherein
(Appendix 10)
The process of controlling the output signal includes:
Calculating the speech speed of the utterance section in the input speech data based on the calculated number of mora for each of the plurality of vowel sections and the time length of the utterance section in the input speech data.
The speech processing program according to appendix 1, wherein
(Appendix 11)
The process of controlling the output signal includes:
When the calculated speech speed is equal to or higher than a threshold value, further includes a process of extending the vowel section in the input speech data to reduce the speech speed.
The speech processing program according to appendix 9, wherein
(Appendix 12)
Computer
Detect a plurality of vowel sections included in the utterance section in the input voice data,
Calculating a time length of each of the detected vowel intervals;
Calculating a frequency distribution for the time lengths of the plurality of vowel intervals, and specifying a minimum value of the time lengths of the plurality of vowel intervals,
Based on the identified minimum value of the time length and the frequency distribution, calculate the number of mora corresponding to the time length of each of the plurality of vowel intervals,
Controlling an output signal corresponding to the utterance interval in the input voice data according to the calculated number of mora;
A voice processing method characterized by executing processing.
(Appendix 13)
A vowel section detection unit for detecting a plurality of vowel sections included in the utterance section in the input voice data;
Calculating a time length of each of the plurality of detected vowel intervals and calculating a frequency distribution of the time lengths of the plurality of vowel intervals; Based on the specified minimum value of the time length and the frequency distribution, the number of mora determining unit that determines the number of mora corresponding to the time length of each of the plurality of vowel sections,
A speech speed calculation unit that calculates the speech speed of the utterance section in the input speech data based on the determined number of mora and the time length of the utterance section;
An audio processing apparatus comprising:

DESCRIPTION OF SYMBOLS 1 Speech speed estimation apparatus 2,1620 Sound collection apparatus 3 Display apparatus 8 Computer 9A, 9B Speaker 10 Call system 11 Call processing apparatus 12 Information processing apparatus 13,1630 Receiver 14A, 14B Switch 15 Network 16 Telephone 21,1610 Call processing part 25 mobile phone terminal 26 speech speed adjustment unit 30 base station 110 speech acquisition unit 120 utterance interval detection unit 130 vowel interval detection unit 140 mora number determination unit 141 frequency distribution calculation unit 142 peak detection unit 143 mora number calculation unit 150 speech rate calculation unit 160, 162 Output unit 170 Speech rate control unit 171 Basic period detection unit 172 Waveform processing unit 173 Extension control unit 190 Storage unit 191 Audio data 192 Target extension rate

Claims (9)

Detect a plurality of vowel sections included in the utterance section in the input voice data,
Calculating a time length of each of the detected vowel intervals;
Calculating a frequency distribution for the time lengths of the plurality of vowel intervals, and specifying a minimum value of the time lengths of the plurality of vowel intervals,
Based on the identified minimum value of the time length and the frequency distribution, calculate the number of mora corresponding to the time length of each of the plurality of vowel intervals,
Controlling an output signal corresponding to the utterance interval in the input voice data according to the calculated number of mora;
A voice processing program for causing a computer to execute processing. In the process of specifying the minimum value of the time length,
In the frequency distribution, specify a minimum value among a plurality of time lengths at which the frequency peaks.
The voice processing program according to claim 1. In the process of specifying the minimum value of the time length,
Identify a plurality of time lengths in which the frequency peaks in the frequency distribution;
Based on the plurality of time lengths, calculating a time length between the adjacent peaks in the frequency distribution,
The average value of the time length between the calculated peaks is specified as the minimum value of the time length.
The voice processing program according to claim 1. In the process of calculating the frequency distribution,
Of all the vowel sections included in the utterance section, calculate the frequency distribution by excluding the last vowel section in the utterance section.
The voice processing program according to claim 1. In the process of calculating the number of mora,
An integer value closest to a value obtained by dividing the time length of the vowel section by the minimum value of the time length is set as the number of mora of the vowel section.
The voice processing program according to claim 1. The process of controlling the output signal includes:
Including calculating a speech speed of the utterance section in the input speech data based on the determined number of mora of each of the plurality of vowel sections and a time length of the utterance section in the input speech data.
The voice processing program according to claim 1. The process of controlling the output signal includes:
When the calculated speech speed is equal to or higher than a threshold, further includes a process of extending the vowel section in the input speech data to reduce the speech speed.
The voice processing program according to claim 6. Computer
Detect a plurality of vowel sections included in the utterance section in the input voice data,
Calculating a time length of each of the detected vowel intervals;
Calculating a frequency distribution for the time lengths of the plurality of vowel intervals, and specifying a minimum value of the time lengths of the plurality of vowel intervals,
Based on the identified minimum value of the time length and the frequency distribution, calculate the number of mora corresponding to the time length of each of the plurality of vowel intervals,
Controlling an output signal corresponding to the utterance interval in the input voice data according to the calculated number of mora;
A voice processing method characterized by executing processing. A vowel section detection unit for detecting a plurality of vowel sections included in the utterance section in the input voice data;
Calculating a time length of each of the plurality of detected vowel intervals to calculate a frequency distribution for the time length of the plurality of vowel intervals, and specifying a minimum value of the time length of each of the plurality of vowel intervals; Based on the specified minimum value of the time length and the frequency distribution, the number of mora determining unit that determines the number of mora corresponding to the time length of each of the plurality of vowel sections,
A speech speed calculation unit that calculates the speech speed of the utterance section in the input speech data based on the determined number of mora and the time length of the utterance section;
An audio processing apparatus comprising:

Images