JP4313724B2 - Audio reproduction speed adjustment method, audio reproduction speed adjustment program, and recording medium storing the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a program with which the reproducing speed of recorded voice is adjusted without loosing nature of original voice and individuality of an uttered person and to provide a recording medium which stores the program. <P>SOLUTION: The voice reproducing speed adjusting method includes a voice inputting step 22, a spectrum featured vector group computing step 23 which converts voice signals into a spectrum featured vector group, a spectrum featured change vector group computing step 24 which computes a spectrum featured change vector from the spectrum featured vector and a spectrum change amount computing step 25 which computes a spectrum change amount from the spectrum featured change vector within a certain constant time window and continues to conduct computations while moving the time window along a time axis from the beginning of uttering to the end. In the method, the voice reproducing speed is adjusted by segmenting and reproducing the inputted voice waveforms corresponding to an interval in which the spectrum change amount exceeds a threshold value and by updating the threshold value. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to an audio playback speed adjustment method, an audio playback speed adjustment program, and a recording medium storing the same, and in particular, when playing back recorded audio, the playback speed is increased while maintaining the naturalness of the audio by signal processing. The present invention relates to an audio reproduction speed adjustment method to be adjusted, an audio reproduction speed adjustment program, and a recording medium storing the same.

Conventional audio playback speed adjustment methods, that is, methods of fast listening and slow listening of reproduced speech include the following.
As a first method, there is a method of setting a sampling rate of a signal at the time of reproduction of a recorded audio signal to a value different from a rate at the time of recording (see Patent Document 1). As an example, if audio data obtained by observing 8000 samples per second during recording is played back at 16000 samples per second during playback, twice the playback speed can be obtained. In the case of recording by analog tape, if the tape speed at the time of playback is made faster than the tape speed at the time of recording, a higher playback speed can be obtained. However, this method causes a problem that the pitch frequency related to the pitch of the reproduced voice changes. If the sampling rate of the signal at the time of reproduction is greatly increased or decreased compared with the rate at the time of recording, the naturalness of the voice will be lost to the extent that it is difficult to identify who the speaker is.

As a second method, there is a method in which a silent section existing between utterances is detected and a voice is reproduced without skipping the silent section (see Patent Document 2). This second method does not lose the naturalness of the sound, but has a problem that the sound part cannot be reproduced at high speed.
As a third method, there is a method in which a portion with low power of audio is skipped and only a portion with high power is reproduced. However, this third method has a problem in that consonants with low power are lost and it is difficult to hear the reproduced sound.
JP 05-316597 A JP 05-080796 A

  The present invention calculates a spectrum change amount indicating how much the sound spectrum changes at each time of the input sound, and based on this spectrum change amount, a section for reproducing sound, a section for skipping, and a section for extending the sound. The structure of determining is adopted, the sound reproduction speed adjustment method, the sound reproduction speed adjustment program for adjusting the reproduction speed of the recorded sound without losing the naturalness of the original sound, and the individuality of the speaker, the sound reproduction speed adjustment program, and the same The recording medium is provided.

A speech input step for inputting a speech signal, a spectral feature vector sequence calculating step for converting the speech signal into a spectral feature vector sequence, and a spectral feature change vector sequence calculating step for calculating a spectral feature change vector from the spectral feature vector And a spectral change calculation step for calculating a spectral change amount from a spectral feature change vector within a certain time window and moving the time window along the time axis from the beginning to the end of the utterance The input speech waveform corresponding to the section where the spectrum change amount exceeds the threshold is cut out , and the input speech waveform corresponding to the section where the spectrum change amount does not exceed the threshold is deleted and reproduced, and the threshold is changed. The audio playback speed adjustment method that adjusts the audio playback speed by configuring

A speech input step for inputting a speech signal, a spectral feature vector sequence calculating step for converting the speech signal into a spectral feature vector sequence, and a spectral feature change vector sequence calculating step for calculating a spectral feature change vector from the spectral feature vector And a spectral change calculation step for calculating a spectral change amount from a spectral feature change vector within a certain time window and moving the time window along the time axis from the beginning to the end of the utterance In a section where the amount of spectral change is smaller than the threshold, a part of the corresponding input speech waveform is repeated , and in a section where the amount of spectral change is not smaller than the threshold, it is stored and played as it is, and the threshold is changed. Adjust the audio playback speed by adjusting the audio playback speed You configure.

[3] The audio reproduction speed adjustment method according to any one of [1] and [2], wherein the audio reproduction speed adjustment method is such that the amount of change in the spectrum is a dynamic measure .
Claim 4: An audio signal is input, the audio signal is converted into a spectral feature vector sequence, a spectral feature change vector is calculated from the spectral feature vector, and a spectral change amount is calculated from the spectral feature change vector within a certain time window. The time window is calculated while moving along the time axis from the beginning to the end of the utterance, and the input speech waveform corresponding to the section where the amount of spectrum change exceeds the threshold is cut out , and the amount of spectrum change exceeds the threshold. An audio playback speed adjustment program is provided which instructs the computer to delete and play back an input audio waveform corresponding to a non-interval and change the threshold to adjust the audio playback speed.

Claim 5: An audio signal is input, the audio signal is converted into a spectral feature vector sequence, a spectral feature change vector is calculated from the spectral feature vector, and a spectral change amount is calculated from the spectral feature change vector within a certain time window. and, and, the time window towards the end from the start of the utterance calculated while moving along the time axis, in a small interval than the spectral variation threshold, repeating a portion of the corresponding input speech waveform, the spectrum In a section in which the amount of change is not smaller than the threshold value, the program is stored and reproduced as it is, and a voice playback speed adjustment program is configured to give a command to the computer to adjust the voice playback speed by changing the threshold value.

  Claim 6: A recording medium storing the sound reproduction speed adjustment program according to any one of claims 4 and 5 is configured.

  According to the present invention, the section for deleting the voice and the section for inserting the voice are determined focusing on the fact that the amount of change in the spectrum is related to human speech perception. Is not misunderstood as a different phoneme, nor does it impair naturalness. Unlike the compression and expansion of the time axis of the speech waveform, the speech waveform is reproduced while maintaining the pitch frequency, so that the individuality of the speaker is not impaired. Furthermore, even if stationary noise is superimposed on the speech, the spectral change of one stationary noise is very small compared to the speech, so that the spectral change of the speech can be observed, which affects the detection of the spectral change interval. There is no. FIG. 6 is a diagram showing this, and is an observation result when “baNgumiaNnai” is uttered in a state where stationary noise exists. Even if low level stationary noise exists in the vicinity of the time axis, the amount of change in the spectrum of the noise itself is small, so that it can be seen that the amount of change in the spectrum is large only at the phoneme boundary.

  Each vowel and consonant phoneme has typical spectral features. Humans basically perceive the sound by perceiving this spectral difference. However, detailed speech perception experiments show that humans are very sensitive to perception near the phone boundary where the spectrum changes abruptly, and perceive phonemes based on this spectral change ( References: Furui, “Role of Spectral Change Information in Speech Perception”, Acoustical Society of Japan Auditory Society Material, H85-6, 1985). The present invention utilizes this human perceptual characteristic. That is, when calculating the spectral feature vector of the original voice and the spectral change amount from this feature vector and making the voice playback speed faster than the original voice speed, the voice waveform corresponding to the part where the spectral change amount is large is left, and the spectrum The audio waveform corresponding to the small change portion is deleted and reproduced. The part having a large amount of spectrum change is mainly a phoneme boundary and a plosive section. The portion where the amount of change in the spectrum is small is a steady portion of consonants such as vowels and friction sounds, and a silent section. Since there is no spectrum change in the silent section, it is skipped according to this method. The extent to which the reproduction speed is increased is determined by setting a threshold value for the amount of change in the spectrum, and reproducing the speech waveform in the section equal to or greater than the threshold value. If this threshold value is increased, only the part where the spectrum change is more severe is reproduced, and the reproduction speed is increased. On the other hand, if the threshold value is decreased, a section with a relatively small spectrum change is also reproduced, so that the reproduction speed becomes close to the original sound.

On the other hand, when the playback speed is slower than that of the original voice, the time length of the section having a small spectrum change amount is lengthened. When the time length of the section is lengthened, the section length is lengthened by inserting a representative waveform of the section. If the waveform is simply stretched along the time axis, the sound's naturalness will be lost because the pitch frequency of the sound changes in the same way as when the tape is stretched and played back. In addition, since a section with a large amount of spectrum change is a section that is very important for phoneme perception, it is stored and reproduced as it is, and the waveform is not processed.
As described above, according to the present invention, the playback speed is adjusted by deleting and adding a speech waveform based on human speech perception characteristics, so that the speech playback speed can be achieved without impairing the naturalness and speaker nature of speech. Can be adjusted.

Here, the best mode for carrying out the invention will be described in detail with reference to FIG.
In FIG. 1, a speech waveform, a spectrum, and a spectrum change amount are displayed from the lowermost region. The displayed utterance is “baNgumiaNnai” uttered by an adult male. A solid line that runs vertically is a line that indicates a phoneme boundary given by human eyes. As is apparent from the spectrum, each phoneme has a unique spectral pattern. That is, the amount of change in spectrum is large at the phoneme boundary. Referring to FIGS. 4 and 5, these show the waveform, spectrum, and spectral variation when the same word “baNgumiaNnai” is spoken slowly. Comparing FIG. 4 and FIG. 5 with FIG. 1, it can be seen that the duration of the steady spectrum portion is longer when the voice is spoken slowly. The present invention realizes quick listening by finding a stationary part with a small spectrum change from the time pattern of the spectrum change amount and deleting the corresponding waveform. Conversely, the slow listening is realized by extending the normal part of the normal utterance in FIG. In either case, the phoneme boundary having a large spectral change near the solid line running vertically is preserved.

A first embodiment will be described with reference to FIG. The first embodiment is an example in which fast listening is performed to increase the playback speed compared to the original voice.
In an audio input step 22, an audio signal is input through an acoustic-electric converter such as a microphone. The input speech signal is cut out in a spectral feature vector sequence calculation step 23 in a time window of 30 ms and subjected to spectral analysis. The spectrum analysis may be a linear prediction method (LPC) based on an all-pole model or an FFT method. The time window is further shifted by a shift width of 10 ms, and the spectrum analysis of the audio signal is performed along the time axis. Eventually, a spectrum feature vector expressing the spectrum shape is calculated every 10 ms. For example, an LPC cepstrum, an FFT cepstrum, an LPC spectrum, an FFT spectrum, or a mel spectrum or mel cepstrum expressed by logarithmizing the frequency axis of these spectra is calculated as a spectrum feature vector. Next, in a spectrum feature change vector sequence calculation step 24, a new 90 ms time window is provided for the spectrum feature vector sequence, and a change vector of the spectrum feature vector within the time window is calculated. For example, a linear primary regression coefficient of a spectral feature vector sequence within 90 ms is used. Thereby, the gradient of the change pattern of the feature vector within the 90 ms time window is calculated. When the spectrum change is large, the absolute value of the regression coefficient also increases. The regression coefficient of the spectral feature vector is calculated independently in each dimension. Without using the regression coefficient, it is possible to use the following difference value which requires simpler calculation.

Δx _i (t) = x _i (t + Δt) −x _i (t−Δt)
Here, x _i (t) indicates the i-th order of the spectral feature vector at time t. Δt is a half value of the time window width for calculating the amount of change. This difference value also represents the slope of the spectrum change. The spectrum change amount is calculated from the regression coefficient vector or the difference vector. The amount of change in the spectrum uses, for example, a dynamic scale calculated by the following formula (reference: Hiyama, Itakura “Personality information included in the dynamic scale of speech”, Acoustical Society of Japan 1979 Spring Research Presentation Conference Proceedings, 3-2-7, pp589-590).

_{^{D (t) = Σ P i}} = 1 (Δx i (t) 2)
Here, P indicates the number of dimensions of the vector. This value can be regarded as a scalar quantity indicating a spectral change centered at time t. This is calculated in the dynamic scale calculation step as a specific example in the spectrum change amount calculation step 25. Since the spectrum change is severe at the phoneme boundary, the dynamic scale shows a peak near the phoneme boundary. As described above, a section where the value of the dynamic scale is large, that is, a section where the spectrum change is severe is an important place for human speech perception, and this section is searched for in the spectrum change section detection step 26. At this time, a configuration is adopted in which a threshold for the dynamic scale is given to the spectrum change interval detection step 26 and an interval equal to or greater than the threshold is detected. If the threshold value is set high, the detected section becomes a section where the spectrum changes more drastically. Therefore, the ratio of the section to be reproduced to the whole is decreased and the fast listening speed is increased. If the threshold is set low, the speed of the original voice is closer. Input audio is stored in the input audio buffer 27. The voice waveform is read out from this, the voice waveform corresponding to the section detected in the spectrum change section detecting step 26 is cut out in the voice extracting step 28, sent to the voice reproducing step 29, the voice is reproduced, and the process is terminated.

In the above description, the numerical value of the time window width is specified, but this is for easy understanding of the description, and the time window width is not limited to this.
A second embodiment will be described with reference to FIG. The second embodiment is an example in which a slow listening is performed with a playback speed slower than that of the original sound.
The point of the second embodiment is that the waveform of a section with a small spectrum change is repeated. The steps up to the spectral change amount calculation step 35 for calculating the spectral change amount (dynamic scale) from the voice input are the same as those in the first embodiment. In the second embodiment, the spectral change interval detection step 36 finds an interval with a small dynamic measure, that is, an interval with a small spectral change. Input audio is stored in the input audio buffer 37. The input speech waveform is read from here, and in each section detected in the spectrum change section detection step 36, a representative waveform located near the section center is cut out and repeated at the section center. This number of repetitions is determined so as to be proportional to the detected section length, for example. This operation is performed in the speech waveform repetition step 38. The processed voice waveform is played back in the voice playback step 39 and the process ends. For example, if the voice of a section with a large spectrum change is processed to extend the voice or insert the voice, it will cause a problem that the voice perception may be heard from another place that affects the voice perception, or the naturalness may be impaired. In the second embodiment, only the section where the spectrum change is small is processed, so that the naturalness of the reproduced sound is not impaired. In the above description, the numerical value of the time window width is specified, but this is for easy understanding of the description, and the time window width is not limited to this.

The figure explaining the principle of this invention. The figure explaining a 1st Example. The figure explaining 2nd Example The figure explaining the principle of this invention. The figure explaining the principle of this invention. The figure which shows an observation result when saying "baNgumiaNnai" in the state where stationary noise exists.

Explanation of symbols

22, 32 Voice input step 23, 33 Spectral feature vector sequence calculation step 24, 34 Spectral change calculation step 25, 35 Spectral change calculation step 26, 36 Spectral change section detection step 27, 37 Input voice buffer 28 Voice extraction step 38 Voice waveform repetition step 29, 39 Voice playback step

Claims (6)

An audio input step for inputting an audio signal; a spectral feature vector sequence calculating step for converting the audio signal into a spectral feature vector sequence; a spectral feature change vector sequence calculating step for calculating a spectral feature change vector from the spectral feature vector; A spectral change amount calculating step for calculating a spectral change amount from a spectral feature change vector within a fixed time window and moving the time window along the time axis from the beginning to the end of the utterance; The input speech waveform corresponding to the section where the spectrum change amount exceeds the threshold is cut out , and the input speech waveform corresponding to the section where the spectrum change amount does not exceed the threshold is deleted and reproduced, and the speech is changed by changing the threshold. An audio playback speed adjustment method, characterized by adjusting a playback speed. An audio input step for inputting an audio signal; a spectral feature vector sequence calculating step for converting the audio signal into a spectral feature vector sequence; a spectral feature change vector sequence calculating step for calculating a spectral feature change vector from the spectral feature vector; A spectral change amount calculating step for calculating a spectral change amount from a spectral feature change vector within a fixed time window and moving the time window along the time axis from the beginning to the end of the utterance; In a section where the amount of spectral change is smaller than the threshold, a part of the corresponding input speech waveform is repeated , and in a section where the amount of spectral change is not smaller than the threshold, the part is stored and played as it is, and then the voice is changed Audio playback speed characterized by adjusting playback speed Section method. In the audio | voice reproduction speed adjustment method as described in any one of Claim 1 and Claim 2,
The method for adjusting a voice reproduction speed, wherein the spectral change amount is a dynamic scale . Input an audio signal, convert the audio signal into a spectral feature vector sequence, calculate a spectral feature change vector from the spectral feature vector, calculate a spectral change amount from the spectral feature change vector within a certain time window; and The time window is calculated while moving along the time axis from the beginning to the end of the utterance, and the input speech waveform corresponding to the section where the spectral variation exceeds the threshold is cut out . Also, the spectral variation does not exceed the threshold. An audio reproduction speed adjustment program that instructs a computer to delete and reproduce an input audio waveform corresponding to a section and adjust the audio reproduction speed by changing the threshold value. Input an audio signal, convert the audio signal into a spectral feature vector sequence, calculate a spectral feature change vector from the spectral feature vector, calculate a spectral change amount from the spectral feature change vector within a certain time window; and The time window is calculated while moving along the time axis from the beginning to the end of the utterance, and a part of the corresponding input speech waveform is repeated in the section where the spectral variation is smaller than the threshold , and the spectral variation is the threshold. An audio reproduction speed adjustment program for giving a command to the computer to adjust the audio reproduction speed by changing the threshold value in a section that is not smaller than that .   A recording medium storing the audio reproduction speed adjustment program according to claim 4.