JP5863472B2 - Speaking speed conversion device and program thereof - Google Patents

Description

The present invention relates to a speech speed converting device and a program to convert the speech rate when reproducing audio content.

In recent years, pre-recorded audio contents such as audio books and audio contents distributed via communication media such as the Internet have become widespread. Accordingly, there is an increasing demand from users to listen to these audio contents at high speed. As a technique for solving this demand, a technique for uniformly increasing the playback speed of audio content is generally used.
This method linearly expands and contracts the audio waveform according to the reproduction magnification. Moreover, when expanding / contracting a voice waveform, a technique for shortening the time while maintaining the height (pitch) of the original voice without causing a change in pitch like an analog voice signal is disclosed in Patent Documents. (For example, refer to Patent Document 1).

However, in the method of expanding / contracting the voice waveform as described above, although it depends on the speaking speed of the original voice, the triple-speed playback is regarded as a limit that humans can hear.
Therefore, as a technique for facilitating listening even when the playback speed is increased, a technique has been proposed in which a silent section in the audio signal is partially deleted and the time is assigned to the audio playback time (for example, Patent Documents). 2).

  This method is a method of reproducing a voice as slowly as possible within the time when the voice signal is converted into a voice signal having a target time length by converting the voice speed with a designated conversion magnification. That is, in this method, when the voice reproduction speed is increased by the speech speed conversion, a part of the silent section is deleted, and the ratio of the voice reproduction time in the target time length is increased. As a result, this method increases the ratio of the time length of the voice to the target time length and does not delete the silent section, and the voice is played back slowly compared to the case where the speech speed is converted to the target time length. Will be.

In addition, a method has been proposed in which the audio signal is partially deleted while the audio reproduction speed is maintained to increase the reproduction speed of the video corresponding to the audio (for example, see Patent Document 3).
In this method, an audio signal is written in a ring memory, and the audio signal is reproduced at a reproduction speed of 1 × speed at the time of reading, and a corresponding video signal is reproduced at an n × speed. At this time, in this method, an audio signal within the capacity of the ring memory is reproduced at a single speed, but an audio signal exceeding the capacity is deleted.

JP-A-9-16193 JP-A-10-301598 JP-A-8-147874

In the method of Patent Document 1 described above, it is generally considered that 3 × speed playback is the limit that humans can hear, and there is a problem that the content cannot be grasped if playback is performed at a higher speed.
On the other hand, when a person visually reads a sentence, the content of the sentence can be grasped faster by performing so-called oblique reading than when the sentence is reproduced by voice at a triple speed. In particular, there is a demand from visually impaired people to listen to audio content at high speed at least as fast as oblique reading.

Moreover, it is possible to improve the ease of hearing at the time of high-speed reproduction by the method of Patent Document 2 described above. However, even those who say that 3 times the playback speed is sufficient are tired of watching for a long time. Therefore, further improvement in the ease of hearing has been desired for this method.
Also, with the technique disclosed in Patent Document 3, an audio signal that exceeds the ring buffer is unconditionally deleted. Therefore, with this method, there is a problem that a voice having important meaning content may be deleted from the voice signal, and it is difficult to grasp the content.

  The present invention has been made in view of the above problems and demands, and plays back audio content at high speed while grasping the content as if reading the text obliquely, and also during high-speed playback. It is an object of the present invention to provide a speech speed conversion device that can be easily heard.

  The present invention has been made to solve the above-mentioned problems. First, the speech speed conversion apparatus according to the present invention partially deletes audio contents and reproduces them at a specified reproduction magnification. And it was set as the structure provided with an audio | voice content storage means, an acoustic feature-value storage means, an area information storage means, a deletion area search means, and an output time length adjustment means.

In such a configuration, the speech rate conversion apparatus stores in advance the audio content that is subject to speech rate conversion in the audio content storage unit. Further, the speech speed conversion apparatus stores in advance the acoustic feature amount for each time of the audio content in association with the time in the acoustic feature amount storage unit. This acoustic feature amount is a physical feature amount as sound of speech , and is pitch (physical voice height) and power (physical voice magnitude) , or pitch or power .

  Further, the speech speed converting apparatus stores in advance the voice section and the non-voice section of the voice content in association with the time of the voice content in the section information storage unit. The voice segment and the non-speech segment can be classified, for example, depending on whether the voice power is larger or smaller than a predetermined threshold. Here, the voice section refers to a section where the speaker speaks, and the non-voice section refers to a section where the speaker does not speak. The non-speech section includes noise, silence and the like in addition to the section where the speaker is not speaking.

  Then, the speech speed conversion device uses the deletion section search means to extract a section in which the change in the acoustic feature amount is smaller than a predetermined reference in the speech section immediately before the non-speech section from the end time of the speech section. Search as a deletion section of audio content. That is, the deletion section search means sets a voice section having a small change in acoustic feature amount as a section to be deleted from the voice content immediately before the non-voice section, such as when there is little change in the sound volume. As described above, it is determined that the speech section having a small change in the acoustic feature value is weak in the intention of the speaker to convey to the other party, and is deleted in the present invention.

  Then, the speech speed conversion device reproduces the output time length of the audio content excluding the plurality of deletion intervals searched by the deletion interval search means by the output time length adjustment means, with the specified time length of the original audio content being specified. The audio content excluding the deletion section is extended and output so that the output time length of the magnification is obtained. At this time, since the deletion section is set, an extra reproduction time is secured for the sound to be output by the amount deleted. As a result, compared to the case where the original audio content is directly converted into the speech speed, the audio to be output is reproduced slowly.

  In addition, the speech speed conversion device of the present invention is further characterized by further comprising an acoustic feature amount extraction means and a section information detection means.

In such a configuration, the speech speed conversion apparatus extracts the acoustic feature amount from the audio content by the acoustic feature amount extraction unit, and writes it in the acoustic feature amount storage unit in association with the time. This acoustic feature quantity is a physical feature quantity such as power and pitch.
Further, the speech speed conversion apparatus detects the voice section and the non-speech section in the voice content by the section information detection unit, and writes the voice section and the non-speech section in the section information storage unit in association with the time.

  As described above, the acoustic feature amount extraction unit and the section information detection unit preliminarily analyze the audio content based on the acoustic feature, extract features such as an acoustic feature amount for each time, a voice segment, and a non-speech segment, and a deletion segment. Prepare for exploring. As a result, the speech rate conversion apparatus of the present invention can perform speech rate conversion with any audio content as input.

Further, the speech speed conversion apparatus of the present invention, the acoustic feature quantity, when a power showing the pitch and loudness indicates the height of the voice, the deletion interval search section, a pitch reference search means, the power reference Search means and deletion interval determination means are provided .

  In such a configuration, the speech speed conversion device searches, as a deletion section, a section in which the pitch change is smaller than a predetermined reference, by using the pitch reference search unit, retroactively from the end time of the speech section. In addition, the speech speed conversion device searches, as a deletion section, a section in which the power change is smaller than a predetermined reference retroactively from the end time of the voice section by the power reference search means. That is, the speech speed conversion device searches for a deletion section having a different time length from the end time by the pitch and power in the same voice section.

  Then, the speech rate conversion apparatus responds to the predetermined ratio of the weight of the pitch and the power from the deletion section searched by the pitch reference search means and the deletion section searched by the power reference search means by the deletion section determination means. Then, the deletion section in the voice section is determined. In other words, if the pitch weight is large, a section close to the deletion section searched by the pitch reference search means is set, and if the power weight is large, a section close to the deletion section searched by the power reference search means is set. It will be.

  As a result, the speech speed conversion apparatus of the present invention can determine the deletion section according to the weight of the pitch and power. Moreover, the speech rate conversion apparatus of the present invention can perform speech rate conversion suitable for the language by changing the weight in advance according to the language characteristics of each country. For example, in a language with little change in power, it is possible to set a deletion section in which the pitch is prioritized by increasing the weight of the pitch.

Further, when the acoustic feature quantity is a pitch indicating the height of the voice, deletion section searching means comprises a pitch reference search means.

  In such a configuration, the speech speed conversion device searches, as a deletion section, a section in which the pitch change is smaller than a predetermined reference, by using the pitch reference search unit, retroactively from the end time of the speech section.

Further, when the acoustic feature quantity is a power indicating the magnitude of voice, deletion section searching means comprises a power reference search means.

  In such a configuration, the speech speed conversion device searches, as a deletion section, a section in which the power change is smaller than a predetermined criterion, retroactively from the end time of the voice section, by the power reference search means.

  In the speech speed conversion device of the present invention, the time when the deletion section search means searches for the deletion section retroactively from the end time of the voice section does not exceed a predetermined time from the start time of the voice section. It is characterized in that the time length of the deletion section to be performed is in a range not exceeding a predetermined maximum time length.

  In such a configuration, the speech rate converting apparatus limits the time length when searching for a deletion section in the voice section by the deletion section searching means, so that all the voices in the voice section are not set as the deletion section. At least a predetermined time length from the beginning is not deleted. As a result, the speech rate conversion apparatus of the present invention ensures a predetermined time length from the beginning of at least a series of speech intervals (expiratory paragraphs) regardless of changes in the acoustic feature amount. Therefore, the speech speed conversion apparatus of the present invention makes it easy for the user to grasp the meaning content even if the voice is partially deleted.

  Further, in the speech speed converting apparatus according to the present invention, the deletion section search means further selects another section in the non-speech section that has a predetermined time length or more and that has left a predetermined time length shorter than the time length. It is characterized by the deletion section.

In such a configuration, the speech rate conversion apparatus assigns the deleted section to the playback time of the voice by setting a part of the non-voice section as the deleted section by the deleted section searching means.
As a result, the speech speed converting apparatus according to the present invention can secure a time margin for assigning other voices by the amount corresponding to the deletion of the voice and non-voice sections. And when the audio | voice content conversion apparatus of this invention reproduces | regenerates audio | voice content at the same reproduction speed, it will reproduce | regenerate audio | voice slowly rather than the conventional speech speed conversion, and a user becomes easy to hear.

Further, the present invention is a program for causing a computer to function as any of the above speech rate conversion devices.

The present invention has the following excellent effects.
According to the present invention, it is possible to reproduce audio content at high speed by deleting a part of the audio section of the audio content. As a result, the user can listen to the audio content partially at high speed as if reading the text obliquely. Further, according to the present invention, it is possible to secure a time margin for allocating other speech by the amount corresponding to the deletion of the speech section. Therefore, according to the present invention, when the audio content is reproduced at the same reproduction speed, the audio is reproduced more slowly than the conventional speech speed conversion, so that the user can easily listen.

It is a block block diagram which shows the structure of the speech rate converter which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the deletion area in the non-voice area which a deletion area search means searches in the speech-speed converter which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the deletion area in the audio | voice area which a deletion area search means searches in the speech rate conversion apparatus which concerns on embodiment of this invention. It is explanatory drawing for demonstrating the method in which the output time length adjustment means controls the output length of audio | voice content except a deletion area in the speech speed conversion apparatus which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the speech-speed converter which concerns on embodiment of this invention. It is a block block diagram which shows the structure of the speech-speed converter which concerns on other embodiment of this invention. It is a block block diagram which shows the structure of the speech-speed converter which concerns on other embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
[Configuration of speech speed converter]
First, with reference to FIG. 1, the structure of the speech speed converter 1 which concerns on embodiment of this invention is demonstrated. This speech speed conversion device 1 performs speech speed conversion by partially deleting a speech section as well as a non-speech section such as silence when reproducing audio content (audio signal) at a speed higher than the 1 × speed playback. Is to do. In other words, the speech speed conversion device 1 performs partial reproduction of audio content as if a person reads a sentence represented by printed characters obliquely visually.
Note that the speech speed conversion device 1 deletes the portion where the change in the acoustic feature amount is reduced at the end of the exhalation paragraph divided by inspiration, which is a series of utterances. Prevent omissions as much as possible.
Here, the speech rate conversion apparatus 1 includes an acoustic analysis unit 10, a storage unit 20, a deletion section search unit 30, and an output time length adjustment unit 40.

The acoustic analysis means 10 performs acoustic analysis on the input audio content, and extracts time-specific acoustic feature amounts (acoustic feature information) and section information of speech sections and non-speech sections (including silence sections). . The acoustic analysis unit 10 writes the acoustic feature information and section information extracted by the analysis to the storage unit 20 and notifies the deletion section search unit 30 that the analysis is completed.
Here, the acoustic analysis unit 10 includes a power extraction unit 11, a pitch extraction unit 12, and a voice section detection unit 13.

  The power extraction means (acoustic feature quantity extraction means) 11 extracts power (sound intensity and magnitude), which is one of acoustic feature quantities, from audio content (audio signal) input from the outside. . A general method may be used as a power extraction method in the power extraction unit 11. For example, the power extraction unit 11 calculates the power (power spectrum) by frequency-converting (FFT) the audio content at a predetermined frame width at predetermined time intervals and squaring the amplitude value.

  The power extraction unit 11 smoothes (smooths) the signal level with the passage of time of power. For example, the power extraction unit 11 smoothes the sequential change in power at a cutoff frequency of about 6 to 10 Hz. As a result, the power extraction unit 11 can extract the change in power with the lapse of time from the audio content by smoothing the change in power and suppressing the influence of noise.

  The power extraction unit 11 writes the extracted power value (dB) with the passage of time in the storage unit 20 as one of the acoustic feature information in association with the time from the start of the audio content. That is, the power extraction unit 11 writes an instantaneous value at a certain time in the power smoothing waveform in the storage unit 20 in association with the time.

  The pitch extraction means (acoustic feature quantity extraction means) 12 extracts a pitch (sound pitch), which is one of acoustic feature quantities, from audio content (audio signal) input from the outside. A general method may be used as the pitch extraction method in the pitch extraction means 12. For example, the pitch extraction unit 12 obtains the autocorrelation function of the power spectrum extracted by the power extraction unit 11 and extracts the pitch (fundamental frequency) as the period interval of the maximum value of the coefficient of the autocorrelation function.

  Note that the pitch extraction unit 12 smoothes (smooths) the signal level with the passage of time of the pitch, similarly to the power extraction unit 11. For example, the pitch extraction unit 12 smoothes the pitch change at a cutoff frequency of about 10 Hz. As a result, the pitch extraction unit 12 can extract a change in pitch over time by excluding an acoustic component that is not perceived in general conversational voice from the audio content.

  The pitch extraction unit 12 writes the extracted pitch value (Hz) with the passage of time in the storage unit 20 as one of the acoustic feature information in association with the time from the start of the audio content. That is, the pitch extraction unit 12 writes an instantaneous value at a certain time in the pitch smoothing waveform in the storage unit 20 in association with the time.

  The voice section detecting means (section information detecting means) 13 detects a voice section including voice and a non-voice section not containing voice (including a silent section) from the voice content (sound signal) input from the outside. Is.

A general technique may be used as the voice / non-speech section detection technique in the voice section detector 13.
For example, when the power extracted by the power extraction unit 11 is larger than a predetermined threshold, the voice section detection unit 13 determines that the time section is a voice section, and sets the rest as a non-voice section. Note that this threshold value may be adaptively changed according to the level of the audio signal, or a technique described in Japanese Patent Laid-Open No. 10-301593 may be used.

  That is, the audio section detection means 13 holds the maximum value and minimum value of power within a past predetermined time in a memory or the like (not shown) for the input audio content, and holds the maximum value. A threshold for power that is smaller by a predetermined value is determined. Then, when the difference between the maximum value and the minimum value of the power becomes smaller than a predetermined reference value, the voice section detection unit 13 increases the threshold value according to the difference. As a result, it is possible to discriminate between a speech segment and a non-speech segment while sequentially adapting to changes in the speech level.

  The voice section detection means 13 stores the start time and end time of the detected voice section and non-speech section (including the silent section) in association with the time from the start of the audio content as section information. Write to 20. In addition to the start time and end time, the section information includes type information indicating whether the section is an audio section or a non-audio section, and an identification indicating the first section from the beginning of the audio content. Contains information (eg, serial number).

  Here, the acoustic analysis means 10 sequentially performs acoustic analysis from the input audio content. However, the acoustic analysis unit 10 may perform acoustic analysis on the audio content stored in the storage unit 20 after the audio content is written in the storage unit 20 in advance.

The storage means (audio content storage means, acoustic feature amount storage means, section information storage means) 20 is an audio content input from outside, acoustic feature information (power, pitch) analyzed by the acoustic analysis means 10, and section information. (Speech segment, non-speech segment) is stored. The storage means 20 can be a general storage device such as a hard disk.
The acoustic feature information and the section information stored in the storage unit 20 are referred to by the deletion section searching unit 30 and used when searching for a deletion section of the audio content.

In addition, it is assumed that the deletion section (deletion section information) of the audio content searched by the deletion section search means 30 is written in the storage unit 20.
The audio content, section information, and deletion section information stored in the storage unit 20 are referred to by the output time length adjusting unit 40 and used when adjusting the output time length of the partially deleted audio content.

The deletion section search means 30 searches for a section to be deleted in the audio section and the non-audio section of the audio content based on the deletion condition input from the outside.
The deletion section search means 30 sets a section having a predetermined time length or more as a deletion section in the non-voice section according to the deletion condition. Further, the deletion section search means 30 sets a section having a change in acoustic feature amount less than a predetermined reference in the voice section retroactively from the end time of the voice section as a deletion section of the audio content according to the deletion condition. This deletion condition will be described in detail later.
The deletion section search means 30 writes the deletion section information indicating the section to be deleted in the storage means 20, and notifies the output time length adjustment means 40 that the deletion section has been searched.
Here, the deletion section search means 30 includes a non-speech section search means 31, a non-speech section part deletion means 32, a speech deletion section search means 33, and a speech section part deletion means 34.

The non-speech segment searching unit 31 searches for a non-speech segment (target non-speech segment) having a predetermined time length or longer based on the segment information stored in the storage unit 20.
The non-speech segment searching means 31 searches for a non-speech segment in which the difference between the start time and the end time stored as segment information is longer than a predetermined time length as a deletion condition. This predetermined time length is a time length for excluding the short non-voice interval in the exhalation paragraph from being subject to deletion, and is set to 300 ms, for example.
The non-speech segment searching unit 31 outputs the identification information (serial number) of the searched target non-speech segment to the non-speech segment partial deleting unit 32 and the speech deletion segment searching unit 33.

  The non-speech segment partial deletion means 32 partially deletes a section in the target non-speech section searched by the non-speech section search means 31 while leaving a section for a minimum length of time that is set in advance as a deletion condition. To do. For example, the minimum length of the non-voice interval to be left is 100 ms.

  That is, the non-speech segment partial deletion unit 32 sets the time obtained by adding the minimum time length to the end non-speech segment from the start time in the target non-speech segment stored in the storage unit 20 as the end time, and the identification information (serial Number)) as new section information (deleted section information) corresponding to the number). In the target non-speech section, the partial section to be deleted does not necessarily need to be on the end side of the target non-speech section, and may be on the start end side.

The voice deletion section search means 33 starts from the end of the voice section in the voice section (target voice section) immediately before the target non-voice section searched by the non-voice section search means 31 from the end of the voice section. This is to search for a section to be deleted with little feature change.
In this way, by setting the speech segment immediately before the target non-speech segment searched by the non-speech segment search means 31 as a search target, it is possible to prevent the voice from being deleted during the exhalation paragraph.
Here, the voice deletion section search means 33 includes a power reference search means 331, a pitch reference search means 332, and a deletion section determination means 333.

  The power reference search means 331 determines a time interval in which a change in power, which is one of acoustic feature values, is smaller than a predetermined change amount in a speech segment (target speech segment) immediately before the target non-speech segment. Search backward from the end of.

  That is, the power reference search unit 331 uses the power value stored in the storage unit 20 corresponding to the start time of the target non-speech section as a reference value, and ends the speech segment (target speech segment) immediately before the target non-speech segment. By going back in time from the time in the direction of the start time of the voice section, a section in which the amount of change between the power value stored in the storage unit 20 and the reference value is smaller than a predetermined change is obtained. And the power reference search means 331 makes the start time of the calculated | required area the start time (power reference deletion start time) of the deletion area calculated | required with the power value.

Here, it is assumed that the maximum time is set in advance for the time for which the power reference search means 331 goes back in the direction of the start time from the end time of the voice section. Further, the power reference search means 331 does not go back more than a predetermined time interval from the start time of the audio interval even within a predetermined maximum time. As a result, a section that is not subject to deletion is set at least for a predetermined time or more from the beginning of the voice section, and the sentence head portion in the expiratory paragraph is not deleted. Note that the power reference search unit 331 does not treat a section in which the time length of the speech section is shorter than a predetermined time length as a target speech section to be deleted.
The power reference search unit 331 outputs the deletion section start time (power reference deletion start time) in the voice section to the deletion section determination unit 333.
An example of the deletion section searched by the power reference search unit 331 will be described more specifically later.

  The pitch reference searching means 332 selects a time interval in which a change in pitch, which is one of acoustic feature values, is smaller than a predetermined change amount in the audio interval (target audio interval) immediately before the target non-audio interval. Search backward from the end of.

  That is, the pitch reference search unit 332 uses the pitch value stored in the storage unit 20 corresponding to the start time of the target non-speech segment as a reference value, and ends the speech segment (target speech segment) immediately before the target non-speech segment. The time is traced back from the time in the direction of the start time of the voice section, and a section in which the change amount between the pitch value and the reference value stored in the storage unit 20 is smaller than a predetermined change is obtained. Then, the pitch reference search unit 332 sets the start time of the obtained section as the start time of the deletion section (pitch reference deletion start time) obtained from the pitch value.

Here, it is assumed that the maximum time is set in advance for the time for which the pitch reference search means 332 goes back in the direction of the start time from the end time of the voice section. Similarly to the power reference search unit 331, the pitch reference search unit 332 does not go back more than a predetermined time interval from the start time of the voice interval even within a predetermined maximum time. As a result, a section that is not subject to deletion is set at least for a predetermined time or more from the beginning of the voice section, and the sentence head portion in the expiratory paragraph is not deleted. Note that the pitch reference search means 332 does not handle a section in which the time length of the speech section is shorter than a predetermined time length as a target speech section to be deleted.
This pitch reference searching means 332 outputs the deletion section start time (pitch reference deletion start time) in the voice section to the deletion section determination means 333.
An example of the deletion section searched by the pitch reference search unit 332 will be described in more detail later.

  Based on the power reference deletion start time searched for by the power reference search means 331 and the pitch reference deletion start time searched for by the pitch reference search means 332, the deletion section determination means 333 deletes the corresponding voice section ( Start time). Note that the end time of the deletion section is the same as the end time of the voice section.

Here, the deletion section determination means 333 sets in advance which of the power and the pitch is to be weighted, and determines the deletion section according to the weight (ratio). For example, when the power weight is m, the pitch weight is n, the power reference deletion start time is t _pw , and the pitch reference deletion start time is t _pi , the deletion section determination unit 333 uses the following equation (1): by, to calculate the start time t _d of the deletion section.

The deletion section determination unit 333 outputs the determined deletion section (start time) to the voice section partial deletion unit 34.
Here, the deletion section determining means 333 determines the deletion section according to the weight (ratio) between the power and the pitch, but the time section with the earlier power reference deletion start time or pitch reference deletion start time is selected. It is good also as a deletion area in the said audio | voice area, and it is good also considering the time area of the later one of a power reference deletion start time or a pitch reference deletion start time as a deletion area in the said audio | voice area.

  For example, how the deletion section determining unit 333 determines the deletion section based on the power and the pitch may be determined in advance according to the language type of the audio content. For example, in a language with little change in power according to the characteristics of the language of each country, it is possible to set a deletion section in which the pitch is prioritized by increasing the weight of the pitch. Alternatively, in the case of a language with little change in pitch, it is possible to set a deletion section in which power is prioritized by increasing the power weight.

The voice section partial deletion means 34 is for partially deleting the deletion section searched by the voice deletion section search means 33 from the voice section immediately before the target non-voice section.
That is, the voice segment partial deletion unit 34 sets the end time of the voice segment corresponding to the segment information stored in the storage unit 20 to the time immediately before the start time of the deletion segment searched by the voice deletion segment search unit 33. It is set and written in the storage means 20 as new section information (deleted section information).

  Here, referring to FIGS. 2 and 3 (refer to FIG. 1 as appropriate), a specific example of a method for searching for a section to be deleted in the audio content by the deletion section searching means 30 will be described.

[Deleted section of non-voice section]
First, a section to be deleted in the non-voice section will be described with reference to FIG.
As shown in FIG. 2, the deletion section searching means 30 sets a non-speech section (including a silent section) whose section length is equal to or longer than a predetermined time length as a target non-speech section Seg1, and leaves a predetermined minimum remaining time length leaveS1. The remaining section is set as the deletion section. For example, the time length of the target non-speech section Seg1 is set to 300 ms or more, and the minimum time length leaveS1 is set to 100 ms.

That is, the non-speech segment searching means 31 searches the non-speech segment having a segment length of 300 ms or more from the audio content as the target non-speech segment Seg1. Then, the non-speech section part deletion unit 32, from the time obtained by adding the time length LeaveS1 leave minimum start time _{t 1s} of the subject non-speech section Seg1 _(t 1s + leaveS1), until the end time _{t 1e} of the target non-speech segments Seg1 This section is the deletion section.
Thus, from the start time _{t 1s} of the subject non-speech section Seg1, so that the time length of the non-speech interval of leaveS1 is left without being removed.

[Audio section deletion section]
Next, a section to be deleted in the voice section will be described with reference to FIG. Note that the speech segment to be partially deleted is the speech segment (target speech segment Seg2) immediately before the target non-speech segment Seg1 described in FIG. This target speech section Seg2 is a section longer than a predetermined minimum remaining time length leaveS2.

As shown in FIG. 3, the deletion section searching means 30 is configured to set the target non-voice section Seg1 in the target voice section Seg2 that is immediately before the target non-voice section Seg1 and is longer than the minimum remaining time length leaveS2. Based on the acoustic feature at the start time t _{1 s} , the change amount of the acoustic feature is determined in advance from the end time t _2e to the search final time t _stop where the amount of change with the acoustic feature is larger than a predetermined reference. A section that is smaller than the reference is set as a deletion section. At this time, it is assumed that the time length between the search final time t _stop and the end time t _2e does not exceed the predetermined maximum deletion time length cutMax at the maximum, and the deletion interval search means 30 starts from the beginning of the target speech interval Seg2. It is assumed that the search is not performed until the time interval of the remaining time length leaveS2. For example, the maximum deletion time length cutMax is 250 ms, and the minimum remaining time length leaveS2 is 50 ms.

That is, the power reference search means 331 and pitch reference search means of the voice delete interval search unit 33 332 times to search the deletion section retroactively from the end time t _2e of the target speech section Seg2 is, the start time of the target speech section Seg2 t _The target non-speech section Seg1 is within a range that does not exceed a predetermined time (t _2s + leaveS2) from _2s , and that the time length of the deletion section to be searched does not exceed the predetermined maximum time length (maximum deletion time length cutMax). A section having a small change with respect to the acoustic feature serving as the head reference is set as a deletion section.

  Hereinafter, conditions for continuing the search in the target speech section Seg2, that is, conditions (deletion conditions) in which changes in acoustic features are less than a predetermined reference will be described with examples.

(When referring to power)
First, conditions for the power reference searching unit 331 to search for a deletion section with reference to power in the target voice section Seg2 will be described. Even when the following conditions are satisfied, the search time is limited by the maximum deletion time length cutMax and the minimum time length leaveS2 as described above.

<Example 1>
As an example 1, the power reference search means 331 uses PW _BASE as the power value at the start time t _1s of the target non-speech segment Seg1 (instantaneous value of the smoothing waveform at time t _1s ), and the power value at the search time (smoothing waveform at the search time). instantaneous value) when the _{PW the NOW} of, among the following equation (2) satisfies the condition, continue searching.

Here, th1 is a predetermined threshold value, for example, 10 (dB).
The power reference search unit 331 sets the search time when the condition is no longer satisfied as the search final time t _stop .
According to this example 1, the power of the end portion of the voice section is compared with the power of the head of the non-voice section, and the section to be deleted is specified on the condition that the difference is small.

<Example 2>
As an example 2, the power reference search means 331 uses PW _BASE as the power value at the start time t _1s of the target non-speech segment Seg1 (the instantaneous value of the smoothing waveform at time t _1s ), and the power value at the search time (smoothing waveform at the search time). Search is continued while satisfying the following expression (3), where PW _NOW is the instantaneous value), PW _{MAX is} the maximum power value in the target speech section Seg2, and PW _MIN is the minimum value.

Here, th2 is a predetermined coefficient for adjusting the threshold, and is set to 0.1, for example.
The power reference search unit 331 sets the search time when the condition is no longer satisfied as the search final time t _stop .
The second example is the same as the first example in that the power of the end portion of the voice section is required to have a small difference as compared with the first power of the non-voice section. However, considering that the difference varies depending on the speaker, the threshold value is changed according to the power in the speech section. As a result, even when the speaker changes in the audio content, it is possible to appropriately specify the optimum deletion section.

(When referring to the pitch)
Next, conditions for the pitch reference searching unit 332 to search for a deletion section with reference to the pitch in the target speech section Seg2 will be described.
The pitch reference search means 332 uses PT _BASE as the pitch value (smooth frequency of the pitch waveform at time t _1s ) at the start time t _1s of the target non-speech section Seg1, and the pitch value at the search time (smooth frequency of the pitch waveform at the search time). Is set to PT _NOW , the search is continued while satisfying the following expression (4).

Here, th3 is a predetermined threshold value, for example, 2.
The pitch reference search means 332 sets the search time at which this condition is no longer satisfied as the search final time t _stop .
According to this example, the section to be deleted is specified on the condition that the pitch of the end portion of the voice section is smaller than a predetermined multiple of the leading pitch of the non-voice section.
Even when this condition is satisfied, the search time is limited by the maximum deletion time length cutMax and the minimum time length leaveS2 as described above.
Returning to FIG. 1, the description of the configuration of the speech speed conversion device 1 will be continued.

  The output time length adjusting unit 40 partially deletes the audio content based on the section (deletion section information) to be deleted from the audio section and the non-speech section searched by the deletion section searching unit 30 and specified. The output time length of the audio content is adjusted so as to achieve the playback speed. Here, the output time length adjusting means 40 includes an expansion / contraction rate calculating means 41 and an output time length changing means 42.

  The expansion / contraction rate calculating means 41 is configured such that the reproduction time length of the audio content deleted from the deletion section searched by the deletion section searching means 30 is the time length of reproducing the audio content before deletion at the specified reproduction speed (reproduction magnification). The expansion / contraction rate of the voice section is calculated so as to be the same. For the non-speech section, the expansion / contraction rate is set to “1”, and the expansion / contraction is not performed.

Specifically, the expansion / contraction rate calculating means 41 sets the total time length of the audio section in the audio content before deletion as P _O , the total time length of the non-audio section as Q _O , and the designated playback speed (playback magnification) as R _O , where the total time length of the audio section in the audio content after deletion of the deletion section is P _D and the total time length of the non-audio section is Q _D , the expansion / contraction rate R _{D of the} audio section is expressed by the following equation (5). Is calculated.

The expansion / contraction rate calculating unit 41 outputs the calculated expansion / contraction rate of the voice section to the output time length changing unit 42.
When audio content is played back at high speed, basically, the expansion / contraction rate calculating means 41 calculates the expansion / contraction rate in the direction of shortening the audio section. However, when the reproduction magnification is small and the deletion interval in the audio section is long, the expansion / contraction rate calculating means 41 may calculate the expansion / contraction rate in the direction of extending the remaining audio section.

  The output time length changing means 42 is for changing the output time length of the audio content from which the deletion section searched by the deletion section searching means 30 is deleted based on the expansion ratio calculated by the expansion ratio calculation means 41. That is, the output time length changing unit 42 reads out the audio data of the audio content for each section (audio section, non-speech section) based on the section information and the deletion section information stored in the storage unit 20, and outputs the output time length. Adjust.

Here, when the deletion section is determined by the deletion section information for the voice section, the output time length changing means 42 reads out the voice data from which the deletion section has been deleted from the storage means 20, and the expansion / contraction rate calculation means 41. The time length is changed with the expansion / contraction rate calculated in.
Further, the output time length changing means 42 reads the voice data for the voice section as it is from the storage means 20 for the voice section for which the deletion section is not determined, and uses the expansion ratio calculated by the expansion ratio calculation means 41 for the time. Change the length.

  Here, in order to expand / contract the audio data according to the expansion / contraction rate, the audio waveform may be thinned / repeated according to the pitch period, and the audio waveforms may be overlapped and connected with a time length corresponding to the expansion / contraction rate. . For such expansion and contraction of voice data, a general speech speed conversion method may be used. For example, techniques such as Japanese Patent No. 3327936 and Japanese Patent No. 2955247 can be used.

The output time length changing means 42 reads out the voice data (non-speech data) from which the deletion section is deleted from the storage means 20 when the deletion section is determined by the deletion section information for the non-speech section. It outputs as it is without performing.
Further, the output time length changing means 42 reads the voice data (non-voice data) for the non-voice section from the storage means 20 for the non-voice section for which the deletion section is not defined, and outputs it as it is without expansion / contraction. .

  In this way, the speech speed conversion apparatus 1 adjusts the output time length by providing a deletion section not only in the non-speech section but also in the speech section, so that even if the playback speed is the same as the conventional speech speed conversion, The time length for the partially reproduced sound can be assigned longer than before, and the sound can be easily heard during the reproduction.

Here, referring to FIG. 4 (refer to FIG. 1 as appropriate), the speech speed conversion processing of the audio content in the speech speed conversion apparatus 1 will be schematically described.
FIG. 4A shows audio content data before speech speed conversion, and shows a state in which an audio section and a non-voice section are included. Note that the non-speech segment is a target non-speech segment to be deleted for a predetermined time length or longer. Further, it is assumed that the speech section includes a section B in which the change in the acoustic feature amount is small before the non-speech section.
That is, the speech speed converting apparatus 1 performs acoustic analysis on the audio content by the acoustic analysis unit 10, so that the section information of the voice section and the non-voice section (including the silent section) and the power for specifying the sections A and B are identified. And acoustic feature information such as pitch.

  FIG. 4B shows a state in which a deletion section is set in the audio content of FIG. That is, as shown in FIG. 4B, the speech speed conversion apparatus 1 uses the deletion section search means 30 to identify the section B with a small change in the acoustic feature amount shown in FIG. 4A as the deletion section D1. At the same time, for the non-speech section, a section having a predetermined time length or more is specified as the deletion section D2. Then, the speech speed conversion apparatus 1 sets only the section A as a reproduction target in the voice section.

FIG. 4C shows data after the speech speed converting apparatus 1 converts the speech content of FIG. 4A to the speech speed. Here, the reproduction speed is tripled as an example.
That is, the speech speed conversion apparatus 1 deletes the deletion sections D1 and D2 from the audio content by the output time length adjusting unit 40, so that the total time length becomes 3 times the audio content of FIG. As described above, the output time length of the voice section is adjusted. Here, it is shown that only the section A in the audio section of the audio content in FIG. 4A is converted into the section A1 in FIG.

  FIG. 4D shows data after the speech speed of the audio content in FIG. 4A is converted by the conventional speech speed conversion. In the conventional method, it is assumed that the data (D2) is partially deleted from the non-voice section. This conventional method shows that the sections A and B of the audio section of the audio content in FIG. 4A are converted into the sections A2 and B2 in FIG. 4D, respectively. That is, in this conventional method, the speech speed conversion is also performed for the section B with a small change in the acoustic feature amount that the speech speed conversion apparatus 1 deletes from the speech section.

As can be seen by comparing FIG. 4 (c) and FIG. 4 (d), even in the same playback speed, among the audio sections shown in FIG. The time length of the audio data is the time length of the section A1 in (c) and the time length of the section A2 in (d), and a longer time length can be secured in FIG. 4 (c). As described above, the speech converted by the speech speed conversion device 1 is reproduced more slowly than the speech converted by the conventional method, and is easy to hear.
Although the configuration of the speech speed conversion device 1 has been described above, the speech speed conversion device 1 can be operated by a program (speech speed conversion program) that causes a general computer to function as each of the means described above. The program can also be recorded and distributed on a computer-readable recording medium such as a CD-ROM.

As described above, the speech speed conversion apparatus 1 can generate voice content that can be reproduced at high speed by deleting voice with little change in acoustic characteristics. As a result, the speech speed converting apparatus 1 can further increase the playback speed, which is conventionally limited to 3 × speed, and can listen to the audio content in the same way that a person visually reads a sentence obliquely.
Further, even when the speech speed converting apparatus 1 plays back at the same playback speed as before, since the time allocated to the playback voice is relatively long, it can be converted into audio content that is easier to hear than before.

[Operation of speech speed converter]
Next, referring to FIG. 5 (refer to FIG. 1 as appropriate for the configuration), the operation of the speech speed converting apparatus 1 will be described.
First, the speech speed conversion apparatus 1 performs acoustic analysis on the input audio content by the acoustic analysis means 10 (step S1). That is, the speech speed converting apparatus 1 extracts power (sound intensity and magnitude), which is one of acoustic features, by the power extraction unit 11 of the acoustic analysis unit 10, and the pitch ( (Pitch) is extracted. Furthermore, the speech speed converting apparatus 1 detects a voice section including voice and a non-voice section including no voice (including a silent section) from the voice content by the voice section detecting unit 13 of the acoustic analysis unit 10. These acoustic feature amounts and section information are stored in the storage unit 20. The input audio content is also stored in the storage unit 20.

  Then, the speech speed conversion apparatus 1 refers to the section information stored in the storage unit 20 by the non-speech section search unit 31 of the deletion section search unit 30 and refers to the non-speech section (target A non-voice section) is searched (step S2). Then, the speech speed conversion apparatus 1 uses the non-speech section partial deletion unit 32 of the deletion section search unit 30 to leave a section in the target non-speech section, leaving a section for a minimum length of time that is preset as a deletion condition. It deletes partially, and it writes in the memory | storage means 20 as new area information (deletion area information) (step S3).

  Then, the speech speed conversion apparatus 1 refers to the acoustic feature information stored in the storage unit 20 by the voice deletion section search unit 33 in the voice section immediately before the target non-speech section searched in step S2. From the end of the voice section, a section (deletion section) to be deleted with a small change in acoustic features is searched (step S4).

  That is, the speech speed conversion apparatus 1 uses the power reference search means 331 of the voice deletion section search means 33 to change the power change in a voice section (target voice section) immediately before the target non-voice section from a predetermined amount of change. Search for a few deleted sections retroactively from the end of the speech section. Also, the speech speed conversion apparatus 1 uses the pitch reference search means 332 of the voice deletion section search means 33 to delete a deletion section in which the change in pitch is smaller than a predetermined change amount from the end of the voice section in the same voice section. Explore retrospectively. Then, the speech speed conversion apparatus 1 uses the deletion interval determination means 333 of the voice deletion interval search means 33 to obtain a predetermined weight of power and pitch from the deletion intervals searched independently based on the power and the pitch. Based on this, the deletion interval is determined.

Then, the speech speed conversion apparatus 1 partially deletes the deletion section searched in step S4 from the voice section by the voice section partial deletion unit 34 and writes it to the storage unit 20 as new section information (deleted section information). (Step S5).
If the speech information conversion apparatus 1 has not searched for all target non-speech sections in the section information (No in step S6), the speech speed conversion apparatus 1 returns to step S2 and searches for the next target non-speech section.

  On the other hand, when all the target non-speech sections have been searched (Yes in step S6), the speech speed conversion apparatus 1 uses the output time length adjustment unit 40 to create new section information (deletion section) stored in the storage unit 20. On the basis of the information), the audio content is partially deleted, and the output time length of the audio content is adjusted so that the designated reproduction speed is obtained.

In other words, the speech speed conversion apparatus 1 uses the expansion rate calculation means 41 of the output time length adjustment means 40 so that the playback time length of the audio content from which the deletion section is deleted is the audio before deletion at the specified playback speed (playback magnification) The expansion / contraction rate of the audio section is calculated so as to be the same as the time length for reproducing the content (step S7). Then, the speech speed converting apparatus 1 converts the audio data of the audio content into the segment (based on the segment information and the deletion segment information stored in the storage unit 20 by the output time length changing unit 42 of the output time length adjusting unit 40. Reading is performed for each voice section and non-voice section), and the output time length is adjusted based on the expansion / contraction rate (step S8).
With the above operation, the speech speed conversion apparatus 1 can output audio content that can be reproduced at high speed by deleting the audio data even in the audio section.

Although the embodiment of the present invention has been described above, the present invention is not limited to this embodiment.
For example, here, the speech speed conversion device 1 is configured to include the acoustic analysis means 10, but data (acoustic feature information, section information) corresponding to the audio content is analyzed in advance by an external analysis device. For example, only the data may be input and stored in the storage unit 20.

  Further, the present invention is characterized in that the speech section is partially deleted, and the non-speech section partial deletion means 32 is not an essential configuration. However, by providing the non-speech segment part deletion means 32, it is possible to assign a playback time of the voice to the deleted non-speech segment, so that it can be said that it is a more preferable mode.

Here, in the speech rate conversion apparatus 1, the power extraction unit 11 and the pitch extraction unit 12 of the acoustic analysis unit 10 smooth the respective acoustic features and write them into the storage unit 20. However, the power extraction unit 11 and the pitch extraction unit 12 may write the acoustic features at the extracted time as they are.
In that case, the voice deletion section searching means 33 may sequentially smooth the acoustic features stored in the storage means 20 and search for the deletion section using the instantaneous value for each time.

In addition, here, the speech speed conversion apparatus 1 uses both power and pitch as acoustic features, but either one may be used.
For example, as shown in the configuration of the speech speed conversion apparatus 1B in FIG. 6, the power extraction means 11, the power reference search means 331, and the deletion interval determination means 333 are omitted from the configuration of the speech speed conversion apparatus 1 (FIG. 1). By configuring, speech speed conversion may be performed using only the pitch as an acoustic feature.

  Further, for example, as shown in the configuration of the speech speed conversion apparatus 1C in FIG. 7, the pitch extraction means 12, the pitch reference search means 332, and the deletion interval determination means 333 are omitted from the configuration of the speech speed conversion apparatus 1 (FIG. 1). Thus, the speech speed conversion may be performed using only power as an acoustic feature.

DESCRIPTION OF SYMBOLS 1 Speech speed converter 10 Acoustic analysis main means 11 Power extraction means (acoustic feature amount extraction means)
12 Pitch extraction means (acoustic feature quantity extraction means)
13 Voice section detection means (section information detection means)
20 storage means
(Audio content storage means, acoustic feature quantity storage means, section information storage means)
30 Deletion Interval Search Unit 31 Non-Voice Interval Search Unit 32 Non-Voice Interval Partial Deletion Unit 33 Voice Deletion Interval Search Unit 331 Power Reference Search Unit 332 Pitch Reference Search Unit 333 Deletion Interval Determination Unit 34 Voice Segment Partial Deletion Unit 40 Output Time Length Adjustment Means 41 Expansion / contraction rate calculation means 42 Output time length changing means

Claims (6)

A speech speed conversion device that partially deletes audio content and plays back at a specified playback magnification,
Audio content storage means for storing the audio content in advance;
The audio content time per voice pitch power indicating the size of the pitch and voice indicating the or pitch or power as acoustic features, the acoustic feature storage unit that stores in advance in association with the time,
Section information storage means for storing in advance the voice section and the non-voice section of the voice content in association with the time of the voice content;
Wherein immediately before the speech section of the non-speech section, going back from the end time of the speech section, the fewer sections than the reference change is a predetermined acoustic feature quantity, deletion interval search that searches a deletion section of said audio content Means,
The deletion section so that the output time length of the audio content excluding the plurality of deletion sections searched by the deletion section search means becomes the output time length of the designated reproduction magnification with respect to the time length of the original audio content. Output time length adjusting means for expanding and outputting audio content excluding
A speech speed conversion device comprising: An acoustic feature extraction means for writing from the audio content the extracted acoustic features, in association with a time in the acoustic feature value memory,
Section information detection means for detecting a voice section and a non-voice section in the audio content, and writing the voice section and the non-voice section in the section information storage means in association with time;
The speech speed conversion apparatus according to claim 1, further comprising: When the acoustic feature amount is a pitch indicating the pitch of the voice and a power indicating the volume of the voice,
The deletion section search means includes
Pitch reference searching means for searching for a section having a change in pitch smaller than a predetermined criterion retroactively from the end time of the voice section as the deletion section;
Power reference search means for searching, as the deletion section, a section whose power change is less than a predetermined criterion, retroactively from the end time of the voice section;
Deletion for determining a deletion section in the voice section from a deletion section searched by the pitch reference search means and a deletion section searched by the power reference search means according to a predetermined ratio of pitch and power weight Section determination means ,
When the acoustic feature amount is a pitch,
The deletion section search means includes
Pitch reference search means for searching as a deletion section a section having a change in pitch that is smaller than a predetermined reference retroactively from the end time of the voice section,
When the acoustic feature is power,
The deletion section search means includes
Power reference search means for searching, as the deletion section, a section whose power change is less than a predetermined criterion, retroactively from the end time of the voice section,
The speech speed converting apparatus according to claim 1 or 2, further comprising:   The deletion section searching means is such that the time for searching for the deletion section retroactively from the end time of the voice section does not exceed a predetermined time from the start time of the voice section, and the time length of the deletion section to be searched is The speech speed converting apparatus according to claim 1 or 2, wherein a range not exceeding a predetermined maximum time length is set.   The deletion section search means is characterized in that, in a non-speech section having a predetermined time length or longer, another section that has a predetermined time length shorter than the time length is further set as a deletion section of the audio content. The speech speed conversion device according to claim 1 or 2. A speech speed conversion program for causing a computer to function as the speech speed conversion device according to any one of claims 1 to 5.

Images