JP2013015693A - Spoken word analyzer, method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a spoken word analyzer that extracts a section of prominence from voice data without using learning data.SOLUTION: A spoken word analyzer synthesizes a synthesized voice in a reading tone that has a language label, by regarding a prominence section extracted voice, in which a prominence section applied with the language label is extracted, as an input. Then, the analyzer extracts a fundamental frequency sequence X1 that is a fundamental frequency sequence of the prominence section extracted voice and a fundamental frequency sequence X2 that is a fundamental frequency sequence of the synthesized voice, with the prominence section extracted voice and the synthesized voice as inputs. A prominence section extracting unit in the analyzer extracts a prominence section of the prominence section extracted voice and outputs prominence section information, on the basis of a correlation between the fundamental frequency sequence X1 and the fundamental frequency sequence X2 regarding a direction of variation in fundamental frequency between accent phrases and a comparison between the fundamental frequency sequence X1 and the fundamental frequency sequence X2 regarding a quantity of variation in fundamental frequency between the accent phrases, with the fundamental frequency sequences X1, X2, and the language label as inputs.

Description

  The present invention relates to a speech analysis device, method and program for automatically extracting speech segments corresponding to prominence of emphasis and suppression in spoken speech.

  For example, “expression” naturally uttered in scenes where speech is spoken according to movie scenes, scenes of storytelling of fairy tales, scenes of product advertisements through media such as television, and telephone response scenes at call centers etc. In “rich speech”, the emphasis on emphasis and suppression is frequently used. Such a standoff is a relative one that becomes apparent compared to some standard. Therefore, it is difficult to automatically extract a table from a given voice only when the reference is unknown. Until now, a standing section has been designated in advance, and voices spoken with standing are recorded and used in that section.

  In conventional automatic automatic assignment, the automatic assignment of “emphasis” or “not enhancement (non-emphasis)” is formulated as a binary discrimination problem, and the location of “emphasis” is extracted using a binary discriminator. It was. This method is disclosed in Non-Patent Document 1. Non-Patent Document 1 requires learning speech data that is manually labeled in advance in the emphasis section. The learning voice is given a label indicating non-emphasis at the same time that the emphasis section is labeled and where there is no emphasis.

  The binary discriminator is a language related to prosody such as “a sequence of category labels representing speech units such as syllables”, “a numerical value indicating a phrase or a position in a sentence of the speech unit”, and “a position of an accent nucleus included in the phrase”. “Category labels that represent features” and “difference values between fundamental frequencies of synthesized speech synthesized using a normal speech synthesizer and the original speech data for learning” are used as input variables. A binary label is constructed as an output variable. Using this constructed binary discriminator, new speech data other than learning data is subjected to binary discrimination of emphasis or non-emphasis, and a stand-alone section called emphasis is extracted from the speech data.

J. Xu and L. ・ H. Cai, “Automatic emphasis labeling for emotional speech by measuring prosody generation error”, Proceedings of ICIC, 2009, pp. 177-186, 2009.

  However, the conventional method requires learning data to which emphasis / non-emphasis labels are attached in order to extract the stand-up interval. In order to construct a binary discriminator that discriminates the standing section with high accuracy, a large amount of learning data accurately labeled is required. In order to prepare this correctly labeled audio data, there is no choice but to rely on human hands, and the cost is high.

  Thus, it is difficult to automatically extract the standing section, and in many of the studies before Non-Patent Document 1, a label of the standing is previously attached to the text, and a person stands out at the place where the label is attached. The voice was recorded by performing the utterance with the. However, with this method, it is difficult to construct a speech database that includes natural utterance data and utterances including such emphasis and non-emphasis at a natural rate.

  This invention is made in view of such a subject, and it is possible to efficiently extract a stand-by section from audio data without preparing audio data to which manual emphasis / non-emphasis labels are assigned in advance. An object of the present invention is to provide a possible speech analysis device, method and program thereof.

  The speech analysis device according to the present invention has a speech segment extraction target speech provided with a language label as an input, and includes a speech synthesizer, a fundamental frequency sequence extractor, and a standoff segment extractor. The speech synthesizer receives the language label as an input and synthesizes a speech with a reading tone having the language label. The fundamental frequency sequence extraction unit receives the synthesized speech and the speech that is to be extracted as a standup interval, and receives a basic frequency sequence X1 that is a basic frequency sequence of the standout interval extraction target speech and a basic frequency sequence X2 that is a basic frequency sequence of the synthesized speech. To extract. The stand-by section extraction unit receives the fundamental frequency sequences X1 and X2 and the language label as input, and the correlation between the fundamental frequency sequences X1 and X2 with respect to the direction of variation of the fundamental frequency between the accent phrases, and between the accent phrases Based on the comparison between the fundamental frequency series X1 and X2 with respect to the fluctuation amount of the fundamental frequency at, the utterance interval of the utterance interval extraction target speech is extracted and the erection interval information is output.

  According to the speech analysis apparatus of the present invention, it is possible to obtain stand-by section information from the stand-by-section extraction target speech without requiring a large amount of learning data to which accurate emphasis / non-emphasis labels are attached. Therefore, it is possible to eliminate a high cost for operating a human who prepares learning data. In addition, since it is possible to extract accurate standing sections from speech data uttered naturally, the speech analysis device of this invention greatly accelerates research and development based on speech with natural standing. Contribute to doing. In addition, since the synthesized speech generated from the language label becomes a reading speech with a small fluctuation range, it is possible to obtain clear section information with a definition based on the speech.

The figure which shows the function structural example of the speech analysis apparatus 100 of this invention. The figure which shows the operation | movement flow of the speech analysis apparatus 100. The figure which shows the function structural example of the stand-up area extraction part 30. FIG. The figure which shows the operation | movement flow of the stand-up area extraction part 30. FIG. The figure which shows the determination flow of the standup determination means. The figure which shows an example of a motion of the concrete accent phrase average M1 _i and M2 _i .

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows an example of the functional configuration of a speech analysis apparatus 100 according to the present invention. The operation flow is shown in FIG. The speech analysis device 100 includes a speech synthesizer 10, a fundamental frequency sequence extractor 20, a standup section extractor 30, and a controller 40. The function of each part of the speech analysis apparatus 100 is realized by reading a predetermined program into a computer composed of, for example, a ROM, a RAM, a CPU, etc., and executing the program by the CPU.

  Prior to the description of the embodiments, points of emphasis and suppression will be defined. A point of emphasis or suppression can be defined as a relative change in one utterance or a plurality of utterance sequences. In this example, the standard necessary for measuring relative changes is the speech uttered in a reading tone without emphasis or suppression, and the speech in the reading tone is compared with the speech uttered in an expressive manner. Thus, the location where the change occurs is taken as the extraction target. Although this change appears as a difference in various physical quantities such as fundamental frequency fluctuation, speech duration, and voice quality, this embodiment focuses on the fluctuation due to fundamental frequency fluctuation. That is, a point where the fundamental frequency is relatively high is defined as “emphasis”, and a point where the fundamental frequency is relatively low is defined as “suppression”. In this embodiment, the unit of the emphasis and suppression part is defined as the unit of the accent phrase.

  The speech synthesizer 10 receives the language label given to the speech extraction target speech, and synthesizes a speech with a reading tone having the language label (step S10).

  Language labels include “speech segment types such as phonemes and syllables and start / end times of those speech segments”, “pause segment start / end times”, “accent phrase boundaries and their start / end times and accents. Phrase accent type ". From these pieces of information, it is possible to automatically calculate and give “the length of the accent phrase” and “the position of each voice unit in the accent phrase”.

  The synthesized speech synthesized by the speech synthesis unit 10 is a conventional speech synthesizer (reference: T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, “Simultaneous modeling of spectrum, pitch and duration. in HMM-based speech synthesis, “Proceedings of EUROSPEECH, pp. 2347-2350, 1999.) Depending on the mode of the speech synthesizer 10, the fundamental frequency sequence X2 may be output simultaneously with the synthesized speech.

  The basic frequency sequence extraction unit 20 receives the synthesized speech synthesized by the speech segment extraction target speech and the speech synthesis unit 10 as an input, and the fundamental frequency sequence X1 that is the fundamental frequency sequence of the speech synthesis target speech and the basics of the synthesized speech. A basic frequency series X2 that is a frequency series is extracted (step S20). When the speech synthesizer 10 outputs the synthesized speech and its fundamental frequency sequence X2 at the same time (see the broken line in FIG. 1), the fundamental frequency sequence extraction unit 20 uses the fundamental frequency that is the fundamental frequency sequence of the stand-by interval extraction target speech. Only the series X1 is extracted. The fundamental frequency is defined as the shortest cycle of the periodic signal, and is perceived as the pitch of the voice on hearing. In the case of speech, neither the opening / closing time interval of the vocal cords nor the waveform is constant, so there is no strict meaning of the fundamental frequency, but the instantaneous frequency of the fundamental wave is taken as the fundamental frequency. The fundamental frequency can be obtained every 1 ms, for example. The unit of the fundamental frequency is originally Hz, but it may be a value as it is or a value converted to a natural logarithm with the base being e (Napier number).

  As for the fundamental frequency, for example, a method of extracting a fundamental wave component by designing a bandpass filter that can extract only the fundamental wave and performing wavelet transform using the impulse response as a mother wavelet is known (references: H. Katayose, A. de Cheveigne and RD Patterson, “Fixed point analysis of frequency to instantaneous frequency mapping for accurate estimation of F0 and periodicity,” Eurospeech '99, pp. 2781-2784 (1999).). There are a plurality of other methods for extracting the fundamental frequency, such as a method of obtaining from the autocorrelation coefficient in the time domain. Obtaining the fundamental frequency itself is a conventional technique and will not be described in detail.

  The table interval extraction unit 30 receives the fundamental frequency sequence X1, the fundamental frequency sequence X2, and the language label extracted by the fundamental frequency sequence extraction unit 20, and compares the fundamental frequency sequences. As described above, the fundamental frequency sequence X2 may be directly input from the speech synthesizer 10. Since the language label information is identical between the fundamental frequency sequence X1 of the speech to be extracted and the fundamental frequency sequence X2 of the synthesized speech, it is possible to compare the two fundamental frequency sequences. The inter-phrasing phrase extraction unit 30 compares the fundamental frequencies of the fundamental frequency sequences X1 and X2, extracts the prominent section of the speech to be extracted, and outputs the stand-by section information (step S30). The control unit 40 controls time-series operations between the above-described units.

  As described above, the speech analysis apparatus 100 according to the present invention synthesizes synthesized speech from the language label attached to the speech extraction target speech that is naturally spoken in an expressive tone. Then, the stand-by section is extracted by comparing the fundamental frequencies having the same language labels of the fundamental frequency sequence X1 of the stand-by section extraction target speech and the basic frequency sequence X2 of the synthesized speech. Therefore, unlike the conventional method, it is possible to extract the standing section from the target speech without using the speech data for learning. As described above, the speech synthesizer 10 and the fundamental frequency sequence extraction unit 20 can be realized by the conventional technique. However, the speech analysis device 100 of the present invention is the entire device shown in FIG. The configuration itself is new. A particularly new part is a stand-up section extraction unit 30 that extracts stand-up sections from two fundamental frequency sequences.

  FIG. 3 shows an example of a more detailed functional configuration of the stand-up section extraction unit 30, and its operation will be described. The standing section extracting unit 30 includes an average value / standard deviation calculating unit 31, a normalizing unit 32, an accent phrase average series extracting unit 33, and a standing determination unit 34. In FIG. 4, the operation | movement flow of the standing area extraction part 30 is shown.

  The average value / standard deviation calculating means 31 receives the fundamental frequency sequence X1 and the fundamental frequency sequence X2 output from the fundamental frequency sequence extraction unit 20 as input, and the utterance average μ1 that is the average value of the fundamental frequency over the entire utterance and the utterance average μ2, and the standard deviation utterance standard deviation σ1 and utterance standard deviation σ2 are calculated (step S31).

  The normalizing means 32 obtains an utterance normalization sequence M1 and an utterance normalization sequence M2 that are utterance normalization sequences obtained by dividing the basic frequency sequences X1 and X2 by the utterance averages μ1 and μ2 by the utterance standard deviations σ1 and σ2. Obtained (step S32, equation (1)).

  By this normalization processing, individual differences among speakers can be removed, and the accuracy of extracting the stand-up section can be improved.

The accent phrase average series extraction unit 33 receives the utterance normalization series M1, the utterance normalization series M2, and the language label, and divides each utterance normalization series M1 and M2 into accent phrases i to obtain an i-th accent phrase. Accent phrase averages M1 _i and M2 _i which are average values of the normalized values are calculated (step S33). i is 1 to n.

The standing judgment means 34 receives the accent phrase averages M1 _i and M2 _i as input, and stands out based on the correlation of the direction of variation and the comparison of the variation amount with respect to a pair of adjacent accent phrases up to the last accent phrase n. The standing section information representing the section is output (step S34).

The desk determination means 34 receives the accent phrase averages M1 _i and M2 _i as input and calculates the average values μ1 _i (formula (2)) and μ2 _i (formula (3)) of adjacent accent phrase sets, The respective correlation coefficients ρ12 _i , the respective fluctuation amounts Δ1 _i and Δ2 _i, and the fluctuation amount ratio r _i obtained by dividing the fluctuation amount Δ1 _i by the fluctuation amount Δ2 _i are obtained, emphasis and suppression of the accent phrase i, and emphasis Outputs stand-by section information representing stand-up sections that are neither suppressed nor suppressed.

An equation for obtaining the correlation coefficient ρ12 _i is shown in Equation (4).

Equations (5), (6), and (7) are used to obtain the fluctuation amounts Δ1 _i , Δ2 _i and the fluctuation amount ratio r _i .

  FIG. 5 shows an operation flow of the tabletop determination means 34, and the determination flow will be described. The operation flow of FIG. 5 shows that i + 1 is the accent phrase number, i is the first accent phrase number in the sentence, n is the last accent phrase number, and i and i + 1 accent phrases are used. Is the emphasis or suppression, or the following process for classifying into a stand-up section without emphasis or suppression is repeated until the last accent phrase n (step S340).

The standing determination means 34 obtains a correlation coefficient ρ12 _i for each accent phrase i (step S341). Then, the fluctuation amounts Δ1 _i and Δ2 _i between the adjacent accent phrases i and i + 1 and the fluctuation amount ratio r _i are obtained (step S342).

When the correlation coefficient ρ12 _i is a positive correlation (YES in step S343), when the fluctuation amount Δ1 _i > the fluctuation amount Δ2 _i and the fluctuation amount ratio r _i is outside the threshold interval, the accent phrase i + 1 is an enhancement interval. Determination is made (YES in step S344). When the fluctuation amount ratio r _i exceeds, for example, 120%, it is determined as emphasis. If NO is determined in step S344, if the variation amount Δ1 _i <the variation amount Δ2 _i and the variation amount ratio r _i is outside the threshold section, the accent phrase i + 1 is determined to be a suppression section (YES in step S345). ). When the fluctuation amount ratio r _i is, for example, less than 80%, it is determined to be suppressed. When it is determined NO in step S345, it is determined that the accent phrase i + 1 is neither emphasized nor suppressed.

  If it is determined in step S343 that there is no positive correlation (NO in step S343), it is determined whether the correlation is negative (step S346). If it is determined in step S346 that there is no negative correlation, it is determined that the accent phrase i + 1 is neither emphasized nor suppressed (NO in step S346).

If it is determined in step S343 that there is a negative correlation (YES in step S346), if the variation Δ1 _i is positive and the variation Δ2 _i is negative (YES in step S347), the accent phrase i + 1 is emphasized. It is determined. When it is determined NO in step S347, if the variation Δ1 _i is negative and the variation Δ2 _i is positive (YES in step S348), the accent phrase i + 1 is determined to be suppressed. If NO is determined in step S348, it is determined that the accent phrase i + 1 is neither emphasized nor suppressed.

FIG. 6 shows the movement of the accent phrase average series M1 _i and M2 _i , and the operation of the standup judgment means 34 will be further described. The horizontal direction in FIG. 6 represents a change in accent phrase and represents elapsed time, and the vertical direction represents a normalized value. The movement of the accent phrase average sequence of accent phrases i and i + 1 in FIG. 6 shows an example of the case where the accent phrase i + 1 of the speech to be extracted is extracted as emphasized. Since the change in the average value μ1 _i between the accent phrases is in the plus direction and the change in the average value μ2 _i between the accent phrases is in the minus direction, the correlation coefficient ρ12 _i is a negative correlation. In this case, in the operation flow of FIG. 5, YES is determined in step S346, and since the variation Δ1 _i is in the positive direction and the variation Δ2 _i is in the negative direction, YES is determined in step S347, and the accent phrase i + 1 is determined to be emphasized.

Next, since the change in the average value μ1 _{i + 1} between the accent phrases i + 1 and i + 2 is in the minus direction and the change in the average value μ2 _{i + 1} between the accent phrases is in the minus direction, the correlation coefficient ρ12 _{i + 1} is positive. (YES in step S343). In this case, the fluctuation amount ratio r _{i + 1} is clearly within the threshold interval. That is, since the fluctuation amount ratio r _{i + 1} is in the range of 80% to 120%, it is determined that the accent phrase i + 2 is neither emphasized nor suppressed (NO in step S345).

As described above, the correlation coefficient of the change between the accent phrase average series M1 _i derived from the speech to be extracted and the accent phrase average series M2 _i derived from the synthesized speech, and the variation Δ1 of each. _{By using i} and Δ2 _i and the variation ratio r _i , the accent phrase can be determined. As described above, according to the speech analysis device 100 of the present invention, it is possible to obtain stand-by section information from speech data while eliminating a large amount of learning data to which accurate emphasis / non-emphasis labels are attached. . In addition, since the synthesized speech generated from the language label becomes a reading speech with a small fluctuation range, it is possible to obtain clear section information with a definition based on the speech.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (5)

The speech to be extracted from the standing section with the language label is input.
A speech synthesizer for synthesizing a synthesized speech with a reading tone with the language label;
Using the synthesized speech and the speech that is to be extracted as a stand-up interval, the basic frequency sequence X1 that is the basic frequency sequence of the speech that is to be extracted as a stand-by interval and the basic frequency sequence X2 that is the basic frequency sequence of the synthesized speech are extracted. A fundamental frequency sequence extraction unit to
Using the fundamental frequency sequences X1 and X2 and the language label as input, the correlation between the fundamental frequency sequences X1 and X2 with respect to the direction of fluctuation of the fundamental frequency between accent phrases and the fundamental frequency between the accent phrases. A stand-up section extraction unit that extracts a stand-up section of the above-described stand-by section extraction target speech based on the comparison between the fundamental frequency series X1 and X2 with respect to the amount of fluctuation, and outputs stand-up section information; ,
Talking word analysis device. In the speech analysis device according to claim 1,
The above stand section extraction unit
Using the fundamental frequency sequence X1 and the fundamental frequency sequence X2 as input, an utterance average μ1 and an utterance average μ2 that are average values of fundamental frequencies over the entire utterance, and an utterance standard deviation σ1 and an utterance standard deviation σ2 that are standard deviations are obtained. Mean value / standard deviation calculation means for calculating,
Normalization means for obtaining an utterance normalization sequence M1 and an utterance normalization sequence M2, which are utterance normalization sequences obtained by dividing a value obtained by subtracting the utterance average from the basic frequency sequences X1 and X2 by an utterance standard deviation;
The utterance normalization sequence M1, the utterance normalization sequence M2, and the language label are input, the respective sequences are divided into accent phrases i, and the accent of the utterance normalization sequence in each accent phrase i (i: 1 to n). An accent phrase average series extracting means for obtaining phrase averages M1 _i and M2 _i ;
Using the above accent phrase averages M1 _i and M2 _i as input, a standup that represents a standup interval based on the correlation of the direction of change and the comparison of the amount of change with respect to a set of adjacent accent phrase averages up to the last accent phrase n Standing judgment means for outputting section information;
A speech analysis device characterized by comprising: The speech to be extracted from the standing section with the language label is input.
A speech synthesis process for synthesizing a spoken tone synthesized speech with the language label;
Using the synthesized speech and the speech that is to be extracted as a stand-up interval, the basic frequency sequence X1 that is the basic frequency sequence of the speech that is to be extracted as a stand-by interval and the basic frequency sequence X2 that is the basic frequency sequence of the synthesized speech are extracted. A basic frequency sequence extraction process,
Using the fundamental frequency sequences X1 and X2 and the language label as input, the correlation between the fundamental frequency sequences X1 and X2 with respect to the direction of fluctuation of the fundamental frequency between accent phrases and the fundamental frequency between the accent phrases. A comparison between the fundamental frequency series X1 and X2 with respect to the amount of fluctuation, and a table interval extraction process for extracting a table interval of the table segment extraction target speech and outputting table interval information based on the comparison ,
A word analysis method with a story. In the speech analysis method described in claim 3,
The above-mentioned standing section extraction process is
Using the fundamental frequency sequence X1 and the fundamental frequency sequence X2 as input, an utterance average μ1 and an utterance average μ2 that are average values of fundamental frequencies over the entire utterance, and an utterance standard deviation σ1 and an utterance standard deviation σ2 that are standard deviations An average value / standard deviation calculation step to be calculated;
A normalization step for obtaining an utterance normalization sequence M1 and an utterance normalization sequence M2 which are utterance normalization sequences obtained by dividing a value obtained by subtracting the utterance average from the basic frequency sequences X1 and X2 by an utterance standard deviation;
The utterance normalization sequence M1, the utterance normalization sequence M2, and the language label are input, the respective sequences are divided into accent phrases i, and the accent of the utterance normalization sequence in each accent phrase i (i: 1 to n). An accent phrase average sequence extraction step for obtaining phrase averages M1 _i and M2 _i ;
Using the above accent phrase averages M1 _i and M2 _i as input, a standup that represents a standup interval based on the correlation of the direction of change and the comparison of the amount of change with respect to a set of adjacent accent phrase averages up to the last accent phrase n A standing judgment step for outputting section information;
A word analysis method characterized by including.   A program for causing a computer to function as the speech analysis device according to claim 1 or 2.

Images