JP6622681B2 - Phoneme Breakdown Detection Model Learning Device, Phoneme Breakdown Interval Detection Device, Phoneme Breakdown Detection Model Learning Method, Phoneme Breakdown Interval Detection Method, Program - Google Patents

Description

  The present invention relates to a speech recognition technique, and more particularly to a technique for detecting a phoneme breakage section that has occurred due to unclear pronunciation.

  Speech recognition technology related to natural utterances is widely used in various applications such as dialog analysis at call centers, creation of minutes in meetings, and chat conversation between humans and robots.

  There are several methods for conventional speech recognition. For example, there is a method of generating a maximum likelihood phoneme sequence by generating a template for each phoneme using a large amount of prepared speech data as learning data and sequentially applying the template to the speech data to be recognized (non-patent document). Reference 1).

  There is also a method using DNN (Deep Neural Networks) (Non-patent Document 2). In this method, by learning DNN that outputs phonemes with speech features as input, speech features of speech data to be recognized are directly converted into phonemes, and phoneme sequences are generated. By preparing a large amount, a very good speech recognition rate can be obtained.

  In addition, in order to recognize the utterances of dysarthria due to athetoid cerebral palsy, there is a method of reducing the spectrogram fluctuation by extracting features using CNN (Convolutional Neural Networks) (Non-patent Document 3).

F. Jelinek, "Continuous speech recognition by statistical methods", Proceedings of the IEEE, Vol.64, No.4, pp.532-556, 1976. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, B. Kingsbury, "Deep Neural Networks for Acoustic Modeling in Speech Recognition ", IEEE Signal Processing Magazine Vol.29, Issue 6, pp.82-97, 2012. Yuki Takashima, Wataru Nakakaka, Tetsuya Takiguchi, Yasuo Ariki, “Examination of phoneme labeling based on mixed normal distribution for speech recognition of dysarthria”, IEICE, IEICE Technical Report, vol. 115, no. 100, pp.71-76, 2015.

  In any of the methods, erroneous recognition may occur, but among them, a particular problem is when the speech recognition rate is significantly reduced. There are several possible causes.

  The current speech recognition technology uses speech features learned from pre-prepared training speech data as knowledge for speech recognition, so if the noise environment or speaker's way of speaking deviates significantly from the average, speech recognition The rate drops significantly. Examples of cases that deviate significantly from the average are cases where the noise environment is exposed to a new noise environment that is not included in the speech data for learning, or sudden noise with strong non-stationarity occurs. In terms of the speaker's way of speaking, there are cases where the speaker utters with strong emotions, or the volume of the voice is extremely large (extremely small). Such a case becomes a cause of deterioration, and if it occurs in part or all of an utterance, the speech recognition rate is significantly lowered.

In addition, some current speech recognition technologies use information about what kind of word follows the currently focused word. Once misrecognition occurs, a phenomenon (pitfall error) in which subsequent words are misrecognized in a chained manner may occur (reference non-patent document 1). This pitfall error also significantly reduces the voice recognition rate.
(Reference Non-Patent Document 1) Taichi Asami, Yoshiaki Noda, Satoshi Takahashi, “Analysis of Speech Recognition Errors Focusing on Pit Fall Errors”, Proceedings of the Acoustical Society of Japan March 2008, 1-10-18, pp.53 -54, 2008.

  Accordingly, an object of the present invention is to provide a phoneme breakage section detection technique capable of detecting a phoneme breakup section in which erroneous recognition occurs in a chain due to one phoneme breakup.

  According to one aspect of the present invention, a learning phoneme segment information sequence is assigned to learning speech data, a phoneme label indicating a phoneme, an utterance start time and an utterance end time of the phoneme, and the phoneme being unclear. A learning phoneme segment information sequence including a phoneme collapse flag which is either a phoneme collapse label or a label indicating otherwise, and the learning speech data and the learning phoneme segment information sequence are used for the learning. A phoneme label associated with a vowel phoneme label or a phoneme breakage label indicating a vowel phoneme included in the phoneme section information, a phoneme breakage flag of the learning phoneme label, and a phoneme breakage flag of the learning phoneme label A learning phoneme information extraction unit for extracting a learning phoneme segment speech feature amount that is a speech feature amount corresponding to a segment from a phoneme utterance start time to an utterance end time; A phoneme collapse decision tree learning unit that learns a phoneme collapse decision tree that is a model for detecting phoneme collapse from a phoneme label, a phoneme collapse flag of the learning phoneme label, and a phoneme section speech feature value for learning. .

  According to the present invention, it is possible to learn a phoneme collapse decision tree that is a model for detecting phoneme collapse of a vowel during speech recognition.

The figure which shows an example of a phoneme area information series. The figure which shows an example of the phoneme area information series for learning. The figure which shows an example of a structure of the phoneme collapse detection model learning apparatus. The figure which shows an example of operation | movement of the phoneme collapse detection model learning apparatus. The figure which shows an example of a structure of the phoneme information extraction part 110 for learning. The figure which shows an example of operation | movement of the phoneme information extraction part 110 for learning. The figure which shows an example of the phoneme collapse decision tree which is a phoneme collapse detection model. The figure which shows an example of a structure of the phoneme breaking area detection apparatus. The figure which shows an example of operation | movement of the phoneme breakage area detection apparatus 200. FIG. The figure which shows an example of the recognition result by the speech recognition part 230. FIG. The figure which shows an example of a structure of the speech recognition part 230. FIG. The figure which shows an example of a structure of the phoneme collation part 250. FIG. The figure which shows an example of operation | movement of the phoneme collation part 250. FIG. The figure which shows an example of operation | movement of the estimated phoneme series production | generation part 241. FIG. The figure which shows an example of operation | movement of the phoneme series comparison part 243. The figure which shows an example of the collation result by the phoneme collation part 250. FIG. The figure which shows an example of the detection result by the phoneme breaking area detection part 270.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

<Definition>
Hereinafter, terms used in each embodiment will be described.

[Audio data]
The voice data is voice data recorded in advance for use in learning (specifically, learning of phoneme breaking decision trees) and voice recognition. The voice data is a voice of a sentence uttered by a speaker, for example, digital data digitized at a sampling frequency of 16 kHz.

[Phoneme interval information series]
The phoneme section information series is a series of information about phonemes (hereinafter referred to as phoneme section information) given to speech data. One phoneme section information series is given to the voice data.

  The phoneme section information includes at least a phoneme label representing a phoneme, and information on the phoneme utterance start time and utterance end time. The utterance start time and utterance end time here are elapsed times when the start point of each utterance is set to 0 [seconds]. An example of a phoneme segment information sequence is shown in FIG.

  The learning data used for learning the phoneme collapse decision tree is a set of learning speech data and a learning phoneme section information sequence. Here, the phoneme section information for learning is a dedicated label (hereinafter referred to as a phoneme breakage label) that indicates a phoneme that has been broken by hand (phoneme that is unclearly pronounced) with respect to the phoneme section information. Are associated with each other. An example of the learning phoneme segment information sequence is shown in FIG. As shown in FIG. 2, the phoneme breakage label may have a form different from the phoneme label, or a phoneme label in which phoneme breakage has occurred may be overwritten with the phoneme breakage label. In this example, by adding the symbol “*”, the phoneme “a”, phoneme “r”, phoneme “u”, phoneme in the second, third, sixth, and eighth rows from the top of the table. “U” indicates that the phoneme is broken. It should be noted that a special symbol such as nil indicating that the phoneme is not broken may be attached instead of adding a symbol to the phoneme that has not broken the phoneme.

  In the following, it is assumed that each phoneme is associated with either a phoneme breakage label or a label indicating that phoneme breakage has not occurred as a phoneme breakage flag indicating the presence or absence of phoneme breakage.

  The task of generating the learning phoneme segment information sequence from the phoneme segment information sequence is somewhat subjective, but for example, the task is limited to a location where a recognition error has been caused greatly as a result of the speech recognition process. By doing this, it is possible to suppress to some extent variations in the application of phoneme breakdown labels by the operator.

<First embodiment>
Hereinafter, the phoneme breakage detection model learning apparatus 100 will be described with reference to FIGS.

[Phoneme breakdown detection model learning apparatus 100]
As shown in FIG. 3, the phoneme collapse detection model learning device 100 includes a learning phoneme information extraction unit 110, a phoneme collapse decision tree learning unit 130, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for processing of the phoneme breakage detection model learning device 100. The phoneme breakage detection model learning device 100 learns and outputs a phoneme breakage decision tree, which is a phoneme breakage detection model, with the learning speech data and the learning phoneme section information sequence as inputs.

  The operation of the phoneme breakage detection model learning apparatus 100 will be described with reference to FIG. The learning phoneme information extraction unit 110 receives the learning speech data and the learning phoneme segment information sequence as input, and the vowel phoneme label indicating the vowel phoneme included in the learning phoneme segment information, the semi-vowel phoneme label indicating the semi-vowel phoneme, and the prompting phoneme Phoneme labels indicating the phoneme labels, and phoneme labels associated with the phoneme breakage labels (hereinafter referred to as learning phoneme labels), and the utterance interval corresponding to the phoneme of the learning phoneme label (that is, the learning phoneme) Extract speech features (hereinafter referred to as learning phoneme segment speech features) from the frame corresponding to the label phoneme utterance start time to utterance end time, and break the phoneme between the learning phoneme label and the learning phoneme label A set of flags and learning phoneme segment speech feature values is output (S110).

  Here, the majority of phoneme breakage is often caused by the fact that phonemes that appear at the end of words like vowels, semi-vowels, and prompting sounds are not properly pronounced, so in addition to the phonemes associated with the phoneme breakage labels, The phonemes of the vowel phoneme label, the semi-vowel phoneme label, and the prompt phoneme label are also selected as the phonemes of the learning phoneme label. Therefore, in Japanese, “a (a)”, “i (i)”, “u (u)”, “e (e)”, “o (o)”, “ng (n)”, “ The phoneme label in which the seven phonemes of q () "and the phoneme collapse label are associated with each other is extracted.

  In the example of FIG. 2, the phoneme “a”, phoneme “r”, phoneme “u”, and phoneme “u” in the second, third, sixth, and eighth rows from the top of the table are broken. Since a label is given, it becomes a phoneme label for learning. Also, since the phoneme “a” in the fourth line from the top of the table is a vowel phoneme label, it becomes a learning phoneme label.

  Therefore, the phoneme collapse decision tree is learned using the phonemes of the vowel phoneme label, the phonemes of the semi-vowel phoneme label, the phonemes of the prompt phoneme label, and the phonemes causing the phoneme collapse.

  Note that the phoneme breakage decision tree may be learned using only the phonemes of the vowel phoneme label and the phonemes causing the phoneme breakage. This is because phoneme breakdown is particularly caused by the fact that vowels are not properly pronounced.

  The phoneme breaking decision tree learning unit 130 learns and outputs a phoneme breaking decision tree using the learning phoneme label, the phoneme breaking flag of the learning phoneme label, and the learning phoneme segment speech feature as input (S130).

  Hereinafter, the configurations and operations of the learning phoneme information extraction unit 110 and the phoneme collapse decision tree learning unit 130 will be described in detail.

  First, the learning phoneme information extraction unit 110 will be described with reference to FIGS. As shown in FIG. 5, the learning phoneme information extraction unit 110 includes a speech feature value generation unit 101 and a learning phoneme selection unit 103. The operation of the learning phoneme information extraction unit 110 will be described with reference to FIG.

  The speech feature quantity generation unit 101 divides the training speech data into frames, generates speech feature quantities, and utterance sections corresponding to the phonemes of each learning phoneme section information (that is, utterance ends from the utterance start time of the phonemes) A pair of a phoneme label, a phoneme breakage flag, and a phoneme segment speech feature, and a pair of a phoneme label and a phoneme breakage flag of the corresponding phoneme. The amount is output (S101). For example, MFCC (Mel-Frequency Cepstrum Coefficients) or FBANK (logarithmic Mel filter bank) may be used as the audio feature amount. In general, N is an integer equal to or greater than 1, and the speech interval of each phoneme corresponds to an N frame, so that N phonetic feature amounts are associated with one phoneme label.

  The learning phoneme selection unit 103 receives a phoneme label, a phoneme breakage flag, and a phoneme segment speech feature as an input, and the phoneme label indicates any one of a vowel, a semi-vowel, and a prompt sound, or the phoneme label If a collapsed label is attached (that is, the speech collapse flag is the symbol “*”), the input phoneme label, the phoneme collapse flag, and the phoneme segment speech feature are learned with the phoneme label for learning and the phoneme collapse flag. It is output as is as a phoneme segment speech feature. On the other hand, in other cases (that is, when the phoneme label does not indicate any vowel, semi-vowel, or prompting sound and does not have a phoneme collapse label), the input phoneme label is discarded as it is and is not output. (S103).

  Next, the phoneme breaking decision tree learning unit 130 will be described. The phoneme collapse decision tree learning unit 130 learns a phoneme collapse decision tree using the learning phoneme label, the phoneme collapse flag, and the learning phoneme segment speech feature as input (S130). As shown in FIG. 7, the phoneme collapse decision tree repeats a Yes-No question for the learning phoneme segment speech feature amount that has entered the root node of the top layer (here, the phoneme segment speech feature for learning). It reaches the lowest leaf node (using a question about the quantity attribute and its answer) and outputs a learning phoneme label and a phoneme collapse flag assigned to the reached leaf node. Hereinafter, a leaf node in which the phoneme breakage flag is a symbol “*”, that is, a phoneme breakage occurs is referred to as a phoneme breakup node.

  In general, learning of a decision tree requires a plurality of attribute / value pairs to cluster each learning data. The number and type of attributes can be arbitrarily determined. However, since learning data is generally large, it is desirable that the attributes and their values are automatically determined according to a certain procedure. For example, the length of the phoneme segment can be used as an attribute. This attribute can be calculated from the number of learning phoneme segment speech features that are learning data. In addition, an average value of F0, which is a feature amount representing the pitch of a sound, can be used as an attribute. It is because it can obtain | require by calculation from phoneme area audio | voice feature-value. The phoneme collapse is mainly due to the phonological blurring phenomenon that drags the phonemes before and after due to physical restrictions of mouth movement. Moreover, the tendency of phoneme disruption is stronger for people who are quick. Therefore, learning of the phoneme collapse decision tree efficiently proceeds by using the attribute and value related to the time change amount and the attribute and value related to the phoneme duration.

In addition, a learning method using entropy can be applied to learning a phoneme collapse decision tree. The learning method using entropy can objectively evaluate the importance of attributes used to construct a phoneme breakdown decision tree, and by making the attributes with high importance close to the root node, a more compact decision tree can be created. Can be configured. Hereinafter, a learning method using entropy will be briefly described. The decision tree is T, the m-th node is R _m , and the number of examples in the node R _m (the number of learning data assigned to the node R _m when clustered according to the decision tree T) is n _m . At this time, the probability P ^ _{m, g} that the label is g at the node R _m is expressed by the following equation (1).

Here, I [] Individual learning data, y _i is the label of the learning data I, ranges adding the Σ is as learning data that is assigned to the node R _m are numbered from 1 to n _m , About all learning data.

Since the predicted value y ^ (m) of the label at node R _m is the label with the highest probability,

It becomes. In learning based on entropy, the cost Q _m (T) of the node R _m is defined by equation (3).

That is, the cost Q _m (T) at the node R _m is obtained by inverting the sign of the entropy at the node R _m (the total entropy of each label) (Q _m (T) ≦ 0).

Here, if the attribute of interest is useful for discrimination of phoneme breakage, there should be a great relationship between the value taken by the attribute and the presence or absence of the phoneme breakage label. That is, for useful attributes, the entropy is small (ie, the cost is high). In fact, in equation (3), if p ^ _{m, g} = 1, that is, there is an attribute that gives the label g with a probability of 100%, its entropy is 0 (and the cost is also 0), and the purity is maximum. Become. That is, the size of the cost indicates the importance of the attribute. For this reason, it is possible to configure a decision tree that is more compact and has high determination performance by arranging a node with a high cost above (near the root node).

  In addition, learning of a decision tree based on entropy has many merits, such as the number of branches following below may be two or more, and pruning to remove unnecessary branches from the constructed tree is easy.

  Hereinafter, the phoneme breaking section detection apparatus 200 will be described with reference to FIGS.

[Phoneme breaking section detection device 200]
As shown in FIG. 8, the phoneme breakage section detection device 200 includes a voice feature value generation unit 210, a voice recognition unit 230, a phoneme collation unit 250, a phoneme breakage section detection unit 270, and a recording unit 290. The recording unit 290 is a component that appropriately records information necessary for the processing of the phoneme breaking segment detection device 200. The phoneme breakage interval detection device 200 receives the recognition speech data as an input, and generates and outputs a maximum likelihood phoneme sequence with a phoneme breakage interval using the phoneme breakage determination tree learned by the phoneme breakage detection model learning device 100. The maximum likelihood phoneme sequence with phoneme collapse section is obtained by adding information of a segment (phoneme collapse segment) where the phoneme is collapsed to the maximum likelihood phoneme sequence as a recognition result by the speech recognition unit 230.

  The operation of the phoneme breaking section detection apparatus 200 will be described with reference to FIG. The voice feature quantity generation unit 210 divides the voice data for recognition into frames, generates a voice feature quantity, and outputs it (S210). The voice feature quantity generation unit 210 generates a voice feature quantity under the same conditions as the voice feature quantity generation in the voice feature quantity generation unit 101.

  The speech recognition unit 230 receives the speech feature value generated in S210 as an input, and corresponds to the maximum likelihood phoneme sequence that is the most likely phoneme sequence of the speech data for recognition and the speech section of each phoneme of the maximum likelihood phoneme sequence. A speech feature amount sequence that is a sequence of speech feature amounts of a frame is generated, and a maximum likelihood phoneme sequence and a speech feature amount sequence are output as recognition results (S230). An example of the recognition result is shown in FIG. An example of the configuration of the speech recognition unit 230 is shown in FIG. In this configuration, the decoder 221 uses each model (acoustic model 225, language model 227, dictionary 229) to generate a recognition result including the maximum likelihood phoneme sequence from the input speech feature. A DNN may be used for the configuration of the voice recognition unit 230.

  The phoneme matching unit 250 receives the maximum likelihood phoneme sequence and the speech feature amount sequence generated in S230 as input, and uses the phoneme collapse decision tree learned by the phoneme collapse detection model learning device 100, thereby including the phonemes included in the maximum likelihood phoneme sequence. A maximum likelihood phoneme sequence with a phoneme break label, which is a sequence of phoneme unit collation results obtained by adding a phoneme break label to a vowel phoneme label indicating a broken vowel phoneme, is generated and output (S250). The phoneme matching unit 250 will be described in detail with reference to FIGS. As shown in FIG. 12, phoneme matching unit 250 includes an estimated phoneme sequence generation unit 241 and a phoneme sequence comparison unit 243. The operation of the phoneme matching unit 250 will be described with reference to FIG.

The estimated phoneme sequence generation unit 241 receives the maximum likelihood phoneme sequence and the speech feature amount sequence, and generates an estimated phoneme sequence using a phoneme collapse decision tree (S241). The operation of the estimated phoneme sequence generation unit 241 will be described in detail (see FIG. 14). Figure 14 is a maximum likelihood phoneme sequence a ₁ ... a _K, the operation of estimating the phoneme sequence generator 241 for outputting an estimated phoneme sequence c ₁ ... c _K audio feature amount sequence as input b ₁ ... b _K Description (Where K is the length of the sequence (that is, the number of phonemes included in the maximum likelihood phoneme sequence)).

  When the phoneme label in the maximum likelihood phoneme sequence indicates a vowel, the estimated phoneme sequence generation unit 241 uses a phoneme collapse decision tree to determine from the speech feature amount corresponding to the vowel in the speech feature amount sequence. The generated phonemes are generated as estimated phonemes (S241-4a). On the other hand, if the phoneme label in the maximum likelihood phoneme sequence indicates a phoneme other than a vowel such as a consonant, the phoneme is generated as an estimated phoneme (S241-4b). An estimated phoneme sequence is generated by sequentially combining these estimated phonemes (S241-7).

The phoneme sequence comparison unit 243 receives the maximum likelihood phoneme sequence and the estimated phoneme sequence generated in S241, and generates a maximum likelihood phoneme sequence with a phoneme collapse label (S243). The operation of the phoneme sequence comparison unit 243 will be described in detail (see FIG. 15). Figure 15 is a maximum likelihood phoneme sequence a ₁ ... a _K, the phoneme collapse labeled maximum likelihood phoneme sequence estimation phoneme sequence as input c ₁ ... c _K phoneme sequence comparison unit 243 outputs the d ₁ ... d _K It is a flowchart explaining operation | movement.

  The phoneme sequence comparison unit 243 sequentially compares each phoneme label of the estimated phoneme sequence generated in S241 and each phoneme label of the maximum likelihood phoneme sequence (S243-3), and if they match, the phoneme label of the maximum likelihood phoneme sequence Only as a phoneme unit collation result (S243-4a). On the other hand, if they do not match, a pair of phoneme labels and phoneme collapse labels of the maximum likelihood phoneme sequence is generated as a phoneme unit collation result (S241-4b). By combining these phoneme unit matching results in order, a maximum likelihood phoneme sequence with a phoneme collapse label is generated and output as a matching result (S243-7). An example of the collation result is shown in FIG.

  The phoneme breakage interval detection unit 270 receives the phoneme breakage interval including the two or more connected phoneme groups to which the phoneme breakage labels are attached, using the maximum likelihood phoneme sequence with the phoneme breakage label generated in S250 as an input. The attached maximum likelihood phoneme sequence is generated and output (S270). Specifically, the maximum likelihood phoneme sequence with phoneme breaking sections is generated as follows. The maximum likelihood phoneme sequence with the phoneme breakage label is viewed in order from the top, and the vowel phoneme label (vowel phoneme label 1) to which the phoneme breakage label is assigned is found. If found, the next vowel phoneme label (vowel phoneme label 2) appearing is found and it is confirmed whether or not a phoneme collapse label is given. If a phoneme breakage label is assigned, a phoneme breakage label is assigned to all phoneme labels such as consonants in between (that is, phoneme labels are assigned to all phoneme labels from vowel phoneme label 1 to vowel phoneme label 2). A collapsing label is given). On the other hand, when the phoneme breakage label is not assigned, the phoneme breakage label is deleted from the found vowel phoneme label (vowel phoneme label 1) to which the phoneme breakage label is found. By repeating this procedure, a phoneme breakage section composed of two or more connected phoneme groups to which a phoneme breakage label is assigned is generated, and a maximum likelihood phoneme sequence with a phoneme breakup section is generated. Accordingly, when only the vowel phoneme label is viewed in the maximum likelihood phoneme sequence, if the phoneme collapse label is assigned to all three adjacent vowel phoneme labels, the third vowel phoneme from the first vowel phoneme label from the front is given. A phoneme collapse label is assigned to all phoneme labels up to the label. An example of the detection result is shown in FIG.

  According to the invention of this embodiment, it is possible to learn a phoneme collapse decision tree that is a model for detecting phoneme collapse of a vowel during speech recognition. Moreover, phoneme collapse can be detected quickly by determining only phoneme collapse of a vowel using the phoneme collapse decision tree. Furthermore, it is possible to detect a phoneme breakage section in which phoneme breakage occurs continuously with two or more phonemes, which significantly reduces the speech recognition rate.

<Modification>
It goes without saying that the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (5)

A phoneme label indicating a phoneme, a speech start time and an end time of the phoneme, a phoneme collapse label indicating that the phoneme is unclear A series of learning phoneme section information including a phoneme collapse flag that is one of the labels indicating that,
A learning phoneme label which is a phoneme label associated with a vowel phoneme label or a phoneme collapse label indicating a vowel phoneme included in the learning phoneme segment information from the learning speech data and the learning phoneme segment information sequence; A learning phoneme that extracts a phoneme collapse flag of the learning phoneme label and a learning phoneme segment speech feature that is a speech feature corresponding to a segment from the speech start time to the speech end time of the phoneme of the learning phoneme label An information extractor;
Phoneme collapse decision tree learning that learns a phoneme collapse decision tree that is a model for detecting phoneme collapse from the phoneme collapse flag of the learning phoneme label, the phoneme label of the learning and the phoneme segment speech feature value for learning. Phoneme breakage detection model learning device. A voice feature quantity generating unit for generating a voice feature quantity from the recognition voice data;
Using the speech feature amount, a maximum likelihood phoneme sequence that is a most likely phoneme sequence of the recognition speech data, and a sequence of speech feature amounts corresponding to a speech segment of each phoneme included in the maximum likelihood phoneme sequence A speech recognition unit for generating a speech feature amount sequence,
A phoneme collapse decision tree learned by the phoneme collapse detection model learning device according to claim 1 is used to cause phoneme collapse included in the maximum likelihood phoneme sequence from the maximum likelihood phoneme sequence and the speech feature amount sequence. A phoneme matching unit that generates a maximum likelihood phoneme sequence with a phoneme collapse label, which is a sequence of phoneme unit matching results obtained by adding a phoneme breakage label to a vowel phoneme label indicating a vowel phoneme,
Phoneme collapse interval detection for generating a maximum likelihood phoneme sequence with a phoneme break interval provided with a phoneme break interval composed of two or more connected phoneme groups to which the phoneme break label is assigned from the maximum likelihood phoneme sequence with the phoneme break label. Phoneme breakage section detection device including a part. A phoneme label indicating a phoneme, a speech start time and an end time of the phoneme, a phoneme collapse label indicating that the phoneme is unclear A series of learning phoneme section information including a phoneme collapse flag that is one of the labels indicating that,
A phoneme break section detecting device is associated with a vowel phoneme label or a phoneme break label indicating a vowel phoneme included in the learning phoneme section information from the learning speech data and the learning phoneme section information sequence. Learning phoneme label, learning phoneme segment flag of the learning phoneme label, and phoneme section speech for learning which is a speech feature amount corresponding to a section from the utterance start time to the utterance end time of the phoneme of the learning phoneme label A phoneme information extraction step for extracting a feature amount;
A phoneme breakage determination tree, which is a model for detecting phoneme breakage of a phoneme from the phoneme breakage flag of the learning phoneme label, the phoneme breakage flag of the learning phoneme label, and the learning phoneme interval speech feature amount. A phoneme breakage detection model learning method including a phoneme breakage decision tree learning step to be learned. A phoneme disruption section detection device that generates a speech feature amount from recognition speech data;
The phoneme breaking segment detection device uses the speech feature value to generate a maximum likelihood phoneme sequence that is a most likely phoneme sequence of the recognition speech data and a speech segment of each phoneme included in the maximum likelihood phoneme sequence. A speech recognition step for generating a speech feature amount sequence that is a sequence of speech feature amounts corresponding to
The phoneme breaking section detection device converts the maximum likelihood phoneme sequence and the speech feature amount sequence into the maximum likelihood phoneme sequence using the phoneme breaking decision tree learned by the phoneme breaking detection model learning method according to claim 3. A phoneme collation step for generating a phoneme breakage label-attached maximum likelihood phoneme sequence that is a series of phoneme unit collation results including a phoneme breakage label to a vowel phoneme label indicating a vowel phoneme that includes phoneme breakage,
The phoneme breakage interval detecting device is a maximum likelihood phoneme with a phoneme breakage interval in which a phoneme breakage interval comprising two or more connected phoneme groups to which the phoneme breakage label is attached is assigned from the maximum likelihood phoneme sequence with the phoneme breakage label. A phoneme breakage interval detection method including a phoneme breakage interval detection step for generating a sequence.   A program for causing a computer to function as the phoneme breakage detection model learning device according to claim 1 or the phoneme breakage detection unit according to claim 2.

Images