JP2017058507A - Speech recognition device, speech recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech recognition device with which it is possible to divide a speech recognition result into sentences without using a sentence boundary discriminator.SOLUTION: The speech recognition device includes a sentence boundary detection unit which, using an utterance-detected speech signal that is a speech signal for which utterance has been detected, a speech recognition result that includes a written expression and a word class generated by speech recognition of the utterance-detected speech signal, and the initial and final times of the utterance or pause of the utterance-detected speech signal, where an utterance section between pauses that are in prescribed length or longer is defined as an utterance, designates the whole or part of a pause in the utterance-detected speech signal that is equal to or longer than a predetermined shortest pause length as a sentence boundary candidate, classifies the sentence boundary candidate into a plurality of clusters on the basis of the feature value of the sentence boundary candidate, and detects the whole or part of a sentence boundary candidate included in a cluster that includes a sentence boundary candidate whose pause length is greater than or equal to a predetermined pause length threshold, as a sentence boundary.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a speech recognition apparatus, a speech recognition method, and a program for recognizing an input speech signal and dividing and outputting a speech recognition result for each sentence.

  When the input speech signal is converted into text by speech recognition, if the speech recognition result is divided for each sentence, when natural language processing (for example, document summarization) based on the sentence is applied to the speech recognition result It is easier to get correct results.

  Non-Patent Document 1 discloses a conventional technique for outputting a speech recognition result (hereinafter referred to as a speech recognition result with a sentence boundary) divided for each sentence. Hereinafter, the speech recognition apparatus of Non-Patent Document 1 will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of the speech recognition device 9 of Non-Patent Document 1. 2 and 3 are flowcharts showing the operation of the speech recognition apparatus 9 of Non-Patent Document 1, in which FIG. 2 is a flowchart showing the discriminator learning operation, and FIG. 3 is a flowchart showing the speech recognition operation.

  As shown in FIG. 1, the speech recognition device 9 of Non-Patent Document 1 includes a text corpus storage unit 90 with a sentence boundary label, a sentence boundary discriminator learning unit 91, a speech segment detection unit 92, a speech recognition unit 93, The sentence boundary identifying unit 94 is included. The text corpus storage unit 90 with sentence boundary labels stores a large number of text corpora (hereinafter referred to as text corpus with sentence boundary labels) to which correct sentence boundary positions (sentence boundary labels) are assigned. As shown in FIG. 2, the sentence boundary discriminator learning unit 91 learns a sentence boundary discriminator to be used for sentence boundary discrimination from a text corpus with a sentence boundary label (S91). As the sentence boundary classifier, an existing classifier capable of binary classification, such as Support Vector Machine (SVM) or a neural network, can be used. The sentence boundary classifier learning unit 91 learns a sentence boundary classifier using an existing algorithm corresponding to the type of classifier. It is assumed that step S91 is executed in advance prior to steps S92 to S94 described later.

  The speech section detection unit 92 detects a speech that is a speech section sandwiched between pauses (hereinafter also referred to as pause sections) longer than a predetermined pause length L1 (for example, about 500 ms) from the speech signal, and speech detected speech A signal is output (S92). The voice section detection unit 92 detects pauses and utterances in the voice signal using the existing VAD technology. The voice recognition unit 93 performs voice recognition on the utterance detected voice signal and outputs a voice recognition result for each utterance (S93).

  In the simplest method, the speech recognition result obtained in step S93 is regarded as a speech recognition result with a sentence boundary as it is. In other words, utterance = sentence. In reality, a plurality of sentences are uttered by one breath, and a case where a plurality of sentences are included in one utterance or a breath (a pose having a length equal to or longer than a predetermined pose length L1) is placed in the middle of the sentence. Since there is a case where one sentence is divided into a plurality of utterances, it is necessary to detect the sentence boundary again by the sentence boundary identifying unit 94 described later.

  The sentence boundary discriminating unit 94 uses the sentence boundary discriminator learned in advance in step S91, and uses the notation and part of speech of the word immediately before / after each word boundary in the speech recognition result as the feature quantity, and the word boundary is determined. It is identified whether or not it is a sentence boundary, and a speech recognition result with a sentence boundary is output (S94).

Sho Sobue, Keiko Yamamoto, Tetsugo Tamura, Satoru Hayami, "Evaluation of Discrimination Model for Estimating Sentence Boundary of Speech Recognition Results", Proc. Of the 15th Annual Conference of the Language Processing Society of Japan, Language Processing Society of Japan, 2009 March, pp.582-585

  Since the prior art uses a sentence boundary classifier that has been learned in advance, sentence boundaries that have different characteristics from the sentence boundary characteristics (previous / immediate word notation and part of speech) in the text corpus used for learning are correctly identified. Can not do it. In particular, there are various variations in sentence boundary characteristics in conversational speech between humans. For example, the expression at the sentence boundary changes variously depending on the degree of familiarity with the conversation partner, whether or not the place to speak is a formal place, the habit of the sentence end expression for each speaker, and the like. It is difficult to cover such variations in advance. For example, sentence boundaries are almost undetectable with a specific user's voice, which may reduce the accuracy of sentence boundary identification and reduce system convenience. .

  In addition, for learning a sentence boundary classifier, it is common to use data in which each word boundary of a speech recognition result text is manually assigned a label indicating whether it is a sentence boundary (sentence boundary label). Since a recognition error is included, it is necessary to work while listening to the original voice in order to give a correct sentence boundary label, which increases the system construction cost due to the large amount of work.

  Accordingly, an object of the present invention is to provide a speech recognition apparatus that can divide a speech recognition result for each sentence without using a sentence boundary discriminator.

  The speech recognition apparatus of the present invention includes a sentence boundary detection unit. An utterance section sandwiched between poses longer than a predetermined pose length is called utterance, and the sentence boundary detection unit recognizes the utterance detected voice signal, which is an utterance detected voice signal, and the utterance detected voice signal. The speech recognition result including the notation and the part of speech generated in this way, and the utterance or pause start and end times of the utterance detected speech signal are used. The sentence boundary detection unit sets a part or all of a pose having a length equal to or longer than a predetermined shortest pose length in a speech signal whose utterance has been detected as a sentence boundary candidate, and selects a plurality of sentence boundary candidates based on the feature amount of the sentence boundary candidate. And a part or all of sentence boundary candidates included in the cluster including sentence boundary candidates having a pause length equal to or greater than a predetermined pause length threshold are detected as sentence boundaries.

  According to the speech recognition apparatus of the present invention, the speech recognition result can be divided for each sentence without using a sentence boundary discriminator.

The block diagram which shows the structure of the speech recognition apparatus of a nonpatent literature 1. FIG. 10 is a flowchart showing a discriminator learning operation of the speech recognition apparatus of Non-Patent Document 1. The float which shows the speech recognition operation | movement of the speech recognition apparatus of a nonpatent literature 1. 1 is a block diagram illustrating a configuration of a voice recognition device according to Embodiment 1. FIG. 3 is a flowchart showing the operation of the speech recognition apparatus according to the first embodiment. The figure which shows the output example of the audio | voice area detection part of the speech recognition apparatus of Example 1. FIG. The figure which shows the example of an output of the speech recognition part of the speech recognition apparatus of Example 1. FIG. FIG. 3 is a block diagram illustrating a configuration of a sentence boundary detection unit of the speech recognition apparatus according to the first embodiment. 6 is a flowchart illustrating an operation of a sentence boundary detection unit of the speech recognition apparatus according to the first embodiment. The figure which shows the example of an output of the sentence boundary feature extraction part of the speech recognition apparatus of Example 1. FIG. The figure which shows the example of an output of the sentence boundary flag provision part of the speech recognition apparatus of Example 1. FIG.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  Hereinafter, the configuration and operation of the speech recognition apparatus according to the first embodiment will be described with reference to FIGS. 4 and 5. FIG. 4 is a block diagram showing the configuration of the speech recognition apparatus 10 of this embodiment. FIG. 5 is a flowchart showing the operation of the speech recognition apparatus 10 of this embodiment. As shown in FIG. 4, the speech recognition apparatus 10 includes a speech section detection unit 102, a speech recognition unit 103, and a sentence boundary detection unit 104. Since the speech recognition apparatus 10 according to the present embodiment does not use a sentence boundary discriminator, the text corpus storage unit 90 with sentence boundary label and the sentence boundary discriminator learning unit 91 described above are unnecessary.

  The voice section detection unit 102 detects the utterance from the voice signal by using the existing VAD technique, and outputs the utterance detected voice signal (S102). The voice section detection unit 102 outputs the start and end times of the utterance or pause (the beginning of the voice signal is set to time 0) together with the utterance detected voice signal. Since the start time of the utterance is equal to the end time of the pose immediately before the utterance and the end time of the utterance is equal to the start time of the pose immediately after the utterance, the start and end times of the utterance or the start and end times of the pose At least one is enough. The start / end times of utterances or pauses are stored in any storage area of the speech recognition apparatus 10 in a table format data structure shown in FIG. FIG. 6 is a diagram illustrating an output example of the speech segment detection unit 102.

  Next, the speech recognition unit 103 recognizes the speech signal that has been detected for speech, and outputs a speech recognition result including the start and end times of the speech or pause, the notation, and the part of speech (S103). The speech recognition result is desirably a certain amount or more (for example, about 100 utterances or more) in order to sufficiently grasp the tendency of the speaker in the sentence boundary candidate clustering described later.

  Note that the speech recognition unit 103 outputs the start and end times due to a short pause that the speech segment detection unit 102 did not detect, in addition to the start and end times output by the speech segment detection unit 102.

  FIG. 7 shows an output example of the voice recognition unit 103. In the output example of FIG. 7, pauses with a start time of 8.66 seconds and an end time of 9.03 seconds are short pauses not included in the output example of FIG. 6 (predetermined pauses that are not detected in step S102). Pose of length L1 or less).

  In a general VAD technique, a pose (and an utterance sandwiched between poses) longer than a predetermined pose length L1 that is normally set to about 500 ms is detected. Therefore, when the speech section detection unit 102 is configured using a general VAD technique, the speech section detection unit 102 does not detect a short pose shorter than the predetermined pause length L1. For this reason, the speech recognition unit 103 detects a short pose shorter than the predetermined pose length L1.

  Next, the sentence boundary detection unit 104 sets a part or all of poses having a length equal to or longer than a predetermined shortest pose length T1 in the speech signal that has been detected as speech as a sentence boundary candidate, and based on the feature amount of the sentence boundary candidate. Sentence boundary candidates are classified into a plurality of clusters, and a part or all of sentence boundary candidates included in a cluster including sentence boundary candidates having a pause length greater than or equal to a predetermined pause length threshold T2 are detected as sentence boundaries (S104). ).

  Hereinafter, a detailed configuration example and an operation example of the sentence boundary detection unit 104 will be described with reference to FIGS. 8 and 9. FIG. 8 is a block diagram illustrating a configuration of the sentence boundary detection unit 104 of the speech recognition apparatus 10 according to the present embodiment. FIG. 9 is a flowchart showing the operation of the sentence boundary detection unit 104 of the speech recognition apparatus 10 of this embodiment. As shown in FIG. 8, the sentence boundary detecting unit 104 includes a sentence boundary candidate specifying unit 1041, a sentence boundary feature extracting unit 1042, a sentence boundary candidate clustering unit 1043, and a sentence boundary flag adding unit 1044.

  The sentence boundary candidate specifying unit 1041 sets all poses (excluding the initial pose) longer than a predetermined shortest pose length T1 as sentence boundary candidates, assigns a sentence boundary candidate flag to the sentence boundary candidate, A speech recognition result with a candidate flag is output (S1041). T1 is a parameter that defines the shortest value of the length of the pose placed at the sentence boundary, and is usually set in advance to a value of about T1 = 200 ms. An example in which a sentence boundary candidate flag is added to the speech recognition result in FIG. 7 with T1 = 200 ms is shown in the second column from the right in FIG. FIG. 10 is a diagram illustrating an output example of a sentence boundary feature extraction unit 1042 described later. The sentence boundary candidate specifying unit 1041 may add identification information for specifying a sentence boundary candidate for each extracted sentence boundary candidate.

  When the shortest pause length T1 is set to a value shorter than the predetermined pause length L1 in the speech segment detection unit 102 (that is, T1 <L1), all poses detected by the speech segment detection unit 102 are sentence boundary candidates. On the other hand, some or all of the short poses detected by the speech recognition unit 103 are sentence boundary candidates.

  The sentence boundary feature extraction unit 1042 uses the predetermined values N and M (N and M may be integers greater than or equal to 1 and N = M) to determine the N words immediately before and M words immediately after the sentence boundary candidates. A set of notations and parts of speech are extracted as sentence boundary features of the sentence boundary candidate and output (S1042). The values N and M are parameters that specify the range in which the sentence boundary feature appears. For example, N = M = 2 is set. Note that there is no immediately preceding word at the beginning of the speech recognition result and no immediately following word at the end. In this case, the notation and part of speech of a nonexistent word are not acquired. An example of sentence boundary features extracted from each sentence boundary candidate with N = M = 2 is shown in the rightmost column of FIG.

  The sentence boundary feature extraction unit 1042 may store the identification information for specifying the sentence boundary candidate and the obtained sentence boundary feature in association with each other. For example, a column of “sentence boundary candidate identification information” is further added to the right end of the table of FIG. 10, and a flag is set in the column of sentence boundary candidate flags (indicated by a circle in the figure). A serial number (for example, a sentence boundary candidate number) may be given as identification information for identifying a sentence boundary candidate. Alternatively, identification information may be given to the column of sentence boundary candidate flags, and the identification information itself may have a function as a flag.

  The sentence boundary candidate clustering unit 1043 classifies the sentence boundary candidates based on the similarity between the sentence boundary features, and outputs a sentence boundary candidate cluster (hereinafter also simply referred to as a cluster) as a classification result (S1043).

  The sentence boundary candidate clustering unit 1043 first calculates the similarity between all sentence boundary candidate pairs. As the similarity between sentence boundary candidates, the cosine similarity between corresponding sentence boundary features can be used. For example, when the sentence boundary feature of the first sentence boundary candidate is F1 and the sentence boundary feature of the second sentence boundary candidate is F2, the similarity S between the first sentence boundary candidate and the second sentence boundary candidate is expressed as follows: (Where | F | represents the number of elements in the set F).

The sentence boundary candidate clustering unit 1043 associates the calculated similarity between sentence boundary candidates with identification information for specifying two sentence boundary candidates used for similarity calculation, executes an existing clustering technique, Obtain boundary candidate clusters. The sentence boundary candidate clustering unit 1043 can use any method as long as it is a clustering method that receives the similarity between data and does not need to set the number of clusters (the number of clusters is automatically determined). The Chinese Whispers method described in Patent Document 1 can be used. An example of sentence boundary candidate clusters acquired in step S1043 is shown in the second column from the right in FIG. FIG. 11 is a diagram illustrating an output example of a sentence boundary flag adding unit 1044 described later.
(Reference Non-Patent Document 1: Chris Biemann, “Chinese Whispers-an Efficient Graph Clustering Algorithm and its Application to Natural Language Processing Problems,” in Proceedings of the first workshop on graph based methods for natural language processing, pp.73-80, 2006.)
In the example of FIG. 11, a number indicating which cluster belongs to each sentence boundary candidate is assigned. In the example of FIG. 11, two sentence boundary candidates assigned with cluster number 1 belong to the same cluster. By the clustering technique, “identification information for identifying sentence boundary candidates” included in each cluster is stored in association with each identification information for identifying each cluster.

  Next, the sentence boundary flag assigning unit 1044 sets a sentence boundary cluster including sentence boundary candidates having a pose length equal to or greater than a predetermined pause length threshold T2 among sentence boundary candidate clusters, and includes sentence boundaries included in the sentence boundary cluster. Some or all of the candidates are detected as sentence boundaries, and a speech recognition result with sentence boundaries is output (S1044).

  Specifically, the sentence boundary flag assigning unit 1044 identifies, for each cluster, the length of the pose that has become the sentence boundary candidate from the identification information that identifies the sentence boundary candidate included in the cluster, and the length of the pose. Is determined to be greater than or equal to the pause length threshold T2. The sentence boundary flag assigning unit 1044 determines that the cluster is a sentence boundary cluster if there is even one pause having a pause length of T2 or more. The pause length threshold T2 is a parameter that defines the pause length that can be regarded as a sentence boundary with a high probability. For example, T2 can be set to about 1000 ms. In the example of FIG. 11, when T2 = 1000 ms, the length of the pose to which cluster number 2 is assigned is equal to or greater than T2 (1000 ms), so the sentence boundary flag assigning unit 1044 Is a sentence boundary cluster. Normally, the pause length threshold T2 is set to a value longer than the pause length L1 detected by the speech section detection unit 102 (92). For example, L1 = 500 ms and T2 = 1000 ms may be set.

  The sentence boundary flag assigning unit 1044 detects sentence boundary candidates included in the sentence boundary cluster as sentence boundaries, and assigns sentence boundary flags. In the example of FIG. 11, sentence boundary flags are assigned to all sentence boundary candidates belonging to cluster number 2. The sentence boundary flag assigning unit 1044 outputs a speech recognition result with sentence boundaries as a final result.

  As described above, the sentence boundary detection unit 104 outputs a speech recognition result to which a sentence boundary is added without using a sentence boundary discriminator. Since the sentence boundary detection unit 104 analyzes the target speech recognition result itself and performs sentence boundary detection, the sentence boundary is correctly determined in consideration of the conversation partner, the place of speech, and the habit of expression for each speaker in the speech recognition result. Can be detected.

<Effects produced by the speech recognition apparatus 10 according to the first embodiment>
According to the speech recognition apparatus 10 of the present embodiment, the speech recognition result can be correctly divided for each sentence without using a sentence boundary discriminator learned in advance, and sentence boundary accuracy can be obtained with the voice of the specific user described above. As a result, the convenience of the system for the user is improved. In addition, since it is not necessary to create a speech recognition result that is manually assigned with a correct sentence boundary label, which is used for learning of a sentence boundary classifier, the cost of the system operator can be reduced.

<Technical points of the speech recognition apparatus 10 of the first embodiment>
The technical point of the speech recognition apparatus 10 of the present embodiment is that there is a tendency that “a pose longer than a preset pause length threshold T2 (for example, 1000 ms) is a sentence boundary” and “one speech recognition result” In this case, the sentence boundary is detected by utilizing the tendency that the same sentence boundary feature appears repeatedly (that is, if the conversation partner / speaking place / speaker are the same).

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (6)

An utterance section between poses longer than a predetermined pose length is called utterance,
The speech detected speech signal that is a speech signal in which the speech has been detected, a speech recognition result including speech and a part of speech generated by speech recognition of the speech detected speech signal, and the speech of the speech detected speech signal Or, using the start and end times of the pause,
A part or all of a pose having a length equal to or longer than a predetermined shortest pose length in the speech signal that has been detected in speech is set as a sentence boundary candidate, and the sentence boundary candidate is classified into a plurality of clusters based on the feature amount of the sentence boundary candidate. A speech recognition apparatus including a sentence boundary detection unit that classifies and detects part or all of sentence boundary candidates included in a cluster including sentence boundary candidates that have a pause length equal to or greater than a predetermined pause length threshold as sentence boundaries. The speech recognition device according to claim 1,
The sentence boundary detection unit
A sentence boundary candidate clustering unit that classifies the sentence boundary candidates into a plurality of clusters based on a notation of a predetermined number of words immediately before and after the sentence boundary candidates and a similarity between sentence boundary features that are a set of parts of speech. Including speech recognition device. The speech recognition device according to claim 1 or 2,
The sentence boundary detection unit
A speech recognition apparatus that outputs a speech recognition result with a sentence boundary in which the sentence boundary is added to the speech recognition result. The speech recognition device according to any one of claims 1 to 3,
A voice section detecting unit for detecting the utterance from a voice signal and outputting the utterance detected voice signal;
A speech recognition apparatus comprising: a speech recognition unit that recognizes the speech signal with the speech detected and outputs the start and end times and the speech recognition result. An utterance section between poses longer than a predetermined pose length is called utterance,
The speech detected speech signal that is a speech signal in which the speech has been detected, a speech recognition result including speech and a part of speech generated by speech recognition of the speech detected speech signal, and the speech of the speech detected speech signal Or, using the start and end times of the pause,
A part or all of a pose having a length equal to or longer than a predetermined shortest pose length in the speech signal that has been detected in speech is set as a sentence boundary candidate, and the sentence boundary candidate is classified into a plurality of clusters based on the feature amount of the sentence boundary candidate. A speech recognition method in which the speech recognition apparatus executes a step of classifying and detecting a part or all of sentence boundary candidates included in a cluster including sentence boundary candidates having a pause length equal to or greater than a predetermined pause length threshold as sentence boundaries .   A program for causing a computer to function as the voice recognition device according to any one of claims 1 to 4.

Images