JP2015169834A - Speech analysis method, speech analysis program, and speech analysis device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech analysis method, a speech analysis program, and a speech analysis device for accurately correlating voice data to text.SOLUTION: The speech analysis method has a computer execute the steps of: dividing voice data into a plurality of sections; extracting a plurality of words from text; performing voice recognition between the plurality of sections on the basis of the extracted plurality of words; selecting, from the extracted plurality of words, one or more words having the easiness or reliability of voice recognition higher than or equal to a certain level, as a reference word; selecting, from the plurality of sections, a section in which the voice of the reference word is recognized; selecting, from the selected section, one or more sections in which the voice of a word other than the reference word is recognized, as a first section corresponding to the text; and detecting, beginning with a section adjacent to the first section, a range of sections in which the voice of a word other than the reference word is recognized, and selecting a section within the range as a second section corresponding to the text.

Description

  The present invention relates to a voice analysis method, a voice analysis program, and a voice analysis apparatus.

  With the advancement of audio and video recording / playback technology, the minutes system that allows users to refer to the minutes as well as to reproduce the audio data or video data recorded in the meeting with a high sense of presence. . Users (such as those who are absent from the conference) can know the detailed content of the conference by using the minutes system.

  Regarding the minutes system, for example, Patent Document 1 describes that audio data or video data is associated with document data based on the similarity between words.

International Publication No. 2005/027092

  By associating the audio data of the meeting with the text of the minutes, the user can refer to the description location of the minutes corresponding to the audio contents at any time, so that the convenience of the minutes system is improved. However, if the correspondence of the audio data is not accurate, the user becomes difficult to grasp the contents of the meeting. Such a problem exists not only in the minutes system but also in other systems that record and reproduce lectures and lectures.

  Accordingly, the present invention has been made in view of the above problems, and an object thereof is to provide a speech analysis method, a speech analysis program, and a speech analysis device that accurately associate speech data with text.

  The speech analysis method described in this specification includes a step of dividing speech data into a plurality of sections, a step of extracting a plurality of words from text, and speech recognition of the plurality of sections based on the extracted plurality of words. Respectively, a step of selecting one or more words having a certain level of ease of speech recognition or certainty as a reference word from the plurality of extracted words, and the reference word from the plurality of sections. Selecting a section in which the voice of the word is recognized, and one or more sections in which the voices of words other than the reference word are recognized among the plurality of words from the selected section corresponding to the text. The step of selecting as one section, and the section adjacent to the first section as a starting point, the range of the section in which the speech of words other than the reference word is recognized among the plurality of words, Between, and selecting as a second section corresponding to said text, computer executes.

  The speech analysis program described in the present specification divides speech data into a plurality of sections, extracts a plurality of words from text, performs speech recognition of the plurality of sections based on the extracted plurality of words, From the plurality of extracted words, one or more words having a certain level of ease or certainty of speech recognition are selected as reference words, and a section in which the voice of the reference word is recognized from the plurality of sections. Selecting one or more sections in which the speech of words other than the reference word among the plurality of words is recognized as the first section corresponding to the text from the selected section, and the first section And a range of a section in which the speech of a word other than the reference word is recognized among the plurality of words, and a section within the range is defined as a second section corresponding to the text. Selecting Te, and processing, it causes the computer to execute.

  The speech analysis device described in the present specification includes a dividing unit that divides speech data into a plurality of sections, an extraction unit that extracts a plurality of words from text, and a plurality of sections based on the extracted plurality of words. A speech recognition processing unit that performs speech recognition, a first selection unit that selects, as a reference word, one or more words that have a certain level of ease or certainty of speech recognition from the plurality of extracted words; A second selection unit that selects a section in which the voice of the reference word is recognized from a plurality of sections, and a voice of a word other than the reference word among the plurality of words is recognized from the selected section. A voice of words other than the reference word among the plurality of words starts from a third selection unit that selects the above section as a first section corresponding to the text, and a section adjacent to the first section. Recognized Detecting a range between, a section within the range, and a fourth selection unit for selecting as a second section corresponding to the text.

  Audio data can be accurately associated with text.

It is a block diagram which shows the speech analyzer which concerns on an Example. It is a block diagram which shows an example of a function structure of a speech analyzer. It is a figure which shows an example of document data and the document data after a division | segmentation. It is a table | surface which shows an example of an extraction word database. It is a figure which shows the example of a division | segmentation of audio | voice data. It is a figure which shows audio | voice data and the audio | voice data after a division | segmentation. It is a figure which shows the comparative example of matching of the area | region of a text and audio | voice data. It is a figure which shows the other comparative example of matching of the area | region of a text and audio | voice data. It is a figure which shows an example of selection of the candidate of a main area. It is a figure which shows an example of selection of the main area. It is a table | surface which shows an example of an analysis result database. It is a figure which shows an example of selection of a subsection. It is a flowchart of the speech analysis program which concerns on an Example.

  FIG. 1 is a configuration diagram illustrating a speech analysis apparatus 1 according to an embodiment. The voice analysis apparatus 1 analyzes voice data such as a meeting by voice recognition processing and associates it with document data such as minutes.

  The voice analysis device 1 is a computer device such as a server device. The voice analysis apparatus 1 includes a CPU 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, an HDD 13, a communication processing unit 14, a portable storage medium drive 15, an input processing unit 16, an image processing unit 17, and the like. It has.

  The CPU 10 is arithmetic processing means, and executes a voice analysis method according to a voice analysis program. The CPU 10 is connected to the units 11 to 17 via the bus 18. Note that the voice analysis device 1 is not limited to one that operates by software, and hardware such as an application specific integrated circuit may be used instead of the CPU 10.

  The RAM 12 is used as a working memory for the CPU 10. The ROM 11 and HDD 13 are used as storage means for storing a voice analysis program for operating the CPU 10. The communication processing unit 14 is, for example, a network card, and is a communication unit that communicates with other devices via a network such as a LAN.

  The portable storage medium drive 15 is a device that writes information to and reads information from the portable storage medium 150. Examples of the portable storage medium 150 include a USB memory (USB: Universal Serial Bus), a CD-R (Compact Disc Recordable), and a memory card. Note that the voice analysis program may be stored in the portable storage medium 150.

  The voice analysis apparatus 1 further includes an input device 160 for performing an information input operation and a display 170 for displaying an image. The input device 160 is input means such as a keyboard and a mouse, and input information is output to the CPU 10 via the input processing unit 16. The display 170 is image display means such as a liquid crystal display, and displayed image data is output from the CPU 10 to the display via the image processing unit 17. In addition, it can replace with the input device 160 and the display 170, and can also use devices, such as a touchscreen provided with these functions.

  The CPU 10 executes a program stored in the ROM 11 or the HDD 13 or a program read from the portable storage medium 150 by the portable storage medium drive 15. This program includes not only the OS (Operating System) but also the above-described voice analysis program. Note that the program may be downloaded from another apparatus via the communication processing unit 14.

  When the CPU 10 executes the voice analysis program, a plurality of functions are formed. Below, the function of the speech analyzer 1 will be described.

  FIG. 2 is a configuration diagram illustrating an example of a functional configuration of the voice analysis device 1. FIG. 2 shows an example of functions formed in the CPU 10 and information stored in the HDD 13.

  The CPU 10 includes a document data dividing unit 101, a word extracting unit (extracting unit) 102, a reference word selecting unit (first selecting unit) 103, a voice recognition processing unit 104, and a voice data dividing unit (dividing unit) 105. Have The CPU 10 further includes a main section candidate selection section (second selection section) 106, a main section selection section (third selection section) 107, and a sub section selection section (fourth selection section) 108. The HDD 13 includes a document database (document DB) 131, a dictionary database (dictionary DB) 132, an extracted word database (extracted word DB) 133, a speech database (speech DB) 134, and an analysis result database (analysis result DB). 135 are stored.

  Each database 131 to 135 may be stored in a storage unit (HDD or the like) of another device. In this case, the CPU 10 accesses the databases 131 to 135 from the communication processing unit 14 via the network.

  The document DB 131 includes document data input from the input device 160 or document data input from the portable storage medium 150 via the portable storage medium drive 15. In this example, the minutes of the meeting are listed as the document data, but the present invention is not limited to this.

  The document data dividing unit 101 divides the document data in the document DB 131 into a plurality of texts (text data). The document data dividing unit 101 divides the document data, for example, by generating a plurality of individual texts from the document data, or by adding a mark for separating the texts in the document to the document data. To do.

  FIG. 3A and FIG. 3B show examples of document data and divided document data, respectively. The document data dividing unit 101 divides the document data using, for example, line breaks, indentation (indentation), or blank lines in the minutes of the minutes as boundaries between texts. As a result, the text of the minutes “About the summer training camp, next Friday ...” is divided into a plurality of texts (A) to (C). The document data division unit 101 notifies the word extraction unit 102 of the completion of the document data division processing.

  When the word extraction unit 102 receives a notification of completion of the document data division process, the word extraction unit 102 extracts words from the texts (A) to (C) by referring to the dictionary DB 132. A plurality of words are registered in the dictionary DB 132. The word extraction unit 102 registers the extracted words in the extracted word DB 133 for each of the texts (A) to (C).

  FIG. 4 shows an example of the extracted word DB 133. In FIG. 4, “text” indicates text identification information (A to C in this example), and “word” indicates an extracted word. The “reference flag” and “voice data section” will be described later.

  In this example, “summer season”, “training camp”, “opinion”, “recruitment”, “Friday”, and “Kagawa” are extracted from the text (A). Also, from the text (B), “autumn”, “test period”, “building 3”, “in / out”, “case”, “previous”, “Sagawa”, and “contact” are extracted, and the text (C ), “Next time”, “next week”, and “Friday” are extracted. As will be described later, the extracted words are used for speech recognition processing of speech data and association with each text (A) to (C). The word extraction unit 102 notifies the reference word selection unit 103 of completion of the word extraction process.

  When the reference word selection unit 103 receives a notification of completion of the word extraction process, the reference word selection unit 103 has, for each text (A) to (C), one or more words that have a certain level of ease or certainty of speech recognition from the extracted words. Are selected as reference words. As will be described later, the reference word is used to select a main section of speech data corresponding to the texts (A) to (C).

  More specifically, the reference word selection unit 103 selects, as a reference word, one or more words having the largest number of mora or a certain number or more from the plurality of extracted words. Here, the number of mora is the number of syllables, that is, the number of vowels and “n” included in a word.

  In this example, the reference word selection unit 103 selects a word having a mora number of 5 or more as a reference word. Therefore, “Friday” (number of mora = 5) in the texts (A) and (B) and “No. 3 building” (number of mora = 6) in the text (C) are selected as reference words. The reference word selection unit 103 registers the selected reference word in the extracted word DB 133.

  The “reference flag” in the extracted word DB 133 illustrated in FIG. 4 indicates that the word is a reference word when “1”, and indicates that the word is not a reference word when “0”. . Therefore, in the text (A) column, “reference flag” of “Friday” indicates “1”, and “reference flag” of other words indicates “0”. In the text (B) column, “reference flag” of “No. 3 building” indicates “1”, and “reference flag” of other words indicates “0”. In the text (C) column, “Friday” “reference flag” indicates “1” and other words “reference flag” indicates “0”.

  Thus, by using the number of mora, it is possible to easily detect a word that can be easily recognized from a plurality of words extracted from the texts (A) to (C). The reference word selection means is not limited to this.

  The reference word selection unit 103 may select, as a reference word, one or more words having the highest score indicating the likelihood of words in speech recognition or a certain value or more. The score is calculated using, for example, a reciprocal of a distance between a word model and a speech parameter calculated using dynamic programming or the like in a model using a probability model. Therefore, in this case, the reference word selection unit 103 acquires a word score from the speech recognition processing unit 104. Thereby, the word with high certainty of the result of speech recognition can be easily detected from a plurality of words extracted from the texts (A) to (C). Note that the reference word selection unit 103 may select a reference word based on both the number of mora and the score. The reference word selection unit 103 notifies the main section candidate selection unit 106 of the completion of the reference word selection process.

  The voice DB 134 includes voice data input from another device via the communication processing unit 14 or voice data input from the portable storage medium 150 via the portable storage medium drive 15. In this example, the audio data of the conference recorded by an IC (Integrated Circuit) recorder or the like is exemplified as the audio data. However, the audio data is not limited to this, and may be audio data included in video data, for example. Note that the content (minutes) of the document data in the document DB 131 is created according to the content of the conference indicated by the audio data.

  The audio data dividing unit 105 divides the audio data in the audio DB 134 into a plurality of sections. The audio data dividing unit 105 divides the audio data, for example, by separating the audio data into individual data for each section, or by adding a mark for separating the sections to the audio data.

  FIG. 5A and FIG. 5B show a division example (1) and a division example (2) of audio data, respectively. The waveform of the audio data shown in FIGS. 5A and 5B shows a change in the strength of the sound with respect to time.

  In the division example (1), the voice data division unit 105 divides the voice data every certain time T, thereby acquiring the sections (1) to (3) of the voice data. According to the method of division example (1), the data amount of each section (1) to (3) of the audio data can be made equal.

  Also, in the division example (2), the voice data division unit 105 detects a silent section M that continues for a predetermined time or more from the voice data, and divides the voice data by the silent section M to thereby obtain a section (1) of the voice data. Obtain (3). According to the method of the division example (2), the data amount of each section (1) to (3) of the audio data can be reduced to about one sentence.

  As described above, since the division of the voice data is not performed corresponding to the division of the document data, the voice analysis apparatus 1 does not necessarily associate the sections of the text and the voice data in a one-to-one relationship. . That is, there are cases where a plurality of sections of speech data are associated with one text, and sections where speech data are different are associated with the same text.

  FIGS. 6A and 6B show the audio data and the divided audio data, respectively. In this example, the audio data dividing unit 105 divides the audio data by the method of the above division example (2), but is not limited thereto, and for example, the method of the above division example (1) may be used. . In FIG. 6A and FIG. 6B, the voice of the voice data is expressed as characters.

  The voice data dividing unit 105 divides the voice indicated by the voice data “Summer training camp will be held this year. The next meeting will be held next Friday” into sections (1) to (4). . Since the division process is performed by detecting a silent section of a certain time, the voice is divided in units of one sentence.

  For this reason, the voice data section (1) includes the voice of “I will hold a summer camp this year”, and the voice data section (2) contains “Destination, what I want to do… Please tell me ". In addition, the voice data section (3) includes the voice “Please contact me during the fall test period.” The voice data section (4) The meeting will be held next Friday. "

  The speech recognition processing unit 104 performs speech recognition of a plurality of sections obtained by dividing speech data based on a plurality of words extracted from the texts (A) to (C). For the speech recognition process, for example, a probabilistic model method using a hidden Markov model is used. The speech recognition processing unit 104 refers to the extracted word DB 133 and determines whether the speech of the extracted word is recognized in the sections (1) to (4) of the speech data for each of the texts (A) to (C). And the determination result is registered in the extracted word DB 133.

  In the extracted word DB 133 illustrated in FIG. 4, the “1” to “4” columns in the “voice data section” column indicate whether or not each word is recognized in the voice data sections (1) to (4) (“1 ": With voice recognition," 0 ": without voice recognition). For example, the voices “Summer” and “Training Camp” in the text (A) are recognized in the section (1) of the voice data, but are not recognized in the other sections (2) to (4). Therefore, the “1” field in the “voice data section” field corresponding to “summer” and “camp” in the text (A) indicates “1”, and “2” in the “voice data section” field. The “4” columns indicate “0” respectively. The voice recognition processing unit 104 notifies the main section candidate selection unit 106 of the completion of the voice recognition process.

  The speech analysis apparatus 1 refers to the extracted word DB 133 in which the speech recognition determination results of the sections (1) to (4) of the speech data are registered, so that the speech data of each speech (A) to (C) is stored. Associate sections (1) to (4). Here, if the association is performed based only on the above reference word, the following problem occurs.

  FIG. 7 shows a comparative example of correspondence between texts (A) and (B) and sections (1) to (3) of voice data. In this example, the section (2) of the voice data includes the voice of the reference word “Friday” (see the dotted frame), and is associated with the text (A) including “Friday” (see the dotted frame) (circle). See sign). The section (3) of the voice data includes the voice of the reference word “Building No. 3” (see the dotted frame) and is associated with the text (B) including “No. 3 Building” (see the dotted frame) (see the circle). ).

  However, since the voice data section (1) does not include the voices of the reference words “Friday” and “No. 3 building”, the voice content of the voice data section (1) is the text (A). Despite being in common with the content, it cannot be associated with the text (A). Therefore, in this example, the range of the voice data section associated with the text (A) is not accurately specified.

  FIG. 8 shows another comparative example in which texts (A) to (C) are associated with voice data sections (1) to (4). In this example, the section (3) of the voice data includes the voice of the reference word “Building No. 3” (refer to the dotted frame) and is therefore associated with the text (B) including “No. 3 Building” (refer to the dotted frame). (See circle).

  However, since the reference word “Friday” (see the dotted frame) is included in the plurality of texts (A) and (B) and the plurality of sections (2) and (4) of the voice data, the section (2 ), (4) is difficult to determine which of the texts (A), (B) is associated (see “?”). Thus, when the reference word overlaps in a plurality of texts (A) and (B) and a plurality of sections (2) and (4) of speech data, it is difficult to associate the sections of the text and speech data.

  Therefore, when only the reference word is used, the sections of the texts (A) to (C) and the voice data (1) to (4) cannot be accurately associated. For this reason, as will be described below, the speech analysis apparatus 1 converts a section in which other words are also speech-recognized from a section of speech data in which a reference word in the text is speech-recognized, to a main section (first section) corresponding to the text. 1 section). Furthermore, the speech analysis apparatus 1 detects a range of a section in which the speech of a word other than the reference word is recognized starting from a section adjacent to the main section, and the section within the range is defined as a sub-section (first section) corresponding to the text. 2 section). Thereby, the speech analyzer 1 associates the main section and the sub section of the sound data accurately with the text.

  When the main section candidate selection unit 106 receives each notification of completion of the voice recognition process and the selection process of the reference word, the main section candidate selection unit 106 selects a section in which the voice of the reference word is recognized from the plurality of sections (1) to (4) of the voice data. Select as a candidate for the main section. More specifically, the main section candidate selection unit 106 refers to the extracted word DB 133 to recognize words whose “reference flag” is “1” for the sections (1) to (4) of the voice data. The presence or absence of is detected. Note that selection of candidates for the main section is performed for each of the texts (A) to (C).

  More specifically, the main section candidate selection unit 106 selects “1” to “4” in the “voice data section” corresponding to the word (reference word) whose “reference flag” is “1”. A column indicating “1” is detected. The main section candidate selection unit 106 detects, for example, the “2” field and the “4” field of the “voice data section” corresponding to the reference word “Friday” in the text (A).

  FIG. 9 shows an example of main section candidate selection. FIG. 9 shows how main section candidates are selected after the audio data shown in FIG. 6 is divided.

  In this example, the main section candidate selection unit 106 selects a main section candidate corresponding to the text (A). The reference word “Friday” (see the dotted line frame) of the text (A) is recognized by speech in the sections (2) and (4) of the voice data. Therefore, the main section candidate selection unit 106 selects the sections (2) and (4) of the speech data as main section candidates corresponding to the text (A).

  As for the text (B), since the section of the audio data including the reference word “No. 3 building” is only the section (3), the section (3) is the main section corresponding to the text (B). Selected as a candidate. In addition, since the text (C) includes “Friday”, the sections (2) and (4) of the voice data are selected as candidates for the main section corresponding to the text (C), similarly to the text (A). The The main section candidate selection unit 106 notifies the main section selection unit 107 of completion of the main section candidate selection process.

  When the main section selection unit 107 receives a notification of completion of the main section candidate selection process, the main section selection unit 107 converts one or more sections in which speech of words other than the reference word is recognized from the sections selected as main section candidates into text The main section corresponding to (A) to (C) is selected. More specifically, the main section selection unit 107 refers to the extracted word DB 133 so that the “reference flag” is “0” for the sections (1) to (4) of the voice data selected as candidates for the main section. The presence / absence of voice recognition of the word “” is detected. The main section is selected for each text (A) to (C).

  FIG. 10 shows an example of selection of the main section. FIG. 10 shows how the main section is selected after the main section candidates shown in FIG. 9 are selected.

  In this example, the main section selection unit 107 selects the main section corresponding to the text (A) from the sections (2) and (4) of the audio data selected as the main section candidates. Words other than the reference word (“Friday”) of the text (A), “opinion” and “Kagawa” (see the dotted frame) are recognized in the voice data section (2), but the voice data section ( In 4), voice recognition is not performed. For this reason, the main section selection unit 107 selects the section (2) of the voice data as the main section corresponding to the text (A) (see a circle). The section (4) of the voice data is not selected because it does not include the voices of words other than the reference word that is common to the text (A) (see the crosses).

  As described above, the main section selection unit 107 selects the main section of the audio data corresponding to the texts (A) to (C) based on not only the reference word but also a word other than the reference word. Therefore, even if the reference word is overlapped in a plurality of sections of the plurality of texts (A) to (C) and the speech data (1) to (4), the speech analysis apparatus 1 is different from the comparative example shown in FIG. In contrast, it is possible to accurately associate sections of text and audio data.

  Note that “No. 3 building”, which is the reference word of the text (B), is included only in the section (3) of the audio data, so the section (3) is selected as the main section corresponding to the text (B). The text (C) is selected as the main section corresponding to the text (C) because the section (4) of the voice data includes the “next week” and “next week” voices common to the text (C).

  As described above, the main section selection unit 107 selects words other than the reference word common to the texts (A) to (C) from the sections (1) to (4) of the speech data selected as the main section candidates. Based on the presence or absence of voice recognition, the main section corresponding to the texts (A) to (C) is selected. At this time, the main section selection unit 107 determines, from the candidates for the main section, the section in which the voices of words other than the reference word among the plurality of words in the texts (A) to (C) are most recognized, or the reference word One or more sections in which a certain number or more of voices of other words are recognized may be selected as the main section.

  For example, when a section in which three or more voices of words other than the reference word are recognized is selected as the main section, the number of words in the text (A) that can be recognized in the section (2) of the voice data is “ Excluding “Friday”, there are only two of “opinion” and “Kagawa”, so they are not selected as the main section. In addition, the section (4) of the voice data has only two words “next time” and “next week” except for “Friday” except for “Friday”. Not selected. However, in the section (3) of the speech data, the number of words in the text (B) that can be recognized by the speech is 7 such as “autumn”, “test period”, “in / out”, etc. Since there is, it is selected as the main section.

  As described above, if one or more sections in which a certain number of words other than the reference word are recognized are set as the main section, the main section can be selected strictly. Therefore, when there are many sections of audio data selected as main section candidates, the main section can be selected with high accuracy. This effect can be obtained in the same way when the section in which the speech of words other than the reference word is recognized most is the main section.

  The main section selection unit 107 may assign a numerical value used as a reference for selecting the main section for each word included in each of the sections (1) to (4) of the audio data. In this case, for example, a large numerical value is assigned to a word having a large number of mora or a word included only in the texts (A) to (B), and a small numerical value is assigned to the other words, whereby the text (A) The main section corresponding to ~ (B) is selected more accurately.

  The main section selection unit 107 registers the selected main section in the analysis result DB 135 in order to associate the selected main section with the texts (A) to (C). More specifically, the main section selection unit 107 identifies identification information (A to C in this example) and identification information (1 to 4) of voice data selected as the main section (in this example). 1 to 4) are output to the analysis result DB 135.

  FIG. 11 shows an example of the analysis result DB 135. In FIG. 11, “text” indicates text identification information (A to C in this example). Also, the “1” to “4” fields in the “voice data section” field indicate the presence / absence of the correspondence relationship between the texts (A) to (C) in the sections (1) to (4) of the voice data (“1”: Corresponding relationship exists, “0”: no corresponding relationship).

  In the example of FIG. 11, among the “1” to “4” columns of the “voice data section” column corresponding to the text (A), only the “1” column and the “2” column are “1”. The sections of the audio data corresponding to (A) are shown as section (1) and section (2). Here, the section (1) of the voice data is selected as the main section in the above example, while the section (2) of the voice data is selected as the subsection described later.

  In the example of FIG. 11, according to the above example, the voice data section (3) is registered in the “3” column of the “voice data section” column corresponding to the text (B), and the text (C) is supported. The section (4) of voice data is registered in the “4” field of the “voice data section” field. After registering the main section, the main section selection unit 107 notifies the sub-section selection unit 108 of completion of the main section selection process.

  Upon receiving the notification of completion of the main section selection process, the subsection selection unit 108 starts with a section adjacent to the main section, and a word other than the reference word among the plurality of words in the texts (A) to (C). The range of the section in which the voice is recognized is detected. Then, the subsection selection unit 108 selects a section within the detected range as a subsection (second section) corresponding to the texts (A) to (C). That is, the sub-section selection unit 108 detects a section corresponding to the sub-section from the sections (1) to (4) of the audio data that are connected in front of and behind the main section in time series.

  More specifically, the sub-interval selection unit 108 refers to the extracted word DB 133 so that words whose “reference flag” is “0” sequentially from the sections (1) to (4) adjacent to the main section. The presence or absence of voice recognition is detected. In addition, selection of a subsection is performed for every text (A)-(C).

  FIG. 12 shows an example of selection of sub-intervals. FIG. 12 shows how the sub-section is selected after the main section shown in FIG. 10 is selected.

  In this example, the subsection selection unit 108 selects a subsection corresponding to the text (A). The sub-interval selection unit 108 starts from the sections (1) and (3) adjacent to the main section (2) of the text (A), and the range of sections in which words other than the reference word of the text (A) are voice-recognized. Is detected.

  The words “summer season” and “camp” (see the dotted frame) other than the reference word of the text (A) are recognized by speech in the section (1) adjacent to the main section (2) of the text (A). For this reason, the subsection selection unit 108 selects the section (1) of the audio data as a subsection corresponding to the text (A). (See circle).

  Since there is no section further adjacent to the section (1) of the speech data selected as the subsection, the subsection selection unit 108 performs speech recognition of words other than the reference word for the other adjacent section (3) of the main section. Detect the presence or absence. However, the section (3) of the speech data is not selected as a sub-section because it does not include the speech of words other than the reference word that is common to the text (A) (see the x mark). For this reason, the subsection selection unit 108 ends the detection of the subsection range.

  As described above, the sub-section selection unit 108 detects the range of the sub-section of the audio data corresponding to the texts (A) to (C) based on words other than the reference word. Therefore, the speech analysis apparatus 1 can associate the texts (A) to (C) with the sections (1) to (4) of the speech data in which the speech of the reference word is not recognized. Thus, unlike the comparative example shown in FIG. 7, the voice analysis device 1 can accurately specify the range of the voice data section associated with the texts (A) to (C). .

  In addition, for text (B), sections (2) and (4) adjacent to section (3), which is the main section, are selected as sub-sections because they do not contain the word speech common to text (B). Not. As for the text (C), the section (3) adjacent to the main section (4) corresponding to the text (C) is not selected as a sub-section because it does not include the word speech common to the text (C).

  As described above, the sub-section selection unit 108 detects a range of a section in which the speech of words other than the reference word in the texts (A) to (C) is recognized, starting from a section adjacent to the main section, A section within the range is selected as a subsection corresponding to the texts (A) to (C). At this time, the sub-section selection unit 108 may detect a range of one or more sections in which a certain number or more of voices of words other than the reference word in the texts (A) to (C) are recognized.

  For example, when detecting the range of a section in which three or more speeches of words other than the reference word are recognized, the number of words in the speech (A) in which the speech data can be recognized is “Friday”. Excluding, there are only two, “summer season” and “camp”, so they are not selected as sub-intervals.

  In this way, if one or more sections in which a certain number or more of speeches of words other than the reference word are recognized are set as sub-sections, the range of the sub-section can be strictly detected. Note that the sub-section selection unit 108 may assign a numerical value as a reference for detecting the range of the sub-section for each word included in each of the sections (1) to (4) of the audio data. In this case, for example, a large numerical value is assigned to a word having a large number of mora or a word included only in the texts (A) to (B), and a small numerical value is assigned to the other words, whereby the text (A) The sub-interval corresponding to ~ (B) is selected more accurately.

  Further, when the sub-section selection unit 108 detects the range of the sub-section, the sub-section selection unit 108 detects the number of speech recognition of words other than the reference word in each of the texts (A) to (B) (the number of matching words), The detected section may be selected as a subsection corresponding to the text having the largest number of speech recognitions.

  The sub-section selecting unit 108 registers the selected sub-section in the analysis result DB 135 in order to associate it with the texts (A) to (C). More specifically, the main section selection unit 107 identifies identification information (in this example, A to C) of text and identification information (in this example) of sections (1) to (4) of audio data selected as sub-sections. 1 to 4) are output to the analysis result DB 135.

  Thereby, in the analysis result DB 135, correspondences between the texts (A) to (C) and the sections (1) to (4) of the voice data are registered. In the present embodiment, one or more sections (main sections) of highly related speech data using reference words and other words common to the speech data (1) to (4) and the texts (A) to (C). And the sub-interval) are associated with the text, so that an accurate correspondence relationship is registered in the analysis result DB 135.

  The analysis result DB 135 obtained in this way is used, for example, when a user of the minutes system refers to the minutes. Thereby, the user can refer to the exact description location of the minutes corresponding to the content of the audio data of the reproduced meeting, so that the convenience of the minutes system is improved.

  Next, a speech analysis program that executes the speech analysis method described above will be described. FIG. 13 is a flowchart of the speech analysis program according to the embodiment.

  First, the audio data dividing unit 105 divides the audio data in the audio DB 134 into a plurality of sections (1) to (4) (step St1). As a method for dividing the audio data, for example, the method shown in FIG. 5A or 5B is used.

  Next, the document data dividing unit 101 divides the document data in the document DB 131 into a plurality of texts (A) to (C) (step St2). As a document data dividing method, as described above, there is a method of dividing document data using a line break, indentation, or blank line in a sentence as a boundary between texts.

  Next, the word extraction part 102 extracts a word from each text (A)-(C) based on dictionary DB132 (step St3). Next, the word extraction unit 102 registers the extracted word in the extracted word DB 133 (see FIG. 4) (step St4). Note that the processing of steps St2 to St4 may be executed before the processing of step St1.

  Next, the speech recognition processing unit 104 performs speech recognition processing for each section (1) to (4) of the speech data based on the words extracted from the texts (A) to (C) (steps). St5). Next, the speech recognition processing unit 104 registers the presence / absence of speech recognition of each word in the extracted word DB 133 for each of the texts (A) to (C) (step St6).

  Next, the reference word selection unit 103 selects one of the texts (A) to (C) (step St7). The subsequent processing is described assuming that the text (A) is selected, but any text (A) to (C) may be selected in the processing of step St7.

  Next, the reference word selection unit 103 uses, as reference words, one or more words (“Friday”) having a certain level of ease or certainty of speech recognition from words extracted from the selected text (A). The selected word is registered in the extracted word DB 133 (step St8).

  In the process of step St7, as described above, the reference word selection unit 103 selects one or more words having the largest number of mora or a certain number or more from the plurality of words extracted from the selected text (A). It may be selected as a word. Alternatively, the reference word selection unit 103 selects, as a reference word, one or more words having a maximum score or a certain value or more indicating a word likelihood in speech recognition from a plurality of words extracted from the selected text (A). May be.

  Next, the main section candidate selection unit 106 selects a section in which the speech of the reference word is recognized from the sections (1) to (4) of the voice data as main section candidates (step St9). As described with reference to FIG. 9, the main section candidate selecting unit 106 selects the section (2) and the section (4) of the speech data in which “Friday”, which is the reference word of the text (A), is speech-recognized. A candidate for the main section corresponding to the text (A) is selected.

  Next, the main section selection unit 107 corresponds to one or more sections in which the voices of words other than the reference word are recognized among the plurality of words of the text (A) from the candidates of the main section. To be selected as the main section (first section) to be performed (step St10). As described with reference to FIG. 10, the main section selection unit 107 performs voice recognition of “opinion” and “Kagawa” of the text (A) from the candidate sections (2) and (4) of the main section. The voice data section (2) is selected as the main section corresponding to the text (A).

  In the processing of step St10, as described above, the main section selection unit 107, except for the reference word among the plurality of words of the text (A), from the sections (2) and (4) selected as the main section candidates. The section in which the most speech of the word is recognized may be selected as the main section. Alternatively, the main section selection unit 107 recognizes a predetermined number or more of words other than the reference word among the plurality of words of the text (A) from the sections (2) and (4) selected as the main section candidates. One or more sections thus selected may be selected as the main section.

  Next, the main section selection unit 107 registers the correspondence between the text (A) and the main section in the analysis result DB 135 (see FIG. 11) (step St11). Thereby, the section (2) of the voice data is associated with the text (A).

  Next, the subsection selection unit 108 selects sections (1) and (3) adjacent to the main section corresponding to the text (A) (step St12). Next, the sub-section selection unit 108 refers to the extracted word DB 133 to determine whether or not speech recognition is performed on words other than the reference word of the text (A) in the selected adjacent sections (1) and (3). (Step St13).

  When there is speech recognition of a word other than the reference word (Yes in step St13), the sub-section selecting unit 108 selects the selected adjacent section (1) as a sub-section (step St14). As described with reference to FIG. 12, the sub-interval selection unit 108 determines the “summer season” and “camp” of the text (A) from the adjacent sections (1) and (3) of the main section corresponding to the text (A). Is selected as the sub-interval corresponding to the text (A). Note that, in the processing of steps St13 and St14, the sub-section selection unit 108, as described above, from the sections (1) and (3) adjacent to the main section, among the plurality of words of the text (A), the reference word A section in which a certain number or more of speeches of other words are recognized may be selected as a sub-section.

  Next, the subsection selection unit 108 registers the correspondence between the text (A) and the subsection in the analysis result DB 135 (step St15). Thereby, the section (1) of the voice data is associated with the text (A).

  Next, the sub-section selecting unit 108 selects an adjacent section of the sub-section (step St16) and executes the process of step St13 again. The sub-interval selection unit 108 repeats the processing of steps St13 to St16 to detect a range of a segment in which the speech of a word other than the reference word of the text is recognized starting from a segment adjacent to the main segment. Is selected as a sub-interval corresponding to the text.

  When there is no speech recognition of words other than the reference word (No in step St13), the reference word selection unit 103 determines whether all texts (A) to (C) have been selected (step). St17). That is, the reference word selection unit 103 determines whether or not the processes in steps St8 to St16 have been executed for all the texts (A) to (C).

  When all the texts (A) to (C) have been selected (Yes in step St17), the reference word selection unit 103 ends the process. In addition, when all the texts (A) to (C) have not been selected (No in Step St17), the reference word selection unit 103 selects other texts (B) and (C) (Step St18). After selecting the other texts (B) and (C), the processes in steps St8 to St13 are performed again for the selected text. In this way, the processing of the voice analysis program is executed.

  9, 10, and 12 exemplify the case where there is one reference word, the present invention is not limited to this, and there may be a plurality of reference words. When a plurality of reference words are selected, the main section candidate selecting unit 106 selects a section of audio data including all the reference words as a main section candidate.

As described above, the speech analysis method according to the embodiment is a method in which the computer (CPU) 10 executes the following steps.
Step (1): The voice data is divided into a plurality of sections (1) to (4).
Step (2): Extract a plurality of words from the texts (A) to (C).
Step (3): Based on the extracted plurality of words, speech recognition in a plurality of sections (1) to (4) is performed.
Step (4): One or more words having a certain level of ease or certainty of speech recognition are selected as reference words from the extracted plurality of words.
Step (5): A section in which the voice of the reference word is recognized is selected from a plurality of sections.
Step (6): One or more sections in which the voices of words other than the reference word are recognized among the plurality of words from the selected section are defined as a first section corresponding to the texts (A) to (C) (main (Section).
Step (7): Starting from a section adjacent to the first section, a range of a section in which the voice of a word other than the reference word is recognized among a plurality of words is detected, and the section within the range is defined as text (A). To (C) are selected as the second section (sub-section).

  According to this configuration, a reference word that is easily recognized by speech or a reference word that is highly reliable as a result of speech recognition is extracted from the text, and the sections (1) to (4) of the speech data in which the speech of the reference word is recognized. Among them, the first section in which the speech of other words extracted from the text is also recognized is associated with the texts (A) to (C). Further, a range of the second section in which the speech of another word extracted from the text is recognized is detected from the section of the speech data continuous to the first section, and is associated with the text.

  Therefore, it is possible to associate one or more sections of speech data with high relevance with text using a reference word and other words common to the speech data and text. Therefore, according to the speech analysis method according to the embodiment, speech data can be accurately associated with text.

The speech analysis program according to the embodiment is a program that causes the computer (CPU) 10 to execute the following processing.
Process (1): The audio data is divided into a plurality of sections (1) to (4).
Process (2): A plurality of words are extracted from the texts (A) to (C).
Process (3): Based on the extracted plurality of words, speech recognition is performed in a plurality of sections (1) to (4).
Process (4): One or more words having a certain level of ease or certainty of speech recognition are selected as reference words from a plurality of extracted words.
Process (5): A section in which the voice of the reference word is recognized is selected from a plurality of sections.
Process (6): One or more sections in which the voices of words other than the reference word are recognized among the plurality of words from the selected section are defined as a first section corresponding to the texts (A) to (C) (main (Section).
Process (7): Starting from a section adjacent to the first section, a range of a section in which the speech of a word other than the reference word is recognized among a plurality of words is detected, and the section within the range is converted into text (A) To (C) are selected as the second section (sub-section).

  Since the speech analysis program according to the embodiment includes the same configuration as the speech analysis method described above, the same effects as those described above can be obtained.

  The speech analysis apparatus 1 according to the embodiment includes a dividing unit (speech data dividing unit) 105, an extracting unit (word extracting unit) 102, and a speech recognition processing unit 104. The speech analysis apparatus 1 further includes a first selection unit (reference word selection unit) 103, a second selection unit (main section candidate selection unit) 106, a third selection unit (main section selection unit) 107, and a fourth And a selection unit (sub-section selection unit) 108.

  The dividing unit 105 divides the audio data into a plurality of sections (1) to (4). The extraction unit 102 extracts a plurality of words from the texts (A) to (C). The speech recognition processing unit 104 performs speech recognition for the plurality of sections (1) to (4) based on the extracted plurality of words. The first selection unit 103 selects, as a reference word, one or more words having a certain level of ease or certainty of voice recognition from a plurality of extracted words.

  The second selection unit 106 selects a section in which the voice of the reference word is recognized from the plurality of sections. The third selection unit 107 selects, from the selected section, one or more sections in which the voices of words other than the reference word among the plurality of words are recognized as the first section corresponding to the texts (A) to (C). Select as (main section). The fourth selection unit 108 detects a range of a segment in which a voice of a word other than the reference word is recognized among a plurality of words, with a segment adjacent to the first segment as a starting point. A second section (sub-section) corresponding to (A) to (C) is selected.

  Since the speech analysis apparatus 1 according to the embodiment includes the same configuration as the speech analysis method described above, the same effects as those described above are achieved.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium (except for a carrier wave).

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, the following additional notes are disclosed regarding the above description.
(Supplementary note 1) dividing audio data into a plurality of sections;
Extracting a plurality of words from the text;
Performing speech recognition of the plurality of sections based on the extracted plurality of words,
Selecting one or more words having a certain level of speech recognition ease or certainty as a reference word from the plurality of extracted words;
Selecting a section in which the voice of the reference word is recognized from the plurality of sections;
Selecting one or more sections from which the speech of words other than the reference word is recognized among the plurality of words as a first section corresponding to the text;
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A voice analysis method, wherein the computer executes the step of selecting as the second section.
(Supplementary Note 2) In the step of selecting the reference word, one or more words having a maximum number of mora or a certain number or more are selected as the reference word from the plurality of extracted words. The voice analysis method described in 1.
(Additional remark 3) In the process of selecting the said reference word, the said 1 or more word whose score which shows the word likeness in speech recognition is the most or more than a fixed value is selected as the said reference word. The voice analysis method described.
(Supplementary Note 4) In the step of selecting the first section, from the selected section, among the plurality of words, a section in which the voice of a word other than the reference word is most recognized, or a section other than the reference word 4. The speech analysis method according to any one of appendices 1 to 3, wherein one or more sections in which a certain number of words are recognized are selected as the first section.
(Supplementary Note 5) In the step of selecting the second section, one or more voices of words other than the reference word among the plurality of words are recognized starting from a section adjacent to the first section. The speech analysis method according to any one of appendices 1 to 4, wherein the range of the section is detected.
(Appendix 6) Dividing audio data into multiple sections,
Extract multiple words from text,
Based on the plurality of extracted words, perform speech recognition of the plurality of sections,
From the extracted plurality of words, one or more words having a certain level of ease or certainty of speech recognition are selected as reference words,
From the plurality of sections, select a section in which the voice of the reference word is recognized,
From the selected section, one or more sections in which the voices of words other than the reference word are recognized among the plurality of words are selected as the first section corresponding to the text,
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A speech analysis program that causes a computer to execute processing selected as the second section.
(Supplementary Note 7) In the process of selecting the reference word, one or more words having a maximum number of mora or a certain number or more are selected as the reference word from the plurality of extracted words. The voice analysis program described in 1.
(Supplementary note 8) In the supplementary note 6, the step of selecting the reference word selects, as the reference word, the one or more words having a score indicating the likelihood of a word in speech recognition or having a certain score or more. The voice analysis program described.
(Supplementary Note 9) In the step of selecting the first section, from the selected section, among the plurality of words, a section in which the voice of a word other than the reference word is most recognized, or a section other than the reference word 9. The speech analysis program according to any one of appendices 6 to 8, wherein one or more sections in which a certain number of words are recognized are selected as the first section.
(Supplementary Note 10) In the step of selecting the second section, one or more voices of words other than the reference word among the plurality of words are recognized starting from a section adjacent to the first section The speech analysis program according to any one of appendices 6 to 9, wherein the range of the section is detected.
(Supplementary Note 11) A dividing unit that divides audio data into a plurality of sections;
An extractor for extracting a plurality of words from the text;
A speech recognition processing unit that performs speech recognition of the plurality of sections based on the extracted words;
A first selection unit that selects, as a reference word, one or more words having a certain level of ease or certainty of speech recognition from the plurality of extracted words;
A second selection unit that selects a section in which the voice of the reference word is recognized from the plurality of sections;
A third selection unit that selects, from the selected section, one or more sections in which voices of words other than the reference word are recognized among the plurality of words as a first section corresponding to the text;
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A speech analysis apparatus comprising: a fourth selection unit that selects the second section.
(Supplementary note 12) The supplementary note 11 is characterized in that the first selection unit selects, as the reference word, one or more words having a maximum number of mora or a certain number or more from the plurality of extracted words. Voice analysis device.
(Supplementary note 13) The first selection unit according to Supplementary note 11, wherein the first selection unit selects, as the reference word, the one or more words having a maximum score or a certain value or more indicating a word-likeness in speech recognition. Voice analysis device.
(Additional remark 14) The said 3rd selection part is the area in which the audio | voice of words other than the said reference word was recognized most among these words from the selected area, or the audio | voice of words other than the said reference word. 14. The speech analysis apparatus according to any one of appendices 11 to 13, wherein one or more sections in which a certain number is recognized are selected as the first section.
(Additional remark 15) The said 4th selection part is the starting point of the area adjacent to the said 1st area, Of one or more areas from which the audio | voice of words other than the said reference word was recognized more than among the said several words. The speech analyzer according to any one of appendices 11 to 14, wherein a range is detected.

1 Speech analyzer 10 CPU
101 document data division unit 102 word extraction unit (extraction unit)
103 reference word selection unit (first selection unit)
104 voice recognition processing unit 105 voice data dividing unit (dividing unit)
106 Main section candidate selection section (second selection section)
107 Main section selector (third selector)
108 Subsection selection unit (fourth selection unit)

Claims (7)

Dividing the audio data into a plurality of sections;
Extracting a plurality of words from the text;
Performing speech recognition of the plurality of sections based on the extracted plurality of words,
Selecting one or more words having a certain level of speech recognition ease or certainty as a reference word from the plurality of extracted words;
Selecting a section in which the voice of the reference word is recognized from the plurality of sections;
Selecting one or more sections from which the speech of words other than the reference word is recognized among the plurality of words as a first section corresponding to the text;
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A voice analysis method, wherein the computer executes the step of selecting as the second section.   2. The method according to claim 1, wherein in the step of selecting the reference word, one or more words having a maximum number of mora or a predetermined number or more are selected as the reference word from the plurality of extracted words. Voice analysis method.   2. The speech according to claim 1, wherein, in the step of selecting the reference word, the one or more words having a maximum score or a certain value or more indicating a word likelihood in speech recognition are selected as the reference word. Analysis method.   In the step of selecting the first section, from the selected section, among the plurality of words, a section in which the speech of a word other than the reference word is most recognized, or a speech of a word other than the reference word The speech analysis method according to any one of claims 1 to 3, wherein one or more recognized sections are selected as the first section.   In the step of selecting the second section, a range of one or more sections in which a predetermined number or more of voices of the words other than the reference word are recognized from the section adjacent to the first section as a starting point The speech analysis method according to claim 1, wherein the speech analysis method is detected. Divide audio data into multiple sections,
Extract multiple words from text,
Based on the plurality of extracted words, perform speech recognition of the plurality of sections,
From the extracted plurality of words, one or more words having a certain level of ease or certainty of speech recognition are selected as reference words,
From the plurality of sections, select a section in which the voice of the reference word is recognized,
From the selected section, one or more sections in which the voices of words other than the reference word are recognized among the plurality of words are selected as the first section corresponding to the text,
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A speech analysis program that causes a computer to execute processing selected as the second section. A dividing unit for dividing the audio data into a plurality of sections;
An extractor for extracting a plurality of words from the text;
A speech recognition processing unit that performs speech recognition of the plurality of sections based on the extracted words;
A first selection unit that selects, as a reference word, one or more words having a certain level of ease or certainty of speech recognition from the plurality of extracted words;
A second selection unit that selects a section in which the voice of the reference word is recognized from the plurality of sections;
A third selection unit that selects, from the selected section, one or more sections in which voices of words other than the reference word are recognized among the plurality of words as a first section corresponding to the text;
Starting from a section adjacent to the first section, a range of a section in which speech of a word other than the reference word is recognized among the plurality of words is detected, and a section in the range corresponds to the text A speech analysis apparatus comprising: a fourth selection unit that selects the second section.

Images