JP2013088488A - Speech search device, speech search method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech search device, a speech search method, and a program which have high search accuracy.SOLUTION: A search object acquisition unit 62 acquires a triphone string A where triphones each of which has a phoneme included in speech data of a search object as a center phoneme and includes a phoneme directly preceding the center phoneme and a phoneme directly succeeding the center phoneme are arranged in time sequence. An input unit 64 inputs a search word to be searched for speech data. A search word acquisition unit 66 converts the search word inputted by the input unit 64, to a phoneme string to acquire a triphone string B. A search unit 67 calculates similarities between a triphone string C obtained by eliminating the first and last biphones of the triphone string B and partial strings included in the triphone string A and extracts a partial string for which the calculated similarity meets a prescribed condition, from the triphone string A. An output unit 69 displays the reproduction start time in speech data, of the partial string extracted by the search unit 67.

Description

  The present invention relates to a voice search device, a voice search method, and a program.

  When searching for speech contained in audio data by a keyword, it is common to model the audio data. One such model is a triphone (see, for example, Patent Documents 1 and 2). A triphone is a phoneme model of three phonemes. The triphone uses each phoneme included in the voice data as a central phoneme, and includes a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side. Finally, the voice data is modeled into a triphone sequence in which triphones are arranged in time series.

  When searching for a voice included in the voice data converted into the triphone string using a keyword, the keyword must also be modeled into the triphone string. The keyword triphone sequence is compared with a portion of the triphone sequence of voice data. As a result of the comparison, a portion having high similarity to the keyword triphone string is searched from the triphone string of the voice data.

JP 2006-11257 A JP 2011-39468 A

  Since the first phoneme of the keyword has no phoneme on the front side, the phoneme model having the first phoneme as the central phoneme is not a triphone but a biphone of two phonemes. Similarly, since the last phoneme of the keyword does not have a phoneme on the rear side, the phoneme model having the last phoneme as the central phoneme is biphone. In other words, the first and last phoneme models of the keyword triphone string are biphones.

  The comparison between the triphone string of the voice data and the triphone string of the keyword is a comparison between a triphone string that is all triphones and a triphone string that partially includes biphones. The biphones included in the keyword triphone string are factors that reduce the similarity between the two and cause noise in the search. As a result, the keyword search accuracy decreases.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a voice search device, a voice search method, and a program with high search accuracy.

In order to achieve the above object, a voice search device according to the present invention provides:
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition unit for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence And a search unit that extracts a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output unit that outputs information related to the audio data corresponding to the partial sequence extracted by the search unit;
Is provided.

  According to the present invention, search accuracy is increased.

It is a block diagram which shows the hardware constitutions of the voice search device which concerns on Embodiment 1 of this invention. It is a block diagram which shows the function structure of the speech search device which concerns on Embodiment 1 of this invention. It is a figure which shows the triphone row | line | column A produced | generated based on the audio | voice data of search object. It is a figure which shows the phoneme row | line | column, the triphone row | line | column B, and the triphone row | line | column C which were produced | generated based on the search word. It is a figure which shows the node defined by two triphone strings. It is a figure which shows the content of the distance table between triphones. It is a figure explaining the recurrence formula used by continuous DP matching. It is a figure explaining the extraction method of a partial sequence. It is a flowchart of the voice search process which concerns on Embodiment 1 of this invention. It is a flowchart of the search process which concerns on Embodiment 1 of this invention. It is a flowchart of the search process which concerns on Embodiment 2 of this invention. It is a flowchart of the search process which concerns on Embodiment 3 of this invention. It is a flowchart of the search process which concerns on Embodiment 4 of this invention.

(Embodiment 1)
Embodiment 1 of the present invention will be described in detail with reference to the drawings. First, a hardware configuration of the voice search device 100 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the speech search apparatus 100 includes a ROM 1 (Read Only Memory), a RAM 2 (Random Access Memory), an external storage device 3, an input device 4, an output device 5, and a CPU 6 (Central Processing Unit). ).

  The ROM 1 stores an initial program for performing various initial settings, hardware inspection, program loading, and the like. The RAM 2 temporarily stores various software programs executed by the CPU 6, data necessary for executing these software programs, and the like. The external storage device 3 is, for example, a hard disk or the like and stores various software programs, data, and the like. These software programs include application software programs and basic software programs such as an OS (Operating System).

  Various data includes audio data. The audio data is, for example, audio data relating to audio such as news broadcasts, recorded conference audio, movie audio, and the like.

  The input device 4 is, for example, a keyboard. The input device 4 inputs text data and the like input by a user using a keyboard to the CPU 6. The output device 5 includes, for example, a screen such as a liquid crystal display, a speaker, and the like. The output device 5 displays text data output by the CPU 6 on a screen and outputs audio data from a speaker.

  The CPU 6 reads the software program stored in the external storage device 3 into the RAM 2 and controls the execution of the software program, thereby realizing the following functional configuration.

  Next, the functional configuration of the voice search device 100 will be described with reference to FIG. The voice search device 100 includes a voice data storage unit 61, a search target acquisition unit 62, a triphone string storage unit 63, an input unit 64, a conversion table storage unit 65, a search word acquisition unit 66, a search unit 67, and an inter-triphone distance storage. Unit 68 and output unit 69. The audio data storage unit 61, the triphone string storage unit 63, the conversion table storage unit 65, and the inter-triphone distance storage unit 68 are constructed in the storage area of the external storage device 3.

  The voice data storage unit 61 stores voice data to be searched. The search target acquisition unit 62 acquires a triphone string A (first phoneme model string) based on the search target voice data stored in the voice data storage unit 61. The triphone is a phoneme model including a central phoneme, a phoneme nearest to the front side of the phoneme, and a phoneme nearest to the rear side. In the triphone row A, triphones having each phoneme included in the audio data to be searched as a central phoneme are arranged in time series. The triphone train A is generated using, for example, a triphone HMM (Hidden Markov Model) as an acoustic model. The search target acquisition unit 62 functions as a triphone phoneme recognition engine that generates the triphone string A using the triphone HMM.

  The triphone is expressed by “front nearest phoneme−central phoneme + rear latest phoneme”. An array of triphones arranged in chronological order is a triphone sequence. For example, as illustrated in FIG. 3, a part of the triphone string A generated based on the audio data includes “a−o + N”, “o−N + s”, “N−s + e”, “se− + sh”. ”,“ E−sh + i ”,“ sh−i + N ”,“ i−N + g ”,“ N−g + o ”,“ go− + sh ”,“ o−sh + o ”,“ sh−o + r ”,“ or−r + i ” ”,“ R−i + k ”,“ i−k + o ”,“ k−o: + z ”,“ o−z + a ”, and“ z−a + g ”.

  The search target acquisition unit 62 generates a triphone string A in advance based on the audio data, and stores it in the triphone string storage unit 63. At this time, the search target acquisition unit 62 creates an information table in which each generated triphone is associated with the reproduction start time of the audio data corresponding to the triphone and stores the information table in the triphone string storage unit 63. . In addition, the search target acquisition unit 62 causes the triphone string storage unit 63 to store the generated audio data corresponding to each triphone in association with the reproduction start time of the information table.

  The input unit 64 inputs a search term to search for voice data. More specifically, the input unit 64 inputs a search term input by the user using the keyboard of the input device 4, for example, text data of “signal processing” to the search term acquisition unit 66.

  The conversion table storage unit 65 stores a conversion table in which text data and phonemes are associated with each other. The search word acquisition unit 66 acquires a triphone string B (second phoneme model string) based on the search word input by the input unit 64. More specifically, first, the search word acquisition unit 66 converts the search word input by the input unit 64 into a phoneme string. In the conversion of the search word into the phoneme string, the search word acquisition unit 66 refers to the conversion table stored in the conversion table storage unit 65 and converts the text data into a phoneme. Subsequently, the search word acquisition unit 66 arranges the converted phonemes in time series order and converts them into phoneme strings.

  The search word acquisition unit 66 uses each phoneme constituting the converted phoneme string as a central phoneme, and triphones including the central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side are arranged in chronological order. A triphone string B (second phoneme model string) is generated.

  For example, when the search term input by the input unit 64 is “signal processing”, as shown in FIG. 4, the search term acquisition unit 66 sets “signal processing” as a phoneme string “sh”, “i”. , “N”, “g”, “o:”, “sh”, “o”, “r”, “i”. Subsequently, the search word acquisition unit 66 sets “sh + i”, “sh−i + N”, “i−N + g”, “N−g + o”, “g−” as the triphone string B based on the converted phoneme string. o: + sh ”,“ o−sh + o ”,“ sh−o + r ”,“ o−r + i ”, and“ r−i ”are generated. As described above, the generated triphone string B includes biphones at the beginning and end.

  The search unit 67 determines the similarity between the triphone sequence C (third phoneme model sequence), which is a triphone sequence excluding the first and last triphones of the triphone sequence B, and the partial sequence included in the triphone sequence A. calculate. The search unit 67 generates a triphone string C based on the triphone string B output from the search word acquisition unit 66. As shown in FIG. 4, in the triphone line C, the first biphone “sh + i” and the last biphone “ri” of the triphone line B are removed, and “sh−i + N” and “i−N + g” are removed. , “N−g + o”, “go: + sh”, “o−sh + o”, “sh−o + r”, “o−r + i”.

  The search unit 67 calculates the similarity by continuous DP (Dynamic Programming) matching between the triphone sequence A and the triphone sequence C. For the continuous DP matching, any known method may be used. In the present embodiment, the search unit 67 executes the continuous DP matching by the following method.

  First, the search unit 67 defines a set of nodes including a row corresponding to each triphone included in the triphone column C and a column corresponding to each triphone included in the triphone column A. In the following, as shown in FIG. 5 as an example, the nodes are arranged in 7 rows corresponding to 7 triphones in the triphone column C and 10 columns corresponding to 10 triphones included in the triphone column A. The case will be described. Note that a node located in i row and j column is a node (i, j), and a set of nodes shown in FIG. 5 is a search table.

  Next, the search unit 67 defines a cost X (i, j) for each node (i, j). The cost X (i, j) is the triphone corresponding to the node that is traced when the node (i, j) is traced from the node corresponding to the first triphone of the triphone string C (i = 1 node). The matching error between the column C and the partial column of the triphone column A, that is, the similarity is shown.

  The cost X (i, j) is updated as follows. First, the search unit 67 calculates the cost X (cost X (1, 1) to cost X (1, 10)) of the node (node (1, 1) to node (1, 10)) in the first row of the search table. In addition, the distance between the first triphone of the triphone line C and the triphone of the triphone line A corresponding thereto is substituted as an initial value.

  More specifically, if the distance between the i-th triphone in the triphone row C and the j-th triphone in the triphone row A is d (i, j), the search unit 67 calculates the cost X (1, j ) Is substituted for d (1, j). Further, the search unit 67 eliminates the cost X (cost X (2, 1) to cost X (from the node (2, 1) to the node (7, 1)) in the first column excluding the node (1, 1). In 7, 1)), a value sufficiently larger than the assumed maximum value of cost X is substituted as an initial value.

  The distance d (i, j) is obtained with reference to the inter-triphone distance table stored in the inter-triphone distance storage unit 68. As shown in FIG. 6, the inter-triphone distance table stores distances d obtained experimentally for all combinations of triphones. In the example of FIG. 6, the distances d of “a−a + i”, “a−a + u”, and “o−w + o” with respect to “a−a + a” are “5”, “7”, and “100”, respectively. The distances “a−a + u” and “o−w + o” with respect to “a−a + i” are “20” and “99”, respectively. The distance d of “o−w + o” with respect to “a−a + u” is “98”. The distance d between the same triphones is “0”.

  The distance d between triphones may be, for example, the average of the distances between the three phonemes included in the triphone. In this case, the distance between phonemes is obtained in advance from the feature amount of each phoneme using an acoustic model. The phoneme feature amount is, for example, a short-time spectrum in the frequency band of the phoneme waveform data.

  The nodes to which the initial values are assigned are connected to each other through paths corresponding to triphone matching or replacement, insertion, and dropout. Here, with reference to FIG. 7, the coincidence or replacement, insertion and dropout will be described in detail. As shown in FIG. 7, the match or replacement is a connection between the node (i-1, j-1) and the node (i, j). The insertion is a connection between the node (i-1, j-2) and the node (i, j). The dropout is a connection between the node (i−2, j−1) and the node (i, j).

  Next, the search unit 67 updates the cost X (i, j) with the following recurrence formulas corresponding to the match or replacement, insertion, and dropout, respectively.

  The upper part of the recurrence formula corresponds to the case of matching or replacement. In this case, the search unit 67 adds d (i, j) to the cost X (i-1, j-1). The middle stage of the recurrence formula corresponds to the case of insertion. In this case, the search unit 67 adds the average of d (i, j−1) and d (i, j) to the cost X (i−1, j−2), and further adds the insertion cost α. The lower stage of the recurrence formula corresponds to the case of dropout. In this case, the search unit 67 adds d (i, j) and d (i-1, j) to the cost X (i−2, j−1), and further adds the dropout cost β. Note that the insertion cost α and the dropout cost β are preset constants.

  Note that the recurrence formula is a formula corresponding to, for example, the case where two triphone strings are inserted, and d (i, j-2) and d (i, j) are added to the cost X (i-1, j-3). −1) and the average of d (i, j) are added, and an arbitrary modification such as adding an expression for adding the insertion cost γ to the recurrence formula is possible.

  The search unit 67 updates X (i, j) as described above. Next, the search unit 67 selects a node having the smallest cost X (7, j) among the nodes with i = 7. Hereinafter, a case where the cost X (7, 9) is the smallest will be described as an example.

  As shown in FIG. 8, the search unit 67 goes back the route from which the selected cost X (7, 9) is calculated, thereby matching the first triphone string A matched with the first triphone in the triphone string C. Identifies the third triphone. As a result, the search unit 67 extracts a partial sequence including the third to ninth portions of the triphone sequence A. At this time, the cost X (7, 9) corresponds to the similarity between the triphone sequence C and the partial sequence included in the triphone sequence A.

  As described above, by executing the continuous DP matching, the search unit 67 extracts the partial sequence having the smallest calculated cost X as the predetermined condition from the triphone sequence A.

  The search unit 67 selects the node with the lowest cost X. However, the search unit 67 determines whether the cost X is less than a predetermined threshold or is within a predetermined order in ascending order of the cost X. The partial sequence may be extracted by selecting.

  The search unit 67 that has extracted the partial sequence outputs information relating to the audio data corresponding to the extracted partial sequence. The information regarding the audio data corresponding to the extracted partial sequence is, for example, the reproduction start time corresponding to the first triphone of the extracted partial sequence and the audio data corresponding to the extracted partial sequence. The search unit 67 refers to the information table stored in the triphone string storage unit 63, acquires the reproduction start time and audio data, and outputs the reproduction start time and audio data to the output unit 69.

  The output unit 69 outputs information related to the audio data corresponding to the partial sequence extracted by the search unit 67 to the output device 5. More specifically, the output unit 69 displays the reproduction start time output from the search unit 67 via a screen provided in the output device 5. Here, when an instruction to reproduce audio data corresponding to the partial sequence extracted via the input device 4 is input by the user, the output unit 69 outputs the audio data output from the search unit 67 via a speaker. May be output.

  Next, the flow of the voice search process by the voice search device 100 in this embodiment will be described in detail with reference to FIG. As a premise, it is assumed that the triphone string storage unit 63 has already stored the triphone string A shown in FIG.

  The input unit 64 inputs text data corresponding to the search term “signal processing” to the search term acquisition unit 66 (step S1). The search term acquisition unit 66 generates a triphone string B based on the input text data (step S2). Subsequently, the search unit 67 executes a search process for extracting a partial sequence from the triphone sequence C (step S3).

  Here, the search process (step S3) will be described in detail with reference to FIG. The search unit 67 generates a triphone sequence C excluding the first and last biphones of the triphone sequence B (step S11). Subsequently, the search unit 67 performs continuous DP matching between the triphone sequence C and the triphone sequence A (step S12). Subsequently, the search unit 67 extracts a partial sequence having the smallest cost X from the result of continuous DP matching (step S13). Next, returning to FIG. 9, the output unit 69 displays the reproduction start time corresponding to the extracted partial sequence via the screen (step S4). Then, the voice search device 100 ends the voice search process.

  As described above in detail, the speech search apparatus 100 according to the present embodiment performs continuous DP matching using the triphone sequence C excluding the first and last biphones of the triphone sequence B. Therefore, compared to the case where the continuous DP matching is performed using the triphone string B including biphones, it is possible to prevent an increase in cost X due to the fact that the number of phonemes of the biphone does not match the number of phonemes of the triphone. This increases the search accuracy.

(Embodiment 2)
Next, Embodiment 2 of the present invention will be described in detail. The hardware configuration and functional configuration of the voice search device 100 in the present embodiment are the same as those in the first embodiment, but the function of the search unit 67 is different. When the search term is short, that is, when the number of triphones constituting the triphone row B is small, if the biphone is deleted, the number of triphones constituting the triphone row C is reduced, and the amount of data related to the search term is reduced. May be extremely low. As a result, the search accuracy may be lowered.

  Therefore, when the number of triphones constituting the triphone train B is less than the threshold, the search unit 67 determines the cost Y (second similarity) between the triphone train B and the partial train included in the triphone train A. Degree). Then, the search unit 67 extracts the partial sequence having the smallest calculated cost Y from the triphone sequence A. On the other hand, when the number of triphones constituting the triphone string B is equal to or greater than the threshold, the search unit 67 determines the cost X (first similarity) between the triphone string C and the partial string included in the triphone string A. Degree).

  The flow of the voice search process by the voice search device 100 according to this embodiment will be described in detail. The flow of the voice search process according to the present embodiment is different from the search process (step S3) in the voice search process according to the first embodiment. In the following, with reference to FIG. 11, the difference from the first embodiment will be mainly described in the search process of the present embodiment.

  The search unit 67 determines whether or not the number of triphones constituting the triphone train B is equal to or greater than a threshold value (step S21). When the number of triphones constituting the triphone row B is equal to or greater than the threshold (step S21; Yes), the search unit 67 executes steps S22 to S24 in the same manner as steps S11 to S13 of the first embodiment. . Subsequently, the voice search device 100 returns to the voice search process, executes step S4, and ends the voice search process.

  On the other hand, when the number of triphones constituting the triphone train B is less than the threshold (step S21; No), the search unit 67 performs continuous DP matching between the triphone train B and the triphone train A (step S25). . Subsequently, the search unit 67 extracts a partial sequence having the smallest cost Y (step S24). Then, the voice search device 100 returns to the voice search process, executes step S4, and ends the voice search process.

  As described above in detail, according to the present embodiment, the cost X is calculated using the triphone sequence C or the triphone sequence B is used according to the number of triphones constituting the triphone sequence B. Whether to calculate the cost Y is selected. By doing so, when the number of triphones in the triphone train B is small, it is possible to prevent the data amount from becoming extremely small by excluding the first and last biphones in the triphone train B. As a result, the search accuracy is increased.

(Embodiment 3)
Next, Embodiment 3 of the present invention will be described in detail. The hardware configuration and functional configuration of the voice search device 100 in the present embodiment are the same as those in the first embodiment, but the function of the search unit 67 is different. The search unit 67 calculates both the cost X (first similarity) and the cost Y (second similarity), and extracts a substring having the smallest costs X and Y.

  The flow of the voice search process by the voice search device 100 according to this embodiment will be described in detail. The flow of the voice search process according to the present embodiment is different from the search process (step S3) in the voice search process according to the first embodiment. In the following, with reference to FIG. 12, the search process of the present embodiment will be described mainly with respect to the differences from the first embodiment.

  The search unit 67 performs continuous DP matching between the triphone train B and the triphone train A (step S31). Subsequently, the search unit 67 generates a triphone string C (step S32). Subsequently, the search unit 67 performs continuous DP matching between the triphone train C and the triphone train A (step S33). Then, the search unit 67 extracts a partial sequence having the smallest costs X and Y (step S34). Subsequently, the voice search device 100 returns to the voice search process, executes step S4, and ends the voice search process.

  As described above in detail, according to the present embodiment, when the number of triphones constituting the triphone train B is small and the amount of data is extremely reduced by removing the biphone, the cost Y is used. A subsequence is extracted. On the other hand, if the number of triphones is sufficient and it is advantageous to exclude biphones, the subsequence is extracted based on the cost X. As described above, since the partial sequence can be extracted in a flexible manner corresponding to the number of triphones, the search accuracy can be further improved.

(Embodiment 4)
Next, Embodiment 4 of the present invention will be described in detail. The hardware configuration and functional configuration of the voice search device 100 in the present embodiment are the same as those in the first embodiment, but the function of the search unit 67 is different. The search unit 67 reduces the weights of the first and last biphones of the triphone train B from the remaining triphones constituting the triphone train B, and the triphone train B and the partial trains included in the triphone train A The sub-sequence with the smallest calculated cost Z is extracted from the triphone sequence A.

  For example, the search unit 67 multiplies the distance d between the first biphone “sh + i” of the triphone train B and the triphone of the triphone train A corresponding to this by 1/2, A value obtained by multiplying the last biphone “ri” and the triphone distance d of the triphone line A corresponding thereto by ½ is calculated. The search unit 67 performs continuous DP matching using the calculated value. Then, the search unit 67 calculates the cost Z of the triphone sequence B and the partial sequence included in the triphone sequence A, and extracts the partial sequence having the smallest calculated cost Z.

  Next, the flow of the voice search process by the voice search device 100 in the present embodiment will be described in detail. The flow of the voice search process according to the present embodiment is different from the search process (step S3) in the voice search process according to the first embodiment. Hereinafter, with reference to FIG. 13, the search process of the present embodiment will be described mainly with respect to the differences from the first embodiment.

  The search unit 67 performs continuous DP matching between the triphone train B and the triphone train A (step S41). However, in calculating the cost Z, the search unit 67 multiplies the distance d between the first and last biphones of the triphone row B and the triphones of the triphone row A by 1/2. Subsequently, the search unit 67 extracts a partial sequence having the smallest cost Z (step S42). Then, the voice search device 100 returns to the voice search process, executes step S4, and ends the voice search process.

  As described above, according to the present embodiment, the weight of the distance between the first and last biphones of the triphone row B and the triphones of the triphone row A in the cost calculation is reduced. By doing so, it is possible to reduce the increase in the cost Z due to the fact that the number of phonemes of the first and last biphones of the triphone sequence B do not match the number of phonemes of the triphone of the triphone sequence A. As a result, search accuracy is increased.

  Further, in each of the above embodiments, the search unit 67 may extract a partial sequence on the condition that either the cost is less than a predetermined threshold or the cost is within a predetermined order in ascending order. It was. In this way, when a plurality of partial sequences that match or are similar to the search word are extracted, the user can check a plurality of search results.

  In the above embodiments, the similarity is calculated by continuous DP matching. Thereby, the similarity can be calculated efficiently.

  In each of the above embodiments, the phone model is a triphone. When expressed in phonemes without using a triphone, for example, the phoneme “r” is expressed as “r” regardless of the phonemes nearest to the front side and the back side. On the other hand, using a triphone, the same phoneme “r” is expressed by including information that the front nearest phoneme is “o” and the front nearest phoneme is “i”, for example, “o−r + i”. can do. Thus, since the triphone includes a larger amount of data, the phoneme can be expressed more precisely, and the search accuracy is improved.

  Note that the audio data to be searched stored in the audio data storage unit 61 may be read by the CPU 6 from a medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc) or a server via an Internet line. You may make it download from etc. The data format of the audio data may be a WAVE format, an AIFF (Audio Interchange File Format) format, or the like.

  The search term is not limited to a word such as “signal processing” but may be a keyword, a phrase, or the like including a plurality of words.

  Note that the search unit 67 may generate a triphone sequence excluding either the first or last biphone of the triphone sequence B as the triphone sequence C.

  In the second embodiment, the threshold of the number of triphones constituting the triphone row B is preferably 10 or more and less than 20 and more preferably 10 or more.

  Further, the input unit 64 may perform voice recognition processing or the like on voice data as a search term input via the input device 4. The input unit 64 inputs the text data obtained by converting the voice data by the voice recognition process to the search word acquisition unit 66. More specifically, the input device 4 includes, for example, a microphone, and the input unit 64 converts voice data corresponding to “signal processing” uttered by the user to the microphone into text data by voice recognition processing. To the search term acquisition unit 66. By doing so, the user can utter and input a search word, and convenience is further improved.

  In each of the embodiments described above, the search unit 67 outputs the reproduction start time of the first triphone among the triphones constituting the extracted partial sequence to the output unit 69. By doing so, the user can reproduce the audio data from the portion matching the search word by designating the displayed reproduction start time and instructing the reproduction of the audio data. Further, the search unit 67 refers to the information table stored in the triphone sequence storage unit 63, acquires audio data including the extracted partial sequence and several seconds before and after it from the triphone sequence storage unit 63, and Audio data may be output to the output unit 69. In this case, the output unit 69 outputs the audio data output from the search unit 67 via a speaker included in the output device 5. By doing so, the user can check the search result including the front and back of the voice that matches the search word, thereby further improving convenience.

  In addition, the search unit 67 may output the reproduction start time of the first triphone in the extracted partial sequence together with the audio data including the extracted partial sequence and several seconds before and after the extracted partial sequence to the output unit 69. In this case, the search unit 67 may output the reproduction start time of the first triphone in a plurality of partial sequences that satisfy a predetermined condition. The search unit 67 outputs, to the output unit 69, audio data including the upper part having a low cost X, for example, the reproduction start times of the five partial sequences, the partial sequences associated with the respective reproduction start times, and several seconds before and after the partial sequences. May be. In this case, the output unit 69 displays the playback start times of the top five partial columns on the screen. Further, the output unit 69 may output audio data corresponding to the reproduction start time selected by the user among the reproduction start times of the upper five subsequences indicated.

  In addition, when there are a plurality of search target audio data, the search unit 67 acquires the data name of the audio data including the extracted partial sequence from the audio data storage unit 61 and outputs it to the output unit 69. Also good. In this case, the output unit 69 displays the data name of the audio data output from the search unit 67 on the screen.

  Further, the search unit 67 may omit the process of tracing back the route, and extract a partial sequence including triphones corresponding to a predetermined number of nodes from the node with the lowest cost X among the nodes of i = 7. . If the process of tracing back the route is omitted in this way, the partial sequence is roughly extracted, but the amount of calculation required to extract the partial sequence can be suppressed.

  In each of the above-described embodiments, the phoneme feature value is a short-time spectrum in the frequency band of the phoneme waveform data, but the phoneme feature value is a logarithmic value of the short-time spectrum in the phoneme frequency band. Also good.

  In addition, the search target acquisition unit 62 and the search word acquisition unit 66 may each generate a biphone string based on the search target speech data and the phoneme string obtained by converting the search word. In this case, since there is no phoneme immediately before the first phoneme and immediately after the last phoneme, the first and last phoneme models are monophones such as “sh” and “i”. The search unit 67 may perform continuous DP matching between the biphone strings excluding the first and last monophones (“sh”, “i”) of the search word biphone strings and the partial strings included in the search target biphone strings. .

  In each of the above embodiments, the distance d between the triphones is the average of the distances between the three phonemes included in the triphone, but is calculated by increasing the weight of the distance between the central phonemes of the triphone. A weighted average of distances between three phonemes may be used.

  The voice search device 100 includes a television receiver, a terrestrial digital broadcast receiver, a recording device, a recording device, a personal computer (PC), a mobile terminal such as a mobile phone, a personal handy-phone system (PHS), a personal digital device (PDA). Assistant or Personal Data Assistance), game machine, or the like.

  In each of the above embodiments, the software program to be executed is stored in a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read-Only Memory), a DVD, or an MO (Magneto-Optical disc). It is also possible to configure a system that executes the above-described processing by distributing and installing the software program.

  Further, the software program may be stored in a disk device or the like of a predetermined server device on a communication network such as the Internet, and may be downloaded, for example, superimposed on a carrier wave.

  In addition, when the above functions are realized by sharing the OS, or when the functions are realized by cooperation between the OS and the application software program, only the part other than the OS may be stored in the medium and distributed. Moreover, you may download.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition unit for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence And a search unit that extracts a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output unit that outputs information related to the audio data corresponding to the partial sequence extracted by the search unit;
A voice search device comprising:

(Appendix 2)
The search unit
When the number of the phoneme models constituting the second phoneme model sequence is less than a threshold value, a second similarity between the second phoneme model sequence and a partial sequence included in the first phoneme model sequence Calculating a degree, and extracting a partial sequence in which the calculated second similarity satisfies a predetermined condition from the first phoneme model sequence;
The voice search device according to Supplementary Note 1, wherein

(Appendix 3)
The search unit
Both the first similarity and the second similarity between the second phoneme model sequence and the partial sequence included in the first phoneme model sequence are calculated, and the first and second similarities are predetermined. A partial sequence that satisfies the following condition is extracted from the first phoneme model sequence:
The voice search device according to Supplementary Note 1, wherein

(Appendix 4)
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition unit for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search unit that calculates a similarity with a partial sequence included in one phoneme model sequence, and extracts a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output unit that outputs information related to the audio data corresponding to the partial sequence extracted by the search unit;
A voice search device comprising:

(Appendix 5)
The search unit
The partial sequence is extracted with the predetermined condition that either the similarity is equal to or higher than a predetermined threshold and the similarity is within a predetermined order in descending order.
The voice search device according to any one of appendices 1 to 4, wherein

(Appendix 6)
The search unit
Calculating the similarity by continuous DP matching;
The voice search device according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
The phoneme model is
A triphone including a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side.
The voice search device according to any one of appendices 1 to 6, characterized in that:

(Appendix 8)
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition process for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition step of acquiring a second phoneme model sequence in which phoneme models including one are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence And a search step of extracting a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output step of outputting information on the audio data corresponding to the partial sequence extracted in the search step;
Voice search method including

(Appendix 9)
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition process for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition step of acquiring a second phoneme model sequence in which phoneme models including one are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search step of calculating a similarity with a partial sequence included in one phoneme model sequence, and extracting a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output step of outputting information on the audio data corresponding to the partial sequence extracted in the search step;
Voice search method including

(Appendix 10)
Computer
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order Search target acquisition unit that acquires model columns,
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence A search unit that extracts a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output unit that outputs information about the audio data corresponding to the partial sequence extracted by the search unit;
Program to function as.

(Appendix 11)
Computer
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order Search target acquisition unit that acquires model columns,
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search unit that calculates a similarity with a partial sequence included in one phoneme model sequence, and extracts a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output unit that outputs information about the audio data corresponding to the partial sequence extracted by the search unit;
Program to function as.

  DESCRIPTION OF SYMBOLS 1 ... ROM, 2 ... RAM, 3 ... External storage device, 4 ... Input device, 5 ... Output device, 6 ... CPU, 61 ... Audio | voice data storage part, 62 ... Search object acquisition part, 63 ... Triphone row | line storage part, 64 ... input unit, 65 ... conversion table storage unit, 66 ... search word acquisition unit, 67 ... search unit, 68 ... inter-triphone distance storage unit, 69 ... output unit, 100 ... voice search device

Claims (11)

A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition unit for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence And a search unit that extracts a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output unit that outputs information related to the audio data corresponding to the partial sequence extracted by the search unit;
A voice search device comprising: The search unit
When the number of the phoneme models constituting the second phoneme model sequence is less than a threshold value, a second similarity between the second phoneme model sequence and a partial sequence included in the first phoneme model sequence Calculating a degree, and extracting a partial sequence in which the calculated second similarity satisfies a predetermined condition from the first phoneme model sequence;
The voice search device according to claim 1. The search unit
Both the first similarity and the second similarity between the second phoneme model sequence and the partial sequence included in the first phoneme model sequence are calculated, and the first and second similarities are predetermined. A partial sequence that satisfies the following condition is extracted from the first phoneme model sequence:
The voice search device according to claim 1. A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition unit for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search unit that calculates a similarity with a partial sequence included in one phoneme model sequence, and extracts a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output unit that outputs information related to the audio data corresponding to the partial sequence extracted by the search unit;
A voice search device comprising: The search unit
The partial sequence is extracted with the predetermined condition that either the similarity is equal to or higher than a predetermined threshold and the similarity is within a predetermined order in descending order.
The voice search device according to claim 1, wherein the voice search device is a voice search device. The search unit
Calculating the similarity by continuous DP matching;
The voice search device according to claim 1, wherein the voice search device is a voice search device. The phoneme model is
A triphone including a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side.
The voice search device according to any one of claims 1 to 6. A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition process for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition step of acquiring a second phoneme model sequence in which phoneme models including one are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence And a search step of extracting a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output step of outputting information on the audio data corresponding to the partial sequence extracted in the search step;
Voice search method including A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order A search target acquisition process for acquiring a model column;
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition step of acquiring a second phoneme model sequence in which phoneme models including one are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search step of calculating a similarity with a partial sequence included in one phoneme model sequence, and extracting a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output step of outputting information on the audio data corresponding to the partial sequence extracted in the search step;
Voice search method including Computer
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order Search target acquisition unit that acquires model columns,
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The first similarity between a third phoneme model sequence that is a phoneme model sequence excluding at least one of the first and last phoneme models of the second phoneme model sequence and a subsequence included in the first phoneme model sequence A search unit that extracts a partial sequence in which the calculated first similarity satisfies a predetermined condition from the first phoneme model sequence,
An output unit that outputs information about the audio data corresponding to the partial sequence extracted by the search unit;
Program to function as. Computer
A first phoneme in which each phoneme included in the speech data to be searched is a central phoneme, and a phoneme model including the central phoneme and at least one of the phoneme nearest to the front side and the phoneme nearest to the back side is arranged in time series order Search target acquisition unit that acquires model columns,
A search word to be searched for the speech data is converted into a phoneme string, each phoneme constituting the converted phoneme string is set as a central phoneme, and at least a central phoneme, a phoneme nearest to the front side of the central phoneme, and a phoneme nearest to the rear side A search word acquisition unit that acquires a second phoneme model sequence in which phoneme models including one of them are arranged in chronological order;
The weight of at least one of the first and last phoneme models in the second phoneme model sequence is reduced from the remaining phoneme models constituting the second phoneme model sequence, and the second phoneme model sequence and the second phoneme model sequence are reduced. A search unit that calculates a similarity with a partial sequence included in one phoneme model sequence, and extracts a partial sequence in which the calculated similarity satisfies a predetermined condition from the first phoneme model sequence;
An output unit that outputs information about the audio data corresponding to the partial sequence extracted by the search unit;
Program to function as.

Images