JP2011128903A - Sequence signal retrieval device and sequence signal retrieval method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sequence signal retrieval device and a sequence signal retrieval method for efficiently processing a plurality of candidates from a sequence signal including an error. <P>SOLUTION: The retrieval processing of voice data by the use of a retrieval word is achieved by representing a two-dimensional array with a syllable string of a voice recognition result of voice data and a syllable string of a retrieval word, constructing a plane as an element of the two-dimensional array by the use of a distance between syllables, that is, similarity, and detecting a straight line on the plain. The use of indexing with a distance to be considered enables high-speed detection and the consideration of a plurality of candidates of the voice recognition enables highly precise detection. In addition, indexing with a distance to be considered allows a search from a candidate close in a distance or an approximately close candidate so that the search need not be suspended and an appropriate threshold can be set. Furthermore, the search is not suspended with a threshold, and even if noise is large, a solution can be found sequentially from a candidate close in distance or an approximately close candidate. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

The present invention relates to a search device and a search method, and in particular, a search device for searching for a sequence signal such as a search for words used in speech data or a search from text data including errors after OCR. It relates to the search method.

With the expansion of audio / image / video recording / editing equipment and the development of the information communication network including the Internet, anyone can easily create and publish content, and the information explosion of multimedia content has progressed. It's getting on. In many cases, metadata other than the file name and title is not given to these contents, and it is difficult to reach the target contents only with the conventional text-based search technology. On the other hand, in the case of content including spoken language, a search using language information can be performed by using a large vocabulary continuous speech recognition technology. Search technology for such spoken language information is “spoken document search (Spoken).
Document Retrieval) or simply “Voice Search”, it is an indispensable technology in the information explosion era of multimedia contents.

The problem of specifying the position where an input search word (referred to as a query or a pattern) appears in the voice data in the voice document search is the search word detection (Spoken).
It is called “Term Detection (STD)”, “Known Item Retrieval”, “Speech Keyword Search”, or “Speech Search”, and is a problem that has been actively researched in the field of speech information processing. In 1997, a speech document search track (SDR) of evaluation type workshop TREC organized by NIST, USA
Track) evaluated Known Item Retrieval [Non-Patent Document 1]. In 2006, NIST again spoke
Since Term Detection is set as a research topic, STD research that focuses on the detection of unknown words has been actively conducted since then [Non-Patent Document 2]. A special session for speech document search was also organized in 2009 at ICASSP (International Conference on Acoustic, Speech and Signal Processing), which is a representative international conference on speech information processing. In Japan, the working group of the IPSJ Spoken Language Information Processing Study Group is developing a test collection for STD evaluation [Non-patent Document 3].

Most conventional methods for STD deal with recognition errors in speech recognition within the framework of a search method for text that is relatively researched. TDR SDR
In Track, it is shown that performance can be improved by using a plurality of candidates lower than the second in addition to the first candidate of the speech recognition result. Subsequent methods include a method for efficiently expressing a plurality of recognition result candidates, for example, as a typical method,
Network [Non-Patent Document 4] and TALE (Time-Anchored Lattice)
(Expansion) [Non-Patent Document 5], and most of the search uses an indexing method similar to a text search method without error. Since these methods use an exact index, there is a problem that it cannot be detected at all if there is a leak in indexing. In particular, the handling of recognition vocabulary words (also called unknown words) in speech recognition is a problem. On the other hand, a method using both the phoneme and syllable recognition results has been proposed, but the recognition rate is a problem. In addition, if the number of recognition candidates used for dealing with misrecognition is increased, the number of indexes increases and search efficiency deteriorates.

  On the other hand, Teshima et al. Have proposed a search method in which an indexing method using a suffix array is applied to STD [Non-Patent Document 6]. Although it is positioned as a method in which approximate character string matching is applied to STD, it is difficult to handle a plurality of recognition candidates with a suffix array (or suffix tree), and there is a problem in search accuracy. In addition, since any of the conventional methods applies the text-based indexing method as it is, the index itself includes only binary information corresponding to match / mismatch. Therefore, there is a problem that it is necessary to calculate a distance using DP (Dynamic Programming) matching or the like during or after the search.

On the other hand, the problem of finding a portion similar to a pattern in text data is “approximate character string matching (Approximate
String Matching) ”. Approximate string matching is an online method that performs matching without preprocessing the text assuming that the text is given on the fly, and preprocessing the text assuming that the text is given in advance. Thus, there are two types of off-line methods for performing indexing.

  Further, the indexing methods for offline approximate character string matching are classified into the following three types [Non-patent Document 7].

(1) Method based on suffix tree (Suffix Tree) or suffix array (Suffix Array) (2) Method using n-gram index (3) Method using index on metric space [Non-patent Document 8]

When these methods are used for STD, the above (1) cannot handle a plurality of candidates for speech recognition, and the above (2) requires that the detected position includes a search word and a perfect match portion of a certain ratio or more. There is a problem such as. In addition, any of the above conventional methods (1) and (2) is a method for the problem setting “search for all candidates within a certain error range”, and “the top N candidates with the shortest distance” In order to adapt to the problem of “determining“ On the other hand, the method using the index on the metric space of (3) has not been applied to the STD so far.

John S. Garofolo and Cedric G. P. Auzanne and Ellen M. Voorhees, “The TREC SpokenDocument Retrieval Track: A Success Story”, In Proceedings of TREC-9, page107-129, 1999. NIST, “TheSpoken Term Detection Evaluation”: http://www.itl.nist.gov/iad/mig/tests/std/2006/index.html Yoshiaki Ito, Hiromitsu Nishizaki, Hiroki Hujo, Hiroki Nanjo, Tomoaki Akiba, Kiyoaki Aikawa, Tatsuya Kawahara, Seiichi Nakagawa, Tomoko Matsui, Yoichi Yamashita, “Development of Test Collection for Searching Search Terms in Speech -Interim Report-”, Information Proceedings of IPSJ, Vol.2009-SLP-78 No.4, 2009. L. Mangu, E. Brill and A. Stolcke: “Finding Consensus in Speech Recognition: Word Error Minimization and Other Applications of Confusion Network” Computer Speech and Language, Vol. 14, No. 4, pp. 373-400, 2000. P. Yu, Y. Shi and F. Seide, “Approximate Word-Lattice Indexing with Text Indexers: Time-Anchored LatticeExpansion”, In Proceedings of International Conference on Acoustic, Speech and Signal Processing (ICASSP), pp.5248-5251, 2008 . Shigeki Teshima, Koichi Katsuta, Tsuneo Nitta: “High-speed keyword speech search system using Suffix Array”, Proceedings of the Acoustical Society of Japan 2009 Fall Meeting, 1-R-26 (2009-9). G. Navarro, et al., “Indexingmethods for approximate string matching”, IEEE Data Eng. Bull., 24 (4): 12-27, 2001. G. Navarro, et al., “A Metric Index for Approximate String Matching”, Theoretical Computer Science (TCS) 352 (1-3): 266-279, 2006.

  Since the conventional method is based on an indexing method for unambiguous text, (1) a method of handling a plurality of candidates, such as a speech recognition result, and (2) a method of handling a continuous distance , So there was a problem in sacrificing either accuracy or efficiency. Regarding (1), since binary indexing of match / mismatch is applied, a plurality of candidates cannot be handled at all, or if a plurality of candidates are handled, the number of candidates increases and efficiency deteriorates. Regarding (2), when handling ambiguous information such as speech, it is necessary to handle continuous distances (likelihood, reliability, probability, etc.).

  The conventional method aims to handle discrete distances (n characters are different, etc.), and the index itself does not use distance information. Therefore, there is a problem that it is necessary to calculate the distance using DP matching or the like at the time of search or after the search. Further, the conventional method has a problem that a threshold value is required for searching, and it is difficult to set an appropriate threshold value.

Therefore, an object of the present invention is to provide a search device and a search method capable of efficiently performing a plurality of candidates and simplifying search processing.

  In order to achieve the above object, the present inventors have made the present invention as a result of intensive studies.

  That is, the present invention according to the sequence signal search apparatus is configured to divide the sequence signal information to be searched for each predetermined unit and extract the signal feature for each unit, and the signal feature extracted by the signal feature extraction unit Similar distance calculation means for calculating the distance indicating the similarity between the feature quantity and the reference signal feature, and the feature vector in which the signal features are sequentially arranged from the minimum value of the similarity distance calculated by the similarity distance calculation means Feature vector generation means for generating each signal feature, the sequence signal information, the reference signal feature, a distance indicating the similarity of the feature amount of the signal feature, a storage device for storing the feature vector, and search signal information Search signal feature extracting means for extracting a search signal feature for each unit, and for each reference signal feature that matches the search signal feature, And a signal feature sequence generating means for generating a predetermined signal sequence by sequentially selecting from the minimum value of each feature vector, and the signal feature sequence generated by the signal feature sequence generating means is a part of the search signal feature Alternatively, it is possible to make a gist of a series signal search device including a determination unit that determines a search result when all are specified, and an output unit that outputs the search result.

  The determination means of the invention stores a column in which the search signal information is arranged from the top and a column in which the sequence signal information is arranged from the top, and the signal feature sequence on a matrix composed of the two columns When the signal feature sequences generated by the generation unit are aligned in a straight line, the determination unit can determine as a search result.

  Further, in the above invention, the sequence signal information may be speech data, and the signal feature and the reference signal feature may be signal features characterized by phonemes, syllables, or phonemes or syllable n-grams.

  On the other hand, the present invention relating to the sequence signal search method is a sequence signal search method comprising a preprocessing step and a runtime processing step in a search method via a computer connected to information stored in a storage device. The pre-processing step divides the sequence signal information to be searched into predetermined units, extracts a signal feature for each unit, and extracts the signal features and reference signal features extracted by the signal feature extraction step. For each reference signal feature, a similarity vector calculation process for calculating a distance indicating the similarity between the feature quantities and a feature vector in which signal features are sequentially arranged from the minimum value of the similarity distance calculated by the similarity distance calculation process. The search signal feature extraction process that divides the search signal information into predetermined units and extracts the search signal features for each unit. A signal feature sequence generating step of generating a predetermined signal sequence by aligning the feature vectors for each reference signal feature that matches the search signal features and sequentially selecting from the minimum value of each feature vector; and the signal feature sequence generation A sequence comprising: a determination process for determining a search result when a signal feature sequence generated by the process specifies a part or all of a search signal feature; and an output process for outputting the search result The gist of the signal search method.

  As a determination process of the above invention, a column in which the search signal information is arranged from the top and a column in which the sequence signal information is arranged from the top are stored, and the signal feature sequence is stored on a matrix composed of the two columns. When the signal feature sequences generated by the generation process are arranged in a straight line, it can be a determination process for determining as a search result.

  In the above invention, the signal feature extraction step is a signal feature extraction step of dividing speech data into phonemes, syllables or phoneme or syllable n-grams and extracting signal features for each unit, The search signal feature extraction process may be a search signal feature extraction process in which character data is divided in units of phonemes, syllables or phonemes or syllable n-grams, and signal features are extracted for each unit.

  The search method with the above configuration is considered to be classified as a technique using an index in the metric space among the conventional techniques, whereas the technique in the conventional technique directly indexes in the metric space of the entire character string. The present invention divides a metric space into linearly reconfigurable partial metric spaces, and performs metric space indexing in each partial space.

The search method according to the present invention is one of STD methods for searching a portion similar to an input search word from large-scale voice data at high speed in descending order of similarity. In addition to keyword search from speech data, the distance between each position of the search target data, in particular continuous, is a unit in which the search term is divided, such as phonemes, syllables, n-grams of syllables and phonemes in the case of speech search The present invention is generally applicable to a similar part search from a series in which a distance or similarity by value is defined. For example, the present invention can be applied to approximate character string collation and search from text including errors after OCR. In particular, since the search target data can be applied even when there are a plurality of candidates at each position, the search target data is suitable for a similar column search from a sequence including an error.

  According to the present invention, high-speed detection is possible by using indexing in consideration of distance, and high-precision detection is also possible by considering a plurality of speech recognition candidates.

  Indexing in consideration of distance has a good property that the search is advanced from candidates that are close or approximately close. In addition, since the index does not take distance into consideration, the prior art needs to treat possible candidates equivalently, and therefore requires a threshold value to stop the search, and it is difficult to set an appropriate threshold value. However, according to the search method of the present invention, there is no need to set a threshold value, and this problem can be avoided.

Thus, the search device of the present invention enables a new STD system operation method due to its nature. Also, according to the search method of the present invention, the search is not terminated at the threshold value, so that solutions can be found in order from candidates that are close to each other or approximately close to each other no matter how loud the noise is. It has a good nature. This property allows for unprecedented system operation. For example, if operation is performed until the first N solutions are found, or if the detection takes time, it is suggested that there are many conflicting candidates with similar distances and there are no search terms in the first place, so that good results cannot be obtained. Therefore, it is possible to perform operations that take advantage of this property when constructing a system, such as stopping the search within a certain period of time.

It is a schematic diagram of the search term detection by the straight line detection on xy plane. It is a schematic diagram of a syllable distance arrangement. It is a schematic diagram which shows the vector (stack) of the position i sorted in order of distance. It is a flowchart of a pre-processing (indexing) algorithm. It is a flowchart of a search algorithm. It is a flowchart of a search algorithm. It is a schematic diagram which shows the example of neighborhood production | generation. It is a schematic diagram of the distance between syllables in consideration of Confusion Network. It is a schematic diagram showing the distance between syllables which considered insertion, deletion, and an error. It is a schematic diagram of the index data creation apparatus for the pre-processing in the embodiment of the present invention. It is a schematic diagram of the search processing apparatus in the embodiment of the present invention. It is a graph which shows the STD experimental result of continuous DP in embodiment of this invention. It is a graph which shows the STD experiment result in embodiment of this invention.

  Embodiments of the present invention will be described below. The basic principle of the present invention is characterized by an efficient indexing (preprocessing) means using the nature of the STD problem and a search method (runtime processing) using a straight line detection means.

  Examples of the series signal information include audio data, text data, music data, and image data. In the present embodiment, the audio data will be described as an example. Moreover, examples of the unit for dividing the speech data include phonemes, syllables, or phonemes or n-grams of syllables. For ease of explanation, the speech data is divided in units of syllables, and for each syllable, A case where individual syllables are recognized as signal features will be described. Note that the signal features and their similarities in each syllable are characterized by an analysis method that focuses on the difference in the utterance method as well as the physical difference between the pronounced speech waveform and the reference syllable waveform. The scale may be determined by criteria. More specifically, syllables can be modeled by a Hidden Markov Model (HMM), and the distance between the models can be used as a scale. Therefore, the “syllable” shown in the following description may include the “signal feature for each syllable” characterized as described above, and the reference signal feature is an accurate pronunciation. This means the signal characteristics for each syllable.

Consider a plane in which the syllable string of the speech recognition result of the voice data is taken on the x axis and the syllable string of the search word is taken on the y axis (FIG. 1). STD can be regarded as a problem of detecting a straight line from this plane. The xy plane is a two-dimensional array D [i] [j] of speech data length n × search word length m in which a distance (similarity) between syllables is substituted as an element.
(0 ≦ i <n, 0 ≦ j <m).

  At this time, for example, in order to detect a straight line y = x + c with a slope of 1 (which is the most common matching case), Equation 1 is calculated for each i (0 ≦ i <n), and a place where T [i] is small Can be formulated into the problem of detecting as a similar part.

Hereinafter, the sum of <Expression> related to j (0 ≦ j <m−1) is expressed as Σ_j <Expression>.

  In the case of image straight line detection, since a two-dimensional array D [i] [j] is obtained on the spot, all calculations must be performed online.

  On the other hand, in the case of STD, since it can be assumed that the speech data is known before the search, a certain division unit of the search word (phonemes, syllables, syllables and n-grams of phonemes, etc., hereinafter referred to as syllables) a It is possible to calculate the distance array D (a) [i] (0 ≦ i <n) at each position of the audio data offline (FIG. 2). At the time of retrieval, the array of distances D (a [0]) [i], D (a [1]) [i is determined by the syllable series a [0], a [1],. ],..., D (a [m−1]) [i] can be arranged in order in the y-axis direction to form a two-dimensional array D [i] [j] (ie, D [i] [j] = D (a [j]) [i]).

  Furthermore, the distance array D (a) [i] can be sorted offline in ascending order of distance. That is, the signal features are sequentially arranged from the minimum value of the similar distance. S (a) is a stack in which each position i in the audio data for syllable a is sorted in ascending order of distance according to D (a) [i], and the top of the stack is S (a). Top (FIG. 3).

  A flowchart of the algorithm for calculating D (a) [i] and S (a) described above is shown in FIG. Hereinafter, the procedure will be described with reference to FIG.

  First, syllable Confusion Network CN [i] is prepared by means such as voice recognition of voice data (step A1). The syllable set A is initialized and all syllables are substituted (step A2). One syllable a is extracted from A, and position i is initialized to 0 (step A3). The distance between a and CN [i] is calculated, assigned to the distance array D (a) [i], and i is incremented (step A4). If the position i reaches the document length n, the process returns to the next step A6, and if not, the process returns to step A4 (step A5). According to the obtained D (a) [i], the document positions are sorted in ascending order to obtain the sorted position vector S (a) (step A6). If syllables still remain in the syllable set A, the process returns to step A3 (step A7). The obtained position vector S (a) and distance vector D (a) [i] (both vectors are referred to as a feature vector) are output (step A8).

  When S (a) calculated in the preprocessing is used as a search index, an efficient detection method for detecting in order from a probable position can be configured. More specifically, high-speed search word detection is possible according to the following procedure.

  A flowchart of the search process is shown in FIG. Hereinafter, the procedure will be described with reference to FIG.

First, a search word is input, and a stack sequence S (a [0]) calculated in advance by preprocessing according to the syllable sequence a [0], a [1],. , S (a [1]),..., S (a [j]),. R = {}, count [i] = 0
(0 ≦ i <n) is initialized (step B1). A set U = {S (a [0]). top, S (a [1]). top,..., S (a [j]). top, S (a [m-1]). top}, j is selected according to a certain criterion, and S (a [j]). remove top from stack S (a [j]), position i = S (a [j]). For top-j, 1 is added (voted) to a counter count [i] that counts the number of times of voting (step B2). For a certain value k, count [i] ≧ k
If is not yet established, the process returns to step B2 (step B3). The position i is added to the search result candidate set R (step B4).
For each candidate i in R, the one that satisfies a certain criterion is output as a search result. The output candidate is removed from R (step B5). If the end condition is not satisfied, the process returns to step B2 (step B6).

  In the above procedure, it should be noted that basically no distance calculation is required for each candidate. Also, as will be described later, depending on the criteria used in step B5, the distance between each candidate

However, it is possible to calculate without adding an expensive calculation only by adding the distance D (a [j]) [i + j] of each syllable obtained in advance.

  As a criterion for selecting j in step B2, an element having the smallest distance in U (argmin_j D (a [j]) [S (a [j]). Top]) may be considered. The end condition of step B6 includes “until the first search result is obtained”, “until the first N search results are obtained”, “until the distance of the search results exceeds a certain threshold value”, and “a certain period of time” It can be considered “until it elapses”. Further, step B5 does not necessarily need to be executed in each iteration. For example, an implementation may be considered in which step B4 is executed once after a predetermined number of times.

  The selection of k in step B3 and the criterion in step B5 is related to accuracy and efficiency. If k in step B3 is reduced, the efficiency is improved, but the accuracy is lowered. Therefore, the result is evaluated according to the criterion of step B5, and a safe one is output.

  The simplest implementation method is that k = m (search word length), and no criteria is imposed on step B5, and the search result is output immediately. Further, in order to improve the efficiency, k may be set small without imposing the criterion of step B5. For example, if k = αm for a certain real number α (0 <α <1), it is output when a ratio α is voted for the search word length.

  A flow chart of these simple implementations that do not impose criteria in step B5 is shown in FIG.

  In FIG. 6, Step B1, Step B2, Step B3, and Step B6 are the same as the corresponding steps in FIG. In step B45 in FIG. 6, the position i is output as it is as a search result.

  In FIG. 5, when k = 1 in particular, when the next criterion 1 is used in step B5, an optimal solution algorithm that guarantees that a candidate that is worse (larger in distance) than a candidate that is output in subsequent iterations is not output is can get.

(Criteria 1) Σ_j D (a [j]) [i + j] ≦ Σ_j S (a [j]). top holds.

  Furthermore, if the candidates are output in order from the candidate with the smallest distance in step B5, the algorithm outputs the solutions in ascending order of the distance. Further, the set R can be efficiently implemented by using a data structure that retains the order such as a binary search tree or a B-tree.

  The validity of Criterion 1 is clear from the following lemma.

(Lemma 1) The final distance T [i] defined by Equation 2 at the position i that has not yet been voted is the sum of the stack top distances Σ_j S (a [j]). top

以上である。

That's it.

(Proof) Since the stack is sorted in ascending order, for any syllable position j for a position i that has not yet been voted,

が成り立つ。よって、

Holds. Therefore,

が成り立つ。（証明終）

Holds. (End of proof)

(Lemma 2) Σ_j S (a [j]). The candidate position i having a distance smaller than the top is voted at a syllable position j at least once.

(Proof) Clarified by the even number of Lemma 1.

As described above, in addition to the candidate set R, Σ_j
S (a [j]). It can be seen that there is no candidate whose distance is smaller than top, and it is guaranteed that an optimum solution is obtained.

  The above is the basic algorithm, but various variations are possible. In the description so far, the case of the simplest straight line y = x + c having a slope of 1 is assumed. This corresponds to a Hamming distance between character strings as a distance scale. By applying neighborhood generation, which is a technique used in the field of approximate character string matching, voting destinations T [i] can be prepared for straight lines and broken lines that are close to straight lines, and the same calculation is possible. It can be applied to edit distances and other distances which are distance scales between character strings (FIG. 7).

In addition, an arbitrary distance obtained from the syllable a and the position i in the audio data can be used for the distance array D (a) [i]. Confusion, a compact method for expressing multiple speech recognition results
Allowing deviation from a straight line in consideration of distances considering multiple candidates using Network, TALE (Time-Anchored Lattice Expansion), etc., distances expressing the plausibility of recognition, insertion errors and deletion errors Therefore, it is possible to use a more complicated distance such as a distance considering the distance between adjacent syllables (FIG. 9).

[Preprocessing] Index data (stack S (a) shown in the above embodiment) is created in the following procedure. (Fig. 10)

1. Audio data 201 recorded in a storage device 102 such as a hard disk or CD-ROM is prepared. The speech data 102 is recognized using a speech recognition decoder installed on the computer 101, and a syllable Confusion that represents a plurality of recognition candidates as a recognition result.
A network 202 is created on the storage device 102. Instead of the syllable confusion network 202, a simpler syllable string as a recognition result may be used as it is. Instead of syllables, arbitrary units obtained by dividing a search word such as phonemes, syllable / phoneme n-grams, and the like may be used (hereinafter collectively referred to as syllables).

2. For the syllable a, the computer 101 calculates the distance from each of the multiple syllable candidates of each value i of the syllable Confusion Network 202, and obtains the minimum value of the distance between syllables. The distance calculation results for all positions are created on the storage device 102 as the syllable distance array 203. The above operation is repeated for all syllables to create a syllable distance array 203 for each syllable.

3. For each syllable a, the positions i are sorted in ascending order using a computer in accordance with the syllable distance array 203, and a vector 204 of syllable positions, which is the output, is created on the storage device.

[Search Processing] Search processing is performed using the syllable position vector (stack) created in the preprocessing. (Fig. 11)

1. The voice data 401 and the syllable position vector 403 for each syllable created by [preprocessing] are prepared in a storage device 302 such as a hard disk. When using a method that requires a distance between a syllable and a position during search processing, a syllable distance array 402 for each syllable is also prepared in the storage device 302. When the speech recognition result text is used as the search result, the speech recognition result text may be prepared on the storage device instead of the speech data 401.

2. A search word (syllable string) given by the system user is input to the computer 301 using an input device 303 such as a keyboard. For all the syllables included in the input syllable string, the corresponding syllable position vector 403 is read from the storage device 302 to the computer 301, and a stack is formed on the computer memory. One stack is prepared for each syllable constituting the search word, as many as the number of syllables of the search word. When using a method that requires a distance between a syllable and a position during the search process, the corresponding syllable distance array 402 is also read into the memory of the computer 301.

  At this time, as the input device 303, any input device capable of inputting a syllable string such as voice recognition and handwritten character recognition can be used in addition to a keyboard. The syllable distance array 402 can be arranged in the storage device 302 as it is instead of being read into the memory, and can be referred to by random access when necessary. The syllable position vector 403 does not need to be read from the storage device into the memory at once, but the vector can be divided into blocks from the head to a certain length, and can be read one block at a time as necessary. In addition, a suffix portion in which a large distance among syllable vectors is recorded is unlikely to be used, and can be deleted to save the storage capacity of the storage device 302.

3. While referring to and manipulating the stack prepared for each syllable on the memory of the computer 301, the search word appearance position is obtained by the method described in the above embodiment. When using a method that requires a distance between a syllable and a position during the search process, the process is also performed with reference to the syllable distance array 402 on the computer memory (or on the storage device).

4). In accordance with the detected search word appearance position, the voice near the search word appearance position is extracted from the audio data 401 on the storage device 302 and used as a search result. When the text of the speech recognition result is used, the text at the search word appearance position is used as the search result. The range to be output as a search result can be arbitrarily selected according to the use, such as the search term itself, a wider range of sentences and documents including the search term. The search result may be presented to the user as it is using the output device 304 such as a voice reproduction device or a display device, or to any device that provides various services such as machine translation, a voice synthesizer, and a Web server. It can also be used as input.

  The storage device 302, the input device 303, the output device 304, and the computer 301, which are the elements constituting the device described in the above [Preprocessing] and [Search processing], can be directly connected or distributed on the network. It is also possible to configure the apparatus by arranging and interconnecting by communication.

  Further, the audio data 401, the syllable distance array 402, and the syllable position vector 403, which are data on the storage device 302, can be arranged in different storage devices. For example, the audio data 401 among the data on the storage device 302 can be arranged in a storage device on a server over a network, and only the syllable position vector can be arranged in a storage device directly connected to the computer 301.

  It is also possible to construct the input / output device 303 with a client on the Web and the other on the Web server.

  Implementation of the apparatus shown in the examples and an evaluation experiment were performed in the following procedure.

  An STD system was constructed for speech data of 177 lectures (about 44 hours) in the “Japanese Spoken Language Corpus (CSJ)”, a corpus that recorded lectures and mock lectures in Japanese. The CSJ 1-best speech recognition results included in the test collection for speech document search using the CSJ as a search target are expanded into phoneme strings, and indexed using phonemes as division units.

  As the distance measure between the phoneme a and the search target position i, the Hamming distance between phoneme discrimination features [Non-Patent Document 6] was used.

  As a search term, the search term of the test collection [Non-Patent Document 3] for search term detection targeting the CSJ was used.

Comparison was made with continuous DP matching using a distance threshold under the same conditions as in the above example. The experimental results are shown in (FIG. 12) and (FIG. 13). From this result, it can be seen that the search efficiency is greatly improved without degrading the search performance.

101 ... Computer 102 ... Storage device 201 ... Audio data 202 ... Confusion Network
203 ... syllable distance array 204 ... syllable position vector 301 ... calculator 302 ... storage device 303 ... input device 304 ... output device 401 ... audio data 402 ... syllable distance array 403 ... syllable position vector

Claims (6)

Signal feature extraction means for dividing the sequence signal information to be searched into predetermined units and extracting signal features for each unit;
Similarity distance calculation means for calculating a distance indicating the similarity of the feature quantity between the signal feature extracted by the signal feature extraction means and the reference signal feature;
Feature vector generation means for generating for each reference signal feature a feature vector in which signal features are sequentially arranged from the minimum value of the similarity distance calculated by the similarity distance calculation means;
A storage device that stores the sequence signal information, the reference signal feature, a distance indicating the similarity of the feature quantity of the signal feature, and the feature vector;
Search signal feature extracting means for dividing search signal information into predetermined units and extracting search signal features for each unit;
Signal feature sequence generating means for aligning the feature vectors for each reference signal feature that matches the search signal features and sequentially selecting from the minimum value of each feature vector to generate a predetermined signal sequence;
A determination unit that determines a search result when the signal feature sequence generated by the signal feature sequence generation unit specifies a part or all of a search signal feature;
A sequence signal search apparatus comprising: output means for outputting the search result. The determination unit stores a column in which the search signal information is arranged from the top and a column in which the sequence signal information is arranged from the top, and the signal feature sequence generation unit on the matrix composed of the two columns. 2. The series signal search device according to claim 1, wherein the sequence signal search device is a determination unit that determines a search result when the generated signal feature sequence is aligned in a straight line. The sequence signal information is speech data, and the signal feature and the reference signal feature are phonemes, syllables, or signal features characterized by n-grams of phonemes or syllables. The series signal search device described. In a method for searching through a computer connected to information stored in a storage device, a sequence signal search method comprising a pre-processing step and a runtime processing step,
The pre-processing process divides the sequence signal information to be searched into predetermined units, and extracts a signal feature for each unit;
A similarity distance calculation step of calculating a distance indicating the similarity of the feature quantity between the signal feature extracted by the signal feature extraction step and the reference signal feature;
A feature vector generating step of generating, for each reference signal feature, a feature vector in which signal features are sequentially arranged from the minimum value of the similar distance calculated by the similar distance calculating step;
The runtime processing process divides the search signal information into predetermined units and extracts a search signal feature for each unit,
Aligning the feature vectors for each reference signal feature that matches the search signal features and sequentially selecting from the minimum value of each feature vector to generate a predetermined signal sequence; and
When the signal feature sequence generated by the signal feature sequence generation process specifies a part or all of the search signal features, a determination process for determining as a search result;
A sequence signal search method comprising: an output process for outputting the search result. The determination process stores a column in which the search signal information is arranged from the beginning and a column in which the sequence signal information is arranged from the beginning, and is performed by the signal feature sequence generation process on a matrix composed of the two columns. 5. The sequence signal search method according to claim 4, wherein the sequence signal search method is a determination process of determining the search result when the generated signal feature sequence is aligned in a straight line. The signal feature extraction step is a signal feature extraction step of dividing speech data into phonemes, syllables or phoneme or syllable n-grams as units, and extracting signal features for each unit, and the search signal feature extraction step is 6. The search signal feature extracting process of dividing character data into phonemes, syllables or n-grams of phonemes or syllables, and extracting signal features for each unit. Sequence signal search method.

Images