JP3798991B2 - Audio signal search method, audio signal search apparatus, program thereof, and recording medium for the program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a technology of a speech search system, and more particularly to a speech signal search method for searching a speech signal section similar to an input or designated speech signal from a speech database.,soundThe present invention relates to a voice signal retrieval device, a program thereof, and a recording medium for the program.
[0002]
[Prior art]
As a prior art of a method for retrieving a speech signal similar to an input speech signal from a speech database, there is one described in Reference Document 1 below.
[Reference 1] Takashi Endo, Masayuki Nakazawa, Hironobu Takahashi, Ryuichi Oka: “Data Representation and Spotting Mutual Search by Self-Organizing Network of Voice and Video”: 1998 Annual Conference of Japanese Society for Artificial Intelligence (12th), S5 -04, pp.122-125.
This is a method of calculating similarity between speech signals by dynamic matching similar to DP matching using an IPM (Incremental Path Method) network, and retrieving speech signals similar to the input speech signal from the speech database. It is. According to this method, by using dynamic matching, the similarity can be calculated even for an audio signal that is nonlinearly expanded and contracted in the time axis direction.
[0003]
[Problems to be solved by the invention]
When attempting to search for audio signals using audio signals, it is necessary to calculate the similarity between the audio signals. In general, a voice signal nonlinearly expands and contracts on the time axis even when the same character string is uttered. Therefore, in order to calculate the similarity between audio signals, it is necessary to use dynamic matching such as DP matching or hidden Markov model (HMM) in order to cope with nonlinear expansion and contraction.
[0004]
However, not all audio signals expand and contract nonlinearly on the time axis. As an example of an audio signal that does not expand and contract non-linearly, a singing voice is given first. There is a potential tempo in the singing voice, so if it is a short period, the tempo of the entire period may be slow or fast, but the tempo will not be disturbed within that period . In other words, if the singing voice is within a short period, it may linearly expand and contract in the entire section, but does not expand and contract in a non-linear manner within that section.
[0005]
Another example of an audio signal that does not undergo non-linear expansion / contraction is an announcer's speech. Announcers are very good at speaking the same words many times with the same tone, so if the words are the same, they will not be nonlinearly stretched. Furthermore, even if the utterance of a general person is as short as one word, it can be considered that there is almost no non-linear expansion / contraction in the section even though linear expansion / contraction occurs in the entire section.
[0006]
In the prior art, dynamic matching is used when calculating the similarity between audio signals. Therefore, dynamic matching that can cope with non-linear expansion and contraction is performed on speech that is only linear expansion and contraction. The dynamic matching has a problem that the calculation time is longer and the search time is longer than the static matching for calculating the distance such as the Euclidean distance and the Manhattan distance between the fixed-length vectors.
[0007]
[Means for Solving the Problems]
In order to solve the above-described problems, the present invention makes it possible to collate feature quantities extracted using a time window in retrieval of speech data at high speed using static matching instead of dynamic matching. For this reason, the voice data of the voice time-series feature amount is linearly expanded and contracted to obtain fixed-length data having a certain time interval length. By doing so, the voice data similar to the search key can be searched at high speed by simple matching between fixed-length data.
[0008]
Specifically, the speech signal retrieval apparatus of the present invention includes speech time series feature amount extraction means, time window cutout means, partial speech time series feature amount fixed length lengthening means, and search key speech time series feature amount extraction means. And voice storage means, search condition input means, feature quantity information comparison means, and display format generation means. Furthermore, a speech time series feature amount linear compression means and a search key speech time series feature amount linear compression means may be provided.
[0009]
The voice time-series feature quantity extraction unit extracts a voice time-series feature quantity from a voice signal that is a time-series signal.
[0010]
The time window cutout means first prepares a reference time window which is a time window having a reference length. Then, a partial speech time-series feature amount having a reference time window length is extracted from the speech time-series feature amount extracted by the speech time-series feature amount extracting unit while the reference time window is gradually shifted. Next, a plurality of types of time windows are prepared with the reference time window as the center. The partial speech time-series feature amount is similarly cut out in the time windows of these lengths.
[0011]
The partial speech time-series feature amount fixed length increasing means is a partial speech time-series feature in which the partial time-series feature amounts cut out by the time window cut-out means in a plurality of types of time windows are linearly expanded and contracted to match the reference time window length. A sequence feature quantity is generated and extracted as a speech time series feature quantity vector.
[0012]
The speech time series feature amount linear compression means linearly compresses the speech time series feature amount of the reference time window length generated by the partial speech time series feature amount fixed length means to a certain length to obtain a speech time series feature. Extract quantity vector.
[0013]
The search key voice time series feature quantity extraction means extracts a search key voice series feature quantity of a reference time window length from a voice signal having a reference time window length inputted as a search key, and extracts it as a search key voice time. Extracted as a series feature vector.
[0014]
The search key speech time series feature quantity linear compression means linearly compresses the speech time series feature quantity of the reference time window length extracted by the search key voice time series feature quantity extraction means to the same length as the speech time series feature quantity vector. Then, the search key speech time series feature vector is extracted.
[0015]
The speech storage means stores the input speech signal to be searched, creates an index from the speech time series feature vector extracted by the speech time series feature amount linear compression means, and stores it together with the speech time series feature vector Further, time window cutout information, which is information for associating the speech time series feature vector with the speech signal section from which the speech time series feature vector is extracted, is also stored.
[0016]
The search condition input means inputs a condition for designating the search key when the voice signal section stored in the voice storage means is used as the search key.
[0017]
The feature quantity information comparing means statically matches the search key voice time series feature quantity vector extracted by the search key voice time series feature quantity linear compression means and the voice time series feature quantity vector stored in the voice storage means by similarity matching. And set the similarity. As a result, it is possible to perform similarity calculation faster than dynamic matching. Then, the speech time-series speech feature vector is output in descending order of similarity to the search key speech time-series feature vector.
[0018]
  The display format generation means associates the voice time-series feature vectors output by ranking from the feature quantity information comparison means with the voice signal section of the extraction source based on the time window cutout information stored in the voice storage means. Output to the display device.
  Further, the search condition input means allows the user to input designation information using one of the search results output by the display format generation means as a search key, and the voice data obtained from the designated search result is used as the search key. The display format generation means has means for outputting the search result.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
[Embodiment 1]
FIG. 1 is a configuration diagram for explaining Embodiment 1 of the present invention. In the first embodiment, an externally input voice is used as a search key.
[0020]
The operation of the present invention includes a speech accumulation phase P1, a speech time series feature vector extraction phase P2 called from it, a speech search phase P3, and a search key speech time series feature vector extraction phase P4 called from it. The The operation of each phase will be described below.
[0021]
[A] Speech accumulation phase P1 and speech time series feature vector extraction phase P2
FIG. 2 is a flowchart for explaining the operations of the speech accumulation phase P1 and the speech time series feature vector extraction phase P2.
[0022]
First, a search target voice signal is input from the search target voice signal input device 20, and the voice storage unit 14 stores this voice signal in the storage device 15 (step S1).
[0023]
Next, the voice time series feature quantity extraction unit 11 extracts a voice time series feature quantity from the inputted voice signal (step S2). As the speech time series feature, for example, the low-order term of the Mel frequency cepstrum coefficient, the first order difference, the second order difference, the sound power, the sound power of each band by the filter bank analysis, etc. are represented by multidimensional vectors. Those arranged in chronological order can be used. An example of speech time series feature is described in Reference Document 2 below.
[Reference 2] Kiyohiro Shikano et al .: “IT text speech recognition system”, Ohmsha, 2001.
Next, in the time window cutout unit 12, first, as shown in FIG. 3, a reference time window is set as a reference time window, and this reference time window is shifted little by little while the reference time window length portion is set. A voice time-series feature amount is cut out (step S3).
[0024]
As shown in Fig. 4, multiple types of time windows centered on the length of the reference time window are set, and as with the reference time window, the time window is shifted gradually and the time window length is adjusted. Partial voice time-series feature amounts are cut out (step S4). In an experimental example to be described later, the reference time window is 150 frames and has a length of about 26 milliseconds per frame. The lower limit of the time window length was 118 frames, and the upper limit was 182 frames. If partial time series feature quantities are cut out using these time windows, information related to the cut-out of the time window is stored in the storage device 15 as time window cut-out information in the voice storage unit 14.
[0025]
Further, as shown in FIG. 5, the partial speech time-series feature amount fixed length increasing unit 13 linearly expands / contracts the partial speech time-series feature amounts clipped by a plurality of types of time windows in the time axis direction. Align to the length of the reference time window (step S5), generate a partial speech time series feature quantity of the reference time window length, and set it as a speech time series feature quantity vector (step S6).
[0026]
Then, an index is constructed from the obtained speech time-series feature vector, and is stored in the storage device 15 of the speech storage unit 14 together with the speech time-series feature vector (step S7). As an index structure of a multidimensional space vector, for example, SR-tree described in the following Reference 3 or A-tree described in Reference 4 can be used.
[Reference 3] Norio Katayama and Shin'ichi Satoh: “The SR-Tree: An Index Structure for High-Dimensional Nearest Neighbor Queries”, In Proc. ACM SIGMOID International Conference on Management of Data, pp. 368-380, May 1997.
[Reference 4] Yasushi Sakurai, Masatoshi Yoshikawa, Shunsuke Uemura, and Haruhiko Kojima: “The A-Tree: An Index Structure for High-Dimensional Spaces Using Relative Approximation”, In Proc. Of the 26th International Conference on Very Large Data Bases (VLDB), pp.516-526, Cairo, September 2000.
[B] Speech retrieval phase P3 and retrieval key speech time-series feature vector extraction phase P4
FIG. 6 is a flowchart for explaining the operations of the speech search phase P3 and the search key speech time-series feature vector extraction phase P4.
[0027]
First, an audio signal having a reference time window length serving as a search key is input using the search key audio signal input device 22 (step S10).
[0028]
Next, the search key voice time series feature quantity extraction unit 18 extracts the search key voice time series feature quantity of the reference time window length from the input voice signal of the reference time window length (step S11). As the search key voice time series feature quantity, the same feature quantity as the voice time series feature quantity in the voice accumulation phase P1 described above is used. This search key voice time series feature quantity is set as a search key voice time series feature quantity vector (step S12).
[0029]
Further, the feature information comparison unit 16 calculates a similarity distance between the obtained search key speech time series feature vector and the speech time series feature vector stored in the storage device 15 by the speech storage unit 14 (step S13). This distance calculation is performed using static matching such as a Euclidean distance or a Manhattan distance between fixed-length vectors, and using an index stored in the storage device 15 of the voice storage unit 14. This makes it possible to perform distance calculation faster than dynamic matching. Then, the speech signal sections of the speech time-series feature vector are ranked in order from the shortest distance (step S14).
[0030]
Finally, the display format generation unit 17 uses the time window cutout information stored in the storage device 15 by the audio storage unit 14 to associate the audio time-series feature vector with the extraction source audio signal section, and displays the display device. 21 (step S15). In this display, for example, when displaying the search result of the part corresponding to the search key from an hourly music program, how many minutes and seconds are from the top of the program in order from the highest ranking result. The list information indicating the audio signal interval and the playback button for that portion are displayed, and when the playback button is pressed, the audio for that portion is output. As a result, it is possible to save the searcher from having to search for a speech signal section suitable for the search purpose. Also, if the database has information such as the song name for the search object, the song name and the like of the search result can be displayed together.
[0031]
[Embodiment 2]
FIG. 7 is a configuration diagram for explaining the second embodiment of the present invention. In the second embodiment, the feature amount extracted using the time window is compressed to reduce the size, and a database having a smaller size can be constructed and searched.
[0032]
In the second embodiment, a linearly compressed partial speech time-series feature quantity that has been linearly expanded and contracted to the length of the reference time window is used as a speech time-series feature vector, and accordingly, a reference time is used. A search key speech time-series feature value vector obtained by linearly compressing the search key speech time-series feature value of the window length on the time axis to the same length as the speech time-series feature value vector is used. Therefore, compared with the first embodiment, the speech time-series feature vector extraction phase P2 called from the speech accumulation phase P1 and the search key speech time-series feature vector extraction phase P4 called from the speech search phase P3 are different. It becomes street.
[0033]
[A] Speech accumulation phase P1 and speech time series feature vector extraction phase P2
FIG. 8 is a flowchart for explaining the operations of the speech accumulation phase P1 and the speech time series feature vector extraction phase P2.
[0034]
As in the first embodiment, an audio signal is input from the search target audio signal input device 20 (step S20). The voice time series feature quantity extraction unit 11 extracts a voice time series feature quantity from the input voice signal (step S21), and the time window cutout part 12 shifts the reference time window little by little while changing the reference time window length. A partial voice time series feature amount is cut out (step S22). Further, partial speech time-series feature quantities are cut out even in a plurality of types of time windows (step S23), and the partial speech time-series feature fixed length unit 13 extracts a plurality of types of partial speech time-series features. The amount is linearly expanded / contracted on the time axis to generate a partial speech time series feature amount aligned with the reference time window length (step S24).
[0035]
Next, as shown in FIG. 9, the speech time series feature amount linear compression unit 30 linearly compresses the partial speech time series feature amounts aligned with the length of the reference time window in the time axis direction. A feature vector is extracted (step S25). As a result, the number of dimensions of the speech time-series feature vector is reduced, the amount of similarity calculation can be reduced, and the capacity of the storage device 15 to be stored can be reduced.
[0036]
Further, in the same manner as in the first embodiment, an index is constructed from the obtained speech time series feature vector, and is stored in the storage device 15 of the speech storage unit 14 together with the speech time series feature vector.
[0037]
[B] Speech retrieval phase P3 and retrieval key speech time-series feature vector extraction phase P4
FIG. 10 is a flowchart for explaining the operations of the speech search phase P3 and the search key speech time-series feature vector extraction phase P4.
[0038]
As in the first embodiment, the search key voice signal input device 22 is used to input a voice signal having a reference time window length as a search key (step S30). A search key voice time-series feature quantity having a reference time window length is extracted from the input voice signal having the reference time window length (step S31).
[0039]
Then, as shown in FIG. 9, the search key speech time-series feature amount linear compression unit 31 linearly compresses the speech time-series feature amount of the reference time window length in the time axis direction, and obtains the search key speech time-series feature amount vector. Extract (step S32). Note that the length of the search key speech time series feature vector is the same as the speech time series feature vector generated in the speech accumulation phase P1.
[0040]
Further, as in the first embodiment, the feature key information comparison unit 16 obtains the search key speech time series feature vector obtained and the speech time series feature vector stored in the storage device 15 by the speech storage unit 14. (Step S33), and the speech time-series feature vectors are ranked in order of increasing distance (step S34).
[0041]
Finally, in the same manner as in the first embodiment, the display format generation unit 17 uses the time window cutout information stored in the storage device 15 of the audio storage unit 14 to convert the audio time series feature vector into the audio signal from which it is extracted. The information is output to the display device 21 in association with the section (step S35). For example, when displaying the search result of the part corresponding to the search key from an hourly music program, the audio signal interval such as how many minutes and how many seconds from the top of the program is displayed in order from the highest ranking result. The list information to be displayed and the playback button for that part are displayed, and when the play button is pressed, the part is displayed in a display format that can output the sound of that part.
[0042]
[Embodiment 3]
11 and 12 are configuration diagrams for explaining Embodiment 3 of the present invention. In the third embodiment, instead of inputting a search key from the outside, when a search is performed once and a search result is obtained, a search key is newly specified from the search result, and voice data similar to the search key is obtained. It can be searched.
[0043]
Multiple search results using voice data as a search key are displayed on the screen as a list in order of similarity, and other similar voice data is searched using the voice data obtained as the search result as a new search key. .
[0044]
For example, in the case of music search, when the “music title” portion of the search result display screen is touched, the music is selected as the search result, and is played back from the portion corresponding to the search key or the beginning of the music and output as a sound. Further, when the “rank” part on the screen is touched, the search result of the music (for example, a part similar to the voice data (3 to 4 seconds) of the original search key in the music) is used as a new search key. Further, a similar music search is performed.
[0045]
As described above, in the third embodiment, the search result is selected again from the search results and the result closest to the search purpose is selected. The speech accumulation phase P1 of the speech signal retrieval apparatus 10 shown in FIG. 11 is the same as the speech accumulation phase P1 of the first embodiment shown in FIG. 1, and the speech accumulation phase P1 of the speech signal retrieval apparatus 10 shown in FIG. This is the same as the voice accumulation phase P1 of the second embodiment shown in FIG. The voice search phase P3 is different from the first and second embodiments described above, and is as follows.
[0046]
[A] Voice search phase P3
FIG. 13 is a flowchart for explaining the operation of the voice search phase P3.
[0047]
As a previous step, search is performed in the same manner as in the first and second embodiments, and the search results are ranked in the order of similarity and displayed on the display device 21 (step S40). If the displayed search result sufficiently matches the search purpose according to the instruction from the user, the search is terminated (step S41).
[0048]
If the search purpose is not met, the search condition input unit 23 causes the search result that is already displayed to be designated as the search key, and selects the result closest to the search purpose. (Step S42). For this reason, when the results ranked in step S40 are displayed on the display device 21, two input buttons are displayed for each result. One is a button that is pressed when designating as a search key, and the other is a button that is pressed when the resulting voice is uttered. The user can specify a search key from the search results by pressing the former button on the display device 21.
[0049]
When the search key is designated, the feature amount information comparison unit 16 searches the stored speech sections having a high similarity with the search key and ranks them in descending order of similarity (step S43). Further, the display format generation unit 17 generates a display format in which the search key can be selected from the search result, and displays it on the display device 21 (step S40).
[0050]
If the displayed search result sufficiently matches the search purpose, the search is terminated (step S41). If it still does not match sufficiently, the search key is selected again, the search is performed again, and the process is repeated until a result that sufficiently matches the search purpose is obtained (steps S40 to S43).
[0051]
[Embodiment 4]
In the above first to third embodiments, partial speech time-series feature quantities having different lengths are extracted from a speech signal to be searched using a plurality of types of time windows, and these partial speech time-series feature quantities are extracted. Is stored in the speech accumulating unit 14 as a speech time-series feature vector to be searched.
[0052]
In the fourth embodiment, the linear expansion / contraction of the partial speech time-series feature amount cut out by the time window is not performed for the search target but for the input as a search key. That is, in the fourth embodiment, the partial speech time series features are not extracted from the search target speech signal using time windows of a plurality of types of lengths, but only the reference time window is used. The amount is extracted and stored in the speech storage unit 14 as a speech time series feature amount vector to be searched. On the other hand, for speech signals input as search keys, partial speech time series features with different lengths are extracted using time windows of different lengths, and these partial speech time series features with different lengths are extracted. Scale the amount linearly to the length of the reference time window.
[0053]
Even if linear expansion / contraction is performed on a key input as a search key, the same search results as those in the first to third embodiments can be obtained. In addition, after aligning with the length of the reference time window, the amount of data to be collated at the time of retrieval can be reduced by performing linear compression to a certain length as in the second embodiment as necessary.
[0054]
〔Experimental result〕
In order to confirm the effectiveness of the present invention, two types of experiments were performed on the singing voice data created for the experiment in the second embodiment.
[0055]
First, in the first experiment, it was confirmed that the matching corresponding to linear expansion / contraction according to the present invention is more effective in singing voice search than the matching not corresponding to linear expansion / contraction. It was confirmed that the present invention is sufficiently effective in the search of singing voice even when compared with the conventional method in which matching is implemented.
[0056]
[A] Experimental conditions common to both experiments
Fifteen subjects are divided into three groups, female, male and mixed, and a list of 62 song names is given to each group. A person who knows the song in the song name list sings a part of the phrase (about 10 seconds) and stores 62 × 3 singing voices (about 30 minutes in total) in the database.
[0057]
Twelve songs are arbitrarily selected from the singing voices of one subject group, and about one phrase (reference time window length: 150 frames, about 4 seconds) is taken out as a search key. And the same phrase part of the singing voice of the test subject of two other groups is searched as a matching result.
[0058]
In this experiment, a singing voice is searched from audio data in a wave file format with a sampling frequency of 44100 Hz, a quantization bit number of 16 bits, and one channel. A time-series feature value is extracted using a time window from the five-dimensional low-order term of the mel frequency cepstrum coefficient as a voice feature value and arranged on the time axis.
[0059]
The average of the average search length is used as an evaluation criterion for the search results. This is an evaluation criterion that represents the time and effort required to find a result suitable for the search purpose from the search results. It is assumed that the suitability up to the 20th among the ranked search results is judged, the ranking is up to the 20th, and even if there is a conformance result below that, the search could not be performed.
[0060]
[B] Explanation of average search length
Here, the average of the average search length used as the evaluation criterion of the search result will be described. The average search length is described in Reference 5 below.
[Reference 5] Takenobu Tokunaga: “Language and Calculation 5 Information Retrieval and Language Processing”, University of Tokyo Press, 1999.
The average search length is a scale for evaluating a set ranked as a search result. When the ranked results are returned as the search results, the searcher must actually judge the suitability of the search results from the top results one by one. The average search length takes into account the process of determining the suitability of a searcher, and the user must determine how much the searcher must judge the suitability of a result in order to obtain the required number of match results. It is a scale that measures the effort of the.
[0061]
For example, if the search result is the ordered set S₁, S₂, S_ThreeSuppose we were able to divide However, the order between sets is S₁, S₂, S_ThreeIn this order, ○ and × represent the conformity result and nonconformity result, respectively.
[0062]
S₁: {○, ×, ×, ×}
S₂: {○, ○, ○, ○, ×, ×}
S_Three: {○, ○, ×, ×}
Now, the searcher wants to obtain one match result. First, the set S₁Will be inspected. Since there is no order in this set, the expected number of results that must be examined before finding a match is
1 × 1/4 + 2 × 1/4 + 3 × 1/4 + 4 × 1/4 = 2.5
It becomes.
[0063]
This indicates that, in order to find one match result from the search results, the searcher must determine the suitability of 2.5 search results on average. That is, the average search length required to find one matching result from this search result is 2.5.
[0064]
To find two matching results, the set S₁After checking all, set S₂The expected value of the number of results to be examined is
(4 + 1) × 4/6 + (4 + 2) × 4/15 + (4 + 3) × 1/15 = 5.4
It becomes. That is, the average search length required to find two matching results is 5.4.
[0065]
In the above example, the search results were given in a ranked set, but even if the search results are all ordered, an average search is considered if each set has one element. The length can be calculated.
[0066]
As can be seen from the above, the average search length is not a single measure but a value depending on the number of necessary matching results. Therefore, as the average of the average search length, a value of the average search length per necessary matching result is calculated.
[0067]
The average number x of average search lengths, where i is the required number of matching results, M is the total number of matching results, and x (i) is the average search length necessary to find i required matching results._avIs
x_av= (1 / M) Σ_{i = 1} ^M{X (i) / i}
It is represented by
[0068]
For example, let us consider a case where the matching result is retrieved in the second and sixth positions as a result of the retrieval. When the number of necessary matching results is 1, the average search length is 2, and when the number of necessary search results is 2, the average search length is 6. The average of these average search lengths is (2/1 + 6/2) / 2 = 4.
[0069]
[C] First experiment
As a method that does not support linear expansion and contraction, we will experiment by extracting time-series features using only one type of time window. On the other hand, as a method corresponding to linear expansion and contraction, an experiment is performed by the method according to the present invention using nine types of time windows. The average value of the average search length of both methods is compared.
[0070]
[D] Results of the first experiment
FIG. 14 shows the result of comparing the average search lengths of the first experiment. The x in the figure indicates that the matching result could not be retrieved. Although the average search length is the same for songs B, E, and H, the method according to the present invention is higher for all other songs. That is, comparing the case where the linear expansion / contraction with the conventional method is not used and the case where the linear expansion / contraction according to the present invention is used for the speech data to be searched through the fixed-length time window, the average search length is short, It was found that there is an effect that the search accuracy is improved from 25% to about 2 times. Therefore, it can be said that the method corresponding to the linear expansion and contraction according to the present invention is effective in the search for singing voice.
[0071]
[E] Second experiment
As a conventional method capable of dealing with nonlinear expansion and contraction, the average search length average of the conventional method and the method according to the present invention is compared by using the voice search function of “CrossMediator for Video V2.0 (R1)” of Media Drive Co., Ltd. Also, check whether the search time can be reduced by simple matching.
[0072]
The search time is manually measured 10 times from when the button for starting the search on the display unit is pressed until the search result is displayed, and the average value is used as the search time. In this experiment, a personal computer with a CPU of Pentium 4 (1.7 GHz) manufactured by Intel Corporation and a main storage capacity of 654,812 KB was used.
[0073]
[F] Results of the second experiment
FIG. 15 shows the result of comparing the average search lengths in the second experiment. The x in the figure indicates that the matching result could not be retrieved. From FIG. 15, although the results of the method according to the present invention are slightly lower in the songs C and G, the results of all the other results are equivalent or better. It can be seen that the method according to the present invention that does not use non-linear matching is able to search at least as much as the conventional method.
[0074]
Next, using the song B in FIG. 15 as a search key, the search was repeated once and the average was compared as the search time. The conventional method took 4.29 seconds, but the method according to the present invention is about 2.42 seconds faster, confirming a reduction in search time by simple matching. That is, when comparing the conventional method using nonlinear expansion and contraction with the method using linear expansion and contraction according to the present invention, the method according to the present invention is comparable to the search accuracy, but the processing speed (CPU burden) is improved by about 56%. I found out that
[0075]
Obviously, the effectiveness of the present invention is basically the same in the first and fourth embodiments.
[0076]
The processing of each embodiment described above can be realized by a computer and a software program, and the program is stored in an appropriate recording medium such as a portable medium memory, a semiconductor memory, and a hard disk that can be read by the computer. Then, it can be executed by a computer by reading out from there. The program can be downloaded from another computer via a communication line, and can be installed and executed.
[0077]
【The invention's effect】
In the conventional method of searching for a speech signal section having a high similarity from the speech signals to be searched using the speech signal of the search key, a dynamic matching that can cope with the nonlinear expansion / contraction of the speech signal is used for the distance calculation representing the similarity. However, in the present invention, when the audio signal is mainly linear expansion / contraction, the dynamic matching is used for the distance calculation by using the static matching corresponding to only the linear expansion / contraction for the distance calculation expressing the similarity. Compared to the case of using it, the calculation amount is reduced and the search time is reduced.
[0078]
In the present invention, the speech signal section obtained as a search result can be used as a search key, and all speech signal sections suitable for the search purpose in the search target speech signal can be obtained by the first search. Even if it is not, it is possible to perform a narrowed search by re-selecting a speech signal section suitable for the search purpose from among the speech signal sections of the search results as a search key, and there is a possibility of obtaining an audio signal suitable for the search purpose. Has the effect of increasing.
[Brief description of the drawings]
FIG. 1 is a configuration diagram for explaining Embodiment 1 of the present invention;
FIG. 2 is a flowchart for explaining operations in a voice accumulation phase and a voice time-series feature vector extraction phase.
FIG. 3 is a diagram illustrating processing of a time window cutout unit.
FIG. 4 is a diagram for explaining an example of a time window cutout method;
FIG. 5 is a diagram for explaining a linear expansion / contraction method for partial audio time-series feature amounts;
FIG. 6 is a flowchart for explaining operations in a voice search phase and a search key voice time-series feature vector extraction phase.
FIG. 7 is a configuration diagram for explaining a second embodiment of the present invention;
FIG. 8 is a flowchart for explaining operations in a speech accumulation phase and a speech time-series feature vector extraction phase.
FIG. 9 is a diagram illustrating a method of linearly compressing a partial speech time-series feature quantity having a reference time window length in the time axis direction.
FIG. 10 is a flowchart for explaining operations in a voice search phase and a search key voice time-series feature vector extraction phase.
FIG. 11 is a configuration diagram for explaining a third embodiment of the present invention;
FIG. 12 is a configuration diagram for explaining a third embodiment of the present invention;
FIG. 13 is a flowchart illustrating an operation in a voice search phase.
FIG. 14 is a diagram showing the results of a first experiment.
FIG. 15 is a diagram showing the results of a second experiment.
[Explanation of symbols]
P1 voice storage phase
P2 Speech time series feature vector extraction phase
P3 voice search phase
P4 Search key voice time-series feature vector extraction phase
10 Voice signal search device
11 Voice time-series feature extraction unit
12 hour window cutout
13 Partial speech time-series feature fixed length unit
14 Voice storage unit
15 Storage device
16 Feature information comparison part
17 Display format generator
18 Search key voice time series feature extraction unit
20 Search target audio signal input device
21 Display device
22 Search key voice signal input device
23 Search condition input device
30 Speech time-series feature linear compression unit
31 Search Key Voice Time Series Feature Quantity Linear Compression Unit
40 Search condition input part

Claims (8)

Extracting partial speech time series features with different lengths from the speech signal to be searched using multiple types of time windows;
A process of linearly expanding and extracting the extracted partial speech time-series feature quantities of different types to match the length of the reference time window, which is the basis for calculating the similar distance during search,
A speech time-series feature quantity vector to be searched for comparing a partial speech time-series feature quantity aligned with the length of the reference time window with a speech time-series feature quantity vector obtained from a speech signal input as a search key And the process of accumulating as
Extracting a search key voice time series feature vector of the length of the reference time window from a voice signal input or designated as a search key;
A similarity distance between the speech time series feature vector accumulated as the search target and the search key speech time series feature vector is calculated, and the speech signal of the search key and the speech signal of each speech signal section to be searched are calculated. The process of calculating similarity,
A process of ranking and outputting search results in order of similarity based on similarity calculation results ,
The user is made to input specification information using one of the output search results as a search key, and the search is performed again by calculating the similarity using the voice data obtained from the specified search result as a search key. And a process of outputting a result . A process of extracting partial speech time series features from a speech signal to be searched using a reference time window of a length that is a reference for calculating a similar distance at the time of search;
A process of storing the extracted partial speech time series feature quantity as a speech time series feature quantity vector to be searched for comparison with a speech time series feature quantity vector obtained from a speech signal inputted as a search key;
A process of extracting partial speech time-series features with different lengths from a speech signal input or designated as a search key using time windows of different lengths;
A process of linearly expanding and contracting the extracted partial speech time-series feature quantities of a plurality of lengths to match the length of the reference time window;
A partial voice time series feature quantity aligned with a length of the reference time window is set as a search key voice time series feature quantity vector, and the voice time series feature quantity vector stored as the search target and the search key voice time series feature quantity vector Calculating a similarity distance between the voice signal of the search key and the voice signal of each voice signal section to be searched;
A process of ranking and outputting search results in order of similarity based on similarity calculation results ,
The user is made to input specification information using one of the output search results as a search key, and the search is performed again by calculating the similarity using the voice data obtained from the specified search result as a search key. And a process of outputting a result . In the process of extracting partial speech time-series features with different lengths from the speech signal using time windows of different types,
Extract voice time series feature from the audio signal, cut out the partial time series feature from the reference time window of the reference length little by little, and center on the length of the reference time window The speech signal search method according to claim 1 or 2, wherein the partial speech time-series feature amount is cut out in a similar manner even in a plurality of types of time windows. The speech time-series feature vector and the search key speech time-series feature vector accumulated as the search target are linear compression of a partial speech time-series feature amount having a reference time window length to a predetermined length. The speech signal search method according to claim 1, wherein the speech signal search method is provided. Means for extracting partial speech time-series feature quantities having different lengths from a speech signal to be searched using a plurality of types of time windows;
Means for linearly expanding / contracting the extracted partial time series feature quantities of multiple lengths to match the length of the reference time window, which is the basis for calculating the similar distance during search,
A speech time-series feature quantity vector to be searched for comparing a partial speech time-series feature quantity aligned with the length of the reference time window with a speech time-series feature quantity vector obtained from a speech signal input as a search key As a means of accumulating as
Means for extracting a search key voice time series feature quantity vector of the length of the reference time window from a voice signal input or designated as a search key;
A similarity distance between the speech time series feature vector accumulated as the search target and the search key speech time series feature vector is calculated, and the speech signal of the search key and the speech signal of each speech signal section to be searched are calculated. Means for calculating similarity,
A means for ranking and outputting the search results in order of similarity based on the similarity calculation results ;
The user is made to input specification information using the voice data obtained from the output search result as a search key, and the search is performed again by calculating the degree of similarity using one of the specified search results as a search key. Means for outputting a result . Means for extracting a partial speech time-series feature amount from a speech signal to be searched using a reference time window having a length as a reference for calculating a similar distance at the time of search;
Means for storing the extracted partial voice time-series feature quantity as a voice time-series feature quantity vector to be searched for comparison with a voice time-series feature quantity vector obtained from a voice signal input as a search key;
Means for extracting partial speech time-series feature amounts having different lengths from a speech signal input or designated as a search key using a plurality of types of time windows;
Means for linearly expanding / contracting the extracted partial time series feature quantities of multiple lengths to match the length of the reference time window, which is the basis for calculating the similar distance during search,
A partial voice time series feature quantity aligned with a length of the reference time window is set as a search key voice time series feature quantity vector, and the voice time series feature quantity vector stored as the search target and the search key voice time series feature quantity vector Means for calculating the similarity distance between the voice signal of the search key and the voice signal of each voice signal section to be searched;
A means for ranking and outputting the search results in order of similarity based on the similarity calculation results ;
The user is made to input specification information using one of the output search results as a search key, and the search is performed again by calculating the similarity using the voice data obtained from the specified search result as a search key. Means for outputting a result . An audio signal search program for causing a computer to execute the audio signal search method according to any one of claims 1 to 4 . A recording medium for an audio signal search program, characterized in that a program for causing a computer to execute the audio signal search method according to any one of claims 1 to 4 is recorded on a computer-readable recording medium. .

Images