JP6208631B2 - Voice document search device, voice document search method and program - Google Patents

Description

  The present invention relates to a technique for searching an audio document such as an audio file or an audio archive.

  In recent years, it has become easier to record audio such as smartphones and IC recorders, and a large amount of audio documents have been accumulated. It is not easy to find a desired voice document from a large number of voice documents. Therefore, voice document retrieval technology is required.

  In many cases, the voice document search is basically realized by the same kind of technique as the text search. The voice document is converted into text by voice recognition technology, and the user performs a search by giving the text as a query. An audio document containing utterance content similar to a text query is the search result.

  However, one of the features of voice document search is the need for speaker-specific search. That is, it searches for “who said what?”. For example, when there are a large number of audio documents recorded from a company meeting, if it is desired to re-listen to the remarks of the supervisor, it is necessary to perform a search by designating the supervisor.

  In the conventional method, speaker specification has also been realized by the same kind of technique as text search. That is, a speaker label (speaker name) is assigned to the voice document, and the speaker label is passed as a query at the time of retrieval. Since the speaker label is also a text, as a result, the speaker can be specified by the same framework as the text search.

  However, it is difficult to assign speaker labels to all of a large number of voice documents. Non-Patent Document 1 reports a system in which speaker name information based on speaker identification is expected to be applied to metadata production, and the registration of speaker names is continuously repeated. However, speaker names are not assigned to all voices, and this system is created at a cost by broadcasting stations and the like because revenue is expected from long-term use of content. It is not actually forced by ordinary users.

  Therefore, a technique for mechanically attaching a speaker label is used. One example is the technique of Non-Patent Document 2. In Non-Patent Document 2, a speaker model combined with an utterance tendency is prepared in advance, and it is verified which speaker model the voice document matches. The speaker model that best matches the speech document (here, the speech document is assumed to be uttered by one speaker) is selected, and the corresponding speaker label is assigned to the speech document.

Akio Kobayashi, Takahiro Oku, Shinichi Honma, Shohei Sato, Satoshi Imai, “Automatic Transcription System for News Programs for Content Utilization”, IEICE Transactions, vol. J93-D (10), pp. 2085- 2095, 2010 Keimuro Yamamuro and Katsunobu Ito, “Speaker Annotation Considering Digital Broadcast Captioning Information and Tendency”, Information Processing Society of Japan 74th Annual Conference, 2012 (1), pp. 619-620, 2012

  When searching for a desired voice document from a database of a large number of voice documents, the function of specifying a speaker is extremely useful. In order to be able to specify a speaker, it is necessary to attach a speaker label (speaker name) to each voice document. However, it is practically difficult to assign speaker labels to all voice documents.

  In the conventional voice document search method using the speaker label, there is a problem that a voice document without a speaker label cannot be properly detected in the speaker-specified search.

  In the conventional speech document retrieval technology that mechanically assigns speaker labels, it is necessary to prepare speaker models for all speakers in advance. However, in an actual speech document search, when a large amount of speech documents exists, an unknown speaker inevitably exists, and this assumption does not hold.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a speaker specification type voice document search technique that does not require a speaker label.

  In order to solve the above problems, an audio document search device of the present invention is an audio document storage unit that stores a plurality of audio documents by a plurality of speakers, and a speaker feature vector of a speaker to be searched Speaker feature vector space similarity calculation unit that calculates speaker similarity from the speaker feature vector and the speaker feature vector of the speaker who uttered the speech document, and a search result that outputs a speech document with high speaker similarity And an output unit.

  The voice document search technology of the present invention realizes a speaker-specified type voice document search that does not require a speaker label by giving a restriction using a speaker feature vector as a query. It is no longer necessary to provide an appropriate speaker label for the voice document, and the operation and cost associated with the maintenance of the voice document can be reduced.

  In voice document search, high speed is required and it is necessary to consider input of text query. According to the present invention, high-speed speaker-specific speech document search is realized by calculating the similarity between a query and a speech document using a similarity measure with a small calculation cost on a vector space of speaker features and text.

FIG. 1 is a diagram illustrating a functional configuration of the voice document search apparatus according to the first embodiment. FIG. 2 is a diagram illustrating a processing flow of the voice document search method according to the first embodiment. FIG. 3 is a diagram illustrating a functional configuration of the voice document search apparatus according to the second embodiment. FIG. 4 is a diagram illustrating a processing flow of the voice document search method according to the second embodiment. FIG. 5 is a diagram illustrating a functional configuration of an audio document search apparatus according to a modification of the first embodiment. FIG. 6 is a diagram illustrating a functional configuration of an audio document search device according to a modification of the second embodiment. FIG. 7 is a diagram illustrating a method for generating a target speaker feature vector. FIG. 8 is a diagram illustrating a processing flow of a voice document search method according to a modification of the first embodiment. FIG. 9 is a diagram illustrating a processing flow of a voice document search method according to a modification of the second embodiment. FIG. 10 is a diagram illustrating a functional configuration of the voice document search apparatus according to the third embodiment. FIG. 11 is a diagram illustrating a processing flow of the voice document search method according to the third embodiment.

  The present invention has been devised for the purpose of making it possible to detect a speech by a specified search even in such a speech document on the assumption that a speech document having no speaker label exists in a database of speech documents. An apparatus and method for voice document retrieval. This voice document search apparatus and method also ensures high speed search required for voice document search.

  In the present invention, a speaker to be searched is specified by using a speaker feature vector as a query instead of a speaker label. Each voice document on the database is also converted into speaker feature vectors, and a speaker-specified type of voice document search is realized based on the similarity between the speaker feature vectors.

  The speaker feature is configured to be one vector (hereinafter referred to as a speaker feature vector) for the voice document. Then, the similarity is calculated based on the similarity between the two vectors in the vector space. The similarity between vectors may be a similarity measure widely used in the field of information processing, such as cosine similarity. In general, the calculation of the similarity between vectors often has a low calculation cost, and employing such a scale enables high-speed search.

  In the speech document search apparatus and method according to the first embodiment of the present invention, the similarity based on the text query and the similarity of the speaker are added together to determine the speech document that is the final search result.

  The method of calculating the text similarity is arbitrary, and is not limited to a vector expression. For example, in a search engine on the web (WEB), the query is limited to a small set of words. The search is performed by taking AND and OR of documents in which each word appears, and the number of appearance words of the query word is treated as the similarity. As for the text similarity in the first embodiment, such text similarity that is not calculated as a vector space is allowed.

  On the other hand, in a document search using a text sentence as a query, a vector (hereinafter referred to as a word vector) having elements such as the appearance frequency of each word in a total document is widely used as a feature vector. The query and each document on the database are expressed by one word vector, and a search is performed after defining a measure such as cosine similarity.

  An audio document search apparatus and method according to a second embodiment of the present invention creates a new vector (hereinafter referred to as a speaker feature word vector) by connecting a speaker feature vector and a word vector, and provides a speaker feature word vector. Calculate the similarity in space. Although the second embodiment imposes restrictions on processing on the text side as compared with the first embodiment, it is not necessary to separately calculate the similarity between the speaker feature and the text. A higher speed can be expected than when the similarity between the speaker feature vectors and the similarity between the word vectors are calculated separately.

  In this invention, the query is a speaker feature vector, but it is obvious that the speaker feature vector can be mechanically calculated from speech data in a predetermined step. Therefore, as an embodiment, the case of using voice data as a query is also within the scope of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[First embodiment]
As shown in FIG. 1, the speech document search apparatus according to the first embodiment includes a speech query input unit 10, a text query input unit 11, a speaker feature vector extraction unit 12, a word vector extraction unit 13, and a speaker feature vector space similarity. For example, a degree calculation unit 14, a word vector space similarity calculation unit 15, a similarity addition unit 16, a search result output unit 17, an audio document storage unit 18, and a similarity storage unit 19 are included.

  The voice document retrieval device is a specially configured computer in which a special program is read into a known or dedicated computer having a central processing unit (CPU), a main storage device (RAM), and the like. Device. For example, the voice document search device executes each process under the control of the central processing unit. The data input to the voice document search device and the data obtained in each process are stored in the main storage device, for example, and the data stored in the main storage device is read out as necessary and used for other processing. Is done. Further, at least a part of each processing unit of the voice document search apparatus may be configured by hardware such as an integrated circuit.

  Each storage unit included in the voice document search device includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory, or It can be configured with middleware such as a relational database or key-value store. Each storage unit included in the voice document search device only needs to be logically divided, and may be stored in one physical storage device.

  The voice document storage unit 18 stores voice documents uttered by a plurality of speakers.

  With reference to FIG. 2, the voice document search method of the first embodiment will be described.

  In step S10, speech data or a speaker feature vector of a speaker to be searched is input as a query (hereinafter referred to as a speech query) to the speech query input unit 10. When voice data is input, the voice query is sent to the speaker feature vector extraction unit 12. When the speaker feature vector is input, the voice query is sent to the speaker feature vector space similarity calculation unit 14. In this case, the speech document search apparatus may not include the speaker feature vector extraction unit 12. A specific configuration of the speaker feature vector will be described later.

  In step S11, text data or a word vector to be searched is input to the text query input unit 11 as a query (hereinafter referred to as a text query). When text data is input, the text query is sent to the word vector extraction unit 13. When a word vector is input, the text query is sent to the word vector space similarity calculation unit 15. In this case, the voice document search device may not include the word vector extraction unit 13. Details of the word vector will be described later.

  In step S12, the speaker feature vector extraction unit 12 extracts a speaker feature vector from the input speech query. The extracted speaker feature vector is sent to the speaker feature vector space similarity calculation unit 14.

  In step S13, the word vector extraction unit 13 extracts a word vector from the input text query. The extracted word vector is sent to the word vector space similarity calculation unit 15.

  In step S14, the speaker feature vector space similarity calculation unit 14 calculates a predetermined similarity measure from the speaker feature vector of each voice document stored in the voice document storage unit 18 and the input speaker feature vector. The speaker similarity calculated according to the above is calculated. The calculated speaker similarity is sent to the similarity summation unit 16.

  Any similarity measure can be used for the similarity between the speaker feature vectors. One representative form is cosine similarity. Another representative form is an inner product value.

  In step S15, the word vector space similarity calculation unit 15 calculates the text similarity based on a predetermined similarity measure from the word vector of each audio document stored in the audio document storage unit 18 and the input word vector. Calculate the degree. The calculated text similarity is sent to the similarity summation unit 16.

  As the similarity between the word vectors, an arbitrary similarity measure can be used similarly to the speaker feature vector.

  In step S16, the similarity summation unit 16 calculates the similarity between the voice document and the query by adding the speaker similarity and the text similarity according to a predetermined method. The calculated similarity is stored in the similarity storage unit 19.

  The method of summing the speaker similarity and the text similarity includes addition, multiplication, logarithmic addition, and a weighted operation. It is recommended to artificially determine a weight that is considered optimal in terms of search accuracy and the like through preliminary experiments.

  In step S <b> 17, the search result output unit 17 outputs an audio document having a high similarity stored in the similarity storage unit 19 as a search result. The output format may be a recognition result text obtained by voice recognition of a voice document, or the voice data itself included in the voice document may be reproduced as a voice waveform.

[Second Embodiment]
As shown in FIG. 3, the speech document search apparatus according to the second embodiment includes a speech query input unit 10, a text query input unit 11, a speaker feature vector extraction unit 12, a word vector extraction unit, as in the first embodiment. 13. A search result output unit 17, a voice document storage unit 18, and a similarity storage unit 19, for example, include a voice document speaker feature word vector creation unit 20, a query speaker feature word vector creation unit 21, and a speaker feature word vector space. A similarity calculation unit 22 is further included.

  With reference to FIG. 4, the audio | voice document search method of 2nd embodiment is demonstrated. Below, it demonstrates centering on difference with the above-mentioned 1st embodiment.

  In step S <b> 20, the voice document speaker feature word vector creation unit 20 recognizes the speaker feature vector extracted from the voice document stored in the voice document storage unit 18 and the voice document stored in the voice document storage unit 18. The word vector extracted from the recognized recognition result text is connected to create a speech document speaker feature word vector. The created voice document speaker feature word vector is sent to the speaker feature word vector space similarity calculation unit 22.

  In step S21, the query speaker feature word vector creation unit 21 connects the speaker feature vector input from the speaker feature vector extraction unit 12 and the word vector input from the word vector extraction unit 13 to obtain a query. Create speaker feature word vectors. The created query speaker feature word vector is sent to the speaker feature word vector space similarity calculation unit 22.

  In step S22, the speaker feature word vector space similarity calculation unit 22 outputs the voice document speaker feature word vector output by the voice document speaker feature word vector generation unit 20 and the query speaker feature word vector generation unit 21. The similarity between the voice document and the query is calculated according to a predetermined similarity measure from the query speaker feature word vector to be stored, and the result is stored in the similarity storage unit 19.

[Specific examples of feature vectors]
Hereinafter, a specific configuration of each feature vector used in the first embodiment and the second embodiment will be described in detail.

  One form of the speaker feature vector is, for example, a feature value called i-vector. Details of i-vectors are described in “H. Aronowitz and O. Barkan,“ Efficient approximated i-vector extraction ”, Proceedings of ICASSP, pp. 4789-4792, 2012. (Reference 1). Another form of the speaker feature vector is a vector of speaker-dependent components extracted using Joint Factor Analysis (JFA). The vector obtained by JFA is also described in the above reference 1.

  Both the i-vector and the vector obtained by JFA are connected by connecting the average vector of each Gaussian distribution (GMM: Gaussian Mixture Model) that has been applied to speech data (super vector). ) Is subjected to matrix decomposition by a predetermined method. Considering this, another form of the speaker feature vector is a vector obtained by matrix decomposition so that speaker components can be extracted by a predetermined method from the GMM super vector. Another form of speaker feature vector is the GMM supervector. The GMM supervector is one of the options in that it retains the speaker component. However, it contains a large amount of components other than the speaker, and depending on the setting, it is extremely high in dimension compared to other vectors, and there is a concern about the influence on the search speed. In addition, even a vector obtained by a method that does not use a GMM super vector is a category of a speaker feature vector as long as it is a feature vector that exhibits the effect of identifying a speaker.

  The text similarity may be a score used in an existing text search.

  Another form of speaker feature vector and word vector is a normalized vector. The normalized vector of vector X,

(Where || X || is the norm of X), a vector that normalizes speaker feature vector V

Is a normalized vector of the word vector W

It is.

  Another form of speaker feature vector and word vector is a weighting vector. If the weighted vector of vector X is aX (where a is a constant), vector aV weighted by speaker feature vector V or vector weighted by normalized vector

A vector aW weighted by word vector W or vector weighted by normalized vector

It is.

  The speaker feature word vector is a vector obtained by simply connecting the speaker feature vector and the word vector. For example, for speaker feature vector V and word vector W:

It is good to do.

  The advantage of normalization is that the calculation of cosine similarity can be replaced with an inner product operation with a smaller amount of calculation. Weighting can be used in the second embodiment to introduce a degree of similarity equivalent to the first embodiment. For example, in the summation of similarities in the first embodiment, when a weighted sum of cosine similarity is used,

So, the inner product of 2 times

Only the amount of calculation can be reduced.

Is a vector calculated in advance, so only two inner products are required. Note that the subscript d represents an audio document, and the subscript q represents a query. further,

It is. The right side is exactly the second embodiment. This equality relationship shows that the same similarity can be used in the first embodiment and the second embodiment.

[Modification]
In the first embodiment or the second embodiment, the user needs to prepare speaker feature vectors or voice data of the search target speaker. This condition is realistic. In the present invention, since the speaker label is not denied, the voice document of the target speaker (non-target) can be obtained as long as the data has the speaker label. This speech document is different from the content of the utterance to be originally searched, that is, it is not the target, but represents the characteristics of the speaker. By inputting as a text query, a target speech document with a matching speaker and text may be searched.

  The voice data once used as a query may be managed like an e-mail address book. That is, if voice data with speaker labels that can be used as queries is given names and IDs such as speaker labels that can be easily identified and managed in an address book, By calling the target speaker's data from the book, it is possible to search for a speaker-specified voice document. At this time, it is more efficient to register the speaker feature vector in the address book in order to avoid the feature vector calculation step.

  In the modification of the first embodiment or the second embodiment, the speaker label and the speaker feature vector are associated with each other in the address book format, and the name of the speaker label or the like given to the target speaker by the user. An audio document search apparatus and method for searching for an audio document by inputting an ID or ID.

  As shown in FIG. 5, the speech document search apparatus that is a modification of the first embodiment is similar to the first embodiment in that the text query input unit 11, the word vector extraction unit 13, and the speaker feature vector space similarity calculation. Unit 14, word vector space similarity calculation unit 15, similarity summation unit 16, search result output unit 17, voice document storage unit 18, and similarity storage unit 19, for example, speaker label input unit 30, target speaker characteristics A vector storage unit 31 and a speaker feature vector extraction unit 32 are further included.

  As shown in FIG. 6, the voice document search apparatus as a modification of the second embodiment is similar to the second embodiment in that a text query input unit 11, a word vector extraction unit 13, a search result output unit 17, and a voice document are used. A storage unit 18, a similarity storage unit 19, a voice document speaker feature word vector creation unit 20, a query speaker feature word vector creation unit 21, and a speaker feature word vector space similarity calculation unit 22 are included, for example, and speaker label input It further includes a unit 30, a target speaker feature vector storage unit 31, and a speaker feature vector extraction unit 32.

  The target speaker feature vector storage unit 31 stores speaker feature vectors to which speaker labels are assigned. The speaker label is a name or ID for identifying the speaker, and is assigned so that the speaker feature vector stored in the target speaker feature vector storage unit 31 can be uniquely identified.

  A method for generating speaker feature vectors stored in the target speaker feature vector storage unit 31 will be described with reference to FIG. Arbitrary audio data is input to the audio data input unit 33. The speaker feature vector extraction unit 12 extracts a speaker feature vector from the input voice data, similarly to the speaker feature vector extraction unit of the first embodiment. The speaker feature vector may have any form as described above. The speaker label name assigning unit 34 associates the speaker label name with the speaker feature vector by assigning the speaker label input by the user to the speaker feature vector output from the speaker feature vector extracting unit 12. And stored in the target speaker feature vector storage unit 31.

  FIG. 8 shows a processing flow of a voice document search method that is a modification of the first embodiment, and FIG. 9 shows a processing flow of a voice document search method that is a modification of the second embodiment. Below, it demonstrates centering around difference with the above-mentioned 1st embodiment and 2nd embodiment.

  In step S30, one of the speaker labels assigned to the target speaker is input to the speaker label input unit 30.

  In step S <b> 32, the speaker feature vector extraction unit 32 extracts the speaker feature vector stored in the target speaker feature vector storage unit 31 using the input speaker label. In the subsequent processing, the extracted speaker feature vector may be handled as the speaker feature vector extracted from the voice query in the first embodiment or the second embodiment.

  On the other hand, when searching for a speaker having no speech document with a speaker label in the target speaker feature vector storage unit 31, it is necessary to obtain speech data by some method. If it is a celebrity's voice, it will be possible to obtain voice data by web search. If you have a relatively close relationship, such as an acquaintance, you can record audio and register it in your address book. Alternatively, the query can be obtained by sharing the address book of an acquaintance who registers the target speaker.

  The speaker feature vector used for the query is obtained as an output result when speech data to which a speaker label is attached or speech data obtained from outside the database as described above is input to the speaker feature vector extraction unit. Any desired speaker label name may be assigned by the user and recorded in the target speaker feature vector storage unit in which the speaker label name and the speaker feature vector are associated with each other.

[Comparison with similar prior art]
Similar voice selection methods and devices are described in Shigeo Morishima et al., “New Video Technology“ Dive Into the Movie ””, IEICE Journal, Vol. 94, No. 3, pp. 250-268, March 2011 ( Reference 2) ”. In Reference Document 2, speech is used as input, and the speaker similarity with each speech file (speech document) registered in advance is calculated. Reference 2 is an idea similar to the present invention in this respect, but cannot be used for speech document retrieval because it decisively lacks consideration regarding calculation cost. In fact, it is premised on processing with a high calculation cost, such as calculating a dynamic scale between feature quantities and GMM likelihood. Also, no consideration is given to how to specify text. The feature amount in Reference 2 is extracted for each frame (segment time), and one matrix (order of feature amount × number of frames) is obtained for audio data. It is not a single vector as in the present invention, and this point is also different.

  I-vector, which is one example of speaker feature vectors, is a technology developed in the field of speaker verification. In this field, a method for collating speakers based on the cosine similarity of i-vector was proposed. In speaker verification, the similarity between the speech of all speakers registered in advance and the input speech is calculated. This is a part in common with the present invention. However, i-vector cosine similarity does not provide sufficient speaker verification and identification accuracy, and then transitions to use statistical techniques (such as Probabilistic Linear Discriminant Analysis). (For details, see Pavel Matejka, Ondrej Glembek, Fabio Castaldo, Md. Jahangir Alam, Oldrich Plchot, Patrick Kenny, Lukas Burget, Jan Cernocky, “Full-covariance UBM and heavy-tailed PLDA in i-vector speaker verification. ", ICASSP 2011, pp. 4828-4831 (reference 3)"). That is, it is difficult to identify and identify a speaker by a simple method such as calculating the similarity of two points on a vector space, such as the cosine similarity of i-vector. Furthermore, the problem becomes more serious when a large database such as the subject of the present invention is targeted.

  However, the inventors have noticed that the i-vector cosine similarity works sufficiently accurately by paying attention to the following features of the voice document search. The first feature is that a text query exists in a speech document search. The text query greatly limits the candidates. This effectively limits the number of speakers.

  The second feature is that in the audio document search, the accuracy in which the target audio document is included in the plurality of higher candidates is more important than the accuracy of the first candidate. In c-similarity of i-vector, it is often difficult to hit first. However, it is relatively easy to list the target speech document as a top candidate.

  The third feature is that the voice document search uses a longer voice compared to the voice used in the field of speaker verification and the like. Since long speech is targeted, speaker characteristics can be extracted accurately, and improvement in accuracy can be expected.

  Furthermore, since high-speed search is also required for voice document search, a low-computation method such as i-vector cosine similarity is suitable.

  The inventors focused on these various features unique to voice document search, giving speaker feature vectors to queries, and similarity between two points in vector space, such as cosine similarity of i-vectors I found that calculating speaker similarity as a degree is suitable for voice document search, and even those in the relevant field (speaker verification and voice document search field) can easily recall Absent.

[Third embodiment]
As a configuration that makes use of the first feature of the above-described voice document search, the voice document search apparatus and method according to the third embodiment will be described.

  As shown in FIG. 10, the speech document search apparatus according to the third embodiment is similar to the modification of the first embodiment in that a text query input unit 11, a word vector extraction unit 13, a word vector space similarity calculation unit 15, The search result output unit 17, the audio document storage unit 18, the similarity storage unit 19, the speaker label input unit 30, the target speaker feature vector storage unit 31, and the speaker feature vector extraction unit 32, for example, include a high similarity candidate storage And a speaker feature vector space similarity calculation unit 41.

  With reference to FIG. 11, the audio | voice document search method of 3rd embodiment is demonstrated. Below, it demonstrates centering on difference with each above-mentioned embodiment.

  In step S15, the word vector space similarity calculation unit 15 calculates the text similarity based on a predetermined similarity measure from the word vector of each audio document stored in the audio document storage unit 18 and the input word vector. The degree information is calculated, and the candidate information of the voice document having a high text similarity is stored in the high similarity candidate storage unit 40.

  In step S <b> 41, the speaker feature vector space similarity calculation unit 41 limits the similarity to the speaker feature vector by limiting the candidate information stored in the high similarity candidate storage unit 40 to a higher-ranked candidate. , And a candidate with high similarity to both the word vector and the speaker feature vector is output as a search result.

  In the configuration of the third embodiment, the similarity of speaker feature vectors is calculated for a limited number of candidates, so that search results can be obtained quickly, and similarity based on word vectors and speakers can be obtained. Since the number of candidates can be controlled independently for each degree of similarity based on the feature vector, it is possible to provide a search accuracy that cannot be obtained in the first embodiment or the second embodiment.

[Experimental result]
Voice database for smartphones and voice question answering system are made into a database. There are various sound collection environments and noise is also included. The number of voice documents in the database is approximately 110,000. Since one file corresponds to one utterance, it corresponds to 110,000 utterances. Each voice document is automatically represented for each file by the speech recognition technology, the start and end of the utterance are determined, the utterance content is transcribed, and then expressed by a word vector whose element is the frequency of the content word in the recognition result. . Similarly, i-vectors were also extracted in advance as speaker feature vectors for each file and registered in the database. 5000 queries were given and the search accuracy was compared. Cosine similarity was used as a similarity measure for both speakers and text.

  In the experiment, speaker labels are given to all voice documents in the database. At the time of retrieval, the accuracy is compared between when a speaker label is given as a query and when a speaker feature vector (i-vector) is given instead of a speaker label. However, the text used as a query is a correct transcript of the target speech document (a transcript given manually).

  As the speaker feature vector of the query, voice documents of the same speaker are randomly extracted from those other than the target voice document on the database, and the speaker feature vector is used. That is, the experiment was performed in a situation where the voice document having the highest speaker similarity is not the target voice document.

  Table 1 shows the retrieval accuracy obtained as a result of the above experiment.

  The target of the query is 100 people. When the speaker label is given to the query, the search accuracy is extremely high because it is an ideal state, and the 3-best (the ratio that the target speech document is included in the top three candidates) exceeds 90%. In the present invention, 70% is correct for 2-best and 80% is correct for 5-best. It was 2-best, almost the same as the 1-best accuracy with speaker label. From this experimental result, it can be seen that it operates with accuracy that can withstand practical use.

[effect]
According to the present invention, in speaker-specified type audio document search, a speaker with a high similarity of speaker feature vectors can be obtained without assigning speaker labels to all audio data of a large amount of audio documents. An audio document can be obtained as a search result. That is, it is not necessary to prepare a voice document that assigns an appropriate speaker label to each, and it is possible to reduce operation and cost related to the maintenance of the voice document.

  In the similarity calculation of the speaker feature vector, a high-speed speaker-specified type speech document search can be realized by using a similarity scale with a low calculation cost.

  In the similarity calculation at the time of search, a combined value combining the similarity of the speaker feature vector and the similarity of the word vector (text query) can be applied, and the optimal similarity calculation is selected from the viewpoint of search accuracy etc. Can be used. Alternatively, after limiting the candidates for the speech document based on the word vector similarity calculation result, the search accuracy is improved in the form of obtaining the top candidate of the similarity based on the speaker vector similarity calculation, It is also possible to increase the processing speed.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 10 Speech query input part 11 Text query input part 12 Speaker feature vector extraction part 13 Word vector extraction part 14 Speaker feature vector space similarity calculation part 15 Word vector space similarity calculation part 16 Similarity summation part 17 Search result output part 18 Speech document storage unit 19 Similarity storage unit 20 Speech document speaker feature word vector creation unit 21 Query speaker feature word vector creation unit 22 Speaker feature word vector space similarity calculation unit 30 Speaker label input unit 31 Target speaker Feature vector storage unit 32 Voice data input unit 33 Speaker label name assigning unit 40 High similarity candidate storage unit 41 Speaker feature vector space similarity calculation unit

Claims (3)

A voice document storage unit for storing a plurality of voice documents by a plurality of speakers;
A word vector space similarity calculating unit that calculates a text similarity from a word vector of a text to be searched and a word vector of a recognition result text obtained by voice recognition of the voice document;
A high similarity candidate storage unit for storing candidate information for specifying the voice document having the high text similarity;
Among the candidate information stored in the high similarity candidate storage unit, the high candidate with the high text similarity is specified by the target speaker feature vector that is the speaker feature vector of the speaker to be searched and the candidate information. a speaker feature vector space similarity calculating section for calculating a speaker similarity and a speaker feature vectors of the speaker who uttered the sound document that,
A search result output unit for outputting the voice document having both the text similarity and the speaker similarity;
Only including,
The speaker feature vector is a feature vector obtained by performing matrix decomposition on a super vector of a mixed Gaussian distribution obtained by performing adaptive processing on speech data uttered by a speaker.
Voice document retrieval device. A plurality of voice documents by a plurality of speakers are stored in the voice document storage unit,
A word vector space similarity calculating unit that calculates a text similarity from a word vector of a text to be searched and a word vector of a recognition result text obtained by voice recognition of the voice document;
A high similarity candidate storing step for storing candidate information for specifying the voice document having a high text similarity in the high similarity candidate storing unit;
The purpose of the speaker feature vector space similarity calculation unit is a speaker feature vector of a speaker to be searched with respect to a higher candidate having a high text similarity among the candidate information stored in the high similarity candidate storage unit A speaker feature vector space similarity calculating step for calculating a speaker similarity from a speaker feature vector and a speaker feature vector of a speaker who has spoken the voice document specified by the candidate information ;
A search result output unit for outputting the voice document having both the text similarity and the speaker similarity high;
Only including,
The speaker feature vector is a feature vector obtained by performing matrix decomposition on a super vector of a mixed Gaussian distribution obtained by performing adaptive processing on speech data uttered by a speaker.
Voice document search method. A program for causing a computer to function as the voice document search device according to claim 1 .

Images