JP4080965B2 - Information presenting apparatus and information presenting method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information presentation apparatus and an information presentation method having a function of presenting information by voice in response to a search request from a user.
[0002]
[Prior art]
In recent years, the means for presenting information by voice has been increasing along with the multimediaization of distributed data and the multimodalization of search devices. For example, even in the Internet search, it is possible to read out a searched Web page by voice by using a voice synthesis technique (see, for example, Patent Document 1). In addition, it is possible to search not only document data but also audio / video by using voice recognition technology (see, for example, Patent Document 2).
[0003]
However, in the conventional voice presentation method, it is difficult to understand on what basis the presented data is selected. In addition, unlike the presentation by text data, the presentation by voice is low in listability, so if you do not listen to all of the searched data from the beginning to the end, you will often not notice the basis of the search, and you will also notice a search mistake It often takes time.
[0004]
For example, in Patent Literature 1, when Web information is read aloud, a link is read separately from other parts, but there is a problem that it is not possible to know where a hit word part that matches a query is.
[0005]
Further, for example, in Patent Document 2, there is a problem that even if the search result for the recorded voice is heard, it is not known which part hits the search request. In addition, when the recorded speech is converted into a character string using the speech recognition technology and the corresponding part is searched using the text search technology based on the search request, if the speech recognition result is wrong, the word is different from the search request. However, if the user does not know which part is hit, the user must listen to the end, and the search term may not appear even if it is heard.
[0006]
Conventionally, when presenting search results as character data, there are also techniques for inverting the character of the hit part or changing the color of the character, but this technique is applicable to the presentation of information by voice. I could not.
[0007]
[Patent Document 1]
JP 2002-358194 A
[0008]
[Patent Document 2]
JP 2002-366552 A
[0009]
[Problems to be solved by the invention]
As described above, the conventional information presentation technology using voice has problems such as difficult to understand the basis of selection of the presented information, difficulty in recognizing a search error, and time required for the user to reach the target data. It was.
[0010]
The present invention has been made in view of the above circumstances, and an object thereof is to provide an information presentation apparatus and an information presentation method that enable more effective information presentation by voice.
[0011]
[Means for Solving the Problems]
An information presentation apparatus according to the present invention acquires content by searching an input unit for inputting a search sentence and a database storing content including character string data or audio data based on the search sentence. An acquisition unit; a detection unit that detects a hit word that matches the search sentence from character string data or audio data included in the content acquired by the acquisition unit; and the hit word detected by the detection unit Presentation means for presenting information related to voice by voice.
[0012]
Further, the present invention relating to an apparatus or a method has a function for causing a computer to execute a procedure corresponding to the invention (or for causing a computer to function as a means corresponding to the invention, or for a computer to have a function corresponding to the invention. It is also established as a program (for realizing) and also as a computer-readable recording medium on which the program is recorded.
[0013]
According to the present invention, it is possible to present information more effectively by voice by presenting information related to the detected hit word by voice. For example, when presenting voice data, the hit word is highlighted and presented, so that the basis for selecting the data can be easily understood.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the invention will be described with reference to the drawings.
[0015]
(First embodiment)
In the present embodiment, an example of an information presentation device having a voice browser function of outputting a character string included in retrieved data as an example will be described.
[0016]
FIG. 1 shows a configuration example of an information presentation apparatus according to the first embodiment of the present invention.
[0017]
Here, the data to be searched is content including a character string, and audio output by hit word detection or hit word emphasis is on the character string portion included in the searched data.
[0018]
The hit word is a character string that matches the search sentence among the character strings included in the retrieved data (or the character string that is the basis for retrieving the data). In particular, when a search is performed based on a search word (character string) in the search sentence, the hit word matches a search word (character string) in the search sentence among character strings included in the searched data. It is a string. In addition, when there are a plurality of search words (character strings) in the search sentence, the hit word is a character that matches one of the search words (character strings) in the search sentence among the character strings included in the searched data. Is a column. In this case, matching means, for example, matching, but other configurations are possible. For example, among the character strings included in the searched data, a character string belonging to a subordinate concept of a search word (character string) in a search sentence, a character string belonging to a superordinate concept of a search word (character string) in the search sentence, It is also possible to configure as a character string that is similar to the search word (character string), a character string that matches the search word (character string) in the search sentence, or any combination thereof.
[0019]
If the search is a QA search, the hit word may be an answer to the search sentence. A method for obtaining an answer to a search sentence by a QA search technique is a known technique (for example, Question Answering Challenge (QAC-1): An Evaluation of Question Answering Task at NTCIRWorkshop 3, Jun'ichi FUKUMOTO, Tsuneaki KATO, and Fumito MASUI (URL: http: : //research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/) can be used.
[0020]
In the present embodiment, a hit word is described as an example of a character string that matches a search word (character string) in a search sentence among character strings included in searched data.
[0021]
In FIG. 1, the information presentation apparatus 1 includes an input unit 101 for inputting a search sentence, a search unit 103 for searching data that matches the search sentence from a data server 8 connected via the network 7, and the searched data. A hit word detection unit 104 that detects a portion corresponding to a hit word, a speech synthesis unit 107 that converts searched data (a character string portion) into speech, and emphasizes a portion corresponding to a hit word in the detected data. A data processing unit 105 that performs data processing, an output unit 106 that outputs data in which a portion corresponding to a hit word is emphasized, and a communication unit 108 that performs data communication via the network 7.
[0022]
The network 7 is, for example, the Internet, and the data is, for example, a Web page, but is not limited to this (the network may be a LAN or the like, and the data is a document of various editing software) Data etc.).
[0023]
In the above description, the information presentation apparatus 1 accesses the data server 8 having the database storing the data via the network 7, but instead of or in addition to the data server 8, the information presentation apparatus 1 May be built in, and the search unit 103 may search the database for data that matches the search text. In this case, the communication unit 108 is not always necessary.
[0024]
The information presenting apparatus 1 can be configured as a dedicated machine or can be configured using a general-purpose computer. Further, for example, it can be realized by a mobile phone terminal having various functions.
[0025]
The information presentation device 1 may also have a function of displaying the retrieved data and various messages on the display screen. For example, the retrieved data may be selectable to be displayed on the screen without outputting the sound, outputting the sound without displaying the screen, or outputting the sound while displaying the screen. The function to be applied can be applied to the sound output when the sound is output without displaying the screen or the sound is output while displaying the screen.
[0026]
FIG. 2 shows an example of the overall processing procedure of the information presentation apparatus 1.
[0027]
First, in step S <b> 1, the user inputs a search sentence by text or voice via the input unit 101. The input method is not limited as long as a final text search sentence can be obtained. For example, both text input (keyboard input) and voice input are possible. It may be.
[0028]
When performing voice input, the input unit 101 only needs to have a voice recognition unit (not shown) that recognizes the user's voice and converts it into a character string (search sentence). A known voice recognition technique may be used.
[0029]
Regardless of which method is used as the input method, the search sentence is given as a character string to the search unit 103. Therefore, for the sake of simplicity, the search sentence is input to the input unit 101 as text data. The explanation will be focused on the case where it is input.
[0030]
Next, in step S <b> 2, the search unit 103 searches through the communication unit 108 for data (for example, a web page) that matches the search sentence. For data search using a character string (search sentence), a known document search technique or Internet search technique may be used. Specifically, for example, while searching for data containing the same word (or character string) in the search sentence converted to a character string, the same as the word (or character string) in the search sentence The higher the data contained, the higher the score is given, and the data is rearranged in the order of score. Alternatively, it is possible to search for data including an answer to a search sentence for QA search. Data including the most probable answer is given a higher score, and processing such as sorting the retrieved data in the order of score is performed.
[0031]
Next, in step S <b> 3, the hit word detection unit 104 extracts hit words for the retrieved data having the first rank score. At this time, it is detected in which part of the data the hit word appears, and hit word information indicating the appearance position is recorded.
[0032]
The hit word information for the search data may be stored as a separate file from the search data, or the hit word information may be described in the search data.
[0033]
FIG. 3 shows an example of hit word information in the former case (when the file is held as a file different from the Web page). In this example, information indicating how many times the hit word appears in the data from the beginning, and information indicating how many characters from the beginning of the hit word in the data start and end in the data It is recorded. If there are multiple types of hit words and the number of appearances should be distinguished for each hit word, hit word information may be provided for each hit word as shown in FIG. Regardless of the contents, one hit word information may be provided as shown in FIG.
[0034]
Further, for example, when there are a plurality of types of hit words, a form of recording the total number of appearances when counting regardless of the hit words in addition to the number of appearances when counting for each hit word as shown in FIG. Is also possible.
[0035]
Various other forms are possible. For example, for the end position, information indicating the number of characters from the start position may be described. In addition, for example, when the number of appearances is the second or later, information indicating the number of characters from the start position or the end position in the number of appearances one time before the start position may be described. Further, since one of the start position and the end position can be specified according to the number of characters of the hit word, only one of the start position and the end position may be recorded.
[0036]
FIG. 5 shows an example of hit word information in the latter case (when described in a Web page). In this example, the hit word is the start tag <hw id = 1 t = 1> and end tag </ hw> is used to indicate a hit word (note that hw in the tag name is an example and is not limited to this). The id in the start tag is an ID number assigned to distinguish hit words (for example, assigned by the hit word detection unit 104), but it is of course possible to add the ID number. In addition, t in the start tag is the number of appearances of the same hit word (here, hit word having id = 1), and t = 1 is the first appearance in the data. Yes. In the case where the hit word is not distinguished with respect to the number of appearances, t represents the number of appearances regarding all hit words. Of course, a configuration without t is also possible.
[0037]
Further, for example, when there are a plurality of types of hit words, as shown in FIG. 6, in addition to the number of appearances t when counting for each hit word, the total number of appearances s when counting regardless of the hit word is recorded. The form to do is also possible.
[0038]
Next, in step S4, the hit word is emphasized and the data is output as voice. Hereinafter, this process will be described in detail.
[0039]
First, the speech synthesizer 107 converts a character string of searched data (for example, a web page) into speech. A well-known speech synthesis technique for converting a character string into speech may be used.
[0040]
At this time, the data processing unit 105 refers to the hit word information and performs voice conversion while emphasizing a portion corresponding to the hit word in the retrieved data.
[0041]
The search data in which the hit word is emphasized is presented to the user by voice from the output unit 106.
[0042]
Here, various forms are possible as the emphasis method. Specific examples are listed below.
-For the portion corresponding to the hit word, the volume is increased compared to the other portions.
・ When playing the part corresponding to the hit word, sound a notification sound such as “Pawn” before playing the hit word.
・ For the part corresponding to the hit word, repeat the voice a predetermined number of times.
-Change the tone (for example, gender) at the time of speaking the part corresponding to the hit word from the tone at the time of speaking the other part.
・ Change the playback speed of the part corresponding to the hit word to the playback speed of the other parts.
・ Place a voice such as “hit word” or “it is a hit word” immediately before the utterance of the part corresponding to the hit word (Note that this voice is uttered in a different tone from that used when uttering search data. You may do it).
-When using a display screen, make a difference so that it can be visually recognized, for example, when the portion corresponding to the hit word is uttered, the display screen glows red.
and so on.
[0043]
A plurality of types of emphasis methods may be used together. For example, there are various methods such as a method of increasing the volume and sounding a notification sound and a method of changing both the tone and the reproduction speed.
[0044]
Further, when there are a plurality of types of hit words, the emphasis content may be changed for each hit word.
[0045]
For example, there is a method in which the emphasis forms are the same and the parameters at that time are changed according to the hit word. For example, when changing the tone when uttering a portion corresponding to a hit word from the tone when uttering other portions, the tone may be changed for each hit word. In addition, for example, the number of notification sounds that are generated when reproducing a portion corresponding to a hit word may be changed for each hit word.
[0046]
Further, for example, the emphasis form may be changed for each hit word. For example, for the portion corresponding to the first hit word, the volume is increased, for the portion corresponding to the second hit word, the notification sound is sounded, and for the portion corresponding to the third hit word, the tone is changed. It may be changed.
[0047]
Also, for example, for the portion corresponding to the first hit word, the volume is increased, for the portion corresponding to the second hit word, the notification sound is sounded once, and for the portion corresponding to the third hit word. A method such as sounding a notification sound twice is also possible.
[0048]
On the other hand, with or instead of highlighting hit words, the detected hit words (e.g., all detected hit words, or the first detected hit word, or hit word) before the data is voiced out. The first detected signal may be output as a sound. Thus, the user can listen to the data after confirming the detected hit word. In particular, when a search sentence is input by voice input, the voice recognition technology does not necessarily completely convert the voice into a character string, and may contain a mistake. By listening to and confirming the hit word detected before listening to the output, it is possible to confirm whether or not the input speech recognition has been performed correctly (for example, a hit word different from the intended one due to misrecognition is uttered) In such a case, since the data search result cannot be expected to be desired, the processing can be stopped without listening to the voice output of the search data, and the search sentence can be input again from the voice input).
[0049]
Furthermore, when emphasizing hit words, the number of appearances of hit words in the data may be presented to the user. For example, when a character string as a hit word is uttered, a voice such as “nth hit word (is)” or “nth (is)” may be inserted. When there are a plurality of types of hit words, the number of appearances for each hit word is presented in the same manner as the number of appearances irrespective of the content of the hit word. In addition, when there are multiple types of hit words and the number of occurrences for each hit word is presented, for example, a specific hit word XXX such as “the third XXX” is shown for each hit word. You may make it insert | pinch the audio | voice containing. In addition, as another method, it is also possible to generate a notification sound for hit word playback as many times as the number of appearances (for example, “Pawn pawn pawn” instead of “third time”) Ring).
[0050]
On the other hand, as described above, instead of or along with the presentation of the number of appearances when emphasizing the hit word, the total number of occurrences of all hit words and / or each hit before or after the data is voiced. Information such as the total number of appearances for each word may be output by voice.
[0051]
Here, if the user who has heard the presentation of the number of appearances wants to hear the data again from the “nth hit word”, the user designates the desired number of appearances n and inputs a reproduction command. In response to this, the information presenting apparatus may be provided with a function of reproducing the data from a portion corresponding to the “nth hit word” (or a certain number of characters before or a certain time before). In this case, since the hit word information has already been created, referring to this, the speech output may be performed from the corresponding portion (after the speech synthesis and data processing if the processed data does not remain). In order to distinguish hit words, if the user designates a pair of a desired hit word xxx and a desired number of appearances n and inputs a reproduction command, the “nth May be played back from the portion corresponding to the hit word “XX” (or a certain number of characters before or a certain time before).
[0052]
In the above-described procedure example, only the data having the first rank score among the searched data is emphasized with the hit word and the data is output by voice. For example, from the first rank to the predetermined rank. These data may be sequentially output by voice with emphasis on hit words.
[0053]
FIG. 7 shows another example of the overall processing procedure of the information presentation apparatus 1.
[0054]
Here, the description will focus on the point that the example of the procedure in FIG. 7 differs from the example of the procedure in FIG.
[0055]
First, in step S <b> 11, the user inputs a search sentence by text or voice via the input unit 101. Here, for the same purpose as described with reference to FIG. 2, the description will be focused on the case where the search sentence is input to the input unit 101 as text data.
[0056]
Next, in step S <b> 12, the search unit 103 searches for data that matches the search sentence through the communication unit 108.
[0057]
Next, in step S <b> 13, the search unit 103 presents a list of information related to the searched data on the display screen of the output unit 106. In this list, for example, information about search data is arranged in the order of scores obtained at the time of search.
[0058]
Next, in step S14, the input unit 101 receives an instruction from the user.
[0059]
Next, in step S15, if the instruction content input by the user via the input unit 101 in step S14 is an instruction to select desired data in the list presented in step S13, the process proceeds to step S16. If the instruction content indicates an end, the process of FIG. 7 ends.
[0060]
Next, in step S <b> 16, the hit word detection unit 104 extracts hit words for the data selected by the user in step S <b> 14.
[0061]
In step S17, the hit word is emphasized in the same way as in the procedure example of FIG. 2, and the data is output as voice, and the process returns to step S14.
[0062]
Note that the overall processing procedure of the information presenting apparatus 1 is not limited to the procedure examples of FIGS. 2 and 7 and can be implemented with various modifications.
[0063]
For example, instead of the procedure example of FIG. 7, hit word detection processing is performed for all data retrieved subsequent to step S <b> 12 of FIG. 7, and information on the data up to the predetermined order is sorted in order of score. It is possible to make the user selectable by presenting, and perform voice output processing on the data selected by the user. Alternatively, for example, instead of the procedure example of FIG. 7, hit word detection processing is performed on data from the first order to a predetermined order following step S <b> 12 of FIG. 7, and information about the retrieved data is obtained. Sort and present in order of score to allow user selection, and perform voice output processing on the data selected by the user (after hit word detection processing if the data selected by the user has not been hit word detection processing) You may do it.
[0064]
As described above, according to the present embodiment, the user can grasp and confirm the reason why the information presenting apparatus presented the data by listening to the emphasized hit word. In addition, if a character string different from the search sentence is emphasized, it can be known that the search sentence input by voice is recognized erroneously, and the search can be performed again immediately.
[0065]
In this embodiment, data including a character string is targeted, but the present invention can also be applied to other data. For example, voice data to which keyword information is attached is searched based on the keyword information, the searched voice data is converted into character string data, and the character string data is handled in the same manner as the searched data described so far. By doing so, it is also possible to target audio data.
[0066]
(Second Embodiment)
In the present embodiment, an example of an information presentation apparatus having a voice search function for searching voice data will be described.
[0067]
FIG. 8 shows a configuration example of an information presentation apparatus according to the second embodiment of the present invention.
[0068]
Here, the data to be searched is an audio content including an audio part that utters a character string, and an audio output by hit word detection or hit word emphasis is a part corresponding to the character string included in the searched audio data. (Parts capable of voice recognition processing) are targeted. Of course, the present invention can also be applied to a portion of the audio data in content including video data accompanied with such audio data.
[0069]
The hit word is as described in the first embodiment. In the present embodiment, the hit word is described as a portion corresponding to a character string that matches the search word (character string) in the search sentence in the voice data.
[0070]
In FIG. 8, the present information presentation apparatus 2 is connected to an input unit 201 for inputting a search sentence, a voice recognition unit 202 for converting voice data into character string data, and voice data suitable for the search sentence via a network 27. A voice search unit 203 for searching from the voice data server 28, a hit word detection unit 204 for detecting a portion corresponding to a hit word from the searched voice data, and a portion corresponding to a hit word in the searched voice data are emphasized. A reproduction processing unit 205 that processes the data corresponding to the hit word, an output unit 206 that outputs voice data in which the portion corresponding to the hit word is emphasized, and converts the voice data into character string data, and an index (document index) of the converted character string data A document index database 208 for storing data, and a communication unit 209 for performing data communication via the network 27. Eteiru.
[0071]
The network 27 is, for example, a LAN, but is not limited to this, and may be the Internet or the like. The audio data may be based on any format.
[0072]
In the above description, the information presentation apparatus 2 accesses the voice data server 28 having the database storing the voice data via the network 27. Instead of or in addition to this, the information presentation apparatus 2 The voice database 207 storing voice data may be provided, and the voice search unit 203 may search the voice database 207 for voice data that matches the search sentence.
[0073]
Further, when the voice data server 28 has a function of providing a document index, the document index provided by the voice data server 28 may be acquired and referenced at the time of voice data search, or the document index may be stored in advance. You may make it download to the index database 208. FIG.
[0074]
In addition, this information presentation apparatus 2 can be configured as a dedicated machine or can be configured using a general-purpose computer. Further, for example, it can be realized by a mobile phone terminal having various functions.
[0075]
In FIG. 9, an example of the whole process sequence of this information presentation apparatus 2 is shown.
[0076]
Here, a voice search system is described as an example of the information presenting apparatus. The voice search system is a system that can search data recorded in a voice recorder, for example. When searching for voicemail in the voicemail system, searching for audio data recorded from a conference, searching for audio data recorded from a radio broadcast, or searching for an audio track of data recorded from a TV program Can be used. It is assumed that such a voice database 207 is stored in the voice data server 28 or the voice database 207.
[0077]
First, as necessary, the voice recognition unit 202 converts voice data (for all or part of voice data stored in the voice data server 28 or the voice database 207) into a set of character strings (that is, a document). (Step S301). The voice data is converted into character string data using a known voice recognition technique.
[0078]
A time stamp (for example, start time and end time) is attached to the converted character string obtained by converting the voice data, and which part of the original voice data each word of the converted character string corresponds to. Create a document index that you can understand.
[0079]
When converting voice data into character string data, the voice data may be separated for each document by dividing the voice data at a position where the utterance is interrupted for a certain period of time or longer in the voice data (however, voice mail This process is not always necessary when the audio data is originally divided into contents as in the above.
[0080]
Then, for each audio data, a document index that associates each document created from the audio data, that is, each character string of the character string set, and a time stamp indicating a position in the audio data corresponding to each character string, Accumulated in the document index database 208.
[0081]
Note that the speech recognition technology does not necessarily convert the speech into a character string completely without error, and may include errors. Therefore, if there is a misrecognition, it is stored in the document index database 208 with the error included.
[0082]
Next, a user who wants to search for voice data inputs a search sentence by text, voice, or the like via the input unit 201 (step S302). This input method is not limited as long as a text search sentence is finally obtained. For example, both text input (keyboard input) and voice input are possible. It may be possible.
[0083]
When performing voice input, the voice recognition unit 202 recognizes the user's voice and converts it into a character string (search sentence).
[0084]
Note that, regardless of which method is used as the input method, the search sentence is given as a character string to the voice search unit 203. Therefore, in the following, for ease of explanation, the search sentence is input as text data. The explanation will be focused on.
[0085]
Next, the voice search unit 203 uses the document index database 208 to search for a document (corresponding voice data) that matches the search sentence input by the user (step S303).
[0086]
A known method may be used as the voice search method. For example, a full-text search is performed on the document index database 208 based on the search sentence, whereby a suitable document can be searched, and voice data corresponding to the document can be found based on the time stamp. Further, for example, a matching document can be searched from a document set by a search technique such as TF / IDF. It is also possible to search for a document including an answer to a search sentence by QA search. The search result is, for example, a document list arranged in the order of score (see the first embodiment).
[0087]
Next, the hit word detection unit 204 uses the document index database 208 to detect hit words existing in the searched document. As a detection result, information indicating the character string of the detected hit word and the position where the character string appears in the original voice data is obtained (step S304). The appearance position is, for example, the start time and end time of a hit word indicating the position in the voice data obtained when the voice recognition unit 202 converts the voice data into a document. When there are a plurality of portions corresponding to the hit word in the voice data, all the partial voices are detected.
[0088]
Next, the reproduction processing unit 205 first reproduces only the audio from the start time to the end time of the hit word for the audio data corresponding to the retrieved document having the first rank score (step S304). When there are a plurality of hit words, for example, the sound of the portion corresponding to the hit word is reproduced continuously.
[0089]
The user hears the voice of the hit word and determines that the search is correctly performed when the character string in the search sentence is included, and determines that the search cannot be performed correctly when the character string is not included (step S305). ). If the user is able to search correctly, for example, an instruction to indicate that the search has been made correctly, for example, by instructing by voice such as “correct” or “play” or by pressing an “OK” button or “play” button. If the search is not correctly performed, for example, an instruction is given to indicate that the search has not been performed correctly by, for example, indicating “different” or “next” by voice or pressing the “next” button.
[0090]
In step S305, when an instruction indicating that the search has been correctly performed is input from the user, the audio data corresponding to this document is reproduced (for example, from the top) (step S307).
[0091]
On the other hand, in step S305, if the user cannot search correctly (when the user hears a hit word, it is not a character string in the search sentence), it may be an error even if not all the voice data is heard. Since it is understood, the reproduction of the audio data is skipped (step 306), and the process returns to step S304. Then, the same processing is repeated for the audio data corresponding to the retrieved document having the next ranking score.
[0092]
Here, the hit word is described as a portion corresponding to the character string that matches the search word in the search sentence. However, as described in the first embodiment, the character string that belongs to the subordinate concept of the search word in the search sentence. , A character string belonging to a higher-order concept of a search word in the search sentence, a character string similar to the search word in the search sentence, or a character string configured as any combination thereof, or a response part to the search sentence in the case of a QA search Even when the user hears a hit word and a word that does not match the search sentence is reproduced, it can be easily determined that the search result is incorrect.
[0093]
Thus, the user can promptly detect a search error and instruct to reproduce the next audio data without listening to all of the audio data.
[0094]
Instead of the above-described procedure example, for example, only the speech from the start time to the end time of the hit word is sequentially reproduced for the speech data from the first rank to the predetermined rank in the retrieved voice data. Various procedures such as a procedure for reproducing the audio data selected by the user from the top are possible.
[0095]
Here, as in the first embodiment, if a hit word appears during the reproduction of the audio data corresponding to the searched document, the hit word may be emphasized and reproduced. For highlighting hit words, the same configuration examples and various variations as those described in the first embodiment can be used.
[0096]
Further, in the above description, processing is performed using a search statement using a character string and the document index database 208 that documents voice data (that is, by comparing character strings instead of voice data). Spoken speech input (or speech synthesized speech input from the keyboard) and speech data to be searched for in an intermediate language such as speech waveform, phoneme, syllable, phonetic symbol, etc. A configuration for performing a search process, a hit word detection process, and the like by comparison is also possible.
[0097]
As described above, according to the present embodiment, the user can grasp and confirm the reason why the information presenting apparatus presented the data by listening to the hit word. In addition, when a character string different from the search sentence is emphasized as a hit word, there was a misrecognition in voice recognition when creating a document index from voice data, or a search sentence entered by voice was mistakenly recognized, etc. And immediately move to the reproduction of the audio data of the next score rank, or immediately restart the search.
[0098]
(Third embodiment)
In the present embodiment, an example of an information presentation device having a video search function for searching video data will be described.
[0099]
In FIG. 10, the structural example of the information presentation apparatus which concerns on the 3rd Embodiment of this invention is shown.
[0100]
Here, the video data to be searched is accompanied by audio data including an audio portion that utters a character string, and audio output by hit word detection or hit word emphasis is audio data accompanying the searched video data. The part corresponding to the character string included in (a part capable of speech recognition processing) is targeted. Of course, the present invention can also be applied to such contents only with audio data (without video).
[0101]
The hit word is as described in the first embodiment. In the present embodiment, the hit word is described as a portion corresponding to a character string that matches the search word (character string) in the search sentence in the voice data.
[0102]
In FIG. 10, the information presenting apparatus 3 includes an information input / output unit 301 for inputting a search sentence and various instructions and outputting a search result, and a video connected to the network 37 with video data suitable for the search sentence. A data search unit 303 for searching from the data server 38, a hit word detection unit 304 for detecting a portion corresponding to a hit word from the searched video data, and a portion corresponding to a hit word in the searched video data are emphasized. A reproduction processing unit 305 for processing and a communication unit 309 for performing data communication via the network 37 are provided.
[0103]
The information input / output unit 301 is a user interface using a GUI screen.
[0104]
The network 37 is a LAN, for example, but is not limited to this, and may be the Internet or the like. The video data may be based on any format.
[0105]
In the above description, the information presentation apparatus 3 accesses the video data server 38 having the database storing the video data via the network 37, but instead of or in addition to this, the information presentation apparatus 3 However, a video database (video DB) 307 that stores video data may be built in, and the data search unit 303 may search the video database 307 for video data that matches the search text. In this case, the communication unit 309 is not always necessary.
[0106]
In addition, this information presentation apparatus 3 can be comprised as a special purpose machine, or can be comprised using a general purpose computer. Further, for example, it can be realized by a mobile phone terminal having various functions.
[0107]
In FIG. 11, an example of the whole process sequence of this information presentation apparatus 3 is shown.
[0108]
Hereinafter, a case where the TV program information is searched by the information presentation device 3 will be described as a specific example.
[0109]
The TV program information search of this specific example assumes a case where TV news is searched for each article, and the news that the user wants to see can be searched.
[0110]
The video DB 307 is a database of a video recording device such as a hard disk recorder, and includes video and audio. For example, news is recorded, and the news is divided and stored for each article. In this specific example, it is assumed that the information presentation apparatus 3 presents a video that matches the user's search text from the video DB 307 to the user.
[0111]
First, in step S <b> 21, the user inputs a search sentence by text, voice or the like via the information input / output unit 301. Similar to the first and second embodiments, the input method is not limited as long as a text string search sentence can be finally obtained. For example, even if only text input (keyboard input) is possible, only voice input is possible. Either or both of these may be possible. When performing voice input, the information input / output unit 301 may have a voice recognition unit (not shown) that recognizes a user's voice and converts it into a character string (search sentence).
[0112]
Whichever method is used as the input method, the search sentence is given as a character string to the data search unit 303. In the following, the search sentence is input as text data in order to simplify the description. The explanation will be focused on the case.
[0113]
Next, the data search unit 303 searches the video DB 307 for video data that matches the search text (step S22). For this search, for example, a well-known technique for performing a voice search using an audio track of video data can be used (for example, Katsunobu Ito, Satoshi Fujii, Tetsuya Ishikawa, “On-demand lecture system using voice document search”, IPSJ Research Report, 2001-SLP-39, pp.165-170, Dec. 2001).
[0114]
Next, the hit word detection unit 304 detects a hit word from the searched video data (step S23). For example, the method described in the second embodiment may be used to detect the hit word. That is, for example, a document index may be used, or it may be directly detected by an intermediate language such as a speech waveform, phoneme, syllable, or phonetic symbol.
[0115]
Next, a search result screen is displayed (step S24).
[0116]
FIG. 12 shows an example of the search result screen.
[0117]
In the example of FIG. 12, the retrieved data is displayed as thumbnails (reference numeral 403 in the figure) and arranged in the order of ranking (reference numeral 401 in the figure). In the example of FIG. 12, since the data ranked third is audio content without video, the thumbnail is blank. In addition, on the screen of FIG. 12, various other information regarding each content may be presented. Information relating to each content includes, for example, information indicating a channel or a broadcasting station on which the content is broadcast, information indicating whether it is a terrestrial wave or satellite broadcast, information indicating whether it is a television broadcast or a radio broadcast, a broadcast date, Various information such as information indicating time and an address where the data is stored can be considered.
[0118]
Next, in step S25, the information input / output unit 301 receives an instruction from the user.
[0119]
Next, in step S26, the instruction content input by the user via the information input / output unit 301 in step S25 may be an instruction to reproduce the hit word of the desired data on the search result screen displayed in step S24. For example, the process proceeds to step S27, and if it is an instruction to reproduce data of desired data, the process proceeds to step S28. If the instruction content indicates an end, the process of FIG. 7 ends.
[0120]
Here, the processing in step S27 will be described in detail.
[0121]
In the search result screen of FIG. 12 displayed in the previous step S24, there is a [Hit word] button (reference numeral 402 in the figure) along with each data. When the user clicks the hit word button for the desired data with a mouse or the like in step S25, the process transits from step S26 to S27, and the hit word reproduction screen shown in FIG. 13 appears. The word part is played.
[0122]
The hit word playback screen of FIG. 13 includes an area 501 for displaying video corresponding to the sound of the hit word, a search statement 502, a timeline 503 corresponding to the hit word, and a pointer 504 for displaying the current playback position. When there are a plurality of hit words, the timeline 503 corresponds to a combination of a plurality of hit words.
[0123]
As shown in FIG. 14, a button may be provided for each type of hit word. In this case, a screen corresponding to the type of hit work clicked by the user is displayed. If there are a plurality of hit words of this type, the timeline 503 corresponds to a combination of a plurality of hit words.
[0124]
The user can easily determine whether the search result is correct or not by listening to the hit word on such a screen. If a word different from the search sentence is reproduced as a hit word, it is understood that it is not necessary to listen to all of the voice data.
[0125]
Here, the hit word is described as a portion corresponding to the character string that matches the search word in the search sentence. However, as described in the first embodiment, the character string that belongs to the subordinate concept of the search word in the search sentence. , A character string belonging to a higher-order concept of a search word in the search sentence, a character string similar to the search word in the search sentence, or a character string configured as any combination thereof, or a response part to the search sentence in the case of a QA search Even when the user hears a hit word and a word that does not match the search sentence is reproduced, it can be easily determined that the search result is incorrect.
[0126]
In the example of FIG. 13, there are an [OK] button 505 and an [NG] button 506 below the time bar to be reproduced.
[0127]
If the user hears the playback of the hit word and the same word as the search sentence is played, the [OK] button 505 is pressed. On the other hand, if a word different from the search sentence is reproduced as a hit word, the [NG] 506 button is pressed.
[0128]
When the [OK] button 505 is pressed, the character string of the query corresponding to this hit word is added to this voice data together with a positive weight (+ a).
[0129]
When the [NG] button 506 is pressed, a query character string is added to this voice data together with a negative weight (-b).
[0130]
The positive weight to be added may be limited to (+ a), and the negative weight to be added may be limited to (−b).
[0131]
This additional character string is used in subsequent searches.
[0132]
For example, when the hit word is reproduced with the voice data searched with the search character string “XX”, when the hit word is reproduced correctly and the user presses the [OK] button, this voice is reproduced. It can be seen that this part of the data contains the voice “OO”. For this reason, the character string “XX” is added to the audio data with a positive weight. On the other hand, when this hit word is played back, if “XX” is played instead of “XX” and the user presses the [NG] button, the character string “XX” is displayed in this audio data. Added with negative weight. If there is a character string “XX” in the search sentence from the next time, voice data with a positive weight “XX” in the additional character string will be searched preferentially (or the score will be higher). become. On the other hand, speech data having a negative weight “OO” in the additional character string is not searched (or the score is lowered).
[0133]
The score considering the additional character string is, for example, as follows.
[0134]
The similarity score between the current search sentence (search character string) and the additional character string added to the voice data is Sa (for example, + a or -b described above), and the similarity score between the new search sentence and the voice data ) As Sd, and the final score
S = αSd + (1−α) Sa
And Here, α is an appropriate number between 0 and 1.
[0135]
As for the similarity Sd between the new search sentence and the voice data, the similarity Sd can be obtained using a search character string indicating the new search sentence and a character string document obtained by converting the voice data into a character string. It is also possible to obtain the similarity Sd by using the intermediate language such as speech waveform, phoneme, syllable, phonetic symbol, etc., for the similarity between the new search sentence and the target speech data.
[0136]
When the search character string hits the additional character string, for example, as illustrated in FIG. 12, a mark (in this example, a star mark) 404 is added to the head of the title, and the additional character string is hit. You may make it understand.
[0137]
When the process of step S27 is completed, the process returns to step S25.
[0138]
Next, the processing in step S28 will be described in detail.
[0139]
In the search result screen of FIG. 12 displayed in the previous step S24, there is a thumbnail (reference numeral 403 in the figure) together with each data. In step S25, when the user clicks a thumbnail for the desired data with a mouse or the like, the process proceeds from step S26 to S28, and the data reproduction screen shown in FIG. 15 appears, and the data specified by the user (video data or audio data). ) Is played.
[0140]
In the data reproduction screen of FIG. 15, there are an area 601 for reproducing a video, a search statement 602, a timeline 603 corresponding to the entire content, and a pointer 604 for displaying the current reproduction position. In FIG. 15, a mark 607 is added to the timeline 603 at the position where the hit word appears. As a result, it can be determined at which position the hit word appears.
[0141]
As shown in FIG. 16, the hit word and the text before and after it may be displayed in the vicinity of each mark 607 as a character string (in FIG. 16, XX represents the hit word). , ... represents the text before and after that).
[0142]
When the process of step S27 is completed, the process returns to step S25.
[0143]
In this embodiment, as in the first embodiment, if a hit word appears during the reproduction of the audio data corresponding to the searched document, the portion may be reproduced with emphasis. For highlighting hit words, the same configuration examples and various variations as those described in the first embodiment can be used.
[0144]
As a result, even if the speech recognition of the system has been wrong so far, it can be seen that there is no search sentence after all the retrieved data is reproduced, and it is wrong if all the erroneous search results are not reproduced. I didn't understand. With the hit word emphasis function of this system, it is possible to notice a system error in advance without reproducing the data, so that the time for searching for target data is greatly shortened.
[0145]
Each of the above functions can be realized even if it is described as software and processed by a computer having an appropriate mechanism.
The present embodiment can also be implemented as a program for causing a computer to execute predetermined means, causing a computer to function as predetermined means, or causing a computer to realize predetermined functions. In addition, the present invention can be implemented as a computer-readable recording medium on which the program is recorded.
[0146]
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.
[0147]
【The invention's effect】
According to the present invention, it is possible to present information more effectively by voice.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration example of an information presentation apparatus according to a first embodiment of the present invention.
FIG. 2 is a flowchart showing an example of a processing procedure of the information presentation apparatus according to the embodiment.
FIG. 3 is a diagram showing an example of hit word information
FIG. 4 is a diagram showing another example of hit word information
FIG. 5 is a diagram showing still another example of hit word information
FIG. 6 is a diagram showing still another example of hit word information
FIG. 7 is a flowchart showing another example of the processing procedure of the information presentation apparatus according to the embodiment.
FIG. 8 is a diagram showing a configuration example of an information presentation device according to a second embodiment of the present invention.
FIG. 9 is a flowchart showing an example of a processing procedure of the information presentation apparatus according to the embodiment.
FIG. 10 is a diagram showing a configuration example of an information presentation apparatus according to a third embodiment of the present invention.
FIG. 11 is a flowchart showing an example of a processing procedure of the information presentation apparatus according to the embodiment.
FIG. 12 is a diagram showing an example of a search result screen
FIG. 13 shows an example of a hit word playback screen.
FIG. 14 is a diagram showing another example of the search result screen
FIG. 15 shows an example of a data playback screen.
FIG. 16 is a diagram showing another example of the data reproduction screen.
[Explanation of symbols]
1, 2, 3 ... Information presentation device, 8 ... Data server, 7, 27, 37 ... Network, 18 ... Audio data server, 28 ... Image data server, 101, 201 ... Input unit, 103 ... Search unit, 104, 204 , 304 ... hit word detection section, 105 ... data processing section, 106, 206 ... output section, 107 ... speech synthesis section, 108, 209, 309 ... communication section, 109 ... database, 202 ... voice recognition section, 203 ... voice search , 205, 305... Reproduction processing unit, 207... Audio database, 208... Document index, 301.

Claims (2)

An input means for inputting a search sentence;
Based on the search sentence, an acquisition means for searching a database storing content including audio data or video data accompanied by audio data and acquiring the content;
Detecting means for detecting, as a hit word , a voice data portion in which a character string matching the search sentence is recognized from the voice data included in the acquired content;
On the display screen, a first image for specifying that reproduces the video data with audio data or audio data included in the content, as before Symbol Hit word from by Ri in the audio data to said detecting means A second image for designating reproduction of the detected audio data portion , and when the first image is selected by the user, the audio data or audio data included in the content is displayed. reproduces the video data with, in the case where the image of the second has been selected by the user, and a user interface means for reproducing the detected audio data portion as the hit word,
When the second image is selected, the user interface means includes a third image for inputting a positive evaluation and a negative result for the reproduction result of the audio data portion detected as the hit word. A fourth image for inputting the evaluation is displayed on the display screen, and when the third image is selected by the user, a positive evaluation is stored in association with the content, and the user When the fourth image is selected, a negative evaluation is stored in association with the content,
It said acquisition means, in accordance with the calculated search score for the content, along with it's also sort the obtained content, if the evaluation of the positive in correspondence with the content is stored, for the content the sorting order of the search score is increased or the contents to the top, if not constant for evaluation is stored, also shall be the that do not get the allowed or the content reduces the search score for the content An information presenting apparatus characterized by that. An information presentation method for an information presentation apparatus comprising search text input means, acquisition means, detection means, and user interface means,
The search text input means for inputting a search text;
The acquisition means searches the database storing content including audio data or video data accompanied by audio data based on the search sentence, and acquires the content;
A step wherein the detecting means, for detecting the in the speech data contained in the acquired the content, the audio data portion matching string is recognized by the search statement, as a hit word,
Said user interface means, on the display screen, a first image for specifying that reproduces the video data with audio data or audio data included in the prior SL content, the audio Ri by said detecting means and it displays a second image for specifying to play the detected audio data portion as before Symbol hit word from the data, if the first image is selected by the user, the content reproduces the video data with audio data or audio data included, if the image of the second has been selected by the user, and a step of reproducing the detected audio data portion as the hit word,
When the second image is selected, the user interface means includes a third image for inputting a positive evaluation and a negative result for the reproduction result of the audio data portion detected as the hit word. A fourth image for inputting the evaluation is displayed on the display screen, and when the third image is selected by the user, a positive evaluation is stored in association with the content, and the user When the fourth image is selected, a negative evaluation is stored in association with the content,
It said acquisition means, in accordance with the calculated search score for the content, along with it's also sort the obtained content, if the evaluation of the positive in correspondence with the content is stored, for the content the sorting order of the search score is increased or the contents to the top, if not constant for evaluation is stored, also because you and that do not get the allowed or the content reduces the search score for the content An information presentation method characterized by being.

Images