JP2012181358A - Text display time determination device, text display system, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a text display time determination device, a text display system, a text display time determination method, and a text display time determination program, which can generate a caption easy to read and understand for users when sequentially converting an input voice into text to display.SOLUTION: A text display device comprises a recognition result creation means 81 and a display time determination means 82. The recognition result creation means 81 creates a recognition result obtained by sequentially recognizing an input voice and converting it into text. The display time determination means 82 determines a display time for each sentence contained in the recognition result on the basis of the utterance time of the voice.

Description

  The present invention relates to a text display time determination device, a text display system, a text display time determination method, and a text display time determination program for converting input speech in real time and determining the display time of the converted text.

  In order to improve the efficiency of creating subtitles to be displayed on a television or a movie, a method of automatically converting speech into characters using speech recognition technology is known. By manually correcting the recognition result, the time and labor of the work can be saved as compared with the method of creating subtitles while the user confirms the contents of the pre-recorded video.

  In addition, since the work can be made more efficient by using voice recognition technology for the creation of captions, captions can be added even to live broadcast programs such as news that the distribution video is determined immediately before airing. Patent Document 1 describes a caption shift correction device that performs caption broadcasting on a live broadcast program such as news. The correction apparatus described in Patent Document 1 has a function of converting subtitles into data immediately before broadcasting by a relay method and pre-editing the converted subtitles. In addition, the correction apparatus described in Patent Document 1 has a switching function based on block management to cope with real-time input. Therefore, for example, when subtitles are urgently replaced, such as confirmation and correction of subtitles in a program or live broadcast that is brought in close to the broadcast, pre-processing that has been difficult until now can be performed smoothly.

  Patent Document 2 describes a notation character string conversion method for performing voice recognition more accurately. In the method described in Patent Document 2, when a notation character string is created by extracting feature amounts from voice data input by an operator, this notation character string is displayed on the display means in an unconfirmed state. Then, when a conversion command is given for the target portion of the displayed character string, conversion based on this command is performed.

  Patent Document 3 describes a caption output device that outputs captions corresponding to speech sounds that are uttered based on text prepared in advance for program broadcasting. The caption output device described in Patent Document 3 can reliably output text immediately before a commercial as caption by setting a text collation range for a certain recognition result to be longer than the length of the recognition result. .

  Note that Patent Document 4 and Non-Patent Document 1 describe a method for detecting a speech rate.

JP 2007-202094 A JP 2000-10971 A JP 2009-182859 A JP-A-9-146575

Masahiro Ohno and four others, “Real-time caption generation based on simultaneous monologue speech summarization”, Information Processing Society of Japan, v.SLP-62-10, 2006, pp.51-56

  Generally, in a system that displays a recognition result created simultaneously with speech recognition of speech as subtitles, the recognition result is displayed as soon as one recognition process is completed. For this reason, the display time of the recognition result depends on the processing time of the speech recognition performed immediately after the recognition process that created the recognition result. Therefore, if the utterance immediately after creating the recognition result is short and the utterance recognition process ends quickly, even if the recognition result (number of displayed characters) displayed before that utterance is long, the next utterance is immediately Results may be displayed. In such a case, the user may not be able to read the recognition result.

  Recently, there is an increasing need to make speech into text at the same time as speaking in seminars, lectures, university lectures, and conferences. However, like the correction device described in Patent Document 1, the method of creating subtitles for a news program requires a lot of human resources, and it is necessary to secure a place for installing dedicated equipment and editing work in the venue. It is necessary to do. For this reason, it is difficult to use for these purposes because of cost, manpower, and burden of equipment installation.

  Furthermore, the correction apparatus described in Patent Document 1 is an apparatus based on the premise that subtitle character input is performed in advance by a relay method using a plurality of terminals. When urgent live broadcasting correspondence is performed using the correction apparatus described in Patent Document 1, it is necessary to sequentially input subtitles. Therefore, it is difficult to always apply the correction device described in Patent Document 1 to a scene where subtitles that are converted into text by sequentially recognizing input speech are displayed.

  In the notation character string conversion method described in Patent Document 2, a notation character string is created on the basis of voice data, and then an operator gives a conversion command. As described above, it is difficult to continuously provide a voice recognition result that is constantly created by a method in which an operator issues a command for a written character string each time.

  In addition, the subtitle output device described in Patent Document 3 is based on the premise that a performer of a television program does not utter anything other than the predetermined text content. Therefore, the method cannot be applied to a method of recognizing output sound each time and displaying the recognition result in subtitles.

  Accordingly, the present invention provides a text display time determination device, a text display system, a text display time determination method, and a text display time that can generate subtitles that are easy to read and understand for the user when the input speech is sequentially converted into text. The purpose is to provide a decision program.

  The text display time determination device according to the present invention recognizes input speech sequentially and creates a recognition result by creating a recognition result, and displays each sentence included in the recognition result based on the speech utterance time. The display time determining means for determining the time is provided.

  A text display system according to the present invention includes a voice input device that inputs voice, a voice recognition device that recognizes voice input to the voice input device, and a recognition result display device that displays a voice recognition result by the voice recognition device. A speech recognition device for recognizing the speech input to the speech input device in order to create a recognition result that is converted into text, and for each sentence included in the recognition result based on the speech utterance time Display time determining means for determining a display time.

  A method for determining a text display time according to the present invention creates a recognition result obtained by sequentially recognizing input speech, and determines the display time for each sentence included in the recognition result based on the speech utterance time. It is characterized by.

  A text display time determination program according to the present invention is included in a recognition result based on a recognition result creation process for creating a recognition result obtained by sequentially recognizing input speech and converting it into text, and a speech utterance time. A display time determination process for determining the display time for each sentence is executed.

  According to the present invention, it is possible to generate subtitles that are easy to read and understand for the user when the input voice is sequentially converted into text and displayed.

It is a block diagram which shows the example of the text display system in the 1st Embodiment of this invention. It is explanatory drawing which shows the example of the recognition result after speech recognition. It is a block diagram which shows the example of a speech recognition means and a dictionary storage means for speech recognition. It is explanatory drawing which shows the example of the information defined in a dictionary. It is explanatory drawing which shows the example of the information which a conversion database memory | storage means memorize | stores. It is a flowchart which shows the example of operation | movement of the text display system in 1st Embodiment. It is explanatory drawing which shows the example of a speech recognition process and a recognition result conversion process. It is explanatory drawing which shows the example of the text display system in the modification of 1st Embodiment. It is a block diagram which shows the example of the text display system in the 2nd Embodiment of this invention. It is explanatory drawing which shows the example of the information which a keyword database storage means memorize | stores. It is a flowchart which shows the example of operation | movement of the text display system in 2nd Embodiment. It is a block diagram which shows the example of the minimum structure of the text display time determination apparatus by this invention. It is a block diagram which shows the example of the minimum structure of the text display system by this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing an example of a text display system according to the first embodiment of the present invention. The text display system in this embodiment is described as voice input means 1, voice recognition means 2, voice recognition dictionary storage means 3, recognition result conversion means 4, and conversion database storage means 5 (hereinafter referred to as conversion DB 5). ), Display time determination means 6 and text display means 7.

  The voice input means 1, the voice recognition means 2, the recognition result conversion means 4, the display time determination means 6 and the text display means 7 operate by program control. As for these means, all the means may be included in one terminal. These means may be included in different terminals for each means, and may be connected to each other via the Internet, a LAN (Local Area Network), or the like.

  The voice input unit 1 notifies the voice recognition unit 2 of the input voice. A file representing sound (hereinafter referred to as a sound file) may be input to the sound input means 1.

  The voice recognition dictionary storage means 3 stores various information used when the voice recognition means 2 performs voice recognition. The contents of the voice recognition dictionary storage means 3 will be described later.

  The voice recognition unit 2 sequentially recognizes the voice input to the voice input unit 1 and creates a text recognition result. Further, the voice recognition means 2 calculates a time when each word included in the voice is uttered (hereinafter referred to as an utterance time length). Specifically, the voice recognition unit 2 analyzes the input voice file and calculates the acoustic feature amount and the utterance time length of each word included in the voice file. Further, the voice recognition means 2 refers to the voice recognition dictionary storage means 3 and extracts a word or word string closest to the acoustic feature of the voice file from the stored words or word strings. Then, the voice recognition unit 2 outputs the extracted word or word string as a result of voice recognition. At this time, the voice recognition means 2 associates the speech duration length calculated based on the voice file, the notation, the reading, and the part of speech stored in the voice recognition dictionary storage means 3 in units of words, and the associated contents. The recognition result conversion means 4 is notified as a voice recognition result.

  FIG. 2 is an explanatory diagram illustrating an example of a recognition result after speech recognition. The recognition result illustrated in FIG. 2 indicates that the sentences included in the speech data include the words SP1 to SPn, and the speech duration, notation, reading, and part of speech are associated with each word.

  Here, the contents of the voice recognition means 2 and the voice recognition dictionary storage means 3 will be further described. FIG. 3 is a block diagram showing an example of the voice recognition means 2 and the voice recognition dictionary storage means 3. The voice recognition unit 2 includes a voice detection unit 21, a voice analysis unit 22, and a voice collation unit 23. The speech recognition dictionary storage means 3 stores an acoustic model 31, a language model 32, and a dictionary 33.

  The acoustic model 31 includes a standard pattern for each Japanese phoneme.

  The language model 32 is a model in which appearance probabilities of words included in a dictionary 33 described later are converted into data. The language model 32 includes information defining connection relationships between Japanese words and phonemes, grammatical rules defining connection relationships between words, and the like.

  The dictionary 33 is a word-to-be-recognized word, and the notation and reading of the corresponding part, part-of-speech information, etc. are defined in units of words or word strings. FIG. 4 is an explanatory diagram illustrating an example of information defined in the dictionary 33. In the example shown in FIG. 4, the dictionary 33 indicates that word unit notation, reading, and part of speech are included. For example, the reading of the notation “VPC” illustrated in FIG. 4 is “Buipsy”, and the part of speech is “noun”. Further, the reading of the notation “SaaS” is “Saas”, and the part of speech is “noun”.

  The acoustic model 31, the language model 32, and the dictionary 33 are stored in advance in the speech recognition dictionary storage unit 3 by a user or the like.

  The voice detection unit 21 separates voice and noise from the input voice file, detects a section including the voice, and sends it to the voice analysis unit 22. As a method for detecting the sound by the sound detection unit 21, for example, a method using the power of the sound indicated by the sound file can be used. In this voice detection method, the power of the voice indicated by the voice file is calculated sequentially. Then, the time when the power exceeds a predetermined threshold for a certain period of time is determined as the voice start time, and the time when the power falls below the predetermined threshold for a certain period of time is determined as the voice end time. To do. The voice detection unit 21 sequentially transmits the voice from the voice start time to the voice end time as voice indicating one sentence to the voice analysis unit 22. Note that the voice detection unit 21 may determine a word (sequence) separated by punctuation points as one sentence, and may determine a word (sequence) up to the speech separation as one sentence.

  The voice analysis unit 22 performs an acoustic analysis of the voice extracted by the voice detection unit 21 and sends an acoustic feature expressing the feature of the voice to the voice collation unit 23.

  When the speech collation unit 23 receives the acoustic features of the speech from the speech analysis unit 22, the speech collation unit 23 performs speech recognition using a standard Japanese phoneme pattern stored in the acoustic model 31 and the language model 32, and performs speech recognition. Output the result as text.

  The conversion DB 5 stores a predetermined word or word string and a notation converted to be shorter than the word or word string (hereinafter referred to as a converted notation) in association with each other. Specifically, the conversion DB 5 stores a word or a word string that needs to be converted from a word or a word string included in the speech recognition result in association with the converted notation. Note that the notation converted to be shorter than the word or the word string (that is, the notation after conversion) includes those that display nothing. In addition to the notation, the conversion DB 5 may store reading of the notation and attributes of the notation. For example, when the notation is a word, the corresponding part of speech is set as the notation attribute. When the notation is a word string, the notation attribute is set to be a word string. The conversion DB 5 stores a post-conversion notation in advance by a user or the like.

  FIG. 5 is an explanatory diagram illustrating an example of information stored in the conversion DB 5. The example shown in FIG. 5 indicates that the notation, reading, attribute, and converted notation of the word or word string are stored in association with each other. For example, the word “Etto” illustrated in FIG. 5 is “Etter” for reading and “Filler” for attribute representing part of speech. Further, by setting the item of the notation after conversion of the word “um” as blank (that is, displaying nothing), it means “deleting the word”. In addition, the word string “That's why” illustrated in FIG. Further, by setting the item of the attribute of the word string as “word string”, it means “it is a word string composed of a plurality of words”. Furthermore, it means that the converted notation of this word string is “I”.

  The recognition result conversion means 4 converts the word or word notation included in the recognition result into a converted notation corresponding to the word or word string stored in the conversion DB 5. Specifically, the recognition result conversion unit 4 refers to the conversion DB 5 and converts or deletes the recognition result to the converted notation for the word or word string that needs to be converted among the recognition results generated by the speech recognition unit 2. I do.

  For example, if the recognition result by the speech recognition means 2 is “Well, the matter will be considered from now on”, the recognition result conversion means 4 deletes the word “Ehto” and the word string “Well. Is "is converted to" is ". As a result, the recognition result is converted to “I will consider the matter from now on”.

  The display time determination means 6 determines the display time for each sentence included in the recognition result based on the speech utterance time recognized by the voice recognition means 2. Specifically, the display time determination unit 6 calculates the display time for each sentence to be displayed based on the recognition result converted by the recognition result conversion unit 4. Here, the number of words included in the displayed text may be one or plural. In addition, what is necessary is just to make one sentence into one sentence from the audio | voice start time determined by the audio | voice detection part 21 to the audio | voice end time. The display time determination unit 6 calculates the display time for each sentence based on, for example, Equation 1 exemplified below.

T = S × W (1)
(However, S = S1 + S2 + ... + Sn)

  Here, n is the number of words included in one sentence to be displayed (that is, the sentence after being converted by the recognition result conversion means 4), and Sn is a word n included in the sentence to be displayed. This is the length of the utterance time of the word before S is the total utterance time length of the word n, and W is a weight value for S. Hereinafter, the weight value W may be referred to as a display weight value.

  The display weight value W can be calculated by the following equation (2).

  W = utterance speed per sentence / average utterance speed (2)

  The utterance speed here is the utterance speed of the text before conversion, and can be expressed by the number of phonemes within a fixed time, as described in Patent Document 4 and Non-Patent Document 1, for example. The number of phonemes within a certain time can be calculated by “number of phonemes included in one sentence / the length of utterance of one sentence”. The average utterance speed may be the average value of utterance speeds calculated for the sentence up to immediately before the display weight value W is calculated. Note that the method of calculating the display weight value W is not limited to the above method. The display time determination means 6 may determine the value of the display weight value W from the actual measurement value of the optimal display time.

  Thus, since the display time is calculated from a value obtained by weighting the utterance time length (actually measured value) of each word, the display time is correlated with the utterance speed (that is, the phoneme number / sentence utterance time). I can say that there is.

  The text display means 7 receives the converted recognition result and displays the recognition result for the calculated display time.

  The voice recognition unit 2, the recognition result conversion unit 4, and the display time determination unit 6 are realized by a CPU of a computer that operates according to a program (text display time determination program). For example, the program is stored in a storage unit (not shown) of the voice recognition device, and the CPU reads the program and operates as the voice recognition unit 2, the recognition result conversion unit 4, and the display time determination unit 6 according to the program. May be. Moreover, each of the speech recognition unit 2, the recognition result conversion unit 4, and the display time determination unit 6 may be realized by dedicated hardware.

  The voice recognition dictionary storage means 3 and the conversion DB 5 are realized by a magnetic disk, for example. The voice input unit 1 is realized by, for example, a microphone, and the text display unit 7 is realized by, for example, a display device.

  Next, the operation will be described. FIG. 6 is a flowchart showing an example of the operation of the text display system in the first embodiment.

  First, when a voice is input via the voice input means 1 (step A1), the voice recognition means 2 receives voice data from the voice input means 1, and recognizes the voice by referring to the voice recognition dictionary storage unit 3. (Step A2). At this time, the voice recognition means 2 calculates the utterance time length of the word or word string included in the voice recognition result together with the voice recognition.

  Subsequently, when the recognition result conversion unit 4 receives a recognition result including a word or a word string from the speech recognition unit 2, the recognition result conversion unit 4 refers to the conversion DB 5 to determine whether or not the word corresponding to the recognition result is included ( Step A3). When a word corresponding to the conversion DB 5 exists, the recognition result conversion unit 4 converts the word into a corresponding notation (ie, a notation after conversion) and notifies the display time determination unit 6 (step A4). When there is no word corresponding to the conversion DB 5, the recognition result conversion unit 4 notifies the display time determination unit 6 as it is without performing the conversion process.

  The display time determining means 6 determines the time for displaying the received recognition result based on the recognition result received from the recognition result converting means 4 and the utterance time of the word included in the recognition result (step A5). . The text display means 7 displays the recognition result for the length of time determined by the display time determination means 6 (step A6).

  FIG. 7 is an explanatory diagram illustrating an example of a voice recognition process and a recognition result conversion process. FIG. 7 illustrates notation when the speech recognition unit 2 recognizes speech in step A2 in FIG. 6 and notation when the recognition result conversion unit 4 converts (deletes) unnecessary words in step A4.

  For example, as a result of the speech recognition in step A2, the notation is determined to be "Uh, so we will hold a joint meeting now." In step A3, the recognition result conversion means 4 refers to the conversion DB 5, and as a result, It is assumed that a word (a conversion target term) corresponding to the included word “eto” is found. Then, the recognition result conversion means 4 deletes the unnecessary word “Ut” in step A4, and creates a result converted into “Now we will hold a joint meeting now”.

  As described above, according to the present embodiment, the recognition result conversion unit 4 sequentially recognizes input speech and creates a recognition result that is converted into text. Then, the display time determining means 6 determines the display time for each sentence included in the recognition result based on the speech utterance time. Specifically, the recognition result conversion unit 4 converts the notation of the word or word string included in the input speech recognition result into a converted notation corresponding to the word or word string stored in the conversion DB 5. Then, the display time determination unit 6 determines the display time for each sentence included in the recognition result converted into the post-conversion notation. With such a configuration, it is possible to generate subtitles that are easy to read and understand for the user when the input speech is sequentially converted into text.

  That is, the text display system according to the present embodiment determines the display content and display time when displaying the converted character on the terminal in consideration of the actual utterance content and utterance time. Can be generated.

  In addition, when the content being spoken is sequentially recognized and displayed as subtitles, the user is presented with both the sound currently being spoken and the subtitles of the recognition result. Specifically, the timing when the recognition result is displayed is always after the end of the utterance. If the recognition result subtitle display is too early or too late, it may give the user an unnatural feeling and hinder understanding of the content being spoken. Therefore, it is desirable to display the recognition result based on the utterance time of the original voice as much as possible. In the present embodiment, it is possible to display subtitles that are easy to read and understand for the user by deleting words that are unnecessary (meaningless) by the recognition result conversion means 4 and shortening words that are not very important.

  Next, a modification of this embodiment will be described. The text display system in this modification is implemented by a plurality of devices, and each device is connected via the Internet. FIG. 8 is an explanatory diagram showing an example of a text display system in the present modification. The text display system illustrated in FIG. 8 includes a voice sending terminal 10, a voice recognition server 20, and a recognition result display terminal 30.

  The voice sending terminal 10 includes a voice input unit 1. The speech recognition server 20 includes speech recognition means 2, speech recognition dictionary storage means 3, recognition result conversion means 4, conversion DB 5, and display time determination means 6. The recognition result display terminal 3 includes text display means 7. The contents of the voice input unit 1, the voice recognition unit 2, the voice recognition dictionary storage unit 3, the recognition result conversion unit 4, the conversion DB 5, the display time determination unit 6, and the text display unit 7 are the same as those in the first embodiment. It is the same.

  As described above, even if the configuration is via the Internet, subtitles that are easy to read and understand for the user can be generated when the input speech is converted into text and displayed.

Embodiment 2. FIG.
FIG. 9 is a block diagram showing an example of a text display system according to the second embodiment of the present invention. In addition, about the structure similar to 1st Embodiment, the code | symbol same as FIG. 1 is attached | subjected and description is abbreviate | omitted. The text display system according to this embodiment includes a voice input unit 1, a voice recognition unit 2, a voice recognition dictionary storage unit 3, a recognition result conversion unit 4, a conversion DB 5, a display time determination unit 6, and a text display. Means 7, important word extraction means 8, and important word database storage means 9 (hereinafter referred to as important word DB 9) are provided.

  That is, the text display system according to the present embodiment further includes an important word extraction unit 8 and an important word DB 9 in the configuration of the text display system according to the first embodiment.

  The important word DB 9 stores a weight value by which the utterance time length is multiplied for each word. Specifically, the important word DB 9 stores a weight value to be given to the utterance time length of a word that is assumed to be important among the words included in the speech recognition result. Hereinafter, this weight value may be referred to as an utterance time length weight value.

  FIG. 10 is an explanatory diagram illustrating an example of information stored in the keyword DB 9. The example shown in FIG. 10 indicates that the word notation, reading, part of speech, and weight value assigned to the word are stored in association with each other. For example, the notation “revenue” illustrated in FIG. 10 has a reading “Shueki” and a part of speech “Noun”. Further, the weight value assigned to the utterance time length of the word is “1.3”.

  The important word DB 9 stores an utterance time length weight value for each word in advance by a user or the like. For example, a personal name, a number, or a product name may be extracted from the speech recognition dictionary 3 and the extracted word may be stored in the important word DB 9. In addition, the words extracted from the user dictionary for speech recognition may be stored, or the words included in the important word list created in advance by the user may be stored in the important word DB 9.

  Moreover, an arbitrary value can be set as the utterance time length weight value set for each word. For example, the appearance frequency of a word may be obtained from a corpus that has created a language model, and a greater weight may be set for a more frequent word. Also, an arbitrary value may be set for each word based on the user's experience.

  The keyword extraction unit 8 multiplies the utterance time length of the word included in the speech recognition result generated by the speech recognition unit 2 by the weight value of the corresponding word stored in the keyword DB 9. Specifically, the important word extraction means 8 refers to the important word DB 9, and when a word included in the recognition result generated by the speech recognition means 2 exists in the important word DB 9, the corresponding weight is set as the utterance time of the recognition result. Multiply the length. For example, the keyword extraction unit 8 may assign a weight value to the utterance time length on the basis of Equation 3 illustrated below.

  Sm ′ = Sm × I (3)

  Here, m is the number of words included in the recognition result converted by the speech recognition means 2, and Sm is the utterance time length of the word m. I represents the weight (speech time length weight value) of the word m stored in the important word DB 9, and Sm ′ represents the utterance time length of the word m after the important word extraction means 8 gives the weight. . For example, it is assumed that the word “revenue” is included in the recognition result and the utterance time length of the word is 1.0. Here, when the weight value = 1.3, the keyword extraction unit 8 calculates the utterance time length Sm ′ after processing as 1.0 × 1.3 = 1.3.

  The voice recognition unit 2, the recognition result conversion unit 4, the display time determination unit 6, and the keyword extraction unit 8 are realized by a CPU of a computer that operates according to a program (text display program). Further, each of the speech recognition unit 2, the recognition result conversion unit 4, the display time determination unit 6, and the keyword extraction unit 8 may be realized by dedicated hardware. The voice recognition dictionary storage means 3, the conversion DB 5, and the keyword DB 9 are realized by, for example, a magnetic disk.

  Next, the operation will be described. FIG. 11 is a flowchart showing an example of the operation of the text display system in the second embodiment. Note that the processes from step A1 to step A2 from when a voice is input until voice recognition is performed are the same as those in the first embodiment.

  Next, the keyword extraction unit 8 refers to the keyword DB 9 and assigns the weight of the word included in the recognition result generated by the speech recognition unit 2 to the utterance time length of the recognition result (step B1).

  Thereafter, the processing from converting the recognition result to determining the display time and displaying the text is the same as the processing from step A3 to step A7 in the first embodiment.

  As described above, according to the present embodiment, the important word extracting unit 8 multiplies the utterance time length of each word included in the input speech recognition result by the corresponding weight value stored in the important word DB 9. Therefore, in addition to the effect of the first embodiment, when an important word is included, the recognition result can be displayed longer than usual, so that the subtitle can be more easily seen.

  Next, an example of the minimum configuration of the present invention will be described. FIG. 12 is a block diagram showing an example of the minimum configuration of the text display time determination device according to the present invention. The text display time determination device according to the present invention is based on a recognition result creation unit 81 (for example, speech recognition unit 2) that sequentially recognizes input speech and creates a text recognition result, and a speech utterance time. Display time determining means 82 (for example, display time determining means 6) for determining the display time for each sentence included in the recognition result is provided.

  With such a configuration, it is possible to generate subtitles that are easy to read and understand for the user when the input speech is sequentially converted into text.

  In addition, the text display time determination device includes a post-conversion storage means (for example, a post-conversion storage means for storing a predetermined word or word string and a post-conversion post-conversion that is shorter than the word or word string) Conversion DB 5) and recognition result conversion means for converting a word or word string notation included in the input speech recognition result into a converted notation corresponding to the word or word string stored in the converted notation storage means ( For example, recognition result conversion means 4) may be provided. Then, the display time determining unit 82 may determine the display time for each sentence included in the recognition result converted into the post-conversion notation.

  Further, the post-conversion notation storage means may store a notation (for example, blank) indicating that the word or the word string is deleted as the post-conversion notation. Then, the recognition result conversion means may delete the word or word string from the recognition result when the post-conversion notation of the word or word string is a notation indicating deletion. In this way, unnecessary (nonsense) speech can be deleted, so that subtitles that are easier to read and understand for the user can be displayed.

  The text display time determination device further includes an utterance time length calculation unit (for example, a voice recognition unit 2) that calculates, for each word, an utterance time length that is a time when a word included in the input speech recognition result is uttered. You may have. Then, the display time determining means 82 uses the display time of the sentence included in the recognition result converted into the converted notation based on the utterance time length of the word before the word included in the sentence is converted into the converted notation. May be determined.

  Further, the display time determining means 82 uses a value obtained by multiplying the ratio of the utterance speed of the sentence to be displayed with respect to the average value of the utterance speed (for example, the display weight value W) by the sum of the utterance time lengths of the words as the display time. It may be determined (for example, determined based on Equation 1 and Equation 2).

  Further, the text display time determination device stores weight value storage means (for example, important value) for storing a speech time length weight value (for example, speech time weight value I), which is a weight value to be multiplied by the speech time length, for each predetermined word. Word DB 9) and weight value assigning means (for example, important word extracting means 8) for multiplying the utterance time length of each word included in the input speech recognition result by the corresponding utterance time length weight value. Also good. With such a configuration, when an important word is included, the recognition result can be displayed longer than usual, so that the subtitle can be more easily seen.

  FIG. 13 is a block diagram showing an example of the minimum configuration of the text display system according to the present invention. The text display system according to the present invention includes a voice input device 70 (for example, the voice sending terminal 10) for inputting voice, and a voice recognition device 80 (for example, the voice recognition server 20) for recognizing the voice input to the voice input device 70. And a recognition result display device 90 (for example, a recognition result display terminal 30) for displaying a speech recognition result by the speech recognition device 80.

  The voice recognition device 80 includes a recognition result creation unit 81 (for example, the voice recognition unit 2) and a display time determination unit 82 (for example, the display time determination unit 6). The contents of the recognition result creation means 81 and the display time determination means 82 are the same as the contents illustrated in FIG. Even with such a configuration, it is possible to generate subtitles that are easy to read and understand for the user when the input speech is sequentially converted into text.

  The present invention is suitably applied to a text display system that converts input speech in real time and displays the converted text.

DESCRIPTION OF SYMBOLS 1 Voice input means 2 Voice recognition means 3 Voice recognition dictionary storage means 4 Recognition result conversion means 5 Conversion database storage means 6 Display time determination means 7 Text display means 8 Important word extraction means 9 Important word database storage means 10 Voice sending terminal 20 Speech recognition server 21 Speech detection unit 22 Speech analysis unit 23 Speech collation unit 30 Recognition result display terminal 31 Acoustic model 32 Language model 33 Dictionary 100 Internet

Claims (11)

A recognition result creating means for sequentially recognizing input speech and creating a recognition result converted into text;
A text display time determination device comprising: display time determination means for determining a display time for each sentence included in the recognition result based on the speech utterance time. A post-conversion storage means for storing a predetermined word or word string in association with a post-conversion notation that is a notation obtained by converting the length of the word or word string to a short length;
Recognition result conversion means for converting the notation of a word or word string included in the input speech recognition result into a converted notation corresponding to the word or word string stored in the converted notation storage means,
The text display time determination device according to claim 1, wherein the display time determination means determines a display time for each sentence included in the recognition result converted into the converted notation. The post-conversion notation storage means stores, as post-conversion notation, a notation indicating that a word or word string is deleted,
The text display time determination device according to claim 2, wherein the recognition result conversion means deletes the word or the word string from the recognition result when the post-conversion notation of the word or the word string is a notation indicating deletion. Utterance time length calculating means for calculating, for each word, an utterance time length that is a time when a word included in the input speech recognition result is uttered;
The display time determining means determines the display time of the sentence included in the recognition result converted into the converted notation based on the utterance time length of the word before the word included in the sentence is converted into the converted notation. The text display time determination device according to claim 2 or 3. 5. The text display time according to claim 4, wherein the display time determination means determines a value obtained by multiplying a ratio of the utterance speed of the sentence to be displayed with respect to the average value of the utterance speed by a sum of the utterance time lengths of words as the display time. Decision device. Weight value storage means for storing an utterance time length weight value, which is a weight value multiplied by the utterance time length, for each predetermined word;
The text display time determination device according to claim 4 or 5, further comprising weight value giving means for multiplying the utterance time length of each word included in the input speech recognition result by the corresponding utterance time length weight value. . A voice input device for inputting voice;
A speech recognition device for recognizing speech input to the speech input device;
A recognition result display device for displaying a voice recognition result by the voice recognition device;
The voice recognition device
A recognition result creating means for sequentially recognizing speech input to the speech input device and creating a recognition result that is converted into text;
And a display time determining means for determining a display time for each sentence included in the recognition result based on the speech utterance time. Recognize input speech and create a recognition result that is converted into text,
A text display time determination method, comprising: determining a display time for each sentence included in the recognition result based on the speech utterance time. The notation of the word or word string included in the input speech recognition result is associated with a predetermined word or word string and a converted notation that is a notation obtained by converting the length of the word or word string to a shorter length. Convert to the corresponding converted notation stored in the stored converted notation storage means,
The text display time determination method according to claim 8, wherein the display time is determined for each sentence included in the recognition result converted into the post-conversion notation. On the computer,
A recognition result creation process that sequentially recognizes input speech and creates a text recognition result; and
A text display time determination program for executing display time determination processing for determining a display time for each sentence included in the recognition result based on the speech utterance time. On the computer,
The notation of the word or word string included in the input speech recognition result is associated with a predetermined word or word string and a converted notation that is a notation obtained by converting the length of the word or word string to a shorter length. Executing recognition result conversion processing for converting into a corresponding post-conversion notation stored in the post-conversion post-notation storage means,
The text display time determination program according to claim 10, wherein the display time is determined for each sentence included in the recognition result converted into the post-conversion notation in the display time determination process.

Images