JP2020057401A - Display support device, method and program - Google Patents

Abstract

To provide a display support device capable of supporting comprehension of contents.SOLUTION: A display support device in an embodiment includes an acquisition part, a first processing part, a second processing part and a display control unit. The acquisition part acquires a character string. The first processing part segments the character string using a first unit which is a unit representing a meaningful unit and generates a plurality of segmented character strings. The second processing part detects a second unit which is a unit larger than the first unit with respect to the character string. The display control unit controls the display of the plurality of segmented character strings while differentiating the first unit and the second unit from each other.SELECTED DRAWING: Figure 1

Description

  Embodiments of the present invention relate to a display support device, a method, and a program.

As a scene in which the speech recognition result is displayed as a caption, for example, there is a scene in which a speech recognition result is displayed as a caption for a conference participant who cannot hear the speech in the conference and has difficulty understanding the content for the purpose of guaranteeing information. In addition, as a scene in which a translation result is displayed in a caption, in a conference system used in a conference in which participants in different languages are native speakers, a scene in which a machine translation result in different languages or a translation result by hand is displayed in a caption is displayed. is there.
When displaying subtitles as described above, when the subtitles are switched sequentially as the content progresses, or when the old subtitles that are already displayed are pushed out of the screen little by little by the newly displayed subtitles In addition, the time when the user can see the subtitles is limited. Therefore, in order to clarify the contents of subtitles and to improve the quality of translation, there is a technique for dividing a sentence of a speech recognition result or a translation result displayed as subtitles.

JP 2007-18098 A

  However, in an actual operation, it is difficult to distinguish the subtitle components by simply displaying the sentence of the divided speech recognition result or translation result as it is. Furthermore, since the sentence structure cannot be completely analyzed, the user who refers to the contents of the subtitles cannot catch up with the understanding, which may hinder information transmission.

  The present disclosure has been made to solve the above-described problem, and has as its object to provide a display support device, a method, and a program that can support understanding of contents.

  The display support device according to the present embodiment includes an acquisition unit, a first processing unit, a second processing unit, and an expression control unit. The obtaining unit obtains a character string. The first processing unit divides the character string into first units, which are units representing meanings, to generate a plurality of divided character strings. The second processing unit detects a second unit of the character string that is a unit larger than the first unit. The display control unit performs display control for distinguishing the first unit and the second unit when displaying the plurality of divided character strings.

FIG. 1 is a block diagram illustrating a display support device according to a first embodiment. The figure which shows an example of the table which a 1st process part refers. FIG. 7 is a diagram illustrating an example of a processing result of a first processing unit. The figure which shows an example of the table which a 2nd process part refers. FIG. 9 is a diagram illustrating an example of a processing result of a second processing unit. 5 is a flowchart illustrating the operation of the display control unit according to the first embodiment. FIG. 4 is a diagram illustrating a specific example of display control according to the display control unit according to the first embodiment. The figure which shows the 1st another example of the display control in a display control part. The figure which shows the 2nd another example of the display control in a display control part. FIG. 9 is a block diagram showing a display support device according to a second embodiment. The figure which shows an example of the conversion pattern referred by the expression conversion part. The figure which shows an example of the processing result of the expression conversion part. FIG. 9 is a diagram illustrating a specific example of display control by a display control unit according to the second embodiment. FIG. 10 is a block diagram showing a display support device according to a third embodiment. FIG. 7 is a diagram illustrating an example of a processing result of a translation unit. FIG. 14 is a diagram showing a specific example of display control by a display control unit according to the third embodiment. The figure which shows the example of the machine translation in the case of the Japanese-English translation regarding the 1st example in which the division | segmentation position of the division | segmentation character string of a 1st language changes. The figure which shows the example of the machine translation in the case of the Japanese-Chinese translation regarding the 1st example in which the division | segmentation position of the division | segmentation character string of a 1st language changes. The figure which shows the example of the machine translation in the case of the Japanese-Chinese translation regarding the 2nd example in which the division | segmentation position of the division | segmentation character string of a 1st language changes. The figure which shows the example of the machine translation in the case of the Japanese-English translation about the 2nd example in which the division | segmentation position of the division | segmentation character string of a 1st language changes. The figure which shows the example of another of the processing result of the translation part in the case of Japanese-English translation. The figure which shows another example of the processing result of the translation part in the case of Japanese-Chinese translation. FIG. 14 is a block diagram showing a display support device according to a fourth embodiment. The figure which shows the 1st example of the word order determination rule referred by a word order determination part. The figure which shows an example of the processing result of a word order determination part. FIG. 14 is a diagram showing a specific example of display control by a display control unit according to a fourth embodiment. FIG. 18 is a diagram illustrating an example of a processing result of a first processing unit according to the fifth embodiment. 15 is a flowchart illustrating the operation of the display control unit according to the fifth embodiment. FIG. 18 is a diagram showing a specific example of display control by a display control unit according to a fifth embodiment. 21 is a flowchart showing the operation of the display control unit according to a first modification of the fifth embodiment. FIG. 19 is a diagram showing a specific example of display control by a display control unit according to a first modification of the fifth embodiment. 20 is a flowchart showing the operation of the display control unit according to a second modification of the fifth embodiment. FIG. 19 is a diagram showing a specific example of display control by a display control unit according to a second modification of the fifth embodiment. The figure which shows another example of the process when the expression was converted by the expression conversion part. FIG. 31 is a diagram showing a specific example of display control for the process shown in FIG. 30.

  Hereinafter, a display support device, a method, and a program according to the present embodiment will be described in detail with reference to the drawings. In the following embodiments, portions denoted by the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate.

(First embodiment)
The display support device according to the first embodiment will be described with reference to the block diagram of FIG. The display support device 100 according to the first embodiment includes an acquisition unit 101, a first processing unit 102, a second processing unit 103, and a display control unit 104.

  The obtaining unit 101 obtains an input character string based on an input from a user. Various commonly used methods such as keyboard input, handwritten character recognition, and microphone input for receiving voice can be applied as input from the user.

  When the input from the user is by voice, the obtaining unit 101 obtains a character string of a voice recognition result of the voice as an input character string, and further obtains pause information. The acquisition unit 101 may acquire, as pause information, a state in which a silent period in a voice continues for a predetermined time or more. In addition, if the setting is such that the input of voice is started after the button is pressed at the time of microphone input, the obtaining unit 101 may obtain the pause information by detecting the on / off of the button.

  When the input from the user is a text input such as a keyboard input or a handwritten character recognition process, the acquiring unit 101 acquires the determined information together with the input character string. The acquisition unit 101 may acquire depression of an enter key or input of a period, a period, and the like as final information. Further, if the user interface has a screen display for confirming an input such as an input confirmation button on the user interface, the acquisition unit 101 may acquire, as confirmation information, that the display is touched or clicked with a mouse or the like.

The first processing unit 102 receives an input character string and pose information or confirmation information from the acquisition unit 101. The first processing unit 102 performs a morphological analysis on the input character string, and converts the input character string subjected to the morphological analysis into a first language unit (simply (Also referred to as one unit) to generate a plurality of divided character strings.
For the morphological analysis, any general morphological analysis technique such as the CKY method or the longest match method can be used. If the input character string acquired by the acquisition unit 101 has already been given information on the result of the morphological analysis, the first processing unit 102 does not need to perform the morphological analysis.

  The first processing unit 102 assigns a role label to each divided character string. Role labels include, for example, a label indicating the type of case when using a phrase that is a case element, a label indicating that it is a simple sentence, and a phrase located at the end of a sentence when using a tense verb phrase that is the end of a sentence Label, a conjunction indicating the composition or context of the sentence, an adverb or a label indicating the context when using an equivalent expression, a label indicating that the element is a parallel element when using a parallel element, and a pause. There is a label indicating a pose when used as one first language unit.

  The second processing unit 103 receives from the first processing unit 102 a plurality of divided character strings to which role labels have been assigned. The second processing unit 103 detects a second language unit (also simply referred to as a second unit), which is a language unit larger than the first language unit, from the divided character string. The second processing unit 103 assigns an end label to the divided character string at the end of the second language unit. The end label includes, for example, a label indicating the end of a clause (also referred to as the end of a clause), the end of a sentence (also referred to as the end of a sentence), and a label indicating that the sentence is a multiple sentence. The second processing unit 103 detects the arrangement of labels and poses in the first language unit assigned to the input divided character string, determines the end of a clause, the end of a sentence, and the like, and assigns the end label to the corresponding divided character string. What is necessary is just to give.

  The display control unit 104 receives from the second processing unit 103 a plurality of divided character strings to which role labels (and end labels) are added. The display control unit 104 performs display control for distinguishing the first language unit and the second language unit when displaying a plurality of divided character strings based on the role label and the end label. The display control may be any control as long as the relationship between a plurality of divided character strings can be distinguished and displayed so that the user can easily understand the relationship. For example, when displaying a plurality of divided character strings on a display or the like, an indent (indentation) is displayed to distinguish the first language unit, a font color is changed, or an underline is drawn or italicized. Or display a divided character string at the end of the second language unit, insert blank lines or draw a separator line, or draw a figure such as a so-called speech bubble that surrounds the displayed first language unit. Or to do so. Details of the display control will be described later.

Next, an example of a table referred to by the first processing unit 102 is shown in FIG.
In the table 200 shown in FIG. 2, the first language unit detection pattern 201 and the role label 202 are stored in association with each other.

  The first language unit detection pattern 201 is a pattern for dividing an input character string into first language units. Here, a pattern of a character string that appears at the beginning of a sentence when the input character string is divided as a phrase, and Indicates the pattern of the character string that appears at the end when the input character string is separated as a phrase. The role label 202 indicates a label indicating the feature of the first language unit detection pattern 201, and here is a label indicating a type of case. The first processing unit 102 determines whether a morpheme string that is a character string that is a result of the morphological analysis of the input character string matches the first language unit detection pattern 201. If it is determined that the morpheme string matches the first language unit detection pattern 201, the first processing unit 102 generates a divided character string by using the end of the morpheme string as a delimiter position, and detects the matched first language unit detection. A role label 202 corresponding to the pattern 201 is given.

  Specifically, for example, the first language unit detection pattern 201 “sentence / first / ni” is associated with the role label 202 “[sentence adverb]”. In FIG. 2, the “head of sentence” in the first language unit detection pattern 201 may refer to the position immediately after the pause information or the confirmation information, and the “content word” indicates a character string that appears in the input character string.

  If there is pause information or confirmation information immediately after the morpheme located at the end of the divided character string, that information may be added to the divided character string as a role label.

Next, an example of a processing result of the first processing unit 102 is shown in FIG.
The first processing unit 102 can obtain the processing results of the divided character strings 301 to 310 as shown in FIG. 3 and the role labels 202 assigned to the respective character strings. More specifically, since the divided character string 301 “E / First / First / Ni” matches the first language unit detection pattern 201 “Sentence / First / Ni” by referring to the table 200, the divided character string 301 A corresponding role label 202 “[sentence adverb]” is assigned to the column 301.

Next, an example of a table referred to by the second processing unit 103 is shown in FIG.
In the table 400 shown in FIG. 4, the second language unit detection pattern 401 and the end label 402 are stored in association with each other.

  Here, the second language unit detection pattern 401 indicates a pattern of a character string appearing at the end of a clause and at the end of a sentence, which is a language unit larger than a phrase in terms of grammar. The end label 402 is a label indicating the end of a section or the end of a sentence. Specifically, the second language unit detection pattern 401 “mas / de / mo” is associated with the end label 402 “<< section end >>”.

  The second processing unit 103 determines whether the divided character string matches the second language unit detection pattern 401. If it is determined that the divided character string matches the second language unit detection pattern 401, the second processing unit 103 assigns the tail label 402 corresponding to the matched second language unit detection pattern to the divided character string. .

Next, an example of a processing result of the second processing unit 103 will be described with reference to FIG.
The second processing unit 103 can obtain the processing results of the divided character strings 501 to 510, the role label 202 assigned to each character string, and the end label 402 as shown in FIG.

  More specifically, since the divided character string 503 “... / mas / de / mo” matches the second language unit detection pattern 401 “mas / de / mo” by referring to the table 400, The corresponding end label 402 “<< section end >>” is assigned to the divided character string 503. Similarly, since the divided character string 510 “de / I / translation / is” matches the second language unit detection pattern 401 “de / i / translation / is” by referring to the table 400, the divided character string 510 The column 510 is provided with a corresponding end label 402 “<< sentence end >>”.

  Next, the operation of the display control unit 104 will be described with reference to the flowchart in FIG. Note that “display” in the description of FIG. 6 means that the display control unit 104 controls the display device (not shown) such as an external display to display a character string or a graphic. Or display as a telop. It is also assumed that the display control unit 104 holds a variable related to the number of indents when displaying a divided character string.

In step S601, the display control unit 104 sets the number of indents to zero.
In step S602, the display control unit 104 displays the first dividing line in the display.
In step S603, the display control unit 104 performs the set indentation and displays the divided character string. Since the number of indents is zero in the first processing, the display control unit 104 may display the divided character string from the beginning of the line.

In step S604, the display control unit 104 determines whether or not the divided character string displayed in step S603 is the end of a sentence, that is, whether or not an end-of-sentence label is given to the divided character string. If the end-of-sentence label has been assigned, the process proceeds to step S608. If the end-of-sentence label has not been assigned, the process proceeds to step S605.
In step S605, the display control unit 104 determines whether the divided character string displayed in step S603 is the end of a section, that is, whether or not a section end label is added. If the end-of-section label has been assigned, the process proceeds to step S609. If the end-of-section label has not been assigned, the process proceeds to step S606.

In step S606, the display control unit 104 increments the number of indents being set by one.
In step S607, the display control unit 104 determines whether there is a next divided character string. If there is a next divided character string, the process returns to step S603, and the same processing is repeated. If there is no next divided character string, the process proceeds to step S610.
In step S608, since the end of the sentence can be known from the sentence end label, the display control unit 104 displays a dividing line and sets the number of indents to zero. Thereafter, the flow returns to step S603, and the same processing is repeated.

In step S609, since the end of the section is known from the section end label, the display control unit 104 displays a blank line and sets the number of indents to zero. Thereafter, the flow returns to step S603, and the same processing is repeated.
In step S610, the display control unit 104 displays a dividing line. Thus, the operation of the display control unit 104 ends.

A specific example of display control performed by the display control unit 104 according to the first embodiment will be described with reference to FIG.
Here, an example is shown in which display control is performed on the processing result of the second processing unit 103 shown in FIG. 5 and displayed as subtitles on a display or a screen. Since the divided character string 501 is the first divided character string and the number of indents is zero, the divided character string 501 is displayed after the initial delimiter line 700 is displayed (display 701). Since the divided character string 501 is neither a sentence end label nor a section end label, the number of indents is incremented by one in step S606, and the number of indents becomes one.

  As for the next divided character string 502, the number of indents is 1, so the display start position of the divided character string 502 is moved to the right by one interval (referred to as one indent) defined as the indent and displayed. (Display 702). Note that the example of FIG. 7 is horizontally written Japanese, so indentation is performed on the right side. However, in the case of vertical writing, indentation may be performed in the downward direction. In the case of a language written from right to left such as Arabic, Just indent to the left.

  Similarly to the divided character string 501, the divided character string 502 is neither a sentence end label nor a section end label, so the number of indents is incremented by one, and the number of indents becomes two.

Since the number of indents for the next divided character string 503 is 2, the divided character string 503 is displayed from the position moved to the right by two indents (display 703).
Here, since the end-of-section label is assigned to the divided character string 503, a blank line 704 is displayed after the display 703 by the processing in step S609, and the number of indents is reset to zero.

  When the same processing is performed on the divided character strings 504 to 509, a display 705 to a display 708, a blank line 709, and a display 710 to a display 712 are displayed.

  When processing is performed on the last divided character string 510, since the divided character string 510 is provided with the end-of-sentence label, the display 712 is displayed, and then the end separator line 713 is displayed by the processing in step S610. .

  After displaying the divided character string to which the end-of-section label is added, the character string may be separated not by a blank line but by a dividing line that can be distinguished from the dividing line at the beginning and at the end of the sentence. In other words, any display format can be used to distinguish between the end of a section and the end of a sentence.

Next, another example of display control in the display control unit 104 will be described with reference to FIGS. 8A and 8B.
Here, for convenience of explanation, it is assumed that the divided character strings 501 to 510 shown in FIG. 5 form one phrase.

  If the character string up to the end of one section is long, the number of indents may increase when the processing is performed according to the flowchart shown in FIG. 7 and may not fit on a display surface such as a display. In such a case, for example, as shown in FIG. 8A as a first alternative, when the number of indents reaches a threshold, the indentation may be displayed without changing the number of indents. For example, if the threshold value of the number of indents is 5, the subsequent divided character string may be displayed without increasing the number of indents to 6 or more and keeping the number of indents at 5. Further, for the divided character strings displayed with the same number of indents, a process of further increasing the visibility of the user, such as reducing the font size or changing the font color, may be performed.

  As a second alternative, as shown in FIG. 8B, when the number of indents reaches the threshold, the number of indents may be reset to zero, and the display may be repeated until the number of indents reaches the threshold again.

  According to the first embodiment described above, when a divided character string is displayed, display control for distinguishing the first language unit and the second language unit is performed, for example, a phrase unit, a clause unit, or a sentence unit. Different units can be displayed so that the difference can be understood. Thereby, the visibility of a character string displayed as a caption or a telop can be improved, and it is possible to assist the user in understanding the content.

(Second embodiment)
A display support device according to the second embodiment will be described with reference to the block diagram of FIG. The display support device 900 according to the second embodiment includes an acquisition unit 101, a first processing unit 102, a second processing unit 103, a display control unit 104, and an expression conversion unit 901.
The operations of the acquisition unit 101, the first processing unit 102, and the second processing unit 103 are the same as those in the first embodiment, and a description thereof will not be repeated.

The expression conversion unit 901 receives the divided character string to which the role label (and the tail label) is added from the second processing unit 103, and converts the expression of the divided character string corresponding to the conversion rule into another expression based on the conversion rule. Convert. Note that the expression conversion unit 901 receives not the divided character string processed by the second processing unit 103 but the character string subjected to morphological analysis from the first processing unit 102, and performs input morphological analysis based on the conversion rule. The column representation may be converted. At this time, the first processing unit 102 may send the converted input character string to the second processing unit 103.
The display control unit 104 performs display control on the converted divided character string.

Next, an example of a conversion pattern referred to by the expression conversion unit 901 will be described with reference to FIG.
In a table 1000 shown in FIG. 10, a conversion target pattern 1001 and a conversion pattern 1002 are stored in association with each other. The conversion target pattern 1001 indicates a character string pattern to be converted. The conversion pattern 1002 is a pattern indicating how to convert the conversion target pattern 1001. In the example of FIG. 10, a pattern for removing stagnation, rephrasing, typo, filler, and the like is used as the conversion pattern 1002.

  Specifically, the conversion target pattern 1001 “er” and the conversion pattern 1002 “(blank)” are associated with each other. That is, if the filler is “er”, “er” is deleted.

  In the example of FIG. 10, it is assumed that a process of so-called sentence removing fillers and the like is performed. However, the present invention is not limited to this, and conversion from spoken language to written language or vice versa, conversion from dialect to standard language, or Various expressions may be converted, such as a reverse conversion, a conversion from one local dialect to another local dialect, and a conversion from a standard language to a so-called character language spoken by a character.

Next, an example of the processing result of the expression conversion unit 901 is shown in FIG.
FIG. 11 is an example in which the expression of the divided character string is converted by the expression conversion unit 901 with respect to the processing result of the second processing unit 103 shown in FIG.

For example, regarding the divided character string 1101, in the example of FIG. 5, the divided character string 501 is “E / First / First / First”, but the filler is removed by the processing of the expression conversion unit 901 to be “First / First”. Is converted.
Also, regarding the divided character string 1107, in the example of FIG. 5, the divided character string 507 is "ma / this / 2 / one / no / method / use / use / use / mao / mashi / te", and the section end label Is given, but by the processing of the expression conversion unit 901, it is converted into “2 / two / method / use / use / use / is / masu”, and the end-of-section label is converted to the end-of-sentence label.

  Next, a specific example of display control by the display control unit 104 according to the second embodiment is shown in FIG. As shown in FIG. 12, a display is performed in which redundant words such as fillers displayed in FIG. 7 have been deleted. Further, a dividing line 1209 is displayed in a line below the display 1208 of the divided character string converted from the section end label to the sentence end label. As a result, the display becomes easier for the user to understand the contents of the caption.

  According to the second embodiment described above, the expression conversion unit converts the expression of the divided character string in order to perform the sentence processing, so that the subtitle display that is more readable for the user can be performed. Can assist in understanding the content. In addition, when converting an expression into another expression such as a dialect, it is possible to widen variations in subtitle display.

(Third embodiment)
A display support device according to the third embodiment will be described with reference to the block diagram of FIG. The display support device 1300 according to the third embodiment includes an acquisition unit 101, a first processing unit 102, a second processing unit 103, a display control unit 104, an expression conversion unit 901 and a translation unit 1301. The operations of the acquisition unit 101, the first processing unit 102, the second processing unit 103, and the expression conversion unit 901 are the same as those in the second embodiment, and a description thereof will be omitted.

  The translation unit 1301 receives a plurality of divided character strings converted as necessary from the expression conversion unit 901 and performs machine translation of the plurality of divided character strings from the first language to the second language. The machine translation technique can use any commonly used translation engine, such as a rule-based machine translation engine, an example-based machine translation engine, or a statistics-based machine translation engine. The display control unit 104 performs display control on a plurality of machine-translated divided character strings.

Next, an example of the processing result of the translation unit 1301 will be described with reference to FIG.
FIG. 14 shows an example in which the machine translation process is performed on the divided character strings 1101 to 1110 whose expressions have been converted by the expression conversion unit 901 shown in FIG. A case of translating from Japanese to a second language English (referred to as Japanese-English translation) is shown. As shown in the divided character strings 1401 to 1410, translation can be performed for each divided character string. Note that the label that has already been given is kept as it is.

  A specific example of display control by the display control unit 104 according to the third embodiment will be described with reference to FIG. By displaying the translated content as subtitles as shown in FIG. 15, it is easy for a user who can understand the second language to understand the content.

  When translating from the first language to the second language, it may be desirable to change the length of the divided character string as a translation unit depending on the type of the second language. For example, in Japanese-English translation and when translating from Japanese to Chinese (referred to as Japanese-Chinese translation), the translation unit is set according to each language. It is desirable to change the character string separation position).

Here, a first example in which a delimiter position of a divided character string in the first language changes due to a difference in grammar according to the type of the second language will be described with reference to FIGS. 16A and 16B.
FIG. 16A shows an example of machine translation for Japanese-English translation. FIG. 16B shows an example of machine translation in the case of daytime translation.

  In the case of the Japanese-English translation shown in FIG. 16A, in Japanese, the character string is divided into a divided character string 1601 “and then finally” and a divided character string 1602 “in the context of a conference voice utilization system”. In the case of medium translation, in Japanese, the divided character string 1603 is combined into one, "and finally the system for utilizing the conference voice."

  Conversely, in the case of the Japanese-English translation shown in FIG. 16A, the divisional character string 1604 "I tried to start the trial by actually using the place where the voice is transcribed or translated, etc. "And one divided string in the Japanese-Chinese translation of FIG. 16B, the divided string 1605" to transcribe or translate voice "and the divided string 1606" And a part of the divided character string 1607 "I'm trying to start a trial ...".

  In determining the translation unit as described above, the translating unit 1301 receives the information on the type of the second language (also referred to as target language information), and determines the translation unit in the second language unit based on a preset translation unit rule of the language. Join or split the split string of The target language information may be obtained from the user by designation. If the second language is determined in advance, the first processing unit 102 may generate the divided character string by referring to the translation unit rule at the stage of generating the divided character string.

Next, a second example in which the delimiter positions of the divided character strings in the first language are changed due to differences in the grammar of the second language will be described with reference to FIGS. 17A and 17B.
FIG. 17A shows an example of machine translation in the case of day-to-day translation. FIG. 17B shows a machine translation example in the case of Japanese-English translation.

  In general, in the case of Japanese-Chinese translation, the number of characters in Chinese is often smaller than that in Japanese, so that Chinese corresponding to the Japanese content can be displayed with subtitles. On the other hand, in the case of Japanese-English translation, the number of characters may be larger than in Japanese. Therefore, in order to prevent characters from protruding from the caption display space, non-essential contents may not be translated.

  In the day-to-day translation shown in FIG. 17A, the divided character string 1701 corresponds to “I wish you good”.

However, in the Japanese-English translation shown in FIG. 17B, since there is no translation indicating the corresponding content in order to reduce the number of characters, the translation of the divided character string 1701 “I wish you good” is omitted.
The translation unit 1301 may determine a keyword that is not translated into a translated sentence in advance as a translation unit rule, and perform a translation process as shown in FIGS. 17A and 17B.

  Another example of the process of the translation unit 1301 will be described with reference to FIGS. 18A and 18B. FIG. 18A shows an example of machine translation in the case of Japanese-English translation. FIG. 18B shows an example of machine translation for day-to-day translation.

Depending on the type of the second language, there is a case where the translation corresponding to a certain one of the divided character strings is more natural when the translation result is reflected over the translated sentences of the two divided character strings. For example, as shown in FIG. 18A, the translation unit 1301 translates the Japanese divided character string 1801 “We are doing speech recognition” into English “as we are working on the speech recognition”. Here, "as" corresponding to "so" may be translated as a translation of the corresponding divided character string.
On the other hand, as shown in FIG. 18B, when translating “no” of the Japanese divided character string 1801 into Chinese,

Is translated so as to appear in the translation of the two divided character strings. The translation unit 1301 may determine a keyword to appear in the translation of the two divided character strings in advance as a translation unit rule, and perform the translation processing as shown in FIGS. 18A and 18B.

  According to the third embodiment described above, by displaying the content translated from the first language to the second language, it becomes easy for a user who can understand the second language to understand the content, and the user can change the content. Can help you understand.

(Fourth embodiment)
A display support device according to the fourth embodiment will be described with reference to the block diagram of FIG. The display support device 1900 according to the fourth embodiment includes an acquisition unit 101, a first processing unit 102, a second processing unit 103, a display control unit 104, an expression conversion unit 901, a translation unit 1301, and a word order determination unit 1901.
The operations of the acquiring unit 101, the first processing unit 102, the second processing unit 103, the expression converting unit 901 and the translating unit 1301 are the same as those in the third embodiment, and the description is omitted here.

The word order determination unit 1901 receives the translated character string from the translation unit 1301, and determines the display order of the plurality of divided character strings based on the word order determination rule of the second language. That is, the plurality of divided character strings are rearranged in a natural order along the grammatical order of the second language. In addition, the word order determination unit 1901 reassigns a tail label as necessary. Note that the word order determination unit 1901 may rearrange a plurality of divided character strings if the order is unnatural such as inversion at the stage of the first language.
The display control unit 104 performs display control on the rearranged divided character strings.

Next, a first example of a word order determination rule referred to by the word order determination unit 1901 will be described with reference to FIG.
The word order determination table 2000 shown in FIG. 20 stores a word order pattern 2001 of the first language and a word order pattern 2002 of the second language in association with each other. The example of FIG. 20 shows an example in which the first language is Japanese and the second language is English. Here, the order of the role labels given to the divided character strings is determined by the order of the role labels such that the order becomes natural as the English word order.

  Specifically, for example, the Japanese word order pattern 2001 is “[sentence adverb] → [object] → [predicate]”, and the English word order pattern 2002 is “[sentence adverb] → [predicate] → [object]”. Are associated with each other.

Next, an example of the processing result of the word order determination unit 1901 will be described with reference to FIG.
In the example of FIG. 14 before the word order is rearranged, the order of the divided character strings 1401 to 1403 in the second language unit is “first [sentence adverb]”, “about machine translation [object]”, "We will introduce [predicate] << end of the end >>".

  The word order determination unit 1901 rearranges the divided character strings 1401 to 1403 in the order of [sentence adverb] → [predicate] → [object] based on the word order determination rule. Further, the word order determination unit 1901 reassigns a tail label to the last divided character string of the second language unit. As a result, the divided character strings 2101 to 2103 are arranged in the order of “first [sentence adverb]”, “we will introduce [predicate]”, “about machine translation [object] << end-of-section >>”. Note that the rearrangement of the end label may be performed by the second processing unit 103.

Next, a specific example of display control by the display control unit 104 according to the fourth embodiment will be described with reference to FIG.
After the word order is changed as shown in FIG. 22, the display order of the translated sentences becomes the word order of [first], [we will introduce], and [about machine translation]. Is displayed.

  According to the fourth embodiment described above, the user who uses the second language sorts the plurality of divided character strings into a natural order along the grammatical order of the second language by the word order determination unit. Can display more natural subtitles, and can assist the user in understanding the contents.

(Fifth embodiment)
In the fifth embodiment, it is assumed that there are a plurality of speakers in the first language.
The display support device according to the fifth embodiment is realized by using the display support device described in any one of the above embodiments.

  The acquisition unit 101 according to the fifth embodiment acquires unique speaker information for each speaker for identifying the speaker who has input (uttered) the input character string, in addition to the input character string. The method of acquiring the speaker information may be, for example, by preparing a microphone connected to the acquisition unit 101 for each speaker, or may be a general speaker identification using beamforming or voice feature. Techniques may be used to identify speakers and acquire speaker information.

  In addition, the first processing unit 102 receives the input character string and the speaker information from the acquisition unit 101, and performs a speech based on the speaker information for a plurality of divided character strings obtained in the same manner as in the above-described embodiment. A speaker label for classifying each speaker is provided.

  The display control unit 104 receives a plurality of divided character strings to which a speaker label has been assigned from the second processing unit 103 and, when displaying the plurality of divided character strings, distinguishes each speaker based on the speaker label. While performing the display control, the first language unit and the second language unit are distinguished.

FIG. 23 illustrates an example of a processing result of the first processing unit 102 according to the fifth embodiment.
As shown in FIG. 23, a speaker label 2300 is provided in addition to the role label 202 and the end label 402.
The method of determining a speaker label is, for example, based on the speaker information acquired by the acquisition unit 101, when the identification information of the user (such as the IP address or the identification information of the user including the user ID) is obtained in advance. The information may be identified according to the information, or a label such as speaker A or speaker B may be given to distinguish different speaker information.

Next, the operation of the display control unit 104 according to the fifth embodiment will be described with reference to the flowchart in FIG.
The processing of step S601, steps S603 to S607, and step S609 is the same as that of the above-described embodiment, and thus the description is omitted.

In step S2401, the display control unit 104 prepares a new balloon.
In step S2402, the display control unit 104 ends the current speech balloon and sets the number of indents to zero since the end of sentence label is assigned to the divided character string. The process returns to step S2401, and the same processing is repeated. Thus, one sentence is represented by one balloon.
In step S2403, since the processing has been completed for all the divided character strings, the balloon ends. The process returns to step S2401, and the same processing is repeated.

  Note that a process for clarifying the distinction between speakers may be performed, such as changing the frame line of the speech balloon and the color of the frame line for each speaker. Further, in the fifth embodiment, speech balloons are assumed for speaker identification, but any display may be used as long as it can be identified for each speaker.

Next, a specific example of display control by the display control unit 104 according to the fifth embodiment is shown in FIG.
FIG. 25 is an example in which display control is performed on the processing result of the second processing unit 103 shown in FIG. A balloon is displayed for each speaker and further for each sentence, such as a balloon 2501 for the utterance of the speaker A, a balloon 2502 for the utterance of the speaker B, and a balloon 2503 for the utterance of the speaker C. Can be improved.

(First Modification of Fifth Embodiment)
The operation of the display control unit 104 according to the first modification of the fifth embodiment will be described with reference to the flowchart in FIG.
The processing of step S604, step S605, step S607, step S2401, and step S2403 is the same as that of the above-described embodiment, and thus the description is omitted.

In step S2601, the display control unit 104 displays the content of the divided character string in a box smaller than the balloon and within a size that fits in the balloon.
In step S2602, the display control unit 104 ends the current balloon, returns to step S2401, and repeats the same processing.
In step S2603, the display control unit 104 displays a blank line, returns to step S2601, and repeats the same processing.

Next, a specific example of display control by the display control unit 104 according to a first modification of the fifth embodiment is shown in FIG.
As shown in FIG. 27, the divided character strings are displayed in a box 2701 in the balloon, so that the visibility of the subtitles can be improved.

(Second Modification of Fifth Embodiment)
A second modification of the fifth embodiment will be described with reference to the flowchart in FIG. Steps S607, S2401, S2403, and S2603 from step S603 to step S605 are the same as those in the above-described embodiment, and thus description thereof is omitted.

In step S2801, the display control unit 104 displays a period at the end of the divided character string when the end-of-section label or the end-of-sentence label is not assigned.
In step S2802, the display control unit 104 displays a reading point at the end of the divided character string, ends the current balloon, and returns to step S2401 to repeat the same processing.

Next, FIG. 29 shows a display example according to a second modification of the fifth embodiment.
Also in FIG. 29, similar to FIG. 27, the divided character strings are displayed with punctuation marks in the balloon, so that the visibility of the subtitles can be improved.

Next, another example of the display control when the expression is converted by the expression conversion unit 901 is shown in FIG.
FIG. 30 shows a case where the divided character string 2309 in FIG. 23 is converted by the expression conversion unit 901, and the attached label is changed from a section end label to a sentence end label. The display control unit 104 changes the color of the character string of the divided character string 3001.

  Next, another example of the display when the expression is converted by the expression conversion unit 901 is shown in FIG. As described above, by changing the color of the converted portion, the user can easily grasp the changed portion. The color is not limited to being changed, and may be underlined or italicized.

  According to the fifth embodiment described above, in response to input from a plurality of users, speech is separated for each speaker using a speech balloon, and further, a divided character string in the speech balloon is the same as in the above-described embodiment. Performing the appropriate display control makes it easy to distinguish between speakers, and can assist the user in understanding the contents.

The instructions shown in the processing procedure shown in the above-described embodiment can be executed based on a program that is software. A general-purpose computer system stores this program in advance, and by reading this program, it is possible to obtain the same effect as the above-described effect of the display support device. The instructions described in the above-described embodiments are, as programs that can be executed by a computer, magnetic disks (flexible disks, hard disks, etc.) and optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD ± R, DVD ± RW, Blu-ray (registered trademark) Disc, etc.), a semiconductor memory, or a recording medium similar thereto. As long as the recording medium is readable by a computer or an embedded system, its storage format may be any. If the computer reads the program from the recording medium and causes the CPU to execute the instructions described in the program based on the program, the same operation as the display support device of the above-described embodiment can be realized. Of course, when the computer acquires or reads the program, the program may be acquired or read through a network.
Also, an OS (Operating System) running on the computer, database management software, MW (Middleware) such as a network, etc., based on instructions of a program installed in the computer or the embedded system from the recording medium realizes the present embodiment. May be executed.
Further, the recording medium in the present embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium in which a program transmitted via a LAN, the Internet, or the like is downloaded and stored or temporarily stored.
Further, the number of recording media is not limited to one, and a case where the processing in the present embodiment is executed from a plurality of media is also included in the recording medium in the present embodiment, and the configuration of the medium may be any configuration.

Note that the computer or the embedded system according to the present embodiment is for executing each process according to the present embodiment based on a program stored in a recording medium, and includes an apparatus including one such as a personal computer and a microcomputer. This device may have any configuration such as a system in which the devices are connected to a network.
Further, the computer in the present embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, and the like included in an information processing device, and is a general term for devices and devices that can realize the functions in the present embodiment by programs. ing.

  Although several embodiments of the present invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

  100, 900, 1300, 1900: display support device, 101: acquisition unit, 102: first processing unit, 103: second processing unit, 104: display control unit, 200, 400 , 1000, 2000 ... table, 201 ... first language unit detection pattern, 202 ... role label, 301-310, 501-510, 1101-1110, 1401-1410, 1601-1607, 1701, 1801 , 2101 to 2110, 2301 to 2312, 3001 ... divided character string, 401 ... second language unit detection pattern, 402 ... tail label, 701 to 703, 705 to 708, 710 to 712, 1201 to 1203 , 1205-1208, 1210-1212, 1501-1503, 1505-1508, 1510-1512, 201 to 2203, 2205 to 2208, 2210 to 2212 ... display, 704, 709, 1204, 1504, 2204 ... blank line, 700, 713, 1200, 1209, 1213, 1500, 1509, 1513, 2200, 2209 , 2213 ... Separation line, 901 ... Expression conversion unit, 1001 ... Conversion target pattern, 1002 ... Conversion pattern, 1301 ... Translation unit, 1802, 1803 ... Translation sentence, 1901 ... Word order determination unit, 2001: Word order pattern of the first language, 2002: Word order pattern of the second language, 2300: Speaker label, 2501, 502, 2503 ... Balloon, 2701 ... Box .

Claims (11)

An acquisition unit for acquiring a character string;
A first processing unit that divides the character string by a first unit that is a unit representing a unit of meaning and generates a plurality of divided character strings;
A first assigning unit that assigns a role label to each of the first character strings of the plurality of divided character strings;
A second processing unit that detects a second unit that is a unit larger than the first unit for the character string;
A second assigning unit that assigns an end label to a second character string at the end of the second unit of the plurality of divided character strings;
When displaying the plurality of divided character strings, the first unit and the second unit are distinguished based on whether the end label is assigned to each divided character string and the feature indicated by the role label. A display control unit that performs display control,
The role label represents a role of the first character string, and the role includes pause information used as one of the first units and determination information used as one of the first units. , Including
The display support device, wherein the second processing unit detects an arrangement of the role label and the pause information or the determination information, and determines one of a clause end and a sentence end. A conversion unit configured to convert the expression of the divided character string into another expression based on a conversion rule;
The display support device according to claim 1, wherein the display control unit performs display control on the converted divided character string.   When displaying the plurality of divided character strings, the display control unit performs a process of indenting and displaying a second divided character string following the first divided character string, and the second divided character string is the second divided character string. The display support device according to claim 1, wherein when the unit is located at the end of the unit, a process of displaying a blank line after displaying the second divided character string is performed. A translation unit configured to machine translate the plurality of divided character strings from a first language to a second language;
The display support device according to any one of claims 1 to 3, wherein the display control unit performs display control on the machine-translated divided character string. A determining unit configured to determine a display order of the plurality of divided character strings based on a word order determining rule of the second language;
The display support device according to claim 4, wherein the display control unit performs display control on the divided character string in which the display order is determined.   The display support device according to claim 1, wherein the character string is a character string obtained as a result of speech recognition.   The display support device according to any one of claims 1 to 6, wherein the first unit is a phrase, and the second unit is a clause or a sentence.   The display support device according to any one of claims 1 to 6, wherein the first unit is a simple sentence, and the second unit is a compound sentence. The acquisition unit further acquires speaker information for each speaker,
The display support device according to any one of claims 1 to 8, wherein the display control unit performs display control separately for each speaker. Get a string,
Dividing the character string by a first unit which is a unit representing a unit of meaning, generating a plurality of divided character strings,
Assigning a role label to each of the first character strings of the plurality of divided character strings,
Detecting a second unit, which is a unit larger than the first unit, for the character string;
An end label is assigned to a second character string at the end of the second unit of the plurality of divided character strings,
When displaying the plurality of divided character strings, the first unit and the second unit are distinguished based on whether the end label is assigned to each divided character string and the feature indicated by the role label. Perform display control,
The role label represents a role of the first character string, and the role includes pause information used as one of the first units and determination information used as one of the first units. , Including
Detecting the second unit is a display support method for detecting an arrangement of the role label and the pause information or the determination information and determining one of a clause end and a sentence end. Computer
Acquisition means for acquiring a character string;
First processing means for dividing the character string by a first unit which is a unit representing a unit of meaning and generating a plurality of divided character strings;
First assigning means for assigning a role label to each of the first character strings of the plurality of divided character strings;
A second processing unit that detects a second unit of the character string that is larger than the first unit;
Second assigning means for assigning an end label to a second character string at the end of the second unit of the plurality of divided character strings,
When displaying the plurality of divided character strings, the first unit and the second unit are distinguished based on whether the end label is assigned to each divided character string and the feature indicated by the role label. A program for functioning as display control means for performing display control,
The role label represents a role of the first character string, and the role includes pause information used as one of the first units and determination information used as one of the first units. , Including
The second processing means is a display support program for detecting an arrangement of the role label and the pause information or the determination information, and determining one of a section end and a sentence end.