JP2020074020A - Summary generation device, summary generating method and summary generation program - Google Patents

Abstract

To reduce a preparation work.SOLUTION: A summary generation device includes an identity extraction unit, a segment candidate generation unit, and a structuring estimation unit. The identity extraction unit extracts identity information of a word included in text information. The segment candidate generation unit generates a segment candidate which is a constitutional unit for a display based on the extracted identity information. The structuring estimation unit estimates structural information including global structural information to local structural information based on the generated segment candidate and an estimation model for structuring.SELECTED DRAWING: Figure 3

Description

  Embodiments of the present invention relate to a summary generation device, a summary generation method, and a summary generation program.

  2. Description of the Related Art Conventionally, with the improvement in accuracy of voice recognition technology, a system utilizing the voice recognition technology for creating a memorandum at a meeting or the like has been proposed. In such a situation, there is a technology that supports the preparation of minutes of a meeting or the like, which is time-consuming for manual preparation. For example, there is a technique for preparing a minutes template and analyzing it, and then creating a minutes according to the minutes template.

JP, 2015-138103, A

  However, the conventional technique has a problem that preparation for preparing minutes is complicated. Specifically, in the related art, since the template of the minutes prepared in advance is used, the preparation in advance for creating the minutes is complicated.

  The problem to be solved by the present invention is to provide a summary generation device, a summary generation method, and a summary generation program that can reduce the preparation work.

  The summary generation device according to the embodiment includes a feature extraction unit, a segment candidate generation unit, and a structured estimation unit. The feature extraction unit extracts feature information of a word included in the text information. The segment candidate generation unit generates a segment candidate that is a constituent unit for display based on the extracted feature information. The structuring estimation unit estimates structural information including global structural information to local structural information based on the generated segment candidates and the structuring estimation model.

FIG. 1 is a diagram showing a system configuration of a summary generation system according to an embodiment. FIG. 3 is a block diagram showing a hardware configuration of the summary generation device according to the embodiment. FIG. 3 is a block diagram showing a functional configuration of a summary generation device according to the embodiment. The figure which shows the voice recognition process result which concerns on embodiment. The figure which shows the morphological analysis processing result which concerns on embodiment. The figure which shows the command expression which concerns on embodiment. The figure which shows the property information of a part of speech and semantic class which concern on embodiment. The figure which shows the information derived from the system which concerns on embodiment. The figure which shows the result of the segmentation which concerns on embodiment. The figure explaining the extraction of the candidate of the structured estimation which concerns on embodiment. FIG. 6 is a diagram illustrating display format candidates according to the embodiment. 6 is a flowchart showing a flow of processing by the feature extraction unit according to the embodiment. 6 is a flowchart showing the flow of processing by a segment candidate generation unit according to the embodiment. 6 is a flowchart showing a flow of processing by a structured estimation unit according to the embodiment. 6 is a flowchart showing a flow of processing by the display format conversion unit according to the embodiment.

(Embodiment)
FIG. 1 is a diagram illustrating a system configuration example of a summary generation system 1 according to the embodiment. As shown in FIG. 1, the summary generation system 1 includes a summary generation device 100 and a terminal device 200. In the summary generation system 1, each device can communicate wirelessly or by wire. Further, the summary generation system 1 may include a plurality of terminal devices 200. The summary generation system 1 creates a summary document visualized as text with a format structure from voice data in a meeting or the like.

  In the configuration described above, the terminal device 200 acquires voice data in a conference or the like, and transmits the acquired voice data to the summary generation device 100 via the network. The voice data is acquired from a microphone or the like connected to the terminal device 200. In the conference, one microphone may be used or a plurality of microphones may be used. Since a conference may be held through different bases, the summary generation system 1 may include a plurality of terminal devices 200. The terminal device 200 is an information device such as a PC (Personal Computer) or a tablet terminal.

  The summary generation device 100 acquires voice data from the terminal device 200, detects an explicit summary instruction of the speaker or an expression of the structured request included in the utterance, and estimates an appropriate display unit (segment). Then, the summary generation device 100 rearranges each segment according to the content thereof in accordance with a termination instruction or the like from the speaker, converts the segments into various display formats, and outputs the display formats. The summary generation device 100 is an information processing device such as a server device.

  FIG. 2 is a block diagram showing a hardware configuration example of the summary generation device 100 according to the embodiment. As shown in FIG. 2, the summary generation device 100 includes a CPU (Central Processing Unit) 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, and a communication unit 15. The above units are connected to each other by a bus 11.

  The CPU 12 controls the overall operation of the summary generation device 100. The CPU 12 controls the overall operation of the summary generation device 100 by executing a program stored in the ROM 13 or the like using the RAM 14 or the like as a work area. The RAM 14 temporarily stores information about various processes and the like, and is used as a work area when a program stored in the ROM 13 or the like is executed. The ROM 13 stores a program for realizing the processing by the summary generation device 100. The communication unit 15 communicates with an external device such as the terminal device 200 via a wireless or wired network. The hardware configuration shown in FIG. 2 is an example, and in addition to the above, a display unit for outputting the processing result, an operation unit for inputting various kinds of information, and the like may be included.

  FIG. 3 is a block diagram illustrating a functional configuration example of the summary generation device 100 according to the embodiment. As shown in FIG. 3, the summary generation device 100 includes a voice recognition unit 110, a feature extraction unit 120, a segment candidate generation unit 130, a structured estimation unit 140, an instruction unit 160, and a display format conversion unit 170. Have. Some or all of the above-mentioned units may be realized by software (program) or may be realized by a hardware circuit. The summary generation device 100 also stores the structure estimation model 150 in the ROM 13 or the like.

  The voice recognition unit 110 performs voice recognition processing on voice data. More specifically, the voice recognition unit 110 receives an input of voice data transmitted from the terminal device 200. Then, the voice recognition unit 110 executes a voice recognition process, and generates text information including character data of each utterance and information of time when the utterance was made. FIG. 4 is a diagram showing an example of a voice recognition processing result according to the embodiment. As shown in FIG. 4, the text information that is the voice recognition result includes character data of each utterance and information about the time when the utterance was made.

  Further, the voice recognition unit 110 identifies a speech section, a silent section, or the like as the acoustic feature of the voice data, and detects the time length of these sections. The voice recognition unit 110 does not have to be included in the summary generation device 100, and the feature extraction unit 120 in the subsequent stage performs processing based on the execution result of the voice recognition process / acoustic feature extraction process. Also good.

  The feature extraction unit 120 extracts feature information included in the text information. More specifically, the feature extraction unit 120 applies the morphological analysis process to the text information generated by the voice recognition unit 110. FIG. 5 is a diagram illustrating an example of a morphological analysis processing result according to the embodiment. As shown in FIG. 5, the morphological analysis processing divides the language into the smallest units of meaningful language. Then, the feature extraction unit 120 identifies the part-of-speech information included in the text information and performs the semantic class analysis process. If it is a noun, more detailed information based on the property information (for example, information such as a person's name and date) is extracted as the feature information. For example, in the semantic class analysis process, keywords such as presence / absence of a quantity expression or unit expression, a person name, an organization name, an event name, and a technical term are extracted as feature information. Subsequently, the feature extraction unit 120 determines a logical element for the text information. For example, in the determination of the logical element, it is determined whether or not the ordered bulleted expression, the command expression for structuring, and the like are included in the text information. If the text information includes logical elements, the feature extraction unit 120 adds metadata to those logical elements.

  FIG. 6 is a diagram showing an example of a command expression according to the embodiment. As shown in FIG. 6, the command expressions are “conclusion”, “TODO (todo)”, “terminate”, “terminate”, and the like. FIG. 7 is a diagram illustrating an example of part-of-speech property information and a semantic class according to the embodiment. As shown in FIG. 7, the part-of-speech property information and the semantic class are “DATE” for “10 days” and “PERSON” for “West Exit”.

  After that, the feature extraction unit 120 determines the segment label for the text information. A segment label is a name that represents the role of a segment (display unit), the property information of the semantic class or noun extracted in the previous stage, or the text of an utterance that does not have such an element, the command of structuring instruction. It is metadata given according to whether or not (instruction command) is included. For example, the structuring instruction command indicates an instruction command that suggests the start of structuring, and corresponds to "bullet start", "here to table", "here to table format", and the like. The feature extraction unit 120 also adds the utterance section and the silent section detected by the voice recognition unit 110 as peripheral information.

  Information derived from the summary generation system 1 can also be used as the feature information. For example, the feature extraction unit 120 detects the speaker ID based on the logged-in user of the microphone or the connected terminal device 200 as the system-derived information, and operates in conjunction with the usage time of the conference room or the scheduler. If there is detailed conference information such as the conference title that can be referred to, conference time, conference information such as participants and conference rooms, and information about individual speakers inputting audio during the conference, these will be acquired as feature information. I'll do it. FIG. 8 is a diagram showing an example of information derived from the system according to the embodiment. As shown in FIG. 8, the system-derived information is “A” for “speaker ID” and “2015/10/23” for “meeting date / time”.

  The segment candidate generation unit 130 generates a variation of the minimum structural unit candidate for structuring. The candidate of the minimum structural unit in structuring corresponds to a sequence of the same character type such as speaker, paragraph, phrase, kanji and katakana, and a character string delimited in units of semantic class, word, part of speech, etc. in descending order of granularity. .. More specifically, the segment candidate generation unit 130 reads the text information generated by the voice recognition unit 110 and the feature information extracted by the feature extraction unit 120. Then, the segment candidate generation unit 130 detects the segment label existing in each feature information. For example, in the detection of a segment label, a start instruction, an end instruction, a label that can be a clue to structuring, and the like are detected.

  Subsequently, the segment candidate generation unit 130 groups the feature information that has been read and accumulated so far. For example, in grouping, regular repeated appearances of similar elements, appearance patterns of feature information of different types, and the like are detected, and such repeated units are set as the same group. For example, as a similar element, a repetition of elements (three elements) such as "date, place, arbitrary text sentence" appears regularly.

  Further, if the segment label includes the structuring end instruction, the segment candidate generation unit 130 orders the feature information grouped by then. As an example of ordering, a method of defining ordering in advance for the type of feature information, and then defining the ordering fixedly, or in the specific example of the extracted feature information, the character length (character length included in each feature ( There are a method of ordering based on the average character length) and a method of ordering based on the number of contained specific elements (semantic classes).

  FIG. 9 is a diagram showing an example of a result of segmentation according to the embodiment. As shown in FIG. 9, the segmentation generates text information to which a command expression included in the text for each time, property information of a part of speech, a semantic class, system-derived information, and the like are added.

  The structured estimation unit 140 estimates structural information based on the segment information. More specifically, the structured estimation unit 140 reads the segment information generated by the segment candidate generation unit 130. Then, the structured estimation unit 140 reads the structure estimation model from the structure estimation model 150. The structure estimation model is learned using, as learning data, a format example suitable for display and a result edited / determined in the past. Based on such a structure estimation model, the structured estimation unit 140 presents suitable structured candidates in order by giving combinations and patterns of appearance of feature information. In the initial presentation of the structural information, the structured result with the highest likelihood is presented from the ordered segment pattern.

  Then, the structured estimation unit 140 receives a confirmation instruction from the user. The instruction from the user is accepted via the instruction unit 160. For example, if the user has no problem with the current presentation candidate, the presentation candidate is presented as a confirmed structured result. On the other hand, when the confirmation instruction from the user is not obtained (when the request to present the next candidate is accepted), the next structured result is presented. When presenting the next structured result, not only the combination of segments may be changed but also variations may be changed and presented retroactively to how to extract the segment itself. The presentation of the structured result may be output from the terminal device 200 or the summary generation device 100.

  FIG. 10 is a diagram illustrating an example of candidate extraction for structured estimation according to the embodiment. As shown in FIG. 10, based on the segment information, the information on the global structure to the information on the local structure is used as the structuring result.

  The display format conversion unit 170 converts the confirmed structured result into a display format for the user to browse. More specifically, the display format conversion unit 170 reads the structured result determined by the structured estimating unit 140. Then, the display format conversion unit 170 reads the display format conversion model. The display format conversion model describes the definition pattern of the display format to be presented according to the structured result, and is described by CSS (Cascading Style Sheets), XSLT (XSL Transformations), etc. It is possible to

  Then, the display format conversion unit 170 presents an initial conversion result from the structured result and the display format conversion model. In response to this presentation, when a confirmation instruction from the user is received via the instruction unit 160, the conversion result is output as a summary document. On the other hand, when the confirmation instruction from the user is not obtained (when the request to present the next candidate is accepted), the conversion result with the next highest likelihood is presented. The presentation of the conversion result may be output from the terminal device 200 or the summary generation device 100.

  FIG. 11 is a diagram illustrating an example of display format candidates according to the embodiment. As shown in FIG. 11, when presenting information of any structure in the structured result, the information is displayed in chronological order, displayed for each person's name, or only items and people are displayed.

  FIG. 12 is a flowchart showing an example of the flow of processing by the feature extraction unit 120 according to the embodiment. As shown in FIG. 12, the feature extraction unit 120 acquires the text information generated by the voice recognition unit 110 (step S101). Then, the feature extraction unit 120 applies morphological analysis processing to the acquired text information (step S102). Subsequently, the feature extraction unit 120 identifies the part of speech information included in the text information and executes the semantic class analysis process (step S103). After that, the feature extraction unit 120 determines a logical element for the text information (step S104).

  Then, the feature extraction unit 120 determines the segment label for the text information (step S105). Subsequently, the feature extraction unit 120 adds the utterance section and the silent section detected by the voice recognition unit 110 as peripheral information (step S106). After that, the feature extraction unit 120 detects the speaker ID based on the microphone or the login user of the terminal device 200 as the information derived from the system (step S107). Then, the feature extraction unit 120 detects conference detailed information and the like managed by the external device (step S108).

  FIG. 13 is a flowchart showing an example of the flow of processing by the segment candidate generation unit 130 according to the embodiment. As shown in FIG. 13, the segment candidate generation unit 130 reads the text information generated by the voice recognition unit 110 and the feature information extracted by the feature extraction unit 120 (step S201). Then, the segment candidate generation unit 130 reads the segment label included in each feature information (step S202). Then, the segment candidate generation unit 130 groups the feature information accumulated by the reading so far (step S203). After that, the segment candidate generation unit 130 orders the feature information grouped at this point (step S204).

  Then, the segment candidate generation unit 130 determines whether or not the segment label includes a structuring end instruction (step S205). At this time, when the segment label includes the structuring end instruction (step S205: Yes), the segment candidate generation unit 130 orders the grouped feature information (step S206). On the other hand, when the segment label does not include the structured end instruction (step S205: No), the segment candidate generation unit 130 returns to the process of step S201.

  FIG. 14 is a flowchart showing an example of the flow of processing by the structured estimation unit 140 according to the embodiment. As shown in FIG. 14, the structured estimation unit 140 reads the ordered segment information generated by the segment candidate generation unit 130 (step S301). Then, the structured estimation unit 140 reads the structure estimation model from the structure estimation model 150 (step S302). Subsequently, the structured estimation unit 140 presents initial structural information (structural candidates) by giving combinations and patterns of appearance of feature information based on the read segment information and structure estimation model ( Step S303).

  Then, when the structured estimation unit 140 receives a confirmation instruction from the user in response to the presentation of the structure information (step S304: Yes), the structured estimation unit 140 adds the presented candidate as the confirmed structure information (step S305). Further, when the confirmation instruction from the user is not accepted in response to the presentation of the structural information (the request for presenting the next candidate is accepted) (step S304: No), the structured estimation unit 140 has a high score in the second place. The structural information candidates are presented (step S306). After the candidates are presented, the process returns to step S304 and waits for a confirmation instruction from the user.

  FIG. 15 is a flowchart showing an example of the flow of processing by the display format conversion unit 170 according to the embodiment. As shown in FIG. 15, the display format conversion unit 170 reads the structured result determined by the structured estimating unit 140 (step S401). Then, the display format conversion unit 170 reads the display format conversion model (step S402). Then, the display format conversion unit 170 presents an initial conversion result from the structured result and the display format conversion model (step S403).

  After that, when the confirmation instruction from the user is received in response to the presentation of the conversion result (step S404: Yes), the display format conversion unit 170 outputs this conversion result as a summary document (step S405). Further, when the confirmation instruction from the user is not accepted in response to the presentation of the conversion result (the request for the presentation of the next candidate is accepted) (step S404: No), the candidate of the conversion result having a high score in the second run is presented. Yes (step S406). After presenting the candidates, the process returns to step S404 and waits for a confirmation instruction from the user.

  According to the embodiment, segments are estimated from the result of voice recognition processing on voice data based on expressions for explicit instructions and structured requests by a speaker, and the segments are recombined according to their contents, and Since the information is converted into the display format and presented, it is possible to reduce the preparation work.

  Moreover, the summary generation device 100 according to the above-described embodiment can be realized by using a general-purpose computer device as basic hardware. The program to be executed has a module configuration including the functions described above. The program to be executed is stored in a ROM or the like even if the program is recorded in a computer-readable recording medium such as a CD-ROM, a CD-R, or a DVD as an installable or executable file. It may be incorporated in advance and provided.

  Further, the above-described embodiments are presented as examples, and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. In addition, the respective embodiments can be appropriately combined within a range in which the contents are not inconsistent. Further, each embodiment and its modification are included in the scope and gist of the invention, and are also included in the invention described in the claims and an equivalent range thereof.

100 summary generation device 110 speech recognition unit 120 feature extraction unit 130 segment candidate generation unit 140 structured estimation unit 150 structure estimation model 160 instruction unit 170 display format conversion unit

Claims (9)

A feature extraction unit for extracting feature information of words included in text information,
Based on the extracted feature information, a segment candidate generation unit that generates a segment candidate that is a constituent unit for display,
Based on the generated segment candidates and the estimation model for structuring, a structured estimation unit that estimates a plurality of structural information that is a result of structuring the text information into mutually different structures,
Of the plurality of the structure information, the output control unit that outputs the instructed structure information as a summary of the text information,
Generating device having a. A voice recognition unit for executing voice recognition processing of voice data to generate the text information;
The summary generation device according to claim 1, wherein the feature extraction unit extracts feature information of a word included in the generated text information. The voice recognition unit generates the text information in which the character information of the utterance and the time when the utterance is associated, and extracts the acoustic feature of the voice data,
The summary generation device according to claim 2, wherein the feature extraction unit extracts a time for the character information and the acoustic feature as peripheral information for the word. The output control unit converts the instructed structure information into a display format for browsing and outputs the display format.
The summary generation device according to claim 1. The structured estimation unit, based on the estimation model that is learning data based on past processing results, presents structural information with a higher likelihood than the other structural information for the estimation model,
The output control unit outputs the presented structure information as a summary of the text information when a confirmation instruction is accepted for the presented structure information.
The summary generation device according to claim 1.   The summary generation device according to claim 4, wherein the output control unit changes the display format according to a user instruction. The structured estimation unit presents a plurality of the structural information in order of likelihood with respect to the estimation model,
The output control unit outputs the presented structure information as a summary of the text information when a confirmation instruction is accepted for the presented structure information.
The summary generation device according to claim 1. A summary generation method executed by a summary generation device, comprising:
A step of extracting feature information of words included in the text information,
Based on the extracted feature information, a step of generating a segment candidate that is a constituent unit for display,
Based on the generated segment candidates and the estimation model for structuring, a structured estimation step of estimating a plurality of structural information that is the result of structuring the text information into mutually different structures,
An output control step of outputting the instructed structure information among a plurality of the structure information as a summary of the text information,
A summary generation method including. In the summary generator,
A step of extracting feature information of words included in the text information,
Based on the extracted feature information, a step of generating a segment candidate that is a constituent unit for display,
Based on the generated segment candidates and the estimation model for structuring, a structured estimation step of estimating a plurality of structural information that is the result of structuring the text information into mutually different structures,
An output control step of outputting the instructed structure information among a plurality of the structure information as a summary of the text information,
A summary generator to run.

Images