JP2012208630A - Speech management system, speech management method and speech management program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech management system, a speech management method and a speech management program for making record about speech at meetings and for supporting smooth proceedings progress.SOLUTION: A control part 21 of a meeting support server 20 acquires sound, performs speech recognition processing and records a recognition result in a speech management information storage part 25. Then, the control part 21 performs keyword extraction processing using the speech management information storage part 25. The control part 21 performs delete processing of the proceedings article when the control part 21 determines that there is proceedings progress in a meeting progress management processing carried out based on the keyword. Besides, in an explanation display management processing carried out based on the keyword, the control part 21 searches for reference information and outputs display candidates on a client terminal 10.

Description

  The present invention relates to a statement management system, a statement management method, and a statement management program for creating a record of a statement in a meeting or the like and supporting the proceeding of proceedings.

  When a meeting is held, the minutes may be created after the meeting ends. In order to improve the efficiency of such minutes creation, a minutes creation system is being studied. For example, a technique for creating a minutes while identifying a speaker for each comment content is being studied (for example, see Patent Document 1). In the technique described in this document, the voice recognition unit recognizes the voice of the conference attendee based on the recording contents collected by the microphone. Then, minutes data is created from the recognized voice. The speaker specifying unit specifies a speaker of each message content by voiceprint authentication. The minutes data in which the contents of the statements in the created minutes data are associated with the specified speaker are displayed.

  Further, a technique for extracting and viewing only important portions from recorded conference videos has been studied (for example, see Patent Document 2). The technique described in this document includes index detection means for detecting an important part of the conference in the conference video. Then, a minutes file is created based on the index information.

  In addition, a technique for recognizing a keyword included in voice input to a microphone and performing various operations based on the keyword has been studied (see, for example, Patent Document 3). In the technique described in this document, an input speech recognition process is executed by a keyword spotting method in a speech recognition unit.

Japanese Unexamined Patent Publication No. 2007-233075 (first page, FIG. 1) Japanese Patent Laying-Open No. 2005-341015 (first page, FIG. 1) JP-A-8-339198 (first page, FIG. 1)

  In the technique described in Patent Document 1, the content can be confirmed based on the displayed remark content. However, technical terms may be included in the content of remarks, and the content of remarks may not be properly understood depending on the knowledge level of the conference participants.

  In addition, as for the progress of the conference, there are cases where it is desired to present relevant materials to the conference participants. As described above, when presenting related materials, it may not be possible to concentrate on the conference if it takes time and effort from the conference chairman and the explainer.

In a meeting, the order of agendas is often determined in advance depending on the meeting. In this case, it is convenient if the participant can grasp the current status of the proceedings.
The present invention has been made to solve the above problems, and provides a statement management system, a statement management method, and a statement management program for creating a record of a statement and supporting smooth proceeding of proceedings. There is.

  In order to solve the above-mentioned problems, the invention described in claim 1 includes a keyword storage unit that records related information for a keyword, a speech storage unit that records speech-recognized speech content, and a speech input device. A speech management system comprising control means connected to a display device, wherein the control means acquires speech of a speaker in the speech input device, creates speech content that is speech-recognized, and stores the speech storage A means for recording in the means, a means for performing a keyword spotting process for comparing the message text included in the content of the message and the keyword stored in the keyword storage means, and when the message word matches the keyword And a means for outputting relevant information associated with the keyword to the display device.

  The invention according to claim 2 is summarized in that in the statement management system according to claim 1, an explanatory text about a keyword is recorded in the keyword storage unit as the related information.

  The invention described in claim 3 further comprises user management means for recording user attribute information in the statement management system according to claim 1 or 2, wherein the user attribute information is stored for each keyword. Further recorded, the control means acquires attribute information of the user of each display device from the user management means, extracts a keyword corresponding to the attribute information from the keyword storage means, and extracts the extracted keyword The gist is that the keyword spotting process is performed using.

  According to a fourth aspect of the present invention, in the message management system according to any one of the first to third aspects, the output information is recorded in the keyword storage means for the keyword for which the related information is output to the display device. The gist of the keyword spotting process is to use a keyword for which output information is not recorded.

  According to a fifth aspect of the present invention, in the speech management system according to any one of the first to fourth aspects, a speech statement is included for a keyword for which the control means outputs related information to the display device. The gist further comprises means for counting the number of utterances and changing the display method of the relevant information of the keyword according to the number of utterances.

  The invention according to claim 6 is the statement management system according to any one of claims 1 to 5, wherein the keyword storage means stores a keyword for specifying the progress of the agenda item, The gist of the present invention is that the control means further comprises means for deleting the agenda items output to the display device when the match between the statement word and the keyword is detected.

  The invention according to claim 7 is the statement management system according to any one of claims 1 to 6, wherein the control unit creates a summary using the statement contents recorded in the statement storage unit, The gist of the invention is that the display device further includes means for providing the display device.

The gist of the invention described in claim 8 is that in the statement management system according to claim 7, the control means creates a summary of the content range of the statement designated in the display device.
According to a ninth aspect of the present invention, in the speech management system according to the seventh aspect, the control means specifies the position of the speech sentence extracted by the keyword spotting process in the speech content, and the position of the speech sentence is determined. The gist is to identify the summarization target range and summarize the content of the remarks in this summarization target range.

  According to a tenth aspect of the present invention, there is provided a keyword storage means for recording relevant information for a keyword, a speech storage means for recording speech-recognized speech content, and a control means connected to the speech input device and the display device. A speech management method using a speech management system comprising: a speech management apparatus that acquires speech of a speaker in the speech input device, creates speech-recognized speech content, and records the speech content in the speech storage device Performing a keyword spotting process for comparing the statement word included in the statement content with the keyword stored in the keyword storage means, and when the statement word matches the keyword, the keyword And executing the step of outputting the related information associated with the display device to the display device.

  According to the eleventh aspect of the present invention, there is provided a keyword storage means for recording related information for a keyword, a speech storage means for recording speech-recognized speech content, and a control means connected to the speech input device and the display device. A speech management system comprising: a means for acquiring speech of a speaker in the speech input device, creating speech-recognized speech content, and recording the speech content in the speech storage means; Means for performing a keyword spotting process for comparing the statement text included in the content of the comment and the keyword stored in the keyword storage means; and if the statement word matches the keyword, the association associated with the keyword The gist is to allow information to function as a means for outputting to the display device.

(Function)
According to the first, tenth, or eleventh aspects, the control means acquires the voice of the speaker in the voice input device, creates the voice-recognized voice content, and records it in the voice storage means. Next, the statement text included in the statement content is compared with the keyword stored in the keyword storage means. When the statement word matches the keyword, the related information associated with the keyword is output to the display device. Thereby, it is possible to provide related information related to the remarks to the user.

  According to the second aspect of the present invention, the keyword storage is recorded with the explanatory text about the keyword as the related information. Thereby, the user can acquire the description information about the utterance.

  According to the invention described in claim 3, the attribute information of the user of each display device is acquired from the user management means. Then, a keyword corresponding to the attribute information is extracted from the keyword storage means, and a keyword spotting process is performed using the extracted keyword. Thereby, the related information according to a user's attribute can be provided.

  According to the fourth aspect of the present invention, the output information is recorded in the keyword storage means for the keyword for which the related information is output to the display device, and the keyword in which the output information is not recorded is used in the keyword spotting process. . Thereby, about the already provided information, the duplicate provision can be suppressed.

  According to the fifth aspect of the present invention, the number of utterances including the utterance wording is counted for the keyword for which the related information is output to the display device, and the display method of the keyword related information is changed according to the number of utterances. To do. Thereby, weighting of keywords can be performed, and display according to the weighting can be performed.

  According to the sixth aspect of the present invention, the keyword storage means stores a keyword for specifying the progress of the agenda item. When the control means detects a match between the statement and the keyword, the agenda item output to the display device is erased. Thereby, the progress of proceedings can be provided to the user.

According to the seventh aspect of the present invention, a summary is created using the content of the speech recorded in the speech storage means. Thereby, a summary can be provided to the user.
According to the eighth aspect of the present invention, a summary of the content range of the message specified in the display device is created. Thereby, the summary about the speech of the predetermined range can be provided.

  According to the invention described in claim 9, the position of the statement word extracted by the keyword spotting process in the statement content is specified. Then, the summarization target range is specified using the position of the remark wording, and the content of the remarks in the summarization target range is summarized. Thereby, the summary of the range specified by the keyword can be created.

  According to the present invention, it is possible to provide a speech management system, a speech management method, and a speech management program for creating a record of speech and supporting smooth proceeding of proceedings.

Explanatory drawing of the speech management system of embodiment of this invention. It is explanatory drawing of the data recorded on each data memory | storage part of this embodiment, (a) is a keyword memory | storage part, (b) is a reference information memory | storage part, (c) is a meeting management information memory | storage part, (d) Is an explanatory diagram of a speech management information storage unit. Explanatory drawing of the process sequence of this embodiment. Explanatory drawing of the process sequence of this embodiment. Explanatory drawing of the display screen in the client terminal of this embodiment. Explanatory drawing of the process sequence of other embodiment. Explanatory drawing of the process sequence of other embodiment. Explanatory drawing of the process sequence of other embodiment. Explanatory drawing of the process sequence of other embodiment.

  Hereinafter, an embodiment of a message management system embodying the present invention will be described with reference to FIGS. In the present embodiment, a case where a video conference is performed by connecting client terminals installed at a plurality of bases will be described. In the present embodiment, as shown in FIG. 1, a client terminal 10 and a conference support server 20 connected to each other via a network are used.

  The client terminal 10 is a computer terminal used by a participant (user) of a video conference. The client terminal 10 includes a control unit, an input unit such as a keyboard and a point device, an output unit (display device) such as a display, a microphone (voice input device), a speaker, a camera, and the like.

  The client terminal 10 transmits video / audio data including video data captured by the camera and audio data collected by the microphone to the conference support server 20 via the network. When transmitting video / audio data, the client terminal 10 includes data relating to its own terminal code.

  In addition, when the client terminal 10 acquires video / audio data of another base from the conference support server 20, the client terminal 10 outputs audio from a speaker or displays an image on a display.

  A video conference screen 500 shown in FIG. 5 is output on the display of the client terminal 10. The video conference screen 500 includes a participant display area 510, a conference display column 520, a speech history display column 530, and an explanation display column 540. In the participant display area 510, image data acquired from the client terminal 10 at another base is output. In the on-conference display column 520, the up-to-conference that lists the agenda items of this conference is output. In the speech history display field 530, text obtained by voice recognition of the speech of the conference participant is displayed. In the explanation display field 540, a selection icon for instructing output of reference information related to a statement is displayed.

The conference support server 20 is a computer system that supports a video conference by the client terminal 10 installed at each site.
The conference support server 20 includes a control unit 21, a keyword storage unit 22, a reference information storage unit 23, a conference management information storage unit 24, a speech management information storage unit 25, and a video / audio storage unit 26. In the present embodiment, the keyword storage unit 22 and the reference information storage unit 23 function as a keyword storage unit, and the speech management information storage unit 25 functions as a speech storage unit. The control unit 21 includes control means (CPU, RAM, ROM, etc.), and will be described later (conference relay stage, voice recognition stage, speech recording stage, screen control stage, keyword extraction stage, progress management stage, explanation display). Stage, summary creation stage, etc.). By executing the speech management program (for the audience) for that purpose, the control unit 21, as shown in FIG. 1, the conference relay unit 211, the speech recognition unit 212, the speech recording unit 213, the screen control unit 214, keyword extraction Functions as means 215, progress management means 216, explanation display means 217, and summary creation means 218.

  The conference relay unit 211 executes processing for transferring the video / audio data acquired from the client terminal 10 to the client terminal 10 installed at another base. In addition, the conference relay unit 211 supplies audio data included in the video / audio data to the audio recognition unit 212.

The voice recognition unit 212 executes processing for generating voice recognition text converted into text by voice recognition processing based on voice data acquired from the client terminal 10.
The utterance recording unit 213 executes a process of translating the speech recognition text to generate a translation text. Further, the utterance recording unit 213 executes a process of recording the speech recognition text and the translation text in the utterance management information storage unit 25 by associating the participant code specifying the speaker based on the terminal code.

  The screen control unit 214 executes processing for controlling the display of the video conference screen 500 displayed on the display of each client terminal 10. More specifically, the contents of the conferences that list the items of the video conference are output to the display column 520 of the conference on the video conference screen 500. Further, the speech history display field 530 displays a text obtained by voice recognition of the speech of the conference participant. Further, in the explanation display field 540, a selection icon for selecting reference information related to the utterance is displayed. When the information displayed in the message history display column 530 and the explanation display column 540 overflows, the screen control unit 214 sequentially deletes the display from the old information.

  The keyword extraction unit 215 executes a process (keyword spotting process) for matching the text (speech sentence) recorded in the speech management information storage unit 25 with the keyword recorded in the keyword storage unit 22.

The progress management means 216 executes a process for managing the progress of the conference based on the keywords extracted by the keyword spotting process.
The explanation display unit 217 executes a process for managing the output of the reference information based on the keywords extracted by the keyword spotting process.
The summary creation means 218 executes a process for generating a summary using a known summary creation technique based on the minutes record recorded in the statement management information storage unit 25.

  As shown in FIG. 2A, a keyword management record 220 for performing a keyword spotting process is recorded in the keyword storage unit 22. This keyword management record 220 is recorded when a keyword is registered. The keyword management record 220 records data related to keywords and correspondence.

In the keyword data area, data relating to a word that is a target of the keyword spotting process is recorded.
In the corresponding data area, data related to the response when the corresponding keyword is detected is recorded. In the present embodiment, keyword spotting processing is used for conference progress management, explanation display, or summary creation. For this reason, a flag for specifying conference progress management, explanation display, or summary creation is recorded in the corresponding data area.

  In the reference information storage unit 23, as shown in FIG. 2B, a reference information management record 230 for reference information output to the client terminal 10 is recorded. This reference information management record 230 is recorded when reference information corresponding to a keyword is registered. In the reference information management record 230, data on keywords, reference materials, and display status are recorded.

In the keyword data area, data related to a term that is an output target of explanation information is recorded.
In the reference information data area, related information corresponding to this keyword is recorded. In the present embodiment, explanations (descriptions) of technical terms used in the conference, presentation materials, and internal materials are recorded.

  In the display status data area, a flag for specifying the display status of the reference information is recorded. When the selection icon is displayed in the client terminal 10, the displayed flag is recorded as the output information.

  As shown in FIG. 2C, conference management information for managing the conference is recorded in the conference management information storage unit 24. In this embodiment, a conference summary record 241, a conference management record 242 and a participant management record 243 are recorded. The conference management information storage unit 24 in which the participant management record 243 is recorded functions as a user management unit. The conference summary record 241 to the participant management record 243 are recorded when the conference is decided to be held and the conference management information is registered.

In the conference summary record 241, data relating to the conference code, the conference date and time, and the title are recorded.
In the conference code data area, data relating to an identifier for identifying each conference is recorded. With this conference code, the conference summary record 241 is associated with the management record 242 and the participant management record 243 as soon as the conference is performed.

In the meeting date / time data area, data relating to the date and time of the video conference is recorded.
In the title data area, data relating to the title and purpose of the video conference is recorded.

On the meeting, the management record 242 stores data relating to the agenda order, agenda items, keywords, and end determination.
In the agenda order data area, data for specifying the order of agenda items scheduled for discussion in a video conference is recorded.

In the agenda item data area, data relating to the agenda item of the agenda is recorded. If multiple agendas are scheduled for discussion, all agendas are recorded.
In the keyword data area, data relating to keywords used in the discussion of this agenda is recorded.
In the end determination data area, a flag for determining the end of the discussion on the agenda is recorded. For agendas that have been debated, an end flag is recorded.

In the participant management record 243, data related to the participant code, role code, terminal code, and language is recorded.
In the participant code data area, data relating to an identifier for identifying the participant of the video conference is recorded.

  In the role code data area, data relating to an identifier for identifying the role of the participant in the video conference is recorded. In the present embodiment, a moderator flag is recorded in the participant management record 243 of the moderator.

In the terminal code data area, an identifier for specifying the client terminal 10 used by the participant is recorded. For example, an IP address can be used as a terminal code.
In the language data area, an identifier for identifying the language used by the participant is recorded.

  In the speech management information storage unit 25, as shown in FIG. 2D, information related to the speech of the participant in the video conference is recorded. In the present embodiment, an audio file, a statement history record 251 and a summary management record 252 are recorded. The speech history record 251 is recorded when a speech recognition process described later is executed. The summary management record 252 is recorded when a summary creation process described later is executed.

In the speech history record 251, data relating to the time, the speaker code, and the content of the speech are recorded.
In the time data basin, data related to the date and time when the utterance is recorded is recorded.
In the speaker code data area, data relating to an identifier (participant code) for identifying a speaker is recorded.
In the statement content data area, a speech recognition text obtained by converting speech data by speech recognition processing and a translation text obtained by translating the speech recognition text are recorded.

The summary management record 252 records time and summary data.
In the time data basin, data relating to the time at which the summary creation process was performed is recorded.
In the summary data basin, data related to the summary content generated by the summary creation process is recorded.

  In the video / audio storage unit 26, video / audio data acquired from each client terminal 10 is recorded. The video / audio data is recorded in association with the terminal code of the client terminal 10 that transmitted the video / audio data and the acquisition time of the video / audio data. By using this acquisition time and the time recorded in the speech history record 251, the video / audio data can be associated with the content of the speech history record 251.

Next, the operation in this message management system will be described with reference to FIGS. Here, speech registration processing (FIG. 3), keyword spotting processing (FIG. 3), and summary creation processing (FIG. 4) are performed in this order.
(Speech registration process)
First, the speech registration process will be described with reference to FIG. This process is performed during the video conference.

  Here, the control unit 21 of the conference support server 20 executes a voice acquisition process (step S1-1). Specifically, the conference relay unit 211 of the control unit 21 acquires video / audio data from each client terminal 10. The video / audio data includes data related to the terminal code of the client terminal 10 that has transmitted the data. The conference relay unit 211 records the acquired video / audio data in the video / audio storage unit 26 in association with the terminal code of the client terminal 10 that transmitted the video / audio data and the acquisition time of the video / audio data. Then, the conference relay unit 211 transfers to the client terminal 10 installed at another base. Further, the conference relay unit 211 supplies audio data included in the video / audio data to the audio recognition unit 212.

  Next, the control part 21 of the meeting assistance server 20 performs a speech recognition process (step S1-2). Specifically, the voice recognition unit 212 of the control unit 21 generates voice text by voice recognition processing of the acquired voice data. Then, the voice recognition unit 212 provides the voice recording unit 213 with the voice text.

  Next, the control unit 21 of the conference support server 20 executes a statement recording process (step S1-3). Specifically, the statement recording unit 213 of the control unit 21 extracts the participant management record 243 in which the terminal code of the client terminal 10 that has transmitted the video / audio data is recorded in the conference management information storage unit 24. Then, the utterance recording unit 213 specifies the participant code recorded in the participant management record 243. Furthermore, the statement recording unit 213 specifies the language of the participant who uses each client terminal 10 using the participant management record 243. Next, the utterance recording unit 213 generates a translated text obtained by translating the speech text acquired from the speech recognition unit 212 according to the specified language. Then, the utterance recording unit 213 generates a utterance history record 251 in which the current time, terminal code, and utterance content (speech text and translation text) are recorded from the system timer, and records the utterance history record 251 in the utterance management information storage unit 25. Further, the message recording unit 213 supplies the content of the message to the screen control unit 214.

  Next, the control unit 21 of the conference support server 20 executes a speech history output process (step S1-4). Specifically, the screen control unit 214 of the control unit 21 transmits the audio text and the translated text to each client terminal 10 to each client terminal 10. In this case, the content of the message is output to the message history display field 530 of the video conference screen 500. If the message history display field 530 overflows, the oldest message content is deleted in order.

(Keyword spotting process)
Next, the keyword spotting process will be described with reference to FIG. This process is performed during the video conference. In the present embodiment, a conference progress management process, an explanation display management process, and a summary creation process are executed in the keyword spotting process.

  First, the control part 21 of the meeting assistance server 20 performs a keyword extraction process (step S2-1). Specifically, the keyword extraction unit 215 of the control unit 21 acquires the comment history record 251 newly recorded in the comment management information storage unit 25. Next, the keyword extraction unit 215 determines whether or not the keyword recorded in the keyword storage unit 22 is included in the statement history record 251. If no keyword is included, the keyword spotting process for the statement history record 251 is terminated.

  On the other hand, when a keyword is included, the keyword extraction unit 215 specifies the correspondence recorded in the keyword management record 220. Here, it is assumed that “conference progress management” is recorded in the corresponding data area in association with the keywords “end” and “end” for specifying the progress of the agenda item. In this case, the keyword extraction unit 215 takes over the process to the progress management unit 216.

[Conference progress management process]
First, the conference progress management process will be described.
The control unit 21 of the conference support server 20 executes a speaker specifying process (step S2-2). Specifically, the progress management means 216 of the control unit 21 acquires the speaker code recorded in the statement history record 251 extracted by the keyword spotting process.

  Next, the control part 21 of the meeting assistance server 20 performs the determination process about whether it is a moderator (step S2-3). Specifically, the progress management means 216 of the control unit 21 acquires the participant management record 243 in which the speaker code is recorded in the participant code data area from the conference management information storage unit 24. Here, if the moderator flag is recorded in the role code data area of the participant management record 243, it is determined as a moderator.

If it is determined that the person is not a moderator (“NO” in step S2-3), the control unit 21 of the conference support server 20 ends the conference progress management process.
On the other hand, when it determines with it being a moderator (in the case of "YES" in step S2-3), the control part 21 of the meeting assistance server 20 performs the search process depending on a meeting (step S2-4). Specifically, the progress management means 216 of the control unit 21 specifies the current agenda item by specifying a record that is the management record 242 for which the end flag is not recorded and that has the earliest agenda order. .

  Next, the control part 21 of the meeting assistance server 20 performs the determination process about whether the proceeding proceeded (step S2-5). Specifically, the progress management means 216 of the control unit 21 confirms whether or not the keyword recorded in the specified management record 242 is included in the statement history record 251 that is the target of the keyword spotting process. To do. If a keyword is included, it is determined that the proceedings have progressed.

  If it is determined that no keyword is included and there is no proceeding (if “NO” in step S2-5), the control unit 21 of the conference support server 20 ends the conference progress management process.

  On the other hand, when it is determined that the keyword is included and the proceedings have progressed (in the case of “YES” in step S2-5), the control unit 21 of the conference support server 20 executes the process for deleting the agenda items ( Step S2-6). Specifically, the progress management means 216 of the control unit 21 records an end flag in the end determination data area of the management record 242 as soon as the conference. Further, the progress management unit 216 notifies the screen control unit 214 of the completed item. In this case, the screen control unit 214 grays out the notified agenda items in each client terminal 10 depending on the conference displayed in the display column 520 depending on the conference on the video conference screen 500.

[Explanation display management process]
Next, explanation display management processing will be described. This process is performed when “description display” is recorded in the corresponding data area in association with the keyword of the technical term or the document name in the keyword extraction process (step S2-1). In this case, the keyword extraction unit 215 takes over the processing to the explanation display unit 217.

  Here, the control unit 21 of the conference support server 20 executes a reference information search process (step S2-7). Specifically, the explanation display means 217 of the control unit 21 specifies the reference information management record 230 in which the keyword subjected to the keyword spotting process is recorded in the reference information storage unit 23.

  Next, the control unit 21 of the conference support server 20 executes display status search processing (step S2-8). Specifically, the explanation display unit 217 of the control unit 21 checks whether or not a displayed flag is recorded in the display status data area of the identified reference information management record 230.

  Next, the control part 21 of the meeting assistance server 20 performs the determination process about whether it is already displayed (step S2-9). Specifically, the explanation display unit 217 of the control unit 21 determines that the display is already performed when the displayed flag is recorded in the display status data area of the reference information management record 230.

When it determines with already displayed (in the case of "YES" in step S2-9), the control part 21 of the meeting assistance server 20 complete | finishes description display management processing.
On the other hand, when it is determined that the displayed flag is not recorded and is not already displayed (in the case of “NO” in step S2-9), the control unit 21 of the conference support server 20 executes display candidate output processing. (Step S2-10). Specifically, the explanation display unit 217 of the control unit 21 notifies the screen control unit 214 of the keyword extracted by the keyword spotting process. In this case, the screen control unit 214 displays a selection icon for outputting reference information related to the keyword in the explanation display field 540 of the video conference screen 500 in each client terminal 10. This selection icon is linked to a reference material recorded in the reference information storage unit 23. In this case, the screen control unit 214 records the displayed flag in the display status data area of the reference information management record 230.

  Next, the control unit 21 of the conference support server 20 executes reference information output processing (step S2-11). Specifically, the explanation display unit 217 of the control unit 21 outputs the reference information of the keyword corresponding to the selected icon on the display of the client terminal 10 when detecting the pressing of the selected icon in the client terminal 10.

[Summary creation process]
Next, the summary creation process will be described with reference to FIG.
Here, first, the control unit 21 of the conference support server 20 executes a summary instruction acquisition process (step S3-1). This processing is performed when “summary creation” is recorded in the corresponding data area in association with the keyword “summary” in the keyword extraction processing (step S2-1). In this case, the keyword extraction unit 215 takes over the process to the summary creation unit 218.

  Next, the control part 21 of the meeting assistance server 20 performs a search process of a prior | preceding summary (step S3-2). Specifically, the summary creation means 218 of the control unit 21 searches the statement management information storage unit 25 for the summary management record 252.

  Next, the control part 21 of the meeting assistance server 20 performs the determination process about whether there exists registration of a prior | preceding summary (step S3-3). Specifically, when the summary management record 252 is recorded in the speech management information storage unit 25, the summary creation unit 218 of the control unit 21 determines that the preceding summary is registered.

  When it is determined that the summary management record 252 is not recorded and the preceding summary is not registered (“NO” in step S3-3), the control unit 21 of the conference support server 20 generates the summary from the beginning. Processing is executed (step S3-4). Specifically, the summary creation means 218 of the control unit 21 extracts the speech history record 251 from the start of the video conference from the speech management information storage unit 25. Then, the summary creation means 218 creates a summary by aggregating the content of the comments in the extracted comment history record 251.

  On the other hand, when the summary management record 252 is recorded and it is determined that the preceding summary is registered ("YES" in step S3-3), the control unit 21 of the conference support server 20 summarizes the preceding summary and subsequent summaries. Is generated (step S3-5). Specifically, the summary creation means 218 of the control unit 21 specifies the latest time (most recent summary creation time) among the summary management records 252 recorded in the speech management information storage unit 25. Then, the summary creation means 218 extracts the speech history record 251 in which the time after the latest summary creation time is recorded from the speech management information storage unit 25. Then, the summary creation means 218 creates a summary by aggregating the content of the comments in the extracted comment history record 251.

  Next, the control unit 21 of the conference support server 20 executes summary output processing (step S3-6). Specifically, the summary creation means 218 of the control unit 21 supplies the created summary to the screen control means 214. The screen control unit 214 transmits the summary to each client terminal 10 to each client terminal 10. In this case, the summary is output to the comment history display field 530 of the video conference screen 500.

As described above, according to the present embodiment, the following effects can be obtained.
(1) In the above embodiment, the control unit 21 of the conference support server 20 performs voice acquisition processing (step S1-1), voice recognition processing (step S1-2), and speech recording processing (step S1-3). Execute. As a result, it is possible to keep a record of remarks in a meeting or the like. By using this record, the minutes can be efficiently created.

  Here, the utterance recording unit 213 generates a translated text obtained by translating the speech text acquired from the speech recognition unit 212 according to the specified language. Thereby, even when the conference is held in a foreign language, the content of the speech can be grasped by the language of the conference participants themselves.

  (2) In the above-described embodiment, the control unit 21 of the conference support server 20 executes a speech history output process (step S1-4). In this case, the video conference screen 500 including the message history display field 530 is output on the display of the client terminal 10. Thereby, each participant's remark can be confirmed.

  (3) In the above embodiment, the control unit 21 of the conference support server 20 executes the keyword extraction process (step S2-1). Here, when “conference progress management” is recorded in association with the keyword, the conference progress management process is executed. Then, when it is determined that the keyword is included and the proceedings have progressed (in the case of “YES” in step S2-5), the control unit 21 of the conference support server 20 executes a process for clearing the agenda items ( Step S2-6). Thereby, the participant can grasp the progress of proceedings at the client terminal 10.

  (4) In the above embodiment, the control unit 21 of the conference support server 20 executes the keyword extraction process (step S2-1). Here, when “description display” is recorded in association with the keyword, in the explanation display management process, the control unit 21 performs a reference information search process (step S2-7) and a display status search process (step S2-). 8) is executed. If it is determined that it is not already displayed (in the case of “NO” in step S2-9), the control unit 21 of the conference support server 20 outputs the display candidate (step S2-10), the reference information output process (step S2-10). Step S2-11) is executed. Thus, the participant can efficiently obtain the reference information at the client terminal 10.

  (5) In the above embodiment, when “summary creation” is recorded in association with a keyword, summary creation processing is executed. Here, when it is determined that the registration of the preceding summary is not registered (in the case of “NO” in step S3-3), the control unit 21 of the conference support server 20 executes a summary generation process from the beginning ( Step S3-4). On the other hand, when it is determined that registration of the preceding summary is registered (in the case of “YES” in step S3-3), the control unit 21 of the conference support server 20 executes a summary generation process after the preceding summary ( Step S3-5). And the control part 21 of the meeting assistance server 20 performs the output process of a summary (step S3-6). Thereby, the client terminal 10 can grasp a summary of discussion contents in the conference.

In addition, you may change the said embodiment into the following aspects.
In the above embodiment, the explanation display management process is executed in the keyword spotting process. The correspondence method for the extracted keyword is not limited to these. For example, a keyword specifying an important utterance may be recorded in the keyword storage unit 22. In this case, the explanation display means 217 holds data related to a predetermined time range for searching for the utterance history record 251 including important utterances. This process will be described with reference to FIG.

  First, the control part 21 of the meeting assistance server 20 performs a keyword extraction process similarly to step S2-1 (step S4-1). Here, it is assumed that the keyword extraction unit 215 detects the keyword “important” and takes over the processing to the explanation display unit 217.

  Here, the control unit 21 of the conference support server 20 executes an important message extraction process (step S4-2). Specifically, the explanation display means 217 of the control unit 21 extracts the comment history record 251 including the keyword “important”. Further, the explanation display means 217 extracts a comment history record 251 within a predetermined time range for this comment history record 251. Then, the explanation display means 217 generates a reference information management record 230 in which the content of the message recorded in the message history record 251 is recorded in the reference material data area as an important message, and registers it in the reference information storage unit 23.

  Next, the control unit 21 of the conference support server 20 executes display candidate output processing (step S4-3). Specifically, the explanation display unit 217 of the control unit 21 notifies the screen control unit 214 that an important statement has been made. In this case, the screen control unit 214 displays an important message display icon in the explanation display field 540 of the video conference screen 500 in each client terminal 10. This important message display icon is linked to the reference material (important message) recorded in the reference information storage unit 23.

Next, the control unit 21 of the conference support server 20 executes an important message output process (step S4-4). Specifically, the explanation display unit 217 outputs the important message recorded in the reference information storage unit 23 to the display of the client terminal 10 when the client terminal 10 detects the selection of the important message display icon.
As a result, the client terminal 10 can efficiently confirm important messages.

  In the above embodiment, when it is determined in the summary creation process that there is no registration of the preceding summary (“NO” in step S3-3), the control unit 21 of the conference support server 20 generates the summary from the beginning. Processing is executed (step S3-4). On the other hand, when it is determined that there is registration of the preceding summary (in the case of “YES” in step S3-3), the control unit 21 of the conference support server 20 executes the summary generation processing after the preceding summary (step S3- 5). The method for creating the summary is not limited to this. For example, the position of the comment sentence extracted by the keyword spotting process may be specified and summarized. Specifically, it detects the break of the meeting (end of the agenda), specifies the summary target range for each agenda, and creates a summary. In this case, the time when the agenda item clearing process (step S2-6) is executed is recorded as the end time in the management record 242 as soon as the meeting is in the agenda order, and is started in the management record 242 as soon as the meeting is in the next agenda order. Record as time. This process will be described with reference to FIG.

  First, the control unit 21 of the conference support server 20 executes a conference break detection process (step S5-1). Specifically, the summary creation means 218 of the control unit 21 detects the proceeding of proceedings in the progress management means 216.

  Next, the control unit 21 of the conference support server 20 executes a process for generating a summary of the separation range (step S5-2). Specifically, the summary creation means 218 of the control unit 21 acquires the start time and the end time recorded in the management record 242 as soon as the agenda item ended at the conference break. Then, the summary creating means 218 extracts the message history record 251 in which the start time to the end time are recorded from the message management information storage unit 25. Then, the summary creation means 218 aggregates the content of the extracted speech history record 251 and creates a summary.

Next, the control part 21 of the meeting assistance server 20 performs the output process of the summary similarly to step S3-6 (step S5-3).
Thus, at the stage where each agenda is completed, the client terminal 10 can confirm a summary of the agenda.

  In the above embodiment, a selection icon for instructing output of reference information related to a statement is displayed in the explanation display field 540. When the information displayed in the explanation display field 540 overflows, the display is deleted sequentially from the old information. The display control in the explanation display field 540 is not limited to this. For example, the display method may be changed according to the importance of the keyword. In this case, the number of utterances is recorded in the display status data area of the reference information management record 230 of the reference information storage unit 23. Further, the explanation display means 217 holds a time determination table for determining the display time according to the importance. This process will be described with reference to FIG.

First, the control part 21 of the meeting assistance server 20 performs a keyword extraction process similarly to step S2-1 (step S6-1).
Next, similarly to step S2-7, the control unit 21 of the conference support server 20 executes reference information search processing (step S6-2).

  Next, the control unit 21 of the conference support server 20 executes display status update processing. (Step S6-3). Specifically, the explanation display means 217 of the control unit 21 updates the number of utterances recorded in the display status data area of the reference information management record 230.

  Next, the control unit 21 of the conference support server 20 executes importance evaluation processing. (Step S6-4). Specifically, the explanation display means 217 of the control unit 21 acquires the reference information management record 230 corresponding to all the selection icons displayed in the explanation display field 540, and the number of utterances recorded in the display status data area. Is identified. The explanation display means 217 evaluates the importance according to the number of utterances.

  Next, the control part 21 of the meeting assistance server 20 performs a rearrangement process according to importance (step S6-5). Specifically, the explanation display means 217 of the control unit 21 determines the priority order of the selected icons.

  Next, similarly to steps S2-10 and S2-11, the control unit 21 of the conference support server 20 executes display candidate output processing (step S6-6) and reference information output processing (step S6-7). To do.

  Important keywords are often spoken many times. Therefore, the importance can be evaluated according to the number of utterances, and the display can be changed according to the importance. Thereby, the display time when the selection icon is displayed can be controlled by rearranging the priorities.

  In the above embodiment, the reference information storage unit 23 stores the reference information management record 230 for the explanation information output to the client terminal 10. The control unit 21 of the conference support server 20 executes display candidate output processing (step S2-10) and reference information output processing (step S2-11). In this case, keyword spotting processing is performed for all reference information recorded in the reference information storage unit 23. Instead, the keyword used in the keyword spotting process may be changed according to the attribute information (for example, skill level) of the participant. In this case, participant attribute information (skill level) is recorded in the participant management record 243. Further, the keyword management record 220 records provision target person attribute information (skill level) for the target person who provides this reference information. Then, the keyword extraction unit 215 of the control unit 21 acquires the skill level from the participant management record 243 recorded in the conference management information storage unit 24. Then, the lowest level is determined based on the lowest skill level. Then, the keyword spotting process is performed using the keyword management record 220 in which the provision target person attribute information of the lowest level or higher is recorded. Thereby, provision of unnecessary information can be suppressed according to the skill level of the participant.

  In the above embodiment, the same display candidate is output to all the client terminals 10. Instead of this, display candidates may be changed according to the attribute information (for example, skill level) of the participant. In this case, participant attribute information (skill level) is recorded in the participant management record 243. Further, in the reference information management record 230, provision target person attribute information (skill level) for a target person who provides this reference information is recorded. Then, the explanation display means 217 acquires the skill level recorded in the reference information management record 230 of the keyword specified by the keyword spotting process. The client terminal 10 used by the participant in the participant management record 243 that matches this skill level is specified, and display candidates are output only to the specified client terminal 10. Thereby, for example, reference information is provided to a participant with a low skill level, and provision of unnecessary information to a participant with a high skill level can be suppressed. Therefore, accurate reference information can be provided according to the participant attributes.

  In the above embodiment, the reference information storage unit 23 stores the reference information management record 230 for the reference information output to the client terminal 10. In this reference information management record 230, data on keywords, reference materials, and display status are recorded. Using the reference information storage unit 23, the control unit 21 of the conference support server 20 executes a display status search process (step S2-8). Instead, the display status may be recorded in the keyword management record 220 of the keyword storage unit 22. In this case, the keyword in which the displayed flag is recorded is excluded from the keyword spotting process. Thereby, keyword spotting processing can be efficiently performed for a small number of keywords.

  In the above-described embodiment, the control unit 21 of the conference support server 20 executes a speech recording process (step S1-3). Here, the emotion of the speaker may be recorded. In this case, the control unit 21 is provided with emotion evaluation means. This emotion evaluation means holds an emotion evaluation table in which an emotion index is recorded in correspondence with an emotion pattern composed of evaluation results of rough breathing, intonation, voice itchiness, and sigh. Further, the conference support server 20 is provided with an utterance pattern storage unit that records the statistical value of the emotion evaluation result in the past utterance corresponding to the participant code. The speech history record 251 further includes a data area for recording the emotion index. This process will be described with reference to FIG.

First, the control part 21 of the meeting assistance server 20 performs the audio | voice acquisition process similarly to step S1-1 (step S7-1).
Next, the control unit 21 of the conference support server 20 executes an utterance state evaluation process (step S7-2). In the present embodiment, the emotion evaluation means of the control unit 21 performs evaluation of roughness of breathing, evaluation of inflection, evaluation of voice itch, and evaluation of sigh.

  In the evaluation of the roughness of breathing, the emotion evaluation means acquires the breathing sound collected by the microphone. And it compares with the roughness statistics of this participant's breathing. If it is determined that the breathing sound is larger than the statistical value recorded in the utterance pattern storage unit and the breathing is rough, an evaluation result indicating high emotion is acquired.

  In the evaluation of intonation, the emotion evaluation means calculates a change distribution (a magnitude of change) of the voice volume. Then, it compares with the inflection statistics of this participant. If the intonation is greater than the statistical value, an evaluation result indicating emotional height is obtained.

  In the evaluation of the voice itch, the emotion evaluation means calculates the intelligibility of the utterance. Then, it is compared with the clarity statistics value of this participant. When it is determined that the intelligibility is lower than the statistical value recorded in the utterance pattern storage unit and there is itchiness, an evaluation result indicating depression of mood is acquired.

  In the sigh evaluation, the emotion evaluation means identifies a breathing sound that matches the sigh pattern. Then, the emotion evaluation means counts the number of breath sounds that match the sigh pattern, and calculates the number of times per unit time (sigh frequency). Compare this participant's sigh frequency statistics. When the sigh frequency is higher than the statistical value recorded in the utterance pattern storage unit, an evaluation result indicating a feeling drop is acquired.

  Next, the control part 21 of the meeting assistance server 20 performs an emotion evaluation process (step S7-3). Specifically, the emotion evaluation unit of the control unit 21 acquires an emotion index corresponding to the emotion pattern of the evaluation result of the roughness of breathing, intonation, voice itchiness, and sigh from the emotion evaluation table.

  Next, the control unit 21 of the conference support server 20 executes emotion index recording processing in association with the content of the statement (step S7-4). Specifically, the emotion evaluation means of the control unit 21 records the calculated emotion index in the statement history record 251 regarding the participant's statement.

  As a result, the utterance status at that time can be grasped in the utterance history made into text.

  DESCRIPTION OF SYMBOLS 10 ... Client terminal, 20 ... Conference support server, 21 ... Control part, 211 ... Conference relay means, 212 ... Voice recognition means, 213 ... Message recording means, 214 ... Screen control means, 215 ... Keyword extraction means, 216 ... Progress management Means, 217 ... description display means, 218 ... summary creation means, 22 ... keyword storage section, 23 ... reference information storage section, 24 ... conference management information storage section, 25 ... speech management information storage section.

Claims (11)

Keyword storage means for recording relevant information for keywords;
Speech storage means for recording speech-recognized speech content;
A speech management system comprising a voice input device and a control means connected to a display device,
The control means is
Means for acquiring a voice of a speaker in the voice input device, creating voice-recognized voice content and recording it in the voice storage means;
Means for performing a keyword spotting process for comparing the statement text included in the statement content with the keyword stored in the keyword storage means;
A speech management system comprising: means for outputting related information associated with the keyword to the display device when the speech statement matches the keyword.   The statement management system according to claim 1, wherein an explanatory note about a keyword is recorded in the keyword storage unit as the related information. A user management means for recording user attribute information;
For each keyword, user attribute information is further recorded,
The control means is
From the user management means, obtain attribute information of the user of each display device,
3. The message management system according to claim 1, wherein a keyword corresponding to the attribute information is extracted from the keyword storage unit, and the keyword spotting process is performed using the extracted keyword. For the keyword for which the relevant information is output to the display device, the output information is recorded in the keyword storage means,
The speech management system according to any one of claims 1 to 3, wherein in the keyword spotting process, a keyword in which output information is not recorded is used. The control means is
For the keywords that output related information to the display device, count the number of utterances that included the utterance statement,
5. The speech management system according to claim 1, further comprising means for changing a display method of the related information of the keyword according to the number of speeches. In the keyword storage means, a keyword for specifying the progress of the agenda item is recorded,
6. The apparatus according to claim 1, further comprising means for erasing the agenda item output to the display device when the control means detects a match between a statement word and a keyword. The statement management system according to one.   7. The control device according to claim 1, further comprising means for creating a summary using the content of the speech recorded in the speech storage device and providing the summary to the display device. The statement management system described.   8. The message management system according to claim 7, wherein the control unit creates a summary of a message content range designated on the display device. The control means specifies the position of the statement word extracted by the keyword spotting process in the statement content,
Use the position of this statement to identify the summary scope,
The comment management system according to claim 7, wherein the comment contents in the summary target range are summarized. Keyword storage means for recording relevant information for keywords;
Speech storage means for recording speech-recognized speech content;
A speech management method using a speech management system including a voice input device and a control unit connected to a display device,
The control means is
Acquiring the voice of the speaker in the voice input device, creating the voice-recognized speech content and recording it in the speech storage means;
Performing a keyword spotting process for comparing a statement word included in the statement content with a keyword stored in the keyword storage unit;
And a step of outputting related information associated with the keyword to the display device when the statement word matches the keyword. Keyword storage means for recording relevant information for keywords;
Speech storage means for recording speech-recognized speech content;
A speech management system speech management program comprising a voice input device and a control means connected to a display device,
The control means;
Means for acquiring a voice of a speaker in the voice input device, creating a voice-recognized voice content and recording it in the voice storage means;
Means for performing a keyword spotting process for comparing the statement text included in the statement content with the keyword stored in the keyword storage means;
A statement management program that causes a function to output related information associated with a keyword to the display device when the statement word matches a keyword.

Images