JP6402447B2 - Support device, support method, and program - Google Patents

Description

  The present invention relates to a support device, a support method, and a program.

  Conventionally, in order to increase the efficiency of creating minutes, a system for supporting the creation of minutes by voice recognition processing and converting it into text has already been known.

  For example, Patent Document 1 describes a system including a memo editing unit that associates the progress of a conference with a conference video for the purpose of improving the efficiency of creating minutes. In this system, the voice during the meeting is associated with the memo, and the minutes in which the voice text data and the memo are associated can be created.

  However, in the conventional minutes creation support system, the organizer has grasped the schedule of proceedings, but it was difficult for the participants to grasp the overall flow of the discussion and the development of the current discussion during the meeting. It was. For this reason, in the conventional minutes preparation support system, discussions diverge, they do not meet the purpose, there are many redundant contents, and it takes time to create the minutes.

  The present invention has been made in view of the above, and provides a support device, a support method, and a program that can appropriately support the progress of a meeting and can reduce the time and effort of creating minutes. Objective.

In order to solve the above-described problems and achieve the object, the present invention analyzes a minutes template representing a template of an inputted minutes, and displays a meeting in a graph format having a meeting name and agenda of the meeting as nodes. A conference map generation unit for creating a map; a speech recognition unit for obtaining a speech recognition result in the conference; and determining a distance between words in the conference based on the speech recognition result; The conference map is updated according to the distance of the image and displayed on the image output unit, and when the discussion is divergent because the distance between the words is long, a notification that prompts the convergence of the discussion is displayed on the image output unit. A support device comprising: a conversation assistance control unit; and a minutes output unit that creates a minutes according to the minutes template based on the meeting map and the voice recognition result. To provide.

  ADVANTAGE OF THE INVENTION According to this invention, while progressing a meeting appropriately, there exists an advantageous effect that the effort of preparation of minutes can be made small.

FIG. 1 is a block diagram illustrating a functional configuration example of the support apparatus. FIG. 2 is a block diagram illustrating a detailed configuration example of the conversation assistance control unit. FIG. 3 is a diagram showing an example of a content list and a speech recognition result type list. FIG. 4 is a diagram illustrating an example of the result of labeling the type of content of speech from the speech recognition result. FIG. 5 is a flowchart illustrating an operation example of the support apparatus in the conference preparation stage. FIG. 6 is a diagram illustrating an example of a conference opening notification. FIG. 7 is a diagram illustrating an example of the target material. FIG. 8 is a diagram showing an example of the proceeding progress table. FIG. 9 is a flowchart illustrating an example of creating a conference map by the conference map generation unit. FIG. 10 is a diagram showing an example of a minutes template. FIG. 11 is a diagram illustrating an example of a minutes tree structure. FIG. 12 is a diagram illustrating an example of a conference map at the conference start stage. FIG. 13 is a flowchart illustrating an operation example of the support device during the conference. FIG. 14 is a diagram illustrating an example of conference progress status information and an example of speech recognition results. FIG. 15 is a flowchart illustrating an example of controlling the discussion content. FIG. 16 is a diagram illustrating an example of a part of a conference map in a state where discussions have converged. FIG. 17 is a diagram illustrating an example of a part of the conference map in a state where discussions are diverging and a notification example. FIG. 18 is a flowchart illustrating an example of conference time control. FIG. 19 is a diagram illustrating an example of a completed conference map. FIG. 20 is a flowchart illustrating an operation example of the support apparatus after the conference. FIG. 21 is a diagram showing an example of minutes. FIG. 22 is a diagram illustrating an example of association between conference progress status information and a completed conference map. FIG. 23 is a block diagram illustrating a functional configuration example of the support device according to the modification.

  Hereinafter, embodiments of a support device, a support method, and a program according to the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a functional configuration example of the support device 100. The support device 100 supports the progress of the conference and supports the creation of the minutes. The support device 100 includes an agenda input unit 10, a conference map generation unit 15, a voice recognition unit 20, a conversation assistance control unit 30, a storage unit 40, a minutes output unit 50, an image output unit 60, and a voice. And a combining unit 70. The support device 100 is realized by a computer including a projector or the like, for example. The support device 100 can be applied to, for example, a video conference system (UCS).

  The agenda input unit 10 analyzes the input meeting holding information and creates an agenda progress table corresponding to the agenda progress plan of the meeting. That is, at the conference preparation stage, the user sends conference holding information (conference holding notification) to the agenda input unit 10 in advance via the network. Then, the agenda input unit 10 determines from the meeting holding information whether the meeting can proceed, and if it can proceed, creates an agenda progress plan (procedure progress table), sends a notice of the meeting to each attendee, and cannot proceed If so, the organizer is instructed to review the contents of the conference.

  The agenda input unit 10 inputs a minutes template representing a template of minutes. The agenda input unit 10 sends the input minutes template to the meeting map generator 15 and receives the meeting map generated based on the minutes template from the meeting map generator 15.

  The conference map generation unit 15 generates a conference map based on the received minutes template. The conference map is information in a graph format in which item names such as the conference name and agenda of the conference are at least nodes, and each node is connected by a link.

  The voice recognition unit 20 acquires a voice recognition result in the conference. Further, the speech recognition unit 20 determines the type of a statement sentence based on the prosody information of the statement in the conference, and acquires a speech recognition result including the determination result. In other words, the speech recognition unit 20 uses the prosody or question based on the prosodic information (speech strength, intonation, rhythm, strength, weakness, high / low, etc.) of the participant's speech in the conference captured by the microphone (microphone). Judging sentences. The sentence type is a plain text, a question sentence, a command sentence, an exclamation sentence, a prayer sentence, an emphasized sentence, or the like.

  The conversation assistance control unit 30 monitors the progress of the conference from the proceeding plan (procedure progress table) and the speech recognition result, and updates the proceeding plan (procedure progress table) according to the progress. That is, during the conference, the conversation assistance control unit 30 monitors the proceedings based on the speech recognition results recognized by the speech recognition unit 20, and stores the speech recognition results and the conference progress information regarding the progress of the conference. Send to 40. Then, the conversation assisting control unit 30 reviews the proceeding table when the proceeding progresses early, and when the proceeding progress is delayed, the image output unit 60 in order to entrust the attendees to determine the proceeding. The decision information is output to and the proceeding table is reviewed. For example, the conversation assistance control unit 30 prompts the conference participants to determine the progress according to the progress, and reschedules the progress of the conference.

  In addition, the conversation assistance control unit 30 sends the conference map to the image output unit 60 to be output from the projector. Furthermore, in order to update the conference map according to the content of the discussion on each agenda, the conversation assistance control unit 30 sends a notification prompting the participants of the conference to input information to the image output unit 60 and outputs it from the projector. Let A conference participant generates a sound according to the notification. The conversation assistance control unit 30 updates the content of the conference map based on the voice recognition result of the voice generated according to the notification. Then, when the conference ends and the conference map is completed, the conversation assistance control unit 30 sends the completed conference map to the storage unit 40.

  The storage unit 40 stores a voice recognition result, conference progress status information, and a completed conference map. In addition, the storage unit 40 stores an agenda progress plan (agenda progress table). As a storage destination of information handled in the storage unit 40, for example, a nonvolatile storage device (for example, an auxiliary storage device such as a hard disk) in the support apparatus 100 can be cited.

  The minutes output unit 50 creates the minutes from the meeting progress information on the progress of the meeting, the voice recognition result, and the completed meeting map. That is, the minutes output unit 50 creates the minutes from the result of the meeting progress, the voice recognition result, and the completed meeting map. For example, after the meeting, the minutes output unit 50 creates the minutes based on the meeting progress status information stored (recorded) in the storage unit 40, the voice recognition result, and the completed meeting map. In addition, the minutes output unit 50 transmits the created minutes to the organizer. Thereby, meeting information can be recorded without the attendees writing.

  The image output unit 60 displays confirmation information corresponding to the progress status. That is, the image output unit 60 causes the projector to project confirmation information. The image output unit 60 receives confirmation information from the conversation assistance control unit 30 such as selecting whether to extend the current item or move to the next item.

  The image output unit 60 displays a conference map. Furthermore, the image output unit 60 displays a notification that prompts input of information for updating the conference map in accordance with the contents of the discussion of each agenda item.

  The voice synthesizer 70 synthesizes the confirmation information and outputs the voice. That is, the voice synthesizer 70 synthesizes the information (confirmation information) displayed by the image output unit 60 and outputs the voice from the speaker. The voice synthesizer 70 receives the same confirmation information as that sent from the conversation assistance controller 30 to the image output unit 60. For example, the speech synthesizer 70 converts text data “Do you want to extend the current item or move to the next item?” Into speech data and output from the speaker. This improves the efficiency of the conference progress.

  FIG. 2 is a block diagram illustrating a detailed configuration example of the conversation assistance control unit 30. As shown in FIG. 2, the conversation assistance control unit 30 includes a conversation labeler 31, a conference progress control unit 32, and a timer 33.

  The conversation labeler 31 (granting unit) gives (labeling, etc.) the type of comment content (opinion, question, answer, indication, etc.) to the speech recognition result. In addition, the conversation labeler 31 records the number of pages and the time of the target material of the conference corresponding to the speech of each speaker (conference participant) in the conference. The conversation labeler 31 receives the speech recognition result from the speech recognition unit 20 and receives the time from the timer 33. This saves the trouble of correcting the minutes.

  FIG. 3 is a diagram illustrating an example of a list of types of speech contents and a type list of speech recognition results (prosodic information). FIG. 4 is a diagram showing an example of the result of labeling the type of comment content (opinion, question, answer, indication, etc.) from the speech recognition result. The conversation labeler 31 holds in advance a list of types of speech contents and speech recognition results (plain text, question sentences, etc.) as shown in FIG. 4 is converted into the content type of the message, and the result as shown in FIG. 4 is output.

  Returning to FIG. 2, the description will be continued. The conference progress control unit 32 compares the proceeding plan (procedure progress table) with the conference progress status information. The conference progress control unit 32 compares the number of pages and time recorded by the conversation labeler 31 with the conference progress status information. This improves the quality of the target material. In addition, since the conference progress control unit 32 monitors the conference and updates the proceeding progress table according to the situation, the conference progress status information is output as the last updated proceeding progress table and its required time. . In addition, the proceeding table created at the time of meeting preparation is also stored in the storage unit 40 and compared with the required time of the meeting progress status information output after the meeting to confirm whether it is within the originally scheduled time. Is possible. That is, if it does not fit within the time, it is possible to judge that the amount of information of the target material is too large or insufficiently examined.

  The conference progress control unit 32 receives the speech recognition result and conference progress status information from the conversation labeler 31, receives the proceeding plan (procedure progress table) and the conference map from the agenda input unit 10, and sets the time from the timer 33. receive. In addition, the conference progress control unit 32 sends determination information (user instruction) to the image output unit 60 for entrusting the determination of progress to the participants of the conference.

  In addition, the conference progress control unit 32 sends the conference map to the image output unit 60 to be output from the projector. Furthermore, the conversation assistance control unit 30 causes the image output unit 60 to output a notification for prompting information input for updating the conference map. The conference progress control unit 32 updates the content of the conference map based on the voice recognition result of the voice generated according to the notification.

  Further, the conference progress control unit 32 sends the speech recognition result and the conference progress status information to the storage unit 40. In addition, the conference progress control unit 32 sends the completed conference map to the storage unit 40.

  The timer 33 monitors the allocated time. In addition, the timer 33 notifies the conversation labeler 31 or the conference progress control unit 32 that the allocated time has passed.

  Next, an operation example of the support apparatus 100 in the conference preparation stage will be described. FIG. 5 is a flowchart showing an operation example of the support apparatus 100 in the conference preparation stage.

  First, the agenda input unit 10 receives a conference opening notice and a minutes template from the organizer via the network (step S1). That is, at the conference preparation stage, the organizer sends a conference opening notice and a minutes template to the agenda input unit 10. FIG. 6 is a diagram illustrating an example of a conference opening notification. As shown in FIG. 6, the meeting opening notification has an analyzable data structure such as XML, and includes information on the organizer, date and time, participants, location, and the name or URL of the target material. .

  Returning to FIG. Next, the agenda input unit 10 analyzes the document of the target material from the meeting holding notification (step S2). That is, the agenda input unit 10 obtains a Word document or the like from the URL information of the target material included in the meeting holding notification. FIG. 7 is a diagram illustrating an example of the target material. As shown in FIG. 7, the target material also has an analyzable data structure such as XML, and includes information on chapters, sections, pages, the number of characters, and the number of figures and tables.

  Then, the agenda input unit 10 outputs chapters, sections, pages, the number of characters, and the number of figures and tables as analysis results of the document structure based on the information included in the target material. In addition, the agenda input unit 10 extracts information on the number of participants and the meeting time included in the meeting notification.

  Returning to FIG. Next, the agenda input unit 10 creates an agenda progress table based on the number of participants included in the meeting opening notice and the meeting time (step S3). In other words, the agenda input unit 10 analyzes the meeting holding notice and the document structure of the target material, and creates an agenda progress table. Here, the agenda input unit 10 distributes the meeting time based on the analysis result of the document structure of the target material. For example, the agenda input unit 10 assigns a weight based on the number of characters and the number of figures and tables, multiplies the number of analysis results, and assigns a meeting time based on the ratio.

  FIG. 8 is a diagram showing an example of the proceeding progress table. As shown in FIG. 8, the proceeding table is created by analyzing the document structure of the meeting holding notice and the target material, and includes an allocation time for each item of the target material.

  Returning to FIG. Next, the agenda input unit 10 determines whether or not the proceeding of the proceedings is possible (step S4). If the agenda input unit 10 determines that the proceeding is possible (result of step S4: Yes), the agenda input unit 10 sends the minutes template to the conference map generation unit 15 (step S5). Next, the agenda input part 10 receives the meeting map produced | generated based on the minutes template from the meeting map production | generation part 15 (step S6). Then, the agenda input unit 10 sends the conference map and the proceeding progress table to the conversation assistance control unit 30 (step S7), and ends the process. That is, if the proceedings can be proceeded, the agenda input unit 10 sends a conference map generated based on the minutes template to the conversation assistance control unit 30 and also uses the information indicating the allocated time as a proceeding table to assist the conversation. The data is sent to the control unit 30.

  On the other hand, if the agenda input unit 10 determines that the proceeding is not possible (result of step S4: No), the agenda input unit 10 informs the user that the conference does not hold if the input conference holding notification is input (step S8), and processing Exit. That is, the agenda input unit 10 determines that the proceedings are impossible if the allocated time is insufficient as compared to the time of the participants, and informs the organizer that the conference cannot be held. .

  Next, a method for creating a conference map by the conference map generator 15 at the conference preparation stage will be described. FIG. 9 is a flowchart illustrating an example of creating a conference map by the conference map generation unit 15.

  First, the conference map generation unit 15 receives a minutes template from the agenda input unit 10 (step S11). FIG. 10 is a diagram showing an example of a minutes template. The minutes template is, for example, a document file in which item names are described in text. In the minutes template, as an example, the item names are broken for each line, and each item is distinguished by a hierarchical structure.

  In the example of FIG. 10, the minutes template includes “meeting name”, one or more “agenda”, “discussion contents” for each agenda, “conclusion” for each agenda, “reason” for conclusion, and for each agenda. “Determination items”, “results” of the conference, “action items” determined by the conference, and the like are described as item names. In this minutes template, the hierarchical structure of each item name is represented by the number of steps down. The “conference name” with 0 step down is the item name of the highest hierarchy. “Agenda”, “Result”, and “Action Item” with one step down are item names in the second hierarchy. “Discussion contents”, “conclusion”, and “decision items” with two steps down are item names in the third hierarchy. The “reason” with three step downs is the item name of the fourth layer. In the case of a document file in which text is written, the step-down may be distinguished by a blank or a tab, and if it is an XML file, it may be distinguished by a tag.

  Returning to FIG. 9, the description will be continued. Next, the conference map generation unit 15 reads the item names of the minutes template one by one from the top in order, and executes the loop processing from step S12 to step S17 for each item name. For example, the conference map generation unit 15 reads the text line by line from the first line in the minutes template of the document file in which the item name is described in text.

  In the loop process, first, the conference map generation unit 15 checks the hierarchy of the read item name (step S13). In this example, the conference map generation unit 15 confirms the number of layers from the number of steps down. Next, the conference map generator 15 determines whether or not it is the highest hierarchy (step S14). For example, the conference map generation unit 15 determines whether or not the number of steps down in the read item name is zero. In the case of the highest hierarchy, that is, when the number of steps down is 0 (result of step S14: Yes), the conference map generation unit 15 adds the read item name to the minutes tree structure as a parent node. (Step S15). If it is not the highest hierarchy, that is, if the number of steps down is not 0 (result of step S14: No), the conference map generation unit 15 assigns the read item name to the next higher hierarchy. Is added to the minutes tree structure as a child node below the node of the item name read in (step S16).

  Then, when the reading of all item names of the minutes template is completed, the conference map generation unit 15 ends the loop processing from step S12 to step S17. FIG. 11 shows an example of the minutes tree structure after all item names described in the minutes template have been read. The minutes tree structure of this example has a meeting name as a parent node, and has a first agenda item, a second agenda item, a third agenda item, a result, and an action item as a child node immediately below the parent route. Also, the minutes tree structure of this example has the content, conclusion, and decision items of the discussion as child nodes immediately below the first agenda node, and has a reason as a child node immediately below the conclusion node.

  Returning to FIG. 9, the description will be continued. Next, the conference map generation unit 15 generates a conference map from the minutes tree structure (step S18). The conference map is information in a graph format in which each item name is a node and each node is connected by a link according to a hierarchical structure. FIG. 12 is a diagram illustrating an example of a conference map at the conference start stage. For example, the conference map is displayed in the form of a mind map as shown in FIG. In addition, the link may not be described with the curve like a mind map, but may be described with the straight line.

  Returning to FIG. 9, the description will be continued. Next, the conference map generation unit 15 sends the generated conference map to the agenda input unit 10 as a conference map at the start stage (step S19). If the conference name and agenda are determined in advance, the conference map generation unit 15 adds a specific character string to the conference name and the agenda on the conference map, and sends them to the agenda input unit 10. Also good.

  Next, an operation example of the support apparatus 100 during the meeting will be described. FIG. 13 is a flowchart illustrating an operation example of the support apparatus 100 during a conference.

  First, when the conference is started, the conference progress control unit 32 of the conversation assistance control unit 30 determines whether or not the conference name and the agenda are already input in the conference map (step S31). When the conference name and the agenda have not been input in advance (result of step S31: No), the conference progress control unit 32 sends a notification prompting the input of the conference name and the agenda from the projector via the image output unit 60. Output (step S32). When a notification prompting the user to input a conference name and agenda is given, a conference participant (host) speaks the conference name and agenda in accordance with the notification. Next, the conversation labeler 31 extracts the meeting name and the text information of the agenda from the speech of the meeting participant from the speech recognition result received from the voice recognition unit 20, and the extracted meeting name and the text information of the agenda are stored in the meeting map. Is added to the corresponding node (step S33).

  When the process of step S33 is completed, or when the meeting name and the agenda have been input in advance (result of step S31: Yes), the meeting progress control unit 32 advances the process to step S34.

  Next, the conference progress control unit 32 determines whether there is a next agenda based on the agenda progress table, and specifies the agenda to be advanced (step S34). When the conference is started, the conference progress control unit 32 specifies the first agenda in the agenda progress table.

  When the conference progress control unit 32 determines that there is the next agenda (result of step S34: Yes), the process proceeds to step S35, and when it is determined that there is no next agenda (result of step S34: No) ), The process proceeds to step S45.

  When it is determined that there is the next agenda (result of step S34: Yes), the conversation assistance control unit 30 continues to execute the processes of step S35, step S36, step S37, and step S38 until the discussion ends. In step S <b> 35, the conversation labeler 31 of the conversation assistance control unit 30 determines the number of pages, chapters, or sections of the target material of the conference participant (host) from the speech recognition result received from the speech recognition unit 20. Extract keywords.

  Here, with respect to audio data exchanged during the conference, in order to identify a speaker (for example, A or B) when the audio flies, the voice recognition unit 20 extracts the characteristics of the voice input from the microphone. Then, it is specified by creating a feature model for A or B. Alternatively, a plurality of microphones may be prepared, and the voice recognition unit 20 may identify who is a speaker (participant of the conference) depending on the distance that the sound reaches. In addition, since each individual has a microphone, the speech recognition unit 20 can identify a speaker with higher accuracy.

  In addition, the speech recognition unit 20 prepares a feature model for the organizer in advance to determine the organizer's remarks. If the speech input from the microphone is modeled and matches the feature model, the organizer Judge. Moreover, a microphone for the organizer may be prepared in advance, and the voice recognition unit 20 may determine that the host is an organizer if the input from the microphone is large.

  In step S36, the conversation labeler 31 holds the page number, chapter or section keyword, and the time obtained from the timer 33 as conference progress information (step S36). That is, the conversation labeler 31 holds the keywords extracted from the speech recognition result in the form of conference progress information. FIG. 14 is a diagram illustrating an example of conference progress status information and an example of speech recognition results. As shown on the left side of FIG. 14, the conference progress status information is configured by associating the proceeding table and the speech time (required time) of the participants of the conference. Further, as shown on the right side of FIG. 14, the voice recognition result and the proceeding table are associated with corresponding items.

  In step S37, the conference progress control unit 32 controls the content of the discussion. Specifically, the conference progress control unit 32 executes the flow shown in FIG. That is, the conference progress control unit 32 repeatedly executes the processing from step S62 to step S67 until the discussion of the agenda ends (step S68) after the discussion of the agenda starts (step S61).

  In step S62, the conference progress control unit 32 plots the keyword nodes extracted by the speech recognition under the corresponding discussion content node of the conference map. In this case, the conference progress control unit 32 measures the distance between words and accumulates the measured distance. The distance between words represents the closeness of the meaning of the word, and is short when the meaning is synonymous and long when the meaning is opposite. As an example, the conference progress control unit 32 refers to a table representing the distance between words included in a dictionary prepared in advance, and calculates the distance between words.

  In this case, for example, the conference progress control unit 32 plots keywords having a short distance between words on the conference map so as to be located under the same node, and positions keywords having a long distance between words under different nodes. To plot on the conference map. Therefore, when many keywords having similar meanings are spoken (that is, when the discussion has converged), as shown in FIG. 16, the hierarchical structure of the keywords plotted on the conference map becomes deeper (for example, The number of hierarchies is greater than a predetermined number). In addition, when many keywords having different meanings are remarked (that is, when discussions are diverged), as shown in FIG. 17, the number of keyword hierarchies plotted on the conference map becomes shallow (for example, hierarchies). The number is less than a predetermined number).

  Next, in step S63, the conference progress control unit 32 determines whether or not it is time to converge the discussion, that is, whether or not a predetermined proportion of the minutes allocated in the minutes progress table has elapsed ( For example, it is determined whether or not 80% of the allocated time of the minutes progress table has elapsed. When it is not time to converge the discussion (result of Step S63: No), the conference progress control unit 32 determines in Step S64 whether or not there are many nodes with close distances between words, that is, the discussion converges. Judge whether or not. If the discussion has converged (result of step S64: Yes), in step S65, the conference progress control unit 32 adds a new word node whose distance between words is far from the dictionary to the conference map. For example, in the example of FIG. 16, a new node “FAX function” is added. By automatically adding a new word node, the conference progress control unit 32 can prompt the conference participants to discuss from a new viewpoint. If the discussion has not converged (the result of step S64: No), the conference progress control unit 32 performs no processing.

  When it is time to converge the discussion (result of step S63: Yes), the conference progress control unit 32 determines whether or not there are many nodes far from each other in step S66, that is, the discussion diverges. Judge whether or not. If the discussion is diverging (result of step S66: Yes), in step S67, the conference progress control unit 32 displays, for example, a notification that prompts the convergence of the discussion. For example, in the example of FIG. 17, the notification “The allocation time will soon be finished, so please converge the discussion” is displayed. By displaying the notification that prompts the convergence of the discussion, the conference progress control unit 32 can control the progress time so that the discussion is converged within the allocated time. If the discussion has not diverged (result of step S66: No), the conference progress control unit 32 performs no processing.

  The conference progress control unit 32 repeatedly executes the above processing until the discussion ends while periodically checking the timer 33 (steps S61 to S68). In this way, the conference progress control unit 32 can control the content of the discussion.

  Returning to FIG. In step S38, control of the conference progress time is executed. Specifically, the conference progress control unit 32 executes the processing from step S71 to step S80 shown in FIG.

  First, in step S71, the conference progress control unit 32 receives a request for monitoring the time of the next item (next agenda) from the conversation labeler 31, or has passed the allocated time of the item (the agenda) from the timer 33. Get notified. Next, in step S72, the conference progress control unit 32 compares the proceeding progress table with the current time, and determines whether or not the proceeding progress is early.

  If it is determined that the proceedings are equal (result of step S72: Match), the conference progress control unit 32 ends the processing of this flow without performing any processing.

  When it is determined that the proceedings are fast (result of step S72: Yes), in step S73, the conference progress control unit 32 redistributes the spare time to the items following the proceedings and updates the proceedings Then, the process of this flow is finished.

  When it is determined that the proceedings are slow (result of step S72: No), in step S74, the conference progress control unit 32 extends the discussion of the current item to the participants of the conference or moves to the discussion of the next item. Confirmation information for confirming is sent to the image output unit 60. Next, in step S75, the conference progress control unit 32 determines whether to extend the current item or move to the next item from the voice recognition result. If it is determined to move to the next item (result of step S75: No), the conference progress control unit 32 ends the process of this flow.

  If it is determined that the current item is to be extended (result of step S75: Yes), in step S76, the conference progress control unit 32 assigns the extended time to the proceeding table for the current item, and re-assigns the allocated time for the next item and thereafter. Configure and update the agenda. Next, in step S77, the conference progress control unit 32 determines whether or not the allocation time of the proceeding table can be reconfigured. When it is determined that the allocation time can be reconfigured (result of step S77: Yes), the conference progress control unit 32 returns the process to step S71 and repeats the process from step S72.

  When it is determined that the allocation time cannot be reconfigured (result of step S77: No), in step S78, the conference progress control unit 32 eliminates the allocation time after the next item for the conference participant, and the current item. Confirmation information for confirming whether to continue or to move to the next item is sent to the image output unit 60. Next, in step S79, the conference progress control unit 32 determines whether to extend the current item or move to the next item from the voice recognition result. If it is determined to move to the next item (result of step S79: No), the conference progress control unit 32 ends the process of this flow.

  When it is determined that the current item is to be extended (result of step S79: Yes), in step S80, the conference progress control unit 32 reconfigures the allocated time after the next item and does not implement items that do not fall within the time. The agenda progress table is updated, the process returns to step S71, and the process is repeated from step S72. Thus, the conference progress control unit 32 can control the conference progress time.

  Returning to FIG. In step S39, the conference progress control unit 32 determines whether or not the discussion on the subject has been completed. For example, the conference progress control unit 32 determines that the discussion on the agenda has ended when a statement such as “end the discussion on the agenda” is made. When the discussion on the subject has been completed (result of step S39: Yes), the conference progress control unit 32 advances the process to step S40.

  In step S <b> 40, the conference progress control unit 32 sends the speech recognition result and the conference progress status information during the discussion of the agenda to the storage unit 40 for storage.

  Next, in step S <b> 41, the conference progress control unit 32 causes the image output unit 60 to output a notification urging input of the agenda conclusion. The conference participants are prompted by this notification and speak the agenda conclusion. Next, in step S42, the conference progress control unit 32 extracts the text of the agenda conclusion from the speech recognition result of the statement, and adds it to the corresponding node of the conference map.

  Next, in step S <b> 43, the conference progress control unit 32 causes the image output unit 60 to output a notification that prompts the user to input agenda items. The conference participants are prompted by this notification and speak the agenda decisions. Next, in step S44, the conference progress control unit 32 extracts the text of the agenda decision item from the speech recognition result of the speech and adds it to the corresponding node of the conference map.

  And conference progress control part 32 returns processing to Step S34, after finishing processing of Step S44. In step S34, the conference progress control unit 32 determines whether or not there is a next agenda. If there is a next agenda, the process from step S35 is repeatedly executed.

  On the other hand, when it is determined that there is no next agenda (result of step S34: No), that is, when the discussion on all the agendas ends and the conference ends, the conversation assistance control unit 30 performs the processing step Proceed to S45.

  In step S <b> 45, the conference progress control unit 32 causes the image output unit 60 to output a notification urging input of the conference result. The conference participant is prompted by this notification and speaks the result of the conference. Next, in step S46, the conference progress control unit 32 extracts the text of the conference result from the speech recognition result of the statement, and adds it to the corresponding node of the conference map.

  Next, in step S <b> 47, the conference progress control unit 32 causes the image output unit 60 to output a notification urging input of a conference action item. The conference participant is prompted by this notification and speaks the conference action item. Next, in step S48, the conference progress control unit 32 extracts the text of the conference action item from the speech recognition result, and adds it to the corresponding node of the conference map. By completing the processing up to step S48, the conference map is completed. For example, a conference map as shown in FIG. 19 is completed.

  Finally, in step S49, the conference progress control unit 32 sends the completed conference map to the storage unit 40, stores it, and ends the processing of this flow. As described above, the support apparatus 100 can appropriately control the content of the discussion during the conference and can control the progress time so that the conference proceeds as scheduled.

  Next, an example of the operation of the support apparatus 100 after the meeting (preparation of minutes) will be described. FIG. 20 is a flowchart illustrating an operation example of the support apparatus 100 after the conference.

  First, in step S91, the minutes output unit 50 receives the conference progress information, the voice recognition result, and the completed conference map from the storage unit 40 after the conference. That is, after the conference, the storage unit 40 sends the conference progress status information, the voice recognition result, and the completed conference map to the minutes output unit 50.

  Next, in step S92, the minutes output unit 50 creates the minutes. That is, the minutes output unit 50 shapes the conference progress information and the speech recognition result associated with each agenda, deletes the allocated time and the required time information, and further, the content of the conference map, the conference opening notification, In addition, the minutes are output.

  FIG. 21 is a diagram showing an example of minutes. As shown in FIG. 21, the minutes output unit 50 includes the conference progress information from which the information on the allocated time and the required time is deleted, the speech recognition result, the conclusion, reason, decision items for each agenda, and the conference Create minutes containing results and action items.

  FIG. 22 is a diagram illustrating an example of association between conference progress status information and a completed conference map. Further, as shown in FIG. 22, the minutes output unit 50 generates and stores link information that associates the conference map, each node, and the corresponding location of the minutes. For example, the minutes output unit 50 generates link information associating the discussion content node of the conference map with the corresponding conference progress information. This makes it possible to refer to the meeting map and the minutes together. For example, if a person who has not attended the meeting clicks on the reasoning statement for a person who has not attended the meeting, the minutes can be efficiently viewed. Can be confirmed.

  Returning to FIG. 20, the description will be continued. Next, in step S93, the minutes output unit 50 transmits the minutes to the organizer via the network and ends the process. That is, the minutes output unit 50 creates the minutes from the received information and transmits the minutes to the organizer.

  As described above, according to the support device 100 according to the present embodiment, by creating a conference map, it is possible to appropriately support the progress of the conference and reduce the time and effort of creating the minutes.

  Next, the support device 100 according to the modification will be described. FIG. 23 is a block diagram illustrating a functional configuration example of the support device 100 according to the modification.

  As illustrated in FIG. 23, the support device 100 may further include a map operation recognition unit 110, a conference comparison unit 120, and an anonymization processing unit 130.

  The map operation recognition unit 110 accepts an operation of associating each node of the conference map with a corresponding portion of the minutes by a conference participant (user). For example, the map operation recognizing unit 110 operates the conference map and the minutes projected on the projector to perform the association operation. Then, the minutes output unit 50 creates link information according to the operation received by the map operation recognition unit 110. This makes it possible to associate the conference map with the minutes more accurately.

  The meeting comparison unit 120 inputs past minutes, for example, via a network or the like. The meeting comparison unit 120 compares the meeting agenda shown in the meeting holding information input by the agenda input unit 10 with the past minutes, and the meeting agenda shown in the meeting holding information is Determine if it has been discussed. The meeting comparison unit 120 notifies the participants of the meeting about the agenda items discussed in the past meeting. Thereby, the meeting comparison part 120 can eliminate the discussion about the duplicate agenda and can save the waste of the meeting.

  The anonymization processing unit 130 reads the voice recognition result stored in the storage unit 40, anonymizes the speaker so that the speaker cannot be specified, and sends it to the minutes output unit 50. In this case, when an audio file is stored in the storage unit 40, the anonymization processing unit 130 can change the sound quality of the audio file and understand the content of the story but cannot specify who is speaking. Like that. Thereby, the anonymization processing unit 130 can maintain security without specifying a speaker.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to each above-mentioned embodiment, A various deformation | transformation is possible within the range which does not deviate from the summary of this invention. Moreover, each above-mentioned embodiment can also be combined arbitrarily.

  Note that the support apparatus 100 of the above-described embodiment can also be realized by using, for example, a general-purpose computer apparatus as basic hardware. That is, each storage unit 40 described above is realized by a hardware circuit. On the other hand, each of the agenda input unit 10, conference map generation unit 15, speech recognition unit 20, conversation assistance control unit 30, minutes output unit 50, image output unit 60, and speech synthesis unit 70 is connected to the computer device described above. This can be realized by an on-board CPU executing a program stored in a storage device such as a ROM or a RAM. The agenda input unit 10, conference map generation unit 15, speech recognition unit 20, conversation assistance control unit 30, minutes output unit 50, image output unit 60, and speech synthesis unit 70 are each an HDD, a CD drive device. This can be realized by executing a program stored in an external storage device. For example, the functions of the agenda input unit 10, the conference map generation unit 15, the speech recognition unit 20, the conversation assistance control unit 30, the minutes output unit 50, the image output unit 60, and the speech synthesis unit 70 are described above. At least a part of them may be realized by a hardware circuit.

  The program executed by the support apparatus 100 of the above-described embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk). The information may be provided by being recorded on a recording medium that can be read by the user. Furthermore, the program executed by the support apparatus 100 according to the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program executed by the support apparatus 100 of the above-described embodiment may be configured to be provided or distributed via a network such as the Internet.

  The program executed by the support device 100 of the present embodiment includes the above-described units (the agenda input unit 10, the conference map generation unit 15, the speech recognition unit 20, the conversation assistance control unit 30, the minutes output unit 50, the image The module configuration includes an output unit 60 and a speech synthesis unit 70). As actual hardware, a CPU (processor) reads out and executes a program from the storage medium, and the respective units are loaded onto the main storage device. The agenda input unit 10, the conference map generation unit 15, the speech recognition unit 20, the conversation assistance control unit 30, the minutes output unit 50, the image output unit 60, and the speech synthesis unit 70 are generated on the main storage device. It may be.

  Further, the support device 100 has the functions of the above-described conversation assistance control unit 30 (conversation labeler 31, conference progress control unit 32, timer 33), and other functions (the agenda input unit 10, the conference map generation unit 15, voice) The recognition unit 20, the storage unit 40, the minutes output unit 50, the image output unit 60, and the voice synthesis unit 70) may be mounted on an external server device that can be connected to the support device 100. The number of external server devices may be one or more. For example, functions other than the conversation assistance control unit 30 (the agenda input unit 10, the voice recognition unit 20, the storage unit 40, the minutes output unit 50, the image output unit 60, the voice The combining unit 70) may be distributed and mounted on two or more external devices.

10 Agenda Input Unit 20 Speech Recognition Unit 30 Conversation Auxiliary Control Unit 31 Conversation Labeler 32 Conference Progress Control Unit 33 Timer 40 Storage Unit 50 Minutes Output Unit 60 Image Output Unit 70 Speech Synthesis Unit 100 Support Device 110 Map Operation Recognition Unit 120 Conference Comparison Part 130 Anonymization processing part

JP 2006-268800 A

Claims (8)

A meeting map generation unit that analyzes a minutes template that represents a template of the input minutes and creates a meeting map in a graph format including at least the meeting name and the agenda of the meeting as a node;
A voice recognition unit for obtaining a voice recognition result in the conference;
The distance between words in the conference is determined based on the speech recognition result, the conference map is updated according to the word in the conference and the distance between the words and displayed on the image output unit, and the distance between the words If the discussion is divergent due to being far away, a conversation assistance control unit that causes the image output unit to display a notification that prompts the convergence of the discussion, and
A minutes output unit for creating minutes according to the minutes template based on the meeting map and the voice recognition result;
A support device comprising: The support apparatus according to claim 1, wherein the minutes output unit further creates link information that associates each node of the meeting map with a corresponding portion of the minutes. A map operation recognizing unit that accepts an operation of associating each node of the conference map with a corresponding portion of the minutes by the user;
The support apparatus according to claim 2, wherein the minutes output unit further creates the link information according to an operation received by the map operation recognition unit. The support apparatus as described in any one of Claim 1 to 3 further provided with the anonymization process part which anonymizes the said speech recognition result so that a speaker cannot be specified. The support apparatus as described in any one of Claim 1 to 4 further provided with the meeting comparison part which compares the agenda of the said meeting and the past minutes. It further includes an agenda input unit that analyzes the conference information that has been input and creates an agenda that corresponds to the agenda of the conference,
The conversation assistance control unit monitors the progress of the conference from the proceeding progress table and the voice recognition result, and updates the proceeding progress table according to the progress status,
The support apparatus according to any one of claims 1 to 5, wherein the minutes output unit creates a minutes from the meeting map, meeting progress information regarding the progress, and the voice recognition result. A meeting map generation step of analyzing a meeting minutes template representing a template of the inputted meeting minutes and creating a meeting map in a graph format having a meeting name and agenda of the meeting as nodes at least;
A voice recognition step of obtaining a voice recognition result in the conference;
The distance between words in the conference is determined based on the speech recognition result, the conference map is updated according to the word in the conference and the distance between the words and displayed on the image output unit, and the distance between the words If the discussion is divergent due to being far away, a conversation assistance control step for causing the image output unit to display a notification for promoting the convergence of the discussion ;
Minutes output step of creating a minutes according to the minutes template based on the meeting map and the voice recognition result;
Support method including.   The program for functioning a computer as an assistance apparatus as described in any one of Claim 1 to 6.

Images