JP2018092365A - Information processing system, information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more appropriately support creation of a minute book by a simpler system.SOLUTION: In a minute book creation support system 1, a mobile terminal 10 transmits, to an integration server, statement information which is obtained by converting a statement made by a conference participant into text data. The integration server 30 integrates the conference participant's statement information transmitted from the mobile terminal 10 in time sequence. At an editor terminal 40, a UI display control unit 312 displays a display screen including a first display region where the statement information integrated in time sequence by the integration server 30 is displayed and a second display region which is a movement destination for an operation of moving the statement information displayed in the first display region and which is associated with processing of adding statement information to a minute book. When the operation of moving the statement information displayed in the first display region to the second display region is performed, a UI input control unit 313 executes the processing of adding the statement information to data of the minute book.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program.

Conventionally, in order to reduce the labor for creating the minutes of a meeting, a system that supports the creation of the minutes of a meeting has been proposed.
In such a system, by using voice recognition, a method is adopted in which all the comments of participants in the meeting are converted into text and recorded, or important comments in the meeting are selected and recorded.
For example, in Patent Document 1, a state in which a conversation between conference participants is established is detected, and based on the detection result, specific speech is extracted from a recording of speech uttered by a conference participant, and the agenda A minutes creation support system for creating minutes is described.

JP 2013-105374 A

However, in the conventional technology that supports the creation of the minutes of the conference, including the technology described in Patent Document 1, a dedicated large-scale facility such as a microphone system for distinguishing and inputting participants' comments is required. There is. In addition, depending on the algorithm of the system, the accuracy of the process of summarizing the contents of the meeting is not necessarily high, and there is a possibility that the minutes of inappropriate contents are created. In particular, when creating a minutes that summarizes the contents of a meeting in parallel with the meeting, it is necessary to select the appropriate remarks in real time and reflect them in the minutes. It becomes difficult.
As described above, in the conventional technology, it is difficult to appropriately support the creation of the minutes by a simple system.

  An object of the present invention is to more appropriately support the creation of minutes by a simpler system.

In order to achieve the above object, an information processing system according to one embodiment of the present invention includes:
A terminal device used by a conference participant, an integrated server that integrates statements made by the conference participant, and an information processing device that creates minutes of the conference,
The terminal device transmits the speech information in which the speech of the participant of the conference is text data to the integrated server,
The integration server integrates the speech information of the conference participants transmitted from the terminal device in time series,
The information processing apparatus includes:
A first display area for displaying the message information integrated in time series by the integration server; and an area to which the operation for moving the message information displayed in the first display area is performed. Display control means for displaying a display screen including a second display area associated with the process of including information in the minutes;
Processing means for executing processing for adding the comment information to the minutes data when an operation of moving the comment information displayed in the first display area to the second display area is performed;
It is characterized by providing.

  According to the present invention, creation of minutes can be more appropriately supported with a simpler system.

It is a figure which shows the system configuration | structure of the minutes creation assistance system which concerns on this invention. It is a figure which shows the hardware constitutions of the information processing apparatus which comprises a portable terminal, a speech recognition server, an integrated server, and an editor terminal. It is a block diagram which shows the function structure of a portable terminal. It is a block diagram which shows the function structure of an integrated server. It is a block diagram which shows the function structure of an editor terminal. It is a schematic diagram which shows an example of a minutes edit screen. It is a flowchart which shows the flow of the meeting participation process which the portable terminal of a minutes creation assistance system performs. It is a flowchart which shows the flow of the integration process which the integrated server of a minutes creation assistance system performs. It is a flowchart which shows the flow of the minutes edit process which the editor terminal of the minutes creation assistance system performs. It is a schematic diagram which shows the other example of a minutes edit screen. It is a schematic diagram which shows the other example of a minutes edit screen.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Constitution]
[System configuration]
FIG. 1 is a diagram showing a system configuration of a minutes creation support system 1 according to the present invention.
The minutes creation support system 1 is configured as an embodiment of an information processing system according to the present invention, and more appropriately supports the creation of minutes by summarizing the contents of a meeting with a simple configuration system. System.

As shown in FIG. 1, the minutes creation support system 1 includes a plurality of mobile terminals 10, a voice recognition server 20, an integration server 30, and an editor terminal 40, which are connected via a network 50. Thus, they can be communicated with each other.
The portable terminal 10 is used by a conference participant, and accepts a participant's remarks (speech input) for the conference for which the minutes are to be created by executing a conference participation process described later. Moreover, the portable terminal 10 transmits the voice of the participant's speech to the voice recognition server 20 and receives text data of the voice recognition result. Then, the mobile terminal 10 transmits the speech of the speech that is the target of speech recognition and the speech recognition result to the integrated server 30.

  The voice recognition server 20 recognizes the voice of the speech accepted by the mobile terminal 10 and sequentially transmits text data of the voice recognition result to the mobile terminal 10. As the voice recognition server 20, it is possible to use an existing server having a voice recognition function or to prepare the voice recognition server 20 exclusively for the minutes creation support system 1.

  The integration server 30 integrates the voice recognition results of the utterances transmitted from the plurality of mobile terminals 10 by arranging them in time series by executing an integration process described later, and sequentially transmits them to the editor terminal 40. Further, the integrated server 30 stores the minutes data transmitted from the editor terminal 40 in a database together with a series of participants' voices transmitted from the plurality of portable terminals 10 and the voice recognition results thereof.

  The editor terminal 40 displays a speech recognition result (text data) of the speech integrated by the integration server 30 by executing a minutes editing process described later, and a user interface (hereinafter, referred to as “editing”). This will be referred to as a “minutes edit screen” as appropriate. On the minutes editing screen, the editor terminal 40 displays the speech recognition result of each participant's utterance transmitted sequentially from the integrated server 30 on a timeline, and accepts an operation on the text data of the utterance displayed on the timeline.

  Specifically, the editor terminal 40 displays an area to be dragged and dropped (hereinafter referred to as “minutes area”) in order to leave the text data of the statement in the minutes on the minutes edit screen, and the time. When the text data of a message displayed in a line is dragged and dropped into the minutes area, the text data of the message is reflected in the data of the minutes. In addition, the editor terminal 40 accepts an operation for adding an important mark indicating that the editor has determined that the message is important to the text data of the message displayed on the timeline on the minutes editing screen. By adding an important mark, it is possible to easily extract an utterance judged to be important later. In addition, adding an important mark can help to summarize the contents of the meeting. Further, the editor terminal 40 transmits a list of text data of the remarks dragged and dropped to the minutes area to the integrated server 30 as minutes data.

  In the present embodiment, in addition to the minutes area, a predetermined area corresponding to the attribute of the statement content (hereinafter referred to as “attribute-specific area”) is displayed on the minutes edit screen. The text data of an utterance whose content is determined to be a specific attribute is appropriately dragged and dropped into an attribute-specific area corresponding to the attribute of the utterance content. As the attribute of the content of remarks, for example, various matters to be determined in the conference such as a task imposed on the participant by the conference and a schedule determined by the conference are set. When the text data of an utterance is dragged and dropped in the attribute-specific area, processing corresponding to the attribute of the utterance content is automatically performed. For example, the assignment to be imposed on the participant is recorded in the minutes, and is registered as a ToDo list on the personal page of the participant who has been assigned the assignment in the groupware system.

[Hardware configuration]
Next, the hardware configuration of the mobile terminal 10, the speech recognition server 20, the integrated server 30, and the editor terminal 40 of the minutes creation support system 1 will be described.
The mobile terminal 10, the voice recognition server 20, the integration server 30, and the editor terminal 40 are configured by an information processing device. For example, the mobile terminal 10 is a smartphone, the voice recognition server 20 and the integration server 30 are server computers, and the editor terminal 40 is an editor terminal 40. It can be configured by a PC (Personal Computer).
FIG. 2 is a diagram illustrating a hardware configuration of the information processing apparatus 800 that constitutes the mobile terminal 10, the voice recognition server 20, the integration server 30, and the editor terminal 40.
As illustrated in FIG. 2, the information processing apparatus 800 includes a CPU (Central Processing Unit) 811, a ROM (Read Only Memory) 812, a RAM (Random Access Memory) 813, a bus 814, an input unit 815, and an output A unit 816, a storage unit 817, a communication unit 818, and a drive 819.

The CPU 811 executes various processes according to a program recorded in the ROM 812 or a program loaded from the storage unit 817 to the RAM 813.
The RAM 813 appropriately stores data necessary for the CPU 811 to execute various processes.

  The CPU 811, ROM 812, and RAM 813 are connected to each other via a bus 814. An input unit 815, an output unit 816, a storage unit 817, a communication unit 818, and a drive 819 are connected to the bus 814.

The input unit 815 is configured with various buttons and the like, and inputs various information according to an instruction operation. The input unit 815 includes a microphone for voice input, and converts the input voice into an electrical signal.
The output unit 816 includes a display, a speaker, and the like, and outputs an image and sound.
The storage unit 817 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various data managed by each server.
The communication unit 818 controls communication performed with other devices via a network.

A removable medium 821 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 819. The program read from the removable medium 821 by the drive 819 is installed in the storage unit 817 as necessary.
In addition, when each apparatus is specifically configured with respect to such a basic configuration of the information processing apparatus 800, additional hardware can be appropriately provided. For example, when the mobile terminal 10 is configured, an imaging unit that captures a digital image can be provided. When the mobile terminal 10 is configured, a touch sensor can be provided as the input unit 815, and a touch panel can be configured by overlapping the display of the output unit 816.

[Functional configuration]
Next, functional configurations of the mobile terminal 10, the integrated server 30, and the editor terminal 40 will be described. In the present embodiment, the voice recognition server 20 has a function of performing voice recognition and outputting a voice recognition result as text data (voice recognition function).

FIG. 3 is a block diagram illustrating a functional configuration of the mobile terminal 10.
As illustrated in FIG. 3, when the CPU 811 of the mobile terminal 10 executes the conference participation process, in the CPU 811, the login processing unit 111, the registration processing unit 112, the speech control unit 113, the display control unit 114, and the voice The output control unit 115 functions.
The login processing unit 111 executes a login process for authenticating the user (conference participant) of the mobile terminal 10 in the minutes creation support system 1. For example, the login processing unit 111 receives input of the user name and password of the user of the mobile terminal 10 and transmits the input to the integrated server 30 to acquire authentication of the user of the mobile terminal 10.

The registration processing unit 112 accepts selection of a conference in which the user of the mobile terminal 10 participates from conferences held in the minutes creation support system 1 and permits the user of the mobile terminal 10 to participate in the conference. If so, registration as a participant with the integrated server 30 is performed.
The utterance control unit 113 sequentially receives input of utterances (speech voices) of users (participants) participating in the conference, and transmits the input voices to the voice recognition server 20. Then, the utterance control unit 113 receives the voice recognition result of the transmitted voice from the voice recognition server 20, and transmits the voice and the voice recognition result to the integration server 30.

The display control unit 114 displays a display screen indicating that the conference is in progress, and the speech recognition result received from the speech recognition server 20 by the utterance control unit 113 is displayed on the display screen. Thereby, the speech of the user of the portable terminal 10 is sequentially displayed. Note that the display control unit 114 may receive time-series data obtained by integrating the speech recognition results (text data) of the speech of each participant from the integration server 30 and display the timeline data.
The voice output control unit 115 sequentially receives the voice of each participant participating in the conference and outputs it as voice. In addition, for the case where a plurality of participants participate in a conference at the same place, a function of turning on / off the sound output by a user operation may be provided.

Next, the functional configuration of the integrated server 30 will be described.
FIG. 4 is a block diagram illustrating a functional configuration of the integrated server 30.
As shown in FIG. 4, when the CPU 811 of the integration server 30 executes the integration process, in the CPU 811, the conference management unit 211, the voice recognition result acquisition unit 212, the user interface generation unit (UI generation unit) 213, The minutes data acquisition unit 214 and the cooperation processing unit 215 function. Further, a speech voice database (speech speech DB) 221, a speech recognition result database (speech recognition result DB) 222, and a minutes database (minutes DB) 223 are formed in the storage unit 817 of the integrated server 30. .

The speech voice DB 221 stores speech speech of each participant participating in the conference in time series. In the present embodiment, the order of voices stored in the speech voice DB 221 is the order in which voices are received by the integrated server 30.
The speech recognition result DB 222 stores speech recognition results of speech of each participant participating in the conference in time series. In the present embodiment, the order of the speech recognition results stored in the speech recognition result DB is the order in which the speech recognition results are received by the integrated server 30 as in the case of the speech. The voice of speech and the voice recognition result are transmitted as a unit from the portable terminal 10, and the storage order thereof corresponds to the voice of speech DB 221 and the voice recognition result DB 222.

The minutes DB 223 stores the minutes data transmitted from the editor terminal 40.
Note that the voice of speech, the speech recognition result, and the minutes data in each conference are stored in the speech voice DB 221, the speech recognition result DB 222, and the minutes DB 223 in association with each other.

  The meeting management unit 211 receives a meeting holding request from the editor terminal 40 and transmits data of a registration screen for registering information regarding the meeting to the editor terminal 40. The conference information includes the conference name, start date and time, names of participants, and the like. The conference management unit 211 generates registration screen data for registering these pieces of information and transmits them to the editor terminal 40. . In the present embodiment, since the participant participates in the conference using the mobile terminal 10, it is not necessary to gather at a specific place, but the conference requests a set of participants. The information regarding the conference includes the location of the conference.

In addition, when information related to the conference is transmitted from the editor terminal 40, the conference management unit 211 accepts registration of participation from the mobile terminal 10 of the user designated as a conference participant. At this time, the conference management unit 211 may transmit a message prompting the user to join the conference to the mobile terminal 10 of the user designated as the conference participant.
Further, in response to the start of the conference being requested from the editor terminal 40, the conference management unit 211 transmits a message indicating that the conference is started to the mobile terminal 10 of each participant. Similarly, in response to a request for termination of the conference from the editor terminal 40, the conference management unit 211 transmits a message indicating that the conference is terminated to the mobile terminal 10 of each participant.
The speech recognition result acquisition unit 212 sequentially receives the speech of each participant and the speech recognition result transmitted from the mobile terminal 10 of each participant, and stores the speech of the speech in the speech voice DB 221 in the order received. At the same time, the speech recognition result of the speech is stored in the speech recognition result DB 222.

The UI generation unit 213 generates data of a display screen (minutes editing screen) for editing the minutes in the editor terminal 40 and transmits the data to the editor terminal 40. The minutes editing screen includes an area for displaying the speech recognition result of each participant's speech on a timeline, and the UI generation unit 213 receives speech speech and the speech recognition result from each participant. In addition, the data of the minutes edit screen with the updated timeline display is sequentially transmitted to the editor terminal 40.
When the minutes data is transmitted from the editor terminal 40, the minutes data acquisition unit 214 stores the minutes data in the minutes DB 223 in association with the voice of the speech in the meeting and the voice recognition result.

  The cooperation processing unit 215 executes various processes related to the matters determined in the conference based on the minutes data. For example, the cooperation processing unit 215 registers a task to be imposed on a participant as a ToDo list on the participant's personal page in the groupware system, or reserves a facility (conference room or Make reservations for equipment, etc.).

Next, the functional configuration of the editor terminal 40 will be described.
FIG. 5 is a block diagram showing a functional configuration of the editor terminal 40.
As shown in FIG. 5, when the CPU 811 of the editor terminal 40 executes the minutes editing process, in the CPU 811, the conference holding processing unit 311, the user interface display control unit (UI display control unit) 312, the user interface The input control unit (UI input control unit) 313 functions. In addition, a minutes storage unit 321 is formed in the storage unit 817 of the editor terminal 40.
The minutes storage unit 321 stores the data of the minutes created by the editor via the minutes editing screen in the editor terminal 40.

The conference holding processing unit 311 requests the integration server 30 to hold a conference, receives registration screen data for registering information related to the conference from the integrated server 30, displays a registration screen, and displays information related to the conference. Accept input. Then, the conference holding processing unit 311 transmits information regarding the input conference to the integrated server 30. Further, the conference holding processing unit 311 requests the integration server 30 to start or end the conference when an operation for instructing the start or end of the conference is performed.
The UI display control unit 312 receives the data on the minutes edit screen from the integrated server 30 and displays the minutes edit screen. In addition, when the UI display control unit 312 receives the data of the minutes edit screen updated from the integrated server 30, the UI display control unit 312 sequentially displays the updated minutes edit screen.

FIG. 6 is a schematic diagram showing an example of the minutes editing screen.
As shown in FIG. 6, the minutes edit screen displayed by the UI display control unit 312 includes a meeting information area Z1, a timeline display area Z2, a minutes area Z3, and an attribute-specific area Z4. Yes.
In the conference information area Z1, information related to the conference such as the conference name, the date and time of the conference, the names of the participants, the distributed materials, and the like are displayed.
In the timeline display area Z2, the speech recognition results of the participant's speech are displayed in time series. In the example shown in FIG. 6, in the timeline display area Z <b> 2, the important utterance mark indicating whether or not the utterance is important, the utterance time, the speaker, and the text data indicating the utterance content are displayed in one line. As described above, the display content of the timeline display area Z2 is sequentially updated by the integrated server 30 every time a speech is made from each participant.
In the timeline display area Z2, the text data of each utterance displays a pronunciation button for outputting the voice of the utterance.
Further, in the timeline display area Z2, a search window for searching for text data of a speech by a keyword or a speaker is displayed.

  The minutes area Z3 is an area for accepting a drag-and-drop operation of the text data of the message displayed in the timeline display area Z2. The text data of the message subjected to the drag-and-drop operation is processed by the UI input control unit 313. It is copied and added to the recording data. Note that the text data of the message dragged and dropped in this way is displayed in time series in the minutes area Z3.

  The attribute-specific area Z4 is composed of areas corresponding to the attributes of the participant's speech content, and accepts a drag-and-drop operation of the text data of the speech displayed in the timeline display region Z2. In the example shown in FIG. 6, as the attribute-specific area Z4, there are a task area Z41 corresponding to the attribute of the task that has been imposed on the participant, and a scheduled area Z42 corresponding to the scheduled attribute determined by the meeting. Is set.

When the text data of the utterance displayed in the timeline display area Z2 is dragged and dropped into the assignment area Z41, the assignment that is imposed on the participant represented by the text data of the utterance is UI input control. Recorded in minutes by part 313. In the example shown in FIG. 6, when an area representing each participant is further set in the task area Z41, and the text data of the message is dragged and dropped into an area corresponding to any participant, that area As an assignment for the participant corresponding to the above, the assignment represented by the text data of the statement can be recorded.
When the text data of the message displayed in the timeline display area Z2 is dragged and dropped onto the schedule area Z42, the schedule (schedule) represented by the text data of the message is recorded by the UI input control unit 313. To be recorded. In the planned area Z42, similarly to the task area Z41, an area corresponding to each participant may be set.

  Returning to FIG. 5, the UI input control unit 313 controls the processing content to be input in response to the editor's operation on the minutes edit screen. For example, when the text data of the utterance displayed on the timeline is dragged and dropped in the minutes area (see the minutes area Z3 in FIG. 6) on the minutes editing screen, the UI input control unit 313 displays the text of the statement. Copy and add data to the minutes data. At this time, the style of the text data of the utterance may be converted for the minutes (conversion from “is / mas” to “is”, etc.).

  In addition, when the UI input control unit 313 performs an operation of adding an important mark indicating that the editor has determined that the message is important to the text data of the message displayed on the timeline on the minutes editing screen. Then, a tag indicating an important utterance is added to the text data of the utterance. Further, the UI input control unit 313 drags and drops the text data of the utterance displayed on the timeline to the attribute-specific area (see attribute-specific area Z4 in FIG. 6) corresponding to the attribute of the utterance content on the minutes editing screen. Then, a decision item corresponding to the attribute of the statement content is added to the minutes data. For example, the task that is imposed on the participant is added to the minutes data.

  In addition, when an operation for instructing the end of the meeting is performed on the minutes editing screen, the UI input control unit 313 transmits the data of the minutes created on the minutes editing screen to the integrated server 30. When the editor continues to edit the minutes, the UI input control unit 313 transmits the minutes data to the integrated server 30 after confirming the completion of the editing of the minutes by the editor. Also good.

  In the minutes edit screen, the timeline display area can display image data such as photos or drawings presented at the meeting by the editor clicking the “Photos” tab, etc. When the image data is dragged and dropped, the UI input control unit 313 adds the image data to the minutes in association with the text data of the message. Also, on the minutes editing screen, you can drag and drop or dialog data such as handouts related to the meeting, photos of drawings, sites and products, etc. It can be added at any position in the minutes by selecting via the screen. Therefore, in the minutes creation support system 1, various information other than the participant's remarks such as materials related to the meeting and the board written during the meeting are also managed in association with the minutes. It becomes possible.

Furthermore, the UI input control unit 313 accepts editing such as arbitrarily inputting characters or changing the order of the contents displayed in the minutes area. Thus, the editor can add a headline representing a topic, enter a description of a photograph or the like, and rearrange the order of statements when the topic changes.
Note that the UI input control unit 313 sequentially receives edits (such as correction of misrecognition in speech recognition) on the text data of the message displayed on the timeline. This editing result is transmitted to the integrated server 30 by the UI input control unit 313 and reflected in the voice recognition result DB 222.

[Operation]
Next, the operation of the minutes creation support system 1 will be described.
[Meeting process]
FIG. 7 is a flowchart showing a flow of conference participation processing executed by the mobile terminal 10 of the minutes creation support system 1.
The conference participation process is started in response to an operation instructing execution of the conference participation process in the mobile terminal 10.

In step S <b> 1, the login processing unit 111 executes login processing for authenticating the user (participant of the conference) of the mobile terminal 10 in the minutes creation support system 1.
In step S <b> 2, the registration processing unit 112 accepts selection of a conference in which the user of the mobile terminal 10 participates.
In step S <b> 3, the display control unit 114 displays a display screen indicating that a meeting is being performed in response to the integration server 30 accepting the participation of the user of the mobile terminal 10 in the meeting.

In step S4, the utterance control unit 113 sequentially receives input of utterances of users (participants) participating in the conference.
In step S <b> 5, the utterance control unit 113 transmits the input voice to the voice recognition server 20.
In step S <b> 6, the display control unit 114 displays the speech recognition result received from the speech recognition server 20.
In step S <b> 7, the utterance control unit 113 transmits the input voice and the voice recognition result to the integrated server 30.

In step S <b> 8, the registration processing unit 112 determines whether or not a message indicating that the conference is ended is received from the integrated server 30.
If a message indicating that the conference is over has not been received from the integrated server 30, it is determined as NO in Step S8, and the process proceeds to Step S4.
When the message indicating that the conference is ended is received from the integrated server 30, it is determined as YES in Step S8, and the conference participation process is ended.
Note that, during the execution of the conference participation process, the mobile terminal 10 may arbitrarily terminate the conference participation process by notifying the integration server 30 of the exit from the conference.

[Integration processing]
Next, the integration process executed by the integration server 30 will be described.
FIG. 8 is a flowchart showing the flow of integration processing executed by the integration server 30 of the minutes creation support system 1.
The integration process is started in response to an operation instructing execution of the integration process in the integration server 30.

In step S <b> 21, the conference management unit 211 receives a conference holding request from the editor terminal 40.
In step S <b> 22, the conference management unit 211 transmits registration screen data for registering information related to the conference to the editor terminal 40.
In step S <b> 23, the conference management unit 211 receives information related to the conference from the editor terminal 40.

In step S <b> 24, the conference management unit 211 accepts registration of participation from the mobile terminal 10 of the user designated as a conference participant.
In step S <b> 25, the UI generation unit 213 generates the minutes edit screen data and transmits the data to the editor terminal 40.
In step S <b> 26, the conference management unit 211 transmits a message indicating that the conference starts to the mobile terminal 10 of each participant in response to the request for starting the conference from the editor terminal 40.

In step S <b> 27, the speech recognition result acquisition unit 212 sequentially receives the speech of each participant and the speech recognition result transmitted from each participant's mobile terminal 10.
In step S <b> 28, the speech recognition result acquisition unit 212 stores the speech speech in the speech speech DB 221 in the order received, and stores the speech recognition result of the speech speech in the speech recognition result DB 222.
In step S <b> 29, the UI generation unit 213 transmits the minutes editing screen data with the updated timeline display to the editor terminal 40 based on the speech of each participant and the voice recognition result.

In step S <b> 30, the conference management unit 211 determines whether or not the editor terminal 40 has requested termination of the conference.
If the end of the conference is not requested from the editor terminal 40, NO is determined in step S30, and the process proceeds to step S27.
On the other hand, when the end of the conference is requested from the editor terminal 40, YES is determined in step S30, and the process proceeds to step S31.

In step S <b> 31, the minutes data acquisition unit 214 receives the minutes data from the editor terminal 40.
In step S32, the minutes data acquisition unit 214 stores the minutes data in the minutes DB 223 in association with the voice of the speech in the meeting and the voice recognition result thereof.
After step S32, the integration process ends.

[The minutes editing process]
Next, the minutes editing process executed by the editor terminal 40 will be described.
FIG. 9 is a flowchart showing the minutes editing process executed by the editor terminal 40 of the minutes creation support system 1.
The minutes editing process is started in response to an operation instructing execution of the minutes editing process at the editor terminal 40.

In step S <b> 41, the conference holding processing unit 311 requests the integrated server 30 to hold a conference.
In step S42, the conference holding processing unit 311 receives registration screen data for registering information related to the conference from the integrated server 30, and displays the registration screen.
In step S <b> 43, the conference holding processing unit 311 receives input of information regarding the conference, and transmits the input information regarding the conference to the integrated server 30.

In step S44, the UI display control unit 312 receives the data on the minutes edit screen from the integrated server 30, and displays the minutes edit screen.
In step S <b> 45, the UI input control unit 313 accepts an operation (such as a click operation of the “conference start button”) that instructs the start of the conference on the minutes edit screen.
In step S46, the conference holding processing unit 311 requests the integrated server 30 to start the conference.
In step S <b> 47, the UI display control unit 312 receives the data of the minutes edit screen with the updated timeline display from the integrated server 30.
In step S48, the UI display control unit 312 sequentially displays the updated minutes editing screen.

In step S49, the UI input control unit 313 determines whether or not an operation for adding an important mark indicating that the editor has determined that the message is an important message is performed on the text data of the message displayed on the timeline. I do.
When an operation for adding an important mark indicating that the editor has determined that the message is important is performed on the text data of the message displayed on the timeline, YES is determined in step S49, and the process is performed in step S50. Migrate to
On the other hand, when an operation for adding an important mark indicating that the editor has determined that the message is an important message is not performed on the text data of the message displayed on the timeline, it is determined as NO in step S49, and the process is performed. Proceeds to step S51.

In step S50, the UI input control unit 313 adds an important mark (adds a tag indicating an important message) to the text data of the message that has been operated.
In step S51, the UI input control unit 313 determines whether editing has been performed on the text data of the message displayed on the timeline.
When the text data of the message displayed on the timeline is edited, YES is determined in step S51, and the process proceeds to step S52.
On the other hand, if the text data of the message displayed on the timeline has not been edited, NO is determined in step S51, and the process proceeds to step S53.

In step S <b> 52, the UI input control unit 313 transmits the content of the edit on the text data of the message displayed on the timeline to the integration server 30.
In step S53, the UI input control unit 313 determines whether or not the text data of the message displayed on the timeline has been dragged and dropped in the minutes area.
When the text data of the message displayed on the timeline is dragged and dropped in the minutes area, YES is determined in step S53, and the process proceeds to step S54.
On the other hand, when the text data of the message displayed on the timeline is not dragged and dropped in the minutes area, it is determined as NO in Step S53, and the process proceeds to Step S55.

In step S54, the UI input control unit 313 copies and adds the text data dragged and dropped to the minutes area to the minutes data.
In step S55, the UI input control unit 313 determines whether or not the text data of the message displayed on the timeline has been dragged and dropped in the attribute-specific area.
When the text data of the message displayed on the timeline is dragged and dropped to the attribute-specific area, YES is determined in step S55, and the process proceeds to step S56.
On the other hand, when the text data of the message displayed on the timeline is not dragged and dropped in the attribute-specific area, NO is determined in step S55, and the process proceeds to step S57.

In step S56, the UI input control unit 313 adds a decision item (schedule or assignment) corresponding to the attribute of the statement content to the minutes data.
In step S <b> 57, the conference holding processing unit 311 determines whether or not an operation for instructing the end of the conference on the minutes editing screen (such as a click operation of the “conference end button”) has been performed.
When the operation for instructing the end of the conference on the minutes editing screen is not performed, the conference holding processing unit 311 determines NO in step S57, and the process proceeds to step S47.
On the other hand, when the operation for instructing the end of the conference on the minutes editing screen is performed, the conference holding processing unit 311 determines YES in step S57, and the process proceeds to step S58.

In step S <b> 58, the UI input control unit 313 transmits the minutes data edited on the minutes editing screen to the integrated server 30. When the editor continues to edit the minutes, the UI input control unit 313 may transmit the minutes data to the integrated server 30 after confirming the completion of the editing of the minutes by the editor. .
After step S58, the minutes editing process ends.

  By such processing, in the minutes creation support system 1, the speech of the conference participant input via the mobile terminal 10 is recognized by the speech recognition server 20, and the speech and the speech recognition result thereof are obtained. It is transmitted to the integrated server 30. In the integrated server 30, the minutes editing screen data for performing timeline display in which the participant's remarks are integrated in time series is transmitted to the editor terminal 40, and each time the participant remarks, the editor terminal The data on the minutes edit screen displayed at 40 is sequentially updated. In the editor terminal 40, the timeline display area in which the participant's utterances are displayed in time series is displayed while being sequentially updated, and the editor can record the minutes for the utterance text data in the timeline display area. Perform various operations related to creation of. Specifically, the editor can add text data displayed in the timeline display area to the minutes data area by dragging and dropping it, or drag and drop it to the attribute-specific area. In this way, the task to be imposed on the participant is added to the minutes, or the schedule represented by the text data of the remark is added to the minutes.

Therefore, according to the minutes creation support system 1, the editor can select appropriate remarks in real time and reflect them in the minutes by a simple process. It is possible to prepare an appropriate minutes summarizing the contents.
Therefore, according to the minutes creation support system 1, it is possible to more appropriately support the creation of minutes with a simpler system.

[Modification 1]
In the above-described embodiment, the minutes editing screen can take various forms in addition to the example shown in FIG.
FIG. 10 is a schematic diagram showing another example of the minutes editing screen.
In the example shown in FIG. 10, the timeline display area Z2 and the minutes area Z3 are enlarged as compared with the example shown in FIG.
Thereby, the visibility of the timeline display area Z2 is increased, and the editor can easily understand the flow of the conference. In addition, the editor can more easily perform an operation of dragging and dropping the text data of the message to the minutes area Z3. Furthermore, the editor can easily understand the display contents of the minutes area Z3, and the editor can more easily perform the operation of editing the minutes in the minutes area Z3.

FIG. 11 is a schematic diagram showing another example of the minutes editing screen.
In the example shown in FIG. 11, the conference information area Z1 has a simplified display content, and the attribute-specific area Z4 is not displayed because the editor has set it to non-display. On the other hand, various menu buttons useful for editing the minutes are displayed at the top of the minutes display screen.
In the example shown in FIG. 11, the minutes area Z3 is edited by the editor, the data of the distribution material of the meeting is posted, and the heading “about on-premises” indicating the content of the discussion is additionally written. In addition, following the headline, the participant's remarks are added to the minutes together with the names of the remarkers by dragging and dropping them. Note that the content of the remarks has been converted for writing in the minutes. Furthermore, following the remarks, the image data of the photos used in the meeting is added to the minutes.
As described above, the form of the minutes editing screen can be in various forms, and the displayed items can be variously changed according to the settings of the editor. Further, a plurality of forms of the minutes editing screen as shown in FIGS. 6, 10, and 11 may be prepared, and the editor may select the minutes editing screen in an easy-to-use form.

  In the above-described embodiment, the minutes creation support system 1 may include a PC for displaying the minutes edit screen without providing the minutes edit function. By participating in the conference via the PC, a participation form such as participating in the conference as an observer becomes possible.

  Moreover, in the above-mentioned embodiment, it is good also as uploading the photograph image | photographed with the portable terminal 10 as a participant's comment. At this time, the uploaded photo may be transmitted to the mobile terminal 10 of another participant. Accordingly, it is possible to easily make a speech at a conference using a photograph, and it is possible for the participant and the editor of the conference to more easily understand the content of the participant's speech.

  Further, in the above-described embodiment, it is also possible to automatically include in the minutes the statement that the editor has attached an important mark on the minutes edit screen. In this case, in the timeline display area, the editor can appropriately add an important mark to the text data of the speech, and can create a minutes in which the important speech is gathered together with the end of the conference.

  In the above-described embodiment, the integrated server 30 or the editor terminal 40 distributes the created minutes data to the participants by e-mail or notifies the link indicating the storage location of the minutes data. It is good also as providing the data of the minutes by doing. As a result, it is possible to promptly inform the participants of the decisions made in the minutes, such as the tasks and schedules of each participant.

Further, in the above-described embodiment, the decision text and the problem may be automatically determined and added to the minutes by subjecting the text data of the speech to natural language processing.
Also, in the above-described embodiment, the minutes created during the meeting may be saved as temporary minutes data, etc., and the editor may edit the minutes data again after the meeting. Good.

In the above-described embodiment, the time reference when integrating the text data of the speech in time series is the order in which the speech recognition results of the speech are received by the integration server 30, but the speech recognition server 20 Other times may be used as a reference, for example, in the order in which voice recognition is started.
In the above-described embodiment, the process for recognizing the speech of the participant's speech may be performed by a device other than the speech recognition server 20 such as the mobile terminal 10 or the integrated server 30.

In the above-described embodiment, the integrated server 30 or the editor terminal 40 connects the network 50 or the like from the camera that captures the image of the whiteboard on which the board is written during the meeting or the scanner that scans the board surface of the whiteboard. Thus, it is also possible to acquire a picture of a postcard or a scanned image at a predetermined timing (for example, at regular time intervals or timings instructed by an editor).
In this case, the editor terminal 40 automatically adds the acquired board photo or scanned image data to the minutes automatically, or the editor appropriately selects and adds them to the minutes. Can do.
As a result, it is possible to appropriately add information on the board written with an important meaning as a record of the meeting to the minutes.

The minutes creation support system 1 configured as described above includes the mobile terminal 10, the integrated server 30, and the editor terminal 40. The editor terminal 40 includes a UI display control unit 312 and a UI input control unit 313.
The mobile terminal 10 transmits speech information in which the speech of the conference participant is text data to the integrated server.
The integration server 30 integrates the speech information of the conference participants transmitted from the mobile terminal 10 in time series.
The UI display control unit 312 moves a first display area (timeline display area) for displaying the utterance information integrated in time series by the integration server 30 and an operation for moving the utterance information displayed in the first display area. A display screen including a second display area (minutes area) associated with the process of including the statement information in the minutes is displayed.
The UI input control unit 313 executes processing for adding the comment information to the minutes data when an operation for moving the comment information displayed in the first display area to the second display area is performed.
Thereby, in the editor terminal 40, the utterance information in which the utterance of the conference participant input via the mobile terminal 10 is converted into text data is integrated in a time series and displayed in the first display area. Then, by performing an operation for moving the comment information in the first display area to the second display area, the comment information is added to the minutes data.
Therefore, it is possible for the editor to select appropriate remarks in real time and reflect them in the minutes by simple processing. In parallel with the progress of the meeting, the editor creates an appropriate minutes that summarizes the contents of the meeting. It becomes possible to do.
Therefore, according to the minutes creation support system 1, it is possible to more appropriately support the creation of minutes with a simpler system.

The speech information is generated by speech recognition of the participant's speech.
Thereby, the participant's remarks can be promptly converted into text data and used for the processing for creating the minutes.

The UI input control unit 313 accepts an operation for adding identification information (for example, an important mark) to the utterance information displayed in the first display area, and adds the utterance information with the identification information added to the minutes data. Execute the process.
As a result, it is possible to create a minutes in which statement information with identification information is gathered together with the end of the conference.

The UI display control unit 312 displays a third display area (attribute-specific area) associated with a specific attribute, which is an area to which an operation for moving the remark information displayed in the first display area is moved. Display further on the screen.
The UI input control unit 313 performs processing according to the comment information and the attributes of the third display area when an operation for moving the comment information displayed in the first display area to the third display area is performed. .
As a result, in the process of creating the minutes of matters that have been discussed as topics in the conference, it is possible to automatically execute processing (schedules for the groupware system, setting of issues, etc.) according to the attributes.

The UI display control unit 312 displays a third display area corresponding to each participant.
Thereby, the process according to the attribute can be automatically executed in the process of creating the minutes for the topic that has been discussed in the conference for each participant.

The UI input control unit 313 converts the style and adds the speech information to the minutes data when an operation to move the speech information displayed in the first display region to the second display region is performed. Execute the process.
Thereby, the participant's remarks can be automatically converted into an expression suitable for the minutes and recorded.

The UI input control unit 313 accepts an operation for editing information displayed in the second display area.
Thereby, it is possible to edit the remark information moved from the first display area and record it in the minutes.

The UI input control unit 313 accepts an operation for adding image data to the second display area.
Thereby, it is possible to record the minutes including the image data.

The UI input control unit 313 accepts an operation of adding material data related to the conference or data of a board written during the conference to the second display area.
As a result, various information other than the participant's remarks, such as materials related to the conference and a board written during the conference, can be integrated and managed in association with the minutes.

The UI input control unit 313 executes processing for providing the generated minutes data to the conference participants.
Thereby, it is possible to promptly provide the minutes data to the conference participants.

In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.
The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIGS. 3 to 5 is merely an example, and is not particularly limited. That is, it is sufficient if the minutes creation support system 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is particularly shown in the example of FIG. It is not limited.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only configured by the removable medium 821 shown in FIG. 2 distributed separately from the apparatus main body in order to provide the user with the program, but is also incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in. The removable medium 821 is configured by, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being incorporated in advance in the apparatus main body is configured by, for example, the ROM 812 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the order, but is not necessarily performed in chronological order, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

  DESCRIPTION OF SYMBOLS 1 Minutes creation support system, 10 portable terminal, 20 voice recognition server, 30 integrated server, 40 editor terminal, 50 network, 800 information processing apparatus, 811 CPU, 812 ROM, 813 RAM, 814 bus, 815 input unit, 816 Output unit, 817 storage unit, 818 communication unit, 819 drive, 821 removable media, 111 login processing unit, 112 registration processing unit, 113 speech control unit, 114 display control unit, 115 audio output control unit, 211 conference management unit, 212 Voice recognition result acquisition unit, 213 User interface generation unit (UI generation unit), 214 Minutes data acquisition unit, 215 Cooperation processing unit, 221 Speech voice database (speech voice DB), 222 Voice recognition result database (voice recognition result DB) 223 proceedings Database (Minutes DB), 311 conference unit, 312 user interface display control unit (UI display control unit), 313 user interface input control unit (UI input control unit), 321 minutes storage unit

Claims (13)

A terminal device used by a conference participant, an integrated server that integrates statements made by the conference participant, and an information processing device that creates minutes of the conference,
The terminal device transmits the speech information in which the speech of the participant of the conference is text data to the integrated server,
The integration server integrates the speech information of the conference participants transmitted from the terminal device in time series,
The information processing apparatus includes:
A first display area for displaying the message information integrated in time series by the integration server; and an area to which the operation for moving the message information displayed in the first display area is performed. Display control means for displaying a display screen including a second display area associated with the process of including information in the minutes;
Processing means for executing processing for adding the comment information to the minutes data when an operation of moving the comment information displayed in the first display area to the second display area is performed;
An information processing system comprising:   The information processing system according to claim 1, wherein the speech information is generated by speech recognition of the participant's speech.   The processing means receives an operation of adding identification information to the comment information displayed in the first display area, and executes a process of adding the comment information to which the identification information is added to minutes data. The information processing system according to claim 1 or 2. The display control means is an area that is a destination of an operation for moving the message information displayed in the first display area, and further displays a third display area associated with a specific attribute on the display screen. Display
The processing means performs processing according to the comment information and the attributes of the third display area when an operation of moving the comment information displayed in the first display area to the third display area is performed. The information processing system according to any one of claims 1 to 3, wherein the information processing system is executed.   The information processing system according to claim 4, wherein the display control unit displays the third display area corresponding to each participant.   The processing means, when an operation for moving the utterance information displayed in the first display area to the second display area is performed, converts the style and adds the utterance information to the minutes data. 6. The information processing system according to claim 1, wherein the information processing system executes a process to perform the processing.   The information processing system according to claim 1, wherein the processing unit receives an operation for editing the information displayed in the second display area.   The information processing system according to claim 1, wherein the processing unit receives an operation of adding image data to the second display area.   9. The apparatus according to claim 1, wherein the processing unit receives an operation of adding data of a material related to the conference or data of a blackboard during the conference to the second display area. The information processing system described.   The information processing system according to any one of claims 1 to 8, wherein the processing unit executes a process of providing the generated minutes data to a participant of the conference. A first display area for displaying utterance information in which utterances of participants in the meeting are text data in time series;
A second display area that is a movement destination of an operation for moving the comment information displayed in the first display area and is associated with a process of including the comment information in the minutes;
Display control means for displaying a display screen including:
Processing means for executing processing for adding the comment information to the minutes data when an operation of moving the comment information displayed in the first display area to the second display area is performed;
An information processing apparatus comprising: A first display area for displaying utterance information in which utterances of participants in the meeting are text data in time series;
A second display area that is a movement destination of an operation for moving the comment information displayed in the first display area and is associated with a process of including the comment information in the minutes;
A display control step for displaying a display screen including
A processing step of executing a process of adding the comment information to the minutes data when an operation of moving the comment information displayed in the first display area to the second display area is performed;
An information processing method comprising: On the computer,
A first display area for displaying utterance information in which utterances of participants in the meeting are text data in time series;
A second display area that is a movement destination of an operation for moving the comment information displayed in the first display area and is associated with a process of including the comment information in the minutes;
A display control function for displaying a display screen including
A processing function for executing processing to add the speech information to the minutes data when an operation to move the speech information displayed in the first display region to the second display region is performed;
A program characterized by realizing.

Images